Insertion support system, insertion support method, and program

Abstract

To provide an insertion support system, insertion support method and program which can support the insertion operation for changing the state of an insertion unit at a target position to the state suitable to observation of a subject.SOLUTION: An insertion support system comprises: a setting unit; a state estimation unit; a path calculation unit; a state detection unit; and an insertion support unit. The setting unit sets the first position and the second position in shape information. The state estimation unit estimates the first state of a tip part at the first position. The path calculation unit calculates a path through which the tip part passes. The state detection unit detects the second state of the tip part at the second position. The insertion support unit outputs to an information notification device insertion support information necessary for the insertion operation such that the tip part reaches the first position through the path from the second position and the state of the tip part changes to the first state from the second state.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an insertion assistance system, an insertion assistance method, and a program. [Background technology]

[0002] Industrial endoscope devices are used to inspect the interior of boilers, pipes, aircraft engines, and the like for abnormalities and corrosion. The endoscope device has an insertion section for acquiring images. A user inserts the insertion section into a subject to acquire images of the inspection site within the subject. The user then observes the images and inspects the inspection site.

[0003] In the medical field, there is a technology that calculates the insertion path of an endoscope by using three-dimensional (3D) image data of a living body. This technology obtains 3D image data using a CT scan, which irradiates the living body with X-rays. When examining thin organs such as the lungs, this technology can determine the optimal insertion path that will not damage the organ walls.

[0004] The technology disclosed in Patent Document 1 provides a method for determining the direction of the tip of an endoscope based on the inner diameter of the bronchus, while the technology disclosed in Patent Document 2 provides a method for selecting a path based on the minimum curvature of the path.

[0005] A similar technique to the above has been disclosed in the field of industrial endoscopes. The technique disclosed in Patent Document 3 sets an insertion path by using 3D shape information of the subject. If the insertion part passes through a protrusion, a very narrow place, or a place that requires the insertion part to bend excessively, it is determined that the insertion path is inappropriate. In this case, an alternative path is suggested.

[0006] The provision of the path calculation function improves the efficiency of the insertion work of the examiner at the examination site. Furthermore, regardless of the examiner's examination skill, the examiner can easily reach the insertion part to the examination site, enabling a highly reliable examination to be performed. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent No. 6030435 [Patent Document 2] Patent No. 5123615 [Patent Document 3] Patent No. 4464640 Summary of the Invention [Problem to be solved by the invention]

[0008] When the insertion part reaches the examination site, the posture of the insertion part may not be ideal, and images that are not suitable for the examination may be acquired. However, conventional techniques do not take into account the state of the insertion part at the target position for observing the examination site.

[0009] The present invention aims to provide an insertion assistance system, an insertion assistance method, and a program that can assist the insertion operation to bring the state of the insertion section at the target position into a state suitable for observing the subject. [Means for solving the problem]

[0010] The present invention provides an insertion assistance system that assists an insertion operation of an insertion section having a tip portion that acquires an optical image of a subject when the insertion section is inserted into the subject, the system including a setting unit that sets a first position and a second position in shape information, the shape information indicating a three-dimensional shape of the subject, the first position indicating a target position, and the second position being different from the first position; a state estimation unit that estimates a first state of the tip portion at the first position based on specifications of the insertion section; and a path through which the tip portion passes when moving from the second position to the first position. From the second position to the first positiona path calculation unit that calculates a path; a state detection unit that detects a second state of the tip portion at the second position; and an insertion support unit that outputs to an information notification device insertion support information required for the insertion operation in which the tip portion travels from the second position to the first position through the path and the state of the tip portion changes from the second state to the first state. and when the three-dimensional shape has a branching portion, the path calculation unit moves the tip portion, which is disposed at the first position in the first state, to the second position in the three-dimensional shape, and records the branching portion that the tip portion passes through while moving from the first position to the second position. The path calculation unit calculates two or more path candidates from the second position to the first position that pass through the branching portion, and selects one of the two or more path candidates as the path. It is an insertion assistance system. The present invention provides an insertion assistance system that assists an insertion operation of an insertion section having a tip section that acquires an optical image of a subject when the insertion section is inserted into a subject, the system including: a setting unit that sets a first position and a second position in shape information, the shape information indicating a three-dimensional shape of the subject, the first position indicating a target position, and the second position different from the first position; a state estimation unit that estimates a first state of the tip section at the first position based on specifications of the insertion section; a path calculation unit that calculates a path from the second position to the first position that the tip section will take when moving from the second position to the first position; and a state detection unit that detects a second state of the tip section at the second position. and an insertion support unit that outputs to an information notification device insertion support information required for the insertion operation, in which the tip portion reaches the first position from the second position through the path and the state of the tip portion changes from the second state to the first state, wherein, when the three-dimensional shape has a branching portion, the path calculation unit calculates two or more path candidates from the second position to the first position that pass through the branching portion, the path calculation unit analyzes the two or more path candidates according to one or more indicators related to at least one of the insertion portion and the subject, and the path calculation unit selects one of the two or more path candidates as the path based on an analysis result of the two or more path candidates.

[0013] In the insertion assistance system of the present invention, an importance level is set in advance for each of two or more indices including the one or more indices, and the path calculation unit analyzes the two or more path candidates according to the two or more indices and the importance level, and the importance level set for each of the two or more indices can be changed.

[0014] In the insertion assistance system of the present invention, the path calculation unit analyzes the two or more path candidates by using at least one of information indicating the size of the insertion portion and information indicating the shape of the subject.

[0015] The insertion assistance system of the present invention further includes a position detection unit that detects the position of the tip when the insertion operation is performed, and the setting unit sets the second position to the position detected by the position detection unit.

[0016] In the insertion assistance system of the present invention, the state estimation unit estimates the first state such that the state of the tip at the first position is at least one of a first observation state, a second observation state, and a third observation state, wherein the first observation state is a state in which the direction of the optical axis of an observation optical system arranged at the tip is perpendicular to the surface of the subject, the second observation state is a state in which a predetermined direction at the tip coincides with a predetermined direction in the three-dimensional shape, and the third observation state is a state in which the distance between the tip and the subject is suitable for observing the subject.

[0017] In the insertion assist system of the present invention, the insertion operation refers to at least one of an operation of moving the insertion portion inside the subject, an operation of bending the insertion portion, and an operation of twisting the insertion portion.

[0018] In the insertion assistance system of the present invention, the insertion assistance information includes at least one of the distance between the second position and the first position, the amount of change in the direction of the tip portion to align the direction of the tip portion with the direction along the path, and the amount of twist to align the twisted state of the insertion portion with the twisted state of the insertion portion in the first state.

[0019] In the insertion assist system of the present invention, the insertion assist information includes a history of positions through which the tip portion has passed.

[0020] In the insertion assist system of the present invention, the path calculation unit calculates the path based on the specifications of the optical adapter attached to the tip portion.

[0021] The present invention provides an insertion assistance method for assisting an insertion operation of an insertion section having a tip portion for acquiring an optical image of a subject when the insertion section is inserted into the subject, the method including: a setting step in which a setting unit sets a first position and a second position in shape information, the shape information indicating a three-dimensional shape of the subject, the first position indicating a target position, and the second position being different from the first position; a state estimation step in which a state estimation unit estimates a first state of the tip portion at the first position based on specifications of the insertion section; and a state estimation step in which a state estimation unit estimates a state of the tip portion at the first position based on specifications of the insertion section. From the second position to the first position a state detection step in which a state detection unit detects a second state of the tip portion at the second position; and an insertion support step in which the insertion support unit outputs insertion support information required for the insertion operation in which the tip portion travels from the second position to the first position through the path and the state of the tip portion changes from the second state to the first state to an information notification device. and when the three-dimensional shape has a branching portion, the path calculation unit moves the tip portion, which is disposed at the first position in the first state, to the second position in the three-dimensional shape, and records the branching portion that the tip portion passes through while moving from the first position to the second position. The path calculation unit calculates two or more path candidates from the second position to the first position that pass through the branching portion, and selects one of the two or more path candidates as the path. This is an insertion assistance method. The present invention provides an insertion assistance method for assisting an insertion operation of an insertion section having a tip portion for acquiring an optical image of a subject when the insertion section is inserted into a subject, the method including: a setting step in which a setting unit sets a first position and a second position in shape information, the shape information indicating a three-dimensional shape of the subject, the first position indicating a target position, and the second position being different from the first position; a state estimation step in which a state estimation unit estimates a first state of the tip portion at the first position based on specifications of the insertion section; a path calculation step in which a path calculation unit calculates a path from the second position to the first position along which the tip portion will pass when moving from the second position to the first position; and a state detection step in which a state detection unit detects the second state of the tip portion at the second position. and an insertion assistance step in which an insertion assistance unit outputs insertion assistance information necessary for the insertion operation, such that the tip reaches the first position from the second position through the path and the state of the tip changes from the second state to the first state, to an information notification device, wherein, when the three-dimensional shape has a branching portion, the path calculation unit calculates two or more path candidates from the second position to the first position, the path candidates passing through the branching portion, the path calculation unit analyzes the two or more path candidates according to one or more indicators related to at least one of the insertion unit and the subject, and the path calculation unit selects one of the two or more path candidates as the path based on an analysis result of the two or more path candidates.

[0022] The present invention provides a program for causing a computer to execute a process for assisting an insertion operation of an insertion section having a tip portion for acquiring an optical image of a subject when the insertion section is inserted into the subject, the process including a setting step of setting a first position and a second position in shape information, the shape information indicating a three-dimensional shape of the subject, the first position indicating a target position, and the second position being different from the first position; a state estimation step of estimating a first state of the tip portion at the first position based on specifications of the insertion section; and a path through which the tip portion passes when moving from the second position to the first position. From the second position to the first position a path calculation step of calculating a path; a state detection step of detecting a second state of the tip portion at the second position; and an insertion support step of outputting to an information notification device insertion support information required for the insertion operation in which the tip portion travels from the second position to the first position through the path and the state of the tip portion changes from the second state to the first state. and in the path calculation step, if the three-dimensional shape has a branching portion, the tip portion disposed at the first position in the first state is moved to the second position in the three-dimensional shape, and the branching portion that the tip portion passes through while moving from the first position to the second position is recorded, and in the path calculation step, two or more path candidates from the second position to the first position that pass through the branching portion are calculated, and one of the two or more path candidates is selected as the path. It is a program. The present invention provides a program for causing a computer to execute a process for assisting an insertion operation of an insertion section having a tip portion for acquiring an optical image of a subject when the insertion section is inserted into the subject, the process including a setting step of setting a first position and a second position in shape information, the shape information indicating a three-dimensional shape of the subject, the first position indicating a target position, and the second position being different from the first position; a state estimation step of estimating a first state of the tip portion at the first position based on specifications of the insertion section; a path calculation step of calculating a path from the second position to the first position that the tip portion will take when moving from the second position to the first position; and a path calculation step of detecting a second state of the tip portion at the second position. and an insertion support step of outputting to an information notification device insertion support information required for the insertion operation, in which the tip reaches the first position from the second position through the path, and the state of the tip changes from the second state to the first state. In the path calculation step, if the three-dimensional shape has a branching portion, two or more path candidates from the second position to the first position that pass through the branching portion are calculated, the two or more path candidates are analyzed in the path calculation step according to one or more indicators related to at least one of the insertion portion and the subject, and one of the two or more path candidates is selected as the path based on the analysis result of the two or more path candidates. [Effects of the Invention]

[0023] According to the present invention, the insertion assistance system, insertion assistance method, and program can assist the insertion operation to bring the state of the insertion section at the target position into a state suitable for observing the subject. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a perspective view showing the overall configuration of an endoscope apparatus according to an embodiment of the present invention. [Figure 2] 1 is a block diagram showing an internal configuration of an endoscope apparatus according to an embodiment of the present invention. [Figure 3] 1 is a block diagram showing the main configuration of an endoscope apparatus according to an embodiment of the present invention. [Figure 4]4 is a flowchart showing the procedure of an operation of the endoscope device according to the embodiment of the present invention. [Figure 5] 4 is a flowchart showing the procedure of an operation of the endoscope device according to the embodiment of the present invention. [Figure 6] 10A and 10B are diagrams illustrating examples of shape information and position information according to an embodiment of the present invention. [Figure 7] FIG. 4 is a diagram illustrating an example of device setting information according to an embodiment of the present invention. [Figure 8] 4 is a flowchart showing the procedure of an operation of the endoscope apparatus according to the embodiment of the present invention. [Figure 9] 4 is a flowchart showing the procedure of an operation of the endoscope device according to the embodiment of the present invention. [Figure 10] 10A to 10C are diagrams illustrating examples of target states of the tip of the insertion section in the embodiment of the present invention. [Figure 11] 10A to 10C are diagrams illustrating examples of target states of the tip of the insertion section in the embodiment of the present invention. [Figure 12] 10A to 10C are diagrams illustrating examples of target states of the tip of the insertion section in the embodiment of the present invention. [Figure 13] 4 is a flowchart showing the procedure of an operation of the endoscope device according to the embodiment of the present invention. [Figure 14] FIG. 10 is a diagram illustrating an example of a route candidate according to an embodiment of the present invention. [Figure 15] FIG. 10 is a diagram illustrating an example of a branch management table according to the embodiment of the present invention. [Figure 16] 4 is a flowchart showing the procedure of an operation of the endoscope device according to the embodiment of the present invention. [Figure 17] 10A and 10B are diagrams illustrating a method for calculating the amount of bending load of an insertion portion in an embodiment of the present invention. [Figure 18] FIG. 1 illustrates a method for analyzing slope characteristics of a candidate path in an embodiment of the present invention. [Figure 19] 10A and 10B are diagrams illustrating examples of restrictions on the placement of insertion portions according to the diameter of the insertion portions in an embodiment of the present invention. [Figure 20]10A and 10B are diagrams illustrating examples of limitations on the placement of the insertion portion according to the rigid length of the insertion portion in the embodiment of the present invention. [Figure 21] 10A and 10B are diagrams illustrating an example of limitations on the placement of an insertion portion according to the bending length of the insertion portion in an embodiment of the present invention. [Figure 22] FIG. 10 is a diagram illustrating an example of an analysis result of a route candidate according to an embodiment of the present invention. [Figure 23] 4 is a flowchart showing the procedure of an operation of the endoscope device according to the embodiment of the present invention. [Figure 24] FIG. 1 is a diagram illustrating a route model according to an embodiment of the present invention. [Figure 25] 10A and 10B are diagrams illustrating a state of an insertion unit when insertion support information is generated in an embodiment of the present invention. [Figure 26] 10A and 10B are diagrams illustrating an example of insertion support information displayed on a display unit according to an embodiment of the present invention. [Figure 27] 10A and 10B are diagrams showing an example of restrictions on the placement of an insertion section according to the bending length of the insertion section in a first modified example of the embodiment of the present invention. [Figure 28] 10A and 10B are diagrams showing an example of restrictions on the placement of an insertion section according to the bending length of the insertion section in a first modified example of the embodiment of the present invention. [Figure 29] 10A and 10B are diagrams showing an example of restrictions on the placement of an insertion section according to the bending length of the insertion section in a first modified example of the embodiment of the present invention. [Figure 30] FIG. 10 is a diagram showing an example of insertion support information displayed on a display unit in a second modified example of the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0025] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 shows the appearance of an endoscopic device 1 (insertion assistance system) according to an embodiment of the present invention. The endoscopic device 1 shown in Fig. 1 has an insertion section 2, a main body section 3, an operation section 4, a display section 5, and an insertion length detection section 6.

[0026] The insertion portion 2 is inserted into the inside of the subject. The user (examiner) performs an insertion operation and inserts the insertion portion 2 into the subject. The insertion portion 2 is a long, thin tube. The insertion portion 2 has a tip portion 20 and a bending portion 21. The tip portion 20 is located at the tip of the insertion portion 2. The tip portion 20 has a hard portion formed of a hard material. An optical adapter 7 is attached to the tip portion 20. The bending portion 21 is located on the base end side of the tip portion 20. The bending portion 21 can be bent in a predetermined direction. The insertion portion 2 converts an optical image of the subject into an imaging signal and outputs the imaging signal to the main body portion 3.

[0027] The main body 3 is a control device that includes a storage section for storing the insertion section 2. The operation section 4 accepts user operations on the endoscope device 1. The display section 5 has a display screen, and displays an image of the subject acquired by the insertion section 2 on the display screen.

[0028] The operation unit 4 is a user interface (input device). For example, the operation unit 4 is at least one of a button, a switch, a key, a mouse, a joystick, a touchpad, a trackball, and a touch panel. The user bends the bending portion 21 by performing a bending operation using the operation unit 4. Alternatively, the user controls the state of lighting by operating the operation unit 4. Furthermore, the user inputs information for setting the state of the endoscope device 1 into the endoscope device 1 by operating the operation unit 4. The input device having the operation unit 4 may be connected to the main body 3 by wire or wirelessly.

[0029] The display unit 5 is a monitor (display) such as an LCD (Liquid Crystal Display). The display unit 5 may be a touch panel. In this case, the operation unit 4 and the display unit 5 are integrated. The user touches the screen of the display unit 5 using a part of their body (e.g., a finger) or a tool. The display unit 5 may be connected to the main body 3 by wire or wirelessly. When the operation unit 4 and the display unit 5 are integrated, an information terminal such as a tablet, smartphone, or personal computer may be used as the terminal including the display unit 5.

[0030] For example, the insertion length detection unit 6 has a rotary encoder and two rollers. The two rollers are arranged to sandwich the insertion portion 2. The two rollers are in contact with the insertion portion 2. When the insertion portion 2 moves, the two rollers rotate. The rotary encoder detects the amount of rotation of at least one of the two rollers, thereby detecting the length (insertion length) of the insertion portion 2 inserted into the space inside the subject. The insertion length corresponds to the position of the tip portion 20.

[0031] The user performs bending and insertion operations while viewing the image displayed on the display unit 5. When the insertion unit 2 is inserted into the subject, the endoscope device 1 assists in the insertion operation of the insertion unit 2. The user finds the examination site and positions the tip unit 20 so that the examination site is captured in the image in an appropriate state. The user then performs the examination of the subject.

[0032] 2 shows the internal configuration of the endoscope device 1. The distal end portion 20 of the insertion portion 2 has a lens 22, an image sensor 23, and a posture sensor 24.

[0033] The main body 3 has an image processing unit 30, a recording unit 31, an external IF (interface) 32, an operation processing unit 33, a position detection unit 34, an attitude detection unit 35, a light source 36, an illumination control unit 37, a motor 38, a curvature control unit 39, a state detection unit 40, a path processing unit 41, an insertion support unit 42, a memory 43, and a power supply unit 44.

[0034] The optical adapter 7 has a lens 70. Light incident on the lens 70 passes through the lens 70 and enters the lens 22. The lenses 70 and 22 constitute an observation optical system. Light incident on the lens 22 passes through the lens 22 and enters the image sensor 23. The image sensor 23 is an image sensor such as a CCD sensor or a CMOS sensor. The image sensor 23 has an image surface 23a onto which the light that has passed through the lens 22 is incident. The image sensor 23 converts the light that has entered the image surface 23a into an image signal.

[0035] The imaging signal generated by the imaging element 23 includes an image of the subject. Therefore, the imaging element 23 acquires an optical image of the subject and generates an image of the subject. The image generated by the imaging element 23 is output to the main body 3.

[0036] The attitude sensor 24 has at least one of a three-axis acceleration sensor, a three-axis gyro sensor, and a geomagnetic sensor. The attitude sensor 24 measures a value related to the attitude of the tip unit 20 and outputs the measured value to the main body unit 3. The value indicates at least one of acceleration, angular velocity, and geomagnetism.

[0037] The image processing unit 30 processes the image signal output from the imaging element 23 to process the image of the subject. For example, the image processing unit 30 performs processes such as noise removal, brightness adjustment, and color adjustment to improve image quality. The image processing unit 30 also performs self-localization estimation such as Simultaneous Localization and Mapping (SLAM) to calculate the position and orientation of the tip portion 20. Furthermore, the image processing unit 30 superimposes insertion assistance information generated by the insertion assistance unit 42 on the image of the subject.

[0038] The image processed by the image processing unit 30 is output to the display unit 5 or the recording unit 31. The display unit 5 displays the image processed by the image processing unit 30. The recording unit 31 has a recording medium and records the image processed by the image processing unit 30 on the recording medium.

[0039] The external IF 32 is connected to an external PC 8. The external PC 8 is a general-purpose personal computer. The external PC 8 outputs subject information. An information terminal such as a tablet or a smartphone may be used instead of the external PC.

[0040] The object information includes shape information, material information, and position information. The shape information indicates the 3D shape (3D model) of the object. For example, the shape information is generated using 3D-CAD. Alternatively, the shape information is generated from images acquired in a previous examination using a technique such as SLAM. The shape information includes 3D coordinates of three or more points on the object. The shape information generated using a technique such as SLAM includes the 3D coordinates and also includes position information and orientation information of the camera that acquired the image. The material information indicates the material of the object. The position information indicates at least one of the start point and the end point of the 3D shape. The start point indicates the position where the tip portion 20 is positioned when the insertion operation is performed. The end point indicates the target position, which is the observation position of the object. For example, the 3D shape indicated by the shape information is displayed on the display unit 5. The user operates the operation unit 4 to specify the start point and the end point of the 3D shape in advance.

[0041] The object information output from the external PC 8 is input to the external IF 32. The external IF 32 outputs the object information to the path processing unit 41.

[0042] The external IF 32 may be connected to a server on a network (cloud). Subject information may be output from the server and input to the external IF 32.

[0043] The external IF 32 may be connected to a recording medium, such as a memory card, that stores the subject information. The subject information may be read from the recording medium and input to the external IF 32.

[0044] The operation processing unit 33 is connected to the operation unit 4. The operation unit 4 outputs information according to the operation performed by the user. The operation processing unit 33 sets the state of the endoscope device 1 to a state according to the information output from the operation unit 4.

[0045] The position detection unit 34 detects the position of the tip portion 20 inside the subject based on the insertion length output from the insertion length detection unit 6. The attitude detection unit 35 detects the attitude of the tip portion 20 based on the value output from the attitude sensor 24.

[0046] The light source 36 is a light emitting diode (LED) or the like, and generates illumination light. The illumination light is emitted from the distal end portion 20 via a light guide 25 disposed in the insertion portion 2. The illumination control unit 37 controls the light source 36 based on information output from the operation unit 4 to turn the illumination on or off and set the illumination intensity.

[0047] The motor 38 is connected to the plurality of angle wires 26. The plurality of angle wires 26 are arranged in the insertion section 2 and connected to the bending section 21. The motor 38 bends the bending section 21 by pulling the plurality of angle wires 26. The bending control section 39 controls the angle of the bending section 21 by controlling the motor 38 based on information output from the operation section 4. In other words, the bending control section 39 controls the attitude of the distal end portion 20.

[0048] The state detection unit 40 detects the position and posture of the tip portion 20. The state detection unit 40 outputs information indicating the detected position and posture to the insertion support unit 42. The path processing unit 41 calculates the optimal path that the tip portion 20 will take until it reaches the target position. The insertion support unit 42 generates insertion support information to support the insertion operation of the insertion unit 2.

[0049] The memory 43 stores information processed by the endoscope device 1. The memory 43 is, for example, a semiconductor memory and may include a RAM area and a ROM area. The power supply unit 44 supplies driving power to each unit of the endoscope device 1.

[0050] At least one of the image processing unit 30, the operation processing unit 33, the position detection unit 34, the attitude detection unit 35, the illumination control unit 37, the bending control unit 39, the state detection unit 40, the path processing unit 41, and the insertion support unit 42 may be configured with at least one of a processor and a logic circuit. For example, the processor is at least one of a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit). For example, the logic circuit is at least one of an ASIC (Application Specific Integrated Circuit) and an FPGA (Field-Programmable Gate Array). The image processing unit 30, etc. may include one or more processors. The image processing unit 30, etc. may include one or more logic circuits.

[0051] The computer of the endoscope device 1 may load and execute a program. The program includes instructions that define the operation of the image processing unit 30, etc. In other words, the functions of the image processing unit 30, etc. may be realized by software.

[0052] The above program may be provided by a "computer-readable recording medium" such as a flash memory. The program may be transmitted from a computer storing the program to the endoscope device 1 via a transmission medium or by transmission waves in the transmission medium. A "transmission medium" that transmits the program is a medium that has the function of transmitting information. Media that have the function of transmitting information include networks (communication networks) such as the Internet and communication lines (communication lines) such as telephone lines. The above program may realize some of the functions described above. Furthermore, the above program may be a difference file (difference program). The functions described above may be realized by combining a program already recorded on a computer with a difference program.

[0053] 3 shows the main configuration of the endoscope device 1. Using FIG. 3, a process for assisting the insertion operation of the insertion portion 2 will be described.

[0054] The configuration shown in FIG. 3 will be outlined below. The path processing unit 41 has a setting unit 410, a target state estimation unit 411 (state estimation unit), and a path calculation unit 412. The setting unit 410 sets an end point (first position) and a start point (second position) in the shape information. The shape information indicates the 3D shape of the subject. The end point indicates a target position. The start point is different from the end point. The target state estimation unit 411 estimates a first state of the tip portion 20 at the end point based on the specifications of the insertion unit 2. The path calculation unit 412 calculates a path that the tip portion 20 will take when moving from the start point to the end point. The state detection unit 40 detects a second state of the tip portion 20 at the start point. The insertion support unit 42 outputs insertion support information to the display unit 5 (information notification device) that is required for an insertion operation in which the tip portion 20 travels from the start point to the end point along the above path and the state of the tip portion 20 changes from the second state to the first state.

[0055] The configuration shown in Fig. 3 will be described in detail. The operation processing unit 33 acquires device setting information based on information output from the operation unit 4. The device setting information includes insertion unit information and optical adapter information. The insertion unit information indicates the specifications of the insertion unit 2. The optical adapter information indicates the specifications of the optical adapter 7.

[0056] The image processing unit 30 may have an image rotation function. If the image processing unit 30 has the image rotation function, the image processing unit 30 can rotate the image in any direction. The device setting information may include information indicating whether the image rotation function is set.

[0057] The types of optical adapter 7, which are one of the specifications of the optical adapter 7, include a direct viewing adapter and a side viewing adapter. A direct viewing adapter acquires an optical image of the subject viewed in the longitudinal direction of the insertion section 2. A side viewing adapter acquires an optical image of the subject viewed in a direction perpendicular to the side surface of the insertion section 2. There are optical adapters that can switch between a direct viewing state and a side viewing state. The device setting information may include information indicating either the direct viewing state or the side viewing state.

[0058] For example, the memory 43 stores device setting information. The user operates the operation unit 4 to input information about the insertion unit 2 and optical adapter 7 to be used to the endoscope device 1. The operation processing unit 33 identifies the insertion unit 2 and optical adapter 7 to be used based on the information output from the operation unit 4. The operation processing unit 33 obtains insertion unit information corresponding to the identified insertion unit 2 from the memory 43, and obtains optical adapter information corresponding to the identified optical adapter 7 from the memory 43.

[0059] As described above, the external PC 8 outputs object information. The object information includes the shape information, material information, and position information described above. The external PC 8 may output device setting information. The device setting information output from the external PC 8 may be input to the external IF 32.

[0060] The image processing unit 30 performs self-position estimation such as SLAM and calculates the position and orientation of the tip unit 20. The image processing unit 30 outputs information indicating each of the calculated position and orientation. The position detection unit 34 detects the position of the tip unit 20 and outputs information indicating the detected position. The orientation detection unit 35 detects the orientation of the tip unit 20 and outputs information indicating the detected orientation.

[0061] The state detection unit 40 detects the position and orientation of the tip unit 20 based on the information output from the image processing unit 30, the position detection unit 34, and the orientation detection unit 35. The state detection unit 40 does not need to use all of the above information. The state detection unit 40 may detect the position of the tip unit 20 by using one or both of the position calculated by the image processing unit 30 and the position detected by the position detection unit 34. The state detection unit 40 may detect the orientation of the tip unit 20 by using one or both of the orientation calculated by the image processing unit 30 and the orientation detected by the orientation detection unit 35. The state detection unit 40 may select the information used to detect the position or orientation of the tip unit 20 depending on the structure of the subject, the required positional accuracy, etc.

[0062] The state detection section 40 may detect the bending state of the bending portion 21 based on information output from the bending control section 39. This allows the state detection section 40 to detect the general shape of the bending portion 21. In order to detect the posture of the distal end portion 20, the state detection section 40 may use the general shape of the bending portion 21 in addition to the posture detected by the image processing section 30 or the posture detection section 35.

[0063] The setting unit 410 sets the end point and the start point based on the position information included in the object information. The setting unit 410 may set the current position of the tip portion 20 as the start point. In this case, the setting unit 410 sets the position of the tip portion 20 output from the state detection unit 40 as the start point.

[0064] The target state estimation unit 411 estimates a target state of the tip portion 20 when the tip portion 20 is virtually positioned at the end point. The target state indicates a state suitable for observing a subject. The setting unit 410 sets the position where the target state is realized as the end point.

[0065] The path calculation unit 412 virtually moves the tip portion 20 from the end point to the start point on the 3D shape of the subject, and records the path taken by the tip portion 20. If the 3D shape does not have a branching portion, the path calculation unit 412 calculates one or more path candidates. If the 3D shape has one or more branching portions, the path calculation unit 412 calculates two or more path candidates taking into account the branching pattern at the branching portion. The path calculation unit 412 analyzes the state of the insertion unit 2 when it passes through each of the path candidates, and selects one of the one or more path candidates as the optimal path.

[0066] The insertion support unit 42 generates insertion support information based on the position and posture of the tip portion 20 detected by the state detection unit 40, the optimal route calculated by the route processing unit 41, and shape information included in the subject information. Specifically, the insertion support unit 42 calculates the amount of deviation between the ideal posture of the tip portion 20 on the optimal route and the current posture of the tip portion 20. At this time, the insertion support unit 42 calculates the amount of deviation between the twisted state of the insertion unit 2 on the optimal route and the current twisted state of the insertion unit 2.

[0067] The insertion support unit 42 generates insertion support information. The insertion support information includes information that supports an operation for adjusting the attitude of the distal end portion 20. The insertion support information also includes information that supports an operation for adjusting the twisted state of the insertion unit 2. The insertion support unit 42 outputs the insertion support information to the display unit 5 via the image processing unit 30.

[0068] 4 shows the main processing executed by the endoscope device 1. Inspection conditions including subject information and device setting information are input to the endoscope device 1 (step SA). After step SA, the route processing unit 41 calculates an optimal route (step SB). After step SB, the insertion support unit 42 generates insertion support information (step SC). When step SC is executed, the processing shown in FIG. 4 ends. The processing shown in FIG. 4 may be repeated.

[0069] Fig. 5 shows the processing executed in step SA shown in Fig. 4. The external IF 32 acquires object information output from the external PC 8 (step SA1). After step SA1, the operation processing unit 33 acquires device setting information based on information output from the operation unit 4 (step SA2). After step SA2, the state detection unit 40 detects the position and posture of the tip portion 20 (step SA3). When step SA3 is executed, the processing shown in Fig. 5 ends.

[0070] The order of steps SA1, SA2, and SA3 is not limited to the order shown in Fig. 5. Steps SA1, SA2, and SA3 may be performed in any order.

[0071] FIG. 6 shows an example of shape information and position information included in object information. The shape information indicates a 3D shape SUB10 of the object. The object shown in FIG. 6 is a pipe. The position information indicates a starting point PS10 and an ending point PE10. The 3D shape SUB10 has a branching portion BR10 and a branching portion BR11. The path from the starting point PS10 to the ending point PE10 branches into two paths at the branching portion BR11. The two paths merge at the branching portion BR10.

[0072] 6 is set in advance. The observation object OT10 is an examination region. The object information also includes this information.

[0073] FIG. 7 shows an example of device setting information. The device setting information includes insertion section information including specifications of the insertion section 2 and optical adapter information including specifications of the optical adapter 7. The insertion section information includes information indicating the diameter DM10 of the insertion section 2. The insertion section information includes information indicating each of the rigid section length L10 and the bending length L11. The rigid section length L10 indicates the length of the rigid section of the tip section 20. The bending length L11 indicates the overall length of the tip section 20 and the bending section 21. The optical adapter information includes information indicating the observation direction OD10 of the optical adapter 7. The observation direction OD10 is parallel to the optical axis of the observation optical system, such as the lens 70. The optical adapter information includes information indicating each of the focal range FR10 of the optical adapter 7 and the angle of view AG10 of the optical adapter 7.

[0074] Fig. 8 shows the processing executed in step SB shown in Fig. 4. The target state estimation unit 411 estimates the target state of the tip portion 20 at the end point (step SB1). After step SB1, the path calculation unit 412 calculates one or more path candidates in which the state of the tip portion 20 at the end point becomes the target state (step SB2). After step SB2, the path calculation unit 412 selects any one of the one or more path candidates as the optimal path (step SB3). When step SB3 is executed, the processing shown in Fig. 8 ends.

[0075] 9 shows the processing executed in step SB1 shown in FIG. 8. Assume that the insertion portion 2 has been inserted into the subject and has reached the examination site. The position and posture of the tip portion 20 in this state are estimated. In the following processing, the tip portion 20 is virtually placed on the 3D shape indicated by the shape information included in the subject information.

[0076] First, the target state estimation unit 411 refers to the position information included in the object information acquired in step SA1, and places the tip portion 20 at the end point indicated by the position information. At this time, the target state estimation unit 411 refers to the optical adapter information included in the device setting information acquired in step SA2. The target state estimation unit 411 places the tip portion 20 so that it is within the field of view (angle of view AG10 in FIG. 7) of the optical adapter 7 and within the focal range (focal range FR10 in FIG. 7) of the optical adapter 7. The target state estimation unit 411 places the tip portion 20 so that the state of the tip portion 20 becomes the target state (step SB100).

[0077] 10, 11 and 12 show examples of target states for the tip portion 20.

[0078] FIG. 10 shows whether the tip portion 20 faces the object. In state ST10 shown in FIG. 10, the tip portion 20 faces the object. At this time, the direction of the optical axis OA10 of the lens 22 arranged in the tip portion 20 is perpendicular to the surface of the object. In other words, the angle AG11 shown in FIG. 10 is 90 degrees. The tip portion 20 acquires an optical image OI10.

[0079] 10, the tip portion 20 does not face the object. At this time, the direction of the optical axis OA10 of the lens 22 arranged in the tip portion 20 is not perpendicular to the surface of the object. The tip portion 20 acquires an optical image OI11.

[0080] The optical image OI10 is more suitable for observing the object than the optical image OI11. The state ST10 is a target state, and the state ST11 is not a target state.

[0081] 11 shows whether the predetermined direction of the tip portion 20 coincides with the predetermined direction of the subject. The predetermined direction of the tip portion 20 is a direction that is preset in the structure of the tip portion 20. The predetermined direction of the subject is a direction that is preset in the 3D shape of the subject.

[0082] 11, the upward, downward, leftward, and rightward directions of the tip portion 20 correspond to the upward, downward, leftward, and rightward directions of the subject, respectively. The tip portion 20 acquires an optical image OI12.

[0083] 11, the insertion portion 2 is twisted. Therefore, the upward direction of the tip portion 20 coincides with the downward direction of the subject, and the downward direction of the tip portion 20 coincides with the upward direction of the subject. The tip portion 20 acquires an optical image OI13.

[0084] The optical image OI12 is more suitable for observing the object than the optical image OI13. The state ST12 is a target state, and the state ST13 is not a target state.

[0085] Figure 12 shows whether the distance between the tip portion 20 and the subject is appropriate for observing the subject. In state ST14 shown in Figure 12, the distance between the tip portion 20 and the subject is D10. At this time, the examination site of the subject is within the field of view of the optical adapter 7, and the focal length of the optical adapter 7 is appropriate. The tip portion 20 acquires an optical image OI14.

[0086] 12, the distance between the tip portion 20 and the subject is D11. At this time, a part of the examination region of the subject is not within the field of view of the optical adapter 7, and the focal length of the optical adapter 7 is not appropriate. The tip portion 20 acquires an optical image OI15.

[0087] The optical image OI14 is more suitable for observing the object than the optical image OI15. The state ST14 is a target state, and the state ST15 is not a target state.

[0088] The target state estimation unit 411 positions the tip unit 20 so that the state of the tip unit 20 becomes a target state similar to at least one of state ST10, state ST12, and state ST14. The target state does not have to be similar to all of state ST10, state ST12, and state ST14. The target state may be similar to one or two of state ST10, state ST12, and state ST14.

[0089] After step SB100, the target state estimation unit 411 determines whether the placement of the tip portion 20 is obstructed by the wall of the subject (step SB101).

[0090] If the target state estimation unit 411 determines in step SB101 that the placement of the tip portion 20 is obstructed by the wall of the subject, the target state estimation unit 411 outputs a message to the display unit 5 via the image processing unit 30 (step SB102). The message prompts the user to replace the optical adapter 7 or notifies the user that the examination cannot be performed.

[0091] If the target state estimation unit 411 determines in step SB101 that the placement of the tip portion 20 is not obstructed by the wall of the subject, the setting unit 410 sets the position where the tip portion 20 is placed in step SB100 as the final end point. In addition, the setting unit 410 sets the posture of the tip portion 20 in step SB100 as the target posture (step SB103). When step SB102 or step SB103 is executed, the processing shown in FIG. 9 ends.

[0092] Fig. 13 shows the processing executed in step SB2 shown in Fig. 8. The setting unit 410 sets the starting point indicated by the position information included in the object information (step SB200).

[0093] The path calculation unit 412 sets the position of the tip portion 20 to the end point set in step SB103. As a result, the path calculation unit 412 places the tip portion 20 at the end point in the 3D shape of the subject. In addition, the path calculation unit 412 sets the orientation of the tip portion 20 to the target orientation set in step SB103 (step SB201).

[0094] After step SB201, the path calculation unit 412 moves the tip portion 20 on the 3D shape so that the tip portion 20 approaches the starting point along the wall of the subject (step SB202).

[0095] If the tip portion 20 passes through a branching section, the path calculation unit 412 records the position of the branching section (step SB203). After step SB203, the path calculation unit 412 sets the movement direction of the tip portion 20 according to the shape of the branching section (step SB204). If the tip portion 20 does not pass through a branching section, steps SB203 and SB204 are not executed.

[0096] After step SB204, the path calculation unit 412 determines whether the tip portion 20 has come into contact with the wall of the subject (step SB205).

[0097] If the path calculation unit 412 determines in step SB205 that the tip portion 20 has come into contact with a wall of the subject, the path calculation unit 412 records the position of the wall (step SB209). After step SB209, the path calculation unit 412 sets the position of the tip portion 20 to the position of the wall (step SB210). After step SB210, step SB202 is executed.

[0098] If the path calculation unit 412 determines in step SB205 that the tip portion 20 is not in contact with the wall of the subject, the path calculation unit 412 determines whether the tip portion 20 has reached the starting point (step SB206).

[0099] If the path calculation unit 412 determines in step SB206 that the tip 20 has not reached the starting point, step SB202 is executed. If the path calculation unit 412 determines in step SB206 that the tip 20 has reached the starting point, the path calculation unit 412 records a path candidate including the position of the branch and the position of the wall (step SB207).

[0100] After step SB207, the route calculation unit 412 determines whether all branching conditions at each branching point have been reflected in the route candidates (step SB208).

[0101] If the route calculation unit 412 determines in step SB208 that there are still branching states that have not been reflected in the route candidates, step SB201 is executed. Thereafter, the route calculation unit 412 executes the process described above and records the positions of branching points and walls in the new route candidates. If the route calculation unit 412 determines in step SB208 that all branching states have been reflected in the route candidates, the process shown in FIG. 13 ends.

[0102] Fig. 14 shows an example of a path candidate. A starting point PS10 and an ending point PE10 are set on a 3D shape SUB10 of a subject. The 3D shape SUB10 has a branching portion BR10 and a branching portion BR11. In Fig. 14, a path candidate PH10 and a path candidate PH11 are shown.

[0103] In step SB201, the path calculation unit 412 sets the position of the tip portion 20 to the end point PE10. In step SB202, the path calculation unit 412 moves the tip portion 20 toward the start point PS10.

[0104] Because the tip portion 20 passes through the branch portion BR10, the path calculation unit 412 records the position of the branch portion BR10 in step SB203. The tip portion 20 can move in the direction DR11 or the direction DR12 at the branch portion BR10. For example, the path calculation unit 412 sets the movement direction of the tip portion 20 to the direction DR11 in step SB204.

[0105] The path calculation unit 412 uses a branch management table for managing the branch status at the branching section. Figures 15(a), 15(b), 15(c), and 15(d) show examples of the branch management table. When the tip end 20 passes through the branching section BR10, the path calculation unit 412 records a first branch status including the direction DR11 at the branching section BR10 in the branch management table. Figure 15(a) shows the branch management table TB10 at this time.

[0106] If the tip portion 20 comes into contact with a wall of the 3D shape SUB10, the path calculation unit 412 executes steps SB209 and SB210. Thereafter, the tip portion 20 passes through the branching portion BR11. The path calculation unit 412 records the position of the branching portion BR11 in step SB203. The tip portion 20 can move in the direction DR12 at the branching portion BR11. The path calculation unit 412 sets the movement direction of the tip portion 20 to the direction DR12 in step SB204. At this time, the path calculation unit 412 associates the direction DR12 at the branching portion BR11 with the first branching state. Figure 15(b) shows the branch management table TB11 at this time.

[0107] When the tip portion 20 reaches the starting point PS10, the branch state including the direction DR12 at the branch point BR10 is not recorded in the branch management table TB11. Therefore, in step SB208, the path calculation unit 412 determines that there remains a branch state that has not been reflected in the path candidate. In step SB201, the path calculation unit 412 resets the position of the tip portion 20 to the end point PE10. In step SB202, the path calculation unit 412 moves the tip portion 20 toward the starting point PS10.

[0108] Because the tip end 20 passes through the branching point BR10, the path calculation unit 412 records the position of the branching point BR10 in step SB203. Because the direction DR12 at the branching point BR10 is not recorded in the branch management table TB11, the path calculation unit 412 sets the movement direction of the tip end 20 to the direction DR12 in step SB204. At this time, the path calculation unit 412 records a second branch state including the direction DR12 at the branching point BR10 in the branch management table. Figure 15(c) shows the branch management table TB12 at this time.

[0109] Thereafter, the tip 20 passes through the branching point BR11. The path calculation unit 412 records the position of the branching point BR11 in step SB203. The path calculation unit 412 sets the moving direction of the tip 20 to the direction DR12 in step SB204. At this time, the path calculation unit 412 associates the direction DR12 at the branching point BR11 with the second branching state. Figure 15(d) shows the branching management table TB13 at this time.

[0110] When the tip end 20 reaches the starting point PS10, the branch management table TB13 stores information on all branch states related to the branch points BR10 and BR11. Therefore, in step SB208, the path calculation unit 412 determines that all branch states have been reflected in the path candidates.

[0111] Fig. 16 shows the processing executed in step SB3 shown in Fig. 8. The path calculation unit 412 analyzes the bending load of the insertion section 2 for each path candidate (step SB300).

[0112] When the insertion portion 2 is bent, the frictional force between the subject and the insertion portion 2 increases. This frictional force applies a bending load to the insertion portion 2 when the insertion portion 2 is inserted into the subject. This makes it difficult for the user to insert the insertion portion 2 into the subject. The more the insertion portion 2 bends, the greater the bending load.

[0113] 17(a), 17(b), and 17(c) show a method for calculating the amount of bending load of the insertion section 2. FIG. 17(a) shows an example of a path candidate. The path candidate PH12 shown in FIG. 17(a) bends by an angle θ1 at a position P11 and by an angle θ2 at a position P12. The angles θ1 and θ2 are 90 degrees.

[0114] Fig. 17(b) shows changes in the amount of bending of the insertion section 2. The horizontal axis in Fig. 17(b) represents the position P on the path candidate PH12, and the vertical axis in Fig. 17(b) represents the amount of bending of the insertion section 2. The amount of bending of the insertion section 2 is shown as an angle θ.

[0115] FIG. 17(c) shows the amount of bending load. The horizontal axis in FIG. 17(c) indicates the position P on the path candidate PH12, and the vertical axis in FIG. 17(c) indicates the amount of bending load Ld. The path calculation unit 412 calculates the differential value of the change in the amount of bending shown in FIG. 17(b), and calculates the absolute value of the differential value as the amount of bending load Ld. At this time, the path calculation unit 412 uses shape information of the object. The path calculation unit 412 can calculate the amount of sudden and large bending.

[0116] The path calculation unit 412 calculates the sum of the absolute values of the differential values for each path candidate. This sum indicates the amount of turning load for the path candidate. If the amount of turning load is very large, the path calculation unit 412 determines that the path candidate cannot be used. If the amount of turning load is small, the path calculation unit 412 determines that the path candidate can be used.

[0117] After step SB300, route calculation section 412 analyzes the slope characteristics of each route candidate (step SB301).

[0118] 18(a) and 18(b) show a method for analyzing slope characteristics. FIG. 18(a) shows the slope characteristics of the route candidate PH13. In FIG. 18(a), the change in height of the route candidate PH13 is shown. The route candidate PH13 ascends from position P13 to position P14 and descends from position P14 to position P15. The gradient of the slope from position P13 to position P14 is T1, and the gradient of the slope from position P14 to position P15 is T2.

[0119] Figure 18(b) shows changes in the gradient of the slope on the route candidate PH13. The horizontal axis in Figure 18(b) represents the position P on the route candidate PH13, and the vertical axis in Figure 18(b) represents the gradient T of the slope.

[0120] If the gradient of the upward slope is steep and long, a large force is required to move the insertion portion 2. The path calculation unit 412 determines whether the gradient of the slope is greater than a threshold value Tth and whether the length of the slope is longer than a threshold value Llmt. At this time, the path calculation unit 412 uses shape information of the subject. In the example shown in FIG. 18(b), the slope T1 is greater than the threshold value Tth. Furthermore, the length of the slope with the slope T1 is longer than the threshold value Llmt. Therefore, the path calculation unit 412 determines that the path candidate PH13 cannot be used.

[0121] The route calculation unit 412 calculates the gradient of the slope for each route candidate. If the gradient is greater than a threshold Tth and the length of the slope is longer than a threshold Llmt, the route calculation unit 412 determines that the route candidate cannot be used. If the gradient of the slope is equal to or less than the threshold Tth, or if the length of the slope with a gradient greater than the threshold Tth is equal to or less than the threshold Llmt, the route calculation unit 412 determines that the route candidate can be used.

[0122] Most of the insertion portion 2, excluding the hard portion, is soft and easily bends. When the insertion portion 2 is climbing an incline, the direction of the tip portion 20 is likely to change to the left or right, and the insertion portion 2 is likely to tip over. If there are no walls or the like on both sides of the insertion portion 2 to prevent the insertion portion 2 from tipping over, the insertion portion 2 may not be able to climb the incline. The path calculation unit 412 may determine whether a path candidate can be used based on the distance between the insertion portion 2 and a wall or the like of the subject.

[0123] After step SB301, the path calculation unit 412 analyzes limitations on the placement of the insertion portion 2 due to the structure of the subject and the structure of the insertion portion 2 in each path candidate (step SB302).

[0124] The order of steps SB301, SB302, and SB303 is not limited to the order shown in Fig. 16. Steps SB301, SB302, and SB303 may be executed in any order.

[0125] 19, 20, and 21 show examples of limitations on the placement of the insertion portion 2 due to the structure of the subject and the structure of the insertion portion 2. FIG.

[0126] FIG. 19 shows an example of restrictions on the placement of the insertion portion 2 depending on the diameter of the insertion portion 2. In state ST16 shown in FIG. 19, the width Ws of the space inside the subject into which the insertion portion 2 is inserted is the same as W1. Width W1 indicates the width of the space necessary for the insertion portion 2 to pass through the space inside the subject. Width W1 is equal to or greater than the diameter of the insertion portion 2. In state ST16, the insertion portion 2 can pass through the space inside the subject.

[0127] 19, the width Ws of the space inside the subject into which the insertion portion 2 is inserted is smaller than W1. In state ST17, the insertion portion 2 cannot pass through the space inside the subject.

[0128] The path calculation unit 412 calculates the width W1 by using the insertion unit information. The path calculation unit 412 calculates the width Ws of the space in each path candidate by using the shape information of the subject. The path calculation unit 412 determines whether the width Ws is equal to or greater than the width W1. If the width Ws is smaller than the width W1, the path calculation unit 412 determines that the path candidate cannot be used. If the width Ws is equal to or greater than the width W1, the path calculation unit 412 determines that the path candidate can be used.

[0129] Figure 20 shows an example of restrictions on the placement of the insertion section 2 according to the length of the rigid portion of the insertion section 2. In state ST18 shown in Figure 20, the minimum width Wm at the curved portion of the subject is equal to or greater than r1. Width r1 indicates the width of the curved portion required for the rigid portion of the tip portion 20 to pass through the curved portion. In state ST18, the insertion section 2 can pass through the curved portion.

[0130] 20, the minimum width Wm of the subject at the curved portion is smaller than r1. In state ST19, the insertion portion 2 cannot pass through the curved portion.

[0131] The path calculation unit 412 calculates the width r1 by using the insertion section information. The path calculation unit 412 calculates the minimum value Wm of the width of the curved portion in each path candidate by using the shape information of the subject. The path calculation unit 412 determines whether the minimum value Wm is equal to or greater than the width r1. If the minimum value Wm is smaller than the width r1, the path calculation unit 412 determines that the path candidate cannot be used. If the minimum value Wm is equal to or greater than the width r1, the path calculation unit 412 determines that the path candidate can be used.

[0132] Figure 21 shows an example of restrictions on the positioning of the insertion section 2 according to the bending length of the insertion section 2. In state ST20 shown in Figure 21, the distance W2 between the walls of the subject is equal to or greater than the distance R1. The distance R1 indicates the distance between the walls required for the insertion section 2 to acquire an optical image of the examination region IP10 when the insertion section 2 is bent. The distance R1 is equal to or greater than the bending length. In state ST20, the tip 20 of the insertion section 2 can face the examination region IP10.

[0133] 21, the distance W2 between the walls of the subject is smaller than the distance R1. In the state ST21, the tip portion 20 of the insertion portion 2 cannot face the examination region IP10.

[0134] The path calculation unit 412 calculates the distance R1 by using the insertion unit information. The path calculation unit 412 calculates the distance W2 for each path candidate by using the shape information of the subject. The path calculation unit 412 determines whether the distance W2 is equal to or greater than the distance R1. If the distance W2 is smaller than the distance R1, the path calculation unit 412 determines that the path candidate cannot be used. If the distance W2 is equal to or greater than the distance R1, the path calculation unit 412 determines that the path candidate can be used.

[0135] After step SB302, the route calculation unit 412 refers to the analysis results of each route candidate in steps SB300 to SB302. The route calculation unit 412 determines whether there is a route candidate whose result is determined to be unusable (step SB303).

[0136] If the route calculation unit 412 determines in step SB303 that there is no route candidate with a result determined to be unusable, the route calculation unit 412 selects the route candidate with the best analysis result as the optimal route (step SB305).

[0137] If the path calculation unit 412 determines in step SB303 that there is a path candidate whose result is determined to be unusable, the path calculation unit 412 outputs a message to the display unit 5 via the image processing unit 30 (step SB304). The message prompts the user to replace the endoscope device 1 or notifies the user that the examination cannot be performed. When step SB304 or step SB305 is executed, the processing shown in FIG. 16 ends.

[0138] FIG. 22 shows an example of the analysis results of each route candidate in each of steps SB300 to SB302. The analysis results of each route candidate include the analysis result of the bending load in step SB300, the analysis result of the tilt characteristics in step SB301, and the analysis result of the restrictions on the placement of the insertion section 2 in step SB302. Each analysis result is one of Result A, Result B, and Result C. Result A indicates that the route candidate is good. Result B indicates that the analysis result is not classified as Result A, but the route candidate can be used. Result C indicates that the route candidate cannot be used.

[0139] The analysis results for each of route candidates 1 to 4 are shown in Fig. 22. The total length (route length) of each route candidate is also shown in Fig. 22. In step SB303, route calculation unit 412 determines whether there is a route candidate having result C.

[0140] The analysis result of the bending load of route candidate 1 is result C. The analysis result of the constraints of route candidate 2 is result C. The analysis result of the slope characteristics of route candidate 3 is result C. Therefore, the route calculation unit 412 does not select route candidates 1 to 3 as the optimal route. Route candidate 4 does not have result C. Therefore, the route calculation unit 412 selects route candidate 4 as the optimal route.

[0141] If there are two or more route candidates that do not have results that are determined to be unusable, the route calculation unit 412 may select the route candidate with the relatively better analysis result as the optimal route. Alternatively, the route calculation unit 412 may select the route candidate with the shortest route length as the optimal route.

[0142] In the above example, the path calculation unit 412 analyzes each path candidate according to two or more indices related to at least one of the insertion unit 2 and the subject. In the above example, a first index, a second index, and a third index are used. The first index relates to the load applied to the insertion unit 2 when the insertion unit 2 bends on each path candidate. The second index relates to the characteristics of the slope in each path candidate. The third index relates to restrictions on the placement of the insertion unit 2 due to the respective structures of the insertion unit 2 and the subject. The path calculation unit 412 may analyze each path candidate according to only one index.

[0143] For example, the route calculation unit 412 executes the following process. The route calculation unit 412 analyzes each route candidate according to each index, and calculates an index value according to the analysis result. For example, the index value of result A shown in FIG. 22 is 2, the index value of result B shown in FIG. 22 is 1, and the index value of result C shown in FIG. 22 is 0. For example, the route calculation unit 412 calculates the sum SUM of the three index values of each route candidate by using the following formula (1): SUM=I1+I2+I3 (1)

[0144] The value I1 in equation (1) represents the value of the first index. The value I2 in equation (1) represents the value of the second index. The value I3 in equation (1) represents the value of the third index. The route calculation unit 412 selects the route candidate with the smallest total SUM as the optimal route.

[0145] A weighting factor may be set for each index according to the importance of each index. The route calculation unit 412 may analyze each route candidate according to each index and weighting factor. For example, the weighting factor for each index indicates whether or not the index is used to select the optimal route. For example, the weighting factor for the first index is 1, the weighting factor for the second index is 1, and the weighting factor for the third index is 0. In this case, the first index and the second index are used to select the optimal route, but the third index is not used to select the optimal route.

[0146] The path calculation unit 412 may perform the following process. The path calculation unit 412 calculates a corrected index value by multiplying each index value by the weighting coefficient for each index. The path calculation unit 412 calculates the sum of the three corrected index values for each path candidate. For example, the path calculation unit 412 calculates the sum SUM of the three corrected index values for each path candidate by using the following equation (2). SUM=C1*I1+C2*I2+C3*I3 (2)

[0147] The value C1 in equation (2) indicates the weighting coefficient for the first index, and the value I1 indicates the value of the first index. The value C2 in equation (2) indicates the weighting coefficient for the second index, and the value I2 indicates the value of the second index. The value C3 in equation (2) indicates the weighting coefficient for the third index, and the value I3 indicates the value of the third index. The route calculation unit 412 selects the route candidate with the smallest total SUM as the optimal route.

[0148] The weighting coefficients are stored in advance in the memory 43. The weighting coefficients may be changeable. For example, the user may input the weighting coefficients to the endoscope device 1 by operating the operation unit 4. The weighting coefficients stored in the memory 43 may be changed to the weighting coefficients input to the endoscope device 1.

[0149] Fig. 23 shows the processing executed in step SC shown in Fig. 4. The insertion support unit 42 calculates the direction of the imaging surface 23a of the imaging element 23 when the distal end portion 20 is placed at the end point (step SC1). At this time, the attitude of the distal end portion 20 is set to the target attitude.

[0150] 24 shows a model of the path. When the tip portion 20 is placed at the end point, the tip portion 20 acquires an optical image of the inspection site IP11. Since the attitude of the tip portion 20 is set to the target attitude, the tip portion 20 faces the inspection site IP11.

[0151] Vectors Ua and Va shown in FIG. 24 are defined. Vector Ua is parallel to the surface of the examination region IP11 and coincides with the reference direction. For example, the reference direction is defined as the upward direction in the examination region IP11. Vector Va is perpendicular to the surface of the examination region IP11. Vectors Ua and Va are defined by a global coordinate system in the subject. The global coordinate system has an X-axis, a Y-axis, and a Z-axis. A position (3D coordinate) in the global coordinate system is defined by the X-coordinate, the Y-coordinate, and the Z-coordinate.

[0152] Meanwhile, a local coordinate system is defined on the imaging surface 23a of the image sensor 23. The local coordinate system has an x-axis, a y-axis, and a z-axis. The direction of the x-axis is the same as the horizontal direction within the imaging surface 23a, and the direction of the y-axis is the same as the vertical direction within the imaging surface 23a. The direction of the z-axis is perpendicular to the imaging surface 23a. A position (3D coordinate) in the local coordinate system is defined by the x-coordinate, the y-coordinate, and the z-coordinate.

[0153] When the tip portion 20 is disposed at the end point, the imaging surface 23a faces the examination part IP11, and the upward direction of the imaging surface 23a coincides with the upward direction of the examination part IP11. Therefore, the direction of the z axis of the local coordinate system coincides with the direction of the vector Va, and the direction of the y axis of the local coordinate system coincides with the direction of the vector Ua.

[0154] After step SC1, the insertion support unit 42 calculates the rotational state of the imaging surface 23a when the distal end portion 20 is moved from the end point to the start point along the optimal path without twisting (step SC2).

[0155] A unit vector i is set on the z-axis of the local coordinate system. The direction of the unit vector i coincides with the direction of the z-axis. The direction of the unit vector i indicates the movement direction of the insertion section 2. When the tip section 20 is positioned at the end point, the unit vector i is parallel to the vector Va.

[0156] The insertion support unit 42 calculates the amount of rotation of the unit vector i when the tip portion 20 passes through a bending position. The bending position is the position of a bifurcation of the subject or the position of the wall of the subject. The insertion support unit 42 calculates the unit vector i0 before the tip portion 20 passes through the bending position. The insertion support unit 42 also calculates the unit vector i1 after the tip portion 20 has passed through the bending position. At this time, the insertion support unit 42 calculates each unit vector as a value in the global coordinate system by using shape information of the subject.

[0157] The insertion support unit 42 uses the relationship between the unit vector i and a rotation matrix to calculate the amount of rotation of the unit vector i at the bend position. The insertion support unit 42 calculates the amount of rotation θ1 of the unit vector i by using a method such as the least squares method. The amount of rotation θ1 is expressed as a matrix in the global coordinate system. The insertion support unit 42 converts the amount of rotation in the global coordinate system into an amount of rotation in the local coordinate system. At this time, the condition that the unit vector i does not rotate around the z-axis (φ=0) is used. The unit vector i can rotate around the x-axis or y-axis.

[0158] The insertion support unit 42 repeats the above process until the tip portion 20 reaches the starting point. The insertion support unit 42 calculates the unit vector i2, etc., and calculates the rotation amount θ2, etc. When the tip portion 20 is positioned at the starting point, the tip portion 20 acquires an optical image of the examination site IP12.

[0159] Vectors Ub and Vb shown in FIG. 24 are defined. Vector Ub is parallel to the surface of the examination site IP12 and coincides with the reference direction. For example, the reference direction is defined as the upward direction in the examination site IP12. Vector Vb is perpendicular to the surface of the examination site IP12. Vectors Ub and Vb are defined in the global coordinate system. When the unit vector i at the starting point coincides with vector Vb, the insertion support unit 42 determines that the tip portion 20 has reached the starting point.

[0160] The insertion support unit 42 calculates the sum of the rotation amounts in the global coordinate system by using the rotation amount of the unit vector i at each bend position between the starting point and the ending point. As a result, the insertion support unit 42 calculates the rotation state of the imaging plane 23a at the starting point. The rotation state indicates the posture of the distal end portion 20 at the starting point.

[0161] At each bending position, the unit vector i does not rotate around the z-axis. When the above-described rotation state of the imaging plane 23a at the starting point is realized, the distal end portion 20 moves from the starting point to the end point without the insertion portion 2 rotating around the z-axis.

[0162] The insertion operation of the insertion portion 2 mainly includes an operation of pushing or pulling the insertion portion 2, an operation of twisting the insertion portion 2, and an operation of bending the bending portion 21. Because these operations are performed, the insertion operation is complicated and requires the user to be skilled in the operation. If the portion of the insertion portion 2 inserted into the subject is long, a large force is required to twist the insertion portion 2. This makes the user's work cumbersome. In step SC2, the insertion support unit 42 calculates the ideal rotation state of the imaging plane 23a at the starting point to prevent the insertion portion 2 from being twisted.

[0163] After step SC2, the insertion support unit 42 generates insertion support information based on the position and posture of the tip portion 20 detected by the state detection unit 40, the optimal path calculated by the path processing unit 41, and the shape information included in the subject information. The insertion support unit 42 outputs the insertion support information to the display unit 5 via the image processing unit 30, and displays the insertion support information on the display unit 5 (step SC3). When step SC3 is executed, the processing shown in FIG. 23 ends.

[0164] The insertion support unit 42 executes the following process in step SC3. FIG. 25 shows the state of the insertion unit 2 when the insertion support information is generated. The insertion support unit 42 calculates the magnitude of the angle AG20 between the current direction DR20 of the tip portion 20 and the direction DR21 of the optimal path PHb. The current direction DR20 of the tip portion 20 is indicated by the posture of the tip portion 20 detected by the state detection unit 40. The direction DR21 of the optimal path PHb is indicated by the optimal path and shape information. The insertion support unit 42 generates an image of an arrow indicating the direction DR21 of the optimal path PHb and the magnitude of the angle AG20, and displays the image on the display unit 5. The image is included in the insertion support information.

[0165] FIG. 26 shows an example of insertion support information displayed on the display unit 5. The display unit 5 displays an image IMG20 processed by the image processing unit 30. A window WD20 for supporting the insertion operation is displayed on the image IMG20. An arrow AR20 is displayed in the window WD20. The length of the arrow AR20 indicates the magnitude of the angle AG20. The range of the angle AG20 is from 0 degrees to 180 degrees. When the angle AG20 is 180 degrees, the arrow AR20 is at its longest.

[0166] The direction of the arrow AR20 indicates the direction of the optimal path PHb on the imaging surface 23a of the imaging element 23. The endoscope device 1 is usually designed so that the upward, downward, leftward, and rightward directions of the imaging surface 23a correspond to the upward, downward, leftward, and rightward directions of the bending operation, respectively. For example, the user tilts the joystick of the operation unit 4 in the direction of the arrow AR20. This causes the tip portion 20 to face in the direction DR21 of the optimal path PHb.

[0167] The insertion support unit 42 calculates the magnitude of the angle AG21 between the optimal upward direction DR22 of the imaging surface 23a and the current upward direction DR23 of the imaging surface 23a. The optimal upward direction DR22 of the imaging surface 23a is calculated in step SC2. The current upward direction DR23 of the imaging surface 23a is indicated by the posture of the tip portion 20 detected by the state detection unit 40. The insertion support unit 42 generates an image of an arrow indicating the magnitude of the angle AG21 and displays the image on the display unit 5. The image is included in the insertion support information.

[0168] An arrow AR21 is displayed in window WD20. The direction of the arrow AR21 indicates the magnitude of the angle AG21. When the arrow AR21 points upward on the display screen of the display unit 5, the twisted state of the insertion unit 2 is ideal. The user twists the insertion unit 2 so that the arrow AR21 points upward.

[0169] The insertion support unit 42 calculates the distance between the position of the tip portion 20 detected by the state detection unit 40 and the end point of the optimal path. The insertion support unit 42 displays distance information indicating the calculated distance on the display unit 5. The distance information is included in the insertion support information. The distance information DI20 is displayed on the image IMG20.

[0170] In the above example, the route processing unit 41 uses shape information included in the subject information output from the external PC 8. When the current position of the tip portion 20 is set as the starting point, the route processing unit 41 may calculate a position on the 3D shape of the subject corresponding to the current position based on the amount of change in position from a reference position on the subject to the current position. For example, the reference position is the position of the entrance where the insertion unit 2 is inserted. The route processing unit 41 may use the calculated position as the starting point. The route processing unit 41 may calculate the orientation of the tip portion 20 at the starting point based on the amount of change in the orientation of the tip portion 20 while the tip portion 20 moves from the reference position to the current position. For example, the orientation of the tip portion 20 at the reference position is known.

[0171] The image processing unit 30 may generate shape information including the position and posture calculated through self-position estimation. The route processing unit 41 may acquire the shape information from the image processing unit 30 in step SA1. The route processing unit 41 may calculate an optimal route in step SB by using the shape information. When the current position of the tip unit 20 is set as the starting point, the starting point is associated with a position on the 3D shape indicated by the shape information. The posture of the tip unit 20 at the starting point is associated with posture information included in the shape information.

[0172] The insertion support unit 42 displays the insertion support information on the display unit 5 (information notification device). The method for outputting the insertion support information is not limited to this.

[0173] The output unit of the insertion support unit 42 may output sound data to a speaker, causing the speaker to generate a sound corresponding to the insertion support information. The insertion support unit 42 may output a control signal indicating a vibration pattern to a vibration generator, causing the vibration generator to generate a vibration having a pattern corresponding to the comparison information. The insertion support unit 42 may output a control signal indicating a light emission pattern to a light source, causing the light source to generate light having a pattern corresponding to the insertion support information.

[0174] The insertion assistance method according to each aspect of the present invention assists the insertion operation of the insertion unit 2 when the insertion unit 2, which has a tip portion 20 that acquires an optical image of the subject, is inserted into the subject. The insertion assistance method includes a setting step, a state estimation step, a path calculation step, a state detection step, and an insertion assistance step. In the setting steps (steps SB200 and SB201), the setting unit 410 sets a first position (end point) and a second position (start point) in shape information. The shape information indicates the 3D shape of the subject. The first position indicates a target position. The second position is different from the first position. In the state estimation step (step SB1), the target state estimation unit 411 estimates a first state of the tip portion 20 at the first position based on the specifications of the insertion unit 2. In the path calculation step (step SB2), the path calculation unit 412 calculates a path that the tip portion 20 will take when moving from the second position to the first position. In a state detection step (step SA3), the state detection unit 40 detects the second state of the tip portion 20 at the second position. In an insertion support step (step SC), the insertion support unit 42 outputs to the display unit 5 (information notification device) insertion support information required for an insertion operation in which the tip portion 20 reaches the first position from the second position through the above-mentioned path and the state of the tip portion 20 changes from the second state to the first state.

[0175] Each aspect of the present invention may include the following modifications. When the 3D shape of the subject has a bifurcation, the path calculation unit 412 moves the tip unit 20, which is disposed at a first position (end point) in a first state, to a second position (start point) in the 3D shape (step SB202). The path calculation unit 412 records the bifurcation that the tip unit 20 passes through while moving from the first position to the second position (step SB203). The path calculation unit 412 calculates two or more path candidates, including a path candidate that passes through the bifurcation (step SB207), and selects one of the two or more path candidates as the above-mentioned path (optimal path) (step SB305).

[0176] Each aspect of the present invention may include the following modifications. When the 3D shape of the subject has a bifurcation, the path calculation unit 412 calculates two or more path candidates, including a path candidate that passes through the bifurcation (step SB207). The path calculation unit 412 analyzes the two or more path candidates according to one or more indicators related to at least one of the insertion unit 2 and the subject (steps SB300, SB301, and SB302). Based on the analysis result of the two or more path candidates, the path calculation unit 412 selects one of the two or more path candidates as the above-mentioned path (optimal path) (step SB305).

[0177] Each aspect of the present invention may include the following modifications: An importance (weighting coefficient) is set in advance for each of two or more indices. The route calculation unit 412 analyzes two or more route candidates according to the two or more indices and the importance (steps SB300, SB301, and SB302). The importance set for each of the two or more indices can be changed.

[0178] Each aspect of the present invention may include the following modifications: The path calculation unit 412 analyzes two or more path candidates by using at least one of information indicating the size of the insertion unit 2 and information indicating the shape of the subject (steps SB300, SB301, and SB302).

[0179] Each aspect of the present invention may include the following modifications: When an insertion operation is performed, the position detection unit (image processing unit 30 or position detection unit 34) detects the position of the tip portion 20. The setting unit 410 sets the second position (starting point) to the position detected by the position detection unit (step SB200).

[0180] Each aspect of the present invention may include the following modifications. The target state estimation unit 411 estimates a first state in which the state of the tip portion 20 at the first position (end point) is at least one of a first observation state, a second observation state, and a third observation state (step SB100). The first observation state (FIG. 10) is a state in which the direction of the optical axis OA10 of the observation optical system arranged in the tip portion 20 is perpendicular to the surface of the object. The second observation state (FIG. 11) is a state in which a predetermined direction at the tip portion 20 coincides with a predetermined direction in the 3D shape of the object. The third observation state (FIG. 12) is a state in which the distance between the tip portion 20 and the object is suitable for observing the object.

[0181] Each aspect of the present invention may include the following modifications: The insertion operation refers to at least one of an operation of moving the insertion portion 2 inside the subject, an operation of bending the insertion portion 2, and an operation of twisting the insertion portion 2.

[0182] Each aspect of the present invention may include the following modifications: The insertion assistance information includes at least one of the distance between the second position (starting point) and the first position (ending point), the amount of change in the direction of the tip portion 20 to align the direction of the tip portion 20 with the direction along the above-mentioned path (optimal path), and the amount of twist to align the twisted state of the insertion portion 2 with the twisted state of the insertion portion 2 in the first state.

[0183] Conventional techniques do not take into account the state of the insertion section 2 at the target position for observing the examination site. Therefore, even if various insertion assistance methods are implemented and the distal end portion 20 reaches the target position, the resulting state of the insertion section 2 may be poor, making it difficult to see the image of the subject. Also, in situations where there are physical constraints inside the subject, it may be necessary to twist or bend the insertion section 2. Furthermore, because the characteristics of the insertion path and the physical constraints of the endoscope are not taken into account, it may actually be difficult to perform the insertion operation according to the insertion assistance.

[0184] The insertion assist unit 42 outputs insertion assist information required for the insertion operation to bring the state of the tip portion 20 at the end point into the target state to the display unit 5. Therefore, the endoscope device 1 can assist the insertion operation to bring the state of the insertion portion 2 at the target position into a state suitable for observing the subject.

[0185] The path calculation unit 412 can calculate a path that applies a small load to the insertion unit 2. Therefore, the magnitude of the force that the user applies to the insertion unit 2 during the insertion operation is reduced.

[0186] The insertion support unit 42 can generate insertion support information for optimizing the posture of the tip portion 20, including the twisted state of the insertion unit 2. This reduces the complexity of the insertion operation. The user does not need to perform the insertion operation by trial and error at the examination site. This improves work efficiency. Furthermore, regardless of the user's examination skill, the user can easily reach the insertion unit 2 to the examination site, allowing for highly reliable examinations.

[0187] The insertion portion 2 may be inserted into the subject by using a driving device having a motor. In this case, too, the load on the insertion portion 2 is reduced by using the optimal path calculated by the path calculation unit 412.

[0188] (First Modification) A first modified example of the embodiment of the present invention will be described. The optical adapter information includes information indicating the optical characteristics of the optical adapter 7. Specifically, the optical adapter information includes information indicating the characteristics of the field of view of the optical adapter 7. In step SB100, the target state estimation unit 411 places the tip portion 20 at the end point according to the characteristics of the field of view of the optical adapter 7.

[0189] 27, 28, and 29 show examples of limitations on the placement of the insertion section 2 according to the bending length of the insertion section 2. The limitations on the placement of the insertion section 2 change according to the characteristics of the field of view of the optical adapter 7.

[0190] Figure 27 shows an example of limitations on the placement of the insertion section 2 regarding the bending length when the optical adapter 7 is a side-viewing adapter. The side-viewing adapter acquires an optical image of the subject viewed in a direction perpendicular to the side surface of the insertion section 2. An arrow AR22 shown in Figure 27 indicates the distance between the side-viewing adapter and the examination site IP20 required to observe the examination site IP20. The tip of the arrow AR22 indicates the target position.

[0191] A range RG20 is set based on the position of the tip of the arrow AR22. The range RG20 is fan-shaped. The width R2 of the range RG20 is the same as the curvature length. The angle AG22 of the range RG20 is set according to the width of the field of view of the side-viewing adapter.

[0192] 27, a part of the arc of the range RG20 is within a space surrounded by the wall of the subject. In the state ST22, the side viewing adapter can acquire an optical image of the examination region IP20.

[0193] 27, the arc of the range RG20 is not within the space surrounded by the walls of the subject. In state ST23, the side viewing adapter cannot acquire an optical image of the examination region IP20.

[0194] The path calculation unit 412 sets the range RG20 by using the insertion unit information and the optical adapter information. The path calculation unit 412 determines whether at least a portion of the arc of the range RG20 is within a space surrounded by the walls of the subject. If the entire arc of the range RG20 is not within that space, the path calculation unit 412 determines that the path candidate cannot be used. If at least a portion of the arc of the range RG20 is within that space, the path calculation unit 412 determines that the path candidate can be used.

[0195] Figure 28 shows an example of limitations on the placement of the insertion section 2 regarding the bending length when the optical adapter 7 is a narrow-angle adapter. A narrow-angle adapter has a narrow field of view. An arrow AR23 shown in Figure 28 indicates the distance between the narrow-angle adapter and the examination site IP21 required to observe the examination site IP21. The tip of the arrow AR23 indicates the target position.

[0196] A range RG21 is set based on the position of the tip of the arrow AR23. The range RG21 is fan-shaped. The width R3 of the range RG21 is the same as the curvature length. The angle AG23 of the range RG21 is set according to the width of the field of view of the narrow-angle adapter.

[0197] 28, the arc of the range RG21 is within the space surrounded by the wall of the subject. In state ST24, the narrow-angle adapter can acquire an optical image of the examination region IP21.

[0198] 28, the arc of the range RG21 is not within the space surrounded by the wall of the subject. In the state ST25, the narrow-angle adapter cannot acquire an optical image of the examination region IP21.

[0199] The path calculation unit 412 sets the range RG21 by using the insertion unit information and the optical adapter information. The path calculation unit 412 determines whether at least a portion of the arc of the range RG21 is within a space surrounded by the walls of the subject. If the entire arc of the range RG21 is not within that space, the path calculation unit 412 determines that the path candidate cannot be used. If at least a portion of the arc of the range RG21 is within that space, the path calculation unit 412 determines that the path candidate can be used.

[0200] Figure 29 shows an example of limitations on the placement of the insertion section 2 regarding the bending length when the optical adapter 7 is a wide-angle adapter. A wide-angle adapter has a wide field of view. An arrow AR24 shown in Figure 29 indicates the distance between the wide-angle adapter and the examination site IP22 required to observe the examination site IP22. The tip of the arrow AR24 indicates the target position.

[0201] A range RG22 is set based on the position of the tip of the arrow AR24. The range RG22 is fan-shaped. The width R4 of the range RG22 is the same as the curvature length. The angle AG24 of the range RG22 is set according to the width of the field of view of the wide-angle adapter.

[0202] 29, a part of the arc of the range RG22 is within a space surrounded by the wall of the subject. In state ST26, the wide-angle adapter can acquire an optical image of the examination region IP22.

[0203] 29, a part of the arc of the range RG22 is within the space surrounded by the wall of the subject. In the state ST27, the wide-angle adapter can acquire an optical image of the examination region IP22.

[0204] When a wide-angle adapter is used, it is easy to allow the subject to be observed at an oblique angle. Even if the space inside the subject is narrow and the curvature angle is small, the wide-angle adapter can acquire an optical image of the subject.

[0205] The path calculation unit 412 sets the range RG22 by using the insertion unit information and the optical adapter information. The path calculation unit 412 determines whether at least a portion of the arc of the range RG22 is within a space surrounded by the walls of the subject. If the entire arc of the range RG22 is not within that space, the path calculation unit 412 determines that the path candidate cannot be used. If at least a portion of the arc of the range RG22 is within that space, the path calculation unit 412 determines that the path candidate can be used.

[0206] The optical adapter 7 may be an optical adapter that can switch between a direct viewing state and a side viewing state. When such an optical adapter is used, the path calculation unit 412 can calculate path candidates in which the influence of the path length is reduced.

[0207] The image processing unit 30 may have an image rotation function. The insertion support unit 42 may determine, based on the device setting information, whether or not the image rotation function is set in the image processing unit 30. If the insertion support unit 42 determines that the image rotation function is set in the image processing unit 30, steps SC1 and SC2 shown in FIG. 23 do not need to be executed.

[0208] Each aspect of the present invention may include the following modifications: The path calculation unit 412 calculates a path (optimal path) based on the specifications of the optical adapter 7 attached to the tip portion 20.

[0209] The path calculation unit 412 can analyze restrictions on the placement of the insertion unit 2 based on the optical characteristics of the optical adapter 7. The insertion support unit 42 can determine whether to support the operation of twisting the insertion unit 2 based on the setting of the image rotation function.

[0210] (Second Modification) A second modification of the embodiment of the present invention will be described. The insertion support information in the second modification includes the insertion history of the insertion section 2.

[0211] The insertion support unit 42 records the position of the tip portion 20 detected by the state detection unit 40 in the memory 43. The insertion support unit 42 generates an image of the 3D shape of the subject based on the shape information included in the subject information. The insertion support unit 42 also reads out one or more positions of the tip portion 20 stored in the memory 43. The insertion support unit 42 superimposes the one or more positions on the image of the 3D shape. The insertion support unit 42 outputs the image to the display unit 5 via the image processing unit 30.

[0212] Fig. 30 shows an example of insertion support information displayed on the display unit 5. Explanation of the same parts as those shown in Fig. 26 will be omitted.

[0213] A window WD21 is displayed on the image IMG20. A 3D shape 3D20 of the subject and an insertion history IH20 are displayed in the window WD21. The insertion history IH20 indicates the history of one or more positions of the tip portion 20. The insertion history IH20 is superimposed on the image of the 3D shape 3D20. An optimal path calculated by the path calculation unit 412 may be superimposed on the image of the 3D shape 3D20.

[0214] Each aspect of the present invention may include the following modifications: The insertion assistance information includes a history of the positions that the tip portion 20 has passed through.

[0215] It is effective to display the actual path that the distal end 20 of the insertion section 2 takes from the insertion position to the current position. The path shows the current overall bending state of the insertion section 2 and the positions of the uphill or downhill slopes that the insertion section 2 has traveled. By referring to the insertion history, the user can determine whether the future insertion operation of the insertion section 2 will be easy.

[0216] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments and their modifications. Addition, omission, substitution, and other modifications of the configuration are possible within the scope of the spirit of the present invention. Furthermore, the present invention is not limited by the above description, but is limited only by the scope of the appended claims. [Explanation of symbols]

[0217] 1 Endoscopic device 2 Insertion section 3 Main body 4 Control section 5 Display section 6 Insertion length detection unit 7 Optical adapter 20 Tip 21 Curved section 22,70 lens 23 Image sensor 23a Imaging surface 24 Attitude Sensor 25 Light Guide 26 Angle wire 30 Image processing section 31 Recording Section 32 External IF 33 Operation processing section 34 Position detection unit 35 Attitude detection unit 36 light source 37 Lighting control unit 38 Motor 39 Bending control section 40 Status detection unit 41 Route processing unit 42 Insertion support unit 43 Memory 44 Power supply section 410 Settings 411 Target state estimation unit (state estimation unit) 412 Route calculation unit

Claims

1. 1. An insertion assistance system that assists an insertion operation of an insertion section having a tip portion for acquiring an optical image of a subject when the insertion section is inserted into a subject, the system comprising: a setting unit that sets a first position and a second position in shape information, the shape information indicating a three-dimensional shape of the subject, the first position indicating a target position, and the second position different from the first position; a state estimation unit that estimates a first state of the distal end portion at the first position based on specifications of the insertion unit; a path calculation unit that calculates a path from the second position to the first position along which the tip portion passes when the tip portion moves from the second position to the first position; a state detection unit that detects a second state of the tip end portion at the second position; an insertion support unit that outputs, to an information notification device, insertion support information required for the insertion operation in which the tip portion reaches the first position from the second position through the path and the state of the tip portion changes from the second state to the first state; and When the three-dimensional shape has a branching portion, the path calculation unit moves the tip portion, which is disposed at the first position in the first state, to the second position in the three-dimensional shape, and records the branching portion that the tip portion passes through while moving from the first position to the second position; The route calculation unit calculates two or more route candidates from the second position to the first position that pass through the branching section, and selects one of the two or more route candidates as the route. Insertion assistance system.

2. 1. An insertion assistance system that assists an insertion operation of an insertion section having a tip portion for acquiring an optical image of a subject when the insertion section is inserted into a subject, the system comprising: a setting unit that sets a first position and a second position in shape information, the shape information indicating a three-dimensional shape of the subject, the first position indicating a target position, and the second position different from the first position; a state estimation unit that estimates a first state of the distal end portion at the first position based on specifications of the insertion unit; a path calculation unit that calculates a path from the second position to the first position along which the tip portion passes when the tip portion moves from the second position to the first position; a state detection unit that detects a second state of the tip end portion at the second position; an insertion support unit that outputs, to an information notification device, insertion support information required for the insertion operation in which the tip portion reaches the first position from the second position through the path and the state of the tip portion changes from the second state to the first state; and When the three-dimensional shape has a branching portion, the path calculation unit calculates two or more path candidates that pass through the branching portion from the second position to the first position; the path calculation unit analyzes the two or more path candidates according to one or more indicators related to at least one of the insertion unit and the subject; The route calculation unit selects one of the two or more route candidates as the route based on an analysis result of the two or more route candidates. Insertion assistance system.

3. a level of importance is preset for each of two or more indices including the one or more indices; the route calculation unit analyzes the two or more route candidates in accordance with the two or more indicators and the importance; The importance level set for each of the two or more indicators can be changed. The insertion assist system according to claim 2 .

4. The path calculation unit analyzes the two or more path candidates by using at least one of information indicating the size of the insertion portion and information indicating the shape of the subject. The insertion assist system according to claim 2 .

5. a position detection unit that detects the position of the tip when the insertion operation is performed; The setting unit sets the second position to the position detected by the position detection unit. The insertion assist system according to claim 1 or 2.

6. the state estimation unit estimates the first state such that the state of the tip end at the first position is at least one of a first observation state, a second observation state, and a third observation state; the first observation state is a state in which the direction of the optical axis of the observation optical system disposed at the tip portion is perpendicular to the surface of the subject; the second observation state is a state in which a predetermined direction at the tip portion coincides with a predetermined direction in the three-dimensional shape, The third observation state is a state in which the distance between the tip and the subject is suitable for observing the subject. The insertion assist system according to claim 1 or 2.

7. The insertion operation indicates at least one of an operation of moving the insertion portion inside the subject, an operation of bending the insertion portion, and an operation of twisting the insertion portion. The insertion assist system according to claim 1 or 2.

8. The insertion assistance information includes at least one of a distance between the second position and the first position, an amount of change in the direction of the tip portion to make the direction of the tip portion coincide with the direction along the path, and an amount of twist to make the twisted state of the insertion portion coincide with the twisted state of the insertion portion in the first state. The insertion assist system according to claim 1 or 2.

9. The insertion support information includes a history of positions through which the tip portion has passed. The insertion assist system according to claim 1 or 2.

10. The path calculation unit calculates the path based on specifications of an optical adapter attached to the tip portion. The insertion assist system according to claim 1 or 2.

11. 1. An insertion assistance method for assisting an insertion operation of an insertion section having a tip portion for acquiring an optical image of a subject when the insertion section is inserted into a subject, the method comprising: a setting step in which a setting unit sets a first position and a second position in shape information, the shape information indicating a three-dimensional shape of the subject, the first position indicating a target position, and the second position being different from the first position; a state estimating step in which a state estimating unit estimates a first state of the distal end portion at the first position based on specifications of the insertion unit; a path calculation step in which a path calculation unit calculates a path from the second position to the first position along which the tip portion passes when the tip portion moves from the second position to the first position; a state detection step in which a state detection unit detects a second state of the tip end portion at the second position; an insertion support step in which an insertion support unit outputs insertion support information required for the insertion operation in which the tip portion reaches the first position from the second position through the path and the state of the tip portion changes from the second state to the first state to an information notification device; and When the three-dimensional shape has a branching portion, the path calculation unit moves the tip portion, which is disposed at the first position in the first state, to the second position in the three-dimensional shape, and records the branching portion that the tip portion passes through while moving from the first position to the second position; The route calculation unit calculates two or more route candidates from the second position to the first position that pass through the branching section, and selects one of the two or more route candidates as the route. Insertion assistance methods.

12. 1. An insertion assistance method for assisting an insertion operation of an insertion section having a tip portion for acquiring an optical image of a subject when the insertion section is inserted into a subject, the method comprising: a setting step in which a setting unit sets a first position and a second position in shape information, the shape information indicating a three-dimensional shape of the subject, the first position indicating a target position, and the second position being different from the first position; a state estimating step in which a state estimating unit estimates a first state of the distal end portion at the first position based on specifications of the insertion unit; a path calculation step in which a path calculation unit calculates a path from the second position to the first position along which the tip portion passes when the tip portion moves from the second position to the first position; a state detection step in which a state detection unit detects a second state of the tip end portion at the second position; an insertion support step in which an insertion support unit outputs insertion support information required for the insertion operation in which the tip portion reaches the first position from the second position through the path and the state of the tip portion changes from the second state to the first state to an information notification device; and When the three-dimensional shape has a branching portion, the path calculation unit calculates two or more path candidates that pass through the branching portion from the second position to the first position; the path calculation unit analyzes the two or more path candidates according to one or more indicators related to at least one of the insertion unit and the subject; The route calculation unit selects one of the two or more route candidates as the route based on an analysis result of the two or more route candidates. Insertion assistance methods.

13. 1. A program for causing a computer to execute a process for assisting an insertion operation of an insertion section having a tip portion for acquiring an optical image of a subject when the insertion section is inserted into the subject, the program comprising: The process comprises: a setting step of setting a first position and a second position in shape information, wherein the shape information indicates a three-dimensional shape of the subject, the first position indicates a target position, and the second position is different from the first position; a state estimating step of estimating a first state of the distal end portion at the first position based on specifications of the insertion portion; a path calculation step of calculating a path from the second position to the first position along which the tip portion passes when the tip portion moves from the second position to the first position; a state detecting step of detecting a second state of the tip portion at the second position; an insertion support step of outputting, to an information notification device, insertion support information required for the insertion operation in which the tip portion reaches the first position from the second position through the path and the state of the tip portion changes from the second state to the first state; and In the path calculation step, if the three-dimensional shape has a branching portion, the tip portion that is disposed at the first position in the first state is moved to the second position in the three-dimensional shape, and the branching portion that the tip portion passes through while moving from the first position to the second position is recorded; In the route calculation step, two or more route candidates that pass through the branching section and are from the second position to the first position are calculated, and one of the two or more route candidates is selected as the route. program.

14. 1. A program for causing a computer to execute a process for assisting an insertion operation of an insertion section having a tip portion for acquiring an optical image of a subject when the insertion section is inserted into the subject, the program comprising: The process comprises: a setting step of setting a first position and a second position in shape information, wherein the shape information indicates a three-dimensional shape of the subject, the first position indicates a target position, and the second position is different from the first position; a state estimating step of estimating a first state of the distal end portion at the first position based on specifications of the insertion portion; a path calculation step of calculating a path from the second position to the first position along which the tip portion passes when the tip portion moves from the second position to the first position; a state detecting step of detecting a second state of the tip portion at the second position; an insertion support step of outputting, to an information notification device, insertion support information required for the insertion operation in which the tip portion reaches the first position from the second position through the path and the state of the tip portion changes from the second state to the first state; and In the route calculation step, when the three-dimensional shape has a branching portion, two or more route candidates passing through the branching portion from the second position to the first position are calculated; In the path calculation step, the two or more path candidates are analyzed according to one or more indicators related to at least one of the insertion portion and the subject; In the route calculation step, one of the two or more route candidates is selected as the route based on an analysis result of the two or more route candidates. program.

Images