Endoscopic system, control method, and program

Abstract

The present invention provides an endoscopic system, control method, and program that do not require complex user operation and can reduce the amount of foreign matter that comes into contact with the observation optical system. [Solution] The endoscope system has a sensor at its tip, a bendable insertion section, and a control unit. The control unit receives a mode signal indicating a mode for automatically bending the insertion section. The control unit acquires data output from the sensor and determines the direction of gravity at the tip based on the data. After receiving the mode signal, the control unit outputs a control signal to bend the insertion section so that the angle between the front direction of the tip and the direction of gravity increases.

Description

Technical Field

[0001] The present invention relates to an endoscope system, a control method, and a program.

Background Art

[0002] Industrial endoscope devices are used for inspections (endoscopic inspections) of abnormalities and corrosion inside boilers, pipes, aircraft engines, heat exchangers, etc. A user inspects an inspection target by using an endoscope device. The endoscope device has an insertion portion for observing the inside of a subject. The user inserts the insertion portion into the inspection target and checks whether an abnormality such as a scratch has occurred in the inspection target.

[0003] In the inspection of a pipe, deposits or oil on the bottom surface of the pipe may adhere to the observation optical system at the tip of the insertion portion, preventing the observation of the subject. The endoscope device disclosed in Patent Document 1 has a function of sucking liquid dirt adhering to the observation optical system by capillary action. Thereby, the user can observe a clear image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the prior art, when a large amount of dirt adheres to the observation optical system, it is difficult to completely remove the dirt. Further, since it is impossible to prevent the adhesion of dirt to the observation optical system, the user needs to perform a complicated operation to move the insertion portion within the subject.

[0006] The present invention aims to provide an endoscope system, control method, and program that can reduce foreign matter contact with the observation optical system without requiring complex operations by the user. [Means for solving the problem]

[0007] The present invention relates to an endoscope system comprising: a bendable insertion section having a sensor at its tip; and a control unit which receives a mode signal indicating a mode for automatically bending the insertion section, acquires data output from the sensor, determines the direction of gravity at the tip based on the data, and after receiving the mode signal, outputs a control signal for bending the insertion section so that the angle between the front direction of the tip and the direction of gravity increases.

[0008] In the endoscope system of the present invention, after the control signal is output, the control unit outputs a state control signal to control the curvature of the insertion portion so that the angle is fixed.

[0009] In the endoscope system of the present invention, the control unit outputs a control signal to bend the insertion portion in a first direction and a second direction perpendicular to the first direction such that the angle increases according to the amount of twisting of the insertion portion.

[0010] In the endoscope system of the present invention, the control unit generates the control signal based on a predetermined value stored in memory, the predetermined value indicating a target value for the angle or a target value for the increase in the angle.

[0011] In the endoscope system of the present invention, the memory stores two or more values ​​as predetermined values, and the control unit retrieves one of the two or more values ​​from the memory.

[0012] In the endoscope system of the present invention, the control unit obtains a predetermined value from the memory that corresponds to the type of optical adapter attached to the tip.

[0013] In the endoscopic system of the present invention, the predetermined value can be changed.

[0014] In the endoscope system of the present invention, the sensor is an acceleration sensor, and the data indicates the acceleration of the tip.

[0015] The endoscope system of the present invention has an image sensor that generates an image, the control unit determines the orientation of the tip based on the direction of gravity, superimposes information indicating the orientation onto the image, and outputs the image with the superimposed information to a display.

[0016] In the endoscope system of the present invention, the sensor is an image sensor that generates an image as data.

[0017] In the endoscope system of the present invention, the control unit determines the direction of gravity based on the brightness at each of two or more positions in the image.

[0018] In the endoscope system of the present invention, the control unit uses the image to determine the distance between the tip and the subject into which the insertion portion is inserted, and outputs a control signal to curve the insertion portion so that the distance increases.

[0019] In the endoscope system of the present invention, the control unit outputs an image to a display for displaying a graphical user interface related to the setting of the mode, and receives the mode signal when the setting of the mode is instructed through the graphical user interface.

[0020] The endoscope system of the present invention has an image sensor that generates an image, and the control unit calculates the component in the direction of gravity on the imaging surface of the image sensor, calculates the angle between the direction of the component on the imaging surface and the reference direction on the imaging surface, and corrects the image by rotating the image by the angle.

[0021] The present invention receives a mode signal indicating a mode for automatically curving a bendable insertion portion having a sensor at its tip, acquires data output from the sensor, determines the direction of gravity at the tip based on the data, and outputs a control signal for curving the insertion portion so that the angle between the front direction of the tip and the direction of gravity increases after the mode signal is received.

[0022] The present invention is a program for causing a computer to execute steps of receiving a mode signal indicating a mode for automatically curving a bendable insertion portion having a sensor at its tip, acquiring data output from the sensor, determining the direction of gravity at the tip based on the data, and outputting a control signal for curving the insertion portion so that the angle between the front direction of the tip and the direction of gravity increases after the mode signal is received.

Advantages of the Invention

[0023] According to the present invention, an endoscope system, a control method, and a program do not require complicated operations by a user and can reduce foreign substances coming into contact with the observation optical system.

Brief Description of the Drawings

[0024] [Figure 1] It is a block diagram showing an example of the configuration of an endoscope system according to the first embodiment of the present invention. [Figure 2] It is a diagram showing an example of the state of the insertion portion in the first embodiment of the present invention. [Figure 3] It is a diagram showing an example of an image displayed on a display in the first embodiment of the present invention. [Figure 4] It is a flowchart showing an example of the procedure of the bending control process in the first embodiment of the present invention. [Figure 5] It is a flowchart showing an example of the procedure of the shock detection process in the first embodiment of the present invention. [Figure 6]This flowchart shows an example of the procedure for the state update process in the first embodiment of the present invention. [Figure 7] This flowchart shows an example of the procedure for the bending lock process in the first embodiment of the present invention. [Figure 8] This figure shows an example of the state of the insertion portion in the first embodiment of the present invention. [Figure 9] This figure shows an example of an image displayed on a display in the first embodiment of the present invention. [Figure 10] This figure shows an example of the state of the insertion portion in the first embodiment of the present invention. [Figure 11] This figure shows an example of an image displayed on a display in the first embodiment of the present invention. [Figure 12] This figure shows an example of the state of the insertion portion in the first embodiment of the present invention. [Figure 13] This figure shows an example of an image displayed on a display in the first embodiment of the present invention. [Figure 14] This figure shows an example of the state of the insertion portion in the first embodiment of the present invention. [Figure 15] This figure shows an example of an image displayed on a display in the first embodiment of the present invention. [Figure 16] This figure shows the relationship between the amount of twist in the insertion portion and the amount of tension of the wire used to bend the insertion portion in the first embodiment of the present invention. [Figure 17] This figure shows an example of the state of the insertion portion in the first embodiment of the present invention. [Figure 18] This figure shows an example of an image displayed on a display in the first embodiment of the present invention. [Figure 19] This figure shows an example of the relationship between the imaging surface of the image sensor and the direction of gravity in the first embodiment of the present invention. [Figure 20] This block diagram shows an example of the configuration of an endoscope system according to a first modification of the first embodiment of the present invention. [Figure 21]This block diagram shows an example of the configuration of an endoscope system according to a second modification of the first embodiment of the present invention. [Figure 22] This block diagram shows an example of the configuration of an endoscope system according to a second embodiment of the present invention. [Figure 23] This flowchart shows an example of the procedure for the curvature control process in a second embodiment of the present invention. [Figure 24] This figure shows an example of an image generated by the image sensor in a second embodiment of the present invention. [Modes for carrying out the invention]

[0025] Embodiments of the present invention will now be described with reference to the drawings. The subject in each embodiment of the present invention is an industrial product. In the following example, the subject is a pipe.

[0026] (First embodiment) A first embodiment of the present invention will now be described. Figure 1 shows the configuration of an endoscope system 1 according to the first embodiment. The endoscope system 1 shown in Figure 1 has an optical adapter 2, an insertion section 3, and a main body 4.

[0027] The optical adapter 2 is attached to the rigid tip portion 34, which includes the tip of the insertion portion 3. The optical adapter 2 has an imaging lens portion 20 and an illumination lens portion 21. The imaging lens portion 20 has one or more imaging lenses and captures light reflected from the inner surface of the pipe. The illumination lens portion 21 has one or more illumination lenses and irradiates illumination light onto the inner surface of the pipe.

[0028] The optical adapter 2 is interchangeable, and two or more optical adapters with different observation directions, field of view angles, or focal positions can be used. For example, a straight-viewing optical adapter may be used to observe the subject in a direction parallel to the longitudinal direction of the insertion section 3. Alternatively, a side-viewing optical adapter may be used to observe the subject in a direction perpendicular to the longitudinal direction of the insertion section 3.

[0029] The insertion section 3 is inserted into the pipe being observed. The insertion section 3 is a long, slender, flexible tube that can be bent. For example, the diameter of the insertion section 3 is 4 to 6 mm. The insertion section 3 has an image sensor 30, a light guide 31, a sensor 32, and a bendable section 33. The image sensor 30, light guide 31, sensor 32 are located at the tip section 34.

[0030] The image sensor 30 is an image sensor such as a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor. Light incident on the imaging lens section 20 of the optical adapter 2 passes through the imaging lens section 20 and forms an optical image on the imaging surface of the image sensor 30. Two or more pixels are arranged on the imaging surface. Each pixel has a photoelectric conversion element (photodiode). The image sensor 30 generates an image (LAW image) based on the optical image formed on the imaging surface. The image generated by the image sensor 30 is output to the main unit 4.

[0031] The light guide 31 is positioned in the insertion section 3 and the main body 4. Illumination light is generated by the main body 4 and output to the optical adapter 2 via the light guide 31. The illumination light enters the illumination lens section 21 of the optical adapter 2 and is irradiated from the illumination lens section 21 into the inside of the pipe.

[0032] Sensor 32 is fixed to the tip 34 and outputs sensor data indicating the direction of gravity at the tip 34. For example, sensor 32 is a 3-axis accelerometer. Sensor 32 may be a combination of an accelerometer and other sensors. For example, sensor 32 includes an accelerometer and at least one of a gyroscope and a geomagnetic sensor.

[0033] The curved portion 33 curves the insertion portion 3 upward (U), downward (D), left (L), or right (R). Alternatively, the curved portion 33 curves the insertion portion 3 upward left (UL), upward right (UR), downward left (DL), or downward right (DR). Each direction is relative to the tip portion 34.

[0034] The main unit 4 includes an image processing unit 40, a light source 41, a light source control unit 42, a motor unit 43, a curvature control unit 44, a display 45, a memory unit 46, an operation unit 47, a curved joystick 48, an A / D converter 49, and a control unit 50.

[0035] The image processing unit 40 converts the format of the image output from the image sensor 30 and outputs the image to the control unit 50.

[0036] The light source 41 is an LED (Light-Emitting Diode) or the like, and generates illumination light. The illumination light is output from the light source 41 to the light guide 31. The light source control unit 42 controls the light source 41.

[0037] The motor section 43 has a first motor and a second motor. The first motor is connected to a wire W1 for bending the curved section 33 in the U direction or the D direction. The second motor is connected to a wire W2 for bending the curved section 33 in the R direction or the L direction. Wires W1 and W2 are connected to the curved section 33. The first motor bends the curved section 33 in the U direction or the D direction by pulling wire W1. The second motor bends the curved section 33 in the R direction or the L direction by pulling wire W2.

[0038] The bending control unit 44 controls the bending state of the insertion section 3 by controlling the drive of the motor section 43.

[0039] The display 45 is a monitor such as an LCD (Liquid Crystal Display). The display 45 displays the image generated by the image sensor 30.

[0040] The memory unit 46 has volatile memory and non-volatile memory. The volatile memory is RAM (Random Access Memory) or DRAM (Dynamic RAM), etc. The volatile memory stores various information processed by the control unit 50. The non-volatile memory is SRAM (Static RAM), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable ROM), or flash memory. The non-volatile memory may be detachable from the main unit 4. The non-volatile memory stores the image generated by the image sensor 30 and various information processed by the control unit 50.

[0041] The control unit 47 has buttons and a touch panel for receiving various instructions from the user. The user can input various instructions to the endoscope system 1 by operating the control unit 47. The touch panel is located on the screen of the display 45. The user can input instructions to change the settings of the endoscope system 1 and instructions necessary for operating the endoscope system 1 by operating the touch panel.

[0042] The curved joystick 48 is a rod-shaped movable member and functions as a physical user interface. The user touches the curved joystick 48 with their finger or other object and applies force to it. This allows the user to tilt the curved joystick 48 upwards, downwards, to the left, or to the right. By tilting the curved joystick 48 in a predetermined direction, the user can curve the insertion part 3. The curved joystick 48 outputs an analog voltage corresponding to the direction in which the curved joystick 48 is tilted and the angle at which the curved joystick 48 is tilted.

[0043] The user may press the curved joystick 48. When the curved joystick 48 is pressed, it outputs an analog voltage. When the curved joystick 48 is pressed, the curve lock process described later is executed. While the user is tilting the curved joystick 48, the curvature angle of the insertion part 3 increases. When the curvature angle of the insertion part 3 reaches the maximum angle, the curvature angle of the insertion part 3 does not increase. When the curve lock process is executed, the curvature angle of the insertion part 3 is fixed even when the user has released their finger from the curved joystick 48.

[0044] The A / D converter 49 converts the analog voltage output from the curved joystick 48 into a digital value and outputs the digital value to the control unit 50. The curved joystick 48 may include the A / D converter 49.

[0045] The control unit 50 controls various parts of the main unit 4 and performs various processes. For example, the control unit 50 performs colorization, noise reduction, and edge enhancement on the image output from the image processing unit 40. The control unit 50 stores the image in the memory unit 46. The image stored in the memory unit 46 may be a still image or a video. The control unit 50 also superimposes information for displaying a graphical user interface (GUI), such as a menu, onto the image. The control unit 50 performs image processing such as color space conversion and gamma correction according to the specifications of the display 45 and outputs the image to the display 45.

[0046] The control unit 50 determines the direction of gravity at the tip 34 based on sensor data output from the sensor 32. The control unit 50 outputs a control signal to the bending control unit 44 to bend the insertion section 3 in the opposite direction to gravity. The bending control unit 44 drives the motor section 43 according to the control signal output from the control unit 50. When the insertion section 3 bends in the opposite direction to gravity, the tip 34 floats above the bottom surface of the pipe. Therefore, the endoscope system 1 can avoid foreign matter such as sediment or oil on the bottom surface of the pipe coming into contact with the optical adapter 2, especially the imaging lens section 20 or the illumination lens section 21. Even if the user twists the insertion section 3, the insertion section 3 will bend in the opposite direction to gravity.

[0047] The endoscope system 1 operates in one of two or more modes. The mode set for the endoscope system 1 is switchable. For example, the two or more modes include an automatic bending mode and a manual bending mode. When the automatic bending mode is set for the endoscope system 1, the control unit 50 performs a process to bend the insertion section 3 in the direction opposite to the direction of gravity. When the manual bending mode is set for the endoscope system 1, the control unit 50 performs a process to bend the insertion section 3 in the direction in which the bending joystick 48 tilts, based on the digital value output from the A / D converter 49.

[0048] The user inputs instructions to the endoscope system 1 to set a mode by operating the buttons or touch panel on the control unit 47. The control unit 47 outputs a mode signal indicating the mode to the control unit 50. Based on the mode signal output from the control unit 47, the control unit 50 sets the endoscope system 1 to either automatic bending mode or manual bending mode. The memory unit 46 stores the automatic bending mode or manual bending mode set in the endoscope system 1.

[0049] The user may input an instruction to the endoscope system 1 to bend the insertion section 3 by operating the touch panel of the operation unit 47. The operation unit 47 may output a signal indicating this instruction to the control unit 50. The control unit 50 may control the bending control unit 44 based on the signal output from the operation unit 47.

[0050] The image sensor 30 continuously performs imaging and sequentially generates two or more live images. The image processing unit 40 sequentially processes the two or more live images. The control unit 50 sequentially acquires the two or more live images processed by the image processing unit 40 and performs the aforementioned image processing. The control unit 50 sequentially outputs the two or more live images to the display 45. The display 45 sequentially displays the two or more live images. The endoscope system 1 can display images of the pipe in real time.

[0051] At least one of the image processing unit 40, light source control unit 42, curvature control unit 44, and control unit 50 may consist of at least one processor and logic circuit. For example, the processor is at least one of a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit). For example, the logic circuit is at least one of an ASIC (Application Specific Integrated Circuit) and an FPGA (Field-Programmable Gate Array). At least one of the image processing unit 40, light source control unit 42, curvature control unit 44, and control unit 50 may include one or more processors. At least one of the image processing unit 40, light source control unit 42, curvature control unit 44, and control unit 50 may include one or more logic circuits.

[0052] The computer of the endoscope system 1 may load a program and execute the loaded program. The program includes instructions that define at least one operation of the image processing unit 40, the light source control unit 42, the bending control unit 44, and the control unit 50. In other words, at least one function of the image processing unit 40, the light source control unit 42, the bending control unit 44, and the control unit 50 may be implemented by software.

[0053] The above program may be provided on a "computer-readable recording medium," such as flash memory. The program may be transmitted from the computer holding the program to the endoscope system 1 via a transmission medium or by transmission waves within the transmission medium. The "transmission medium" for transmitting the program is a medium that has the function of transmitting information. A medium that has the function of transmitting information includes networks such as the Internet and communication lines such as telephone lines. The above program may implement some of the functions described above. Furthermore, the above program may be a differential file (differential program). The functions described above may be implemented by a combination of a program already recorded on the computer and a differential program.

[0054] After the endoscope system 1 is powered on, the user inserts the tip 34 of the insertion section 3 into the pipe. In the following example, the insertion section 3 is inserted into a straight pipe. The pipe into which the insertion section 3 is inserted may be positioned horizontally or inclined with respect to the horizontal plane. The user holds the tip 34 still when it is in contact with the bottom surface of the pipe in a location free of foreign objects. At this point, the tip 34 is parallel to the bottom surface of the pipe. The user then advances the tip 34 towards the back of the pipe. In the following example, a straight-viewing optical adapter is used as optical adapter 2.

[0055] Figure 2 shows an example of the state of the insertion part 3 inserted into the pipe. The three-dimensional coordinate axes, namely the mutually orthogonal X, Y, and Z axes, are shown. The roll direction Dr around the X axis, the pitch direction Dp around the Y axis, and the yaw direction Dy around the Z axis are also shown.

[0056] The XY plane is the same as the horizontal plane. The positive direction of the Z axis is the same as the vertically upward direction. The pipe into which the insertion part 3 is inserted is positioned horizontally. The axial direction of the pipe is parallel to the X axis. The insertion part 3 inside the pipe is not curved. The front direction Dif of the tip part 34 and the optical adapter 2 is parallel to the X axis. The front direction Dif coincides with the optical axis direction of one or more lenses of the imaging lens part 20. The upward direction Diu corresponds to the upward direction in the image generated by the image sensor 30 and is parallel to the Z axis. The gravity direction Dg is opposite to the upward direction Diu. In this case, the angle An between the front direction Dif and the gravity direction Dg is 90 degrees. If the pipe is not positioned horizontally, the angle An is, for example, between 45 degrees and 135 degrees.

[0057] A virtual object OB1, which does not exist in the actual pipe, is positioned at the center of the pipe. The virtual object OB1 is located in front of the tip 34 and the optical adapter 2. To make the inclination of the virtual object OB1 in the image generated by the image sensor 30 clearer, the letter F is shown on the surface of the virtual object OB1.

[0058] Figure 3 shows an example of an image displayed on the display 45 when the insertion section 3 is in the state shown in Figure 2. The image IMG1 shown in Figure 3 includes the auto-curve button BT1, the curve+ button BT2, and the curve- button BT3. The virtual object OB1 is shown in the center of the image IMG1. The upward direction in the image IMG1 coincides with the upward direction Diu shown in Figure 2.

[0059] The control unit 50 superimposes the automatic curve button BT1, the curve + button BT2, and the curve - button BT3 onto the image IMG1. These buttons are GUI elements. The control unit 50 outputs the image IMG1 with these buttons superimposed to the display 45. The display 45 displays the image IMG1 with these buttons superimposed.

[0060] The automatic bending button BT1 is used to set the endoscope system 1 to automatic bending mode. The automatic bending button BT1 is also used to deactivate the automatic bending mode and set the endoscope system 1 to manual bending mode. The bending + button BT2 is used to increase the angle An in automatic bending mode. The bending - button BT3 is used to decrease the angle An in automatic bending mode.

[0061] The user can press each button by pressing the corresponding location on the touch panel. When a button is pressed, the operation unit 47 outputs a signal corresponding to the function of the button to the control unit 50.

[0062] Figure 4 shows an example of the bending control procedure performed by the endoscope system 1. The operation of the endoscope system 1 in the bending control procedure will be explained with reference to Figure 4.

[0063] Immediately after the endoscope system 1 is started, the mode of the endoscope system 1 is set to manual bending mode. The control unit 50 monitors the signal output from the operation unit 47 and determines whether or not the automatic bending button BT1 has been pressed (step S10).

[0064] The user inserts the insertion section 3 into the pipe while viewing the image displayed on the display 45. When the insertion section 3 reaches a location where a foreign object is suspected to be present, the user may press the automatic bending button BT1. The user can set the mode of the endoscope system 1 to automatic bending mode at their desired timing.

[0065] When the automatic bending button BT1 is pressed, the control unit 47 outputs a mode signal indicating automatic bending mode to the control unit 50. If the mode signal indicating automatic bending mode is output from the control unit 47, the control unit 50 determines that the automatic bending button BT1 has been pressed. At this time, the control unit 50 sets the mode of the endoscope system 1 to automatic bending mode. If the mode signal indicating automatic bending mode is not output from the control unit 47, the control unit 50 determines that the automatic bending button BT1 has not been pressed.

[0066] If the control unit 50 determines in step S10 that the automatic bending button BT1 has not been pressed, step S28, described later, is executed. If the control unit 50 determines in step S10 that the automatic bending button BT1 has been pressed, the control unit 50 performs centering of the tip portion 34. At this time, the control unit 50 outputs a control signal to the bending control unit 44 to forcibly straighten the insertion portion 3. The bending control unit 44 controls the motor portion 43 so that the insertion portion 3 becomes straight (step S12).

[0067] Sensor 32 periodically outputs sensor data. The control unit 50 acquires the sensor data output from sensor 32 and stores the sensor data in the memory unit 46. The control unit 50 repeats this process.

[0068] After step S12, the control unit 50 acquires sensor data from the memory unit 46 (step S14).

[0069] After step S14, the control unit 50 determines the direction of gravity using the sensor data. The control unit 50 can determine the direction of gravity in the coordinate system specific to the tip portion 34 (step S16).

[0070] After step S14, the control unit 50 calculates the current angle An based on the direction of gravity determined in step S14. The relative positional relationship between the image sensor 30 and the sensor 32 is predetermined and known. The control unit 50 can calculate the angle An based on this positional relationship (step S18).

[0071] The memory unit 46 stores the target angle in the automatic bending mode in advance. The target angle is a predetermined value of angle An. The target angle is greater than 90 degrees and 180 degrees or less. For example, the target angle is 110 degrees. After step S18, the control unit 50 retrieves the target angle from the memory unit 46. The control unit 50 determines whether the angle An calculated in step S18 is different from the target angle (step S20).

[0072] In the example above, the target angle represents the target value of angle An. The memory unit 46 may store a target value for the increase in angle An. For example, the target value may be 20 degrees. The control unit 50 may calculate the target angle by adding 20 degrees to the initial value of angle An, which is 90 degrees.

[0073] The memory unit 46 may store two or more different target angles. Each of the two or more target angles may correspond to a type of optical adapter 2. For example, the memory unit 46 may store the target angle for a straight-view optical adapter and the target angle for a side-view optical adapter. The control unit 50 may obtain the target angle corresponding to the type of optical adapter 2 from the memory unit 46.

[0074] The user may input information indicating the type of optical adapter 2 into the endoscope system 1 by operating the control unit 47. The control unit 50 may obtain the target angle corresponding to the type indicated by the information input by the user from the memory unit 46.

[0075] The optical adapter 2 may have a storage unit that stores the type of optical adapter 2 and a corresponding identifier. The control unit 50 may detect the identifier stored in the storage unit and identify the type of optical adapter 2 that corresponds to that identifier. The control unit 50 may obtain the target angle corresponding to the identified type from the memory unit 46.

[0076] The range in which the insertion section 3 can bend varies depending on the diameter of the pipe into which the insertion section 3 is inserted. Therefore, each of the two or more target angles stored in the memory section 46 may correspond to the diameter of the pipe.

[0077] If the control unit 50 determines in step S20 that angle An is the same as the target angle, step S24, described later, is executed. If the control unit 50 determines in step S20 that angle An is different from the target angle, the control unit 50 outputs a control signal to the bending control unit 44 to bend the insertion part 3 in the opposite direction to the direction of gravity so that angle An matches the target angle. The bending control unit 44 controls the motor part 43 according to the control signal and bends the insertion part 3 in the opposite direction to the direction of gravity so that angle An matches the target angle (step S22).

[0078] When angle An matches the target angle, the optical adapter 2 does not come into contact with the bottom surface of the pipe. Therefore, any foreign matter on the bottom surface of the pipe does not come into contact with the optical adapter 2.

[0079] When the user twists the insertion part 3, the relative direction of gravity at the tip part 34 changes. Under the condition that the curved state of the insertion part 3 is fixed, the angle An deviates from the target angle according to the amount of twisting of the insertion part 3. Therefore, the control unit 50 outputs a control signal (state control signal) to the curvature control unit 44 to curve the insertion part 3 so that the angle An maintains the target angle.

[0080] After the insertion section 3 has bent so that angle An matches the target angle, the control unit 50 outputs a control signal to the bending control unit 44 to control the bending state of the insertion section 3 so that angle An is fixed at the target angle. The bending control unit 44 controls the motor section 43 according to the control signal to fix angle An at the target angle. The control unit 50 continues the above control while the mode of the endoscope system 1 is set to automatic bending mode.

[0081] The memory unit 46 may store the range of the target angle. The control unit 50 may determine in step S20 whether or not angle An is included in the range of the target angle. If the control unit 50 determines in step S20 that angle An is not included in the range of the target angle, step S24 may be executed. If the control unit 50 determines in step S20 that angle An is included in the range of the target angle, step S22 may be executed. In step S22, the control unit 50 may output a control signal to the bending control unit 44 to bend the insertion unit 3 in the direction opposite to the direction of gravity so that angle An is included in the range of the target angle.

[0082] After step S22, the control unit 50 performs an impact detection process to detect a strong impact at the tip portion 34 (step S24). Details of the impact detection process will be described later. The bending control process does not necessarily have to include the impact detection process.

[0083] After step S24, the control unit 50 performs a state update process to control the bending state of the insertion section 3 in response to the user's operation of the bend + button BT2 or the bend - button BT3 (step S26). Details of the state update process will be described later. The bending control process does not necessarily have to include the state update process.

[0084] After step S26, the control unit 50 performs a bending lock process to fix the curved state of the insertion section 3 (step S28). Details of the bending lock process will be described later. The bending control process does not necessarily have to include the bending lock process.

[0085] After step S28, the control unit 50 monitors the signal output from the operation unit 47 and determines whether or not the power button of the operation unit 47 has been pressed (step S30).

[0086] When the control unit 50 determines in step S30 that the power button has been pressed, the bending control process shown in Figure 4 is terminated. When the control unit 50 determines in step S30 that the power button has not been pressed, the control unit 50 determines whether or not the automatic bending mode is set for the endoscope system 1 (step S32).

[0087] If the control unit 50 determines in step S32 that the automatic bending mode is set for the endoscope system 1, step S14 is executed. If the control unit 50 determines in step S32 that the automatic bending mode is not set for the endoscope system 1, step S10 is executed. At this time, the tip portion 34 is in contact with the bottom surface of the pipe and is parallel to the bottom surface of the pipe.

[0088] When the automatic bending mode is set on the endoscope system 1, the user can change the automatic bending mode to manual bending mode by pressing the automatic bending button BT1. When the automatic bending button BT1 is pressed, the operation unit 47 outputs a mode signal indicating manual bending mode to the control unit 50. When the mode signal indicating manual bending mode is output from the operation unit 47, the control unit 50 sets the mode of the endoscope system 1 to manual bending mode.

[0089] The target angle stored in the memory unit 46 may be changeable. For example, the user may input any value for the target angle to the endoscope system 1 by operating the control unit 47. The control unit 50 may change the target angle stored in the memory unit 46 to an angle corresponding to the value input to the endoscope system 1.

[0090] The control unit 50 may output a GUI image containing two or more values ​​for the target angle to the display 45. The user may input an instruction to the endoscope system 1 to select one of the two or more values ​​through the GUI by operating the touch panel of the operation unit 47. The control unit 50 may change the target angle stored in the memory unit 46 to the angle corresponding to the value indicated by the instruction input to the endoscope system 1.

[0091] FIG. 5 shows an example of the procedure of the impact detection process. Referring to FIG. 5, the operation of the endoscope system 1 in the impact detection process will be described.

[0092] The control unit 50 acquires sensor data from the memory unit 46 (step S240).

[0093] After step S240, the control unit 50 determines whether a strong impact has been applied to the tip 34 based on the sensor data (step S241).

[0094] In step S241, the control unit 50 determines the acceleration at the tip 34. When the acceleration exceeds a predetermined value, the control unit 50 determines that a strong impact has been applied to the tip 34. When the acceleration does not exceed the predetermined value, the control unit 50 determines that no strong impact has been applied to the tip 34.

[0095] When the control unit 50 determines in step S241 that no strong impact has been applied to the tip 34, step S26 shown in FIG. 4 is executed. When the control unit 50 determines in step S241 that a strong impact has been applied to the tip 34, the control unit 50中止 the bending control. At this time, the control unit 50 may execute the same process as step S12 (step S242).

[0096] In the inspection of a thin pipe, the insertion portion 3 may contact the inner wall of the pipe and the bending portion 33 may be damaged. Therefore, when a strong impact is applied to the tip 34, the control unit 50中止 the bending control.

[0097] After step S242, the control unit 50 outputs a message indicating that the bending control has been中止 to the display 45. The display 45 displays the message (step S243). After step S243, step S10 shown in FIG. 4 is executed.

[0098] FIG. 6 shows an example of the procedure of the state update process. Referring to FIG. 6, the operation of the endoscope system 1 in the state update process will be described.

[0099] The memory unit 46 stores in advance a set value for the increase in angle An and a set value for the decrease in angle An. The memory unit 46 may also store in advance two or more set values ​​for the increase in angle An and two or more set values ​​for the decrease in angle An. Each of the two or more set values ​​for the increase in angle An may correspond to a type of optical adapter 2. Each of the two or more set values ​​for the decrease in angle An may correspond to a type of optical adapter 2.

[0100] The control unit 50 monitors the signal output from the operation unit 47 and determines whether or not the curved + button BT2 has been pressed (step S260).

[0101] When the control unit 50 determines in step S260 that the curve + button BT2 has been pressed, the control unit 50 obtains the set value for the increase amount of angle An from the memory unit 46. The control unit 50 outputs a control signal to the curve control unit 44 to increase the amount of curvature of the insertion unit 3 so that angle An increases by that set value. The curve control unit 44 controls the motor unit 43 according to the control signal and curves the insertion unit 3 so that angle An increases by that set value. For example, the curvature state of the insertion unit 3 is changed so that angle An increases by 5 degrees (step S261). After step S261, step S28 shown in Figure 4 is executed.

[0102] If the control unit 50 determines in step S260 that the curved + button BT2 has not been pressed, the control unit 50 determines whether or not the curved - button BT3 has been pressed (step S262).

[0103] If the control unit 50 determines in step S262 that the curve-button BT3 is not pressed, step S28 shown in Figure 4 is executed. If the control unit 50 determines in step S262 that the curve-button BT3 is pressed, the control unit 50 obtains the set value for the amount of angle An reduction from the memory unit 46. The control unit 50 outputs a control signal to the curve control unit 44 to reduce the amount of curvature of the insertion unit 3 so that angle An decreases by that set value. The curve control unit 44 controls the motor unit 43 according to the control signal and curves the insertion unit 3 so that angle An decreases by that set value. For example, the curvature state of the insertion unit 3 is changed so that angle An decreases by 5 degrees (step S263). After step S263, step S28 shown in Figure 4 is executed.

[0104] The setting value for the increase or decrease amount of angle An stored in the memory unit 46 may be changeable. For example, the user may input any setting value to the endoscope system 1 by operating the operation unit 47. The control unit 50 may change the setting value stored in the memory unit 46 to the setting value input to the endoscope system 1.

[0105] The control unit 50 may output a GUI image containing two or more setting values ​​to the display 45. The user may input an instruction to the endoscope system 1 to select one of the two or more values ​​through the GUI by operating the touch panel of the operation unit 47. The control unit 50 may change the setting value stored in the memory unit 46 to the setting value indicated by the instruction input to the endoscope system 1.

[0106] The degree of inclination of the pipe relative to the horizontal plane may change depending on the position. When the automatic bending mode is set on the endoscope system 1, the angle An is maintained at the target angle regardless of the degree of inclination. Therefore, when the inclination of the pipe relative to the horizontal plane gradually increases vertically upward as the insertion section 3 advances, the optical adapter 2 may come into contact with the bottom surface of the pipe. The user can adjust the curvature of the insertion section 3 so that the optical adapter 2 does not come into contact with the bottom surface of the pipe by using the bending button BT3.

[0107] Furthermore, as the insertion section 3 progresses, the inclination of the pipe relative to the horizontal plane gradually increases in the vertical downward direction, which may cause the optical adapter 2 to come into contact with the upper surface of the pipe. The user can adjust the curvature of the insertion section 3 to prevent the optical adapter 2 from coming into contact with the upper surface of the pipe by using the Curve + button BT2.

[0108] Figure 7 shows an example of the bending lock procedure. Referring to Figure 7, the operation of the endoscope system 1 during the bending lock procedure will be explained.

[0109] The user can input a command to lock the bending state to the endoscope system 1 by pressing the bending joystick 48. When the bending joystick 48 is pressed, the bending joystick 48 outputs a digital value to the control unit 50.

[0110] The control unit 50 monitors the digital value output from the curved joystick 48 and determines whether or not the curved state lock has been instructed (step S280).

[0111] When the control unit 50 determines in step S280 that a lock on the curved state has been instructed, the control unit 50 outputs a control signal to the curved control unit 44 to control the curved state of the insertion unit 3 so that the curved state of the insertion unit 3 is fixed in its current state. The curved control unit 44 controls the motor unit 43 in accordance with the control signal to fix the curved state of the insertion unit 3. The control unit 50 continues the above control until a release of the curved state lock is instructed (step S281).

[0112] After an instruction to lock the bending state has been input, the user can input an instruction to release the bending state to the endoscope system 1 by pressing the bending joystick 48 again. When the bending joystick 48 is pressed, the bending joystick 48 outputs a digital value to the control unit 50.

[0113] After step S281, the control unit 50 monitors the digital value output from the curved joystick 48 and determines whether or not it has been instructed to release the lock on the curved state (step S282).

[0114] When the control unit 50 determines in step S282 that it has been instructed to release the lock on the curved state, the control unit 50 outputs a control signal to the curved control unit 44 to control the curved state of the insertion unit 3 so that the curved state of the insertion unit 3 returns to its original state. The curved control unit 44 controls the motor unit 43 in accordance with the control signal and returns the curved state of the insertion unit 3 to its original state (step S283). After step S283, step S30 shown in Figure 4 is executed.

[0115] When step S283 is performed, the curvature of the insertion section 3 returns to the state it was in before the curvature joystick 48 was tilted. When step S283 is performed with the mode of the endoscope system 1 set to automatic curvature mode, the curvature of the insertion section 3 is changed so that angle An matches the target angle.

[0116] If the control unit 50 determines in step S280 that the curved state lock has not been instructed, the control unit 50 monitors the digital value output from the A / D converter 49 and determines whether or not the curved joystick 48 has been tilted by the user (step S284).

[0117] If the control unit 50 determines in step S284 that the curved joystick 48 has not been tilted by the user, step S287, described later, is executed. If the control unit 50 determines in step S284 that the curved joystick 48 has been tilted by the user, the control unit 50 outputs a control signal to the curvature control unit 44 to control the curvature of the insertion part 3 so that the insertion part 3 curves in the direction corresponding to the direction in which the curved joystick 48 was tilted. The curvature control unit 44 controls the motor unit 43 according to the control signal to curve the insertion part 3 (step S285).

[0118] The memory unit 46 stores in advance a setting value that indicates the amount by which the insertion unit 3 is curved when the curved joystick 48 is tilted by the user. In step S285, the control unit 50 outputs a control signal to the curvature control unit 44 to control the curvature state of the insertion unit 3 so that it curves by an amount corresponding to the setting value.

[0119] After step S285, the control unit 50 monitors the digital value output from the A / D converter 49 and determines whether the curved joystick 48 has returned to its original state. The original state is when the curved joystick 48 is not tilted (step S286).

[0120] If the control unit 50 determines in step S286 that the curved joystick 48 has not returned to its original state, step S280 is executed. If the control unit 50 determines in step S286 that the curved joystick 48 has returned to its original state, the control unit 50 determines whether the state in which the curved state lock was instructed (curved state lock) is continuing (step S287).

[0121] If the control unit 50 determines in step S280 that the bending lock has not been instructed, the bending lock state does not continue. If the control unit 50 determines in step S282 that the bending lock has not been instructed, the bending lock state continues.

[0122] If the control unit 50 determines in step S287 that the bending lock state is not continuing, step S30 shown in Figure 4 is executed. If the control unit 50 determines in step S287 that the bending lock state is continuing, step S282 is executed.

[0123] Figure 8 shows an example of the state of the insertion section 3 after step S22 shown in Figure 4 has been performed. The same parts as those shown in Figure 2 will not be explained.

[0124] The control unit 50 outputs a control signal to the bending control unit 44 to bend the insertion part 3 so that angle An matches the target angle. The bending control unit 44 controls the motor unit 43 according to the control signal and bends the insertion part 3 so that angle An matches the target angle. In the example shown in Figure 8, the target angle is 110 degrees. Also, in the example shown in Figure 8, the user does not twist the insertion part 3. The insertion part 3 is bent 20 degrees in the vertical upward direction.

[0125] Figure 9 shows an example of an image displayed on the display 45 when the insertion section 3 is in the state shown in Figure 8. The image IMG2 shown in Figure 9 includes the auto-curve button BT1, the curve + button BT2, and the curve - button BT3. The virtual object OB1 is visible in the image IMG2. The description of the same parts as shown in Figure 3 is omitted.

[0126] Because the insertion part 3 curves vertically upward, the tip part 34 moves vertically upward. The virtual object OB1 in image IMG2 moves downward. Because the insertion part 3 is not twisted, the virtual object OB1 in image IMG2 does not rotate.

[0127] Three examples of the state of the insertion section 3 after it has reached the state shown in Figure 8 will be described. In the first example, the mode of the endoscope system 1 is set to manual bending mode, and the bending state lock is instructed. The user then twists the insertion section 3 90 degrees in the roll direction Dr. Figure 10 shows an example of the state of the insertion section 3 after it has been twisted 90 degrees in the roll direction Dr. The same parts as those shown in Figure 2 will not be described.

[0128] The curved state of the insertion part 3 is maintained while it is rotating in the roll direction Dr. When the insertion part 3 is twisted 90 degrees in the roll direction Dr, the tip 34 is in contact with the bottom surface of the pipe. The forward direction Dif and the upward direction Diu are parallel to the horizontal plane, and the angle An is 90 degrees.

[0129] Figure 11 shows an example of an image displayed on the display 45 when the insertion unit 3 is in the state shown in Figure 10. The image IMG3 shown in Figure 11 includes the auto-curve button BT1, the curve + button BT2, and the curve - button BT3. The virtual object OB1 is visible in the image IMG3. The description of the same parts as shown in Figure 3 is omitted. Because the insertion unit 3 rotates in the roll direction Dr, the virtual object OB1 in the image IMG3 rotates counterclockwise.

[0130] After the insertion section 3 is in the state shown in Figure 8, in the second example, the mode of the endoscope system 1 is automatic bending mode, and the user twists the insertion section 3 90 degrees in the roll direction Dr. Figure 12 shows an example of the state of the insertion section 3 after it has been twisted 90 degrees in the roll direction Dr. The same parts as those shown in Figure 2 will not be explained.

[0131] While the insertion section 3 rotates in the roll direction Dr, the control unit 50 repeatedly executes step S22 and outputs a control signal to the bending control unit 44 to maintain the angle An at the target angle. The bending control unit 44 controls the motor section 43 according to the control signal to maintain the angle An at the target angle. In the example shown in Figure 12, the target angle is 110 degrees. Even if the insertion section 3 is twisted, the tip section 34 does not come into contact with the bottom surface of the pipe.

[0132] Figure 13 shows an example of an image displayed on the display 45 when the insertion section 3 is in the state shown in Figure 12. The image IMG4 shown in Figure 13 includes the auto-curve button BT1, the curve+ button BT2, and the curve- button BT3. The virtual object OB1 is visible in the image IMG4. The description of the same parts as shown in Figure 3 is omitted.

[0133] Because the insertion section 3 rotates in the roll direction Dr, the virtual object OB1 in image IMG4 rotates counterclockwise. The front direction Dif in Figure 12 is different from the front direction Dif in Figure 10. Therefore, the position of the virtual object OB1 in image IMG4 is different from the position of the virtual object OB1 in image IMG3 shown in Figure 11.

[0134] After the insertion section 3 is in the state shown in Figure 8, in the third example, the mode of the endoscope system 1 is automatic bending mode, and the user twists the insertion section 3 45 degrees in the roll direction Dr. Figure 14 shows an example of the state of the insertion section 3 after it has been twisted 45 degrees in the roll direction Dr. The same parts as shown in Figure 2 will not be explained.

[0135] As described above, the control unit 50 outputs a control signal to the curvature control unit 44 to maintain angle An at the target angle. The curvature control unit 44 controls the motor unit 43 according to the control signal to maintain angle An at the target angle. In the example shown in Figure 14, the target angle is 110 degrees.

[0136] Figure 15 shows an example of an image displayed on the display 45 when the insertion section 3 is in the state shown in Figure 14. The image IMG5 shown in Figure 15 includes the auto-curve button BT1, the curve + button BT2, and the curve - button BT3. The virtual object OB1 is visible in the image IMG5. The description of the same parts as shown in Figure 3 is omitted.

[0137] Compared to the state shown in Figure 9, the virtual object OB1 in image IMG5 is rotated 45 degrees counterclockwise. The front-facing Dif in Figure 14 is different from the front-facing Dif in Figure 10. Therefore, the position of the virtual object OB1 in image IMG5 is different from the position of the virtual object OB1 in image IMG3 shown in Figure 11.

[0138] When the insertion section 3 is twisted while the mode of the endoscope system 1 is set to automatic bending mode, the control unit 50 outputs a control signal to the bending control unit 44 to bend the insertion section 3 in the U or D direction and in the R or L direction so that the angle An increases according to the amount of twisting of the insertion section 3. The bending control unit 44 controls the first motor and the second motor of the motor unit 43 according to the control signal. The first motor bends the insertion section 3 in the U or D direction by pulling the wire W1. The second motor bends the insertion section 3 in the R or L direction by pulling the wire W2.

[0139] Figure 16 shows the relationship between the amount of twist of the insertion section 3 in the rolling direction and the amount of tension on wires W1 and W2. When the insertion section 3 is not twisted, wire W1 needs to be pulled by A to curve the insertion section 3 in the U direction so that angle An is 110 degrees. When the insertion section 3 is twisted 45 degrees in the rolling direction, the insertion section 3 needs to be curved in both the U and L directions so that angle An is 110 degrees. In this case, the amount of tension on wires W1 and W2 is 0.7A.

[0140] When the insertion section 3 is twisted 90 degrees in the rolling direction, the wire W1 needs to be pulled by A to curve the insertion section 3 in the L direction so that angle An becomes the target angle. When the insertion section 3 is twisted 135 degrees in the rolling direction, the insertion section 3 needs to be curved in both the D and L directions so that angle An becomes 110 degrees. In this case, the pulling amounts of wires W1 and W2 are 0.7A.

[0141] Figure 16 also shows the amount of tension required for wires W1 and W2 to bend the insertion section 3 when the amount of twist of the insertion section 3 in the rolling direction is 180 degrees, 225 degrees, 270 degrees, or 360 degrees. Figure 16 shows the amount of tension for wires W1 and W2 when a predetermined amount of twist is applied to the insertion section 3 in the ideal state where the insertion section 3 is straight. In practice, a greater amount of tension than that shown in Figure 16 may be required to maintain angle An at the target angle.

[0142] Figure 17 shows the state of the insertion section 3 after step S22 shown in Figure 4 has been performed, with the side-view type optical adapter being used as optical adapter 2. The same parts as those shown in Figure 2 will not be explained.

[0143] The control unit 50 outputs a control signal to the bending control unit 44 to bend the insertion section 3 so that angle An matches the target angle. The bending control unit 44 controls the motor unit 43 according to the control signal and bends the insertion section 3 so that angle An matches the target angle. The target angle of the side-view type optical adapter is different from the target angle of the straight-view type optical adapter. In the example shown in Figure 17, the target angle is 90 degrees. Also, in the example shown in Figure 17, the user does not twist the insertion section 3. The insertion section 3 is bent 90 degrees in the vertically upward direction. The front direction Dif of the side-view type optical adapter 2 is parallel to the X axis.

[0144] When the user twists the insertion part 3 in the low direction Dr, the angle An is maintained at 90 degrees, and the tip part 34 rotates in the yaw direction Dy. The optical axis of the imaging lens part 20 of the optical adapter 2 rotates in the yaw direction Dy. By twisting the insertion part 3 in the low direction Dr, the user can orient the optical axis of the imaging lens part 20 in any direction parallel to the XY plane.

[0145] Figure 18 shows an example of an image displayed on the display 45 when the insertion section 3 is in the state shown in Figure 17. The image IMG6 shown in Figure 18 includes the auto-curve button BT1, the curve+ button BT2, and the curve- button BT3. The virtual object OB1 is visible in the image IMG6. The description of the same parts as shown in Figure 3 is omitted.

[0146] Because the insertion portion 3 curves vertically upward, the tip portion 34 moves vertically upward. The virtual object OB1 is located slightly below the center of image IMG6.

[0147] As mentioned above, the relative positional relationship between the image sensor 30 and the sensor 32 is known. The control unit 50 may perform the following processing in step S16 shown in Figure 4. The control unit 50 generates information indicating the relationship between the direction of gravity and the reference direction on the imaging surface of the image sensor 30. For example, the reference direction is the direction perpendicular to the imaging surface. The control unit 50 superimposes this information onto the image generated by the image sensor 30 and outputs the superimposed image to the display 45. The display 45 displays the image.

[0148] Information showing the relationship between the direction of gravity and the reference direction indicates the orientation of the tip 34. The user can determine the orientation of the tip 34 based on the image displayed on the display 45.

[0149] The control unit 50 may perform the following processing in step S16 shown in Figure 4. Figure 19 shows an example of the relationship between the imaging surface IS1 of the image sensor 30 and the direction of gravity Dg.

[0150] The control unit 50 calculates the component Dgi of the gravity direction Dg on the imaging plane IS1. Component Dgi is parallel to the imaging plane IS1. The control unit 50 calculates the angle Agr between the direction of component Dgi and the reference direction Dref on the imaging plane IS1. The reference direction Dref is parallel to the imaging plane IS1. For example, the reference direction Dref is the downward direction on the imaging plane IS1. The downward direction on the imaging plane IS1 corresponds to the downward direction in the image. The control unit 50 corrects the image by rotating the image generated by the image sensor 30 by the angle Agr. The downward direction in the corrected image coincides with the direction of component Dgi. The control unit 50 outputs the corrected image to the display 45. The display 45 displays the image.

[0151] The endoscope system 1 can display an image generated by the image sensor 30 in a state where the direction of gravity Dg and the corresponding component Dgi coincide with the downward direction in the image. Since the direction of component Dgi in the image does not change, the user can perform the examination without confusion.

[0152] Each embodiment of the present invention comprises an endoscope system 1 and a control unit 50. The endoscope system 3 has a sensor 32 at its tip 34 and is bendable. The control unit 50 receives a mode signal indicating a mode for automatically bending the endoscope system 3. The control unit 50 acquires data output from the sensor 32 and determines the direction of gravity Dg at the tip 34 based on that data. After receiving the mode signal, the control unit 50 outputs a control signal to bend the endoscope system 3 so that the angle An between the forward direction Dif and the direction of gravity Dg at the tip 34 increases.

[0153] Each aspect of the present invention's control method comprises four steps. In the first step (step S10), the control unit 50 receives a mode signal indicating a mode for automatically bending the insertion portion 3. In the second step (step S14), the control unit 50 acquires data output from the sensor 32. In the third step (step S16), the control unit 50 determines the direction of gravity at the tip portion 34 based on the data. After receiving the mode signal, in the fourth step (step S22), the control unit 50 outputs a control signal to bend the insertion portion 3 so that the angle An between the front direction Dif and the gravity direction Dg of the tip portion 34 increases.

[0154] A program according to each aspect of the present invention causes a computer to perform the first to fourth steps described above.

[0155] Each aspect of the present invention may include the following modifications. After the control signal is output, the control unit 50 outputs a state control signal to control the curvature of the insertion portion 3 so that the angle An is fixed.

[0156] Each aspect of the present invention may include the following modifications. The control unit 50 outputs a control signal to bend the insertion portion 3 in a first direction and a second direction perpendicular to the first direction such that the angle An increases according to the amount of twisting of the insertion portion 3. In the above example, the first direction is the U direction or the D direction, and the second direction is the R direction or the L direction.

[0157] Each aspect of the present invention may include the following modifications. The control unit 50 generates a control signal based on a predetermined value stored in the memory unit 46. The predetermined value indicates a target value for angle An or a target value for the increase in angle An.

[0158] Each aspect of the present invention may include the following modifications. The memory unit 46 stores two or more values ​​as predetermined values. The control unit 50 retrieves one of the two or more values ​​from the memory unit 46.

[0159] Each aspect of the present invention may include the following modifications. The control unit 50 obtains a predetermined value from the memory unit 46 that corresponds to the type of optical adapter 2 attached to the tip portion 34.

[0160] Each aspect of the present invention may include the following modifications. The above predetermined values ​​are changeable.

[0161] Each aspect of the present invention may include the following modifications. Sensor 32 is an acceleration sensor. The data output from sensor 32 indicates the acceleration of the tip portion 34.

[0162] Each aspect of the present invention may include the following modifications. The endoscope system 1 has an image sensor 30 that generates an image. The control unit 50 determines the orientation of the tip 34 based on the direction of gravity Dg. The control unit 50 superimposes information indicating the orientation of the tip 34 onto the image generated by the image sensor 30 and outputs the superimposed image to the display 45.

[0163] Each aspect of the present invention may include the following modifications. The control unit 50 outputs an image to the display 45 for displaying a GUI related to the setting of a mode for automatically bending the insertion section 3. When the setting of that mode is instructed through the GUI, the control unit 50 receives a mode signal.

[0164] Each aspect of the present invention may include the following modifications. The endoscope system 1 has an image sensor 30 that generates an image. The control unit 50 calculates the component in the direction of gravity on the imaging surface of the image sensor 30. The control unit 50 calculates the angle between the direction of the component in the direction of gravity on the imaging surface and the reference direction on the imaging surface. The control unit 50 corrects the image generated by the image sensor 30 by rotating it by that angle.

[0165] In the first embodiment, the endoscope system 1 bends the insertion section 3 in automatic bending mode such that the angle An between the frontal direction Dif and the gravity direction Dg of the tip section 34 increases. Therefore, the endoscope system 1 does not require complex operations by the user and can reduce foreign matter that comes into contact with the observation optical system. The user can perform the examination efficiently.

[0166] (First modification of the first embodiment) A first modification of the first embodiment of the present invention will now be described. Figure 20 shows an example of the configuration of an endoscope system 1a according to the first modification of the first embodiment. Parts that are the same as those shown in Figure 1 will not be described. The same blocks as those shown in Figure 1 are assigned the same reference numerals as those shown in Figure 1.

[0167] The endoscope system 1a shown in Figure 20 includes an optical adapter 2, an insertion section 3, a main body 4a, and a base unit 6.

[0168] The optical adapter 2 shown in Figure 20 is the same as the optical adapter 2 shown in Figure 1. The insertion part 3 shown in Figure 20 is the same as the insertion part 3 shown in Figure 1.

[0169] The main unit 4 shown in Figure 1 is replaced with the main unit 4a shown in Figure 20. The main unit 4a includes an image processing unit 40, a light source 41, a light source control unit 42, a motor unit 43, a curvature control unit 44, a control unit 50, and a communication unit 51. The communication unit 51 has a communication circuit and performs wired or wireless communication with the base unit 6 for purposes such as curvature control.

[0170] The base unit 6 includes a display 45, a memory unit 46, an operation unit 47, a curved joystick 48, an A / D converter 49, a control unit 60, and a communication unit 61. The control unit 60 controls each part of the base unit 6. The communication unit 61 has a communication circuit and performs wired or wireless communication with the main unit 4a for purposes such as curve control.

[0171] At least one of the control unit 50 and the control unit 60 performs the curvature control process shown in Figure 4. When it is necessary to share information between the main body 4a and the base unit 6, the control unit 50 and the control unit 60 communicate via the communication unit 51 and the communication unit 61.

[0172] The endoscope system of each aspect of the present invention may have two or more control units. Some or all of these two or more control units may cooperate with each other to perform bending control processing. Therefore, the functions of the control unit 50 shown in Figure 1 may be distributed among two or more control units.

[0173] Two or more control units may execute the processing sequentially. For example, a first control unit may execute a part of the curvature control processing and output the processing result to a second control unit. The second control unit may then execute the rest of the curvature control processing based on the processing result. Alternatively, two or more control units may execute the processing simultaneously.

[0174] The endoscope systems of each aspect of the present invention may include external devices. For example, the external devices may be personal computers, tablet terminals, or cloud servers on a network. The control unit 50 and one or more control units of the external devices may cooperate with each other to perform the bending control process. Alternatively, one or more control units of the external devices may perform the entire bending control process.

[0175] In the first modification of the first embodiment, similar to the first embodiment, the endoscope system 1a does not require complex operations by the user and can reduce foreign matter that comes into contact with the observation optical system.

[0176] (Second modification of the first embodiment) A second modification of the first embodiment of the present invention will now be described. Figure 21 shows an example of the configuration of an endoscope system 1b according to the second modification of the first embodiment. Parts that are the same as those shown in Figure 1 or Figure 20 will not be described. The same blocks as those shown in Figure 1 or Figure 20 are assigned the same reference numerals as those shown in Figure 1 or Figure 20.

[0177] The endoscope system 1b shown in Figure 21 comprises an optical adapter 2, an insertion section 3, a main body 4b, and a base unit 6b. The main body 4b and the base unit 6b are connected by a cable 7.

[0178] The optical adapter 2 shown in Figure 21 is the same as the optical adapter 2 shown in Figure 1. The insertion part 3 shown in Figure 21 is the same as the insertion part 3 shown in Figure 1.

[0179] The main unit 4 shown in Figure 1 is changed to the main unit 4b shown in Figure 21. The main unit 4b has a light source 41, a light source control unit 42, a motor unit 43, and a curvature control unit 44. The base unit 6b has an image processing unit 40, a display 45, a memory unit 46, an operation unit 47, a curved joystick 48, an A / D converter 49, and a control unit 50.

[0180] In the second modification of the first embodiment, similar to the first embodiment, the endoscope system 1b can reduce foreign matter that comes into contact with the observation optical system without requiring complex operations by the user.

[0181] (Second embodiment) A second embodiment of the present invention will now be described. Figure 22 shows the configuration of the endoscope system 1c according to the second embodiment. Parts that are the same as those shown in Figure 1 will not be described. The same blocks as those shown in Figure 1 are assigned the same reference numerals as those shown in Figure 1.

[0182] The endoscope system 1c shown in Figure 22 comprises an optical adapter 2, an insertion section 3c, and a main body 4. The optical adapter 2 shown in Figure 22 is the same as the optical adapter 2 shown in Figure 1. The main body 4 shown in Figure 22 is the same as the main body 4 shown in Figure 1.

[0183] The insertion section 3 shown in Figure 1 is changed to insertion section 3c. The insertion section 3c has an image sensor 30, a light guide 31, and a curved section 33. The insertion section 3c does not have the sensor 32 shown in Figure 1.

[0184] The control unit 50 determines the direction of gravity at the tip portion 34 by using the image generated by the image sensor 30. The control unit 50 outputs a control signal to the bending control unit 44 to bend the insertion portion 3c in the direction opposite to the direction of gravity.

[0185] Figure 23 shows an example of the bending control procedure performed by the endoscope system 1c. Referring to Figure 23, the operation of the endoscope system 1c in the bending control procedure will be explained. The explanation of the same procedure as shown in Figure 4 will be omitted.

[0186] After step S14, the control unit 50 determines the direction of gravity by using the image generated by the image sensor 30 (step S40).

[0187] Step S40 will now be described in detail. The control unit 50 determines the brightness of the image generated by the image sensor 30. Figure 24 shows an example of an image generated by the image sensor 30. For example, the control unit 50 divides the image IMG7 shown in Figure 24 into two or more regions and calculates the brightness value of each region based on the pixel value of the image IMG7. The image IMG7 is divided into 35 regions of the same size.

[0188] The control unit 50 identifies the region with the highest brightness value. Light reflected inside the pipe enters the image sensor 30. The light reflected from the bottom surface of the pipe is the brightest. The brightest region in image IMG7 corresponds to the bottom surface. For example, region R1 shown in Figure 24 has the highest brightness. The control unit 50 determines the position of region R1 in image IMG7. Region R1 is located in the lower right of the center of image IMG7. The control unit 50 determines that the direction of gravity is close to the lower right direction in image IMG7.

[0189] After step S40, the control unit 50 determines the approximate distance between the tip portion 34 and the bottom surface of the pipe. This approximate distance corresponds to the angle between the frontal direction of the tip portion 34 and the direction of gravity (step S42).

[0190] Step S42 will now be explained in detail. When the tip 34 is close to the bottom of the pipe, the bright area in the image generated by the image sensor 30 is wide. When the tip 34 is far from the bottom of the pipe, the bright area in the image is narrow. In other words, the area of ​​the bright area in the image corresponds to the approximate distance between the tip 34 and the bottom of the pipe. For example, the control unit 50 compares the brightness value of each area in the image IMG7 shown in Figure 24 with a reference value stored in the memory unit 46 and identifies areas with brightness values ​​exceeding the reference value. The control unit 50 determines the number of identified areas. This number indicates the area of ​​the bright area in the image IMG7 and corresponds to the approximate distance between the tip 34 and the bottom of the pipe.

[0191] The memory unit 46 stores in advance a target range for the approximate distance between the tip 34 and the bottom surface of the pipe. For example, the target range is greater than or equal to a first number. Alternatively, the target range is greater than or equal to a first number and less than or equal to a second number that is greater than the first number. After step S42, the control unit 50 retrieves the target range from the memory unit 46. The control unit 50 determines whether the approximate distance identified in step S42 falls outside the target range (step S44).

[0192] If the control unit 50 determines in step S44 that the approximate distance is not outside the target range, that is, that it is included in the target range, then step S24 is executed. If the control unit 50 determines in step S44 that the approximate distance is outside the target range, that is, that it is not included in the target range, then the control unit 50 outputs a control signal to the curvature control unit 44 to curve the insertion portion 3c in the opposite direction to the direction of gravity so that the approximate distance increases. For example, if the direction of gravity is the lower right direction in the image generated by the image sensor 30, then the direction in which the insertion portion 3c curves is the lower right direction. The curvature control unit 44 controls the motor unit 43 according to the control signal and curves the insertion portion 3c in the opposite direction to the direction of gravity so that the approximate distance increases. As the approximate distance increases, the angle between the front direction of the tip portion 34 and the direction of gravity increases (step S46). After step S46, step S24 is executed.

[0193] A stereo optical adapter with two fields of view may be used as optical adapter 2. The stereo optical adapter has a first optical system and a second optical system corresponding to the two fields of view. The first optical system and the second optical system form two optical images of the subject on the image sensor 30. The image sensor 30 generates a stereo image corresponding to the first optical image and the second optical image. The stereo image includes a pair of two images (the first image and the second image). That is, the stereo image includes an image of the subject as seen from the first viewpoint and an image of the subject as seen from the second viewpoint.

[0194] In step S42, the control unit 50 may calculate the distance between the tip 34 and the bottom surface of the pipe by performing a stereo measurement using two images included in the stereo image. For example, the control unit 50 may identify a first pixel in the brightest region of one of the two images, and identify a second pixel in the other image that corresponds to the first pixel. Based on the coordinates of the first pixel and the coordinates of the second pixel, the control unit 50 may calculate a three-dimensional distance from the tip 34 to a point in space corresponding to the first pixel. This three-dimensional distance represents the distance between the tip 34 and the bottom surface of the pipe.

[0195] Halation may occur in the image generated by the image sensor 30. Stereo measurements using pixels where halation occurs may contain errors in the measurement results. The control unit 50 may identify areas in the image generated by the image sensor 30 where halation does not occur and calculate the distance between the tip 34 and the bottom surface of the pipe based on the pixels of the brightest area among the identified areas.

[0196] Similar to the first embodiment, the control unit 50 may generate information in step S40 indicating the relationship between the direction of gravity and the reference direction on the imaging surface of the image sensor 30. The control unit 50 may superimpose this information onto the image generated by the image sensor 30, and output the superimposed image to the display 45.

[0197] Each aspect of the present invention may include the following modifications. The endoscope system 1c has an image sensor 30 that generates an image. The control unit 50 acquires the image output from the image sensor 30 and determines the direction of gravity at the tip 34 based on the image.

[0198] Each aspect of the present invention may include the following modifications. The control unit 50 determines the direction of gravity based on the brightness at each of two or more positions in the image generated by the image sensor 30.

[0199] Each aspect of the present invention may include the following modifications. The control unit 50 determines the distance between the tip portion 34 and the subject into which the insertion portion 3c is inserted by using the image generated by the image sensor 30. The control unit 50 outputs a control signal to curve the insertion portion 3c so that the distance increases.

[0200] In the second embodiment, the endoscope system 1c analyzes the image generated by the image sensor 30 in automatic bending mode and bends the insertion section 3c so that the angle between the frontal direction of the tip section 34 and the direction of gravity increases. Therefore, the endoscope system 1c does not require complex operations by the user and can reduce foreign objects that come into contact with the observation optical system. The user can perform the examination efficiently.

[0201] While preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments and their variations. Additions, omissions, substitutions, and other modifications are possible without departing from the spirit of the invention. Furthermore, the present invention is not limited by the foregoing description, but only by the scope of the appended claims. [Explanation of Symbols]

[0202] 1,1a,1b,1c Endoscopic System 2 Optical Adapters 3,3c Insertion part 4,4a,4b Main body 6,6b Base Unit 7 Cables 20 IMAGING LECON 21 Illumination lens section 30 Image sensors 31 Light Guide 32 sensors 33 Curved section 34 Tip 40 Image Processing Unit 41 Light source 42 Light source control unit 43 Motor section 44 Curve Control Unit 45 displays 46 Memory section 47 Operation section 48 Curved Joystick 49 A / D converters 50,60 Control Unit 51, 61 Communications Department

Claims

1. It has a sensor at the tip and a flexible insertion part, A control unit, A mode signal indicating a mode for automatically bending the insertion portion is received. The data output from the aforementioned sensor is acquired, Based on the aforementioned data, the direction of gravity at the tip is determined. A control unit that, after receiving the mode signal, outputs a control signal to curve the insertion portion such that the angle between the front direction of the tip and the direction of gravity increases, An endoscopic system having [a specific feature / feature].

2. After the aforementioned control signal is output, the control unit outputs a state control signal to control the curvature of the insertion portion so that the angle is fixed. The endoscopic system according to claim 1.

3. The control unit outputs the control signal to curve the insertion portion in a first direction and a second direction perpendicular to the first direction, such that the angle increases according to the amount of twisting of the insertion portion. The endoscopic system according to claim 1.

4. The control unit generates the control signal based on a predetermined value stored in memory. The predetermined value indicates the target value of the angle or the target value of the increase in the angle. The endoscopic system according to claim 1.

5. The memory stores two or more values ​​as predetermined values. The control unit obtains one of the two or more values ​​from the memory. The endoscopic system according to claim 4.

6. The control unit obtains from the memory a predetermined value corresponding to the type of optical adapter attached to the tip. The endoscopic system according to claim 5.

7. The predetermined value can be changed. The endoscopic system according to claim 4.

8. The aforementioned sensor is an accelerometer. The aforementioned data indicates the acceleration of the tip. The endoscopic system according to claim 1.

9. It has an image sensor that generates images, The control unit, The attitude of the tip is determined based on the aforementioned direction of gravity, The information indicating the aforementioned posture is superimposed onto the image. The image with the aforementioned information superimposed is output to the display. The endoscopic system according to claim 8.

10. The aforementioned sensor is an image sensor that generates an image as the data. The endoscopic system according to claim 1.

11. The control unit determines the direction of gravity based on the brightness at each of two or more locations in the image. The endoscopic system according to claim 10.

12. The control unit, By using the aforementioned image, the distance between the tip and the subject into which the insertion part is inserted is determined. Outputting the control signal for curving the insertion portion so that the aforementioned distance increases. The endoscopic system according to claim 11.

13. The control unit, An image for displaying a graphical user interface related to the setting of the aforementioned mode is output to the display. When the setting of the mode is instructed through the graphical user interface, the mode signal is received. The endoscopic system according to claim 1.

14. It has an image sensor that generates images, The control unit, The component in the direction of gravity on the imaging surface of the image sensor is calculated. The angle between the direction of the component on the imaging surface and the reference direction on the imaging surface is calculated. The image is corrected by rotating the image by the aforementioned angle. The endoscopic system according to claim 1.

15. It has a sensor at its tip and receives a mode signal indicating a mode in which the flexible insertion part is automatically bent. The data output from the aforementioned sensor is acquired, Based on the aforementioned data, the direction of gravity at the tip is determined. After receiving the mode signal, a control signal is output to curve the insertion portion so that the angle between the frontal direction of the tip and the direction of gravity increases. Control method.

16. The process involves receiving a mode signal that indicates a mode for automatically bending a flexible insertion part, which has a sensor at its tip, and The steps include acquiring data output from the aforementioned sensor, A step of determining the direction of gravity at the tip based on the aforementioned data, After receiving the mode signal, the step of outputting a control signal to curve the insertion portion such that the angle between the front direction of the tip and the direction of gravity increases, A program that causes a computer to execute something.

Images