Endoscope System

Abstract

To efficiently detect the presence / absence of a deviation in the up-down direction of a tip portion of an endoscope and support realization of a proper medical treatment.SOLUTION: An endoscope system comprises: an endoscope which can be inserted to / pulled out from a subject and has an imaging unit including an optical system and an image sensor on a tip part; a sensor for attitude control which is arranged on the tip part and detects at least the three directions being the front-rear direction, up-down direction and left-right direction of the tip part; and a calculation device which calculates the presence / absence of a deviation in the up-down direction of a photographed image captured by the imaging unit on the basis of the attitude information of the three directions detected by the sensor for attitude control. The calculation device displays an indicator indicating the presence / absence of a deviation in the up-down direction of the photographed image and the photographed image in association with each other on a monitor.SELECTED DRAWING: Figure 8A

Description

[Technical field]

[0001] The present disclosure relates to an endoscopic system. [Background technology]

[0002] Patent Document 1 discloses a blood vessel diameter measurement system that includes an endoscope having an image sensor mounted on its tip side that can be inserted into a blood vessel of a subject and capture images of the blood vessel, a driving device that moves the endoscope inserted into the blood vessel toward the base end at a constant speed, and a computing device that measures the blood vessel diameter based on at least one pre-movement image taken before the endoscope is moved by the driving device, at least one post-movement image taken after the endoscope is moved by the driving device, and the distance moved by the endoscope by the driving device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-185081 A Summary of the Invention [Problem to be solved by the invention]

[0004] The configuration of Patent Document 1 is based on the premise that the images captured before and after the movement of the endoscope in order to measure the blood vessel diameter are aligned in their vertical direction (in other words, the up-down direction of the captured images). Here, blood vessels in a subject such as a human body may be irregularly curved in places, and when an endoscope is inserted into or removed from a blood vessel, the tip of the endoscope may rotate, causing the vertical direction to shift. In such cases, a user such as a doctor viewing the endoscopic captured images may not be able to accurately understand the condition of the observed area inside the subject, which may make it difficult to perform appropriate medical procedures.

[0005] The present disclosure provides an endoscope system that efficiently detects whether or not there is any misalignment in the top-bottom direction of the tip portion of an endoscope, and assists in carrying out appropriate medical procedures. [Means for solving the problem]

[0006] The present disclosure relates to an endoscope that is insertable into and retractable from a subject and has an imaging unit at a tip end including an optical system and an image sensor, a posture control sensor that is disposed at the tip end and detects posture information of the tip end in at least three directions, i.e., forward / backward, up / down, and left / right, and an image captured by the imaging unit based on the posture information of the three directions detected by the posture control sensor. The tip of A calculation device that calculates the presence or absence of a misalignment in the top-bottom direction, the calculation device comprising: storing, in a memory, as calibration data, attitude information in the three directions of the tip portion detected by the attitude control sensor in an initial state before the endoscope is inserted into the subject, and using the calibration data to correct the attitude information in the three directions of the tip portion detected by the attitude control sensor after the endoscope is inserted into the subject; The above The tip portion and the Image capture Between the tip during imaging Check for vertical and circumferential misalignment and calculating the attitude of the tip portion when the captured image is captured. The present invention provides an endoscope system that displays on a monitor an indicator indicating the captured image in association with the captured image. Effect of the Invention

[0007] According to the present disclosure, it is possible to efficiently detect whether or not there is a misalignment in the top-bottom direction of the tip portion of an endoscope, thereby supporting the implementation of appropriate medical procedures. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a use case of the endoscope system according to the first embodiment. [Diagram 2] FIG. 2 is a schematic diagram illustrating a hardware configuration of the endoscope system illustrated in FIG. 1; [Diagram 3] FIG. 3 is a functional block diagram illustrating a functional configuration of the endoscope system shown in FIG. [Figure 4] FIG. 3 is a schematic diagram illustrating the hardware configuration of the distal end portion (camera head portion) of the endoscope shown in FIG. 2. [Diagram 5] FIG. 5 is a functional block diagram illustrating the processing in a processor of the camera control unit shown in FIG. [Figure 6] FIG. 11 is a flow diagram illustrating a processing procedure for superimposing an indicator on a captured image. [Figure 7A]FIG. 7 is a schematic diagram illustrating a first example of captured images at each insertion position before the process shown in FIG. 6 is executed; [Figure 7B] FIG. 7 is a schematic diagram illustrating a second example of captured images at each insertion position before the process shown in FIG. 6 is executed; [Figure 7C] FIG. 7 is a schematic diagram illustrating a third example of captured images at each insertion position before the process shown in FIG. 6 is executed; [Figure 7D] FIG. 7 is a schematic diagram illustrating a fourth example of captured images at each insertion position before the process shown in FIG. 6 is executed; [Figure 8A] FIG. 8 is a schematic diagram illustrating a first example of an image after the process shown in FIG. 6 is performed on the captured image shown in FIG. [Figure 8B] FIG. 8 is a schematic diagram illustrating a second example of an image after the process shown in FIG. 6 is performed on the captured image shown in FIG. [Figure 8C] FIG. 8 is a schematic diagram illustrating a third example of an image after the process shown in FIG. 6 is performed on the captured image shown in FIG. [Figure 9] FIG. 13 is a schematic diagram illustrating a hardware configuration of a rotation drive device according to a second embodiment; [Figure 10A] FIG. 11 is a schematic diagram illustrating a first example when a transmission cable is rotated by a rotary drive device; [Figure 10B] FIG. 11 is a schematic diagram illustrating a second example when the transmission cable is rotated by a rotary drive device; [Figure 11] FIG. 1 is a flow diagram illustrating a process for controlling rotation of a transmission cable about an axis. [Figure 12] A graph illustrating how the rotational force around the axis of a transmission cable is controlled to converge to a predetermined value. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, with reference to the drawings as appropriate, a detailed description will be given of a number of embodiments specifically disclosing an endoscope system according to the present disclosure. However, more detailed description than necessary may be omitted. For example, detailed description of already well-known matters or duplicate description of substantially identical configurations may be omitted. This is to avoid the following description becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Also, each of the attached drawings will be referred to according to the direction of the reference numerals. Note that the attached drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0010] For example, in this disclosure, a medical endoscope system (vascular endoscope system, described later) that is inserted into a blood vessel will be described as an example (use case) of the endoscope system, but the use case is not limited to this example. The contents of this disclosure can be appropriately applied to industrial endoscope systems that can observe not only blood vessels but also digestive organs such as the stomach or large intestine, and further, piping in a plant. In other words, the contents of this disclosure can be adopted for various purposes as long as they are used to image a tubular subject.

[0011] In addition, the term "unit" or "device" in the embodiments is not limited to a physical configuration that is mechanically realized by hardware, but also includes a configuration whose functions are realized by software such as a program. Also, the functions of one configuration may be realized by two or more physical configurations, or the functions of two or more configurations may be realized by, for example, one physical configuration.

[0012] (Embodiment 1) The first embodiment will be described with reference to FIGS. 1 to 8. FIG.

[0013] [Use cases in which the endoscope system of this embodiment is used] An example of a use case in which the endoscope system 1 of the present embodiment is used will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of a use case of the endoscope system 1 according to the first embodiment.

[0014] As shown in Fig. 1, an endoscope system 1 according to the present embodiment is a dedicated medical device used during surgery or examination, in which an endoscopic camera 10 is attached to the tip of a transmission cable 2 (including a catheter, etc.). The transmission cable 2 is provided with its outer circumferential surface covered with an insulating film, and connects an endoscopic camera 10 (an example of an endoscope) from its base end to its tip end. The endoscope system 1 targets a human subject, inserts the endoscopic camera 10 connected to the transmission cable 2 into the subject, and captures (observes) an image of an observation site (e.g., blood vessels) within the subject using the endoscopic camera 10.

[0015] The endoscopic camera 10 is sometimes called a vascular endoscope catheter. The outer diameter of the endoscopic camera 10 is, for example, 1.8 mmΦ as a maximum outer diameter, but is not limited to this size. The endoscopic camera 10 is inserted into, for example, a blood vessel inside the subject along a guide wire (not shown) that has been inserted beforehand into the blood vessel inside the subject so as to be insertable / retractable and movable forward / backward.

[0016] Here, the direction in which the endoscopic camera 10 is inserted toward the observation site inside the subject is defined as the advancement direction, and the direction in which the endoscopic camera 10 is withdrawn toward the outside of the subject is defined as the retreat direction. In other words, being insertable and retractable or freely advancing and retreating means that the endoscopic camera 10 can be inserted into the subject and withdrawn. The endoscopic camera 10 can be smoothly inserted to the observation site guided by a guide wire that has been inserted beforehand to the target site of surgery or examination (e.g., an affected area). The endoscopic system 1 may be a catheter in which the endoscopic camera 10 is replaceably attached to the tip of a normal catheter. A catheter is, for example, a medical tube used to drain body fluids or inject a medicinal liquid. In addition to the endoscopic camera 10, a balloon or a stent may be replaceably attached to the catheter.

[0017] In this embodiment, a standard coordinate system Σs is set (see FIG. 1). The standard coordinate system Σs is set so that its Z direction is along the direction of gravity, its Y direction is along the axial direction of the trunk when the patient is lying on the operating table, and its X direction is the left-right width direction of the trunk. That is, the Y axis of the standard coordinate system Σs is along the aforementioned direction of advancement, and FIG. 1 shows an example in which insertion is performed into the femoral vein (groin). In FIG. 1, the first insertion position where the insertion starts is indicated by the symbol "P0", and the second insertion position where the endoscopic camera 10 is further inserted in the subject and advanced further back than the first insertion position P0 is indicated by the symbol "P1".

[0018] As will be described later, in this embodiment, an acceleration sensor 17 (an example of an attitude control sensor) is provided in the camera head 11 of the endoscopic camera 10, and an acceleration sensor coordinate system Σg is set in the acceleration sensor 17 (see FIG. 1). The acceleration sensor 17 can also detect gravitational acceleration, so it is possible to calculate the amount of deviation (deviation angle) in the Z direction between the acceleration sensor coordinate system Σg and the standard coordinate system Σs. This calculation result indicates how much the captured image has rotated and deviated in the Z direction of the standard coordinate system Σs, in other words, in the up-down direction, and an indicator showing the deviation angle is displayed superimposed on the captured image based on the calculation result (see FIG. 8).

[0019] [Endoscope system configuration] The hardware configuration of the endoscope system 1 will be described with reference to Fig. 2 and Fig. 3. Fig. 2 is a schematic diagram illustrating a hardware configuration of the endoscope system 1 shown in Fig. 1. Fig. 3 is a functional block diagram illustrating a functional configuration of the endoscope system 1 shown in Fig. 2. For ease of explanation, the pullback device (see Fig. 3) and the repeater 4 (see Fig. 3) are omitted from Fig. 2.

[0020] As shown in Figures 2 and 3, the endoscopic system 1 includes an endoscopic camera 10, an auto pullback device 3, a repeater 4, a camera control unit 30 (CCU: Camera Control Unit, an example of a computing device), and a monitor 5.

[0021] The endoscopic camera 10 is a high-resolution camera with 480,000 pixels, and an imaging sensor 16 (an example of an image sensor, see below) capable of imaging blood vessels, for example, mounted on its tip (hereinafter also referred to as the "camera head unit 11"). In this embodiment, an acceleration sensor 17 (an example of a posture control sensor, see below) is also mounted on the tip of the endoscopic camera 10 together with the imaging sensor 16. When the endoscopic camera 10 is inserted into, for example, a blood vessel inside a subject, it is capable of imaging the inner wall of the blood vessel (hereinafter also referred to as the "blood vessel wall"). Note that the number of pixels is 480,000 only by way of example, and the number of pixels is not limited to 480,000.

[0022] The automatic pullback device 3 performs an operation of pulling back (in other words, pulling) the endoscopic camera 10, which is guided by the guidewire and inserted into the part to be observed (e.g., blood vessel) toward the base end at a pullback speed (e.g., constant speed). At this time, it can be considered that the length by which the automatic pullback device 3 pulls back the transmission cable 2 (in other words, the endoscopic camera 10) is approximately equal to the length by which the endoscopic camera 10 inserted into a position near the affected area is pulled back toward the base end. When the automatic pullback device 3 pulls back the endoscopic camera 10, the endoscopic system 1 captures images of the blood vessel wall at equal intervals. The automatic pullback device 3 outputs position information of the endoscopic camera 10 and speed information of the imaging position to the repeater 4.

[0023] Here, the speed information of the imaging position of the endoscopic camera 10 is the speed at which the imaging position of the endoscopic camera 10 (i.e., the position of the endoscopic camera 10) moves toward the base end at a constant speed when the endoscopic camera 10 is pulled back toward the base end, and is approximately equal to the pullback speed. The speed information of the imaging position of the endoscopic camera 10 is set by the auto pullback device 3 through the operation of a user such as a doctor. In addition, the start position of imaging is determined, for example, by a user such as a doctor after checking on the monitor 5 an image of an observation site (e.g., blood vessels) of a subject imaged by the endoscopic system 1. In addition, the pullback speed does not have to be constant and may be variable.

[0024] The relay section relays various signals between the endoscopic camera 10 and the camera control unit 30. The various signals include, for example, data signals (imaging data, etc.) of images captured by the endoscopic camera 10, as well as various control signals for the camera control unit 30 to control the endoscopic camera 10. Furthermore, in this embodiment, the various signals also include an acceleration signal detected by the acceleration sensor 17 of the endoscopic camera 10.

[0025] The camera control unit 30 includes a processor 31, a memory 32, an input / output interface 33, an operation unit 34, and a storage 35.

[0026] The camera control unit 30 is electrically connected to the endoscopic camera 10 via the repeater 4, and controls the imaging operation by the endoscopic camera 10 and the generation of a captured image based on a signal of the captured image from the endoscopic camera 10. In this embodiment, the camera control unit 30 acquires an acceleration signal from the acceleration sensor 17 (see FIG. 4) of the endoscopic camera 10, and calculates the amount of deviation (deviation angle) in the Z direction between the acceleration sensor coordinate system Σg and the standard coordinate system Σs based on the acquired result. The camera control unit 30 superimposes a compass animation icon AI (see later, FIGS. 8A to 8C) corresponding to the amount of deviation on the captured image and displays it on the monitor 5.

[0027] The camera control unit 30 is configured to include at least an image input unit (not shown), an image processing unit (not shown), and an image output unit (not shown) as part of its functions. These functions are stored and held in the memory 32 as programs, for example, and are realized by being executed by the processor 31.

[0028] The image input unit receives an image of a blood vessel wall or the like captured by the endoscopic camera 10 through the input / output interface 33. The image processing unit performs processing such as superimposing an indicator such as a compass animation icon AI on the input image of the blood vessel wall or the like. The image output unit outputs the captured image to the monitor 5 through the input / output interface 33. The functions of the image input unit (not shown), image processing unit (not shown), image output unit (not shown), and the like are collectively realized by each of the functional blocks shown in FIG. 5, which will be described later.

[0029] The processor 31 of the camera control unit 30 executes the programs stored in the memory 32 as described above to realize various functions of the endoscope system 1. The processor 31 may be provided by, for example, a GPU (Graphical Processing Unit) suitable for image processing. Instead of a GPU, the processor 31 may be configured by a dedicated electronic circuit designed with an MPU (Micro Processing Unit), a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or an electronic circuit designed to be reconfigurable with an FPGA (Field Programmable Gate Array), or the like.

[0030] The memory 32 is used as the working memory of the processor 31. The input / output interface 33 may be an interface using HDMI (Registered Trademark) (High-Definition Multimedia Interface) or USB (Universal Serial Bus) Type-C that can transfer video data at high speed in addition to a dedicated image input interface.

[0031] The operation unit 34 receives operations by a user such as a doctor. The operation unit 34 may be composed of one or a combination of a mouse, a keyboard, a touch pad, a touch panel, a microphone, or other input devices.

[0032] The storage 35 is a large-capacity storage device and stores data of captured images such as the blood vessel wall captured by the endoscope camera 10. The storage 35 may be configured to include, for example, a secondary storage device (e.g., HDD (Hard Disk Drive) or SSD (Solid State Drive)), or a tertiary storage device (e.g., optical disk, SD card).

[0033] The monitor 5 displays the measurement result of the blood vessel diameter or an image such as the blood vessel wall output from the camera control unit 30. When displaying an image of the blood vessel wall, the monitor 5 can be 3D-displayed as a three-dimensional image inside the blood vessel viewed from the direction desired by the user. The monitor 5 has a display device such as, for example, LCD (Liquid Crystal Display), organic EL (Electroluminescence), or CRT (Cathode Ray Tube).

[0034] The camera control unit 30 and the monitor 5 are mounted in a single housing as one imaging system 90.

[0035] [Regarding the Configuration of the Camera Head Unit of the Endoscope Camera] The configuration of the camera head unit 11, which is the tip portion of the endoscopic camera 10, will be described with reference to Fig. 4. Fig. 4 is a schematic diagram illustrating a hardware configuration of the tip portion (camera head unit 11) of the endoscope shown in Fig. 2.

[0036] The camera head 11 of the endoscopic camera 10 is formed in a cylindrical shape, and its base end is connected to the tip of the transmission cable 2. The camera head 11 is mainly composed of a holder 12 (lens barrel), a lens cover 13, a lens 15 (an example of an optical system), an imaging sensor 16 (an example of an image sensor), and an acceleration sensor 17 (an example of an attitude control sensor).

[0037] The holder 12 is formed in a substantially cylindrical shape, is located at the tip of the endoscopic camera 10 in the insertion direction, and has the lens cover 13 exposed on its tip surface. The lens 15 is formed in a disk shape and is used as a part of the optical system. The lens 15 is disposed adjacent to the lens cover 13 and the optical axis of the lens 15 is disposed so as to substantially coincide with the axis of the holder 12. The lens cover 13 and the lens 15 are each disposed and held on the tip side of the holder 12, and capture imaging light (in other words, light from an observation site (subject) such as blood vessels in a subject is incident). The lens 15 is held with its periphery covered by the holder 12. The lens 15 may be fixed integrally with the lens cover 13 on its surface.

[0038] Additionally, an imaging sensor 16 and an acceleration sensor 17 are disposed on the base end side of holder 12. Additionally, imaging sensor 16 and acceleration sensor 17 are held in a state in which their peripheries are covered by holder 12. Imaging sensor 16 and acceleration sensor 17 have a plurality of terminals, and electric wires are provided to each of the plurality of terminals, which are electrically connected to transmission cable 2.

[0039] The imaging sensor 16 is disposed closer to the tip of the camera head 11 than the acceleration sensor 17, and is disposed adjacent to the lens 15. That is, the imaging sensor 16 is disposed between the lens 15 and the acceleration sensor 17. The imaging sensor 16 is a solid-state imaging element such as a CCD (Charged-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor), and forms an image of light from a subject (e.g., an affected area of ​​an observation site) on an imaging surface through the lens 15, converts the formed optical image into an electrical signal, and outputs a signal of the captured image. In addition, since the lens 15 and the imaging sensor 16 function as an integrated unit, in this embodiment, a configuration including the lens 15 and the imaging sensor 16, which are an optical system, is also referred to as the "imaging unit 14."

[0040] The acceleration sensor 17 is disposed closer to the base end of the camera head unit 11 than the imaging sensor 16. The acceleration sensor 17 converts acceleration or impact, including gravitational acceleration (acceleration in the direction along the direction of gravity) acting on the camera head unit 11, into an electric signal, and outputs a signal such as the acceleration acting on the camera head unit 11. In addition, in this embodiment, as described above, an acceleration sensor coordinate system Σg is set in the acceleration sensor 17. Therefore, the acceleration sensor 17 is provided so as to be able to periodically detect acceleration signals (gravity acceleration and / or impact signals) in three mutually orthogonal (X, Y, Z) directions, for example, the front-rear direction, the up-down direction, and the left-right direction of the tip unit (camera head unit 11), in each of the components. The acceleration sensor 17 outputs the detection results in each of the X, Y, and Z components in the acceleration sensor coordinate system Σg. In addition, since the acceleration sensor 17 can also detect gravitational acceleration, the posture of the tip end (camera head unit 11 in this embodiment) to which the acceleration sensor 17 is attached can be calculated by dividing it into three directional components in the standard coordinate system Σs.

[0041] The inside of the camera head 11 (tip) is filled with a sealant so that there are no gaps. This filling secures the lens cover 13, the lens 15, the image sensor 16, and the acceleration sensor 17 firmly inside the camera head 11 of the endoscope camera 10. Therefore, the acceleration sensor 17 is disposed so that its relative position with respect to the image sensor 16 is fixed, that is, the relative positions of the image sensor 16 and the acceleration sensor 17 are fixed to each other. As a result, when a coordinate system is set in the image sensor 16, the coordinate system can be regarded as a coordinate system in which the acceleration sensor coordinate system Σg is simply translated. In other words, based on the XYZ components of the acceleration sensor coordinate system Σg detected by the acceleration sensor 17, that is, the posture information of the camera head 11 in three directions, it is possible to accurately calculate the amount of deviation (deviation angle, rotational position Pθ (described later)) of the captured image in the vertical direction.

[0042] [Functional configuration of the camera control unit] The functional configuration of the camera control unit 30 will be described with reference to Fig. 5. Fig. 5 is a functional block diagram illustrating the processing in the processor 31 of the camera control unit 30 shown in Fig. 4.

[0043] 5, the camera control unit 30 includes a multiplexed signal demodulation circuit 40, a signal multiplexing transmission circuit 41, a camera synchronization signal generation circuit 42, a pre-processing section 43, a signal amplification processing section 44, a YC separation processing / RGB conversion section 45, a correction processing section 46, a first gain level adjustment section 47, a second gain level adjustment section 48, an area detection processing section 49, an area generation section 50, and a fog feature amount detection data addition / average / storage section 51. The camera control unit 30 further includes an encoder processing section 52, an area display signal replacement / mixing / selection section 53, an indicator generation section 54, a control section 55, a data storage section 56, an external switch 57, and an external communication section 59. The external switch 57 includes a camera setting change button 58.

[0044] The multiplexed signal demodulation circuit 40 demodulates and outputs the multiplexed power supply signal, drive signal, and setting signal for each circuit of the image sensor 16 output from the image sensor 16 in the camera head unit 11. The signal multiplexing transmission circuit 41 multiplexes the captured image signal from the image sensor 16 and each information signal inside the camera head unit 11, and outputs this multiplexed signal.

[0045] The pre-processing unit 43 performs pre-processing such as noise removal and color separation processing using a pixel filter array on the captured image signal output from the imaging sensor 16 of the camera head unit 11. The signal amplification processing unit 44 performs signal processing such as amplification on the captured image signal that has been pre-processed by the pre-processing unit 43.

[0046] The first gain level adjustment section 47 adjusts the amplification factor (that is, the gain) of the captured image signal in accordance with a setting value from the control section 55.

[0047] The YC separation processing / RGB conversion unit 45 separates the captured image signal from the signal amplification processing unit 44 into a luminance signal (Y signal) and a color signal (C signal), and converts the color signal into an RGB signal. The YC separation processing / RGB conversion unit 45 also outputs the separated Y signal to the area display signal replacement / mixing / selection unit 53 via the correction processing unit 46.

[0048] The correction processing unit 46 generates color difference signals (RY), (GY), and (BY) using the RGB signal output from the YC separation processing / RGB conversion unit 45 , and outputs them to the encoder processing unit 52 .

[0049] The second gain level adjustment unit 48 adjusts the gain of a circuit that can divide and set the input level for each region for the correction processing unit 46 in accordance with a setting value read by the control unit 55. The setting value read by the control unit 55 is stored and held in the data storage unit 56.

[0050] The data storage unit 56 stores and holds the intravascular correction setting values ​​as the above-mentioned setting values. The data storage unit 56 also stores and holds intravascular feature amount detection data.

[0051] The area generating unit 50 generates a display area frame signal based on the area designation coordinates output from the control unit 55, and outputs the display area frame signal to the area display signal replacing / mixing / selecting unit 53 and the fog feature amount detection data adding / averaging / storing unit 51.

[0052] The display area frame signal is a signal that displays a correction detection area frame on the display panel of the monitor 5. The size of this correction detection area frame can be set arbitrarily or automatically. For example, when the diameter of the endoscopic camera 10 is small and the imaging range is narrow, the size of the correction detection area frame is set to a small (appropriate) value. When the correction detection area frame is set automatically, for example, when the diameter of the endoscopic camera 10 is small, the imaging range becomes narrow, so the size may be set to be automatically selectable according to the diameter.

[0053] When the area display signal replacement mixing / selection unit 53 receives a display area frame signal from the area generation unit 50, it mixes the display area frame signal with the Y signal from the YC separation processing / RGB conversion unit 45 and outputs the result to the encoder processing unit 52. When area display switch designation coordinates are input from the control unit 55, the area display signal replacement mixing / selection unit 53 determines the position of the correction detection area frame according to the coordinate information.

[0054] The designated coordinates for switching the area display are determined by the amount of operation of the camera setting change button 58 in the menu for moving the correction detection area frame. The correction detection area frame moves to the right by pressing the right button of the left / right / up / down buttons of the camera setting change button 58, for example, and moves to the left by pressing the left button. The correction detection area frame also moves to the top by pressing the top button, and moves to the bottom by pressing the bottom button.

[0055] The area detection processing unit 49 detects the portion where feature detection is desired, for example the area where fog is occurring on the captured image (i.e., the display area of ​​the correction detection area frame), from each signal from the YC separation processing / RGB conversion unit 45, and outputs the result to the fog feature detection data addition / average / save unit 51.

[0056] The fog feature amount detection data addition / average / save unit 51 adds up the fog feature amount detection data for the display area where fog has occurred on the captured image, detected by the area detection processing unit 49, based on the display area frame signal input from the area generation unit 50, calculates the average and statistical frequency (statistical data), and saves the results.

[0057] The indicator generating unit 79 generates character data and graphic data to be superimposed on the captured image in accordance with an indicator display specification such as a character and / or a graphic input from the control unit 55. In this embodiment, as an example of character data, XYZ coordinate values ​​indicating the position of the camera head unit 11 in the standard coordinate system Σs are generated (see Figs. 8A to 8C). As an example of graphic data, a compass animation icon AI indicating the attitude information of the captured image in the standard coordinate system Σs is generated (see Figs. 8A to 8C). These generated indicators are output to the encoder processing unit 52 through the area display signal replacement mixing / selection unit 53 so as to be superimposed on the signal of the captured image. Note that the XYZ coordinate values ​​indicating the position of the camera head unit 11 in the standard coordinate system Σs, which are an example of character data, are calculated by, for example, double integrating the detection value of the acceleration sensor 17 of the camera head unit 11.

[0058] The camera synchronization signal generating circuit 42 generates an image sensor drive signal for driving the image sensor 16 of the camera head unit 11, a synchronization signal for signal processing used in the operation of each unit, and a vertical synchronization signal.

[0059] The external communication unit 59 communicates with an external device (e.g., a computer) and enables the external device to select a display area and to retrieve pixel information or color space information of the display area selected by the external device. Here, the display area selected by the external device is not limited to one, and multiple display areas may be selected simultaneously. By providing the external communication unit 59 in the camera control unit 30, in addition to key operations performed by the unit itself, key operations can be performed on the external device side, improving the degree of freedom of operation. For example, it becomes possible to interactively select a display area using a mouse-type pointer or GUI, perform color correction, and change and save the settings of the camera control unit 30.

[0060] The control section 55 comprehensively controls the operations of each section of the camera control unit 30 in addition to the above-mentioned processing sections.

[0061] For example, the control unit 55 reads out the statistical data and brightness level stored in the fog feature amount detection data addition / average / storage unit 51, calculates a gain setting value to be used during fog correction processing, and sets this gain setting value in the gain level adjustment unit.

[0062] The fog correction process is a known technique, and various types of processing are exemplified. For example, in the fog correction process, in order to increase the contrast of the entire screen, which is reduced by the concentration of the brightness histogram in the intermediate gradations due to the occurrence of fog, the correction process setting value (gain setting value in this embodiment) is changed so as to flatten the brightness histogram. In addition, an external switch 57 including a camera setting change button 58 is connected to the control unit 55. In addition, the control unit 55 communicates with a computer, which is an external device, via an external communication unit 59, and enables the computer to control the camera control unit 30. In addition, the control unit 55 is connected to the external switch 57 including the camera setting change button 58.

[0063] Also, in the present embodiment, as will be described later, the control unit 55 acquires an acceleration signal, that is, attitude information in three directions, from the acceleration sensor 17 of the camera head unit 11 (endoscope camera 10) through the multiplex signal demodulation circuit 40. In the initial state before the endoscope is inserted into the subject, the control unit 55 causes the data storage unit 56 to store and hold the attitude information in the three directions as calibration data.

[0064] After the endoscope camera 10 is inserted into the subject, the control unit 55 calculates the deviation amount (deviation angle) in the Z direction between the acceleration sensor coordinate system Σg and the standard coordinate system Σs, in other words, the deviation amount in the vertical direction of the captured image captured by the imaging unit 14, based on the attitude information in the three directions. Based on the calculation result, the control unit 55 gives an indicator display specification to the indicator generation unit 54. As a result of this assignment, the camera control unit 30 can display an indicator indicating the presence or absence of deviation in the vertical direction of the captured image in association with the captured image on the monitor 5.

[0065] [Processing procedure for superimposing an indicator on a captured image] With reference to FIG. 6, the processing procedure for the camera control unit 30 to superimpose an indicator on the captured image will be described. FIG. 6 is a flowchart illustrating the processing procedure for superimposing an indicator on the captured image.

[0066] As shown in FIG. 6, when the endoscope system 1 is powered on, the endoscope camera 10, the pullback device, the repeater 4, the camera control unit 30, and the monitor 5 (see FIG. 3) are initialized. At this time, in the endoscope camera 10, at least the image sensor 16 and the acceleration sensor 17 are initialized (S11). After the initialization, the control unit 55 of the camera control unit 30 reads out the above-mentioned calibration data from the data storage unit 56 (S12). This calibration data is the orientation information in three directions of the camera head unit 11 of the endoscope camera 10 in the initial state before the endoscope is inserted into the subject, and serves as a reference value for calculating the orientation information in three directions after the endoscope is inserted. In addition, the orientation information in three directions of the calibration data is coordinate-converted from the acceleration sensor coordinate system Σg to the standard coordinate system Σs.

[0067] After the endoscope is inserted into the subject, at a current insertion position, such as the first insertion position P0 or the second insertion position P1, the control unit 55 acquires an acceleration signal from the acceleration sensor 17 of the camera head unit 11 and detects current posture information of the camera head unit 11 in three directions (S13). Since the posture information detected by the acceleration sensor 17 is a value in the acceleration sensor coordinate system Σg, the control unit 55 converts the posture information in the acceleration sensor coordinate system Σg into posture information in the standard coordinate system Σs. The control unit 55 corrects the current posture information by subtracting the value of the calibration data (initial posture information) from the posture information after the conversion (S14).

[0068] The control unit 55 calculates the amount of misalignment in the up-down direction of the image captured by the imaging sensor 16, that is, the rotational position Pθ, based on the corrected current attitude information in the three directions, and calculates whether or not there is any misalignment (S15). The control unit 55 instructs the indicator generation unit 54 to display an indicator such as a character and / or a figure. This indicator display instruction generates, as character data, XYZ coordinate values ​​indicating the position of the camera head unit 11 in the standard coordinate system Σs. Also, as figure data, a compass animation icon indicating the attitude of the captured image in the standard coordinate system Σs is generated. The control unit 55 displays the indicators indicating the calculated rotational position Pθ by superimposing them on the captured image output from the imaging sensor 16 (S16).

[0069] The control unit 55 determines whether the process flow should end (S17). If it is determined that the process flow should not end (NO in S17), the process flow returns to step S13, and the above-mentioned steps S13 to S16 are repeatedly executed.

[0070] That is, the above-mentioned steps S13 to S16 are executed sequentially as long as it is determined that the entire process flow should not end, and during that time, the indicator is updated and displayed on the captured image. This allows a user such as a doctor of the endoscope system 1 to easily grasp how much the camera head 11 (tip) of the endoscope camera 10 has rotated around its axis, in other words, how much it has shifted in the up-down direction. On the other hand, if it is determined that the process flow should end (or has ended) (YES in S17), the entire process flow ends (END).

[0071] [Use case where an indicator is superimposed on a captured image] A use case in which an indicator is superimposed on a captured image will be described with reference to Figures 7A, 7B, 7C, 7D, and 8A, 8B, and 8C. Figures 7A to 7D are schematic diagrams illustrating captured images at each insertion position before the process shown in Figure 6 is executed. Figures 8A to 8C are schematic diagrams illustrating images after the process shown in Figure 6 is executed on the captured image shown in Figure 7.

[0072] Specifically, Fig. 7A is an example of a blood vessel image at the first insertion position P0 shown in Fig. 1. Fig. 7B is an example of a blood vessel image at a second insertion position P1 (see Fig. 1) that is further inserted from the first insertion position P0. Fig. 7C is an example of a blood vessel image rotated 10 degrees clockwise in the drawing when the posture of Fig. 7B at the second insertion position P1 is used as a reference. Fig. 7D is an example of a blood vessel image rotated 10 degrees counterclockwise in the drawing when the posture of Fig. 7B at the second insertion position P1 is used as a reference.

[0073] Fig. 8A is an example of a screen in which the process shown in Fig. 6 is performed on the blood vessel image shown in Fig. 7B and an indicator is superimposed on the screen. Fig. 8B is an example of a screen in which the process shown in Fig. 6 is performed on the blood vessel image shown in Fig. 7C and an indicator is superimposed on the screen. Fig. 8C is an example of a screen in which the process shown in Fig. 6 is performed on the blood vessel image shown in Fig. 7D and an indicator is superimposed on the screen.

[0074] As shown in Fig. 7A, the endoscopic camera 10 is inserted into the subject and an image of an observation site inside the subject at a first insertion position P0 (see Fig. 1) is captured. At the first insertion position P0, a user such as a doctor can intuitively grasp how much the endoscopic camera 10 has rotated around its axis immediately after insertion into the subject. Therefore, when the blood vessel image of Fig. 7A is displayed on the monitor 5, the user can smoothly operate the device without being aware of the misalignment between the up-down direction of the blood vessel image and the up-down direction (direction of gravity).

[0075] On the other hand, when the endoscopic camera 10 is inserted further into the subject and positioned at a second insertion position P1 (see FIG. 1), it is difficult to extract any particular landmarks or feature points from the blood vessel image, as shown in FIGs. 7B to 7D. Therefore, it is difficult for the user to grasp, in the current situation, how much the endoscopic camera 10 positioned at the back has rotated around its axis, based on the captured images shown in FIGs. 7B to 7D.

[0076] Therefore, in this embodiment, the camera control unit 30 calculates whether or not there is a vertical shift in the image captured by the imaging section 14, based on the three-directional posture information detected by the acceleration sensor 17 arranged in the camera head section 11 of the endoscopic camera 10.

[0077] As shown in Figs. 8A to 8C, based on the calculation result, the camera head unit 11 displays on the monitor 5 an indicator indicating the presence or absence of a vertical deviation of the blood vessel image in association with the blood vessel image. Specifically, on the screen of the monitor 5, an indicator display area A2 is disposed adjacent to an imaging area A1 (left side in Figs. 8A to 8C) in which the blood vessel image is displayed (right side in Figs. 8A to 8C). As a text indicator, one side (upper right side in Figs. 8A to 8C) of the indicator display area A2 displays XYZ coordinate values ​​indicating the position of the camera head unit 11 in the standard coordinate system Σs. As a graphic indicator, a compass animation icon indicating the orientation of the blood vessel image in the standard coordinate system Σs is displayed (lower right side in Figs. 8A to 8C).

[0078] In this way, the indicator indicating the presence or absence of misalignment in the vertical direction of the captured image is displayed on the monitor 5 in association with the captured image, so that a user such as a doctor can easily grasp the degree of misalignment of the endoscopic camera 10 around the axis based on the vertical direction when the endoscopic camera 10 is inserted into the subject, regardless of whether the insertion position of the endoscopic camera 10 is at the back or the front, thereby improving the operability of the endoscope system 1. In other words, it is possible to efficiently detect the presence or absence of misalignment in the vertical direction of the tip of the endoscope and support the implementation of appropriate medical procedures. As a result, in the medical field, it is possible to reduce the occurrence of operational errors by users such as doctors and to improve medical safety for patients.

[0079] As described above, the endoscope system 1 of the first embodiment includes an endoscopic camera 10 (an example of an endoscope) that is insertable into and retractable from a subject (e.g., a blood vessel of a human body) and has an imaging section 14 at its tip including a lens 15 (an example of an optical system) and an imaging sensor 16 (an example of an image sensor), an acceleration sensor 17 (an example of an attitude control sensor) that is disposed at the tip and detects attitude information in at least three directions, the front-rear direction, the up-down direction, and the left-right direction, of the tip, and a camera control unit 30 (an example of a calculation device) that calculates the presence or absence of misalignment in the up-down direction of an image captured by the imaging section 14 based on the attitude information in the three directions detected by the acceleration sensor 17. The camera control unit 30 displays an indicator indicating the presence or absence of misalignment in the up-down direction of the captured image on the monitor 5 in association with the captured image.

[0080] Therefore, an indicator indicating the presence or absence of misalignment in the top-bottom direction of a captured image is displayed on the monitor 5 in association with the captured image, so that a user such as a doctor can easily grasp the degree to which the endoscopic camera 10 is misaligned around the axis based on the top-bottom direction when the endoscopic camera 10 is inserted into a subject, regardless of whether the insertion position of the endoscopic camera 10 is at the back or the front, thereby improving the operability of the endoscope system 1. In other words, it is possible to efficiently detect the presence or absence of misalignment in the top-bottom direction of the tip of the endoscope and to support the realization of appropriate medical procedures. As a result, in the medical field, it is possible to suppress the occurrence of operational errors by users such as doctors and to improve medical safety for patients.

[0081] Furthermore, according to the endoscope system 1 of the first embodiment, the acceleration sensor 17 (an example of an attitude control sensor) is disposed so that its relative position with respect to the imaging sensor 16 (an example of an image sensor) is fixed. Therefore, when a coordinate system is set in the imaging sensor 16, the coordinate system can be regarded as a coordinate system obtained by simply translating the acceleration sensor coordinate system Σg. In other words, based on the XYZ components in the acceleration sensor coordinate system Σg including the gravitational acceleration detected by the acceleration sensor 17, i.e., the attitude information in three directions of the camera head unit 11, the amount of deviation (deviation angle, rotational position Pθ) of the captured image in the vertical direction can be calculated with high accuracy.

[0082] Furthermore, according to the endoscope system 1 of the first embodiment, the imaging unit 14 and the acceleration sensor 17 (an example of an attitude control sensor) are fixed by a sealant filled at the tip. Therefore, the imaging sensor 16 and the acceleration sensor 17 are firmly fixed inside the camera head 11 of the endoscope camera 10, so that the amount of deviation in the vertical direction of the captured image can be calculated with higher accuracy based on the attitude information of the camera head 11 in three directions.

[0083] Furthermore, according to the endoscope system 1 of the first embodiment, the camera control unit 30 (an example of a computing device) stores in the memory 32 as calibration data attitude information in three directions detected by the acceleration sensor 17 (an example of an attitude control sensor) in the initial state before the endoscopic camera 10 (an example of an endoscope) is inserted into the subject, and performs correction using the calibration data from the attitude information in three directions detected by the acceleration sensor 17 after the endoscopic camera 10 is inserted into the subject. Therefore, based on the attitude information in three directions of the camera head 11, the amount of misalignment in the up-down direction of the captured image can be calculated with higher accuracy, and the presence or absence of misalignment in the up-down direction of the tip of the endoscope can be detected more efficiently.

[0084] (Embodiment 2) A second embodiment of the present disclosure will be described with reference to Figures 9 to 12. Note that, since the description of the same or equivalent parts as those in the first embodiment described above will be repeated, the same reference numerals will be given to the drawings and the description thereof may be omitted or simplified.

[0085] [Configuration of the rotation drive device 60] A rotary drive device 60 (an example of a drive device) that applies a rotational force around the axis of the transmission cable 2 to the tip portion will be described with reference to Figs. 9 and 10. It is a schematic diagram illustrating a hardware configuration of the rotary drive device 60 of this embodiment. Fig. 9 is a schematic diagram illustrating a hardware configuration of a rotary drive device according to a second embodiment. Fig. 10A is a schematic diagram illustrating a first example when the transmission cable 2 is rotated by the rotary drive device 60. Fig. 10B is a schematic diagram illustrating a second example when the transmission cable 2 is rotated by the rotary drive device 60.

[0086] The endoscope system 1 of this embodiment is configured to further include a rotation drive device 60 compared to the above-described first embodiment.

[0087] The rotation drive device 60 is disposed near the camera control unit 30 or the pull-back device on the base end side of the transmission cable 2 (opposite the tip end to which the endoscopic camera 10 is attached) and attached to the transmission cable 2. The rotation drive device 60 fixes the base end side of the transmission cable 2 and applies a rotational force around the axis (axial direction) of the transmission cable 2 to the tip end via the transmission cable 2. This makes it possible to change and adjust the attitude around the axis (i.e., the rotation angle) of the camera head unit 11 attached to the tip end of the transmission cable 2 (see below).

[0088] 9, the rotary drive device 60 includes a cable holding part 61 that holds the transmission cable 2 rotatably around an axis, a movable shaft holding part 63 that is provided and coupled to the cable holding part 61, and a passive shaft holding part 67 that is also provided and coupled to the cable holding part 61. The cable holding part 61 is formed with an insertion hole 62 (not shown) through which the transmission cable 2 is inserted. The movable shaft holding part 63 and the passive shaft holding part 67 are disposed opposite each other with the axis of the insertion hole 62 of the cable holding part 61 in between. When the cable holding portion 61 holds the transmission cable 2, the transmission cable 2 is inserted into the insertion hole 62, and the transmission cable 2 is held in the insertion hole 62 so that its axis approximately coincides with the axis of the insertion hole 62 and can rotate freely.

[0089] A movable roller 64 is provided on the movable shaft holding portion 63, and the movable roller 64 is supported so that its axis is approximately parallel to the insertion hole 62. A urethane sponge is attached to the outer periphery of the movable roller 64, and the movable roller 64 abuts on the transmission cable 2 with its outer periphery. The attachment of the urethane sponge allows slippage (exceeding the frictional force) between the movable roller 64 and the transmission cable 2, suppressing the transmission of an excessive or unreasonable rotational force (see below) to the transmission cable 2, and makes it possible to prevent, for example, damage to an observation site inside the subject.

[0090] Further, a motor 65 is disposed in the movable shaft holding portion 63, and an output roller 66 is provided at the tip of the output shaft of the motor 65. The outer circumferential surfaces of the output rollers 66 abut against the movable rollers 64, and transmit the driving force of the motor 65 to the movable rollers 64. As a result, the driving force of the motor 65 is transmitted to the base end of the transmission cable 2 via the output roller 66 and the movable roller 64 as a rotational force about the axis of the transmission cable 2.

[0091] A passive roller 68 is provided on the passive shaft holding portion 67, and the passive roller 68 is also rotatably supported so that its axis is approximately parallel to the insertion hole 62, similar to the movable roller 64. Urethane sponge is also attached to the outer periphery of the passive roller 68, and the passive roller 68 abuts against the transmission cable 2 with its outer periphery.

[0092] The passive shaft holding section 67 further includes a slider section 69. The slider section 69 directly holds the passive roller 68 and moves the passive roller 68 so as to approach the axis of the transmission cable 2 while maintaining the axis of the passive roller 68 parallel to the axis of the transmission cable 2. A biasing section 70 made of an elastic member such as a compression spring is attached to the slider section 69, and the slider section 69 is biased to move the passive roller 68 so as to always approach the axis of the transmission cable 2. This biasing movement adjusts the contact surfaces of the transmission cable 2, the movable roller 64, and the passive roller 68 so as to generate a predetermined frictional force. As a result, the movable roller 64 and the passive roller 68 are provided so as to be clamped to such an extent that no excessive or excessive rotational force is transmitted to the base end of the transmission cable 2 held by the cable holding section 61.

[0093] As shown in FIG. 10A, when a rotational force is applied to the base end of the transmission cable 2 around an axis in the forward direction (e.g., clockwise) by driving the motor 65 of the rotation drive device 60, the camera head unit 11 similarly rotates around the axis in the clockwise direction with a delay corresponding to the distance between the base end and the tip end (camera head unit 11).

[0094] On the other hand, as shown in Fig. 10B, when a rotational force is applied to the base end of the transmission cable 2 by driving the motor 65 of the rotation drive device 60 around an axis in the opposite direction (e.g., counterclockwise) different from that in Fig. 10A, the camera head unit 11 rotates around the counterclockwise axis with a delay corresponding to the distance between the base end and the tip end. By applying such a rotation, the camera head unit 11 of the endoscopic camera 10 also rotates in the same manner, and as a result, the image captured by the imaging sensor 16 also rotates, making it possible to capture an image of an observation site inside the subject at a desired rotation angle set in advance.

[0095] In order to control the rotation drive device 60 configured in this manner, the camera control unit 30 calculates (calculates) a rotation force for rotating the tip part around an axis (axial direction) based on the difference between the posture information in three directions detected by the acceleration sensor 17 and a predetermined posture state. Then, the camera control unit 30 instructs the rotation drive device 60 to rotate the tip part around the axis based on the calculation result of the rotation force. That is, as described later, the camera control unit 30 calculates a control value for the motor 65 of the rotation drive device 60 (hereinafter also referred to as the "motor 65 control value") and controls the rotation force of the motor 65 based on the motor 65 control value, thereby defining the rotation position of the camera head part 11 of the endoscopic camera 10 at a desired rotation angle. Note that the value of the target angle Nθ (desired rotation angle in the standard coordinate system Σs) of the camera head part 11 of the endoscopic camera 10 based on the top-bottom direction is set (stored and held) in advance in the data storage unit 56 of the camera control unit 30.

[0096] [Processing procedure for controlling rotation around the axis of the transmission cable] A process for controlling the rotation around the axis of the transmission cable 2, which is executed by the camera control unit 30 of this embodiment, will be described with reference to Fig. 11 and Fig. 12. Fig. 11 is a flow diagram illustrating a process for controlling the rotation around the axis of the transmission cable 2. Fig. 12 is a graph illustrating a state in which the rotation force around the axis of the transmission cable 2 is controlled and converges to a predetermined value.

[0097] 11 is executed sequentially at each of the first insertion positions P0,...,Pn (where n is 0 or a natural number),... on the insertion path when the endoscopic camera 10 is inserted toward or withdrawn from an observation site inside a subject. That is, symbols such as "Pθn" and "Δθn" mean values ​​corresponding to the insertion position Pn (values ​​at the time of reaching the insertion position Pn). Also, "Pn+1" means the next insertion position after a predetermined period of time has elapsed.

[0098] 11, the camera control unit 30 calculates a rotational position Pθn at the current insertion position Pn based on the posture information in three directions from the acceleration sensor 17 (see step S15 in FIG. 6 for the calculation of the rotational position Pθ). The camera control unit 30 reads out the target angle Nθ from the data storage unit 56, and calculates an angle error Δθn (=Pθn-Nθ) which is the difference between the current rotational position Pθn and the target angle Nθ (S21).

[0099] Based on the calculation result of the angle error Δθn, the camera control unit 30 judges whether the angle error Δθn is within a predetermined range near 0 (zero) (S22). If the judgment result judges that the angle error Δθn is within the predetermined range near 0 (zero) (YES in S22), the camera control unit 30 judges not to issue a control instruction (change / update) to the motor 65 of the rotation drive device 60, and maintains the current output of the motor 65 control value (S23), and then the process flow proceeds to step S28. By judging whether the angle error Δθn is within the predetermined range near 0 (zero), in other words by setting a dead zone, it is possible to prevent a slight rotational force from being applied to the tip (camera head unit 11) of the endoscopic camera 10. As a result, it is possible to suppress a slight movement in the camera head unit 11, and suppress, for example, screen sickness of a user such as a doctor, or deterioration over time of the endoscopic camera 10.

[0100] If it is determined that the angle error Δθn is not within a predetermined range near 0 (zero) (NO in S22), the camera control unit 30 determines whether the angle error Δθn is positive (+) or not (S24). That is, here, the camera control unit 30 calculates (calculates) the direction of rotation around the axis of the camera head unit 11 (tip portion) based on the above-mentioned difference (angle error Δθn). If it is determined that the angle error Δθn is positive (+) as a result of the calculation (YES in S24), the camera control unit 30 uses the angle error Δθn and the error gain Km (coefficient) corresponding to the angle error Δθn to calculate a control value for the motor 65 of the rotation drive device 60 in the forward direction, for example, the clockwise direction (see FIG. 10A) at the next insertion position (S25). For example, the control value of the motor 65 at this time is defined as a value linearly related to the value obtained by multiplying the angle error Δθn by the error gain Km by a servo control system such as PD control or PID control as appropriate. As shown in FIG. 12, by such servo control, the angle error Δθn at the current insertion position Pn converges to 0 (zero) within the steady-state error range.

[0101] Returning to Fig. 11, the explanation will be continued. If it is determined that the angular error Δθn is not positive (+), that is, is negative (-) (NO in S24), then the camera control unit 30 uses the angular error Δθn and an error gain Kp (coefficient) corresponding to the angular error Δθn to calculate a control value for the motor 65 of the rotation drive device 60 in the opposite direction to that in step S25, for example, the counterclockwise direction (see Fig. 10B) at the next insertion position (S25). For example, the control value for the motor 65 at this time is also defined as a value linearly related to the value obtained by multiplying the angular error Δθn by the error gain Kp.

[0102] In this way, when the control value of the motor 65 in the forward or reverse direction is calculated (S25, S26), the control unit applies a rotational force around the axis in the clockwise or counterclockwise direction to the proximal end of the transmission cable 2 by driving and controlling the output of the motor 65 of the rotational drive device 60 (S27). Due to this applied rotation, the camera head portion 11 (tip portion) of the endoscope camera 10 also rotates in the same manner. As a result, the imaging sensor 16, for which the relative position with the image sensor becomes fixed by the attitude control sensor, also rotates the captured image, enabling imaging of the observation site within the subject at a desired angle (i.e., the target angle Nθ) in the preset standard coordinate system Σs, for example, in a state where the up-down and left-right directions are normal (for example, an image as shown in Fig. 8A can be obtained).

[0103] The camera control unit 30 determines whether the processing flow should end (S28). If it is determined based on the determination result that the processing flow should end (or has ended) (YES in S28), the entire processing flow ends (END). On the other hand, if it is determined that it should not end (NO in S28), the processing flow updates the variable n to n + 1 and returns to step S21. That is, the insertion position of the endoscope camera 10 is updated to the next insertion position, and steps S21 to S27 are sequentially executed for each insertion position. That is, the camera control unit 30 repeatedly executes the calculation of the rotational force and the instruction to the rotational drive device 60 (S21 to S27).

[0104] By such execution, the camera control unit 30 repeatedly executes the calculation of the rotational force and the instruction to the rotational drive device 60, and while the endoscope camera 10 is inserted into the subject, it can sequentially rotate the tip portion (camera head portion 11) around the axis at a desired rotation angle with reference to the up-down and left-right directions. Therefore, the captured image by the imaging sensor 16 also rotates, enabling imaging of the observation site within the subject at a desired angle. As a result, since the endoscope system 1 images the observation site in a fixed attitude with reference to the up-down and left-right directions, a user such as a doctor can appropriately and easily grasp in which direction the observation site within the subject is being imaged.

[0105] As described above, the endoscope system 1 of the second embodiment further includes a rotation drive device 60 (an example of a drive device) that fixes the base end side of the transmission cable 2 that connects the base end to the tip end of the endoscopic camera 10 (an example of an endoscope) and applies a rotational force around the axis (axial direction) of the transmission cable 2 to the tip end via the transmission cable 2. The camera control unit 30 (an example of a calculation device) calculates a rotational force for rotating the tip end around the axis (axial direction) based on the difference between the three-directional attitude information detected by the acceleration sensor 17 (an example of an attitude control sensor) and a predetermined attitude state, and instructs the rotation drive device 60 to rotate the tip end around the axis (axial direction) based on the calculation result of the rotational force.

[0106] Therefore, the control unit applies a rotational force around a clockwise or counterclockwise axis to the base end of the transmission cable 2 by controlling the drive of the motor 65 of the rotation drive device 60. This rotation also rotates the camera head 11 (tip) of the endoscopic camera 10, and as a result, the image captured by the image sensor 16 also rotates, allowing the observation site in the subject to be imaged at a desired angle set in advance. As a result, the endoscope system 1 images the observation site in a constant posture based on the vertical direction, so that a user such as a doctor can properly and easily grasp in which direction the observation site in the subject is being imaged. As a result, it is possible to support the realization of appropriate medical procedures, suppress the occurrence of operational errors by users such as doctors, and improve medical safety for patients.

[0107] According to the endoscope system 1 of the second embodiment, the camera control unit 30 (an example of a calculation device) further calculates the direction of rotation of the tip in the axial direction based on the difference (angle error Δθn). Therefore, the control unit can appropriately rotate the camera head 11 (tip) of the endoscopic camera 10 around the axis in the forward direction (e.g., clockwise) or the reverse direction (e.g., counterclockwise) depending on the situation by controlling the drive of the output of the motor 65 of the rotation drive device 60.

[0108] Furthermore, according to the endoscope system 1 of the second embodiment, the camera control unit 30 (an example of a computing device) repeatedly executes the calculation of the rotational force and the instruction to the rotation drive device 60 (an example of a drive device). Therefore, the camera control unit 30 repeatedly executes the above-mentioned calculation of the rotational force and the instruction to the rotation drive device 60, and can sequentially rotate the tip portion (camera head unit 11) around the axis at a desired rotation angle while the endoscopic camera 10 is inserted into the subject. This improves the convenience of users such as doctors and can support the realization of more appropriate medical procedures.

[0109] According to the endoscope system 1 of the second embodiment, the difference is a value within a predetermined range centered on zero. In other words, a dead zone is set to prevent a slight rotational force from being applied to the tip (camera head 11) of the endoscope camera 10. As a result, slight movements in the camera head 11 are suppressed, and it is possible to suppress, for example, screen sickness of a user such as a doctor, or deterioration of the endoscope camera 10 over time. The other configurations and effects are the same as those of the first embodiment described above.

[0110] Although several embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure. In addition, the components in the above-mentioned embodiments may be arbitrarily combined within the scope of the invention.

[0111] In the second embodiment, the rotary drive device 60 including the motor 65 has been described as an example of a drive device that applies a rotational force around the axis of the transmission cable 2 to the tip portion (see FIG. 10), but the present invention is not limited to this. Instead of correcting the deviation of the tip portion of the endoscopic camera 10 by rotating the cable on the base end side using such a motor 65, it is also possible to arrange a plurality of artificial muscles (e.g., conductive polymer actuators) at the tip portion of the endoscopic camera 10 and correct the deviation of the tip portion of the endoscopic camera 10 by applying a voltage controlled from the base end side. [Industrial Applicability]

[0112] INDUSTRIAL APPLICABILITY The present disclosure is useful as an endoscope system that can efficiently detect whether or not there is a misalignment in the top-bottom direction of the tip portion of an endoscope and assist in carrying out appropriate medical procedures. [Explanation of symbols]

[0113] 1 Endoscope system 2 Transmission cable 3. Auto pullback device 4 Repeater 5 Monitor 10 Endoscope Camera 11 Camera head 12 Holder 13 Lens cover 14 Imaging unit 15 Lenses 16 Image sensor 17 Acceleration Sensor 30 Camera Control Unit 31 Processors 32 Memory 33 Input / Output Interface 34 Control section 35 Storage 40 Multiplex signal demodulation circuit 41 Signal multiplexing transmission circuit 42 Camera sync signal generator circuit 43 Pre-processing section 44 Signal Amplification Processing Section 45 YC separation processing / RGB conversion section 46 Correction processing section 47 First gain level adjustment section 48 Second gain level adjustment section 49 Area detection processing section 50 Area Generation Unit 51 Fog feature detection data addition, averaging and storage unit 52 Encoder processing unit 53 Area display signal replacement mixing and selection section 54 Indicator Generation Unit 55 Control section 56 Data Storage Section 57 External Switch 58 Camera setting change button 59 External Communications Department 60 Rotational drive unit 61 Cable holder 62 Insertion hole 63 Movable shaft holder 64 Movable roller 65 Motor 66 Output roller 67 Passive shaft holder 68 Passive roller 69 Slider section 70 Pressurizing section

Claims

1. an endoscope that can be inserted into and removed from a subject and has an imaging unit at a tip thereof that includes an optical system and an image sensor; a posture control sensor disposed at the tip portion and configured to detect posture information of the tip portion in at least three directions, i.e., a forward / backward direction, a vertical direction, and a horizontal direction; a calculation device that calculates whether or not there is a misalignment in a top-bottom direction of the tip portion in an image captured by the imaging unit based on the posture information in the three directions detected by the posture control sensor, the arithmetic device stores in a memory as calibration data information on the three directions of the attitude of the tip portion detected by the attitude control sensor in an initial state before the endoscope is inserted into the subject, corrects the three directions of the attitude information of the tip portion detected by the attitude control sensor after the endoscope is inserted into the subject using the calibration data, calculates whether or not there is a misalignment in the top-bottom direction between the tip portion before being inserted into the subject and the tip portion at the time of capturing the captured image, and displays on a monitor an indicator indicating the attitude of the tip portion at the time of capturing the captured image in association with the captured image. Endoscopy system.

2. the attitude control sensor is disposed so that a relative position between the attitude control sensor and the image sensor is fixed; The endoscope system according to claim 1 .

3. The imaging unit and the attitude control sensor are fixed by a sealant filled in the tip portion. The endoscope system according to claim 1 .

4. a drive device that fixes a base end side of a transmission cable connecting a base end portion to the tip end portion of the endoscope and applies a rotational force in an axial direction of the transmission cable to the tip end portion via the transmission cable, the computing device calculates a rotational force for rotating the tip portion in the axial direction based on a difference between the three-directional attitude information detected by the attitude control sensor and a predetermined attitude state, and instructs the drive device to rotate the tip portion in the axial direction based on a result of the calculation of the rotational force. The endoscope system according to claim 1 .

5. The calculation device further calculates a direction of rotation of the tip portion in the axial direction based on the difference. The endoscope system according to claim 4 .

6. the computing device repeatedly executes the calculation of the rotational force and the instruction to the driving device; The endoscope system according to claim 4 .

7. When the difference is within a predetermined range centered on zero, the arithmetic device does not calculate a control value for applying the rotational force, and maintains the current output of the control value to the drive device. The endoscope system according to claim 4 .

Images