Image processing method, control device, and endoscope system

Abstract

An image processing method includes an image acquisition step and an image processing step. In the image acquisition step, a processor acquires a first image and a second image having parallax from each other. The first image and the second image respectively include a first region that represents at least a portion of an observation target and a second region that represents at least a portion of an instrument. In the image processing step, the processor performs image processing on a processing region of at least one of the first image and the second image that includes the second region, and changes an amount of parallax of the processing region.

Description

TECHNICAL FIELD

[0001] The present application relates to an image processing method and an image processing apparatus. BACKGROUND

[0002] Endoscopes are widely used in the medical field and the industrial field. An endoscope used in the medical field is inserted into a body, and images of various sites in the body are taken. By using the images, diagnosis of an observation object and treatment of the observation object are performed. An endoscope used in the industrial field is inserted into an industrial product, and images of various sites in the industrial product are taken. By using the images, inspection of an observation object and treatment of the observation object (removal of a foreign object, etc.) are performed.

[0003] An endoscope apparatus having an endoscope and displaying a stereoscopic image (3D image) is being developed. The endoscope takes a plurality of images based on a plurality of optical images having parallaxes with respect to each other. A monitor of the endoscope apparatus displays a stereoscopic image based on the plurality of images. An observer can obtain information of a depth direction from the stereoscopic image. Therefore, an operator can easily perform treatment on a lesion using a treatment instrument. This advantage is obtained in fields other than fields using an endoscope. This advantage is common in fields in which an observer observes an image while performing treatment using an instrument. For example, this advantage is obtained even in a case where an image taken by a microscope is used.

[0004] An instrument is mostly located between an observation object and an observation optical system. That is, in a stereoscopic image, the instrument is mostly located in front of the observation object. In particular, the stereoscopic image is displayed so that a root of the instrument protrudes on the observer side. Therefore, a convergence angle becomes large, and the observer's eyes easily tire. The convergence angle is an angle formed by two center axes of lines of sight of the left eye and the right eye when the center axes cross each other.

[0005] A technique for displaying a stereoscopic image that an observer easily observes is disclosed in Patent Literature 1. The endoscope apparatus disclosed in Patent Literature 1 processes an image of a region of a subject that appears close to a position of an optical system of an endoscope so that the region is not visible from the image. When a stereoscopic image is displayed, a subject appearing in the region that is not visible is not displayed.

[0006] PRIOR ART DOCUMENT

[0007] PATENT LITERATURE

[0008] Patent Literature 1: Japanese Patent Application Laid-Open No. 2004-187711 SUMMARY

[0009] PROBLEMS TO BE SOLVED BY THE INVENTION

[0010] In the technology disclosed in Patent Literature 1, a part of an image becomes completely invisible, and thus an observer does not easily use an instrument. For example, there is a case where an observer uses a treatment instrument while taking an image through an endoscope. By using the technology disclosed in Patent Literature 1, a root of the treatment instrument becomes invisible, and thus an observer does not easily determine a traveling direction of the treatment instrument.

[0011] An object of the present application is to provide an image processing method and an image processing apparatus capable of reducing fatigue of an observer's eyes caused by an image of an instrument without impairing ease of use of the instrument.

[0012] Means for solving the problem

[0013] According to a first aspect of the present application, an image processing method has an image taking step and an image processing step. In the image taking step, a processor takes a first image and a second image from a first apparatus that outputs the first image and the second image having parallax from each other. The first image and the second image are images of an observation object and an instrument that performs a treatment on the observation object. The first image and the second image respectively include a first region and a second region. The first region of the first image includes a center of the first image. The first region of the second image includes a center of the second image. At least a part of the observation object is projected in the first region. The second region of the first image includes at least one end portion of the first image. The second region of the second image includes at least one end portion of the second image. At least a part of the instrument is projected in the second region. In the image processing step, the processor performs image processing on a processing region of at least one of the first image and the second image that includes the second region, and changes an amount of parallax of the processing region.

[0014] According to a second aspect of the present application, in the first aspect, in the image processing step, the processor changes the amount of parallax of the processing region in such a manner that a distance between a viewpoint and an optical image of the instrument in a stereoscopic image displayed on the basis of the first image and the second image becomes larger.

[0015] According to a third aspect of the present application, in the first aspect, in a stereoscopic image displayed on the basis of the first image and the second image taken in the image taking step, a majority of the observation object is located on an inside of a cross point.

[0016] According to a fourth aspect of the present application, in the first aspect, a shape of the first region is any one of a circle, an ellipse, and a polygon.

[0017] According to a fifth aspect of the present application, in the first aspect, in the image processing step, the processor can change the parallax amount so that the optical image of the processing region becomes planar.

[0018] According to a sixth aspect of the present application, in the first aspect, the processing region can include two or more pixels. In the image processing step, the processor can change the parallax amount so that two or more points of the optical image corresponding to the two or more pixels move in a direction away from the viewpoint. The distances moved by the two or more points can be equal to each other.

[0019] According to a seventh aspect of the present application, in the first aspect, the processing region can include two or more pixels. In the image processing step, the processor can change the parallax amount so that two or more points of the optical image corresponding to the two or more pixels move in a direction away from the viewpoint. The greater the distance between each of the two or more pixels and the first region, the greater the distance moved by each of the two or more points.

[0020] According to an eighth aspect of the present application, in the first aspect, the processing region can include two or more pixels. In the image processing step, the processor can change the parallax amount so that the distance between the viewpoint and each of two or more points of the optical image corresponding to the two or more pixels is equal to or greater than a predetermined value.

[0021] According to a ninth aspect of the present application, in the first aspect, the image processing method can further include a region setting step. Before the image processing step, in the region setting step, the processor can set the processing region based on at least one of the kind of an image generation device having an image pickup element that generates the first image and the second image, the kind of the instrument, and an image pickup magnification.

[0022] According to a tenth aspect of the present application, in the first aspect, the image processing method can further include an instrument detection step and a region setting step. Before the image processing step, in the instrument detection step, the processor can detect the instrument from at least one of the first image and the second image. Before the image processing step, in the region setting step, the processor can set a region in which the instrument is detected as the processing region.

[0023] According to a 11th aspect of the present application, in the 1st aspect, the image processing method can further include a region setting step. In the region setting step, before the image processing step, the processor can determine the position of the 1st region based on at least one of a kind of an image generation device having an image pickup element that generates the 1st image and the 2nd image, a kind of the apparatus, and an image pickup magnification, and set a region excluding the 1st region as the processing region.

[0024] According to a 12th aspect of the present application, in the 1st aspect, the image processing method can further include an observation object detecting step and a region setting step. In the observation object detecting step, before the image processing step, the processor can detect the observation object from at least one of the 1st image and the 2nd image. In the region setting step, before the image processing step, the processor can regard a region in which the observation object is detected as the 1st region, and set a region excluding the 1st region as the processing region.

[0025] According to a 13th aspect of the present application, in the 1st aspect, the image processing method can further include a region setting step. In the region setting step, before the image processing step, the processor can determine the position of the 1st region based on information input by an observer to an input device, and set a region excluding the 1st region as the processing region.

[0026] According to a 14th aspect of the present application, in the 1st aspect, the image processing method can further include a 1st image output step. In the 1st image output step, the processor can output the 1st image and the 2nd image including an image in which the parallax amount of the processing region is changed to one of a display device that displays a stereoscopic image based on the 1st image and the 2nd image and a communication device that outputs the 1st image and the 2nd image to the display device.

[0027] According to a 15th aspect of the present application, in the 14th aspect, the image processing method can further include a mode selection step and a 2nd image output step. In the mode selection step, the processor can select one of the 1st mode and the 2nd mode. In the 2nd image output step, the processor can output the 1st image and the 2nd image obtained in the image obtaining step to one of the display device and the communication device. In a case where the processor selects the 1st mode in the mode selection step, the processor can execute the image processing step and the 1st image output step. In a case where the processor selects the 2nd mode in the mode selection step, the processor can execute the 2nd image output step without executing the image processing step.

[0028] According to a 16th aspect of the present application, in the 15th aspect, in the mode selection step, the processor can select one of the 1st mode and the 2nd mode based on information input by an observer to an input device.

[0029] According to a 17th aspect of the present application, in the 15th aspect, the image processing method can further include a 1st motion detection step. In the 1st motion detection step, the processor can detect a state of motion of an image pickup element that generates the 1st image and the 2nd image. In the mode selection step, the processor can select one of the 1st mode and the 2nd mode based on the state.

[0030] According to an 18th aspect of the present application, in the 15th aspect, the image processing method can further include a search step. In the search step, the processor can search for the implement in at least one of the 1st image and the 2nd image. In a case where the processor detects the implement from at least one of the 1st image and the 2nd image in the search step, the processor can select the 1st mode in the mode selection step. In a case where the processor fails to detect the implement from at least one of the 1st image and the 2nd image in the search step, the processor can select the 2nd mode in the mode selection step.

[0031] According to a 19th aspect of the present application, in the 15th aspect, the image processing method can further include a 2nd motion detection step. In the 2nd motion detection step, the processor can detect a state of motion of the implement. In the mode selection step, the processor can select one of the 1st mode and the 2nd mode based on the state.

[0032] According to a 20th aspect of the present application, an image processing apparatus has a processor. The processor acquires a first image and a second image from a first apparatus that outputs the first image and the second image having parallaxes with each other. The first image and the second image are images of an observation object and an instrument that performs a treatment on the observation object. The first image and the second image include a first region and a second region, respectively. The first region of the first image includes a center of the first image. The first region of the second image includes a center of the second image. At least a part of the observation object is projected in the first region. The second region of the first image includes at least one end portion of the first image. The second region of the second image includes at least one end portion of the second image. At least a part of the instrument is projected in the second region. The processor performs image processing on a processing region of at least one of the first image and the second image that includes the second region, and changes an amount of parallax of the processing region.

[0033] Effects of the Invention

[0034] According to the above-described aspects, the image processing method and the image processing apparatus can reduce fatigue of an observer's eyes caused by an image of an instrument without impairing ease of use of the instrument. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 FIG. 1 is a diagram showing a structure of an endoscope apparatus having an image processing apparatus of a first embodiment of the present application.

[0036] Figure 2 FIG. 2 is a diagram showing a structure of a distal end portion of an endoscope apparatus of the first embodiment of the present application.

[0037] Figure 3 FIG. 3 is a block diagram showing a structure of the image processing apparatus of the first embodiment of the present application.

[0038] Figure 4 FIG. 4 is a diagram showing another connection example of the image processing apparatus of the first embodiment of the present application and a monitor.

[0039] Figure 5 FIG. 5 is a diagram showing an image acquired by the endoscope apparatus of the first embodiment of the present application.

[0040] Figure 6 FIG. 6 is a diagram showing an image acquired by the endoscope apparatus of the first embodiment of the present application.

[0041] Figure 7 FIG. 7 is a diagram showing a position of an optical image of a subject in a stereoscopic image displayed in the first embodiment of the present application.

[0042] Figure 8 is a flowchart showing steps of a process performed by a processor of an image processing apparatus of Embodiment 1 of the present application.

[0043] Figure 9 is a diagram showing a position of an optical image of a subject in a stereoscopic image displayed in Embodiment 1 of the present application.

[0044] Figure 10 is a diagram showing a position of an optical image of a subject in a stereoscopic image displayed in a first modified example of Embodiment 1 of the present application.

[0045] Figure 11 is a diagram showing a position of an optical image of a subject in a stereoscopic image displayed in a second modified example of Embodiment 1 of the present application.

[0046] Figure 12 is a diagram showing a position of an optical image of a subject in a stereoscopic image displayed in a third modified example of Embodiment 1 of the present application.

[0047] Figure 13 is a diagram showing region information in a fourth modified example of Embodiment 1 of the present application.

[0048] Figure 14 is a diagram showing an image in the fourth modified example of Embodiment 1 of the present application.

[0049] Figure 15 is a flowchart showing steps of a process performed by a processor of an image processing apparatus of Embodiment 2 of the present application.

[0050] Figure 16 is a diagram showing a position of an optical image of a subject in a stereoscopic image displayed in Embodiment 2 of the present application.

[0051] Figure 17 is a graph showing disparity information in a first modified example of Embodiment 2 of the present application.

[0052] Figure 18 is a flowchart showing steps of a process performed by a processor of an image processing apparatus of Embodiment 3 of the present application.

[0053] Figure 19 is a flowchart showing steps of a process performed by a processor of an image processing apparatus of Embodiment 4 of the present application.

[0054] Figure 20 is a diagram showing region information in Embodiment 4 of the present application.

[0055] Figure 21is a diagram showing region information in a modification of the fourth embodiment of the present application.

[0056] Figure 22 is a flowchart showing steps of processing executed by a processor of an image processing apparatus according to the fifth embodiment of the present application.

[0057] Figure 23 is a diagram showing region information in a modification of the sixth embodiment of the present application.

[0058] Figure 24 is a flowchart showing steps of processing executed by a processor of an image processing apparatus according to the seventh embodiment of the present application.

[0059] Figure 25 is a flowchart showing steps of processing executed by a processor of an image processing apparatus according to a first modification of the seventh embodiment of the present application.

[0060] Figure 26 is a flowchart showing steps of processing executed by a processor of an image processing apparatus according to a second modification of the seventh embodiment of the present application.

[0061] Figure 27 is a flowchart showing steps of processing executed by a processor of an image processing apparatus according to a third modification of the seventh embodiment of the present application.

[0062] Figure 28 is a flowchart showing steps of processing executed by a processor of an image processing apparatus according to a fourth modification of the seventh embodiment of the present application.

[0063] Figure 29 is a block diagram showing a structure of a periphery of an image processing apparatus according to a fifth modification of the seventh embodiment of the present application.

[0064] Figure 30 is a flowchart showing steps of processing executed by a processor of an image processing apparatus according to the fifth modification of the seventh embodiment of the present application.

[0065] Figure 31 is a flowchart showing steps of processing executed by a processor of an image processing apparatus according to a sixth modification of the seventh embodiment of the present application.

[0066] Figure 32 is a flowchart showing steps of processing executed by a processor of an image processing apparatus according to the eighth embodiment of the present application.

[0067] Figure 33 is a flowchart showing steps of processing executed by a processor of an image processing apparatus according to a modification of the eighth embodiment of the present application. DETAILED DESCRIPTION

[0068] Embodiments of the present application will be described with reference to the accompanying drawings. An example of an endoscope device having an image processing apparatus will be described below. The endoscope included in the endoscope device can be either of a medical endoscope and an industrial endoscope. Embodiments of the present application are not limited to the endoscope device. Embodiments of the present application can also be a microscope or the like. In a case where an observer performs a treatment on an observation target using an instrument while observing a stereoscopic image, the image processing method and the image processing apparatus of each aspect of the present application can be used. The observer is a doctor, a technician, a researcher, or an apparatus manager, or the like.

[0069] (First Embodiment)

[0070] Figure 1 The structure of an endoscope device 1 of a first embodiment of the present application will be described. Figure 1 The illustrated endoscope device 1 has an electronic endoscope 2, a light source device 3, an image processing apparatus 4, and a monitor 5.

[0071] The electronic endoscope 2 has an imaging element 12 Figure 2 ) that acquires an image of a subject. The light source device 3 has a light source that supplies illumination light to the electronic endoscope 2. The image processing apparatus 4 processes an image acquired by the imaging element 12 of the electronic endoscope 2 to generate a video signal. The monitor 5 displays an image based on the video signal output from the image processing apparatus 4.

[0072] The electronic endoscope 2 has a distal end portion 10, an insertion portion 21, an operation portion 22, and a general-purpose cable 23. The insertion portion 21 is elongated and has flexibility. The distal end portion 10 is disposed at the distal end of the insertion portion 21. The distal end portion 10 is hard. The operation portion 22 is disposed at the proximal end of the insertion portion 21. The general-purpose cable 23 extends from the side of the operation portion 22. A connector portion 24 is disposed at the end of the general-purpose cable 23. The connector portion 24 is configured to be attachable to and detachable from the light source device 3. A connection line 25 extends from the connector portion 24. An electric connector portion 26 is disposed at the end of the connection line 25. The electric connector portion 26 is configured to be attachable to and detachable from the image processing apparatus 4.

[0073] Figure 2 The outline structure of the distal end portion 10 will be described. The endoscope device 1 has a first optical system 11L, a second optical system 11R, the imaging element 12, and a treatment instrument 13. The first optical system 11L, the second optical system 11R, and the imaging element 12 are disposed inside the distal end portion 10.

[0074] The first optical system 11L corresponds to the left eye. The second optical system 11R corresponds to the right eye. The optical axis of the first optical system 11L and the optical axis of the second optical system 11R are apart from each other by a prescribed distance. Thus, the first optical system 11L and the second optical system 11R have parallax with each other. The first optical system 11L and the second optical system 11R each have an optical member such as an objective lens. The imaging element 12 is an image sensor.

[0075] The first optical system 11L and the second optical system 11R are formed with windows for taking in light from an object on the end surface of the distal end portion 10. In the case where the electronic endoscope 2 is a binocular type endoscope, two windows are formed on the end surface of the distal end portion 10. One of the two windows is formed in front of the first optical system 11L, and the other of the two windows is formed in front of the second optical system 11R. In the case where the electronic endoscope 2 is a monocular type endoscope, one window is formed in front of the first optical system 11L and the second optical system 11R on the end surface of the distal end portion 10.

[0076] The treatment instrument 13 is inserted into the inside of the insertion portion 21. The treatment instrument 13 is a laser fiber or a forceps or the like. A space (passage) for passing the treatment instrument 13 is formed in the inside of the insertion portion 21. The treatment instrument 13 protrudes forward from the end surface of the distal end portion 10. The treatment instrument 13 can advance toward the front or can retreat toward the rear. Two or more passages can be formed in the insertion portion 21, and two or more treatment instruments can be inserted into the insertion portion 21.

[0077] The illumination light generated by the light source device 3 is irradiated to an object. The light reflected by the object is incident on the first optical system 11L and the second optical system 11R. The light that has passed through the first optical system 11L forms a first optical image of the object on the imaging surface of the imaging element 12. The light that has passed through the second optical system 11R forms a second optical image of the object on the imaging surface of the imaging element 12.

[0078] The imaging element 12 generates a first image based on the first optical image and generates a second image based on the second optical image. The first optical image and the second optical image are formed on the imaging surface of the imaging element 12 at the same time, and the imaging element 12 generates an image (imaging signal) that contains the first image and the second image. The first image and the second image are images of the observation object and the instrument. The first image and the second image have parallax with each other. The imaging element 12 continuously performs imaging and generates a moving image. The moving image includes the first image and the second image of two or more frames. The imaging element 12 outputs the generated image.

[0079] The first optical image and the second optical image can also be formed in turn on the imaging surface of the imaging element 12. For example, the tip portion 10 has a baffle that blocks light passing through one of the first optical system 11L and the second optical system 11R. The baffle is movable between a first position and a second position. When the baffle is disposed at the first position, the baffle blocks light passing through the second optical system 11R. At this time, the first optical image is formed on the imaging surface of the imaging element 12, and the imaging element 12 generates the first image. When the baffle is disposed at the second position, the baffle blocks light passing through the first optical system 11L. At this time, the second optical image is formed on the imaging surface of the imaging element 12, and the imaging element 12 generates the second image. The imaging element 12 outputs the first image and the second image in turn.

[0080] In the example described above, the first optical image is formed by light passing through the first optical system 11L. The first image is generated based on the first optical image. Further, in the example described above, the second optical image is formed by light passing through the second optical system 11R. The second image is generated based on the second optical image. It is also possible to generate the first image based on the second optical image, and to generate the second image based on the first optical image.

[0081] The image output from the imaging element 12 is transmitted to the image processing device 4. In Figure 2 The insertion portion 21, the operation portion 22, the general-purpose cable 23, the connector portion 24, the connection line 25, and the electric connector portion 26 other than the tip portion 10 are omitted in FIG. 1. The image processing device 4 processes the first image and the second image included in the image output from the imaging element 12. The image processing device 4 outputs the processed first image and second image as video signals to the monitor 5.

[0082] The monitor 5 is a display device that displays a stereoscopic image (three-dimensional image) based on the first image and the second image. For example, the monitor 5 is a flat panel display such as a liquid crystal display (LCD), an organic EL display (OLED), or a plasma display. The monitor 5 can also be a projector that projects an image onto a screen. As a method of displaying a stereoscopic image, a circularly polarized light method or an active shutter can be used. In these methods, special-purpose glasses are used. In the circularly polarized light method, lightweight special-purpose glasses that do not require synchronization can be used.

[0083] Figure 3 The structure of the image processing device 4 is shown. Figure 3 The image processing device 4 shown has a processor 41 and a ROM (Read Only Memory) 42.

[0084] The processor 41 is, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit), or the like. The processor 41 can also be configured by an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array), or the like. The image processing apparatus 4 can include one or a plurality of processors 41.

[0085] The first image and the second image are output from the imaging element 12 and input to the processor 41. The processor 41 acquires the first image and the second image from the imaging element 12 (the first device) in an image acquisition step. The first image and the second image output from the imaging element 12 can also be stored in a storage device not shown in the figure. The processor 41 can also acquire the first image and the second image from the storage device. The processor 41 processes at least one of the first image and the second image in an image processing step in order to adjust the position of the optical image of the display tool in the stereoscopic image. The image processing performed by the processor 41 will be described in detail later. The processor 41 outputs the processed first image and the second image to the monitor 5 in a first image output step. Figure 3

[0086] The operation section 22 is an input device having a component operated by an observer (operator). The component is, for example, a button or a switch, or the like. The observer can input various kinds of information for controlling the endoscope apparatus 1 by operating the operation section 22. The operation section 22 outputs information input to the operation section 22 to the processor 41. The processor 41 controls the imaging element 12, the light source apparatus 3, the monitor 5, and the like based on the information input to the operation section 22.

[0087] The ROM 42 holds a program including a command for specifying the action of the processor 41. The processor 41 reads the program from the ROM 42 and executes the read program. The function of the processor 41 can be realized by software. The above-described program can also be provided by a "computer-readable recording medium" such as a flash memory, for example. The program can also be transmitted to the endoscope apparatus 1 from a computer holding the program via a transmission medium or by a transmission wave in the transmission medium. The "transmission medium" that transmits the program is a medium having a function of transmitting information. The medium having the function of transmitting information includes a network (communication network) such as the Internet and a communication line (communication line) such as a telephone line. The above-described program can also realize a part of the above-described function. Furthermore, the above-described program can be a differential file (differential program). The combination of the program already recorded in the computer and the differential program can also realize the above-described function.​

[0088] In Figure 2 and Figure 3 the example shown, the image pickup element 12 and the image processing apparatus 4 are connected by a signal line through the insertion portion 21 or the like. The image pickup element 12 and the image processing apparatus 4 can also be connected in a wireless manner. That is, the image pickup element 12 can also have a transmitter that transmits the first image and the second image in a wireless manner, and the image processing apparatus 4 can also have a receiver that receives the first image and the second image in a wireless manner. The communication between the image pickup element 12 and the image processing apparatus 4 can also be performed via a network such as a LAN (Local Area Network) or the like. The communication can also be performed via a device on the cloud.

[0089] In Figure 1 and Figure 3 the example shown, the image processing apparatus 4 and the monitor 5 are connected by a signal line. The image processing apparatus 4 and the monitor 5 can also be connected in a wireless manner. That is, the image processing apparatus 4 can also have a transmitter that transmits the first image and the second image in a wireless manner, and the monitor 5 can also have a receiver that receives the first image and the second image in a wireless manner. The communication between the image processing apparatus 4 and the monitor 5 can also be performed via a network such as a LAN or the like.

[0090] In Figure 3 the example shown, the processor 41 outputs the first image and the second image to the monitor 5 (display apparatus). The processor 41 can also not output the first image and the second image directly to the monitor 5. Figure 4 Another connection example of the image processing apparatus 4 and the monitor 5 is shown. The processor 41 outputs the first image and the second image to the reception apparatus 6 (communication apparatus). The reception apparatus 6 receives the first image and the second image output from the image processing apparatus 4. The reception apparatus 6 outputs the received first image and second image to the monitor 5. The image processing apparatus 4 and the reception apparatus 6 can be connected by a signal line, or can also be connected in a wireless manner. The reception apparatus 6 and the monitor 5 can be connected by a signal line, or can also be connected in a wireless manner. The reception apparatus 6 can also be replaced by a storage apparatus such as a hard disk drive or a flash memory or the like.

[0091] Referring to Figure 5 The first image and the second image are described. The two images have parallax with respect to each other, but the composition of the two images does not differ greatly. Figure 5 An example of the first image is shown. The matters described below can also be applied to the second image.

[0092] Figure 5The first image 200 shown is an image of an observation target 210 and a treatment instrument 13. The observation target 210 is a region (region of interest) that an observer is interested in. For example, the observation target 210 is a diseased part of a site (organ or blood vessel, etc.) in a living body. For example, the diseased part is a tumor such as cancer. The diseased part is sometimes referred to as a lesion. The periphery of the observation target 210 is a part of the site (subject). The treatment instrument 13 is displayed on the subject. The treatment instrument 13 performs a treatment on the observation target 210. The treatment instrument 13 has forceps 130 and a sheath 131. The forceps 130 are in contact with the observation target 210 and perform a treatment on the observation target 210. The sheath 131 is a support portion that supports the forceps 130. The forceps 130 are fixed to the sheath 131. The treatment instrument 13 can have a snare or an IT knife, etc. in addition to the forceps 130.

[0093] The first image 200 includes a first region R10 and a second region R11. A broken line L10 represents a boundary between the first region R10 and the second region R11. The first region R10 is a region inside the broken line L10, and the second region R11 is a region outside the broken line L10. The first region R10 includes a center C10 of the first image 200. The observation target 210 is reflected in the first region R10. The second region R11 includes at least one end portion of the first image 200. In Figure 5 In the example shown, the second region R11 includes four end portions of the first image 200. The treatment instrument 13 is reflected in the second region R11. The treatment instrument 13 is reflected in a region including an end portion on a lower side of the first image 200.

[0094] A part of the treatment instrument 13 can also be reflected in the first region R10. In Figure 5 In the example shown, a front end portion (forceps 130) of the treatment instrument 13 is reflected in the first region R10, and a portion on a root side (sheath 131) of the treatment instrument 13 is reflected in the second region R11. The forceps 130 are located in front of the observation target 210 and block a part of the observation target 210. The root of the treatment instrument 13 in the first image 200 is a portion of the sheath 131 reflected in an end portion on a lower side of the first image 200. A part of the observation target 210 can also be reflected in the second region R11. That is, a part of the observation target 210 can be reflected in the first region R10, and a remaining part of the observation target 210 can be reflected in the second region R11.

[0095] The second image includes a first region and a second region similarly to the first image 200. The first region of the second image includes a center of the second image. An observation target is reflected in the first region of the second image. The second region of the second image includes at least one end portion of the second image. The treatment instrument 13 is reflected in the second region of the second image.

[0096] Region 1 and Region 2 are defined to distinguish between the region reflecting the object of observation and the region reflecting the treatment device 13. This is not limited to... Figure 5 The first and second regions are clearly defined by a line with a fixed shape, such as the dashed line L10 shown.

[0097] The first and second images may also include a third region that is different from both the first and second regions. Alternatively, a subject different from the observed object may be reflected in the third region. A portion of the observed object or the processing device 13 may also be reflected in the third region. The third region may also be the area between the first and second regions. The third region may also include an end that is different from the end of the image reflecting the processing device 13. The third region may also include a portion of the end of the image reflecting the processing device 13.

[0098] The treatment device 13 is inserted into the body via the insertion part 21. Treatment devices other than the treatment device 13 may also be inserted into the body without going through the insertion part 21 into which the treatment device 13 is inserted. Figure 6 Other examples are shown in the first image. Figure 6 The first image 201 shown is an image of the observed object 210, the treatment device 14, and the treatment device 15. The treatment devices 14 and 15 are inserted into the body without passing through the insertion portion 21. For example, the endoscope device 1 may have at least one of the treatment devices 14 and 15 in addition to the treatment device 13. It may also be combined with... Figure 1 The endoscopic apparatus 1 shown has at least one of a treatment device 14 and a treatment device 15. The types of treatments performed by the treatment device 14 and the treatment device 15 may also be different. The endoscopic apparatus 1 may also not have a treatment device 13.

[0099] exist Figure 5 In the example shown, one treatment device is displayed in the image. Figure 6 In the example shown, two treatment devices are displayed in the image. It is also possible that three or more treatment devices are displayed in the image. Alternatively, at least one of treatment devices 14 and 15, and treatment device 13, may be displayed in the image.

[0100] Reference Figure 7 The position of the optical image of the subject in the stereoscopic image is explained. Figure 7 This shows the position of the optical image of the subject as visually recognized by an observer when a stereoscopic image is displayed on monitor 5 based on the first and second images. Figure 7 In the example shown, it is assumed that processor 41 does not perform any changes to the parallax of the first and second images output from camera element 12. The changes to the parallax will be described later.

[0101] The viewpoint VL corresponds to the left eye of the observer. The viewpoint VR corresponds to the right eye of the observer. The observer captures the optical image of the subject through the viewpoint VL and the viewpoint VR. A point VC that is intermediate between the viewpoint VL and the viewpoint VR can also be defined as the viewpoint of the observer. In the following examples, the distance between the viewpoint of the observer and the optical image of the subject is defined as the distance between the point VC and the optical image of the subject.

[0102] The point at which the optical axis of the first optical system 11L and the optical axis of the second optical system 11R intersect is referred to as the intersection point. The intersection point is sometimes referred to as the convergence point or the zero point, or the like. In the region of the subject on the intersection point, the amount of disparity between the first image and the second image is 0. In the case of displaying a stereoscopic image, the position of the intersection point is set so that the observer easily observes the stereoscopic image. For example, as shown in FIG. 1, the intersection point CP is set on the screen face SC. The screen face SC is sometimes referred to as the display face, the monitor face, or the zero plane, or the like. The screen face SC corresponds to the face 5a of the monitor 5. Figure 7 Figure 1 As shown in FIG. 1, the intersection point CP is set on the screen face SC. The screen face SC is sometimes referred to as the display face, the monitor face, or the zero plane, or the like. The screen face SC corresponds to the face 5a of the monitor 5. Figure 7 As shown in FIG. 1, the intersection point CP is set on the screen face SC. The screen face SC is sometimes referred to as the display face, the monitor face, or the zero plane, or the like. The screen face SC corresponds to the face 5a of the monitor 5.

[0103] Figure 7 As shown in FIG. 1, the intersection point CP is set on the screen face SC. The screen face SC is sometimes referred to as the display face, the monitor face, or the zero plane, or the like. The screen face SC corresponds to the face 5a of the monitor 5.

[0104] The optical image of the object OB2 is located in the region R21 (the region on the front side of the screen face SC) on the front side of the intersection point CP. The optical image of the object OB2 is located between the viewpoint of the observer and the screen face SC. For example, the object OB2 is the portion on the root side of the treatment instrument 13. The distance between the viewpoint of the observer and the optical image of the object OB2 is D2. The distance D2 is smaller than the distance Dl. There are also cases in which the optical images of all of the objects are located in the region R20.

[0105] The region of the first image and the second image that images the object located on the rear side of the intersection point CP in the stereoscopic image is defined as having a positive amount of disparity. For example, the amount of disparity between the region that images the object OB1 in the first image and the region that images the object OB1 in the second image is a positive value. In the case in which the object OB1 is the observation object 210, the observation object 210 is located on the rear side of the intersection point CP in the stereoscopic image. The amount of disparity between the region that images the observation object 210 in the first image and the region that images the observation object 210 in the second image is a positive value.​​Figure 5 The amount of disparity between at least a part of the first region R10 of the first image 200 and at least a part of the first region of the second image is a positive value. The larger the distance D1 between the viewpoint of the observer and the optical image of the object OBl, the larger the absolute value of the amount of disparity becomes, and the further the optical image of the object OBl is from the viewpoint of the observer.

[0106] The regions of the first image and the second image that depict the object located on the near side of the intersection point CP in the stereoscopic image are defined as having a negative amount of disparity. For example, the amount of disparity between the region in the first image that depicts the object OB2 and the region in the second image that depicts the object OB2 is a negative value. In the case where the object OB2 is the portion on the root side of the treatment instrument 13, Figure 5 The amount of disparity between at least a part of the second region R11 of the first image 200 and at least a part of the second region of the second image is a negative value. The smaller the distance D2 between the viewpoint of the observer and the optical image of the object OB2, the larger the absolute value of the amount of disparity becomes, and the closer the optical image of the object OB2 is to the viewpoint of the observer. In the case where the optical image of the object OB2 is close to the viewpoint of the observer, the observer observes the object OB2 as greatly protruding. In this case, the convergence angle is large, and the eyes of the observer are easily fatigued.

[0107] The change of the amount of disparity performed by the processor 41 will be described. The processor 41 performs image processing on a processing region of at least one of the first image and the second image that includes the second region, changes the amount of disparity of the processing region so that the distance between the viewpoint of the observer and the optical image of the instrument becomes larger in the stereoscopic image displayed on the basis of the first image and the second image. This stereoscopic image is the stereoscopic image displayed on the basis of the first image and the second image after the processor 41 has performed the change of the amount of disparity. For example, the processor 41 sets the amount of disparity of the processing region to 0, or to a positive value. Figure 5 The processing region of the first image 200 that includes the second region R11 is shown, and the amount of disparity of the processing region is changed.

[0108] For example, before the processor 41 performs the change of the amount of disparity, the distance between the viewpoint of the observer and the optical image of the object OB2 is D2. The processor 41 performs image processing on at least one of the first image and the second image so that the amount of disparity of the processing region changes in the positive direction. In the case where the amount of disparity of the second region that depicts the treatment instrument 13 is negative, the processor 41 increases the amount of disparity of the processing region that includes the second region. The processor 41 can also change the amount of disparity of the processing region to 0, or can further change the amount of disparity of the processing region to a positive value. After the processor 41 has performed the change of the amount of disparity, the distance between the viewpoint of the observer and the optical image of the object OB2 is larger than D2. As a result, the convergence angle becomes smaller, and the fatigue of the eyes of the observer is reduced.

[0109] Reference is made to Figure 8 The processing performed by the processor 41 will be described. Figure 8 The steps of the processing performed by the processor 41 will be described.

[0110] The processor 41 sets a processing region including the second region (step S100). Step S100 will be described in detail. The overall sizes of the first image and the second image are known. Before step S100 is performed, region information indicating the position of the second region is stored in a memory not shown in the figure. The region information can also include information indicating at least one of the size and the shape of the second region. The processor 41 reads out the region information from the memory in step S100. The processor 41 determines the position of the second region based on the region information. The processor 41 sets a processing region including the second region. The processing region includes two or more pixels. For example, the processing region is the same as the second region, and the first region is not included in the processing region. The processor 41 can also set two or more processing regions. The processor 41 sets the processing region by holding information of the processing region. The processor 41 can also acquire the region information from a device different from the endoscope device 1. Figure 3

[0111] After step S105, the processor 41 acquires the first image and the second image from the imaging element 12 (step S105 (image acquisition step)). The order in which steps S105 and S100 are performed can also be different from that shown in the figure. That is, step S100 can also be performed after step S105 is performed. Figure 8

[0112] After step S100, the processor 41 changes the parallax amount by changing the image data of the processing region of at least one of the first image and the second image (step S110 (image processing step)). The processor 41 can also change the parallax amount of the processing region only in the first image. The processor 41 can also change the parallax amount of the processing region only in the second image. The processor 41 can also change the parallax amount of the processing region of each of the first image and the second image.

[0113] Step S110 will be described in detail. For example, the processor 41 changes the parallax amount of the processing region in such a manner that the optical image of the processing region becomes flat. Thereby, the processor 41 changes the parallax amount of the processing region in such a manner that the optical image of the treatment instrument 13 becomes flat. Specifically, the processor 41 substitutes the data of each pixel included in the processing region in the first image with the data of each pixel included in the second image corresponding to each pixel of the first image. Thus, the same pixels of the two images have the same data. The processor 41 can also substitute the data of each pixel included in the processing region in the second image with the data of each pixel included in the first image corresponding to each pixel of the second image. ​​

[0114] Figure 9 The position of the optical image of the subject visually recognized by the observer when the stereoscopic image is displayed on the monitor 5 based on the first image and the second image is shown. The explanation of the parts shown in common with the parts shown in FIG. 1 is omitted. Figure 7 The explanation of the parts shown in common with the parts shown in FIG. 1 is omitted.

[0115] Figure 9 The optical image of the treatment instrument 13 projected on the processing region is shown. The optical image of the treatment instrument 13 projected on the first region is omitted in Figure 9 Figure 9 An example in which the treatment instrument 13 is projected on the right side of the center of the image in the first image and the second image is shown.

[0116] Before the processor 41 changes the parallax amount of the processing region in the first image, the optical image 13a of the treatment instrument 13 projected on the processing region is displayed on the front side of the screen surface SC. After the processor 41 changes the parallax amount of the processing region in the first image, the parallax amount between the processing region and the region of the second image corresponding to the processing region is 0. The optical image 13b of the treatment instrument 13 projected on the processing region is displayed as a plane including the intersection point CP in the stereoscopic image. For example, the optical image 13b is displayed inside the screen surface SC. The optical image 13b is away from the viewpoint of the observer.

[0117] After the processor 41 changes the parallax amount of the processing region in the first image, a discontinuity of the parallax amount occurs at the boundary between the processing region and the region other than the processing region. That is, a discontinuity of the parallax amount occurs at the boundary between the first region and the second region. The processor 41 can also perform image processing that makes the change of the data of the periphery of the boundary smooth in order to eliminate the discontinuity. Thereby, the boundary becomes less conspicuous, and the image looks natural.

[0118] The processor 41 can also change the parallax amount of the processing region and change the parallax amount of the first region in at least one of the first image and the second image. The method of changing the parallax amount of the first region is different from the method of changing the parallax amount of the processing region. For example, the processor 41 can also change the parallax amount of the first region in such a way that the optical image of the observation object is away to the inside of the intersection point. In the case of changing the parallax amount of the first region, the amount of change of the parallax amount of the first region can also be smaller than the maximum value of the amount of change of the parallax amount of the processing region.

[0119] After the step S110, the processor 41 outputs the first image and the second image including the image in which the parallax amount of the processing region is changed to the monitor 5 (step S115 (first image output step)). For example, the processor 41 outputs the first image in which the parallax amount of the processing region is changed in the step S110 and the second image acquired in the step S105 to the monitor 5. ​

[0120] In step S105, step S110, and step S115, the image corresponding to 1 frame included in the dynamic image is processed. The processor 41 processes the dynamic image by repeating the execution of step S105, step S110, and step S115. After the processing region applied to the first frame is set, the processing region can be applied to the remaining one or more frames. In this case, step S100 is executed once, and step S105, step S110, and step S115 are executed two or more times.

[0121] The processor 41 sets the processing region based on the region information, and thus the position of the processing region is fixed. The processor 41 can easily set the processing region.

[0122] The region information can also indicate the position of the first region. The region information can include information indicating at least one of the size and the shape of the first region in addition to the information indicating the position of the first region. The processor 41 can determine the position of the first region based on the region information, and regard the region in the image other than the first region as the second region. In a case where the first region includes the entire observation object, the observation object is not affected by the change in the parallax amount of the processing region. Thus, the observer can easily perform a treatment on the observation object using the treatment instrument 13.

[0123] In Figure 5 In the example illustrated in FIG. 10, the shape of the first region R10 is a circle. In a case where the shapes of the first image and the second image are circles and the shape of the first region is a circle, the observer does not easily feel discomfort with the images. The shape of the first region can also be an ellipse or a polygon. The polygon has four or more vertices. The shape of the first region can also be a polygon having eight or more vertices.

[0124] In the first embodiment, the processor 41 changes the parallax amount of the processing region including the second region in a manner that the distance between the viewpoint of the observer in the stereoscopic image and the optical image of the instrument becomes larger. Thus, the image processing apparatus 4 can reduce the fatigue of the observer's eyes caused by the image of the instrument without impairing the ease of use of the instrument.

[0125] (First Modification of the First Embodiment)

[0126] A first modification of the first embodiment of the present application will be described. Another method of changing the parallax amount in a manner that the optical image of the treatment instrument 13 becomes flat will be described.

[0127] The processor 41 shifts the position of the data of each pixel included in the processing region in the first image in a prescribed direction in step S110. Thus, the processor 41 changes the parallax amount of the processing region. The prescribed direction is a direction parallel to the horizontal direction of the image. The prescribed direction is a direction in which the negative parallax amount changes to the positive direction. In a case where the first image corresponds to the optical image captured by the first optical system 11L, the prescribed direction is the left direction. In a case where the first image corresponds to the optical image captured by the second optical system 11R, the prescribed direction is the right direction.

[0128] The processor 41 shifts the position of the data of each pixel included in the processing region in a manner such that the optical image of the subject in each pixel moves to a position apart from the screen surface by a distance Al in step S110. The processor 41 changes the parallax amount of each pixel included in the processing region by B1 by performing this processing. The processor 41 can calculate the change amount B1 of the parallax amount based on the distance Al.

[0129] The method of shifting the position of the data of each pixel is described. The processor 41 replaces the data of each pixel included in the processing region with the data of a pixel apart from the prescribed direction by a distance C1. The distance C1 can be the same as the change amount B1 of the parallax amount, or the distance C1 can be calculated based on the change amount B1 of the parallax amount. In a case where a position apart from the pixel of the first image in the direction opposite to the prescribed direction by the distance C1 is not included in the first image, the processor 41 interpolates the data of the pixel. For example, in a case where a position apart from the pixel of the first image in the right direction by the distance C1 is not included in the first image, the processor 41 interpolates the data using the data of the pixel of the second image corresponding to the position. In a case where a position apart from the pixel of the first image in the prescribed direction by the distance C1 is not included in the first image, the processor 41 does not generate the data of the position. The processor 41 can also shift the position of the data of each pixel included in the processing region in the second image in the prescribed direction.

[0130] Figure 10 The position of the optical image of the subject visually recognized by the observer when the stereoscopic image is displayed on the monitor 5 based on the first image and the second image is shown. The description of the parts common to the parts shown in Figure 7 is omitted.

[0131] Figure 10 The optical image of the treatment instrument 13 reflected in the processing region is shown. The optical image of the treatment instrument 13 reflected in the first region is omitted in Figure 10 . Figure 10 An example in which the treatment instrument 13 is reflected in the right side of the center of the image in the first image and the second image is shown.

[0132] Before the processor 41 changes the parallax of the processing area in the first image, the optical image 13a of the processing device 13 projecting onto the processing area is displayed near the front of the screen surface SC. After the processor 41 changes the parallax of the processing area in the first image, the optical image 13b of the processing device 13 projecting onto the processing area is displayed on an imaginary plane PL1 that is a distance A1 away from the screen surface SC. The plane PL1 is directly opposite the observer's viewpoint. The optical image 13b is away from the observer's viewpoint.

[0133] exist Figure 10 In the example shown, plane PL1 is located on the inside of screen surface SC. Plane PL1 can also be located on the near-front side of screen surface SC.

[0134] Before executing step S110, the information indicating distance A1 can also be stored in... Figure 3 The memory is not shown. Processor 41 can also read the information from the memory in step S110. Processor 41 can also obtain the information from a device different from the endoscope device 1.

[0135] Processor 41 can also calculate distance A1 based on at least one of the first image and the second image. For example, distance A1 can also be the same as the distance between the optical image of the subject of the outermost pixel in the first region and the screen surface. In this case, it is less likely to produce discontinuities in the amount of parallax at the boundary between the processing region and the region outside the processing region. That is, it is less likely to produce discontinuities in the amount of parallax at the boundary between the first region and the second region. Therefore, the boundary becomes less noticeable, and the image looks more natural.

[0136] The observer can also specify a distance A1. For example, the observer can operate the operation unit 22 and input the distance A1. The processor 41 can also use the distance A1 input to the operation unit 22.

[0137] After the processor 41 changes the parallax of the processing area, the optical image of the processing device 13 projected onto the processing area is displayed in the stereoscopic image as a plane separated from the screen surface by a distance A1. Therefore, the image processing device 4 can reduce eye strain caused by the device's image without compromising the ease of use of the device. When the optical image of the device is displayed on the inside of the screen surface, the effect of reducing eye strain is enhanced.

[0138] (Second variation of the first embodiment)

[0139] A second variation of the first embodiment of the present invention will be described. Other methods for changing the amount of parallax in a manner that moves the optical image of the processing device 13 away from the observer's viewpoint will be described.

[0140] The processing region includes two or more pixels. The processor 41 changes the parallax amount in the image processing step in a manner that two or more points of the optical image corresponding to the two or more pixels move in a direction away from the viewpoint of the observer (a direction toward the screen surface). The distances moved by the two or more points are equal to each other.

[0141] The processor 41 shifts the positions of the data of the respective pixels included in the processing region in the first image in a prescribed direction in step S110. Thereby, the processor 41 changes the parallax amount of the processing region. The prescribed direction is the same as the direction explained in the first modification of the first embodiment.

[0142] The processor 41 shifts the positions of the data of the respective pixels included in the processing region in a manner that the optical images of the subject in the respective pixels move from the positions of the respective optical images to positions that are apart by a distance A2 inward in step S110. The processor 41 changes the parallax amount of the respective pixels included in the processing region by B2 by performing this processing. Thereby, the optical images of the subject in all the pixels included in the processing region move by the same distance A2. The processor 41 can calculate the change amount B2 of the parallax amount based on the distance A2.

[0143] For example, the processing region includes a first pixel and a second pixel. The distance A2 moved by the optical image of the subject in the first pixel is the same as the distance A2 moved by the optical image of the subject in the second pixel.

[0144] A method of shifting the positions of the data of the respective pixels is explained. The processor 41 substitutes the data of the respective pixels included in the processing region with the data of pixels that are apart by a distance C2 in a direction opposite to the prescribed direction. The distance C2 can be the same as the change amount B2 of the parallax amount, or the distance C2 can be calculated based on the change amount B2 of the parallax amount. The processor 41 substitutes the data of the respective pixels with the data of other pixels by using the same method as the method explained in the first modification of the first embodiment. The processor 41 can also shift the positions of the data of the respective pixels included in the processing region in the second image in the prescribed direction.

[0145] Figure 11 The positions of the optical images of the subject visually recognized by the observer when a stereoscopic image is displayed on the monitor 5 based on the first image and the second image are shown. The explanation of the parts common to the parts shown in Figure 7 is omitted.

[0146] Figure 11 The optical image of the treatment instrument 13 that is projected in the processing region is shown. The optical image of the treatment instrument 13 projected in the first region is omitted in Figure 11 . Figure 11 An example in which the treatment instrument 13 projects to the right side of the center of the image in the first image and the second image is shown.

[0147] Before the processor 41 changes the parallax amount of the processing region in the first image, the optical image 13a of the treatment instrument 13 that is projected in the processing region is displayed on the front side of the screen surface SC. After the processor 41 changes the parallax amount of the processing region in the first image, the optical image 13b of the treatment instrument 13 that is projected in the processing region is displayed at a position that is apart from the optical image 13a by a distance A2 on the inner side. The optical image 13b is apart from the viewpoint of the observer.

[0148] In Figure 11 In the example shown, the optical image 13b of the treatment instrument 13 includes a portion on the inner side of the screen surface SC and a portion on the front side of the screen surface SC. It is also possible that the entire optical image 13b is on the inner side of the screen surface SC or on the front side of the screen surface SC.

[0149] Before the step S110 is executed, information indicating the distance A2 can also be stored in a memory that is not shown in the endoscope apparatus 1. The processor 41 can also read out the information from the memory in the step S110. The processor 41 can also acquire the information from an apparatus that is different from the endoscope apparatus 1. Figure 3

[0150] The observer can also specify the distance A2. For example, the observer can input the distance A2 by operating the operation section 22. The processor 41 can also use the distance A2 that is input to the operation section 22.

[0151] After the processor 41 changes the parallax amount of the processing region, the optical image of the treatment instrument 13 that is projected in the processing region is displayed in the stereoscopic image at a position that is apart from the actual position by the distance A2 on the inner side. Therefore, the image processing apparatus 4 can reduce the fatigue of the eyes of the observer that is caused by the image of the instrument without impairing the ease of use of the instrument.

[0152] The optical image of the subject in all the pixels included in the processing region is moved by the same distance A2. Therefore, the information of the relative depth is maintained in the processing region. As a result, the observer easily operates the treatment instrument 13.

[0153] (Third Modification of the First Embodiment)

[0154] A third modification of the first embodiment of the present application is described. Another method of changing the parallax amount in such a way that the optical image of the treatment instrument 13 is apart from the viewpoint of the observer is described.

[0155] ​The processing region includes two or more pixels. The processor 41 changes the parallax amount in the image processing step in a manner that two or more points of the optical image corresponding to the two or more pixels move in a direction away from the viewpoint of the observer (a direction toward the screen surface). The greater the distance between each of the two or more pixels and the first region, the greater the distance each of the two or more points moves.

[0156] The greater the distance between the treatment instrument 13 and the first region, the more the treatment instrument 13 has a tendency to protrude greatly forward. Therefore, the farther the treatment instrument 13 is from the first region, the greater the distance the treatment instrument 13 needs to move inward from the actual position. It can also be that the smaller the distance between each of the two or more pixels and the end of the image, the greater the distance each of the two or more points of the optical image of the treatment instrument 13 moves.

[0157] The processor 41 shifts the position of the data of each pixel included in the processing region in a prescribed direction in step S110. Thus, the processor 41 changes the parallax amount of the processing region. The prescribed direction is the same as the direction explained in the first modification of the first embodiment.

[0158] The processor 41 calculates the distance A3 that the optical image of the subject in each pixel included in the processing region moves in step S110. The distance A3 is a value corresponding to the two-dimensional distance between each pixel and a reference position of the first region. For example, the reference position is a pixel of the first region closest to each pixel included in the processing region. Other pixels of the first region are located at the end of the first region. The reference position can also be the center of the first region or the center of the first image. The processor 41 shifts the position of the data of each pixel in a manner that the optical image of the subject in each pixel moves from the position of each optical image to a position that is A3 inward. The processor 41 changes the parallax amount B3 of each pixel included in the processing region by performing this processing. Thus, the optical image of the subject in each pixel included in the processing region moves the distance A3 corresponding to the position of each pixel. The processor 41 can calculate the change amount B3 of the parallax amount based on the distance A3.

[0159] For example, the processing region includes a first pixel and a second pixel. The distance between the second pixel and the first region is greater than the distance between the first pixel and the first region. The distance A3 that the optical image of the subject in the second pixel moves is greater than the distance A3 that the optical image of the subject in the first pixel moves.

[0160] The distance A3 by which the optical image of the subject in the pixel included in the processing region and bordering the first region moves can also be 0. In the case where the pixel included in the processing region is close to the first region, the distance A3 by which the optical image of the subject in the pixel moves can also be very small. The distance A3 can also increase exponentially based on the distance between the pixel included in the processing region and the first region.

[0161] A method of shifting the position of the data of each pixel is explained. The processor 41 substitutes the data of each pixel included in the processing region with the data of the pixel that has moved away by a distance C3 in the direction opposite to the prescribed direction. The distance C3 can be the same as the change amount B3 of the parallax amount, or the distance C3 can also be calculated based on the change amount B3 of the parallax amount. The processor 41 substitutes the data of each pixel with the data of another pixel by using the same method as the method explained in the first modification of the first embodiment. The processor 41 can also shift the position of the data of each pixel included in the processing region in the prescribed direction in the second image.

[0162] Figure 12 The position of the optical image of the subject visually recognized by the observer when the stereoscopic image is displayed on the monitor 5 based on the first image and the second image is shown. The explanation of the parts that are the same as the parts shown in Figure 7 is omitted.

[0163] Figure 12 The optical image of the treatment instrument 13 that is projected in the processing region is shown. The optical image of the treatment instrument 13 that is projected in the first region is omitted in Figure 12 . Figure 12 An example in which the treatment instrument 13 projects to the right of the center of the image in the first image and the second image is shown.

[0164] Before the processor 41 changes the parallax amount of the processing region in the first image, the optical image 13a of the treatment instrument 13 that is projected in the processing region is displayed on the front side of the screen surface SC. After the processor 41 changes the parallax amount of the processing region in the first image, the optical image 13b of the treatment instrument 13 that is projected in the processing region is displayed on a position that is away to the inside from the optical image 13a. The point of the optical image 13a that is farthest from the first region moves by the distance A3a. The point of the optical image 13a that is closest to the first region does not move. The point can also move by a distance smaller than the distance A3a. The optical image 13b is away from the viewpoint of the observer.

[0165] In the example shown in Figure 12 , the optical image 13b of the treatment instrument 13 is located on the front side of the screen surface SC. At least a part of the optical image 13b can also be located on the inside of the screen surface SC.

[0166] The information indicating the distance A3 can also be stored in the storage 42 before step S110 is executed.Figure 3 The processor 41 can also read out the information from the memory in step S110. The processor 41 can also acquire the information from a device different from the endoscope device 1.

[0167] After the processor 41 changes the parallax amount of the processing region, the optical image of the treatment instrument 13 reflected in the processing region is displayed in the stereoscopic image at a position that is apart from the actual position by a distance A3 inward. Therefore, the image processing device 4 can reduce the fatigue of the observer's eyes caused by the image of the instrument without impairing the ease of use of the instrument.

[0168] In a case where the optical image of the subject in the pixel included in the processing region and bordering the first region does not move, a discontinuity of the parallax amount is not easily generated at the border of the first region and the processing region. Therefore, the observer does not easily feel discomfort. The processor 41 can also not perform the image processing of making the change of the data of the periphery of the border of the first region and the processing region smooth.

[0169] (4th Modification of the 1st Embodiment)

[0170] The 4th modification of the 1st embodiment of the present application will be described. Before the image processing step, the processor 41 sets the processing region in a region setting step based on at least one of the kind of the image generation device and the kind of the instrument. The image generation device is a device having the imaging element 12 that generates the first image and the second image. In the example shown in the drawing, the image generation device is the electronic endoscope 2. Figure 1

[0171] The position in the image where the treatment instrument 13 is reflected differs depending on the number and position of the channels in the insertion portion 21. The number and position of the channels differ depending on the kind of the electronic endoscope 2 in many cases. Further, the kind of the treatment instrument 13 inserted into the channel is sometimes fixed. The size or shape of the treatment instrument 13 or the like differs depending on the kind of the treatment instrument in many cases. Therefore, the position in the image where the treatment instrument 13 is reflected differs depending on the kind of the electronic endoscope 2 and the kind of the treatment instrument 13 in many cases.

[0172] For example, before step S100 is performed, region information that associates the kind of the electronic endoscope 2, the kind of the treatment instrument 13, and the position of the processing region is stored in a memory not shown in the drawing. The processor 41 reads out the region information from the memory in step S100. The processor 41 can also acquire the region information from a device different from the endoscope device 1. Figure 3

[0173] Figure 13 ​​Examples of the region information are shown. The region information includes information El, information E2, and information E3. The information El indicates the kind of the electronic endoscope 2. The information E2 indicates the kind of the treatment instrument 13. The information E3 indicates the position of the treatment region. The information E3 can also include information indicating at least one of the size and the shape of the treatment region. In the case where the size of the treatment region is always fixed, the information E3 can also not include information indicating the size of the treatment region. In the case where the shape of the treatment region is always fixed, the information E3 can also not include information indicating the shape of the treatment region.

[0174] In the case where the electronic endoscope F1 is used, the electronic endoscope F1, the treatment instrument G1, and the treatment region H1 are associated. In the case where the electronic endoscope F2 is used, the electronic endoscope F2, the treatment instrument G2, and the treatment region H2 are associated. In the case where the electronic endoscope F3 is used, the electronic endoscope F3, the treatment instrument G3, the treatment instrument G4, and the treatment region H3 are associated. Figure 13 In the case where the electronic endoscope F1 is used, the electronic endoscope F1, the treatment instrument G1, and the treatment region H1 are associated. In the case where the electronic endoscope F2 is used, the electronic endoscope F2, the treatment instrument G2, and the treatment region H2 are associated. In the case where the electronic endoscope F3 is used, the electronic endoscope F3, the treatment instrument G3, the treatment instrument G4, and the treatment region H3 are associated. Figure 13 In the case where the electronic endoscope F1 is used, the electronic endoscope F1, the treatment instrument G1, and the treatment region H1 are associated. In the case where the electronic endoscope F2 is used, the electronic endoscope F2, the treatment instrument G2, and the treatment region H2 are associated. In the case where the electronic endoscope F3 is used, the electronic endoscope F3, the treatment instrument G3, the treatment instrument G4, and the treatment region H3 are associated. Figure 13 In the case where the electronic endoscope F1 is used, the electronic endoscope F1, the treatment instrument G1, and the treatment region H1 are associated. In the case where the electronic endoscope F2 is used, the electronic endoscope F2, the treatment instrument G2, and the treatment region H2 are associated. In the case where the electronic endoscope F3 is used, the electronic endoscope F3, the treatment instrument G3, the treatment instrument G4, and the treatment region H3 are associated. Figure 13 In the case where the electronic endoscope F1 is used, the electronic endoscope F1, the treatment instrument G1, and the treatment region H1 are associated. In the case where the electronic endoscope F2 is used, the electronic endoscope F2, the treatment instrument G2, and the treatment region H2 are associated. In the case where the electronic endoscope F3 is used, the electronic endoscope F3, the treatment instrument G3, the treatment instrument G4, and the treatment region H3 are associated.

[0175] The region information can also include only the information El and the information E3. Alternatively, the region information can also include only the information E2 and the information E3.

[0176] The processor 41 judges the kind of the electronic endoscope 2 used and the kind of the treatment instrument 13 used. For example, the observer can also operate the operation section 22 to input information indicating the kind of the electronic endoscope 2 and the kind of the treatment instrument 13. The processor 41 can also judge the kind of the electronic endoscope 2 and the kind of the treatment instrument 13 based on the information.

[0177] The processor 41 can also acquire information indicating the kind of the electronic endoscope 2 and the kind of the treatment instrument 13 from the electronic endoscope 2 when the electronic endoscope 2 is connected to the image processing apparatus 4. The endoscope apparatus 1 can also have a code reader which reads a two-dimensional code and the processor 41 acquires information of the two-dimensional code from the code reader. The two-dimensional code indicates the kind of the electronic endoscope 2 and the kind of the treatment instrument 13. The two-dimensional code can also be attached to the surface of the electronic endoscope 2.

[0178] The processor 41 extracts information of a processing region corresponding to the combination of the electronic endoscope 2 and the treatment instrument 13 used from the region information. For example, in a case where the electronic endoscope F2 and the treatment instrument G2 are used, the processor 41 extracts information of the processing region H2. The processor 41 sets the processing region on the basis of the extracted information.

[0179] Figure 14 An example of the first image is shown. Figure 14 The first image 202 shown is an image of the observation object 210 and the treatment instrument 13. The first image 202 includes a first region R12 and a second region R13. The broken line L11 represents a boundary of the first region R12 and the second region R13. The first region R12 is a region on an upper side of the broken line L11, and the second region R13 is a region on a lower side of the broken line L11. The first region R12 includes a center C11 of the first image 202. The observation object 210 is reflected in the first region R12. The second region R13 includes an end portion on a lower side of the first image 202. The treatment instrument 13 is reflected in the second region R13. The processor 41 sets the second region R13 as a processing region.

[0180] In a case where a specific kind of electronic endoscope 2 is used, the treatment instrument 13 is reflected only in a region on a lower side of the first image 202. In this case, the processor 41 can set the second region R13 shown instead of the second region R11 shown as a processing region. The second region R13 is smaller than the second region R11. Figure 5 Figure 14 In a case where a specific kind of electronic endoscope 2 is used, the treatment instrument 13 is reflected only in a region on a lower side of the first image 202. In this case, the processor 41 can set the second region R13 shown instead of the second region R11 shown as a processing region. The second region R13 is smaller than the second region R11.

[0181] The processor 41 can set a processing region suitable for the kind of the electronic endoscope 2 and the kind of the treatment instrument 13. Therefore, the processing region becomes smaller, and the load of the processor 41 in the process of changing the parallax amount decreases.

[0182] (Second Embodiment)

[0183] A second embodiment of the present application is described. In the second embodiment, a processing region includes a first region and a second region. For example, the processing region is the entirety of a first image or the entirety of a second image.

[0184] The processing region includes two or more pixels. The processor 41 changes a parallax amount of the processing region in such a manner that a distance between a viewpoint of an observer and each of two or more points of an optical image corresponding to the two or more pixels becomes a prescribed value or more.

[0185] The process performed by the processor 41 is described with reference to Figure 15 , with reference to FIG. 8. Figure 15 Steps of the process performed by the processor 41 are shown. The description of the process common to that shown in Figure 8 is omitted.​

[0186] The processor 41 does not execute Figure 8 the step S100. After the step S105, the processor 41 changes the disparity amount of the processing region of at least one of the first image and the second image (step S110a (image processing step)). After the step S110a, the step S115 is executed.

[0187] The step S110a is different from Figure 8 the step S110. The step S110a is described in detail. Hereinafter, an example in which the processor 41 changes the disparity amount of the first image is described. The processor 41 can also change the disparity amount of the second image by using the same method as the following method.

[0188] The processor 41 calculates the disparity amount of each pixel included in the first image. The processor 41 executes this processing on all the pixels included in the first image. For example, the processor 41 calculates the disparity amount of each pixel by using stereo matching.

[0189] The processor 41 executes the following processing on all the pixels included in the first image. The processor 41 compares the disparity amount of the pixel with a prescribed amount B4. In a case where the disparity amount of the pixel is less than the prescribed amount B4, the distance between the viewpoint of the observer and the optical image of the subject in the pixel is less than A4. The observer observes that the subject is largely protruding. In a case where the disparity amount of the pixel included in the first image is less than the prescribed amount B4, the processor 41 changes the disparity amount of the pixel to the prescribed amount B4. In a case where the disparity amount of the pixel included in the first image is the prescribed amount B4 or more, the processor 41 does not change the disparity amount of the pixel. The processor 41 can calculate the prescribed amount B4 of the disparity based on the distance A4. The processor 41 changes the disparity amount of the processing region in such a manner that the distance between the viewpoint of the observer and the optical image of the treatment instrument 13 becomes a prescribed value or more by executing the above-described processing.

[0190] The processor 41 shifts the position of data of at least a part of all the pixels included in the first image to a prescribed direction. Thereby, the processor 41 changes the disparity amount of the processing region. The prescribed direction is the same as the direction described in the first modification example of the first embodiment.

[0191] In a case where the parallax amount of the pixel included in the 1st image is smaller than the prescribed amount B4, the processor 41 replaces the data of the pixel with the data of the pixel that has moved away by a distance C4 in a direction opposite to the prescribed direction. The distance C4 can be the same as the difference between the parallax amount of the pixel and the prescribed amount B4, or the distance C4 can be calculated based on the difference. The processor 41 replaces the data of each pixel with the data of another pixel by using the same method as that explained in the 1st modification of the 1st embodiment. The processor 41 can also shift the position of the data of each pixel included in the processing region in the 2nd image in the prescribed direction.

[0192] The parallax amount of the pixel included in the 1st region including the observation object is mostly equal to or larger than the prescribed amount B4. The parallax amount of the pixel included in a part of the 1st region is sometimes smaller than the prescribed amount B4. In this case, the processor 41 changes the parallax amount of the pixel included in the 1st region by performing the above-described processing. The amount of change in the parallax amount is smaller than the maximum value of the amount of change in the parallax amount of the pixel included in the 2nd region.

[0193] Figure 16 The position of the optical image of the subject that is visually recognized by the observer when the stereoscopic image is displayed on the monitor 5 based on the 1st image and the 2nd image is shown. The explanation of the parts that are the same as those shown in Figure 7 is omitted. Figure 16 An example in which the treatment instrument 13 is reflected on the right side of the center of the image in the 1st image and the 2nd image is shown.

[0194] The distance between the viewpoint of the observer and a part of the optical image 13a of the treatment instrument 13 is smaller than A4 before the processor 41 changes the parallax amount of the 1st image. The minimum value of the distance between the viewpoint of the observer and the optical image 13b of the treatment instrument 13 becomes A4 after the processor 41 changes the parallax amount of the 1st image. The region of the optical image 13a of the treatment instrument 13 that protrudes greatly on the side of the viewpoint of the observer is displayed at a position that has moved away by a distance A4 from the viewpoint of the observer.

[0195] In the example shown in Figure 16 , the prescribed amount B4 of the parallax amount corresponding to the distance A4 is a positive value. Therefore, the optical image 13b of the treatment instrument 13 is located on the inner side of the screen surface SC. The prescribed amount B4 can also be a negative value. In this case, at least a part of the optical image 13b is located on the front side of the screen surface SC. The prescribed amount B4 can also be 0. In this case, at least a part of the optical image 13b is located in the plane including the intersection point CP (the screen surface SC).

[0196] The information indicating the distance A4 can also be stored in the storage 42 before step S110a is performed. Figure 3The memory is not shown. Processor 41 can also read the information from the memory in step S110a. Processor 41 can also obtain the information from a device different from the endoscope device 1.

[0197] The observer can also specify a distance A4. For example, the observer can operate the operation unit 22 and input the distance A4. The processor 41 can also use the distance A4 input to the operation unit 22.

[0198] After the processor 41 changes the parallax of the processing area, the optical image of the processing device 13 is displayed in a stereoscopic image at a position more than A4 away from the observer's viewpoint. Therefore, the image processing device 4 can reduce eye strain caused by the device's image without compromising the ease of use of the device.

[0199] The optical image of the device 13 does not move in the region where the parallax is not altered. Relative depth information is maintained in this region. As a result, the device 13 is easy for the observer to operate.

[0200] (First variation of the second embodiment)

[0201] A first variation of the second embodiment of the present invention will be described. Other methods for changing the parallax of the processing area in such a way that the distance between the observer's viewpoint and the optical image of the processing device 13 is a predetermined value or higher will be described.

[0202] Before executing step S110a, the disparity information representing the change in disparity is stored in... Figure 3 The memory is not shown in the diagram. Figure 17 An example of parallax information is shown. Figure 17 In the diagram, disparity information is shown as a graph. The disparity information shows the relationship between a first disparity value and a second disparity value. The first disparity value is the disparity value possessed by each pixel before the processor 41 changes the disparity value. The second disparity value is the disparity value possessed by each pixel after the processor 41 changes the disparity value. When the first disparity value is A4a or higher, the first and second disparity values ​​are the same. When the first disparity value is less than A4a, the second disparity value is different from the first disparity value. When the first disparity value is less than A4a, the second disparity value is B4 or higher.

[0203] Figure 17 The second parallax value B4 shown is positive. Therefore, the optical image of the processing device 13 is displayed on the inside of the screen surface. The second parallax value B4 can also be negative.

[0204] The processor 41 reads out the parallax information from the memory in step S110a. The processor 41 changes the parallax amount of each pixel included in the first image on the basis of the parallax information. The processor 41 performs this processing on all the pixels included in the first image. The processor 41 can also change the parallax amount of each pixel included in the second image on the basis of the parallax information. The processor 41 can also acquire the parallax information from a device different from the endoscope device 1.

[0205] For example, in the case where the first parallax amount in the region indicated by A4a is smaller than A4a, the graph is shown in a curve. In this case, the observer is less likely to feel discomfort with the image compared to the method explained in the second embodiment. Figure 17

[0206] A second modification example of the second embodiment of the present application will be explained. Before the image processing step, the processor 41 sets a processing region on the basis of at least one of the kind of the image generation device and the kind of the instrument in a region setting step. The image generation device is a device having the imaging element 12 that generates the first image and the second image. In the example shown in FIG. 12, the image generation device is the electronic endoscope 2.

[0207] The method by which the processor 41 sets the processing region is the same as the method explained in the fourth modification example of the first embodiment. The processor 41 changes the parallax amount of the processing region in such a way that the distance between the viewpoint of the observer and the optical image of the treatment instrument 13 becomes a prescribed value or more. Figure 1

[0208] The processor 41 can set a processing region that is appropriate for the kind of the electronic endoscope 2 and the kind of the treatment instrument 13. Therefore, the processing region becomes smaller, and the load on the processor 41 in the processing of changing the parallax amount is reduced.

[0209] The processor 41 can set a processing region that is appropriate for the kind of the electronic endoscope 2 and the kind of the treatment instrument 13. Therefore, the processing region becomes smaller, and the load on the processor 41 in the processing of changing the parallax amount is reduced.

[0210] A third embodiment of the present application will be explained. Before the image processing step, the processor 41 detects the treatment instrument 13 from at least one of the first image and the second image in an instrument detection step. Before the image processing step, the processor 41 sets a region in which the treatment instrument 13 is detected as the processing region in a region setting step.

[0211] The processing performed by the processor 41 will be explained with reference to FIG. 13.

[0212] The processing performed by the processor 41 will be explained with reference to FIG. 13. Figure 18 The steps of the processing performed by the processor 41 will be shown. The explanation of the processing common to the processing shown in FIG. 12 will be omitted. Figure 18 The steps of the processing performed by the processor 41 will be shown. The explanation of the processing common to the processing shown in FIG. 12 will be omitted. Figure 8 The steps of the processing performed by the processor 41 will be shown. The explanation of the processing common to the processing shown in FIG. 12 will be omitted.

[0213] The processor 41 does not perform the processing shown in FIG. 12. Figure 8 ​​The step S100 is shown. After the step S105, the processor 41 detects the treatment instrument 13 from at least one of the first image and the second image (a step S120 (an instrument detection step)). After the step S120, the processor 41 sets a region in which the treatment instrument 13 is detected as a processing region (a step S100a (a region setting step)). After the step S100a, the step S110 is executed.

[0214] Before the step S120 is executed, two or more images of the treatment instrument 13 are stored in a memory (not shown in the figure). The treatment instrument 13 is imaged in each of the images at various angles. An observer can also specify a region in which the treatment instrument 13 is imaged in an image generated in the past by the imaging element 12. The image of the region can also be stored in the memory. Figure 3

[0215] The processor 41 reads out each of the images of the treatment instrument 13 from the memory in the step S120. The processor 41 compares the first image with each of the images of the treatment instrument 13. Alternatively, the processor 41 compares the second image with each of the images of the treatment instrument 13. Thereby, the processor 41 determines a region in which the treatment instrument 13 is imaged in the first image or the second image. The processor 41 sets only the region in which the treatment instrument 13 is imaged as the processing region in the step S100a.

[0216] The processor 41 can execute the step S110 by using the method explained in the first embodiment and various modifications thereof. Alternatively, the processor 41 can execute the step S110 by using the method explained in the second embodiment and various modifications thereof.

[0217] The processor 41 sets a region in which the treatment instrument 13 is imaged as the processing region, and changes the parallax amount of the region. The processor 41 does not set a region in which the treatment instrument 13 is not imaged as the processing region, and does not change the parallax amount of the region. Therefore, an observer does not easily feel discomfort in a region in which the treatment instrument 13 is not imaged in the stereoscopic image.

[0218] (First Modification of the Third Embodiment)

[0219] A first modification of the third embodiment of the present application is explained. The processor 41 detects the treatment instrument 13 from at least one of the first image and the second image in the instrument detection step. The processor 41 detects a tip region including a tip of the treatment instrument 13 in the region in which the treatment instrument 13 is detected in the region setting step. The processor 41 sets a region from which the tip region is removed from the region in which the treatment instrument 13 is detected as the processing region.

[0220] ​The processor 41 determines the region in which the treatment instrument 13 is visualized in the first image or the second image in step S120 by using the above-described method. Further, the processor 41 detects a tip region including a tip of the treatment instrument 13 in the region. The tip region is, for example, a region between the tip of the treatment instrument 13 and a position that is a prescribed distance away from the tip toward the base. The tip region can also be a region including only the forceps 130. The processor 41 sets a region obtained by removing the tip region from the region in which the treatment instrument 13 is visualized as a processing region. The processing region can also be a region including only the sheath 131.

[0221] The parallax amount of the region on the tip side of the treatment instrument 13 is not changed in the first image or the second image. Therefore, information of the relative depth is maintained in the region. As a result, the observer easily operates the treatment instrument 13.

[0222] (Second Modification of the Third Embodiment)

[0223] The second modification of the third embodiment of the present application will be described. The processor 41 sets the processing region based on at least one of the kind of the image generation device and the kind of the instrument in the region setting step. The image generation device is a device having the imaging element 12 that generates the first image and the second image. In the example shown in FIG. 1, the image generation device is the electronic endoscope 2. Figure 1

[0224] The processor 41 does not perform step S120. The processor 41 sets the processing region based on the region information that associates the kind of the electronic endoscope 2, the kind of the treatment instrument 13, and the position of the processing region in step S100a. The processing region is a region obtained by removing a tip region including a tip of the treatment instrument 13 from the entire treatment instrument 13. The processing region can also be a region including only the sheath 131. The method of setting the processing region by the processor 41 is the same as the method described in the fourth modification of the first embodiment.

[0225] The processor 41 does not need to detect the treatment instrument 13 from the first image or the second image. Therefore, the load of the processor 41 is reduced compared to a case in which the processor 41 performs image processing for detecting the treatment instrument 13.

[0226] (Third Modification of the Third Embodiment)

[0227] The third modification of the third embodiment of the present application will be described. The processor 41 detects a region of the treatment instrument 13 obtained by removing a tip region including a tip of the treatment instrument 13 from the first image and the second image in the instrument detection step. The processor 41 sets the detected region as the processing region in the region setting step.

[0228] ​For example, a portion of the treatment instrument 13 obtained by removing the front end region of the treatment instrument 13 has a prescribed color. The prescribed color is different from the color of the subject such as an organ or a blood vessel, and is different from the color of the observation object. For example, a portion of the sheath 131 including the root portion has the prescribed color. The entire sheath 131 can also have the prescribed color. The processor 41 detects a region having the prescribed color in at least one of the first image and the second image in step S120. The processor 41 sets the detected region as the processing region in step S100a.

[0229] A mark can also be attached to a portion of the treatment instrument 13 obtained by removing the front end region of the treatment instrument 13. The shape of the mark can be any shape. The mark can also be a character or a symbol, or the like. Two or more marks can also be attached. The processor 41 can also detect the mark in at least one of the first image and the second image, and set a region including the detected mark as the processing region.

[0230] A prescribed pattern can also be attached to a portion of the treatment instrument 13 obtained by removing the front end region of the treatment instrument 13. The treatment instrument 13 can also have a portion including the root portion to which the pattern is attached and a portion to which the pattern is not attached. The treatment instrument 13 can also have a portion including the root portion to which a first pattern is attached and a portion to which a second pattern different from the first pattern is attached. The portion to which the pattern is attached can also be the entire sheath 131 or a portion thereof. The processor 41 can also detect the prescribed pattern in at least one of the first image and the second image, and set a region including the detected pattern as the processing region.

[0231] The portion of the treatment instrument 13 obtained by removing the front end region of the treatment instrument 13 is configured to be distinguishable from other portions of the treatment instrument 13. Therefore, the accuracy with which the processor 41 detects the region of the treatment instrument 13 set as the processing region is improved.

[0232] (4th Embodiment)

[0233] The 4th embodiment of the present application will be described. The processor 41 judges the position of the first region which differs depending on the situation of observation.

[0234] For example, before the image processing step, the processor 41 judges the position of the first region in a region setting step based on the kind of image generation device which generates the first image and the second image. The processor 41 sets a region obtained by removing the first region as the processing region. The image generation device is a device having the imaging element 12 which generates the first image and the second image. In the example shown in FIG. 10, the image generation device is the electronic endoscope 2. Figure 1

[0235] ​The position of the observation object sometimes differs depending on the site as the subject. The kind of the site and the kind of the electronic endoscope 2 that can be inserted into the site are mostly fixed. Therefore, the position of the observation object mostly differs depending on the kind of the electronic endoscope 2.

[0236] The processing performed by the processor 41 will be described with reference to Figure 19 . Figure 19 The steps of the processing performed by the processor 41 will be described. The description of the processing common to the processing shown in Figure 8 will be omitted.

[0237] The processor 41 does not perform the step S100 shown in Figure 8 . The processor 41 judges the position of the first region, and sets the region after the first region is removed as the processing region (step S125 (region setting step)). After the step S125, the step S105 is performed. The order in which the step S125 and the step S105 are performed can also be different from the order shown in Figure 8 . That is, the step S125 can also be performed after the step S105 is performed.

[0238] The step S125 will be described in detail. Before the step S125 is performed, the region information in which the kind of the electronic endoscope 2 is associated with the position of the first region is stored in the memory not shown in Figure 3 . The processor 41 reads out the region information from the memory in the step S125. The processor 41 can also acquire the region information from a device different from the endoscope device 1.

[0239] Figure 20 An example of the region information will be described. The region information includes the information E1 and the information E4. The information E1 indicates the kind of the electronic endoscope 2. The information E4 indicates the position of the first region. The information E4 can also include information indicating at least one of the size and the shape of the first region. In the case where the size of the first region is always fixed, the information E4 can also not include information indicating the size of the first region. In the case where the shape of the first region is always fixed, the information E4 can also not include information indicating the shape of the first region.

[0240] In the example shown in Figure 20 , the electronic endoscope F1 and the first region I1 are associated. In the example shown in Figure 20 , the electronic endoscope F2 and the first region I2 are associated. In the example shown in Figure 20 , the electronic endoscope F3 and the first region I3 are associated.

[0241] The processor 41 determines the kind of the electronic endoscope 2 used by using the method explained in the fourth modification of the first embodiment. The processor 41 extracts the information of the first region corresponding to the electronic endoscope 2 used from the region information. For example, in the case where the electronic endoscope F2 is used, the processor 41 extracts the information of the first region I2. The processor 41 regards the position shown by the extracted information as the position of the first region, and sets the region after the first region is removed as the processing region.

[0242] The processor 41 can perform the step S110 by using the method explained in the first embodiment and its various modifications. Alternatively, the processor 41 can perform the step S110 by using the method explained in the second embodiment and its various modifications.

[0243] The processor 41 can set the processing region to an appropriate position based on the position of the first region which differs depending on the kind of the electronic endoscope 2.

[0244] (Modification of the Fourth Embodiment)

[0245] A modification of the fourth embodiment of the present application is explained. Another method of determining the position of the first region is explained.

[0246] The processor 41 determines the position of the first region based on the kind of the image generating device and the imaging magnification in the region setting step. The processor 41 sets the region after the first region is removed as the processing region.

[0247] As explained above, the position of the observation object differs depending on the kind of the electronic endoscope 2 in most cases. In addition, the size of the observation object differs depending on the imaging magnification. When the imaging magnification is high, the observation object is largely reflected in the image. When the imaging magnification is low, the observation object is less largely reflected in the image.

[0248] For example, the region information in which the kind of the electronic endoscope 2, the imaging magnification, and the position of the first region are associated with each other is stored in a memory not shown in the endoscope device 1 before the step S125 is performed. The processor 41 reads out the region information from the memory in the step S125. The processor 41 can also acquire the region information from a device different from the endoscope device 1. Figure 3

[0249] Figure 21 ​An example of area information is shown. The area information includes information E1, information E5, and information E4. Information E1 indicates the type of electronic endoscope 2. Information E5 indicates the magnification. Information E4 indicates the location of the first area. For example, information E4 may contain information indicating the location of the outer periphery of the first area, which varies depending on the magnification. Information E4 may also contain information indicating the shape of the first area. If the shape of the first area is always fixed, information E4 may not contain information indicating the shape of the first area.

[0250] exist Figure 21 In the example shown, the electronic endoscope F1, the camera magnification J1, and the first region I4 are associated. Figure 21 In the example shown, the electronic endoscope F1, the camera magnification J2, and the first region I5 are associated. Figure 21 In the example shown, the electronic endoscope F2, the camera magnification J1, and the first region I6 are associated. Figure 21 In the example shown, the electronic endoscope F2, the camera magnification J2, and the first region I7 are associated.

[0251] In addition to including regional information Figure 21 In addition to the information shown, the information may also include information indicating the type of treatment device 13. The area information may also include information indicating the type of treatment device 13 and the camera magnification, but not information indicating the type of electronic endoscope 2. During the area setting step, the processor 41 may determine the location of the first area based on at least one of the type of image generating device, the type of device, and the camera magnification. The processor 41 may also determine the location of the first area based on only one of the type of image generating device, the type of device, and the camera magnification. The processor 41 may also determine the location of the first area based on any two of the type of image generating device, the type of device, and the camera magnification. The processor 41 may also determine the location of the first area based on all of the type of image generating device, the type of device, and the camera magnification.

[0252] In the fourth variation of the first embodiment or the second variation of the second embodiment, the processor 41 may set the processing area based on at least one of the type of image generating apparatus, the type of device, and the camera magnification during the area setting step. The processor 41 may also set the processing area based on only one of the type of image generating apparatus, the type of device, and the camera magnification. The processor 41 may also set the processing area based on any two of the type of image generating apparatus, the type of device, and the camera magnification. The processor 41 may also set the processing area based on all of the type of image generating apparatus, the type of device, and the camera magnification.

[0253] The processor 41 determines the kind of the electronic endoscope 2 used by using the method explained in the fourth modification of the first embodiment. Further, the processor 41 acquires the information of the imaging magnification used from the imaging element 12.

[0254] The processor 41 extracts the information of the first region corresponding to the electronic endoscope 2 used and the imaging magnification from the region information. For example, in the case where the electronic endoscope F2 and the imaging magnification J1 are used, the processor 41 extracts the information of the first region I6. The processor 41 regards the position shown by the extracted information as the position of the first region, and sets the region after the first region is removed as the processing region.

[0255] The processor 41 can set the processing region to an appropriate position based on the position of the first region which differs depending on the kind of the electronic endoscope 2 and the imaging magnification.

[0256] (Fifth Embodiment)

[0257] The fifth embodiment of the present application is explained. Another method of setting the processing region based on the position of the first region is explained.

[0258] Before the image processing step, the processor 41 detects the observation object from at least one of the first image and the second image in an observation object detection step. Before the image processing step, the processor 41 regards the region in which the observation object is detected as the first region, and sets the region after the first region is removed as the processing region in a region setting step.

[0259] Reference Figure 22 The processing performed by the processor 41 is explained. Figure 22 The steps of the processing performed by the processor 41 are shown. The explanation of the processing same as the processing shown in Figure 8 is omitted.

[0260] The processor 41 does not perform Figure 8 shown in Step S100. After Step S105, the processor 41 detects the observation object from at least one of the first image and the second image (Step S130 (observation object detection step)). Step S130 is explained in detail. The processor 41 calculates the parallax amount of each pixel included in the first image. The processor 41 performs this processing on all the pixels included in the first image. For example, the processor 41 calculates the parallax amount of each pixel by using stereo matching.

[0261] The processor 41 detects the pixel that reflects the observation object based on the parallax amount of each pixel. For example, in a case where the observation object is a convex portion or a concave portion, the parallax amount of the pixel that reflects the observation object is different from the parallax amount of the pixel of the subject that reflects the periphery of the observation object. The processor 41 detects the pixel that reflects the observation object based on the distribution of the parallax amount of all the pixels included in the first image. The processor 41 can also detect the pixel that reflects the observation object based on the distribution of the parallax amount of only the pixels included in the region of the first image after the periphery portion is removed.

[0262] The processor 41 regards the region including the detected pixel as a first region. For example, the first region includes the region that reflects the observation object and the region of the periphery thereof. For example, the region of the periphery of the observation object includes the pixel located within a prescribed distance from the outer periphery of the observation object.

[0263] The processor 41 can also detect the pixel that reflects the treatment instrument 13 based on the distribution of the parallax amount described above. The parallax amount of the pixel that reflects the treatment instrument 13 is different from the parallax amount of the pixel of the subject that reflects the periphery of the treatment instrument 13. Since the treatment instrument 13 is located close to the observation object, the difference between the parallax amount of the pixel that reflects the treatment instrument 13 and the parallax amount of the pixel of the subject that reflects the periphery of the observation object is large. Therefore, the processor 41 can distinguish the observation object from the treatment instrument 13. The processor 41 can also remove the pixel that reflects the treatment instrument 13 from the first region.

[0264] When the treatment instrument 13 does not protrude from the end surface of the front end portion 10, the treatment instrument 13 is not reflected in the first image and the second image. At this time, the processor 41 can also detect the observation object from the first image. The processor 41 can also detect the observation object from the second image by performing the same processing as described above.

[0265] After the step S130, the processor 41 sets the region after the first region is removed as a processing region (step S100b (region setting step)). After the step S100b, the step S110 is performed.

[0266] The processor 41 can perform the step S110 by using the method described in the first embodiment and various modifications thereof. Alternatively, the processor 41 can perform the step S110 by using the method described in the second embodiment and various modifications thereof.

[0267] The processor 41 detects the observation object and sets the processing region based on the position of the observation object. The processor 41 can set the processing region that is appropriate for the observation object.

[0268] (First Modification of the Fifth Embodiment)

[0269] A first modification of the fifth embodiment of the present application will be described. Another method of detecting an observation target will be described.

[0270] The processor 41 generates a distribution of colors of all pixels included in the first image in the observation target detection step. The color tone of the observation target is mostly different from the color tone of the subject around the observation target. The processor 41 detects the pixels that represent the observation target based on the generated distribution. The processor 41 can also detect the pixels that represent the observation target based on a distribution of colors of only the pixels included in the region of the first image after the peripheral portion is removed.

[0271] The processor 41 can also detect the pixels that represent the treatment instrument 13 based on the distribution of colors described above. In a case where the treatment instrument 13 has a prescribed color that is different from the color of the observation target, the processor 41 is able to distinguish the observation target from the treatment instrument 13. The processor 41 can also remove the pixels that represent the treatment instrument 13 from the first region. The processor 41 can also detect the observation target from the second image by performing the same processing as described above.

[0272] The processor 41 detects the observation target based on the information of the colors in the image. The load of the processor 41 in the processing of detecting the observation target is reduced compared to a case where the processor 41 detects the observation target based on the distribution of the parallax amount. The processor 41 is able to remove the pixels that represent the treatment instrument 13 from the first region.

[0273] (A second modification of the fifth embodiment)

[0274] A first modification of the fifth embodiment of the present application will be described. Another method of detecting an observation target will be described.

[0275] The endoscope device 1 has a function of special light observation. The endoscope device 1 irradiates a mucosal tissue of a living body with light (narrow band light) including a waveband of a prescribed narrow width. The endoscope device 1 obtains information of a tissue at a desired depth in the living body tissue. For example, in a case where the observation target in the special light observation is a cancer tissue, narrow band light of blue that is suitable for observation of a tissue surface layer is irradiated to the mucosal tissue. At this time, the endoscope device 1 is able to observe the fine blood vessels of the tissue surface layer in detail.

[0276] Before the execution of the step S105, the light source of the light source device 3 generates narrow band light of blue. For example, the center wavelength of the narrow band of blue is 405 nm. The imaging element 12 captures the subject irradiated with the narrow band light, and generates the first image and the second image. The processor 41 acquires the first image and the second image from the imaging element 12 in the step S105. After the execution of the step S105, the light source device 3 can also generate white light.

[0277] Before the step S130 is executed, pattern information indicating a blood vessel pattern of a lesion as an observation object is stored in a memory not shown in the drawings. The processor 41 reads out the pattern information from the memory in the step S130. The processor 41 can also acquire the pattern information from a device different from the endoscope device 1. Figure 3 The processor 41 detects a region having a pattern similar to the pattern indicated by the pattern information from the first image in the step S130. The processor 41 regards the detected region as an observation object. The processor 41 can also detect an observation object from the second image by performing the same processing as the above-described processing.

[0278] When a cancer develops, a unique blood vessel that cannot be seen at a healthy site is generated in a fine blood vessel or the like of a lesion. The shape of the blood vessel generated due to the cancer has a unique pattern corresponding to the degree of progression of the cancer. The pattern information indicates such a pattern.

[0279] The processor 41 detects a region having a pattern similar to the pattern indicated by the pattern information from the first image in the step S130. The processor 41 regards the detected region as an observation object. The processor 41 can also detect an observation object from the second image by performing the same processing as the above-described processing.

[0280] The processor 41 detects an observation object based on a blood vessel pattern of a lesion. Therefore, the processor 41 can detect an observation object with high precision.

[0281] (6th Embodiment)

[0282] The 6th embodiment of the present application will be described. Another method of setting a processing region based on the position of the first region will be described. Before the image processing step, the processor 41 judges the position of the first region based on information input to the operation section 22 by an observer in a region setting step, and sets a region after the first region is removed as a processing region.

[0283] The processing performed by the processor 41 will be described with reference to the above-described Figure 19 The description of the processing common to that shown in the Figure 8 will be omitted.

[0284] The observer operates the operation section 22, and inputs the position of the first region. The observer can input the size or shape of the first region in addition to the position of the first region. In the case where the position of the first region is fixed, the observer can input only the size or shape of the first region. The observer can input the required information by operating a part other than the operation section 22. For example, in the case where the endoscope device 1 has a touch screen, the observer can operate the touch screen. In the case where the image processing device 4 has an operation section, the observer can operate the operation section.

[0285] In step S125, processor 41 determines the position of the first region based on the information input to operation unit 22. If the observer inputs the position of the first region, processor 41 treats the input position as the position of the first region. If the size and shape of the first region are fixed, processor 41 can determine that the first region is located at the position specified by the observer.

[0286] When the observer inputs the position and size of the first region, the processor 41 treats the input position as the position of the first region and the input size as the size of the first region. With the shape of the first region fixed, the processor 41 can determine that the first region is located at the position specified by the observer and has the size specified by the observer.

[0287] When the observer inputs the position and shape of the first region, the processor 41 treats the input position as the position of the first region and the input shape as the shape of the first region. With the size of the first region fixed, the processor 41 can determine that the first region is located at the position specified by the observer and has the shape specified by the observer.

[0288] Processor 41 determines the location of the first region using the method described above. Processor 41 then sets the region after removing the first region as the processing region.

[0289] In step S125, processor 41 may also determine the size of the first region based on the information input to operation unit 22. For example, the observer may only input the size of the first region, and processor 41 may treat the input size as the size of the first region. When the position and shape of the first region are fixed, processor 41 can determine that the first region has the size specified by the observer.

[0290] In step S125, the processor 41 may also determine the shape of the first region based on the information input to the operation unit 22. For example, the observer may only input the shape of the first region, and the processor 41 may regard the input shape as the shape of the first region. When the position and size of the first region are fixed, the processor 41 can determine that the first region has the shape specified by the observer.

[0291] The information that an observer can input is not limited to position, size, and shape. Alternatively, the observer can input items not shown in the above description.

[0292] Before executing step S125, the processor 41 may also acquire the first image and the second image from the imaging element 12 and output the first image and the second image to the monitor 5. The observer may also confirm the position of the first region in the displayed stereoscopic image and input the position to the operation unit 22.

[0293] The processor 41 determines the position of the first region based on the information input to the operation unit 22, and sets the processing region based on that position. The processor 41 can set a processing region suitable for the observer's expectations or observation conditions. The processor 41 can process the image in a manner that is easy for the observer to perform actions on.

[0294] (A variation of the sixth embodiment)

[0295] A variation of the sixth embodiment of the present invention will be described. Other methods for determining the position of the first region based on information input to the operation unit 22 by an observer will be explained.

[0296] The observer operates the operation unit 22, inputting various information. For example, the observer inputs the location within the body, the type of affected area, the patient's age, and the patient's gender. The processor 41 receives the information input into the operation unit 22.

[0297] For example, before executing step S125, the region information that associates the body part, the type of affected area, the patient's age, the patient's gender, and the location of the first region is stored in [the relevant information]. Figure 3 The memory is not shown. In step S125, the processor 41 reads the region information from the memory. The processor 41 can also obtain the region information from a device different from the endoscope device 1.

[0298] Figure 23 An example of region information is shown. The region information includes information E6, information E7, information E8, information E9, and information E4. Information E6 indicates the location containing the object of observation. Information E7 indicates the type of affected area as the object of observation. Information E8 indicates the patient's age. Information E9 indicates the patient's gender. Information E4 indicates the location of the first region. Information E4 may also include information indicating at least one of the size and shape of the first region. If the size of the first region is always fixed, information E4 may not include information indicating the size of the first region. If the shape of the first region is always fixed, information E4 may not include information indicating the shape of the first region.

[0299] exist Figure 23 In the example shown, the location K1, the type of affected area L1, the patient's age M1, the patient's gender N1, and the first region I8 are associated. Figure 23 In the example shown, location K2, type of affected area L2, patient age M2, patient gender N1, and region I9 are associated. Figure 23 In the example shown, location K3, type of affected area L3, patient age M3, patient gender N2, and region I10 are associated.

[0300] The processor 41 extracts information of the 1st region corresponding to the information input to the operation section 22 from the region information. For example, in a case where the site K2, the kind L2 of the affected part, the age M2 of the patient, and the gender N1 of the patient are input to the operation section 22, the processor 41 extracts information of the 1st region I9. The processor 41 judges the position of the 1st region on the basis of the extracted information. The processor 41 sets a region after the 1st region is removed as a processing region.

[0301] The information that the observer can input is not limited to Figure 23 the information shown in the drawing. It can also be that the observer can input items not shown in the above description.

[0302] The processor 41 judges the position of the 1st region on the basis of various information input to the operation section 22, and sets a processing region on the basis of the position. The processor 41 can set a processing region that is suitable for the situation of observation. Even in a case where the observer is not accustomed to the operation of the electronic endoscope 2 or in a case where the observer is not accustomed to the treatment using the treatment instrument 13, the processor 41 can process the image in a manner that the observer can easily perform the treatment.

[0303] (7th Embodiment)

[0304] A 7th embodiment of the present application will be described. The image processing apparatus 4 of the 7th embodiment has two image processing modes. The image processing apparatus 4 operates in either one of a fatigue reduction mode (1st mode) and a normal mode (2nd mode). The processor 41 selects one of the fatigue reduction mode and the normal mode in a mode selection step. In the following example, the processor 41 selects one of the fatigue reduction mode and the normal mode on the basis of information input to the operation section 22 by the observer.

[0305] The processing performed by the processor 41 will be described with reference to Figure 24 . Figure 24 The steps of the processing performed by the processor 41 will be shown. The description of the processing common to the processing shown in Figure 8 will be omitted. For example, when the power of the endoscope apparatus 1 is turned on, the processor 41 performs the processing shown in Figure 24 .

[0306] The processor 41 selects the normal mode (step S140 (mode selection step)). Information indicating the normal mode is stored in a memory not shown in Figure 3 . The processor 41 performs the processing prescribed in the normal mode in accordance with the information.

[0307] After the step S140, the processor 41 acquires the 1st image and the 2nd image from the imaging element 12 (step S145 (image acquisition step)).

[0308] After the step S145, the processor 41 outputs the first image and the second image taken in the step S145 to the monitor 5 (step S150 (second image output step)). The processor 41 can also output the first image and the second image to the receiving device 6 shown in FIG. 1. In a case where the processor 41 selects the normal mode in the step S140, the step S145 and the step S150 are executed. The processor 41 does not change the parallax amount of the processing region. Figure 4 The order in which the step S140 and the step S145 are executed can be different from the order shown in FIG. 1. That is, the step S140 can be executed after the step S145 is executed.

[0309] The order in which the step S160, the step S100, and the step S105 are executed can be different from the order shown in FIG. 1. That is, the step S160 and the step S100 can be executed after the step S105 is executed. Figure 24 The observer can input information indicating a change of the image processing mode by operating the operation section 22. For example, when the insertion section 21 is inserted into the body and the tip section 10 is disposed in the vicinity of the observation object, the observer inputs information indicating a change of the image processing mode to the operation section 22 in order to start the treatment. The operation section 22 outputs the input information to the processor 41.

[0310] After the step S150, the processor 41 monitors the operation section 22 and determines whether a change of the image processing mode is instructed (step S155). In a case where information indicating a change of the image processing mode is input to the operation section 22, the processor 41 determines that a change of the image processing mode is instructed. In a case where information indicating a change of the image processing mode is not input to the operation section 22, the processor 41 determines that a change of the image processing mode is not instructed.

[0311] In a case where the processor 41 determines that a change of the image processing mode is not instructed in the step S155, the step S145 is executed. In a case where the processor 41 determines that a change of the image processing mode is instructed in the step S155, the processor 41 selects the fatigue reduction mode (step S160 (mode selection step)). Information indicating the fatigue reduction mode is stored in a memory not shown in FIG. 1. The processor 41 executes processing prescribed in the fatigue reduction mode in accordance with the information. After the step S160, the step S100 is executed. In a case where the processor 41 selects the fatigue reduction mode in the step S160, the step S100, the step S105, the step S110, and the step S115 are executed.

[0312] Figure 3 The order in which the step S160, the step S100, and the step S105 are executed can be different from the order shown in FIG. 1. That is, the step S160 and the step S100 can be executed after the step S105 is executed.

[0313] The order in which the step S160, the step S100, and the step S105 are executed can be different from the order shown in FIG. 1. That is, the step S160 and the step S100 can be executed after the step S105 is executed. Figure 24 The order in which the step S160, the step S100, and the step S105 are executed can be different from the order shown in FIG. 1. That is, the step S160 and the step S100 can be executed after the step S105 is executed.​

[0314] For example, when the treatment using the treatment instrument 13 ends, the observer inputs information indicating a change of the image processing mode to the operation section 22 in order to pull out the insertion section 21. The operation section 22 outputs the input information to the processor 41.

[0315] After the step S115, the processor 41 monitors the operation section 22, and determines whether a change of the image processing mode is instructed (step S165). The step S165 is the same as the step S155.

[0316] In a case where the processor 41 determines that a change of the image processing mode is not instructed in the step S165, the step S105 is executed. In a case where the processor 41 determines that a change of the image processing mode is instructed in the step S165, the step S140 is executed. The processor 41 selects the normal mode in the step S140.

[0317] In the example described above, the observer instructs the image processing apparatus 4 of a change of the image processing mode by operating the operation section 22. The observer can instruct the image processing apparatus 4 of a change of the image processing mode by using a method different from the method described above. For example, the observer can instruct the image processing apparatus 4 of a change of the image processing mode by using a sound input.

[0318] Figure 24 The step S100, the step S105, and the step S110 illustrated can be replaced with Figure 15 the step S105 and the step S110a illustrated. Figure 24 The step S100 and the step S105 illustrated can be replaced with Figure 18 the step S105, the step S120, and the step S100a illustrated. Figure 24 The step S100 illustrated can be replaced with Figure 19 the step S125 illustrated. Figure 24 The step S100 and the step S105 illustrated can be replaced with Figure 22 the step S105, the step S130, and the step S100b illustrated.

[0319] In a case where the processor 41 selects the fatigue reduction mode, the processor 41 executes a change of the parallax amount of the processing region. Therefore, fatigue of the observer's eyes is reduced. In a case where the processor 41 selects the normal mode, the processor 41 does not execute a change of the parallax amount of the processing region. Therefore, the observer can use an image that the observer is accustomed to watching for observation. The processor 41 changes the parallax amount of the processing region only when a change of the parallax amount of the processing region is required. Therefore, the load of the processor 41 is reduced.

[0320] (1st Modification of the 7th Embodiment)

[0321] A first modification of the seventh embodiment of the present application will be described. The processor 41 automatically selects one of the fatigue reduction mode and the normal mode in the mode selection step.

[0322] The endoscope apparatus 1 has two display modes. The endoscope apparatus 1 displays an image in one of a 3D display mode and a 2D display mode. The 3D display mode is a mode in which a stereoscopic image (three-dimensional image) is displayed on the monitor 5. The 2D display mode is a mode in which a two-dimensional image is displayed on the monitor 5. In a case where the endoscope apparatus 1 is operating in the 3D display mode, the processor 41 selects the fatigue reduction mode. In a case where the endoscope apparatus 1 is operating in the 2D display mode, the processor 41 selects the normal mode.

[0323] The processing performed by the processor 41 will be described with reference to Figure 25 The steps of the processing performed by the processor 41 will be shown. The description of the processing common to that shown in Figure 25 will be omitted. For example, when the power of the endoscope apparatus 1 is turned on, the processor 41 performs the processing shown in Figure 24 . At this time, the endoscope apparatus 1 starts operating in the 2D display mode. Figure 25

[0324] After the step S145, the processor 41 outputs the first image taken in the step S145 to the monitor 5 (step S150a). The monitor 5 displays the first image.

[0325] The processor 41 can also output the second image to the monitor 5 in the step S150a. In this case, the monitor 5 displays the second image. The processor 41 can also output the first image and the second image to the monitor 5 in the step S150a. In this case, the monitor 5 displays the first image and the second image side by side in the horizontal direction or the vertical direction, for example.

[0326] In a case where the imaging element 12 outputs the first image and the second image in sequence, the processor 41 can also take the first image in the step S145 and output the first image to the monitor 5 in the step S150a. Alternatively, the processor 41 can also take the second image in the step S145 and output the second image to the monitor 5 in the step S150a.

[0327] The observer can input information indicating a change of the display mode by operating the operation section 22. For example, when the insertion section 21 is inserted into the body and the distal end section 10 is disposed in the vicinity of the observation object, the observer inputs information indicating a change of the display mode to the operation section 22 in order to start observation using a stereoscopic image. The operation section 22 outputs the input information to the processor 41.

[0328] ​After the step S150a, the processor 41 determines whether the display mode is changed to the 3D mode (step S155a). In a case where the information indicating the change of the display mode is input to the operation section 22, the processor 41 determines that the display mode is changed to the 3D mode. In a case where the information indicating the change of the display mode is not input to the operation section 22, the processor 41 determines that the display mode is not changed to the 3D mode.

[0329] In a case where the processor 41 determines that the display mode is not changed to the 3D mode in the step S155a, the step S145 is executed. In a case where the processor 41 determines that the display mode is changed to the 3D mode in the step S155a, the step S160 is executed.

[0330] For example, at the time of the end of the treatment using the treatment instrument 13, the observer inputs the information indicating the change of the display mode to the operation section 22 in order to start the observation using the two-dimensional image. The operation section 22 outputs the input information to the processor 41.

[0331] After the step S115, the processor 41 determines whether the display mode is changed to the 2D mode (step S165a). In a case where the information indicating the change of the display mode is input to the operation section 22, the processor 41 determines that the display mode is changed to the 2D mode. In a case where the information indicating the change of the display mode is not input to the operation section 22, the processor 41 determines that the display mode is not changed to the 2D mode.

[0332] In a case where the processor 41 determines that the display mode is not changed to the 2D mode in the step S165a, the step S105 is executed. In a case where the processor 41 determines that the display mode is changed to the 2D mode in the step S165a, the step S140 is executed.

[0333] In the above-described example, the observer instructs the change of the display mode to the endoscope device 1 by operating the operation section 22. The observer can also instruct the change of the display mode to the endoscope device 1 by using a method different from the above-described method. For example, the observer can also instruct the change of the display mode to the endoscope device 1 by using a voice input.

[0334] Figure 25 The illustrated step S100, the step S105, and the step S110 can be replaced with Figure 15 the illustrated step S105 and the step S110a. Figure 25 The illustrated step S100 and the step S105 can be replaced with Figure 18 the illustrated step S105, the step S120, and the step S100a. Figure 25 The illustrated step S100 can be replaced with Figure 19 the illustrated step S125.Figure 25 The step S100 and the step S105 shown can be replaced with Figure 22 The step S105, the step S130, and the step S100b shown.

[0335] The processor 41 selects one of the fatigue reduction mode and the normal mode based on the setting of the display mode. Therefore, the processor 41 can switch the image processing mode at an appropriate timing.

[0336] (2nd Modification of 7th Embodiment)

[0337] A 2nd modification of the 7th embodiment of the present application will be described. Another method of switching the fatigue reduction mode and the normal mode will be described.

[0338] The processor 41 detects the state of the movement of the imaging element 12 in the 1st movement detection step. The processor 41 selects one of the fatigue reduction mode and the normal mode based on the state of the movement of the imaging element 12 in the mode selection step.

[0339] In a case where the normal mode is selected, the observer can observe the image habitually watched. The observer needs the fatigue reduction mode when performing the treatment using the treatment instrument 13 that causes the eye fatigue. The processor 41 selects the fatigue reduction mode only when the fatigue reduction mode is needed. When the insertion portion 21 is fixed in the body, the likelihood that the observer performs the treatment using the treatment instrument 13 is high. When the insertion portion 21 is fixed in the body, the imaging element 12 is relatively stationary with respect to the subject. When the imaging element 12 is stationary, the processor 41 switches the image processing mode from the normal mode to the fatigue reduction mode.

[0340] After the treatment using the treatment instrument 13 is completed, the observer is likely to pull out the insertion portion 21. Therefore, the insertion portion 21 is likely to move in the body. When the insertion portion 21 moves in the body, the imaging element 12 moves relatively with respect to the subject. When the imaging element 12 starts to move, the processor 41 switches the image processing mode from the fatigue reduction mode to the normal mode.

[0341] The processing performed by the processor 41 will be described with reference to Figure 26 The processing performed by the processor 41 will be described with reference to Figure 26 Steps of the processing performed by the processor 41 are shown. The description of the processing common to the processing shown in Figure 24 The description of the processing common to the processing shown in

[0342] After step S145, the processor 41 detects the state of the movement of the image pickup element 12 (step S170 (1st movement detection step)). Step S170 is described in detail. For example, the processor 41 calculates the amount of movement between the 1st image or the 2nd image of consecutive 2 frames. This amount of movement indicates the state of the movement of the image pickup element 12. In the case where the image pickup element 12 moves, this amount of movement is large. In the case where the image pickup element 12 is stationary, this amount of movement is small. The processor 41 can also calculate the total of the amounts of movement within a prescribed time. After step S170, step S150 is executed.

[0343] The order in which step S170 and step S150 are executed can also be different from the order shown in Figure 26 . That is, step S170 can also be executed after step S150 is executed.

[0344] After step S150, the processor 41 determines whether the image pickup element 12 is stationary (step S175). In the case where the amount of movement calculated in step S170 is smaller than a prescribed amount, the processor 41 determines that the image pickup element 12 is stationary. In this case, the possibility that the treatment using the treatment instrument 13 is performed is high. In the case where the amount of movement calculated in step S170 is the prescribed amount or more, the processor 41 determines that the image pickup element 12 moves. In this case, the possibility that the treatment using the treatment instrument 13 is not performed is high. For example, the prescribed amount is a small positive value that can distinguish between the state where the image pickup element 12 is stationary and the state where the image pickup element 12 moves. The processor 41 can determine that the image pickup element 12 is stationary only in the case where the state where the amount of movement calculated in step S170 is the prescribed amount or more continues for a prescribed time or more.

[0345] In the case where the processor 41 determines that the image pickup element 12 moves in step S175, step S145 is executed. In the case where the processor 41 determines that the image pickup element 12 is stationary in step S175, step S160 is executed.

[0346] After step S105, the processor 41 detects the state of the movement of the image pickup element 12 (step S180 (1st movement detection step)). Step S180 is the same as step S170. After step S180, step S110 is executed.

[0347] The order in which step S180 and step S110 are executed can also be different from the order shown in Figure 26 . That is, step S180 can also be executed after step S110 is executed. The order in which step S180 and step S115 are executed can also be different from the order shown in Figure 26 . That is, step S180 can also be executed after step S115 is executed.

[0348] After the step S115, the processor 41 determines whether the imaging element 12 is moving (step S185). In a case where the movement amount calculated in the step S180 is greater than a prescribed amount, the processor 41 determines that the imaging element 12 is moving. In this case, the possibility that the treatment using the treatment instrument 13 is not performed is high. In a case where the movement amount calculated in the step S180 is equal to or less than the prescribed amount, the processor 41 determines that the imaging element 12 is stationary. In this case, the possibility that the treatment using the treatment instrument 13 is performed is high. The prescribed amount used in the step S185 is the same as the prescribed amount used in the step S175, for example.

[0349] In a case where the processor 41 determines that the imaging element 12 is stationary in the step S185, the step S105 is executed. In a case where the processor 41 determines that the imaging element 12 is moving in the step S185, the step S140 is executed.

[0350] In the example described above, the processor 41 detects the state of the movement of the imaging element 12 on the basis of at least one of the first image and the second image. The processor 41 can also detect the state of the movement of the imaging element 12 by using a method different from the method described above. For example, an acceleration sensor that detects the acceleration of the front end portion 10 can be arranged inside the front end portion 10. The processor 41 can also detect the state of the movement of the imaging element 12 on the basis of the acceleration detected by the acceleration sensor. The insertion portion 21 is sometimes inserted into the body from a mouthpiece arranged in the mouth of a patient. An encoder that detects the movement of the insertion portion 21 can also be arranged in the mouthpiece or the like into which the insertion portion 21 is inserted. The processor 41 can also detect the state of the movement of the imaging element 12 on the basis of the movement of the insertion portion 21 detected by the encoder.

[0351] Figure 26 The step S100, the step S105, and the step S110 illustrated above can be replaced with Figure 15 the step S105 and the step S110a illustrated above. Figure 26 The step S100 and the step S105 illustrated above can be replaced with Figure 18 the step S105, the step S120, and the step S100a illustrated above. Figure 26 The step S100 illustrated above can be replaced with Figure 19 the step S125 illustrated above. Figure 26 The step S100 and the step S105 illustrated above can be replaced with Figure 22 the step S105, the step S130, and the step S100b illustrated above.

[0352] The processor 41 selects one of the fatigue reduction mode and the normal mode based on a state of movement of the imaging element 12. Thus, the processor 41 can switch the image processing mode at an appropriate timing.

[0353] (3rd Modification of the 7th Embodiment)

[0354] The 3rd modification of the 7th embodiment of the present application will be described. Another method of switching the fatigue reduction mode and the normal mode will be described.

[0355] The processor 41 searches for the treatment instrument 13 in at least one of the 1st image and the 2nd image in the search step. In a case where the processor 41 has detected the treatment instrument 13 from at least one of the 1st image and the 2nd image in the search step, the processor 41 selects the fatigue reduction mode in the mode selection step. In a case where the processor 41 has not detected the treatment instrument 13 from at least one of the 1st image and the 2nd image in the search step, the processor 41 selects the normal mode in the mode selection step.

[0356] The insertion portion 21 needs to be moved at times when the treatment is performed by the treatment instrument 13. Thus, the treatment can be continuously performed even in a case where the imaging element 12 moves. The processor 41 switches the image processing mode depending on whether the treatment instrument 13 is reflected in the 1st image or the 2nd image.

[0357] The processing performed by the processor 41 will be described with reference to Figure 27 . Figure 27 The steps of the processing performed by the processor 41 will be described. The description of the processing common to the processing shown in Figure 24 will be omitted.

[0358] A mark is attached to a front end region including a front end of the treatment instrument 13 in the treatment instrument 13. The shape of the mark can be any shape. The mark can also be a character or a symbol, or the like. Two or more marks can also be attached.

[0359] After the step S145, the processor 41 searches for the treatment instrument 13 in at least one of the 1st image and the 2nd image (step S190 (search step)). For example, the processor 41 searches for the mark attached to the treatment instrument 13 in the 1st image in the step S190. The processor 41 can also search for the mark in the 2nd image. The step S150 is performed after the step S190.

[0360] The order in which the step S190 and the step S150 are performed can also be different from the order shown in Figure 27 . That is, the step S190 can also be performed after the step S150 is performed.

[0361] After the step S150, the processor 41 determines whether the treatment instrument 13 is detected in the image (step S195). For example, in a case where the marker attached to the treatment instrument 13 is reflected in the first image, the processor 41 determines that the treatment instrument 13 is detected in the image. In this case, the possibility that the treatment using the treatment instrument 13 is prepared or the treatment is performed is high.

[0362] Also, in a case where the marker attached to the treatment instrument 13 is reflected in the second image, the processor 41 determines that the treatment instrument 13 is detected in the image. Also, in a case where the marker is reflected in the first image and the second image, the processor 41 determines that the treatment instrument 13 is detected in the image.

[0363] In a case where the marker attached to the treatment instrument 13 is not reflected in the first image, the processor 41 determines that the treatment instrument 13 is not detected in the image. In this case, the possibility that the treatment instrument 13 is not used is high. Also, in a case where the marker attached to the treatment instrument 13 is not reflected in the second image, the processor 41 can also determine that the treatment instrument 13 is not detected in the image. Also, in a case where the marker is not reflected in the first image and the second image, the processor 41 determines that the treatment instrument 13 is not detected in the image.

[0364] In a case where the processor 41 determines that the treatment instrument 13 is not detected in the image in the step S195, the step S145 is executed. In a case where the processor 41 determines that the treatment instrument 13 is detected in the image in the step S195, the step S160 is executed.

[0365] After the step S105, the processor 41 searches for the treatment instrument 13 in at least one of the first image and the second image (step S200 (searching step)). The step S200 is the same as the step S190. After the step S200, the step S110 is executed.

[0366] After the step S115, the processor 41 determines whether the treatment instrument 13 is detected in the image (step S205). The step S205 is the same as the step S195. After the treatment using the treatment instrument 13 is ended, the observer returns the treatment instrument 13 to the insertion portion 21 in many cases. Therefore, the treatment instrument 13 is not reflected in the image.

[0367] In a case where the processor 41 determines in step S205 that the treatment instrument 13 is detected in the image, step S105 is executed. In this case, the possibility that the treatment using the treatment instrument 13 is being performed is high. Therefore, the processor 41 proceeds with the processing in the fatigue reduction mode. In a case where the processor 41 determines in step S205 that the treatment instrument 13 is not detected in the image, step S140 is executed. In this case, the possibility that the treatment using the treatment instrument 13 has ended is high. Therefore, the processor 41 starts the processing in the normal mode in step S140.

[0368] In the above-described example, the processor 41 searches for the mark attached to the treatment instrument 13 in at least one of the first image and the second image. The front end region of the treatment instrument 13 can also have a prescribed color. The prescribed color is different from the color of the subject such as an organ or a blood vessel. The processor 41 can also search for the prescribed color in at least one of the first image and the second image. A prescribed pattern can be attached to the front end region of the treatment instrument 13. The processor 41 can also search for the pattern attached to the treatment instrument 13 in at least one of the first image and the second image. The processor 41 can also search for the shape of the forceps 130 in at least one of the first image and the second image.

[0369] Figure 27 The illustrated step S100, step S105, and step S110 can be replaced with Figure 15 the illustrated step S105 and step S110a. Figure 27 The illustrated step S100 and step S105 can be replaced with Figure 18 the illustrated step S105, step S120, and step S100a. Figure 27 The illustrated step S100 can be replaced with Figure 19 the illustrated step S125. Figure 27 The illustrated step S100 and step S105 can be replaced with Figure 22 the illustrated step S105, step S130, and step S100b.

[0370] The processor 41 selects one of the fatigue reduction mode and the normal mode based on the state of the treatment instrument 13 in at least one of the first image and the second image. When the treatment using the treatment instrument 13 is being performed, the processor 41 can reliably select the fatigue reduction mode.

[0371] (4th Modification of the 7th Embodiment)

[0372] A 4th modification of the 7th embodiment of the present application will be described. Another method of switching the fatigue reduction mode and the normal mode will be described.

[0373] The processor 41 calculates a distance between a reference position of one of the first image and the second image and the treatment instrument 13 in the distance calculation step. The processor 41 selects one of the fatigue reduction mode and the normal mode based on the distance in the mode selection step.

[0374] When the fatigue reduction mode is set, the optical image of the treatment instrument 13 is displayed at a position on the inner side than the actual position in the stereoscopic image. Therefore, the observer sometimes cannot easily determine the actual position of the treatment instrument 13. In the case where the fatigue reduction mode is set, the observer sometimes cannot approach the observation object with the treatment instrument 13. In the case where the treatment instrument 13 sufficiently approaches the observation object, the processor 41 selects the fatigue reduction mode.

[0375] The processing performed by the processor 41 will be described with reference to Figure 28 . Figure 28 The steps of the processing performed by the processor 41 will be described. The description of the processing common to the processing shown in Figure 24 will be omitted.

[0376] A mark is attached to the front end region including the front end of the treatment instrument 13 in the treatment instrument 13. The shape of the mark is irrelevant. The mark can also be a letter or a symbol, or the like. Two or more marks can also be attached.

[0377] After step S145, the processor 41 calculates a distance between a reference position in the first image or the second image and the treatment instrument 13 (step S210 (distance calculation step)). The reference position is, for example, the center of the first image or the second image. The processor 41 detects the mark attached to the treatment instrument 13 in the first image in step S210, and calculates a two-dimensional distance between the reference position of the first image and the mark. The processor 41 can also detect the mark attached to the treatment instrument 13 in the second image in step S210, and calculate a two-dimensional distance between the reference position of the second image and the mark. After step S210, step S150 is executed.

[0378] The order in which step S210 and step S150 are executed can also be different from the order shown in Figure 28 . That is, step S210 can also be executed after step S150 is executed.

[0379] After the step S150, the processor 41 determines whether the treatment instrument 13 is approaching the observation object (step S215). For example, in a case where the distance calculated in the step S210 is smaller than a prescribed value, the processor 41 determines that the treatment instrument 13 is approaching the observation object. In this case, the possibility that the treatment using the treatment instrument 13 is performed is high. In a case where the distance calculated in the step S210 is the prescribed value or more, the processor 41 determines that the treatment instrument 13 is not approaching the observation object. In this case, the possibility that the treatment instrument 13 is not used is high. For example, the prescribed value is a small positive value that can distinguish between the state where the imaging element 12 approaches the observation object and the state where the imaging element 12 is away from the observation object.

[0380] In a case where the treatment instrument 13 is not reflected in the first image and the second image, the processor 41 cannot calculate the distance in the step S210. In this case, the processor 41 can also determine that the treatment instrument 13 is not approaching the observation object in the step S215.

[0381] In a case where the processor 41 determines that the treatment instrument 13 is not approaching the observation object in the step S215, the step S145 is executed. In a case where the processor 41 determines that the treatment instrument 13 is approaching the observation object in the step S215, the step S160 is executed.

[0382] After the step S105, the processor 41 calculates the distance between the reference position in the first image or the second image and the treatment instrument 13 (step S220 (distance calculation step)). The step S220 is the same as the step S210. The step S110 is executed after the step S220.

[0383] After the step S115, the processor 41 determines whether the treatment instrument 13 is away from the observation object (step S225). For example, in a case where the distance calculated in the step S220 is larger than a prescribed value, the processor 41 determines that the treatment instrument 13 is away from the observation object. In this case, the possibility that the treatment using the treatment instrument 13 is not performed is high. In a case where the distance calculated in the step S220 is the prescribed value or less, the processor 41 determines that the treatment instrument 13 is not away from the observation object. In this case, the possibility that the treatment using the treatment instrument 13 is being performed is high. For example, the prescribed value used in the step S225 is the same as the prescribed value used in the step S215.

[0384] In a case where the treatment instrument 13 is not reflected in the first image and the second image, the processor 41 cannot calculate the distance in the step S220. In this case, the processor 41 can also determine that the treatment instrument 13 is away from the observation object in the step S225.

[0385] When the processor 41 determines in step S225 that the treatment instrument 13 is not distanced from the observation object, step S105 is executed. When the processor 41 determines in step S225 that the treatment instrument 13 is distanced from the observation object, step S140 is executed.

[0386] In the example described above, the processor 41 detects the mark attached to the treatment instrument 13 in the first image or the second image. Further, the processor 41 calculates the distance between the region in which the mark is detected and the reference position.

[0387] The front end region of the treatment instrument 13 can also have a prescribed color. The prescribed color is different from the color of the subject such as an internal organ or a blood vessel. The processor 41 can also detect the prescribed color in the first image or the second image. The processor 41 can also calculate the distance between the region in which the prescribed color is detected and the reference position.

[0388] A prescribed pattern can also be attached to the front end region of the treatment instrument 13. The processor 41 can also detect the pattern attached to the treatment instrument 13 in the first image or the second image. The processor 41 can also calculate the distance between the region in which the pattern is detected and the reference position.

[0389] The processor 41 can also detect the shape of the forceps 130 in the first image or the second image. The processor 41 can also calculate the distance between the front end of the forceps 130 and the reference position.

[0390] Figure 28 The illustrated step S100, step S105, and step S110 can be replaced with Figure 15 the illustrated step S105 and step S110a. Figure 28 The illustrated step S100 and step S105 can be replaced with Figure 18 the illustrated step S105, step S120, and step S100a. Figure 28 The illustrated step S100 can be replaced with Figure 19 the illustrated step S125. Figure 28 The illustrated step S100 and step S105 can be replaced with Figure 22 the illustrated step S105, step S130, and step S100b.

[0391] The processor 41 selects one of the fatigue reduction mode and the normal mode based on the distance between the reference position of one of the first image and the second image and the treatment instrument 13. When the treatment instrument 13 approaches the observation object, the processor 41 can reliably select the fatigue reduction mode.

[0392] (5th Modification of the 7th Embodiment)

[0393] A fifth variation of the seventh embodiment of the present invention will be described. Other methods for switching between fatigue reduction mode and normal mode will be described.

[0394] Figure 29 The surrounding structure of the image processing device 4 is shown. (Omitted) Figure 3 The description of the same structure as shown.

[0395] The endoscope device 1 also includes an encoder 16. The encoder 16 is disposed inside the insertion portion 21. The encoder 16 detects the movement of the sheath 131 along the axial direction of the insertion portion 21. For example, the encoder 16 detects the speed of the sheath 131 by detecting the distance the sheath 131 moves at predetermined time intervals. The encoder 16 outputs the detected speed to the processor 41.

[0396] In the second motion detection step, the processor 41 detects the motion state of the treatment device 13. In the mode selection step, the processor 41 selects either a fatigue relief mode or a normal mode based on the motion state of the treatment device 13.

[0397] Reference Figure 30 This is to explain the processing performed by processor 41. Figure 30 This shows the steps of the processing performed by processor 41. (Omitted) Figure 24 The process described is the same as the one shown. For example, when the treatment device 13 is inserted into the channel within the insertion part 21, the processor 41 executes... Figure 30 The process is shown. The processor 41 is able to detect the insertion of the treatment device 13 into the channel based on the speed of the sheath 131 detected by the encoder 16.

[0398] After step S145, processor 41 obtains the speed of sheath 131 from encoder 16 (step S230 (second motion detection step)). After step S230, step S150 is executed.

[0399] The order in which steps S230 and S145 are executed can also be the same as... Figure 30 The order shown is different. That is, step S145 can also be executed after step S230. The order in which steps S230 and S150 are executed can also be different. Figure 30 The order shown is different. That is, step S230 can also be performed after step S150.

[0400] After the step S150, the processor 41 determines whether the treatment instrument 13 is stationary (step S235). In a case where the speed of the sheath 131 taken in the step S230 is less than a prescribed value, the processor 41 determines that the treatment instrument 13 is stationary. In this case, the treatment instrument 13 is sufficiently close to the observation object, and the possibility that the treatment is performed is high. In a case where the speed of the sheath 131 taken in the step S230 is the prescribed value or more, the processor 41 determines that the treatment instrument 13 is in motion. In this case, the possibility that the treatment using the treatment instrument 13 is not performed is high. The prescribed value is, for example, a small positive value that can distinguish between the state where the treatment instrument 13 is stationary and the state where the treatment instrument 13 is in motion.

[0401] In a case where the processor 41 determines that the treatment instrument 13 is in motion in the step S235, the step S145 is executed. In a case where the processor 41 determines that the treatment instrument 13 is stationary in the step S235, the step S160 is executed.

[0402] After the step S105, the processor 41 takes the speed of the sheath 131 from the encoder 16 (step S240 (2nd motion detection step)). The step S240 is the same as the step S230. After the step S240, the step S110 is executed.

[0403] The order in which the step S240 and the step S105 are executed can also be different from the order illustrated in Figure 30 . That is, the step S105 can also be executed after the step S240 is executed. The order in which the step S240 and the step S110 are executed can also be different from the order illustrated in Figure 30 . That is, the step S240 can also be executed after the step S110 is executed. The order in which the step S240 and the step S115 are executed can also be different from the order illustrated in Figure 30 . That is, the step S240 can also be executed after the step S115 is executed.

[0404] After the step S115, the processor 41 determines whether the treatment instrument 13 is in motion (step S245). In a case where the speed of the sheath 131 taken in the step S240 is greater than a prescribed value, the processor 41 determines that the treatment instrument 13 is in motion. In this case, the possibility that the treatment using the treatment instrument 13 is not performed is high. In a case where the speed of the sheath 131 taken in the step S240 is the prescribed value or less, the processor 41 determines that the treatment instrument 13 is stationary. In this case, the possibility that the treatment using the treatment instrument 13 is performed is high. The prescribed value used in the step S245 is, for example, the same as the prescribed value used in the step S235.

[0405] In a case where the processor 41 determines in step S245 that the treatment instrument 13 is stationary, step S105 is executed. In a case where the processor 41 determines in step S245 that the treatment instrument 13 is in motion, step S140 is executed.

[0406] In the example described above, the processor 41 detects the state of motion of the treatment instrument 13 based on the speed of the sheath 131 detected by the encoder 16. The processor 41 can also detect the state of motion of the treatment instrument 13 by using a method different from the method described above. For example, the processor 41 can also detect the treatment instrument 13 from at least one of the first image and the second image. The processor 41 can also detect the state of motion of the treatment instrument 13 by calculating the amount of motion of the treatment instrument 13 over two or more consecutive frames.

[0407] Figure 30 The steps S100, S105, and S110 illustrated can also be replaced with Figure 15 the steps S105 and S110a illustrated. Figure 30 The steps S100 and S105 illustrated can also be replaced with Figure 18 the steps S105, S120, and S100a illustrated. Figure 30 The step S100 illustrated can also be replaced with Figure 19 the step S125 illustrated. Figure 30 The steps S100 and S105 illustrated can also be replaced with Figure 22 the steps S105, S130, and S100b illustrated.

[0408] The processor 41 selects one of the fatigue reduction mode and the normal mode based on the state of motion of the treatment instrument 13. Therefore, the processor 41 can switch the image processing mode at an appropriate timing. Since the encoder 16 detects the speed of the sheath 131, the processor 41 does not need to perform image processing in order to detect the treatment instrument 13. Therefore, the load on the processor 41 is reduced.

[0409] (6th Modification of the 7th Embodiment)

[0410] A 6th modification of the 7th embodiment of the present application is described. Another method of switching the fatigue reduction mode and the normal mode is described.

[0411] When the fatigue alleviation mode is set, the optical image of the treatment instrument 13 is displayed in the stereoscopic image at a position on the inner side than the actual position. Therefore, the observer sometimes does not easily determine the actual position of the treatment instrument 13. In the case where the fatigue alleviation mode is set, the observer sometimes does not easily approach the treatment instrument 13 to the observation object. When the observer approaches the treatment instrument 13 to the observation object, the image processing mode can also be the normal mode. On the other hand, when the treatment instrument 13 is distanced from the observation object, the easiness of observation of the image does not easily affect the operation. At this time, the image processing mode can also be the fatigue alleviation mode. In the following example, the condition for switching the image processing mode is different when the treatment instrument 13 is approached to the observation object and when the treatment instrument 13 is distanced from the observation object.

[0412] The processing performed by the processor 41 will be described with reference to Figure 31 Figure 31 The steps of the processing performed by the processor 41 will be described. The description of the processing common to the processing shown in Figure 24 will be omitted. For example, when the power of the endoscope device 1 is turned on, the processor 41 performs the processing shown in Figure 31 At this time, the endoscope device 1 starts to operate in the 2D display mode.

[0413] After the step S145, the processor 41 calculates the distance between the reference position in the first image or the second image and the treatment instrument 13 (step S210). Figure 31 The step S210 shown in Figure 28 is the same as the step S210 shown in

[0414] After the step S150, the processor 41 determines whether the treatment instrument 13 approaches the observation object (step S215). Figure 31 The step S215 shown in Figure 28 is the same as the step S215 shown in

[0415] In the case where the processor 41 determines that the treatment instrument 13 does not approach the observation object in the step S215, the step S145 is executed. In the case where the processor 41 determines that the treatment instrument 13 approaches the observation object in the step S215, the step S160 is executed.

[0416] After the observer approaches the treatment instrument 13 to the observation object, the observer operates the operation section 22 to change the display mode to the 3D mode. Thereafter, the observer performs the treatment using the treatment instrument 13. After the treatment is completed, the observer operates the operation section 22 to change the display mode to the 2D mode.

[0417] After the step S115, the processor 41 determines whether the display mode is changed to the 2D mode (step S165a). Figure 31 ​The step S165a shown is the same as Figure 25 The step S165a shown is the same as

[0418] In a case where the processor 41 determines in the step S165a that the display mode is not changed to the 2D mode, the step S105 is executed. In a case where the processor 41 determines in the step S165a that the display mode is changed to the 2D mode, the step S140 is executed.

[0419] Figure 31 The step S100, the step S105, and the step S110 shown can be replaced with Figure 15 The step S105 and the step S110a shown. Figure 31 The step S100 and the step S105 shown can be replaced with Figure 18 The step S105, the step S120, and the step S100a shown. Figure 31 The step S100 shown can be replaced with Figure 19 The step S125 shown. Figure 31 The step S100 and the step S105 shown can be replaced with Figure 22 The step S105, the step S130, and the step S100b shown.

[0420] When the treatment instrument 13 approaches the observation object, the processor 41 selects the fatigue reduction mode. When the display mode is changed from the 3D mode to the 2D mode, the processor 41 selects the normal mode. Thus, the easiness of the operation of the treatment instrument 13 and the reduction of the fatigue of the observer's eyes are well balanced.

[0421] (8th Embodiment)

[0422] The 8th embodiment of the present application will be described. The processor 41 processes the processing region so that the optical image of the subject in the processing region is blurred in the stereoscopic image displayed based on the first image and the second image.

[0423] The processing performed by the processor 41 will be described with reference to Figure 32 The processing performed by the processor 41 will be described with reference to Figure 32 Steps of the processing performed by the processor 41 are shown. The description of the processing common to the processing shown in Figure 8 The description of the processing common to the processing shown in

[0424] After the step S105, the processor 41 blurs the processing region of at least one of the first image and the second image (step S250 (image processing step)). After the step S250, the step S115 is executed.

[0425] Step S250 is explained in detail. For example, processor 41 averages the colors of all pixels contained in the processing area of ​​the first image. Specifically, processor 41 calculates the average of the signal values ​​of two or more pixels surrounding the target pixel, and replaces the signal value of the target pixel with this average. Processor 41 performs this processing on all pixels contained in the processing area of ​​the first image. Processor 41 averages the colors of all pixels contained in the processing area of ​​the second image by performing the same processing as described above.

[0426] Processor 41 may, after averaging the colors of each pixel contained in the processing area of ​​the first image, replace the signal values ​​of each pixel contained in the processing area of ​​the second image with the signal values ​​of each pixel contained in the processing area of ​​the first image.

[0427] Figure 15 The step S110a shown can also be replaced by step S250. Figure 18 , Figure 19 , Figure 22 , Figure 24 , Figure 25 , Figure 26 , Figure 27 , Figure 28 , Figure 30 as well as Figure 31 The step S110 shown can also be replaced by step S250.

[0428] After the processor 41 blurs the processing area, it becomes less difficult for the observer to focus on the optical image of the processing device 13 projected onto the processing area. Therefore, eye strain is reduced. The workload on the processor 41 is reduced compared to when the processor 41 changes the amount of parallax.

[0429] (A variation of the eighth embodiment)

[0430] A variation of the eighth embodiment of the present invention will be described. The processor 41 performs mosaic processing on the processing area.

[0431] Reference Figure 33 This is to explain the processing performed by processor 41. Figure 33 This shows the steps of the processing performed by processor 41. (Omitted) Figure 8 The process shown is the same as the description of the process.

[0432] After the step S105, the processor 41 performs mosaic processing on the processing region of at least one of the first image and the second image (step S255 (image processing step)). After the step S255, the step S115 is executed.

[0433] The step S255 is described in detail. For example, the processor 41 divides the processing region of the first image into two or more partial regions. For example, the partial region contains nine or sixteen pixels. The number of pixels contained in the partial region is not limited to nine or sixteen. For example, the shape of the partial region is a square. The shape of the partial region is not limited to a square. The processor 41 makes the colors of all the pixels contained in one partial region the same color. That is, the processor 41 makes the signal values of all the pixels contained in one partial region the same value. The processor 41 can also calculate the average of the signal values of all the pixels contained in one partial region, and replace the signal values of all the pixels contained in the partial region with the average. The processor 41 performs the above-described processing on all the partial regions. The processor 41 performs mosaic processing on the processing region of the second image by performing the same processing as the above-described processing.

[0434] The processor 41 can also replace the signal values of the pixels contained in the processing region of the second image with the signal values of the pixels contained in the processing region of the first image after performing mosaic processing on the processing region of the first image. The processor 41 can also replace the signal values of the pixels contained in the processing region of the first image with the signal values of the pixels contained in the processing region of the second image after performing mosaic processing on the processing region of the second image.

[0435] Figure 15 The step S110a illustrated can be replaced with the step S255. Figure 18 、 Figure 19 、 Figure 22 、 Figure 24 、 Figure 25 、 Figure 26 、 Figure 27 、 Figure 28 、 Figure 30 and Figure 31 The step S110 illustrated can be replaced with the step S255.

[0436] After the processor 41 performs mosaic processing on the processing region, the observer is less likely to focus on the optical image of the treatment instrument 13 that is represented on the processing region. Thus, the fatigue of the observer's eyes is alleviated. The load on the processor 41 is reduced compared to the case where the processor 41 changes the parallax amount.

[0437] (Postscript)

[0438] All of the above-described embodiments can include the following. The endoscope apparatus 1 has a special light observation function. Before performing a treatment by the treatment instrument 13, the light source of the light source apparatus 3 generates narrow-band light. For example, the center wavelength of the narrow-band is 630 nm. The imaging element 12 captures an object irradiated with the narrow-band light, and generates a first image and a second image. The processor 41 acquires the first image and the second image from the imaging element 12 in step S105.

[0439] When the narrow-band light is irradiated to the observation object, a blood vessel passing through the submucosa or the muscularis propria is emphasized in the first image and the second image. When a stereoscopic image is displayed based on the first image and the second image, the observer easily recognizes the blood vessel. Therefore, the observer easily performs a treatment using the treatment instrument 13.

[0440] The above describes preferred embodiments of the present application, but the present application is not limited to these embodiments and variations thereof. Additional, omission, substitution, and other changes of the structure can be made within the scope of the gist of the present application. Furthermore, the present application is not limited by the above description, but is limited only by the scope of the attached claims.

[0441] Industrial applicability

[0442] According to the embodiments of the present application, the image processing method and the image processing apparatus can reduce the fatigue of the observer's eyes caused by the image of the instrument without impairing the ease of use of the instrument.

[0443] Explanation of reference numerals

[0444] 1 Endoscope apparatus

[0445] 2 Electronic endoscope

[0446] 3 Light source apparatus

[0447] 4 Image processing apparatus

[0448] 5 Monitor

[0449] 6 Receiving apparatus

[0450] 10 Distal end portion

[0451] 11L First optical system

[0452] 11R Second optical system

[0453] 12 Imaging element

[0454] 13, 14, 15 Treatment instrument

[0455] 16 Encoder

[0456] 21 Insertion portion

[0457] 22 operation section

[0458] 23 general cable

[0459] 24 connector section

[0460] 25 connection line

[0461] 26 electrical connector section

[0462] 41 processor

[0463] 42 ROM

[0464] 130 forceps

[0465] 131 sheath

Claims

1. An image processing method, wherein a processor performs: acquiring a first image and a second image having parallax with each other from a first device outputting the first image and the second image, the first image and the second image are images of an observation object and an instrument that performs a treatment on the observation object, a first region is set in the first image and the second image, respectively, the first region is a region including a tip end of the instrument and the observation object in a center of one of the first image and the second image, at least a part of the observation object is shown in the first region of the second image, a second region is set in the first image and the second image, respectively, the second region is a region including at least one end of one of the first image and the second image and a root of the instrument other than the first region of each of the first image and the second image, at least a part of the instrument is shown in the second region of the second image, image processing is performed on a processing region including the second region of at least one of the first image and the second image, and a parallax amount of the processing region is changed.

2. The image processing method according to claim 1, wherein the processor changes the parallax amount of the processing region so that a distance between a viewpoint and an optical image of the instrument in a stereoscopic image displayed based on the first image and the second image becomes larger.

3. The image processing method according to claim 1, wherein a shape of the first region of each of the first image and the second image is any one of a circle, an ellipse, and a polygon.

4. The image processing method according to claim 1, wherein the processor changes the parallax amount so that the optical image of the processing region becomes a plane.

5. The image processing method according to claim 1, wherein the processing region includes two or more pixels, the processor changes the parallax amount so that two or more points of an optical image corresponding to the two or more pixels move in a direction away from a viewpoint, distances by which the two or more points move are equal to each other.

6. The image processing method according to claim 1, wherein the processing region includes two or more pixels, the processor changes the parallax amount so that two or more points of an optical image corresponding to the two or more pixels move in a direction away from a viewpoint, the greater a distance between each of the two or more pixels and the first region, the greater a distance by which each of the two or more points moves.

7. The image processing method according to claim 1, wherein the processing region includes two or more pixels, the processor changes the parallax amount so that a distance between a viewpoint and each of two or more points of an optical image corresponding to the two or more pixels is equal to or greater than a predetermined value.

8. The image processing method according to claim 1, wherein The processor sets the processing region based on at least one of a kind of an image generation device having an image pickup element that generates the first image and the second image, a kind of the implement, and an image pickup magnification.

9. The image processing method according to claim 1, wherein The processor detects the implement from at least one of the first image and the second image, A region in which the implement is detected is set as the processing region.

10. The image processing method according to claim 1, wherein The processor judges a position of the first region based on at least one of a kind of an image generation device having an image pickup element that generates the first image and the second image, a kind of the implement, and an image pickup magnification, and sets a region excluding the first region as the processing region.

11. The image processing method according to claim 1, wherein The processor detects the observation object from at least one of the first image and the second image, regards a region in which the observation object is detected as the first region, and sets a region excluding the first region as the processing region.

12. The image processing method according to claim 1, wherein The processor judges a position of the first region based on information input by an observer to an input device, and sets a region excluding the first region as the processing region.

13. The image processing method according to claim 1, wherein The processor outputs the first image and the second image including the image in which the parallax amount of the processing region is changed to one of a display device that displays a stereoscopic image based on the first image and the second image and a communication device that outputs the first image and the second image to the display device.

14. The image processing method according to claim 13, wherein The processor selects one of a first mode and a second mode, In a case where the first mode is selected, the parallax amount is changed, and the first image and the second image are output to one of the display device and the communication device, In a case where the second mode is selected, the first image and the second image are output to one of the display device and the communication device without changing the parallax amount.

15. The image processing method according to claim 14, wherein The processor selects one of the first mode and the second mode based on information input by an observer to an input device.

16. The image processing method according to claim 14, wherein The processor detects a state of motion of an image pickup element that generates the first image and the second image, and selects one of the first mode and the second mode based on the state.

17. The image processing method according to claim 14, wherein The processor searches for the implement in at least one of the first image and the second image, selecting the first mode in a case where the instrument is detected from at least one of the first image and the second image, selecting the second mode in a case where the instrument is not detected from at least one of the first image and the second image.

18. A control device having a processor constituted by hardware, wherein the processor performs: acquiring the first image and the second image having parallax with each other from a first device that outputs the first image and the second image, the first image and the second image are images of an observation object and an instrument that performs a treatment on the observation object, a first region is set in the first image and the second image, respectively, the first region is a region of a front end portion of the observation object and the instrument including a center of one of the first image and the second image, at least a part of the observation object is shown in the first region of the second image, a second region is set in the first image and the second image, respectively, the second region is a region of at least one end portion of one of the first image and the second image and a root portion of the instrument other than the first region of each of the first image and the second image, at least a part of the instrument is shown in the second region of the second image, image processing is performed on a processing region including the second region of at least one of the first image and the second image, and an amount of parallax of the processing region is changed.

19. An endoscope system, wherein the endoscope system has: an endoscope that acquires a first image and a second image having parallax with each other; a control device having a processor constituted by hardware, the processor performs: acquiring the first image and the second image from the endoscope, the first image and the second image are images of an observation object and an instrument that performs a treatment on the observation object, a first region is set in the first image and the second image, respectively, the first region is a region of a front end portion of the observation object and the instrument including a center of one of the first image and the second image, at least a part of the observation object is shown in the first region of the second image, a second region is set in the first image and the second image, respectively, the second region is a region of at least one end portion of one of the first image and the second image and a root portion of the instrument other than the first region of each of the first image and the second image, at least a part of the instrument is shown in the second region of the second image, image processing is performed on a processing region including the second region of at least one of the first image and the second image, and an amount of parallax of the processing region is changed.

Images