Focus control device, focus control method, and endoscope system

Abstract

The present invention is a focus control device (10) that makes a focal point adjustment by causing a focus lens (26ab) included in an imaging optical system (26a) to move forward or rearward in a direction along an optical axis (O) on the basis of an output signal from an imaging unit (26). The focus control device (10) comprises a processor. The processor detects distance information pertaining to the distance from the leading end of the imaging optical system to an object to be imaged. When the distance information lies within the range of a prescribed distance, the processor performs autofocus control for adjusting the position of the focus lens on the optical axis in accordance with changes in the imaging distance.

Description

Focus control device, focus control method, and endoscope system

[0001] The present invention is applicable to, for example, medical equipment such as an endoscope for observing the inside of a living organ, and relates to a focus control device, a focus control method, and an endoscope system for controlling the focus of an imaging optical system.

[0002] Conventionally, endoscopic systems that include an endoscope that captures images of the inside of a subject and acquires image data, a processor that performs various types of image processing on the image data acquired by the endoscope, a display device that displays the image data processed by the processor as a visible image, and a storage device that records or stores the image data have been widely used in, for example, the medical field and the industrial field.

[0003] Furthermore, in the medical field, various examinations are widely performed using this type of endoscope system to observe the inside of living organs, etc. In an examination using a medical endoscope system (hereinafter referred to as an endoscopic examination), for example, an insertion section of an endoscope inserted into the lumen of a living organ, etc., is advanced and retreated along the length of the lumen, such as by being inserted and withdrawn, while an imaging unit provided at the distal end of the insertion section captures images. At this time, the endoscopic images acquired by the imaging unit are displayed as moving images in real time chronologically on a display device. At the same time, the moving image data can also be stored in a storage device.

[0004] During such an endoscopic examination, the doctor or other equipment operator searches for specific lesions such as polyps or tumors occurring on the inner walls of organs, etc., while observing the real-time endoscopic image displayed on the display device (or, after the examination, a replayed endoscopic image based on recorded or stored endoscopic image data).

[0005] Furthermore, during endoscopic examination, when a specific object, such as a tumor, is recognized and observed, an endoscopic treatment tool (hereinafter, referred to as a treatment tool) may be used. In this case, the treatment tool is positioned in front of the object to be observed (the imaging unit side of the endoscope). In such a situation, for example, if autofocus control is performed on the imaging optical system of the imaging unit in an attempt to observe more closely, focus adjustment may be performed on the treatment tool. In such a case, the focus adjustment may not be performed on the object to be observed.

[0006] Therefore, for example, in a camera focus control device disclosed in Japanese Patent Publication No. 2006-106356, in a situation where an obstacle (such as a net) that should not be focused is present between the camera and the subject, the distance from the focus setting position of the photographing lens to the closest end position of the depth of field is set as a judgment distance, and if this judgment distance is equal to or less than (close to) a preset limit value, the operation of AF control is prohibited. Furthermore, by setting the limit value farther away than the specific subject (such as a net) for which focusing is prohibited, it is possible to prevent the AF control from focusing on the specific subject (such as a net). Furthermore, it is possible to focus on a subject in the area where AF control operation is prohibited by operating the MF.

[0007] However, in endoscopic examinations, the imaging distance from the imaging unit to the object is set to an extremely short distance (for example, a few centimeters to a few millimeters), and therefore the depth of field of the imaging optical system used in endoscopic examinations is generally very narrow.

[0008] For this reason, it is considered inappropriate to apply the conventional technology disclosed in the above-mentioned Japanese Patent Publication No. 2006-106356, which sets a forbidden focusing area depending on the distance, as it is to a focus control device used in medical equipment such as endoscopes, image inspection devices, etc.

[0009] An object of the present invention is to provide a focus control device, a focus control method, and an endoscope system that can switch to an appropriate focus control depending on the usage status of the device in medical equipment such as an endoscope, an image inspection device, etc.

[0010] In order to achieve the above-mentioned object, one aspect of the present invention is a focus control device that adjusts focus by moving a focus lens included in an imaging optical system back and forth along an optical axis based on an output signal from an imaging unit, and is equipped with a processor. The processor acquires distance information from the tip of the imaging optical system to an object to be imaged, and if the distance information is within a predetermined distance range, performs autofocus control that adjusts the position of the focus lens on the optical axis in accordance with changes in the imaging distance.

[0011] A focus control method of one aspect of the present invention is a focus control method that adjusts focus by moving a focus lens included in an imaging optical system back and forth along an optical axis based on an output signal from an imaging unit, detects distance information from the tip of the imaging optical system to an object to be imaged, and if the distance information is within a predetermined distance range, performs autofocus control that adjusts the position of the focus lens on the optical axis in accordance with changes in the imaging distance, and if the distance information is greater than a first distance, performs focus control to maintain the focus lens at a first position or a first' position that is farther away than the first position, and if the distance information is equal to or smaller than a second distance, performs focus control to maintain the focus lens at a second position or a second' distance that is closer to the second position.

[0012] An endoscopic system according to one aspect of the present invention comprises an endoscope including an imaging unit including an imaging optical system that forms an optical image of an object and an imaging element that photoelectrically converts the optical image formed by the imaging optical system to obtain an image signal, an endoscope including a focus adjustment mechanism that moves some of the optical lenses included in the imaging optical system forward and backward in a direction along the optical axis, and a processor, wherein the processor detects distance information from the tip of the imaging optical system to the object to be imaged, and if the distance information is within a predetermined distance range, performs autofocus control that adjusts the position of the focus lens on the optical axis in accordance with changes in the imaging distance, and if the distance information is outside the predetermined distance range and the distance information is greater than a first distance, performs focus control that maintains the focus lens at a first position or a first' position that is farther away than the first position, and if the distance information is equal to or smaller than a second distance, performs focus control that maintains the focus lens at a second position or a second' position that is closer to the second position.

[0013] According to the present invention, it is possible to provide a focus control device, a focus control method, and an endoscope system that can switch to an appropriate focus control depending on the usage status of the equipment in medical equipment such as endoscopes, image inspection equipment, etc.

[0014] 9 is a block diagram showing an overall configuration of an endoscope system including a focus control device according to one embodiment of the present invention; a block diagram showing an outline of the internal configuration of an endoscope system including a focus control device according to one embodiment of the present invention; an explanatory diagram of an inference model acquired as a result of learning in a learning device; a schematic diagram conceptualizing the structure of an image sensor applied to an endoscope included in the endoscope system of FIG. 1 and explaining the principle of image-surface phase-difference AF by the image sensor; a schematic diagram showing a first state (during normal search) when the endoscope of the endoscope system of FIG. 1 is in a first observation state (screening state); a schematic diagram showing a second state (when a specific object is detected) when the endoscope of the endoscope system of FIG. 1 is in the first observation state (screening state); a schematic diagram showing a state when the endoscope of the endoscope system of FIG. 1 is in a second observation state (approach state); a schematic diagram showing a state when the endoscope of the endoscope system of FIG. 1 is in a third observation state (detailed observation state); a flowchart showing the operation of an endoscope system according to one embodiment of the present invention; 18 is a flow chart, a graph showing the relationship between the light emission amount L of the light source and the distance D in the endoscopic system of this embodiment, a diagram showing an example of an endoscopic image when the endoscope in the endoscopic system of this embodiment is in the first observation state (screening state), a conceptual diagram of an endoscopic image explaining a situation in which mist is generated during an endoscopic examination using the endoscopic system of this embodiment, a schematic diagram explaining the concept of continuously adjacent pixels in an endoscopic image, a conceptual diagram explaining a situation in which a large-area brightness area exists in an endoscopic image during an endoscopic examination using the endoscopic system of this embodiment, a conceptual diagram explaining a situation in which a vector change occurs in an endoscopic image during an endoscopic examination using the endoscopic system of this embodiment, a conceptual diagram explaining a situation in which multiple small bright points occur in an endoscopic image during an endoscopic examination using the endoscopic system of this embodiment, a conceptual diagram explaining a situation in which three relatively large bright points occur in an endoscopic image during an endoscopic examination using the endoscopic system of this embodiment, and an enlarged view of the area indicated by the rectangular frame F in Figure 18 and including at least one bright point.

[0015] The present invention will be described below with reference to illustrated embodiments, for example, as an application example to an endoscope system. The drawings used in the following description are schematic. Therefore, in these drawings, each component is shown at a size that allows it to be recognized. Therefore, the dimensional relationships and scales of each component may be shown differently for each component. The present invention is not limited to the illustrated embodiments with respect to the quantities, shapes, size ratios, relative positional relationships, and the like of each component shown in the drawings.

[0016] First, before describing the detailed configuration of a focus control device according to one embodiment of the present invention, the schematic configuration of an endoscope system including the focus control device of this embodiment will be described below with reference to Figures 1 and 2. Figure 1 is a schematic diagram showing the overall configuration of an endoscope system including a focus control device according to one embodiment of the present invention. Figure 2 is a block diagram showing the schematic internal configuration of the endoscope system.

[0017] 1 and 2, an endoscope system 1 is mainly configured to include a processor 10, an endoscope 20, a display device 30, a light source device 40, and a storage device 50. The endoscope system 1 shown in Fig. 1 illustrates a general configuration of an endoscope system used in an endoscopic examination in which the inside of an organ, such as the upper digestive tract (esophagus, stomach, duodenum, etc.) or the lower digestive tract (large intestine, etc.) of a subject (patient, etc.), such as a living body, is observed.

[0018] The endoscope 20 is an image inspection device that includes an insertion section 21, an operation section 22, a universal cable 23, and the like.

[0019] The insertion section 21 is a component that is inserted into a subject such as a living organism. The insertion section 21 is formed by continuously connecting a distal section 21a, a bending section 21b, and a flexible tube section 21c, in that order from the distal end. The insertion section 21 is formed in a generally elongated tube shape. The insertion section 21 has a treatment tool insertion channel 21d therein, which is a conduit for inserting an endoscopic treatment tool 27. This treatment tool insertion channel 21d is provided so as to pass through the insertion section 21 from the distal end to the proximal end. The operation section 22 is connected to the proximal end side of the insertion section 21.

[0020] The tip portion 21a is a structural unit provided at the most distal end of the insertion portion 21. As shown in Fig. 2, various structural members (not shown in Fig. 1) such as an illumination unit 25 and an imaging unit 26 are provided inside the tip portion 21a.

[0021] 2, the illumination unit 25 is a structural unit including an optical fiber cable 25a that guides a light beam from the light source device 40 to the tip portion 21a, and illumination lenses 25b, which are a plurality of optical elements that transmit the light beam guided from the light source device 40 and then emit it forward from the tip surface of the tip portion 21a to illuminate an observation area including a lesion or the like within the subject. As shown in FIG. 1, a plurality of illumination lenses 25b are provided on the tip surface of the tip portion 21a of the insertion section 21 of the endoscope 20. The endoscope 20 illustrated in this embodiment has a structural example including three illumination lenses 25b (see FIG. 1).

[0022] The imaging unit 26 is an electronic device unit including an imaging optical system 26a (see FIG. 2) including a plurality of optical lenses (such as an observation window 26aa and a focus lens 26ab) that form an optical image of a specific object to be observed inside the subject (biological tissue including a lesion such as a tumor; hereinafter referred to as the specific object), a photoelectric conversion element (image sensor 26b; see FIG. 2) that generates image information (such as still image data and moving image data) based on the optical image, and a focus lens driving mechanism 26c that drives the focus lens 26ab included in the imaging optical system 26a. The observation window 26aa included in the imaging optical system 26a is provided on the distal end surface of the distal end 21a of the insertion section 21 of the endoscope 20, as shown in FIG. 1.

[0023] 1 is a tubular portion that can be actively bent by a bending operation mechanism (not shown) that acts in conjunction with the operation of a bending operation member 22b (described later). The flexible tubular portion 21c is a tubular member that extends from the distal end of the operation unit 22 and is connected to the proximal end of the bending portion 21b.

[0024] The operation section 22 is connected to the proximal end of the insertion section 21. The operation section 22 is configured to include an operation section main body 22a, a bending operation member 22b, a plurality of operation members 22c, a treatment tool insertion port 22d, and the like.

[0025] The operation unit main body 22a has a generally box-like shape as a whole and constitutes a gripping portion that is held with the fingers of a user (such as a doctor) of the endoscope 20. Various constituent units such as a bending operation mechanism (not shown) are provided inside the operation unit main body 22a. As described above, the insertion section 21 extends from the operation unit main body 22a.

[0026] The bending operation member 22b and the plurality of operation members 22c are operation members for performing various operations of the endoscope 20. The bending operation member 22b and the plurality of operation members 22c are respectively provided at predetermined positions on the outer surface of the operation portion main body 22a.

[0027] The treatment instrument insertion port 22d is provided at a predetermined position near the distal end of the operation portion main body 22a. The treatment instrument insertion port 22d is a proximal opening of the treatment instrument insertion channel 21d of the insertion portion 21. The treatment instrument insertion channel 21d is connected to a distal channel opening 21da provided on the distal end surface of the distal end portion 21a on the distal side.

[0028] With this configuration, the endoscopic treatment tool 27 inserted from the treatment tool insertion port 22d is configured to be able to pass through the treatment tool insertion channel 21d and then protrude outward from the tip channel opening 21da of the tip portion 21a.

[0029] 1 illustrates a clip-shaped biopsy forceps as an example of the endoscopic treatment tool 27, but the form of the endoscopic treatment tool 27 is not limited to the form shown in the figure. There are various other forms of the endoscopic treatment tool 27, such as a drug dispersion catheter, a looped wire snare, an injection needle, a high-frequency knife, grasping forceps, basket-type grasping forceps, and a hemostatic clip. Therefore, in the following description, the endoscopic treatment tool 27 will be simply referred to as a treatment tool, etc. 27.

[0030] The universal cable 23 is a connection cord for connecting the endoscope 20 to the light source device 40 and the processor 10. For this purpose, the universal cable 23 is made of a tubular member extending from the side of the operation unit body 22a of the operation unit 22. A scope connector 23a is provided at the tip of the universal cable 23. This scope connector 23a is connected to the front panel of the light source device 40.

[0031] An electric cable 23b extends from the scope connector 23a. A connector 23c is provided at the tip of the electric cable 23b. The connector 23c is connected to the front panel of the processor 10. As shown in FIG. 2, various signal transmission cables 10a, optical fiber cables 25a, etc. are inserted into the universal cable 23.

[0032] 2, the light source device 40 includes a light source 41 that supplies illumination light to the illumination unit 25 provided inside the tip portion 21a of the insertion section 21 of the endoscope 20. The illumination light beam emitted from the light source 41 of the light source device 40 is transmitted to the illumination unit 25 in the tip portion 21a through the scope connector 23a, the universal cable 23, the operation section 22, an optical fiber cable 25a that is inserted through the insertion section 21, and the like. The illumination light beam then passes through an illumination lens 25b included in the illumination unit 25 in the tip portion 21a and is irradiated toward the observation target area in front of the tip portion 21a.

[0033] The processor 10 is a control device or a circuit unit including a control circuit, a signal processing circuit, etc. that control the entire endoscope system 1. The processor 10 is also configured to include a function as a focus control device (described in detail below; see FIG. 2).

[0034] A control circuit included in the processor 10 receives, for example, an operation instruction signal from the operation member 22c of the operation section 22 of the endoscope 20, and outputs various control signals for driving and controlling the imaging unit 26, the light source device 40, the illumination unit 25, etc. Also, a signal processing circuit included in the processor 10 receives, for example, an imaging signal from the imaging unit 26, and performs predetermined image signal processing, etc.

[0035] For this reason, the processor 10 and the imaging unit 26 are electrically connected by a signal transmission cable 10a, as shown in Fig. 2. The signal transmission cable 10a is inserted and disposed between the connector 23c, the electrical cable 23b, the scope connector 23a, the universal cable 23, the operation section 22, the insertion section 21, and the imaging unit at the tip end 21a. Similarly, the processor 10 and the light source device 40 are electrically connected by the signal transmission cable 10a, as shown in Fig. 2. In this case, the signal transmission cable 10a is connected from the connector 23c through the electrical cable 23b to the scope connector 23a and then to the light source device 40.

[0036] With this configuration, control signals output from the processor 10 and imaging signals output from the imaging unit 26 are transmitted between the imaging unit 26 and the processor 10 through the signal transmission cable 10a. Similarly, control signals output from the processor 10 are transmitted between the light source device 40 and the light source control unit 17 of the processor 10 through the signal transmission cable 10a. Note that one form of the signal transmission cable 10a is a composite cable in which a plurality of cables are bundled together and covered with an outer sheath shield, an outer sheath tube, or the like.

[0037] The display device 30 receives image signals and the like output from the processor 10 and displays endoscopic images and various types of information in a predetermined format. To this end, the display device 30 and the processor 10 are electrically connected using a video cable 24. The display device 30 may be configured as a display device using, for example, a general liquid crystal panel.

[0038] The storage device 50 is a storage device that saves (records or stores) image data and other information files for storage that are generated for storage by various processes performed in the processor 10 based on the image information generated by the imaging unit 26.

[0039] 1 and 2, the storage device 50 is configured as an integral part of the processor 10. As shown in Fig. 2, a temporary storage unit 51 is provided inside the storage device 50 in addition to a main storage unit (not shown) in the storage device 50. The temporary storage unit 51 is configured, for example, by a semiconductor memory or the like, and is a component that functions as a temporary memory area for temporarily storing predetermined image data and the like output from the image processing unit 11 or the like (described later).

[0040] 1 illustrates a configuration in which the storage device 50 is integrally disposed inside the housing of the processor 10, but the present invention is not limited to this. For example, the storage device 50 may be configured as an external storage device configured using a housing separate from the processor 10.

[0041] 2 shows an example of a configuration in which the temporary storage unit 51 is provided inside the storage device 50, but this is not limiting. The temporary storage unit 51 can also be provided integrally inside the storage control unit 13 (see FIG. 2), which will be described later, for example.

[0042] Furthermore, the processor 10 and the light source device 40 are not limited to being separate components as in the exemplary configurations shown in Figures 1 and 2. For example, the processor 10 and the light source device 40 may be integrally configured using a single housing.

[0043] The configuration of the illumination unit 25 is not limited to the above-described configuration (a configuration in which illumination light from the light source device 40 is transmitted to the distal end portion 21a through the optical fiber cable 25a or the like). As a configuration of the illumination unit 25 other than this, for example, a light-emitting element such as an LED (Light Emitting Diode) as an illumination light source may be provided inside the distal end portion 21a, and power supply to the illumination light source (LED) and light emission control thereof may be controlled by a predetermined control circuit (light source control unit 17 or the like) included in the processor 10.

[0044] The endoscope system 1 configured as described above basically has substantially the same configuration as conventional endoscope systems of the same type, and therefore, illustrations and detailed descriptions of other detailed configurations will be omitted.

[0045] The endoscope system 1 configured as described above is configured such that the function of the focus control device is included in the processor 10. Next, the internal configuration of the processor in particular in the endoscope system according to one embodiment of the present invention will be described in detail below with reference to FIG.

[0046] The processor 10 is configured to include an image processing unit 11, a display control unit 12, a memory control unit 13, an image detection unit 14, a distance distribution information acquisition unit 15, a focus control unit 16, a light source control unit 17, an imager driving unit 18, and a memory device 50 including a temporary memory unit 51.

[0047] The image processing unit 11 is a structural unit or circuit unit that acquires an output signal (mainly image information) from the imaging unit 26 and performs various types of information processing based on the acquired image information. In other words, the image processing unit 11 is a signal processing device that generates an image based on the imaging signal from the imaging element 26b. The image processing performed here includes, for example, normal image information processing, display image data processing, storage image data processing, and various other image processing.

[0048] The normal image processing includes an AD conversion process that converts the analog image signal output from the image sensor 26b into a digital image signal, and is a basic image information processing that is normally performed on image information. The normal image processing is assumed to be a conventional process, and a detailed description thereof will be omitted.

[0049] The display image data processing is processing for generating image data for normal display, such as processing for generating image data representing an endoscopic image to be displayed in a predetermined display area on the display screen of the display device 30. The storage image data processing is processing for generating image data for recording.

[0050] The various types of image processing are image processing that are appropriately added to normal image data for display. Specific examples of the various types of image processing include normal image adjustment processing such as brightness adjustment processing and white balance adjustment processing, and image enhancement processing such as edge enhancement processing and texture and color enhancement imaging (TXI).

[0051] The display control unit 12 is a structural unit or circuit unit that controls the display device 30 so that an appropriate display format is displayed according to the display image data output from the image processing unit 11. The display control unit 12 controls the display of the display image data (various settings such as the display position, display area, and display size) as well as the display of various information other than image information (text information, icon information, etc.). Here, the various information other than image information includes examination date and time information, subject (patient) information, medical record information, etc., as well as various alarm information (notification information) and operation guide information that are displayed as appropriate during an endoscopic examination.

[0052] For example, the display control unit 12 performs display control to continuously display endoscopic images in a display area, and when a specific object is detected in an endoscopic image, the display control unit 12 performs display control to add a predetermined display (for example, a frame display) to an image area including the specific object.

[0053] Furthermore, the display control unit 12 includes a guide display unit (not shown). This guide display unit is a structural unit or circuit unit that displays various alarm information (notification information), guidance information, etc. on the display screen of the display device 30 at appropriate predetermined timings for, for example, a device operator such as a doctor during an endoscopic examination.

[0054] The storage control unit 13 is a configuration unit or circuit unit that controls the storage device 50 (including the temporary storage unit 51) so that the storage image data output from the imaging unit 26 or the image processing unit 11 is saved (recorded or stored) in an appropriate storage format. The storage control unit 13 performs storage control to permanently store image data representing an endoscopic image acquired by the endoscope 20 in a storage area of ​​the storage device 50.

[0055] The image detection unit 14 is a structural unit or circuit unit that detects an image area including an image of a specified object (an image of a specific object such as a tumor, a treatment tool, or other object) from the endoscopic image for each frame that is continuously displayed based on image data acquired by the endoscope 20 and output from the image processing unit 11.

[0056] For this purpose, the image detection unit 14 includes an object detection unit 14a, an imaging distance detection unit 14b, a brightness distribution detection unit 14c, a light quantity detection unit 14d, and the like.

[0057] The object detection unit 14 a is a structural unit or circuit unit that detects an image area including a specific object such as a tumor or a treatment tool, based mainly on image information from the output signal from the imaging unit 26 .

[0058] The imaging distance detection unit 14b is a configuration unit or circuit unit that detects the imaging distance to an object (specific target, treatment tool, other object, etc.) in an endoscopic image based mainly on phase difference information or image information from the output signal from the imaging unit 26. Here, the imaging distance may be detected by detecting distance information from the front surface of the imaging optical system 26a to the object (target, treatment tool, other object, etc.), or by detecting three or so regions defined within each predetermined distance range in a stepwise manner.

[0059] In this embodiment, the imaging distance detection unit 14b is assumed to detect three tiered regions, for example: A first region where the imaging distance exceeds a predetermined first distance (e.g., approximately 4 to 5 mm (millimeters)), A predetermined region where the imaging distance is within a predetermined distance range (e.g., approximately 3 to 4 mm (millimeters)), and A second region where the imaging distance is equal to or less than a predetermined second distance (e.g., approximately 1.5 to 2 mm (millimeters)). In the following description, the first distance may be referred to as a "far point," and the second distance may be referred to as a "near point."

[0060] The luminance distribution detection unit 14c is a structural unit or circuit unit that detects the luminance distribution within an image based on luminance information and the like obtained mainly from image information among the output signals from the imaging unit 26. Here, the luminance distribution is the distribution of luminance within an image expressed by the luminance information for each region when the image is divided into predetermined regions based on the luminance information output for each pixel of the imaging element.

[0061] The light intensity detection unit 14d is a structural unit or circuit unit that detects the illumination light intensity of a specified area within an image based on brightness distribution information obtained mainly from image information among the output signals from the imaging unit 26, or on illumination light intensity control information from the light source control unit 17, etc.

[0062] The detection of a predetermined object by the image detection unit 14 may involve, for example, identifying and detecting a predetermined image region in the image based on the brightness distribution, exposure distribution, distance distribution, etc. The detection of a predetermined object by the image detection unit 14 may also involve, for example, performing similar image recognition based on a plurality of object images (similar images, etc.) prepared in advance, or performing data analysis using deep learning or machine learning.

[0063] For example, in recent years, object detection technology using AI (Artificial Intelligence) has been developed that obtains desired inference results (e.g., detection of specific objects such as tumors) through machine learning of endoscopic images acquired by an endoscope.

[0064] Machine learning is a method of obtaining inference results about unknown matters by learning the features, time-series information, spatial information, etc. of known input information and making inferences based on the learning results. That is, in machine learning, a trained model (also called an inference model) is first obtained that enables inference of a determinable output result from specific input information.

[0065] In this case, when generating a trained model, a large amount of information with known input-output relationships is used as training data to obtain highly reliable inference results. For example, in deep learning, a network is designed using a large amount of training data so that an expected output can be obtained for a known input. The trained model obtained through this process can be used independently of the network that trained it.

[0066] Therefore, by providing the processor 10 with this type of inference model, it becomes possible to accurately and quickly perform various judgments or detections, such as the detection of a specified object in the image detection unit 14, the judgment of the imaging distance range in which an image can be captured in focus during pan focus (deep focus) control, the judgment of the usage status of the endoscope during an endoscopic examination, and the detection of the surface vascular pattern of a specific object (detection of information including the position of the vascular pattern within the image).

[0067] Here, the inference model is constructed roughly as follows. For example, a large amount of image data corresponding to input and output is provided as training data to a predetermined network (not shown) for constructing the inference model. Here, the training data includes image data suitable for detection or determination, such as image data of endoscopic images acquired sequentially in time series during an endoscopic examination, image data of images (such as case images) in which a specific object is captured in the endoscopic images, etc. In this case, an annotation may be set to surround the image area of ​​the specific object with a frame or the like.

[0068] By learning using a large amount of training data, a network design is determined so that a predetermined network (not shown) can obtain an output corresponding to an input. For example, Fig. 3 is a conceptual diagram illustrating an inference model obtained as a result of learning in a predetermined learning device.

[0069] 3, when a group of endoscopic images (first training data group 201) containing a specific object (such as a tumor) is input, information about the specific object captured in the image and additional information such as adding a frame display surrounding the image area of ​​the specific object are obtained along with reliability information. This allows the construction of a first inference model 200 that detects the specific object from the endoscopic image.

[0070] When endoscopic image data (time-series image data) 202 taken during an endoscopic examination is input to the first inference model 200 constructed in this manner, the first inference model 200 detects an image area containing the specific object in a frame in which the specific object is captured from the endoscopic image data 202, and outputs image data (inference result image 203) in which a frame display is added to the image area of ​​the specific object. In this manner, a specific object can be detected from an endoscopic image. The image data of this image 203 can then be temporarily stored, or, if necessary, displayed in a specified display area of ​​the display device 30, providing a reference for the endoscope user during the endoscopic examination.

[0071] Similarly, although not shown, when a group of endoscopic images including a treatment tool or the like is input as a second training data group, information on the treatment tool or the like captured in the images is obtained along with reliability information, thereby constructing an inference model for detecting the treatment tool or the like from the endoscopic images.

[0072] Using the inference model constructed in this way, treatment tools and the like can also be detected from endoscopic images.

[0073] "Deep learning" is a multilayered version of the "machine learning" process using neural networks. A typical example is a "forward propagation neural network," which sends information from front to back and makes a judgment. In its simplest form, it requires three layers: an input layer consisting of N1 neurons, a hidden layer consisting of N2 neurons determined by parameters, and an output layer consisting of N3 neurons corresponding to the number of classes to be discriminated. The neurons in the input and hidden layers, and those in the hidden and output layers, are connected by connection weights, and a bias value is added between the hidden and output layers, making it easy to form logic gates. While three layers are sufficient for simple discrimination, increasing the number of hidden layers makes it possible to learn how to combine multiple features during the machine learning process. In recent years, models with 9 to 152 layers have become practical due to their training time, judgment accuracy, and energy consumption.

[0074] Various well-known networks may be used for machine learning. For example, R-CNN (Regions with CNN features) or FCN (Fully Convolutional Networks) using CNN (Convolution Neural Network) may be used. This involves a process called "convolution" that compresses image features, operates with minimal processing, and is strong in pattern recognition. Furthermore, "recurrent neural networks" (fully connected recurrent neural networks) that can handle more complex information and allow information analysis whose meaning changes depending on the order or sequence of information may be used.

[0075] To realize these technologies, conventional general-purpose arithmetic processing circuits such as CPUs and FPGAs can be used, but because much of the processing in neural networks involves matrix multiplication, devices specialized for matrix calculations such as GPUs (Graphic Processing Units) and Tensor Processing Units (TPUs) may also be used. In recent years, such dedicated artificial intelligence (AI) hardware "neural network processing units (NPUs)" have been designed to be integrated and embeddable with CPUs and other circuits, and may even become part of the processing circuit.

[0076] Furthermore, inference models may be obtained by employing various well-known machine learning techniques, not limited to deep learning. For example, techniques such as support vector machines and support vector regression are available. Here, learning involves calculating the weights, filter coefficients, and offsets of a classifier; other techniques include using logistic regression processing. When a machine is to make a judgment, a human must teach the machine how to make the judgment. In this embodiment, a method for deriving an image judgment using machine learning is employed. However, a rule-based method for applying rules acquired by humans through experience or heuristics to make a specific judgment may also be used.

[0077] The distance distribution information acquisition unit 15 in Figure 2 is a configuration unit or circuit unit that acquires distance distribution information (hereinafter simply referred to as distance distribution information) within the imaging screen based on the output signal (imaging signal; image information or phase difference information) from the imaging element 26b.

[0078] Here, the distance distribution information within the image capture screen refers to information that represents the distribution of distances in the front-to-rear direction for an object captured in the image. The front-to-rear direction for an object refers to the front-to-rear direction (in other words, the depth direction) when the object in the image is viewed from the imaging unit 26 (image sensor 26b) provided at the distal end 21a of the endoscope 20, and refers to the direction along which the endoscope 20 moves forward and backward.

[0079] The imaging element 26b included in the imaging unit 26 of the endoscope 20 applied in this embodiment has a plurality of pixels that receive light beams from a subject and output image signals, and some or all of these plurality of pixels perform imaging plane phase difference detection, and an imaging element having a configuration that can contribute to, for example, autofocus control (AF control), predetermined focus control, etc. In this embodiment, the application of this type of imaging element 26b makes it possible to acquire image information and phase difference information.

[0080] A configuration that enables image-surface phase difference detection using an image sensor will be briefly described below. Fig. 4 is a schematic diagram illustrating the principle of image-surface phase-difference AF control that performs image-surface phase difference detection using an image sensor.

[0081] 4 indicates one pixel of the image sensor 26b. The pixel 260 has a photodiode 262 and a microlens 261. Here, the photodiode 262 is formed in a two-part configuration, for example, a right photodiode 262R and a left photodiode 262L.

[0082] The microlens 261 is an optical lens that transmits a light beam that has passed through the imaging optical system 26a and forms an optical image on the light receiving surface of the photodiode 262. The photodiode 262 is a photoelectric conversion element that converts the optical image formed by the microlens 261 into an electrical signal and outputs it.

[0083] The right photodiode 262R receives a light beam LL from the left region (the region indicated by left-hand diagonal hatching in FIG. 4) and forms an image on the light receiving surface, while the left photodiode 262L receives a light beam LR from the right region (the region indicated by right-hand diagonal hatching in FIG. 4) and forms an image on the light receiving surface.

[0084] The pixel 260 configured as described above can calculate the distance between the two images (parallax) from the output signals (phase difference information) of the left and right photodiodes 262R and 262L to determine the defocus amount for the object in the imaging optical system. Furthermore, by combining the output signals of the left and right photodiodes 262R and 262L, the combined signals can be treated as an image signal for one pixel.

[0085] The distance distribution information acquisition unit 15 receives the phase difference information from the output signal acquired by the imaging element 26b in this manner, and acquires distance distribution information of a predetermined region within the imaging surface (e.g., a treatment tool region, a biological tissue region including a specific target, etc.).

[0086] 4, one photodiode is divided into two and the signals from the left and right photodiodes are handled separately to perform pupil division of the microlens, but the present invention is not limited to this configuration. For example, a configuration in which pupil division is performed by using a light-shielding member can be similarly realized.

[0087] Furthermore, the distance distribution information acquisition unit 15 acquires information on the temporal change in distance information in the forward and backward directions of the object based on multiple image data (hereinafter referred to as time-series image data) output continuously in time series from the imaging unit 26.

[0088] In the above-described configuration example, the distance distribution information acquisition unit 15 acquires distance distribution information from phase difference information from the image sensor 26 b having a plurality of pixels for acquiring image signals and detecting phase differences on the imaging surface, but the present invention is not limited to this configuration. For example, in a typical image sensor, all pixels provided on the imaging surface acquire imaging signals.

[0089] In an imaging unit having such a conventional imaging element 26b, for example, at least one of the optical lenses constituting the imaging optical system 26a, namely, the optical lens (focus lens 26ab) that contributes to focus control, is slightly moved back and forth along the optical axis to change the lens position. This causes a change in the contrast of the object in the image, thereby changing the focus state. At this time, distance information can be obtained from the correlation between the lens position and the focus state corresponding to the lens position. Based on the distance information thus obtained, distance distribution information within the screen can be obtained. In other words, a configuration can be adopted in which distance distribution information is obtained based on the image signal from the imaging element 26b.

[0090] In this way, the distance distribution information acquisition unit 15 acquires distance distribution information based on the output signal (phase difference information or image information) of the image sensor 26 b. That is, the endoscopic image generated based on the output signal (image signal) of the image sensor 26 b and the distance distribution information generated based on the output signal (phase difference information or image information) of the image sensor 26 b are related to and correspond to each other.

[0091] 2 is a structural unit or circuit unit that controls the drive of the imaging optical system 26a of the imaging unit 26, adjusts the relative position on the optical axis between the focus lens 26ab and the image sensor 26b, and maintains a predetermined focus lens position. The focus control unit 16 drives the focus lens drive mechanism 26c based on the detection results and distance distribution information of the image detection unit 14 to perform focus control of the imaging optical system 26a. In this case, the focus control unit 16 switches between, for example, autofocus control, pan focus control, and close-range priority focus control depending on the state of the endoscope.

[0092] The light source control unit 17 is a structural unit or circuit unit that drives and controls the light source device 40 and the illumination unit 25. The light source control unit 17 controls the light source device 40 or the illumination unit 25 in response to operation instruction signals from various operation members 22c provided in the operation unit 22, or instruction signals or acquired data based on the detection results of the image detection unit 14 or the distance distribution information acquisition unit 15, and controls switching of the light source type (white light, special light, etc.).

[0093] Here, the light source types include, for example, white light that is irradiated onto the subject when performing normal observation, as well as predetermined special light that is irradiated onto the subject when performing image-enhanced observation such as narrow band imaging (NBI) or red dichromatic imaging (RDI).

[0094] The imager driving unit 18 is a structural unit or circuit unit that drives and controls the imaging element 26b, which serves as the imager element of the imaging unit 26. The imager driving unit 18 receives a predetermined instruction signal or the like and drives and controls the imaging element 26b at a predetermined timing. This causes a predetermined imaging signal to be output from the imaging element 26b. As described above, the image signal of this output signal is output to the image processing unit 11, and the phase difference information is output to the distance distribution information acquisition unit 15.

[0095] The above is the configuration of the endoscope system 1 including the focus control device of this embodiment. Other configurations that have not been described are the same as those of a conventional endoscope system with a general configuration.

[0096] It should be noted that all or part of the processor 10 includes hardware. Here, the processor 10 is configured by a well-known configuration including, for example, a central processing unit (CPU), random access memory (RAM), read only memory (ROM), non-volatile memory, non-volatile storage, and a non-transitory computer readable medium, as well as peripheral devices thereof.

[0097] Software programs to be executed by the CPU and fixed data such as data tables are stored in advance in ROM, nonvolatile memory, nonvolatile storage devices, etc. The CPU reads out the software programs stored in ROM, etc., expands them into RAM, and executes them, and the software programs refer to various data, etc. as appropriate, thereby realizing the functions of the processor 10.

[0098] The processor 10 may also be configured with a semiconductor chip such as an FPGA (Field Programmable Gate Array), etc. Furthermore, the processor 10 may also be configured with an electronic circuit.

[0099] Furthermore, the software program may be recorded or stored, in whole or in part, as a computer program product on a portable disk medium such as a flexible disk, CD-ROM, or DVD-ROM, or on a non-transitory computer readable medium such as a card-type memory, HDD (Hard Disk Drive) device, or SSD (Solid State Drive) device.

[0100] In an endoscopic examination using such an endoscopic system, a doctor or other equipment operator performs so-called screening, searching for lesions such as polyps or tumors occurring on the inner walls of organs as specific objects (hereinafter referred to as specific objects) while observing the endoscopic image displayed on the display device.

[0101] In this case, for example, the specific object to be observed, such as a tumor, is very small and has a color similar to that of the inner walls of surrounding organs, etc., so it requires skill to accurately distinguish and detect the image area of ​​the specific object from the endoscopic image. In addition, in endoscopic examination, the operation of the endoscope itself, such as the operation of pointing the tip of the insertion part of the endoscope (the observation window of the imaging unit) at the desired object and capturing an image, requires skill.

[0102] Generally, the usage states of an endoscope during an endoscopic examination can be broadly divided into, for example, a state in which the above-mentioned screening is being performed (referred to as the first observation state or screening state), a state in which detailed observation of a detected specific object is being performed (referred to as the third observation state or detailed observation state), and a state in which the tip of the endoscope is in the process of transitioning from the screening state to approaching the specific object to be observed in order to perform detailed observation (referred to as the second observation state or approach state).

[0103] Here, the usage states of the endoscope will be briefly described using Figures 5 to 8. Figures 5 and 6 are schematic diagrams showing the state of the endoscope 20 when it is in a first observation state (screening state). Of these, Figure 5 shows a first state (when searching for a specific object) in the screening state, which represents a state in which a specific object has not been detected. Also, Figure 6 shows a second state (when detecting a specific object) in the screening state, which represents a state in which a specific object has been detected.

[0104] In Fig. 5, the reference numeral 101b schematically indicates a lumen of an organ or the like. Fig. 5 shows the insertion section 21 (tip portion 21a) of the endoscope 20 being inserted into the lumen of the organ or the like 101b. Note that the reference numeral 101a in Fig. 5 indicates a specific object such as a tumor present on the inner wall of the organ or the like 101b. Note that under the circumstances shown in Fig. 5, the specific object 101a is undetected by the endoscope 20. In other words, at this time, the specific object 101a is outside the field of view of the endoscope 20. Note that the reference numeral V in Fig. 5 indicates the imaging range (field of view range) of the imaging optical system 26a of the imaging unit 26 built into the tip portion 21a.

[0105] 5, the symbol O indicates the central axis of the insertion portion 21 of the endoscope 20 (or may be considered as the optical axis of the imaging optical system 26a of the imaging unit 26; hereinafter, simply referred to as the central axis). The insertion portion 21 of the endoscope 20 is operated so that the central axis O advances and retreats in a direction along the lumen of the organ 101b.

[0106] When the endoscope 20 is in the screening state, as shown in FIG. 5, the insertion section 21 of the endoscope 20 is arranged in a substantially straight line along the lumen of the organ 101b.

[0107] Next, assume that the endoscope 20 moves from position [A] in Fig. 5 (see also the dotted line in Fig. 6) to position [B] in Fig. 6 (see Fig. 6) along the direction of arrow X in Fig. 5. At this time, as shown in Fig. 6, the specific object 101a is included within the imaging range (field of view) V of the endoscope 20. When the specific object 101a is included within the imaging range (field of view) V of the endoscope 20 in this way, the specific object 101a is detected by the image detection unit 14 (imaging distance detection unit 14b).

[0108] Here, cases in which a specific object 101a is detected include, for example, cases in which it is detected by the image detection unit 14, as described above, as well as cases in which a user of the endoscope 20 (such as a doctor) recognizes a specific object in the image by visually observing the endoscopic image.

[0109] When the specific object 101a is detected by the image detection unit 14, the detection result is notified to the user (doctor, etc.) of the endoscope 20 in a predetermined format. In this case, the notification is displayed in a predetermined format on the display screen of the display device 30. Alternatively, a notification display such as audio may be used. This allows the user (doctor, etc.) of the endoscope 20 to recognize that the specific object 101a has been detected.

[0110] In this way, when the user (doctor, etc.) of the endoscope 20 recognizes the specific object 101a, the user (doctor, etc.) may wish to perform a detailed observation of the specific object 101a. To do so, the user (doctor, etc.) of the endoscope 20 performs an operation to bring the distal end portion 21a of the endoscope 20 closer to the specific object 101a. By performing this operation, the usage state of the endoscope 20 transitions from the screening state to the approach state.

[0111] 7 is a schematic diagram showing the state of the endoscope 20 when it is in the second observation state (approach state). As shown in FIG. 7, when the endoscope 20 is in the approach state, the insertion section 21 of the endoscope 20 is bent with respect to the bending portion 21b, and the bending portion 21b is bent by a predetermined amount in a predetermined direction. The bending direction of the bending portion 21b at this time is set to a direction in which the distal end surface of the distal end portion 21a of the endoscope 20 is positioned to face the detected specific object 101a.

[0112] That is, at this time, the insertion portion 21 of the endoscope 20 is positioned so that the tip surface faces the specific object 101a, and the central axis O of the insertion portion 21 (optical axis O of the imaging optical system 26a) extends from the tip surface of the insertion portion 21 of the endoscope 20 toward the specific object 101a.

[0113] In this state, the user (doctor, etc.) gradually moves the distal end 21 a of the endoscope 20 closer to the specific object 101 a. In this way, during an endoscopic examination, the state change when the endoscope 20 transitions from the screening state to the approach state can be determined by detecting the change over time in the output signal from the imaging unit 26 that is acquired sequentially in time series.

[0114] Next, consider the case where the endoscope 20 transitions from the approach state to the detailed observation state. In the approach state during an endoscopic examination, the distal end portion 21a of the endoscope 20 approaches the specific object 101a, which is the target of observation, sufficiently to reach a predetermined close distance (e.g., a critical distance beyond which a focused state cannot be achieved). Then, while maintaining the distance between the distal end surface of the distal end portion 21a and the specific object 101a, an operation for detailed observation of the specific object 101a is performed. By performing such an operation, the usage state of the endoscope 20 transitions from the approach state to the detailed observation state.

[0115] 8 is a schematic diagram showing the state of the endoscope 20 when it is in the third observation state (detailed observation state). As shown in Fig. 8, when the endoscope 20 is in the detailed observation state, the bending section 21b of the endoscope 20 is maintained bent by a predetermined amount in a predetermined direction, and the distal end surface of the distal end section 21a is positioned in a position facing and close to the specific object 101a that is being observed. Then, a user (doctor, etc.) operates the endoscope 20 so as to observe the specific object 101a from various angles while pointing the distal end section 21a of the endoscope 20 toward the specific object 101a.

[0116] Conventionally, when performing an endoscopic examination using a medical device or examination device such as an endoscope, it is required to always ensure a reliable observation environment and continuously display good images. To achieve this, it is required to perform appropriate focus control of the imaging unit according to the state of the endoscope, etc. (the three states mentioned above).

[0117] Here, appropriate focus control according to the usage state of the endoscope 20 is, for example, as follows.

[0118] Generally, when an endoscope is in a screening state, observation is performed while the endoscope is moving inside a lumen of an organ, etc. In this case, although the endoscope is actually moving, it can be said that an object present on the inner wall surface of the organ, etc., is moving relative to the endoscope. If focus control is performed each time on an object that moves relative to the endoscope (imaging unit), the distance measurement operation and focus lens movement become complicated, which may make it difficult to obtain accurate and stable images. Therefore, during screening operations, it is preferable to fix the focus lens in a predetermined position and use so-called deep focus control, which can ensure a focused state within a predetermined imaging distance range.

[0119] That is, for example, observation in a screening state is performed with the imaging unit and the object being observed at a slight distance (for example, a distance of more than about 4 to 5 millimeters (mm), i.e., a distance of approximately several centimeters). In this case, sufficient observation can be performed within the depth of field of the imaging optical system. Therefore, the focus control in the imaging unit at this time is what is called pan-focus control.

[0120] In this case, deep focus control refers to focus control that, for example, sets the position of the imaging optical system so that an object within a predetermined distance range (for example, within a range of approximately 1 to 3 centimeters in front of the lens tip surface) is generally in focus, and maintains that position. The position of the imaging optical system (focus lens) that is set during such deep focus control is referred to as the first position or first focus position. Here, a position slightly farther away than the first position is defined as the first' position.

[0121] Furthermore, the pan focus control performed here may set a predetermined focus lens position after performing fine focus control according to distance, or it may be focus control that selects one of several predetermined position settings, or it may be two-stage switching control that simply changes from the pan focus state to another predetermined focus lens position (close-distance compatible focus position).

[0122] In addition, when an endoscope is in the approach state, it is generally desirable to constantly perform focus adjustment operations targeting the object to be imaged or observed to keep the object image in focus at all times, and to maintain a clear, panoramic view of a predetermined area including the peripheral portions of the object. Therefore, it is preferable to perform autofocus control (AF control) during the approach operation so that the object and its peripheral portions are always in focus.

[0123] Typically, when in the approach state, imaging is performed from a slightly farther position (e.g., within a range of approximately 3 to 4 millimeters (mm)) than from a close position (described below; e.g., approximately 1.5 to 2 millimeters (mm)). In this case, a deeper depth of field can be ensured than when observing from a close position. Therefore, by utilizing a deep depth of field, it becomes easier to ensure a focused state within a predetermined range even when there is a difference in distance between the front and rear of the object. In this way, by performing autofocus control that performs fine focus adjustment taking into account the object and its peripheral areas, it becomes possible to observe a wide range including the object in a focused state from a bird's-eye view.

[0124] Here, autofocus control that takes into account the object and its peripheral areas is assumed to be, for example, autofocus control that is performed by setting a focus point at a predetermined position between the main part of the object being observed and the peripheral area in front of the object.

[0125] Furthermore, when in the approach state, it is necessary to continue observing the specific target object so as not to lose sight of it. Furthermore, in the approach state, the distal end of the endoscope (imaging unit) is gradually moved closer to the specific target object, thereby enabling magnified observation. For this reason, if the endoscope moves closer while maintaining pan focus control in the approach state, it becomes increasingly difficult to maintain a focused state. Therefore, for these reasons, it is preferable to perform autofocus control in the approach state.

[0126] On the other hand, detailed observation is often performed with the tip surface of the imaging unit (the front surface of the imaging optical system) close to the surface of the object being observed (a specific object such as a tumor) (for example, within 1.5 to 2 millimeters (mm) of the object) in order to display the object as large as possible. In this case, it is known that the depth of field becomes extremely shallow. Furthermore, in such a state, it is often difficult to hold the imaging unit stably. If imaging is performed using autofocus control in such a situation, the image of the object obtained will be unstable.

[0127] On the other hand, it is known that during such close-up photography (magnification photography), the imaging distance between the imaging unit 26 and the object being observed is substantially constant in the central and peripheral regions of the image, and therefore, the focus control at this time is preferably close-range priority focus control, which is suitable for close-range photography (or close-range observation).

[0128] Here, the near-point priority focus control is a focus control in which, for example, when the imaging unit 26 is brought close to an object located in the central area of ​​the screen of the endoscopic image, the focus lens is fixed and maintained at the closest position at which the imaging optical system of the imaging unit can ensure that the object is in focus.

[0129] Specifically, this control involves moving the focus lens in the optical axis direction and maintaining a predetermined position so that a focused state can be achieved on a predetermined object within a close distance range, for example, within approximately 1.5 to 2 millimeters (mm) in front of the lens tip face. In this case, the position of the imaging optical system (focus lens) set during close-range priority focus control is referred to as the second position or second focus position. Here, a position slightly closer than the second position is defined as the second' position.

[0130] Normally, the closer the imaging distance, the shallower the depth of field. In this case, if the imaging unit sways back and forth relative to the object, for example, and focus control is performed to control the movement of the focus lens each time, a stably focused image cannot be obtained.

[0131] Therefore, as described above, in the detailed observation state, the focus lens position is fixed by performing the closest object priority focus control, thereby ensuring a focused state within a predetermined imaging distance range and enabling a stable observation image to be obtained.

[0132] Another example of a method for providing close-range priority focus control is focus control that performs precise focus adjustment within a close-range distance (for example, within an imaging distance of 1.5 to 3 millimeters (mm)). In this case, the focus control is, for example, focus control that performs extremely fine and rapid movement control of the focus lens to ensure a consistently accurate in-focus state.

[0133] In addition to these focus controls, there are various other forms of autofocus control in recent years, such as a form in which autofocus control is performed by tracking a predetermined object detected within the imaging screen, a form in which autofocus control is performed by preferentially selecting either a distant object or a close object within the imaging screen, a form in which autofocus control is performed on an object included in an area within the imaging screen that is arbitrarily specified by the user, and focus control that ensures good focusing within a predetermined distance range in front of the imaging unit by taking into account the depth of field of the imaging optical system. The focus control unit 16 can apply any of these various autofocus control forms by appropriately selecting or combining them.

[0134] The operation of the endoscope system 1 including the focus control device of this embodiment configured as described above will be described below. Fig. 9 is a flowchart explaining the operation of the endoscope system including the focus control device of one embodiment of the present invention. The flowchart shown in Fig. 9 is a processing sequence explaining the operation when an endoscopic examination is performed using the endoscope system 1.

[0135] First, it is assumed that the endoscope system 1 is activated and ready to perform an endoscopic examination. At this time, it is assumed that the endoscope 20 in the endoscope system 1 is inserted into the organ of a subject (e.g., a patient) to be examined. It is assumed that the focus control setting in the initial state after activation of the endoscope 20 is, for example, a pan-focus control setting.

[0136] When the endoscope system 1 of this embodiment is in this state, in step S1 of Fig. 9, the image processing unit 11 of the processor 10 starts processing to acquire the output signal (image data; image information and phase difference information) from the imaging unit 26. At the same time, various predetermined image processing operations are performed for each frame based on the acquired output signal (image data). Specifically, various predetermined image processing operations may be performed, for example, every two frames or every four frames.

[0137] In this case, the image processing performed for each frame does not necessarily have to be performed on all frames acquired according to the frame rate of the image sensor 26b. For example, the image data acquired may be processed on frame data at intervals of several frames.

[0138] Next, in step S2, the processor 10 executes an endoscope state determination process for determining the current state of the endoscope. Here, Fig. 10 is a flowchart showing the endoscope state determination process in detail.

[0139] 10, the processor 10 checks each of the following conditions from the image data processed by the image processing unit 11 based on the output signal from the imaging unit 26. In the following explanation, the conditions are numbered as first, second, etc., but the numbers used in the explanation do not indicate the order of processing but merely serve as line headings when listing the conditions.

[0140] First, it is confirmed whether the light emission amount of the illumination light source is large or not. The light emission amount of the illumination light source can be confirmed by detecting the light amount control signal of the light source 41 by the light source control unit 17.

[0141] Generally, when imaging is performed using an imaging unit in a dark place such as the inside of a lumen of an organ, the object is illuminated with illumination light. In this case, in order to capture a clear image of the object, the light intensity of the illumination light source needs to be increased as the distance to the object increases.

[0142] Fig. 11 is a graph showing the relationship between the light emission amount L of the light source and the distance D in the endoscope system of this embodiment. For example, when the illumination light source shown in Fig. 11 is used, an object at a distance D = 5 mm can be sufficiently illuminated when the light emission amount L is approximately 30% of the maximum light emission amount. Here, the distance D = 5 mm corresponds to the first distance described above.

[0143] In the process of step S11 described above, whether or not the light emission amount is high can be confirmed by, for example, if the light emission amount L is 30% or more, it can be inferred that light source control is being performed to search for a first region beyond a first distance (approximately 4 to 5 mm). From this, it can be determined that the endoscope 20 at this time is in a first observation state (screening state). Furthermore, if the light emission amount L is less than 30%, it can be determined that the endoscope 20 is in a state other than the first observation state.

[0144] During endoscopic examinations, various dye solutions may be sprayed to stain the tissue inside the lumen of an organ or other object before observation. In this case, if a blue dye solution is used, for example, the luminescence intensity must be increased compared to normal (non-stained) cases. Therefore, to confirm that the luminescence intensity is high when such a procedure is performed, the threshold value is changed to, for example, luminescence intensity L = 50%. This threshold change can be achieved, for example, by checking the overall color tone of the endoscopic image based on the image data, and then changing the confirmation threshold value for luminescence intensity L according to the detected image color tone.

[0145] Second, it is confirmed whether or not a luminal region exists in the image. If the presence of a luminal region is confirmed, it is also confirmed whether or not the luminal region is located in an approximately central region. Here, the presence of a luminal region in the image can be confirmed by brightness distribution information in the image acquired based on image data of the endoscopic image, color image detection of black regions, etc.

[0146] Generally, when the endoscope is in the first observation state (screening state), the insertion section 21 is arranged in a substantially linear form so as to follow the inside of the lumen of an organ, etc., as described with reference to Fig. 5. The endoscopic image acquired at this time is, for example, as shown in Fig. 12.

[0147] Fig. 12 is a diagram showing an example of an endoscopic image when the endoscope in the endoscopic system of this embodiment is in the first observation state (screening state; state shown in Fig. 5). In Fig. 12, reference numeral 101 denotes a display frame indicating the display area of ​​the endoscopic image. The endoscopic image is displayed within the range of this display frame 101. Reference numeral 110a in Fig. 12 denotes a dark area in the endoscopic image. Furthermore, reference numeral 110b in Fig. 12 denotes a bright area 110b displayed around the dark area 110a.

[0148] Here, the dark region 110a in the endoscopic image is a region where a clear image is not displayed because the illumination light does not reach sufficiently. In other words, the dark region 110a can also be said to be an underexposed region. On the other hand, the bright region 110b in the endoscopic image is a region that is sufficiently illuminated by the illumination light and has a high brightness distribution. For these reasons, it is assumed that the above-mentioned lumen region mainly corresponds to the dark region 110a.

[0149] Normally, when an endoscope is in the first observation state (screening state) during an endoscopic examination, a dark area 110a is present in the endoscopic image, as shown in Figure 12, and this dark area 110a tends to be located approximately in the central area.

[0150] In this case, whether or not the region is a lumen region is confirmed by checking the size of the area occupied by the region where the dark region 110a exists in the entire endoscopic image. Specifically, for example, as shown in Fig. 12, the display frame 101 of the endoscopic image is divided into a plurality of areas. The example in Fig. 12 shows a case where the display frame 101 is divided into 8 vertical areas and 8 horizontal areas. Note that each number marked with the symbol N on the margin of the display frame 101 in Fig. 12 indicates an area number. These area numbers are assigned for convenience of explanation.

[0151] Here, for example, the area (referenced 101x) enclosed by the four areas from the vertical N3 area to the vertical N6 area and the four areas from the horizontal N3 area to the horizontal N6 area is defined as the central region area 101x. Also, for example, the areas (referenced 101y) enclosed by the thick frame lines at the four corners of the display frame 101 and enclosed by three areas vertically and horizontally are defined as corner areas 101y.

[0152] The following checks are made for each of the areas 101x and 101y defined in this way. First, the minimum brightness of the central area 101x (the brightness of the darkest part within the area) is 50% or less of the maximum value of the average brightness of each corner area 101y. Second, the area of ​​the dark areas 110a in the central area 101x is 20% or more of the area of ​​the entire area of ​​the endoscopic image. Third, the dark areas 110a in the central area 101x are continuous.

[0153] When all of the above first to third conditions are met, it can be determined that the endoscope 20 is in the first observation state (screening state). Furthermore, when any one of the above first to third conditions is not met, it can be determined that the endoscope 20 is in a state other than the first observation state.

[0154] In this way, in the process of step S11 described above, it is confirmed whether or not a luminal region exists in the image, and if so, whether or not the luminal region is located in the approximate central region.

[0155] Third, it is confirmed whether mist or the like is generated in the image. The presence of mist or the like in the image can be confirmed by brightness distribution information in the image acquired based on image data of the endoscopic image.

[0156] During endoscopic examinations, water may be pumped to, for example, clean the front surface of an observation window or to wash away mucus, intestinal debris, and the like from the area to be observed. When this water pumping operation is performed, a liquid such as physiological saline solution is forcefully ejected from the distal end of the endoscope. This can cause fine liquid particles to spread in a mist-like fashion around the distal end of the endoscope 20, generating what is known as mist. Furthermore, during endoscopic examinations, procedures such as cauterization may be performed. In this case, what is known as surgical smoke may be generated.

[0157] In this way, mist, surgical smoke, etc. that are generated during endoscopic examination can obstruct the field of view of the endoscope 20, and if autofocus control is performed at this time, the focus may be on the mist or surgical smoke that is generated in front of the object that is desired to be observed.

[0158] In general, when water supply operations or treatments are performed, the imaging distance is often relatively long. In other words, water supply operations or treatments are rarely performed in situations where detailed observations are being performed (close imaging distances) or when approaching for imaging observations (close imaging distances).

[0159] Therefore, in this embodiment, when mist or the like is detected, the endoscope 20 is determined to be in the first observation state (screening state). Here, detection of mist or the like is performed as follows. Fig. 13 is a conceptual diagram of an endoscopic image illustrating a situation in which mist occurs during an endoscopic examination using the endoscopic system of this embodiment. In Fig. 13, reference numerals 101, 110a, and 110b are the same as in Fig. 12. In Fig. 13, reference numeral 111 represents mist.

[0160] Generally, when mist is generated, the illumination light emitted from the tip of the endoscope 20 is reflected by the liquid particles that form the mist, causing a plurality of fine, high-brightness pixels to be displayed discretely in the endoscopic image. It is known that such a plurality of discrete high-brightness pixels caused by mist are generated discontinuously, for example, as shown by reference numeral 111 in FIG. 13 .

[0161] Therefore, first, whether or not there are a predetermined number of high-luminance pixels in the endoscopic image is detected based on the luminance distribution information. In this case, a pixel is determined to be a high-luminance pixel if, for example, its pixel signal output value is 50% or more of the maximum value.

[0162] In this case, in addition to the above determination, a condition may be added such that the average luminance of an area excluding the pixel of interest within a predetermined area (e.g., an area of ​​5 pixels x 5 pixels) including the pixel of interest is equal to or less than a predetermined value (e.g., 50% or less). This determination assumes that the background of the pixel of interest is, for example, a dark area.

[0163] Next, it is confirmed whether the detected multiple high-brightness pixels are caused by mist. To do this, it is confirmed whether the high-brightness pixels are consecutively adjacent. Here, if the detected high-brightness pixels are caused by mist, they will appear discontinuously. On the other hand, for example, in the contour line of an object or the ridge line of folds inside an organ, high-brightness pixels may be detected consecutively.

[0164] Here, "consecutively adjacent pixels" refers to, for example, the following: Fig. 14 is a schematic diagram illustrating the concept of consecutively adjacent pixels in an endoscopic image. As shown in Fig. 14, consider lines extending in a total of eight directions, with a pixel of interest designated by the symbol P0 as the center: two horizontal directions H1 and H2; two vertical directions V1 and V2; two diagonal directions DR1 and DL1 obtained by tilting the two horizontal directions H1 and H2 counterclockwise by 45 degrees; and two diagonal directions DL2 and DR2 obtained by tilting the two vertical directions V1 and V2 counterclockwise by 45 degrees.

[0165] Also, in Figure 14, an area consisting of a plurality of pixels arranged along the horizontal direction H1 is indicated by the symbol P1, an area consisting of a plurality of pixels arranged along the horizontal direction H2 is indicated by the symbol P2, an area consisting of a plurality of pixels arranged along the vertical direction V1 is indicated by the symbol P3, an area consisting of a plurality of pixels arranged along the vertical direction V2 is indicated by the symbol P4, an area consisting of a plurality of pixels arranged along the right diagonal direction DR1 is indicated by the symbol P5, an area consisting of a plurality of pixels arranged along the right diagonal direction DL1 is indicated by the symbol P6, an area consisting of a plurality of pixels arranged along the left diagonal direction DL2 is indicated by the symbol P7, and an area consisting of a plurality of pixels arranged along the left diagonal direction DR2 is indicated by the symbol P8.

[0166] In this case, the average brightness of each pixel region P1 to P8 on each of the straight lines H1, H2, V1, V2, DR1, DR2, DL1, and DL2 extending in eight directions centered on the pixel of interest P0 is calculated. If any of these average brightnesses is higher than the pixel of interest P0, it is determined that there are consecutively adjacent high-brightness pixels in the endoscopic image. In other words, in this case, it is determined that there is no mist. Furthermore, if it is determined that mist is occurring, it can be determined that the endoscope 20 is in a first observation state (screening state).

[0167] Fourth, it is confirmed whether or not there is a large (huge) brightness area (also called a halation area) in the endoscopic image. The existence of a large brightness area in the image can be confirmed by brightness distribution information in the image acquired based on the image data of the endoscopic image.

[0168] Here, the determination of high-brightness pixels is the same as the determination of high-brightness pixels during mist detection described above: if the pixel signal output value is 50% or more of the maximum value, it is determined to be a high-brightness pixel. If the area of ​​a region made up of such high-brightness pixels occupies 20% or more of the total area of ​​the endoscopic image and the high-brightness pixel regions exist consecutively, it is determined that a large-area brightness region exists.

[0169] In this case, the shape of the detected large-area luminance region is also checked, for example, to make sure that the detected large-area luminance region is not substantially circular or elliptical.

[0170] Fig. 15 is a conceptual diagram illustrating a situation in which a large brightness area exists in an endoscopic image during an endoscopic examination using the endoscopic system of this embodiment. In Fig. 15, reference numeral 101 denotes a display frame indicating the display area of ​​the endoscopic image. The endoscopic image is displayed within the range of this display frame 101. Reference numeral 101a in the figure indicates a specific object such as a tumor. Reference numeral 101b in the figure indicates the inner wall of an organ, etc.

[0171] 15, reference numeral 112 denotes a surgical smoke, and reference numeral 113 denotes a treatment tool, etc. The treatment tool, etc. 113 shown in FIG. 15 is, for example, a hemostatic clip.

[0172] Generally, specific objects such as tumors to be observed in endoscopic examinations are often approximately circular or elliptical in shape. On the other hand, treatment tools and the like are often approximately rectangular in shape. Surgical smoke and the like are often cloud-shaped or irregular in shape. Therefore, when a large-area brightness region is detected and its shape is confirmed to be neither approximately circular nor approximately elliptical, the endoscope 20 can be determined to be in a first observation state (screening state).

[0173] The first to fourth determination results have hysteresis in the time direction. For example, if the detection continues for five consecutive frames, the determination is true, and if the detection continues for less than five consecutive frames, the determination is false.

[0174] The judgment threshold also has hysteresis. In this case, the case where the judgment is true is described, but if the judgment is false, the threshold is set more strictly. For example, if the presence of a large brightness area (halation area) (true) is judged as false, the judgment will be false if the area of ​​the high brightness pixel area is less than 15% of the area of ​​the entire area of ​​the endoscopic image.

[0175] 10, if any one of the first to fourth conditions is satisfied (if any one is "Y"), the process proceeds to step S12. In step S12, the processor 10 determines that the current state of the endoscope 20 is the first observation state (screening state). Thereafter, the series of processes ends and the process returns to the original process (proceeds to step S3 in FIG. 9).

[0176] On the other hand, if none of the first to fourth conditions are met in step S11 of FIG. 10 (all are negative "N"), the process proceeds to step S13.

[0177] In step S13, the processor 10 checks whether the endoscope state determination result for the previous frame was the first observation state. If the previous frame was determined to be in the first observation state, the processor 10 proceeds to the next step S14. If the previous frame was not determined to be in the first observation state, the processor 10 proceeds to step S20.

[0178] In step S14, the processor 10 checks whether the current frame has a large vector change compared to the previous frame. Here, a vector change refers to the following situation. Figure 16 is a conceptual diagram illustrating a situation in which a vector change occurs in an endoscopic image during an endoscopic examination using the endoscopic system of this embodiment.

[0179] In Fig. 16, reference numeral 101 denotes a display frame for an endoscopic image. Reference numerals 101a(1) and 101a(2) denote specific objects. Of these, reference numeral 101a(1) indicated by a solid line denotes the state of the specific object at any one time when, for example, the endoscope 20 is in the first observation state or the second observation state. Reference numeral 101a(2) indicated by a dotted line denotes the state displayed enlarged after a predetermined time has passed since the specific object 101a(1) was observed after the endoscope 20 transitioned to the second observation state. Reference numeral 101b denotes the inner wall of an organ or the like.

[0180] Now, consider a case where the endoscope 20 approaches or moves away from the detected specific object 101a while in the second observation state. For example, when the endoscope approaches the object, the specific object initially appears small, as indicated by reference numeral 101a(1) in Fig. 16, but as the endoscope 20 approaches, it appears larger, as indicated by reference numeral 101a(2) in Fig. 16. In this case, the specific object 101a(1) gradually becomes larger in the direction indicated by the outward arrow in the arrow symbol Ex when viewed from the image data for each frame acquired in chronological order.

[0181] On the other hand, when the endoscope 20 moves away from the specific object 101a, the specific object initially appears large as indicated by reference numeral 101a(2) in FIG. 16 , but as the endoscope 20 moves away, it appears smaller as indicated by reference numeral 101a(1) in FIG. 16 . In this case, the specific object 101a(2) gradually becomes smaller in the direction indicated by the inward arrow in the arrow symbol Ex when viewed from the image data for each frame acquired in time series. The direction and length of the arrow Ex in these cases are referred to as vector change. The change in distance information in this case changes from large (small) to small (large) (from far (near) to near (far)). In other words, the distance in the depth direction (front-to-back direction) gradually changes in the direction of decreasing or increasing.

[0182] Generally, during an endoscopic examination, a large change is observed in the vector change of a specific object in the depth direction relative to the endoscope when the endoscope is in the second observation state. On the other hand, it is known that in the first or third observation state, the vector change in the depth direction is small, but the vector change in the up, down, left, right, and diagonal directions (such as horizontal movement in the XY plane) can be large. In this case, the horizontal movement of the object does not actually mean that the object itself is moving, but rather that the endoscope is moving relative to the object. Therefore, the term "movement of the object" includes relative movement between the endoscope and the object.

[0183] Therefore, in the process of step S14 in Fig. 10, the vector change of the specific object is confirmed, and if the vector change in the depth direction is large, the process proceeds to step S15, whereas if the vector change in the depth direction is small, the process proceeds to step S12.

[0184] In this case, for example, if movement is detected not in the depth direction relative to the imaging unit 26 but in the XY plane of the image, i.e., horizontal, vertical, or diagonal (up, down, left, right, diagonal) movement, this may be because, for example, observation of the object being observed has been canceled and the endoscopic observation field has been moved horizontally. Therefore, in such a case, even if the vector change is large, the process proceeds to step S12.

[0185] Alternatively, apart from such control, for example, if a horizontal vector change is detected for the imaging unit 26 in the processing of step S14 and the vector change is large, a second observation state determination may be made in which the subsequent processing steps in Figure 10 are skipped and AF control is performed.

[0186] In step S15 of Fig. 10, the processor 10 checks whether or not there are multiple small bright spots in the endoscopic image. In this case, the multiple small bright spots in the endoscopic image are assumed to be in the following situation: Fig. 17 is a conceptual diagram illustrating a situation in which multiple small bright spots appear in the endoscopic image during an endoscopic examination using the endoscopic system of this embodiment.

[0187] In Figure 17, the reference numerals 101, 110a, and 110b are the same as those in Figures 12 and 13. Reference numeral 114 denotes a plurality of small bright spots that have occurred in the endoscopic image. Here, the small bright spots are assumed to be high-brightness pixel areas having an area slightly larger than the minute bright spots (high-brightness pixels) at the pixel level caused by mist or the like as shown in Figure 13 above. Specifically, this refers to a case where there is one or less high-brightness pixel area having an area of ​​five or more adjacent pixels, and five or more high-brightness pixel areas having an area of ​​less than five pixels.

[0188] Generally, such a situation in which multiple small bright spots occur occurs when, for example, in the first observation state, the illumination light from the endoscope 20 is reflected by the inner wall of an organ or the like located at a certain distance.

[0189] Therefore, if multiple small bright spots are confirmed in the process of step S15 in Fig. 10, the process proceeds to step S12, whereas if multiple small bright spots are not confirmed, the process proceeds to step S16.

[0190] In step S16, the processor 10 performs a process of detecting the in-focus position. Here, the in-focus position is the distance to the in-focus position (imaging position) of the object in the endoscopic image, i.e., the imaging distance. This imaging distance information can be obtained from phase difference information included in the image data.

[0191] In step S17, the processor 10 checks whether the detection result (imaging distance) of the process in step S16 is on the far point side. Here, when the imaging distance is on the far point side, it means when the imaging distance is on the side close to the first distance (approximately 4 to 5 mm) as described above.

[0192] In the process of step S17, if the detected imaging distance is on the far point side, the process proceeds to step S12, whereas if the detected imaging distance is not on the far point side, the process proceeds to step S18.

[0193] Next, in step S18, the processor 10 checks whether the detection result (imaging distance) of the process in step S16 is on the near point side. Here, when the imaging distance is on the near point side, it means when the imaging distance is on the side close to the second distance (approximately 1.5 to 2 mm) as described above.

[0194] In the process of step S18, if the detected imaging distance is on the near point side, the process proceeds to step S23, whereas if the detected imaging distance is not on the near point side, the process proceeds to step S19.

[0195] In step S19, the processor 10 determines that the current state of the endoscope 20 is the second observation state (approach state), and then ends the series of processes and returns to the original process (proceeds to the process of step S3 in FIG. 9).

[0196] In step S23, the processor 10 determines that the current state of the endoscope 20 is the third observation state (detailed observation state), and then ends the series of processes and returns to the original process (proceeds to the process of step S3 in FIG. 9).

[0197] On the other hand, if the previous frame was not determined to be in the first observation state in the processing of step S13, the process proceeds to step S20, where the processor 10 checks whether the previous frame was determined to be in the third observation state. If the previous frame was determined to be in the third observation state, the process proceeds to the next step S21. If the previous frame was not determined to be in the third observation state, the process proceeds to step S15.

[0198] In step S21, the processor 10 checks whether the vector change in the current frame is large compared to the previous frame. If the vector change is large, the process proceeds to step S22. If the vector change is small, the process proceeds to step S23.

[0199] In step S22, the processor 10 checks whether or not "three relatively large bright points" exist in a predetermined pattern within the endoscopic image.

[0200] 1 and as described above, the endoscope 20 included in the endoscopic system 1 of this embodiment has three illumination lenses 25b provided on the distal end surface of the distal end portion 21a. During endoscopic examination, illumination light emitted from these (three) illumination lenses 25b toward the front (of the observation object) is specularly reflected by the object and displayed as "three relatively large bright points" in the endoscopic image.

[0201] Here, the detection of "three relatively large bright points" in the endoscopic image can be confirmed by brightness distribution information in the image acquired based on the image data.

[0202] Fig. 18 is a conceptual diagram illustrating a situation in which three relatively large bright spots appear in an endoscopic image during endoscopic examination using the endoscopic system of this embodiment. Fig. 19 is an enlarged view of the area indicated by the rectangular frame F in Fig. 18, which includes at least one bright spot.

[0203] In Figure 18, the reference symbols 101, 101a, and 101b are the same as in Figure 15. Also in Figure 18, the reference symbols 115a (1 to 3) and 115b indicate bright spots. The reference symbol F indicates a frame indicating an area including at least one bright spot 115a (1) of the multiple bright spots 115a. The reference symbol Tr indicates a triangle formed by connecting three bright spots 115a (1 to 3).

[0204] First, whether the size of a bright spot appearing in an endoscopic image is "relatively large" is confirmed as follows. When an area containing high-brightness pixels is detected in an endoscopic image, one of the areas is focused on and divided into multiple areas. Specifically, for example, area F in FIG. 18 is divided into 20 horizontal areas and 20 vertical areas, as shown in FIG. 19. In this case, the numbers shown outside area F in FIG. 19 indicate area coordinates. For example, the divided area indicated by symbol Q1 can be represented as coordinates (7, 6), which are represented by abscissa 7 and ordinate 6. Symbols Q2, Q3, Q4, and Q5 can be represented in a similar manner.

[0205] As shown in FIG. 19 , for each of the divided areas, the ratio of high-luminance pixels to the total number of pixels within the area is calculated. If the high-luminance pixels within each divided area are 50% or more, the divided area is determined to contain high-luminance pixels. For example, if the divided area Q1 (7,6) in FIG. 19 contains 40% high-luminance pixels, the divided area Q2 (11,6) contains 15% high-luminance pixels, the divided area Q3 (6,8) contains 100% high-luminance pixels, the divided area Q4 (4,9) contains 40% high-luminance pixels, and the divided area Q5 (11,14) contains 75% high-luminance pixels, then the divided areas Q3 and Q5 can be determined to contain high-luminance pixels.

[0206] For example, if a divided area containing high-brightness pixels includes an area with five or more adjacent divided areas, the area is determined to be a "relatively large" bright spot. The coordinates of the center of the area with five or more adjacent divided areas are then stored. As a specific example, the areas indicated by symbols R1 and R2 in FIG. 19 are areas with "five or more adjacent divided areas containing high-brightness pixels," and can be determined to be "relatively large." In this case, the coordinates of the center of area R1 are (7, 11), and the coordinates of the center of area R2 are (11, 10). Similar confirmation is performed for the other bright spots 115a (2), (3), and 115b.

[0207] In this way, the bright spots 115a (1 to 3) in Figures 18 and 19 are determined to be "relatively large." On the other hand, the bright spot 115b in Figures 18 and 19 is determined to be not "relatively large" (small).

[0208] In this way, it is confirmed whether three "relatively large" bright spots are present in the endoscopic image, and if so, it is confirmed whether the three bright spots are arranged in a predetermined pattern. Here, the predetermined pattern is assumed to be, for example, a pattern in which a triangle (symbol Tr in FIG. 18 ) formed by lines connecting the approximate centers of the three bright spots is arranged so as to substantially surround the approximate center region of the endoscopic image, and the three bright spots are spaced apart from each other by a certain distance or more on the plane of the endoscopic image. Specifically, for example, a pattern in which the length of each side of the triangle (the distance between the centers of the three bright spots) is 20% or more of the length of the short side of the endoscopic image is assumed.

[0209] In this way, if three relatively large bright spots are detected in the endoscopic image during endoscopic examination and their shape matches a predetermined shape, it can be determined that the endoscope 20 is in the third observation state.

[0210] If the presence of three relatively large bright points is confirmed in step S22 of Fig. 10, the process proceeds to step S23, whereas if not, the process proceeds to step S16.

[0211] Thus, in the endoscope state determination process of Figure 10, the first observation state is determined in the process of step S12, the second observation state is determined in the process of step S19, and the third observation state is determined in the process of step S23.

[0212] 9, the processor 10 checks whether the result of the determination made in the process of step S2 (the endoscope-like body determination process of FIG. 10) is the first observation state. If it is determined that the state is the first observation state, the process proceeds to step S7.

[0213] Then, in step S7, the processor 10 sets the position of the focus lens 26ab to the first position (first focus position) through the focus control unit 16 and performs control to maintain that position (pan focus control setting). Note that at this time, if the position of the focus lens 26ab has already been set to the first position, control is performed to maintain that position. Thereafter, the process proceeds to step S6.

[0214] On the other hand, if the first observation state is not determined in step S3, the process proceeds to step S4. In step S4, the processor 10 checks whether the result of step S2 (the endoscope-like body determination process in FIG. 10) indicates the third observation state. If the third observation state is determined, the process proceeds to step S8.

[0215] Next, in step S8, the processor 10 sets the position of the focus lens 26ab to the second position (second focus position) via the focus control unit 16 and performs control to maintain that position (close-object priority focus control setting). Note that at this time, if the position of the focus lens 26ab has already been set to the second position, control is performed to maintain that position. Thereafter, the process proceeds to step S6.

[0216] On the other hand, if the third observation state is not determined in step S4, the process proceeds to step S5, assuming that the determination result in step S2 (the endoscope-like body determination process in FIG. 10) is the second observation state.

[0217] Therefore, in step S5, the processor 10 executes autofocus control (AF control) targeting the specific object via the focus control unit 16. Thereafter, the process proceeds to step S6.

[0218] In step S6, the processor 10 confirms an instruction to terminate the ongoing endoscopic examination. Here, the instruction to terminate the endoscopic examination may be, for example, a predetermined examination termination instruction signal generated by operating a predetermined operating member among the processor 10 of the endoscopic system 1 or the operating members 22c provided on the operating unit 22 of the endoscope 20. In addition, the termination of the endoscopic examination can also be determined, for example, by checking image data of the endoscopic image acquired by the endoscope 20.

[0219] If the instruction to end the test is confirmed in the process of step S6, the series of processes is ended (END). On the other hand, if the instruction to end the test is not confirmed, the process returns to the process of step S1 described above, and the subsequent processes are repeated.

[0220] As described above, the focus control device of this embodiment is a focus control device that adjusts focus by moving the focus lens 26ab included in the imaging optical system 26a forward or backward in a direction along the optical axis O based on an output signal from the imaging unit 26, and is equipped with an imaging distance detection unit 14b that detects distance information from the tip of the imaging optical system 26a to an object to be imaged, and a focus control unit 16 that adjusts the position of the focus lens 26ab on the optical axis O based on the detection result by the imaging distance detection unit 14b, and when the distance information detected by the imaging distance detection unit 14b is within a predetermined distance range (for example, 3 to 4 mm), the focus control unit 16 performs autofocus control that adjusts the position of the focus lens 26ab on the optical axis O in accordance with changes in the imaging distance.

[0221] In addition, if the distance information detected by the imaging distance detection unit 14b falls outside a predetermined distance range (for example, 3 to 4 mm), the focus control unit 16 performs focus control to maintain the focus lens 26ab at a predetermined position.

[0222] In this case, for example, when the detected distance information is greater than a first distance (e.g., approximately 4 to 5 mm), the focus control unit 16 performs focus control (pan focus control) to maintain the focus lens 26ab at a first position (first focus position) or a first' position that is farther away than the first position. Also, when the detected distance information is equal to or smaller than a second distance (e.g., approximately 1.5 to 2 mm), the focus control unit 16 performs focus control (close-point priority focus control) to maintain the focus lens 26ab at a second position (second focus position) or a second' position that is closer than the second position.

[0223] With this configuration, in the endoscopic system 1 including the focus control device of this embodiment, when in the first observation state (screening state), pan focus control can be used to perform bird's-eye observation of the entire wide field of view, making it possible to easily and quickly search for specific objects such as tumors that are desired to be observed.

[0224] Furthermore, when a specific object such as a tumor is detected and the first observation state is transitioned to the second observation state (approach state), autofocus control is performed to maintain a focused state within a predetermined area including the detected specific object, thereby enabling clear images to be continuously obtained. This allows for easy and smooth approach to the specific object while continuing to observe the desired specific object with good images.

[0225] Furthermore, when transitioning from the second observation state to the third observation state (detailed observation state), the specific object desired to be observed can be approached to the closest distance at which a focused state can be obtained, and magnified observation can be performed in a stable state at all times.

[0226] Furthermore, during an endoscopic examination, even if the imaging distance of the endoscope 20 deviates from the second distance during magnified observation in the third observation state due to, for example, the trembling of the hand of the endoscope user (doctor, etc.) holding the endoscope, the focus control may be fixed.

[0227] In the embodiment described above, the predetermined first distance (distance during screening) as the imaging distance is set to, for example, approximately 4 to 5 mm, but is not limited to this. The first distance can be changed as appropriate, for example, to a distance slightly longer than approximately 4 to 5 mm. Similarly, the predetermined distance (distance during approach) and the second distance (distance during detailed observation) can also be changed as appropriate.

[0228] The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the spirit and scope of the invention. For example, while an endoscope has been described as an example, the present invention can be applied to any device that sequentially acquires and evaluates images (such as an image inspection device). Furthermore, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining the disclosed multiple components. For example, if the problem to be solved by the invention can be solved and the effects of the invention can be obtained even if some components are deleted from all the components shown in one embodiment, the configuration from which these components are deleted can be extracted as an invention. Furthermore, components from different embodiments may be appropriately combined. The present invention is not limited by specific embodiments other than as limited by the appended claims.

Claims

1. A focus control device that adjusts focus by moving a focus lens included in an imaging optical system back and forth along the optical axis based on an output signal from an imaging unit, comprising a processor, wherein the processor acquires distance information from the tip of the imaging optical system to an object to be imaged, and when the distance information is within a predetermined distance range, performs autofocus control to adjust the position of the focus lens on the optical axis in accordance with changes in the imaging distance.

2. The focus control device according to claim 1, characterized in that the processor performs focus control to maintain the focus lens at a predetermined position when the distance information is outside the predetermined distance range.

3. The focus control device according to claim 2, characterized in that the processor performs focus control to maintain the focus lens at the first position or a first' position farther away than the first position when the distance information is greater than the first distance, and performs focus control to maintain the focus lens at the second position or a second' position closer to the second position when the distance information is equal to or smaller than the second distance.

4. A focus control device as described in claim 3, characterized in that the first position is a position of the focus lens at which a focused state can be obtained for the object to be imaged that is within a predetermined distance range that has been set in advance, and the second position is a position of the focus lens that corresponds to the shortest distance at which a focused state can be obtained for the object to be imaged.

5. The focus control device according to claim 4, wherein the first distance is 4 to 5 mm, and the second distance is 1.5 to 2 mm.

6. The focus control device according to claim 1, characterized in that the imaging unit is configured to include the imaging optical system including the focus lens and an imaging element that receives an optical image of the object to be imaged formed by the imaging optical system and performs photoelectric conversion, and the processor further obtains distance distribution information within the image based on phase difference information included in the output signal from the imaging unit, and detects the distance information based on the distance distribution information.

7. The focus control device described in claim 3, characterized in that the processor further detects a specific object, acquires vector change information about the object to be imaged in multiple images acquired continuously in time series from the imaging unit, and determines, based on the vector change information, whether the distance information has changed from a state greater than the first distance to within the range of the specified distance, or whether the distance information has changed from a state equal to the second distance or smaller than the second distance to within the range of the specified distance.

8. The focus control device according to claim 7, characterized in that the processor executes autofocus control when it determines based on the vector change information that the change in the distance information is small and that the object being imaged is moving relatively in the X and Y directions.

9. The focus control device according to claim 1, characterized in that the processor further detects the amount of illumination light irradiated from the lighting unit toward the object to be imaged based on data on the amount of light emitted from the light source device controlled in response to an output signal from the imaging unit, and detects the distance information based on the amount of illumination light.

10. The focus control device according to claim 1, characterized in that the processor further acquires brightness distribution information within the image based on the output signal from the imaging unit, and detects the distance information based on the brightness distribution information.

11. The focus control device described in claim 10, characterized in that the processor detects three large bright spots based on the brightness distribution information, and when the three large bright spots are detected, if the three large bright spots are formed at a predetermined distance from each other and a triangle with the central coordinates of the three large bright spots as vertices is formed surrounding the central area of ​​the endoscopic image, performs focus control to maintain the focus lens at the second position or a second' position closer to the second position.

12. The focus control device described in claim 10, characterized in that the processor detects a plurality of discrete bright spots based on the brightness distribution information, and when the processor detects the plurality of discrete bright spots, performs focus control to maintain the focus lens at the first position or a first' lens position that is farther away than the first position.

13. A focus control method for adjusting focus by moving a focus lens included in an imaging optical system back and forth along the optical axis based on an output signal from an imaging unit, the focus control method comprising: detecting distance information from the tip of the imaging optical system to an object to be imaged; and, if the distance information is within a predetermined distance range, performing autofocus control to adjust the position of the focus lens on the optical axis in accordance with changes in the imaging distance; if the distance information is greater than a first distance, performing focus control to maintain the focus lens at a first position or a first' position farther away than the first position; and, if the distance information is equal to or smaller than a second distance, performing focus control to maintain the focus lens at a second position or a second' position closer to the second position.

14. An endoscopic system comprising: an endoscope including an imaging unit including an imaging optical system that forms an optical image of an object, and an imaging element that photoelectrically converts the optical image formed by the imaging optical system to obtain an image signal; an endoscope including a focus adjustment mechanism that moves some of the optical lenses included in the imaging optical system forward and backward in a direction along the optical axis; and a processor, wherein the processor: detects distance information from the tip of the imaging optical system to the object to be imaged; and if the distance information is within a predetermined distance range, performs autofocus control that adjusts the position of the focus lens on the optical axis in accordance with changes in the imaging distance; if the distance information is outside the predetermined distance range and the distance information is greater than a first distance, performs focus control to maintain the focus lens at a first position or a first' position farther away than the first position; and if the distance information is equal to or smaller than a second distance, performs focus control to maintain the focus lens at a second position or a second' position closer to the second position.

15. An endoscope system according to claim 14, wherein the second distance is a distance at which the object to be imaged can be observed under magnification to obtain diagnostic findings.

Images