Focus control device, focus control method, focus control program, and endoscope system

Abstract

The present invention is a focus control device for adjusting a focus position on the basis of an output signal from an imaging unit (26) having an optical system (26b) that is provided to the distal end (21a) of an endoscope (20) insertion part (21) and moves forward and rearward in a direction following an optical axis (O) to perform focus adjustment, wherein the focus control device comprises: a distance distribution information acquisition unit (15) that, from an output signal of the imaging unit, acquires longitudinal-direction distance distribution information pertaining to an object when viewed from the imaging unit, and acquires temporal distance change information pertaining to the object; a range determination unit (15b) that determines a distance change range for the object on the basis of the distance distribution information and the distance change information; and a focus position control unit (16) that adjusts the focus position on the basis of the result of determination performed by the range determination unit.

Description

Focus control device, focus control method, focus control program, and endoscope system

[0001] The present invention relates to a focus control device, a focus control method, a focus control program, and an endoscope system that can acquire clear endoscopic images with a wide focus area regardless of the shape or state of the object being observed.

[0002] Conventionally, endoscopic systems that include an endoscope that captures images of the inside of a subject and acquires image data, a processor that performs various types of image processing on the image data acquired by the endoscope, a display device that displays the image data processed by the processor as a visible image, and a storage device that records or stores the image data have been widely used in, for example, the medical field and the industrial field.

[0003] Furthermore, in the medical field, various examinations are widely performed using this type of endoscope system to observe the inside of living organs, etc. In an examination using a medical endoscope system (hereinafter referred to as an endoscopic examination), for example, an insertion section of an endoscope inserted into the lumen of a living organ, etc., is advanced and retreated along the length of the lumen, such as by being inserted and withdrawn, while an imaging unit provided at the distal end of the insertion section captures images. At this time, the endoscopic images acquired by the imaging unit are displayed as moving images in real time chronologically on a display device. At the same time, the moving image data can also be stored in a storage device.

[0004] During such an endoscopic examination, the doctor or other equipment operator searches for lesions such as polyps or tumors occurring on the inner walls of organs as specific objects (hereinafter referred to as specific objects) while observing the real-time endoscopic image displayed on the display device (or, after the examination, a replayed endoscopic image based on recorded or stored endoscopic image data).

[0005] In general, the interior of an organ or the like containing a specific object such as a tumor that is the subject of observation in an endoscopic examination using an endoscope or the like does not have a uniform planar shape, but rather has a complex shape with protrusions and depressions. In addition, it is well known that specific objects such as tumors are often formed with unevenness on the inner wall surface of the organ or the like.

[0006] Therefore, for example, if the observation object (such as a tumor) has an uneven shape with a difference in distance in the front-to-back direction along the optical axis of the imaging optical system relative to the imaging unit of the endoscope, when focus adjustment is performed with a specific point on the surface of the object as a target, the focus state is ensured within a specific range in the front-to-back direction of the point (within the so-called depth of field).However, there are cases where the focus state cannot be ensured in areas outside the specific range.

[0007] It is generally known that the depth of field of an imaging optical system tends to become shallower as the distance from the imaging unit to the object to be imaged (hereinafter referred to as the imaging distance) becomes shorter, and to become deeper as the imaging distance becomes longer. Therefore, when an image is captured with the imaging optical system positioned closer to the object, the depth of field becomes shallower. As a result, the closer the image is to the object, the narrower the area in focus in front of and behind the object. Furthermore, areas in front of and behind the area outside the in-focus area of ​​the object become out of focus (so-called out-of-focus), which can interfere with observation.

[0008] On the other hand, since endoscopic examinations are performed to observe the inside of living organs, etc., the inside of the organ, including the specific object, is constantly moving due to, for example, the patient's heartbeat and breathing, changes in body position, or the peristaltic movement of the organ. Therefore, in endoscopic examinations in which the tip of the endoscope is positioned facing the specific object inside the living organ to perform imaging, depending on the imaging timing, sometimes an in-focus image can be obtained and sometimes an out-of-focus image can be obtained. For this reason, it has been difficult to consistently and reliably obtain clear, in-focus endoscopic images during endoscopic examinations.

[0009] Therefore, for example, when capturing an image of a subject that moves in a certain cycle and that frequently appears in a specific location among multiple focus detection areas arranged within an imaging screen, a focus adjustment device is disclosed in Japanese Patent Publication No. 9-318865 and the like that searches for the destination of the subject image in order of past appearance frequency, thereby efficiently shortening the search time.

[0010] However, the conventional focus adjustment device disclosed in the above-mentioned Japanese Patent Publication No. 9-318865 and the like attempts to search for the destination of the subject image by determining the movement pattern of the subject image.

[0011] However, the internal movements of living organs, including the specific object to be observed in endoscopic examination, are irregular and do not necessarily follow a constant periodic motion, and therefore it is understood that it is not appropriate to apply the conventional means disclosed in the above publications to devices for performing endoscopic examinations.

[0012] The object of the present invention is to provide a focus control device, a focus control method, a focus control program, and an endoscopic system that, when performing detailed observation of a specific object such as a tumor in medical equipment such as an endoscope or inspection equipment (image inspection device, etc.), ensures a focused state over a wider area and can always obtain clear, high-resolution endoscopic images regardless of the shape or condition of the object being observed.

[0013] In order to achieve the above object, one aspect of the present invention provides a focus control device that is provided at the tip of an endoscope insertion portion and adjusts the focus position based on an output signal from an imaging unit having an optical system that moves back and forth along an optical axis to adjust the focus, and is equipped with: a distance distribution information acquisition unit that acquires, from the output signal of the imaging unit, distance distribution information in the forward and backward directions of an object as seen from the imaging unit and also acquires information on changes in distance over time of the object; a range determination unit that determines a distance change range of the object based on the distance distribution information and the distance change information; and a focus position control unit that adjusts the focus position based on the determination result of the range determination unit.

[0014] A focus control method according to one aspect of the present invention acquires, from an output signal of an imaging unit, distance distribution information in the forward and backward directions of an object as seen from the imaging unit, as well as information on a change in distance over time of the object, determines a distance change range of the object based on the distance distribution information and the distance change information, and adjusts the focus position based on the determination result of the range determination unit.

[0015] A focus control program according to one aspect of the present invention causes a computer to execute a distance distribution information acquisition process that acquires, from an output signal of an imaging unit, distance distribution information in the forward and backward directions of an object as seen from the imaging unit, and acquires information on changes in distance over time of the object; a range determination process that determines a range of change in distance of the object based on the distance distribution information and the distance change information; and a focus position control process that adjusts the focus position based on the determination result of the range determination unit.

[0016] An endoscopic system according to one aspect of the present invention is an endoscopic system comprising: an endoscope including an imaging unit including an imaging optical system that forms an optical image of an object and an imaging element that photoelectrically converts the optical image formed by the imaging optical system to obtain an image signal; a focus adjustment mechanism that moves some of the optical lenses included in the imaging optical system forward and backward in a direction along the optical axis; and a processor, wherein the processor comprises: a distance distribution information acquisition unit that acquires, from an output signal of the imaging unit, distance distribution information in the forward and backward directions of the object as seen from the imaging unit and also acquires information on a change in distance over time of the object; a range determination unit that determines a distance change range of the object based on the distance distribution information and the distance change information; and a focus position control unit that adjusts the focus position based on a determination result of the range determination unit.

[0017] A second aspect of the focus control device of the present invention is a focus control device for an optical system for a tip imaging unit of an endoscope insertion portion, and includes: a specific object detection unit that, when acquiring distance distribution information in the forward and backward directions of an object in front from the output signal of the imaging unit, determines the image pattern of the object for each position corresponding to each distance indicating a different distance in the distance distribution; and a focus position control unit that adjusts the focus position based on the distance distribution information of the object position classified by the image pattern.

[0018] A third aspect of the focus control device of the present invention is a focus control device for an optical system for a distal imaging unit of an endoscope insertion portion, and includes: a specific object detection unit that, when acquiring distance distribution information in the forward and backward directions of an object in front from the output signal of the imaging unit, determines the image pattern of the object for each position corresponding to each distance indicating a different distance in the distance distribution; a range determination unit that acquires information on the change in distance of the object over time and determines the range of change in distance of the object; and a focus position control unit that adjusts the focus position based on the determination result of the range determination unit.

[0019] A focus control method according to a second aspect of the present invention, when acquiring distance distribution information in the forward and backward directions of an object in front from the output signal of an imaging unit, determines an image pattern of the object for each position corresponding to each distance indicating a different distance in the distance distribution, acquires information on the change in distance over time of the object to determine the range of change in distance of the object, and adjusts the focus position based on the determination result of the range of change in distance of the object.

[0020] A focus control program according to a second aspect of the present invention causes a computer to execute, when acquiring distance distribution information of an object in the forward and backward directions from the output signal of an imaging unit, an image pattern determination process for determining an image pattern of the object at each position corresponding to each distance indicating a different distance in the distance distribution, a range determination process for acquiring information on a change in distance of the object over time and determining a range of change in distance of the object, and a focus position control process for adjusting the focus position based on the result of the range determination.

[0021] A focus control method according to a third aspect of the present invention, when acquiring distance distribution information in the forward and backward directions of an object ahead from the output signal of an imaging unit, determines an image pattern of the object for each position corresponding to each distance indicating a different distance in the distance distribution, acquires information on the change in distance over time of the object to determine the distance change range of the object, and synthesizes the imaging results at each focus position obtained by adjusting the focus position over time multiple times based on the determination result of the distance change range of the object.

[0022] According to the present invention, it is possible to provide a focus control device, a focus control method, a focus control program, and an endoscopic system that, when performing detailed observation of a specific object such as a tumor in medical equipment such as an endoscope or inspection equipment (image inspection device, etc.), ensures a focused state over a wider area regardless of the shape or condition of the object being observed, and can always obtain clear, high-resolution endoscopic images.

[0023] 8 is a block diagram showing an overall configuration of an endoscopic system including a focus control device according to one embodiment of the present invention; a block diagram showing an outline of the internal configuration of an endoscopic system including a focus control device according to one embodiment of the present invention; an explanatory diagram of an inference model acquired as a result of learning in a learning device; a schematic diagram conceptualizing the structure of an image sensor applied to an endoscope included in the endoscopic system of FIG. 1 and explaining the principle of image plane phase difference AF by the image sensor; a flowchart explaining the operation of an endoscopic system including a focus control device according to one embodiment of the present invention; a schematic diagram illustrating an example of a situation in which a specific object is observed using an endoscope; a schematic diagram illustrating an example of a situation in which pulsation or the like occurs when a specific object is observed using an endoscope; a flowchart showing a determination process for pulsation or the like in a modified example of the operation of the focus control device according to one embodiment of the present invention; a flowchart showing a focus control switching process which is performed after the determination process for pulsation or the like in FIG. 8; a schematic diagram showing the operation during first focus control along with the change in distance distribution information over time; a schematic diagram showing the operation during second focus control along with the change in distance distribution information over time;

[0024] The present invention will be described below with reference to illustrated embodiments, for example, as an application example to an endoscope system. The drawings used in the following description are schematic. Therefore, in these drawings, each component is shown at a size that allows it to be recognized. Therefore, the dimensional relationships and scales of each component may be shown differently for each component. The present invention is not limited to the illustrated embodiments with respect to the quantities, shapes, size ratios, relative positional relationships, and the like of each component shown in the drawings.

[0025] First, before describing the detailed configuration of a focus control device according to one embodiment of the present invention, the schematic configuration of an endoscope system including the focus control device of this embodiment will be described below with reference to Figures 1 and 2. Figure 1 is a schematic diagram showing the overall configuration of an endoscope system including a focus control device according to one embodiment of the present invention. Figure 2 is a block diagram showing the schematic internal configuration of the endoscope system.

[0026] 1, the endoscope system 1 is mainly configured to include a processor 10, an endoscope 20, a display device 30, a light source device 40, a storage device 50, and an external device 60. The endoscope system 1 shown in Fig. 1 illustrates a general configuration of an endoscope system used for an endoscopic examination in which the inside of an organ, such as the upper digestive tract (esophagus, stomach, duodenum, etc.) or the lower digestive tract (large intestine, etc.) of a subject (patient, etc.), such as a living body, is observed.

[0027] In an endoscopic examination using such an endoscopic system, a doctor or other equipment operator performs so-called screening, searching for specific lesions such as polyps or tumors occurring on the inner walls of organs while observing the endoscopic image displayed on the display device.

[0028] However, in normal cases, specific objects such as lesions are very small and have similar colors to the inner walls of surrounding organs, etc., so it takes skill to accurately distinguish and detect the image area of ​​the specific object from an endoscopic image. In addition, in endoscopic examinations, the operation of the endoscope itself, such as the operation of pointing the tip of the insertion part of the endoscope (the observation window of the imaging unit) at the desired object and capturing an image, requires skill.

[0029] In general, the use states of an endoscope during an endoscopic examination can be broadly divided into, for example, a state in which the above-mentioned screening is being performed (referred to as the screening state), a state in which a detected specific object is being observed in detail (referred to as the detailed observation state), and a state in which the tip of the endoscope is in the process of transitioning from the screening state to the specific object to be observed in order to perform detailed observation (referred to as the approach state).

[0030] Conventionally, when performing endoscopic examinations using medical or inspection equipment such as endoscopes, it has been necessary to perform appropriate focus control of the imaging unit according to the usage conditions of the endoscope, etc., in order to always ensure a reliable observation environment and continue to display good images.

[0031] For example, when an endoscope or the like is in a screening state, a so-called deep focus control setting is often used, in which the position of the imaging optical system of the imaging unit is set so as to obtain a good focus state within a predetermined imaging range, without performing precise focus control, depending on the characteristics of the imaging optical system.

[0032] Furthermore, for example, when an endoscope or the like is in a detailed observation state or an approach state, it is required to appropriately perform focus control to bring the object to be imaged or observed into focus so that the object can be observed in detail with a clear image.

[0033] Therefore, for example, when performing detailed observation, in order to display the object of observation as large as possible, the tip surface of the imaging unit (the front surface of the imaging optical system) is often brought close to the surface of the object of observation (a specific object such as a tumor) to capture the image.

[0034] As shown in FIG. 1, the endoscope 20 is an image inspection device that includes an insertion section 21, an operation section 22, a universal cable 23, and the like.

[0035] The insertion section 21 is a component that is inserted into a subject such as a living organism. The insertion section 21 is formed by continuously connecting a distal section 21a, a bending section 21b, and a flexible tubular section 21c, in that order from the distal end. The insertion section 21 is formed in a generally elongated tubular shape. The insertion section 21 has a treatment tool insertion channel 21d therein, which is a conduit for inserting an endoscopic treatment tool (not shown). This treatment tool insertion channel 21d is provided so as to pass through the insertion section 21 from the distal end to the proximal end. The operation section 22 is connected to the proximal end side of the insertion section 21.

[0036] The tip portion 21a is a structural unit provided at the most distal end of the insertion portion 21. Various structural members (not shown in FIG. 1; see FIG. 2) such as an illumination unit 25 and an imaging unit 26 are provided inside the tip portion 21a.

[0037] Here, the illumination unit 25 is a structural unit that includes an optical element (illumination lens; not shown) that emits a light beam guided from the light source device 40 (described later) forward from the tip surface of the tip portion 21 a to illuminate an observation area including a lesion or the like within the subject.

[0038] The imaging unit 26 is an electronic device unit including an optical lens (imaging optical system 26a; see FIG. 2) that forms an optical image of a specific object (a lesion such as a tumor) to be observed (imaged) inside the subject, and a photoelectric conversion element (imaging element 26b; see FIG. 2) that generates image information (still image data, moving image data, etc.) based on the optical image. Although not shown, the imaging unit 26 also includes a drive mechanism (focus adjustment mechanism, magnification change mechanism, etc.) that moves some of the optical lenses (e.g., a focus lens, a zoom lens, etc.) included in the imaging optical system 26a forward and backward along the optical axis.

[0039] The bending portion 21b is a tubular portion that can be actively bent by a bending operation mechanism (not shown) that acts in conjunction with the operation of a bending operation member 22b (described later). The flexible tube portion 21c is a flexible tubular member that extends from the tip of the operation portion 22 and is connected to the base end of the bending portion 21b.

[0040] The operation section 22 is connected to the proximal end of the insertion section 21. The operation section 22 is configured to include an operation section main body 22a, a bending operation member 22b, a plurality of operation members 22c, a treatment tool insertion port 22d, and the like.

[0041] The operation unit main body 22a has a generally box-like shape as a whole and constitutes a gripping portion that is held with the fingers of a user (such as a doctor) of the endoscope 20. Various constituent units such as a bending operation mechanism (not shown) are provided inside the operation unit main body 22a. As described above, the insertion section 21 extends from the operation unit main body 22a.

[0042] The bending operation member 22b and the plurality of operation members 22c are operation members for performing various operations of the endoscope 20. The bending operation member 22b and the plurality of operation members 22c are respectively provided at predetermined positions on the outer surface of the operation portion main body 22a.

[0043] The treatment instrument insertion port 22d is provided at a predetermined position near the distal end of the operation portion main body 22a. The treatment instrument insertion port 22d is a proximal end opening of the treatment instrument insertion channel 21d of the insertion portion 21. The treatment instrument insertion channel 21d is connected at its distal end to a distal channel opening (not shown) of the distal end portion 21a.

[0044] With this configuration, an endoscopic treatment tool (not shown) inserted through the treatment tool insertion port 22d is configured to pass through the treatment tool insertion channel 21d and then protrude outward from the tip channel opening of the tip portion 21a.

[0045] The universal cable 23 is a connection cord for connecting the endoscope 20 to the light source device 40 and the processor 10. For this purpose, the universal cable 23 is made of a tubular member extending from the side of the operation unit body 22a of the operation unit 22. A scope connector 23a is provided at the tip of the universal cable 23. This scope connector 23a is connected to the front panel of the light source device 40.

[0046] An electric cable 23b extends from the scope connector 23a. A connector 23c is provided at the tip of the electric cable 23b. The connector 23c is connected to the front panel of the processor 10. Various signal transmission cables, optical fiber cables, etc. (not shown) are inserted into the universal cable 23.

[0047] The light source device 40 is a device that supplies illumination light to the illumination unit 25 (see FIG. 2) provided inside the tip portion 21a of the insertion section 21 of the endoscope 20. The illumination light emitted from the light source device 40 is transmitted to the illumination unit 25 in the tip portion 21a through an optical fiber cable (not shown) or the like that is arranged passing through the scope connector 23a, the universal cable 23, the operation section 22, and the insertion section 21. The illumination light then passes through an illumination lens (not shown) included in the illumination unit 25 in the tip portion 21a and is irradiated toward the observation target area in front of the tip portion 21a.

[0048] The processor 10 is a control device, a signal processing device, or a circuit unit that includes a control circuit and a signal processing circuit that control the entire endoscope system 1. The processor 10 is also configured to include a function as a focus control device of this embodiment (described in detail below; see FIG. 2).

[0049] A control circuit included in the processor 10 receives, for example, an operation instruction signal from the operation member 22c of the operation section 22 of the endoscope 20, and outputs various control signals for driving and controlling the imaging unit 26, the light source device 40, the illumination unit 25, etc. Furthermore, a signal processing circuit included in the processor 10 receives, for example, an imaging signal from the imaging unit 26, and performs predetermined image signal processing, etc.

[0050] For this purpose, the processor 10 and the imaging unit 26 are electrically connected by a signal transmission cable (not shown). The signal transmission cable is inserted and arranged from the connector 23c through the electrical cable 23b, the scope connector 23a, the universal cable 23, the operation section 22, and the insertion section 21 to the imaging unit at the tip end 21a.

[0051] With this configuration, control signals output from the processor 10 and imaging signals output from the imaging unit 26 are transmitted between the imaging unit 26 and the processor 10 through the signal transmission cable. Note that one form of the signal transmission cable is, for example, a composite cable in which a plurality of cables are bundled together and covered with an outer sheath shield, an outer sheath tube, or the like.

[0052] The display device 30 receives image signals and the like output from the processor 10 and displays endoscopic images and various types of information in a predetermined format. To this end, the display device 30 and the processor 10 are electrically connected using a video cable 24. The display device 30 may be configured as a display device using, for example, a general liquid crystal panel.

[0053] The storage device 50 is a storage device that saves (records or stores) image data for storage that has been generated for storage by various processes performed by the processor 10 based on image information generated by the imaging unit 26 .

[0054] As shown in Fig. 1, the storage device 50 is configured as an integral part of the processor 10. As shown in Fig. 2, a temporary storage unit 51 is provided inside the storage device 50 in addition to a main storage unit (not shown) in the storage device 50. The temporary storage unit 51 is configured, for example, by a semiconductor memory or the like, and is a component that functions as a temporary memory area for temporarily storing output signals (image data, etc.) from the imaging unit 26 or the image processing unit 11, etc.

[0055] 1 illustrates a configuration in which the storage device 50 is integrally disposed inside the housing of the processor 10, but the present invention is not limited to this. For example, the storage device 50 may be configured as an external storage device configured using a housing separate from the processor 10.

[0056] 2 shows an example of a configuration in which the temporary storage unit 51 is provided inside the storage device 50, but this is not limiting. The temporary storage unit 51 can also be provided integrally inside the storage control unit 13 (see FIG. 2), which will be described later, for example.

[0057] Furthermore, the processor 10 and the light source device 40 are not limited to being configured as separate entities, as in the example configuration shown in Fig. 1. For example, the processor 10 and the light source device 40 may be configured as an integrated unit using a single housing.

[0058] The configuration of the illumination unit 25 is not limited to the above-described configuration (a configuration in which illumination light from the light source device 40 is transmitted to the distal end portion 21a through an optical fiber cable or the like). As a configuration of the illumination unit 25 other than this, for example, a light-emitting element such as an LED (Light Emitting Diode) as an illumination light source can be provided inside the distal end portion 21a, and power supply to the illumination light source (LED) and light emission control thereof can be controlled by a predetermined control circuit included in the processor 10.

[0059] The external device 60 corresponds to, for example, a general-purpose biometric information acquisition device that detects and acquires vital sign information. The external device 60 is connected to the processor 10. With this configuration, various vital sign information acquired by the external device 60 is output to, for example, a pulse period determination unit 15a (described below) in the processor 10. With this configuration, the vital sign information can be used as reference data when the pulse period determination unit 15a determines the period of the pulse, etc.

[0060] The endoscope system 1 configured as described above basically has substantially the same configuration as conventional endoscope systems of the same type, and therefore, illustrations and detailed descriptions of other detailed configurations will be omitted.

[0061] In the endoscope system 1 configured as described above, the focus control device of this embodiment is configured as being included in the processor 10. Next, the internal configuration of the processor in the endoscope system including the focus control device of one embodiment of the present invention will be described in detail below with reference to FIG.

[0062] The processor 10 is configured to include an image processing unit 11 (including a depth stacking processing unit 11a), a display control unit 12 (including a guide display unit 12a), a memory control unit 13, a specific object image detection unit 14, a distance distribution information acquisition and determination unit 15 (including a pulsation cycle determination unit 15a and a pulsation distance range determination unit 15b), a focus control unit 16, an illumination control unit 17, a memory device 50 (including a temporary memory unit 51), etc.

[0063] The image processing unit 11 is a structural unit or circuit unit that receives an output signal (mainly image information) from the imaging unit 26 and performs various types of information processing based on the received image information. The image processing performed here includes, for example, normal image information processing, display image data processing, storage image data processing, and various other types of image processing.

[0064] The normal image processing includes an AD conversion process that converts the analog image signal output from the image sensor 26b into a digital image signal, and is a basic image information processing that is normally performed on image information. The normal image processing is assumed to be a conventional process, and a detailed description thereof will be omitted.

[0065] The display image data processing is processing for generating image data for normal display, such as processing for generating image data representing an endoscopic image to be displayed in a predetermined display area on the display screen of the display device 30. The storage image data processing is processing for generating image data for recording.

[0066] The various types of image processing are image processing that are appropriately added to normal image data for display. Specific examples of the various types of image processing include normal image adjustment processing such as brightness adjustment processing and white balance adjustment processing, and image enhancement processing such as edge enhancement processing and texture and color enhancement imaging (TXI).

[0067] Furthermore, in this embodiment, the image processing unit 11 is configured to include a depth stacking processing unit 11a. This depth stacking processing unit 11a is an image processing circuit that performs image synthesis based on multiple image data sets with different focus states for the same imaging object. An image generated by such image synthesis processing can be displayed as an image with a wider in-focus area (an image with a deeper depth of field) compared to images acquired individually.

[0068] Here, as described above, the depth stacking image processing is performed based on a plurality of image data acquired by the image sensor 26b of the imaging unit 26. To achieve this, when the endoscope 20 is set to an operating mode for performing depth stacking processing during an endoscopic examination, the processor 10 performs the following control.

[0069] That is, the processor 10 controls the driving of the imaging unit 26 (the imaging optical system 26 a and the imaging element 26 b) using the focus control unit 16 (described in detail later). In this case, the driving control of the imaging unit 26 is a control for acquiring multiple image data for the same imaging target while gradually changing the focal position.

[0070] The depth stacking processing unit 11a performs image synthesis processing based on the multiple image data thus acquired, and generates an image with a wide in-focus area (an image with a deep depth of field).

[0071] The above-described example corresponds to a case where a device operator such as a doctor operates a predetermined imaging operation switch or the like at a desired timing to acquire a desired still image, or a case where the process is automatically started by the processor 10 when predetermined conditions are met during an endoscopic examination. In addition, depth stacking processing can also be applied to normal endoscopic observation images (moving images) during an endoscopic examination.

[0072] In general, endoscopic observation images are acquired continuously at, for example, 30 frames per second (fps). When performing depth stacking on such video data, a means for wobbling the focus lens is used. Here, wobbling refers to, for example, moving the focus lens back and forth between two predetermined points.

[0073] Specifically, for example, this is an operation in which the focus lens is moved back and forth between two lens positions, one focusing on the top of the target's head and the other focusing on the base of the target.

[0074] Then, imaging operations are performed at the two positions to acquire multiple pieces of image data (two pieces in this case). The acquired multiple pieces of image data are then subjected to image synthesis processing. This makes it possible to generate a single piece of image data that focuses on the area from the top to the base of the head of the target object, for example. By performing this operation for each frame, multiple pieces of image data that are consecutive in time series can be acquired and played back as a moving image.

[0075] The display control unit 12 is a structural unit or circuit unit that controls the display device 30 so that an appropriate display format is displayed according to the display image data output from the image processing unit 11. The display control unit 12 controls the display of the display image data (various settings such as display position, display area, display size, etc.) as well as the display of various information other than image information (text information, icon information, etc.). Here, the various information other than image information includes examination date and time information, subject (patient) information, medical record information, etc., as well as various alarm information (notification information) that is displayed as appropriate during an endoscopic examination, for example.

[0076] Specifically, the display control unit 12 performs display control to continuously display the endoscopic image in the display area, and when a specific object is detected in the endoscopic image, the display control unit 12 performs display control to add a predetermined display (for example, a frame display) to the image area including the specific object.

[0077] Furthermore, the display control unit 12 includes a guide display unit 12a. The guide display unit 12a is a structural unit or circuit unit that displays various alarm information (notification information), guidance information, and the like on the display screen of the display device 30 at appropriate predetermined timings for, for example, a device operator such as a doctor during an endoscopic examination.

[0078] The storage control unit 13 is a configuration unit or circuit unit that controls the storage device 50 (including the temporary storage unit 51) so that the storage image data output from the imaging unit 26 or the image processing unit 11 is saved (recorded or stored) in an appropriate storage format. The storage control unit 13 performs storage control to permanently store image data representing an endoscopic image acquired by the endoscope 20 in a storage area of ​​the storage device 50.

[0079] The specific object image detection unit 14 is a structural unit or circuit unit that detects an image area including a specific object from an endoscopic image based on image data acquired by the endoscope 20 .

[0080] In detail, the specific object image detection unit 14 detects a specific object (such as a lesion such as a tumor) within each frame of endoscopic image that is displayed continuously based on the image data output from the image processing unit 11.

[0081] The specific object image detection unit 14 detects specific objects, for example, by performing similar image recognition (pattern matching processing, etc.) based on multiple specific object images (case images, etc.) prepared in advance, or by performing data analysis using deep learning or machine learning to detect the desired specific object.

[0082] Of course, the specific object image detection unit 14 can also determine whether a specific object (such as a lesion such as a tumor) exists within the endoscopic image (and, if so, what area of ​​the image the specific object occupies), and can also display images of blood vessel patterns and tissue mutations that can be used for diagnosis within the object area, and may be equipped with a function to determine features that a doctor would want to visually confirm from the image using techniques such as pattern determination, similar image determination, and inference models.

[0083] For example, in recent years, object detection technology using AI (Artificial Intelligence) has been developed that obtains desired inference results (e.g., detection of specific objects such as tumors) through machine learning of endoscopic images acquired by an endoscope.

[0084] Machine learning is a method of obtaining inference results about unknown matters by learning the features, time-series information, spatial information, etc. of known input information and making inferences based on the learning results. That is, in machine learning, a trained model (also called an inference model) is first obtained that enables inference of a determinable output result from specific input information.

[0085] In this case, when generating a trained model, a large amount of information with known input-output relationships is used as training data to obtain highly reliable inference results. For example, in deep learning, a network is designed using a large amount of training data so that expected outputs can be obtained for known inputs. The trained model obtained through this process can be used independently of the network that was used for training.

[0086] Therefore, by providing the processor 10 with this type of inference model, it becomes possible to accurately and quickly perform various judgments or detections, such as the detection of specific objects in the specific object image detection unit 14, the judgment of the imaging distance range in which images can be captured in focus during pan focus control, the judgment of the usage state of the endoscope during endoscopic examination, and the detection of the surface vascular pattern of a specific object (detection of information including the position of the vascular pattern within the image).

[0087] Here, the inference model is constructed roughly as follows. For example, a large amount of image data corresponding to input and output is provided as training data to a predetermined network (not shown) for constructing the inference model. Here, the training data includes image data suitable for detection or determination, such as image data of endoscopic images acquired sequentially in time series during an endoscopic examination, image data of images (such as case images) in which a specific object is captured in the endoscopic images, etc. In this case, an annotation may be set to surround the image area of ​​the specific object with a frame or the like.

[0088] By learning using a large amount of training data, a network design is determined for a specific network (not shown) so that an output corresponding to an input can be obtained. For example, Fig. 3 is an explanatory diagram of an inference model obtained as a result of learning in a specific learning device.

[0089] 3, when a group of endoscopic images (first training data group 201) containing a specific object (such as a tumor) is input, information about the specific object captured in the image and additional information such as adding a frame display surrounding the image area of ​​the specific object are obtained along with reliability information. This allows for the construction of an inference model 200 that detects the specific object from the endoscopic image.

[0090] When endoscopic image data 202 obtained during an endoscopic examination is input to the inference model 200 constructed in this manner, the inference model 200 detects an image area containing the specific object in the frame in which the specific object is captured from the endoscopic image data 202, and outputs image data (inference result image 203) in which a frame display is added to the image area of ​​the specific object. In this manner, the specific object can be detected from the endoscopic image. The image data of this image 203 can then be temporarily stored, or, if necessary, displayed in a specified display area of ​​the display device 30, providing a reference for the endoscope user during the endoscopic examination.

[0091] "Deep learning" is a multilayered version of the "machine learning" process using neural networks. A typical example is a "forward propagation neural network," which sends information from front to back and makes a judgment. In its simplest form, it requires three layers: an input layer consisting of N1 neurons, a hidden layer consisting of N2 neurons determined by parameters, and an output layer consisting of N3 neurons corresponding to the number of classes to be discriminated. The neurons in the input and hidden layers, and those in the hidden and output layers, are connected by connection weights, and a bias value is added between the hidden and output layers, making it easy to form logic gates. While three layers are sufficient for simple discrimination, increasing the number of hidden layers makes it possible to learn how to combine multiple features during the machine learning process. In recent years, models with 9 to 152 layers have become practical due to their training time, judgment accuracy, and energy consumption.

[0092] Various well-known networks may be used for machine learning. For example, R-CNN (Regions with CNN features) or FCN (Fully Convolutional Networks) using CNN (Convolution Neural Network) may be used. This involves a process called "convolution" that compresses image features, operates with minimal processing, and is strong in pattern recognition. Furthermore, "recurrent neural networks" (fully connected recurrent neural networks) that can handle more complex information and allow information analysis whose meaning changes depending on the order or sequence of information may be used.

[0093] To realize these technologies, conventional general-purpose arithmetic processing circuits such as CPUs and FPGAs can be used, but because much of the processing in neural networks involves matrix multiplication, devices specialized for matrix calculations such as GPUs (Graphic Processing Units) and Tensor Processing Units (TPUs) may also be used. In recent years, such dedicated artificial intelligence (AI) hardware "neural network processing units (NPUs)" have been designed to be integrated and embeddable with CPUs and other circuits, and may even become part of the processing circuit.

[0094] Furthermore, inference models may be obtained by employing various well-known machine learning techniques, not limited to deep learning. For example, techniques such as support vector machines and support vector regression are available. Here, learning involves calculating the weights, filter coefficients, and offsets of a classifier; other techniques include using logistic regression processing. When a machine is to make a judgment, a human must teach the machine how to make the judgment. In this embodiment, a method for deriving an image judgment using machine learning is employed. However, a rule-based method for applying rules acquired by humans through experience or heuristics to make a specific judgment may also be used.

[0095] The distance distribution information acquisition determination unit 15 is a structural unit or circuit unit that acquires distance distribution information (hereinafter simply referred to as distance distribution information or depth data) within the imaging screen based on the output signal (image information or phase difference information; described later) from the imaging element 26b, and also acquires information on the temporal change in distance in the forward and backward directions of the object.

[0096] Here, distance distribution information (depth map) within the captured image refers to information that represents the distribution of distances in the forward and backward directions for objects, including specific objects that are the subject of observation, that are captured within the captured image.

[0097] Furthermore, the forward / backward direction of an object refers to the forward / backward direction (in other words, the depth direction) along the optical axis of the imaging optical system when viewing an object including a specific target from the imaging unit 26 (image sensor 26b) provided at the tip 21a of the endoscope 20, the direction of advancement and retreat of the endoscope 20, and the direction along the insertion axis of the endoscope 20.

[0098] As described above, the image pickup element 26b included in the imaging unit 26 of the endoscope 20 applied in this embodiment has a plurality of pixels that receive light beams from a subject and output image signals, and some or all of these plurality of pixels are applied as elements configured to perform imaging plane phase difference detection and contribute to autofocus control (AF control), predetermined focus control, etc. In this embodiment, the application of this type of image pickup element 26b makes it possible to acquire image information and phase difference information.

[0099] Here, a configuration that enables image-surface phase difference detection using an image sensor will be briefly described below. Fig. 4 is a schematic diagram illustrating the principle of image-surface phase difference AF control that performs image-surface phase difference detection using an image sensor.

[0100] 4 indicates one pixel of the image sensor 26b. The pixel 260 has a photodiode 262 and a microlens 261. Here, the photodiode 262 is formed in a two-part configuration, for example, a right photodiode 262R and a left photodiode 262L.

[0101] The microlens 261 is an optical lens that transmits a light beam that has passed through the imaging optical system 26a and forms an optical image on the light receiving surface of the photodiode 262. The photodiode 262 is a photoelectric conversion element that converts the optical image formed by the microlens 261 into an electrical signal and outputs it.

[0102] The right photodiode 262R receives a light beam LL from the left region (the region indicated by left-hand diagonal hatching in FIG. 4) and forms an image on the light-receiving surface, while the left photodiode 262L receives a light beam LR from the right region (the region indicated by right-hand diagonal hatching in FIG. 4) and forms an image on the light-receiving surface.

[0103] The pixel 260 configured as described above can calculate the distance between the two images (parallax) from the output signals (phase difference information) of the left and right photodiodes 262R and 262L to determine the defocus amount for the object in the imaging optical system. Furthermore, by combining the output signals of the left and right photodiodes 262R and 262L, the combined signals can be treated as an image signal for one pixel.

[0104] The distance distribution information acquisition determination unit 15 receives the phase difference information from the output signal acquired by the image sensor 26b in this manner, and acquires distance distribution information (depth map) of a predetermined region within the imaging surface (e.g., a region including a specific object). In this case, the distance distribution information acquisition determination unit 15 functions as a distance distribution information acquisition unit.

[0105] 4, one photodiode is divided into two and the signals from the left and right photodiodes are handled separately to perform pupil division of the microlens, but the present invention is not limited to this configuration. For example, a configuration in which pupil division is performed by using a light-shielding member can be similarly realized.

[0106] Furthermore, the distance distribution information acquisition determination unit 15 acquires information on the change in distance over time in the forward and backward directions of the object based on multiple image data (hereinafter referred to as time-series image data) output continuously in time series from the imaging unit 26.

[0107] Furthermore, the distance distribution information acquisition determination unit 15 determines the unevenness range of the specific object based on the acquired distance distribution information (depth data), and determines the required depth of field according to the unevenness range. In this case, the distance distribution information acquisition determination unit 15 functions as an object unevenness range determination unit and also functions as a depth of field determination unit.

[0108] Here, the unevenness range of a specific object refers to the range of distance difference in the front-to-back direction (depth direction) relative to the imaging unit 26, for example, when the specific object is formed in a shape that protrudes from the inner wall surface of an organ, etc., or when the surface of the specific object is formed in an uneven shape.

[0109] When functioning as an object unevenness range determination unit, the distance distribution information acquisition determination unit 15 determines the distance difference between the unevenness of the object in the front-to-rear direction relative to the imaging unit 26. When functioning as a depth-of-field determination unit, the distance distribution information acquisition determination unit 15 determines the depth of field that can include the determined distance difference between the unevenness of the object in the focus range.

[0110] In the above-described configuration example, the distance distribution information acquisition determination unit 15 acquires the distance distribution information (depth map) from phase difference information from the image sensor 26 b having a plurality of pixels for acquiring image signals and detecting phase differences on the imaging surface, but the configuration is not limited to this. For example, in a typical image sensor, all pixels provided on the imaging surface acquire imaging signals.

[0111] In an imaging unit having such a conventional imaging element 26b, for example, at least one of the optical lenses constituting the imaging optical system 26a, which is an optical lens that contributes to focus control (hereinafter referred to as the focus lens), is slightly moved back and forth along the optical axis to change the lens position. This causes a change in the contrast of objects in the image, thereby changing the focus state. At this time, distance information can be obtained from the correlation between the lens position and the focus state corresponding to the lens position. Based on the distance information thus obtained, distance distribution information (depth map) within the screen can be obtained. In other words, a configuration can be adopted in which distance distribution information (depth map) is obtained based on image signals from the imaging element 26b.

[0112] In this way, the distance distribution information acquisition determination unit 15 acquires a depth map based on the output signal (phase difference information or image information) of the image sensor 26 b. That is, the endoscopic image generated based on the output signal (image signal) of the image sensor 26 b and the depth map generated based on the output signal (phase difference information or image information) of the image sensor 26 b are related to and correspond to each other.

[0113] The distance distribution information acquisition / determination unit 15 further includes a pulsation period determination unit 15a and a pulsation distance range determination unit 15b.

[0114] Of these, the pulsation period determination unit 15a is a configuration unit or circuit unit that detects and determines the period of the change in distance over time of the object caused by pulsation or breathing occurring in the object to be imaged inside a living organ, or a change in body position, or the peristaltic movement of the organ (hereinafter, these are collectively referred to as pulsation, etc.). The pulsation period determination unit 15a detects and determines the period of the change in distance over time of the object based on the distance distribution information and periodic distance change information acquired by the distance distribution information acquisition and determination unit 15.

[0115] The pulsation distance range determination unit 15b is a configuration unit or circuit unit that detects and determines the range of the forward and backward temporal distance change of an object caused by pulsation, etc., based on the distance distribution information and distance change information acquired by the distance distribution information acquisition determination unit 15.

[0116] In general, the period of peristaltic movement of the digestive tract is typically about 3 to 20 times per minute. Furthermore, the heart normally beats about 50 to 100 times per minute when at rest. Meanwhile, the image sensor 26b used in typical endoscopes is generally driven at 30 frames per second (fps) or 60 frames per second (fps). Therefore, compared to the speed at which the image sensor 26b can acquire image data for one frame, the pulsation of a living body can be said to move at a very slow period.

[0117] The focus control unit 16 is a configuration unit or circuit unit that drives and controls the imaging optical system 26a of the imaging unit 26. The focus control unit 16 controls the driving of the imaging optical system 26a according to, for example, a set imaging operation mode (e.g., depth stacking imaging mode, etc.) or based on the detection result of the specific object image detection unit 14 or the determination result of the distance distribution information acquisition and determination unit 15, and executes focus control such as a predetermined automatic focus adjustment operation (hereinafter referred to as autofocus control) or pan focus control.

[0118] In addition, the focus control unit 16 controls the driving of the imaging optical system 26a, adjusts the relative position on the optical axis between the focus lens and the imaging element, and functions as a focus position control unit that maintains a predetermined focus lens position.

[0119] The illumination control unit 17 is a structural unit or circuit unit that drives and controls the light source device 40 and the illumination unit 25. The illumination control unit 17 controls the light source device 40 or the illumination unit 25 in response to operation instruction signals from various operation members 22c provided in the operation unit 22, or instruction signals based on the detection results of the specific object image detection unit 14 or the determination results of the distance distribution information acquisition and determination unit 15, and performs switching control of the light source type (white light, special light, etc.).

[0120] Here, the light source types include, for example, white light that is irradiated onto the subject when performing normal observation, as well as predetermined special light that is irradiated onto the subject when performing image-enhanced observation such as narrow band imaging (NBI) or red dichromatic imaging (RDI).

[0121] The above is the configuration of the endoscope system 1 including the focus control device of this embodiment. Other configurations that have not been described are the same as those of a conventional endoscope system with a general configuration.

[0122] It should be noted that all or part of the processor 10 includes hardware. Here, the processor 10 is configured by a well-known configuration including, for example, a central processing unit (CPU), random access memory (RAM), read only memory (ROM), non-volatile memory, non-volatile storage, and a non-transitory computer readable medium, as well as peripheral devices thereof.

[0123] Software programs to be executed by the CPU and fixed data such as data tables are stored in advance in ROM, nonvolatile memory, nonvolatile storage devices, etc. The CPU reads out the software programs stored in ROM, etc., expands them into RAM, and executes them, and the software programs refer to various data, etc. as appropriate, thereby realizing the functions of the processor 10.

[0124] The processor 10 may also be configured with a semiconductor chip such as an FPGA (Field Programmable Gate Array), etc. Furthermore, the processor 10 may also be configured with an electronic circuit.

[0125] Furthermore, the software program may be recorded or stored, in whole or in part, as a computer program product on a portable disk medium such as a flexible disk, CD-ROM, or DVD-ROM, or on a non-transitory computer readable medium such as a card-type memory, HDD (Hard Disk Drive) device, or SSD (Solid State Drive) device.

[0126] The operation of the endoscope system 1 including the focus control device of this embodiment configured as described above will be described below. Fig. 5 is a flowchart explaining the operation of the endoscope system including the focus control device of one embodiment of the present invention. The flowchart shown in Fig. 5 is a processing sequence explaining the operation when an endoscopic examination is performed using the endoscope system 1.

[0127] First, it is assumed that the endoscope system 1 is activated and ready to perform an endoscopic examination. At this time, it is assumed that the endoscope 20 in the endoscope system 1 is inserted into the organ of a subject (e.g., a patient) to be examined. It is assumed that the focus control setting in the initial state after activation of the endoscope 20 is, for example, a pan-focus control setting.

[0128] When the endoscopic system 1 of this embodiment is in this state, in step S1 of Figure 5, the specific object image detection unit 14 of the processor 10 performs image detection processing of a specific object such as a tumor based on the endoscopic image data sequentially acquired by the imaging unit 26, and confirms whether or not a specific object to be observed has been detected.

[0129] If the detection of a specific object is confirmed, the process proceeds to the next step S2. If the detection of a specific object is not confirmed, the process proceeds to step S8. Note that the specific object image detection process of step S1 is continuously executed while the endoscopic examination is being performed.

[0130] Furthermore, the detection process of the specific object in the process of step S1 described above is performed by the specific object image detection unit 14, but the method is not limited to this. For example, when a device operator such as a doctor recognizes a specific object by visually observing the display on the display device 30, it may be determined that the specific object has been detected.

[0131] In this case, whether or not a specific object has been detected can be confirmed by, for example, checking an output signal output when a device operator such as a doctor operates a predetermined operating member. When a device operator such as a doctor recognizes a specific object, in most cases, the device operator will operate the distal end face of the endoscope insertion portion toward the recognized specific object, and will also perform operations such as moving the distal end face of the endoscope insertion portion closer to the specific object. By detecting this series of operations, it can be determined that the device operator such as a doctor has recognized the specific object.

[0132] In the endoscope system 1 of this embodiment, a search for a specific object is performed using the pan-focus control setting, and when the specific object is detected, the system switches to autofocus control, which automatically performs focus adjustment with a predetermined focus point as a target. Furthermore, when a specific object is detected, in order to observe the specific object, a device operator such as a doctor performs an operation to bring the endoscope 20 closer to the specific object or to fix the endoscope 20 in a predetermined position facing the specific object.

[0133] In the next step S2, the processor 10 receives image data (image information and phase difference information) sequentially output from the image sensor 26b of the imaging unit 26 and starts a process of acquiring image data and depth data for each frame. At the same time, the processor 10 starts a process of temporarily storing various data in the temporary storage unit 51 of the storage device 50 via the storage control unit 13.

[0134] The acquired depth data is data indicating, for example, the unevenness of a specific object. A specific object inside a living organ may, for example, be present in a form that protrudes from the inner wall surface of a lumen of the organ, as shown in Fig. 6. Fig. 6 is a schematic diagram illustrating an example of a situation in which a specific object is observed using an endoscope.

[0135] In Fig. 6, reference numeral 300 denotes an organ or the like. Reference numeral 301 denotes a specific object protruding from the inner wall surface of the organ or the like 300. Reference numeral 301a denotes the top of the specific object 301. Reference numeral 301b denotes the base of the specific object 301 that is closer to the endoscope 20. In Fig. 6, reference numeral O denotes the optical axis of the imaging optical system 26a, and reference numeral V denotes the field of view of the imaging optical system 26a.

[0136] The situation shown in Fig. 6 illustrates a state in which the distal end surface of the distal end portion 21a of the insertion section 21 of the endoscope 20 is positioned facing a specific object 301 detected on the inner wall surface of an organ or the like 300. In this case, in the example shown in Fig. 6, for example, the distance between the distal end surface of the distal end portion 21a of the endoscope 20 (the front surface of the imaging optical system 26a) and the vertex 301a of the specific object 301 is indicated by symbol L1. Similarly, for example, the distance between the distal end surface of the distal end portion 21a of the endoscope 20 (the front surface of the imaging optical system 26a) and the base 301b on the near side of the specific object 301 is indicated by symbol L2. Note that in this case, since distance L1 is greater than distance L2, distance L1 is referred to as the far point distance, and distance L2 is referred to as the near point distance.

[0137] In this case, it is assumed that autofocus control is executed with a predetermined part (for example, the vicinity of the top of the head 301a) of the specific object 301 as the focus point. Here, for example, the symbol DOF1 in FIG. 6 indicates the depth of field.

[0138] In such a situation, it is desirable that the focus point set when performing autofocus control be set so that, for example, the top 301a and the nearer base 301b of the specific object 301 are within the depth of field DOF1 (i.e., so that they are in focus). The focus point can be set based on depth data of the image area including the object 301. In the situation shown in Figure 6 (where the depth of field is DOF1), for example, the area in focus on the surface of the specific object 301 is the area indicated by the thick solid line and denoted by F1.

[0139] 5, in the next step S3, the processor 10 checks an internal clock (not shown) to determine whether a predetermined time has elapsed since the start of the processing in step S2. If it is determined that the predetermined time has elapsed, the process proceeds to the next step S4. If the predetermined time has not elapsed, the process proceeds to step S8.

[0140] The reason why the passage of a predetermined time is confirmed in the processing of step S3 is as follows: By repeating the processing of step S2 for a predetermined time, it is possible to continuously acquire multiple image data and depth data in chronological order. By performing predetermined processing in the processor 10 based on the multiple data thus acquired, it is possible to acquire information on the change in distance over time of the object and the period of the distance change.

[0141] In this way, in order to obtain distance change information, etc., after the detection of the object (step S1), it is necessary to fix the position of the endoscope 20 relative to the detected object for a predetermined time and perform an operation to continue imaging. For this reason, when the detection of the object is confirmed in the processing of the above-mentioned step S1 and the device user, such as a doctor, wants to perform a detailed observation of the detected object, a predetermined notice may be displayed to the user (for example, a guide display such as ``Please fix the endoscope'').

[0142] In step S4, the processor 10 determines whether or not pulsation or the like is occurring in the detected specific object based on the various data acquired in the process of step S2 described above (pulsation or the like determination process).

[0143] Here, a situation where pulsation or the like occurs inside a biological organ or the like will be briefly explained below with reference to Fig. 7. Fig. 7 is a schematic diagram illustrating a situation where pulsation or the like occurs when observing a specific object using an endoscope. Note that, among the symbols used in Fig. 7, the same symbols as those in Fig. 6 indicate the same things and events.

[0144] The diagram indicated by the solid line in Fig. 7 (see symbol [A]) shows the same situation as in Fig. 6. The diagram indicated by the dashed line in Fig. 7 (see symbol [B]) shows the situation after a change over time due to pulsation or the like from the situation in Fig. 6. That is, Fig. 7 shows the situation in which the specific object 301 changes between the situation indicated by [A] and the situation indicated by [B] in a predetermined cycle in the direction indicated by arrow S.

[0145] Here, the symbol DOF1 indicates the depth of field when autofocus control is performed with the focus point near the top of the head 301a of the specific object 301 in the situation (solid line) shown in [A] in Figure 7. The in-focus area at this time is indicated by the thick solid line area indicated by the symbol F1 (similar to Figure 6).

[0146] 7, the symbol DOF2 indicates the depth of field when autofocus control is performed with a predetermined part (for example, near the top of the head 301aa) of the specific object 301 as the focus point in the situation (dashed line) shown in [B] of Fig. 7. The in-focus area at this time is indicated by the thick dashed line area indicated by the symbol F2.

[0147] Consider a situation in which the specific object 301 is changing over time due to pulsation, etc., and the endoscope 20 is fixed in a predetermined position for observation. In this case, as shown in Figure 7, the distance between the distal end surface of the endoscope 20 and the top 301a of the specific object 301 differs between the situation shown in [A] and the situation shown in [B].

[0148] 7, specifically, for example, the far point distance between the distal end surface of the endoscope 20 and the parietal portion 301a in the situation shown by [A] is designated by the symbol L1a, while the far point distance between the distal end surface of the endoscope 20 and the parietal portion 301a in the situation shown by [B] is designated by the symbol L1b.

[0149] In this case, if the far point distance L1a is greater than the far point distance L1b, the depth of field DOF1 in the situation shown in [A] differs from the depth of field DOF2 in the situation shown in [B], and the in-focus areas F1 and F2 are also different.

[0150] Therefore, when observing (imaging) a specific object 301 with the endoscope 20 fixed at a predetermined position inside a living organ or the like, if the position of the specific object 301 changes due to pulsation or the like, the multiple endoscopic images acquired continuously in time series will have different distance distribution information for each frame and will also have different focus areas centered on the object.

[0151] 5, if it is determined in step S4 that a pulsation or the like has been detected, the process proceeds to step S5, whereas if a pulsation or the like has not been detected, the process proceeds to step S8.

[0152] In step S5, the processor 10 checks whether the first focus control is feasible based on the depth data and distance change information of the endoscopic image including the object. Here, the first focus control is a control for setting a focus position that can obtain a depth of field that can cover both the depth of field DOF1 and the depth of field DOF2 based on the unevenness range (distance difference in the front-to-back direction) of the object and the distance change range (distance difference in the front-to-back direction).

[0153] Specifically, for example, from the unevenness range information of the specific object 301 in the situation shown in [A] of Figure 7 (distance information between the top 301a and the base 301b) and the unevenness range of the specific object 301 in the situation shown in [B] of Figure 7 (distance information between the top 301aa and the base 301bb), the distance information of the farthest point (top 301a in [A]) and the distance information of the closest point (base 301bb in [B]) are obtained, and a focus position is set that can obtain a depth of field DOF3 (see Figure 7) that covers these two points (far points).

[0154] Of course, the above-mentioned terms "farthest point" and "nearest point" may be excluded from the unevenness range of the specific object 301 if they are parts that are not relevant to diagnosis, etc. To do this, a technique for determining whether a location is not relevant to diagnosis, etc. is required. Examples of such a determination technique include techniques such as pattern determination from an image, similar image determination, and inference modeling, which can be used to determine features such as blood vessel patterns and tissue variations that are used in diagnosis. It is conceivable that such a function is provided in the specific object detection unit 14.

[0155] In other words, the focus control device is a focus control device for an optical system for a distal imaging unit of an endoscope insertion portion, and when acquiring distance distribution information in the forward and backward directions of an object in front from the output signal of this imaging unit, it determines the image pattern of the object at each distance indicating the distance distribution, and further acquires information on the change in distance over time of the object, and performs focus position control to adjust the focus position based on the distance distribution information of the object position classified by the image pattern and the distance change information.

[0156] Before determining the change in distance over time, the focus control device of the optical system for the tip imaging unit of the endoscope insertion portion may determine the image pattern of the object for each position corresponding to each distance showing a different distance distribution when acquiring distance distribution information in the forward and backward directions of the object in front from the output signal of the imaging unit.

[0157] Here, "for each corresponding position" means that if the distance distribution is divided into three types, for example, 1 mm, 2 mm, and 3 mm, the pattern of the object's position corresponding to 1 mm is determined, the pattern of the object's position corresponding to 2 mm is determined, and the pattern of the object's position corresponding to 3 mm is determined.

[0158] In this case, for example, if there is no pattern that can be used as a diagnostic reference in the object area at a distance of 3 mm, priority can be given to control that focuses on distances of 1 mm and 2 mm, making control easier.

[0159] A pattern is determined for each location of each distance that makes up the distance distribution, but there may be multiple locations for each distance, and the pattern for each location is determined. If there are multiple locations corresponding to each distance, the pattern for each location is determined, and if any one of the multiple locations is important for diagnosis, it is set as the focus distance.

[0160] By adjusting the focus position based on the distance distribution information obtained by taking into account the object positions classified by image patterns as to whether they are important for diagnosis, natural images can be obtained with reasonable control. Such "unimportant for diagnosis" locations can be checked again from a different perspective.

[0161] If it is determined in step S5 that the first focus control can be performed, the process proceeds to step S9, where the first focus control is executed, and then the process proceeds to step S10.

[0162] In step S10, the processor 10 fixes the focus position using the focus control unit 16. After that, the process proceeds to step S8.

[0163] On the other hand, if it is determined in step S5 that it is difficult to deal with the problem using the first focus control, the process proceeds to step S6.

[0164] In step S6, the processor 10 checks whether the second focus control is feasible based on the period of the heartbeat or the like. Here, the second focus control is a focus control that acquires multiple image data sets with different focus states for the same object. Note that the second focus control includes so-called depth stacking image processing that is performed based on the acquired multiple image data sets.

[0165] For example, when a distance change period of a specific object (e.g., a heartbeat of about 50 to 60 times per minute) is detected, and the distance change period is sufficiently slow compared to the frame rate of the imaging unit 26 (e.g., 30 frames per second), it is considered possible to acquire multiple image data at appropriate timing during the period of one heartbeat, etc. Therefore, in this case, it can be determined that the second focus control is possible.

[0166] Specifically, for example, if a distance change period of approximately 50 to 60 times per minute is detected, and the frame rate of the imaging unit 26 is, for example, 30 frames per second, multiple image data can be acquired at appropriate timing during a single heartbeat or other period. For example, first, AF imaging of the vertex 301a is performed in the situation [A] of FIG. 7 , second, AF imaging of the base 301b is performed in the situation [A] of FIG. 7 , third, AF imaging of the vertex 301aa is performed in the situation [B] of FIG. 7 , and fourth, AF imaging of the base 301bb is performed in the situation [B] of FIG. 7 . This allows four pieces of image data to be acquired. Then, depth stacking processing is performed on these four pieces of image data. The image displayed based on the image data generated in this manner can be displayed as an image with a deeper depth of field.

[0167] If it is determined in step S6 that the second focus control can be performed, the process proceeds to step S7, where the second focus control is executed, and then the process proceeds to step S8.

[0168] On the other hand, if it is determined in step S6 that it is difficult to deal with the problem using the second focus control, the process proceeds to step S11.

[0169] In step S11, the processor 10 executes a third focus control. Then, the process proceeds to step S8. Here, the third focus control is an autofocus control that tracks the focus point in accordance with the movement of a specific object based on the detected distance change period of the object. In this case, if it is not possible to track the movement of the object, the processor 10 may switch to a control that performs normal autofocus control for each frame.

[0170] In step S8, the processor 10 confirms an instruction to terminate the ongoing endoscopic examination. Here, the instruction to terminate the endoscopic examination may be, for example, a predetermined examination termination instruction signal generated by operating a predetermined operating member among the processor 10 of the endoscopic system 1 or the operating members 22c provided on the operating unit 22 of the endoscope 20. In addition, the termination of the endoscopic examination can also be determined, for example, by checking image data of the endoscopic image acquired by the endoscope 20.

[0171] If the instruction to end the test is confirmed in the process of step S8, the series of processes is ended (END). If the instruction to end the test is not confirmed, the process returns to the process of step S1 and the subsequent processes are repeated.

[0172] As described above, in the endoscopic system 1 including the focus control device of this embodiment, the pulsation of the object is determined by detecting information on the change in distance over time of the detected specific object, the period of the change in distance, the range of the change in distance in the forward and backward directions, etc. from the distance distribution information (depth map) obtained based on the image data for each frame continuously acquired in time series (see steps S2 to S4 in Figure 5).

[0173] Then, by switching focus control appropriately and performing imaging operations according to the shape and condition of the object, the period of the object's pulsation, etc., it is possible to acquire endoscopic images in which the in-focus state is ensured over a wider area (see steps S5 to S11 in FIG. 5). In this way, it is possible to consistently acquire clear, high-definition endoscopic images, which contributes to the efficiency of endoscopic examinations and the implementation of highly accurate examinations.

[0174] Next, a modified example of the operation of the endoscope system 1 including the focus control device of the above embodiment, particularly the process of determining pulsation, etc., will be described in detail using the flowchart of Fig. 8. Fig. 8 is a flowchart showing the process of determining pulsation, etc., in the modified example of the operation of the focus control device of this embodiment.

[0175] First, in step S21 of Fig. 8, the processor 10 executes an object detection process. This object detection process corresponds to the process of step S1 of Fig. 5. Therefore, the details are as described above. If an object is detected, the process proceeds to the next step S22. If an object is not detected, the process is repeated.

[0176] When an object is detected and the process proceeds to step S22, in step S22 the processor 10 executes autofocus control (focus adjustment operation) on the detected object.

[0177] Generally, when an object is detected, the endoscope is operated so that the detected object is imaged in the approximate center region of the image sensor 26b. Furthermore, when autofocus control is performed, the focus point is often set, for example, based on the center region of the image capture screen. Therefore, in this case, it is assumed that autofocus control is performed with the focus point set near the top of the object's head.

[0178] Next, in step S23, the processor 10 performs an imaging operation and temporarily records the acquired image data in the temporary storage unit 51. The image data recorded at this time is designated by the symbol lm1.

[0179] Next, in step S24, the processor 10 records various information, such as in-focus position information, when the image data 1m1 was captured. The information about the in-focus position (focus position) recorded at this time is indicated by the symbol Z1. This in-focus position information Z1 is data corresponding to the image data 1m1.

[0180] Next, in step S25, the processor 10 performs a process of comparing previous image data recorded before the image data lm1 with the current image data lm1.

[0181] In the next step S26, the processor 10 checks whether there is an image change between the previous image data and the current image data lm1. Here, the image change refers to, for example, a change in distance distribution information. In this case, the processor 10 (distance distribution information acquisition and determination unit 15) functions as a change determination unit that determines a change in distance in the forward and backward directions of the object. Here, if it is confirmed that there is an image change, the process proceeds to step S31. On the other hand, if there is no image change, the process proceeds to step S27.

[0182] In step S27, the processor 10 maintains and fixes the focus lens position based on the currently set in-focus position information Z1, and then returns to the process of step S21 to repeat the subsequent processes.

[0183] On the other hand, if an image change is confirmed in the process of step S26 described above and the process proceeds to step S31, the processor 10 executes the object detection process again in step S31. This object detection process is substantially the same as the process of step S21 described above.

[0184] If an object is detected, the process proceeds to the next step S32. If an object is not detected, it is considered that, for example, the object detected last time (in the process of step S21) has been lost, or the camera has switched to search mode without observing the object. In this case, the focus control setting is switched to the pan focus setting, and the process returns to the process of step S21 described above. Then, the subsequent processes are repeated.

[0185] Next, when an object is detected in the process of step S31 described above and the process proceeds to the process of step S32, the processor 10 executes autofocus control (focus adjustment operation) for the detected object in step S32. This process is the same as the process of step S22 described above.

[0186] Subsequently, in step S33, the processor 10 performs an imaging operation and temporarily records the acquired image data in the temporary storage unit 51. The image data recorded at this time is designated by the symbol lm2.

[0187] Next, in step S34, the processor 10 records various information, such as in-focus position information, when the image data lm2 was captured. The information about the in-focus position (focus position) recorded at this time is indicated by symbol Z2. This in-focus position information Z2 is data corresponding to the image data lm2.

[0188] Next, in step S35, the processor 10 performs a process of comparing the previous image data lm1 with the current image data lm2.

[0189] In the next step S36, the processor 10 checks whether there is an image change between the previous image data lm1 and the current image data lm2. In this case, the processor 10 (distance distribution information acquisition and determination unit 15) functions as a change determination unit that determines a change in distance in the forward and backward directions of the object. If an image change is confirmed, it is determined that there is a possibility that the object is pulsating, and the process proceeds to step S38. If there is no image change, the process proceeds to step S37.

[0190] In step S37, the processor 10 maintains and fixes the focus lens position based on the currently set in-focus position information Z2, and then returns to the process of step S31 to repeat the subsequent processes.

[0191] Meanwhile, in step S38, the processor 10 controls the guide display unit 12a of the display control unit 12 to display the pulsation determination result using the display device 30, etc. In this case, the display of the pulsation determination result is a display that notifies or explains the current examination status, etc. to the device operator. Specifically, for example, a text display such as information notifying that the detected object under observation may be pulsating, or information urging caution not to move the position of the endoscope during operation, or an icon display consisting of predetermined pictograms, etc., may be considered.

[0192] Next, in step S41, the processor 10 executes the object detection process again. This object detection process is substantially the same as the processes in steps S21 and S31 described above.

[0193] If an object is detected, the process proceeds to the next step S42. If an object is not detected, the focus control setting is switched to the deep focus setting, and the process returns to the above-mentioned step S21. Then, the subsequent steps are repeated.

[0194] Next, when an object is detected in the process of step S41 described above and the process proceeds to the process of step S42, the processor 10 executes autofocus control (focus adjustment operation) for the detected object in step S42. This process is similar to the processes of steps S22 and S32 described above.

[0195] Subsequently, in step S43, the processor 10 performs an imaging operation and temporarily records the acquired image data in the temporary storage unit 51. The image data recorded at this time is designated by the symbol lm3.

[0196] Next, in step S44, the processor 10 performs a process of comparing the image data lm1 with the current image data lm3.

[0197] In the next step S45, the processor 10 checks whether the image data lm3 is similar to the image data lm1. If the image data lm3 is similar, it is determined that the object is experiencing pulsation or the like. In this case, the processor 10 (distance distribution information acquisition / determination unit 15) functions as a change determination unit that determines distance changes in the front-to-back direction of the object. Note that, if it is determined that the object is experiencing pulsation or the like, the processor 10 also acquires information such as the period of pulsation or the like (period of distance change) and the range of distance change in the front-to-back direction. Then, the process proceeds to the processing sequence (focus control switching process) of FIG. 9.

[0198] On the other hand, if it is determined in step S45 that the image data lm1 and the image data lm3 are not similar to each other, the process proceeds to step S46.

[0199] In step S46, the processor 10 resets the determination that there is a possibility that the object is pulsating (see Y in S36 in FIG. 8), and then returns to the process of step S21 to repeat the subsequent processes.

[0200] FIG. 9 is a flowchart showing the focus control switching process that is performed after the process of determining pulsation, etc. in FIG.

[0201] In step S51 of Fig. 9, the processor 10 determines whether or not pulsation or the like is occurring in the object (pulsation or the like determination process). The process performed here is the same as the process of step S4 of Fig. 5. If it is determined that pulsation or the like is occurring in the object, the process proceeds to step S52. If it is determined that pulsation or the like is not occurring in the object, the process returns to step S21 of Fig. 8 (see reference numeral 9B in Figs. 8 and 9).

[0202] Next, in step S52, the processor 10 executes a first focus control, which is the same process as step S9 in FIG.

[0203] In step S53, the processor 10 checks whether a cycle of a pulsation or the like has been detected. If a cycle of a pulsation or the like has been detected, the process proceeds to step S58. If a cycle of a pulsation or the like has not been detected, the process proceeds to step S54.

[0204] In step S54, the processor 10 compares the image data lm1 with the image data lm2.

[0205] In step S55, the processor 10 checks whether the image data lm1 and lm2 are similar. If the image data lm1 and lm2 are not similar, the processor 10 proceeds to step S56. If the image data lm1 and lm2 are similar, the processor 10 proceeds to step S57.

[0206] In step S56, the processor 10 maintains and fixes the focus lens position based on the currently set in-focus position information, and then returns to the processing of step S41 in Fig. 8 (see reference numeral 9A in Figs. 8 and 9).

[0207] In step S57, the processor 10 resets the determination of the pulsation of the object (see Y in S51 in FIG. 9), and then returns to the processing of step S21 in FIG. 8 (see reference numeral 9B in FIGS. 8 and 9).

[0208] On the other hand, if a period such as a pulsation is detected in step S53, the process proceeds to step S58, where the processor 10 executes either the second focus control or the third focus control. The second focus control is the same process as step S7 in Fig. 5. The third focus control is the same process as step S11 in Fig. 5. Either of the focus controls may be selected and set as needed.

[0209] Here, the actions during the first to third focus control will be briefly explained using the drawings. FIG. 10 is a schematic diagram showing the action during the first focus control along with the change in distance distribution information over time. In FIG. 10, the vertical axis represents the reciprocal (1 / D) of the distance D based on the distance distribution information. This means that the further upward from the origin 0 in FIG. 10, the closer the distance distribution is to the origin, and the closer the distance distribution is to the origin, the farther the distance distribution is to the origin. In FIG. 10, the horizontal axis represents time. Moving to the right from the origin in FIG. 10 indicates the passage of time.

[0210] 10, the graph [HD] shows the change over time in the distance distribution of the top of the target's head, the graph [BT] shows the change over time in the distance distribution of the top of the target's head, and the graph [Focus] (dashed line) shows the change over time in the focus point.

[0211] In each period indicated by symbols [A1], [A2], ... in Fig. 10, the object is in the state indicated by [A] in Fig. 7. In addition, in each period indicated by symbols [B1], [B2], [B3], ... in Fig. 10, the object is in the state indicated by [B] in Fig. 7. In other words, the object changes between the state [A] and the state [B] at a predetermined time interval. This state is referred to as a state in which pulsation, etc., is occurring.

[0212] Here, for example, the depth of field when focus adjustment is performed with a predetermined position near the base [BT] as the focus point during the [B1] period in Fig. 10 is indicated by the symbol DOF2. Also, the depth of field when focus adjustment is performed with a predetermined position near the base [BT] as the focus point during the [A1] period in Fig. 10 is indicated by the symbol DOF1.

[0213] In this case, when the object changes from the state [B1] in Fig. 10 to the state [A1], for example, the in-focus state changes to the out-of-focus state or the out-of-focus state changes to the in-focus state in the areas indicated by [X1] and [X2] in Fig. 10. This causes a loss of clarity in some areas of the image, which is one of the factors that hinders observation.

[0214] Therefore, in the first focus control, a focus position is set to obtain a depth of field DOF3 that covers both the depth of field DOF2 in the situation [B] and the depth of field DOF1 in the situation [A]. This depth of field DOF3 has a depth that allows a focused state within the desired region (the region of the object to be observed) to be always ensured even if the object pulsates or the like and the distance distribution changes over time.

[0215] FIG. 11 is a schematic diagram showing the operation during second focus control along with the change in distance distribution information over time. The symbols used in FIG. 11 are the same as those in FIG. 10 . As shown in FIG. 11 , during second focus control, as described with reference to FIG. 7 , for example, AF imaging of the base of the head in the situation [A2] in FIG. 11 is performed first (see symbol [S1]), then AF imaging of the top of the head in the situation [A2] in FIG. 11 is performed (see symbol [S2]), then AF imaging of the base of the head in the situation [B3] in FIG. 11 is performed (see symbol [S3]), and finally AF imaging of the top of the head in the situation [B3] in FIG. 11 is performed (see symbol [S4]). In this way, multiple pieces of image data (four in this case) are acquired. Then, focus stacking image processing is performed on this multiple pieces of image data (four or more). An image is displayed based on the image data generated as a result.

[0216] FIG. 12 is a schematic diagram showing the effect of the third focus control along with the change in distance distribution information over time. The symbols used in FIG. 12 are also the same as those in FIG. 10. As shown in FIG. 12, during the third focus control, the focus point tracks the movement of the object. Therefore, the area in focus changes as the object changes. However, because the imaging frame rate is sufficiently fast relative to the period of the object's pulsation, etc., it is considered that the time during which an out-of-focus area appears is very short. In such cases, it can be expected that good endoscopic images can be obtained even with the third focus control.

[0217] As explained above, the above modification can also provide the same effects as the above embodiment.

[0218] In the above-described modified example, if the determination result in the process of step S45 in FIG. 8 is that pulsation or the like is occurring in the object, the process proceeds to the process sequence in FIG.

[0219] However, instead of such processing, for example, if the determination result of the processing in step S45 in Fig. 8 indicates that pulsation or the like is occurring, a processing sequence may be adopted in which the processing proceeds to the processing in step S4 in Fig. 5. In this case, the processing sequence starting from step S21 in Fig. 8 and ending at step S45 may be considered to correspond to the processing in steps S2 and S3 in Fig. 5.

[0220] This processing sequence can also provide the same effects as those of the embodiment and the modified example.

[0221] The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the spirit and scope of the invention. For example, while an endoscope has been described as an example, the present invention can be applied to any device that sequentially acquires and evaluates images (such as an image inspection device). Furthermore, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining the disclosed multiple components. For example, if the problem to be solved by the invention can be solved and the effects of the invention can be obtained even if some components are deleted from all the components shown in one embodiment, the configuration from which these components are deleted can be extracted as an invention. Furthermore, components from different embodiments may be appropriately combined. The present invention is not limited by specific embodiments other than as limited by the appended claims.

Claims

1. A focus control device that is provided at the tip of an endoscope insertion portion and adjusts the focus position based on an output signal from an imaging unit having an optical system that moves back and forth along an optical axis to adjust the focus, comprising: a distance distribution information acquisition unit that acquires, from the output signal of the imaging unit, distance distribution information in the forward and backward directions of an object as seen from the imaging unit and also acquires information on changes in distance over time of the object; a range determination unit that determines the range of change in distance of the object based on the distance distribution information and the distance change information; and a focus position control unit that adjusts the focus position based on the determination result of the range determination unit.

2. A focus control device according to claim 1, wherein the distance distribution information acquisition section acquires the distance distribution information based on phase difference information or image information from the output signal from the imaging unit.

3. The focus control device described in claim 1, characterized in that the focus position control unit sets a focus position that can ensure a depth of field that includes the farthest point and the closest point within the distance change range of the object based on the judgment result of the range judgment unit.

4. A focus control device as described in claim 1, further comprising a period determination unit that determines the period of the distance change of the object based on the distance distribution information and the distance change information, and the range determination unit determines the range of the distance change of the object based on the determination result of the period determination unit.

5. The focus control device according to claim 4, wherein the distance distribution information acquisition unit repeatedly acquires the distance distribution information at a cycle shorter than the cycle of change in the distance of the object.

6. A focus control device as described in claim 4, further comprising a change determination unit that determines a change in distance to the object in the forward and backward directions, and wherein the focus position control unit predicts a focus position according to a distance change period of the object based on the determination results by the change determination unit and the range determination unit, and performs focus position tracking control.

7. A focus control device as described in claim 4, further comprising a specific object image detection unit that detects an image area including a specific object from an image based on image data acquired by the imaging unit, and wherein the focus position control unit performs autofocus control on the specific object when the specific object is detected by the specific object image detection unit.

8. A focus control device according to claim 4, further comprising: a temporary storage unit that temporarily stores a plurality of output signals output sequentially in time series from the imaging unit; and a depth stacking processing unit that performs image synthesis processing based on a plurality of image data with different focus states for the same object, wherein the focus position control unit performs focus control multiple times to obtain a plurality of image data with different focus states for the same object based on the judgment results of the range judgment unit and the period judgment unit, the imaging unit performs imaging for each of the multiple focus controls by the focus position control unit and outputs a plurality of image data, the temporary storage unit temporarily stores the plurality of image data, and the depth stacking processing unit performs image synthesis processing based on the plurality of image data.

9. The focus control device described in claim 7, characterized in that the specific object image detection unit has an inference model constructed using a large amount of specific object image data as training data and annotated by displaying a frame surrounding the image area containing the specific object.

10. A focus control device as described in claim 4, further comprising a guide display unit, which displays a guide to call attention when it is determined based on the judgment results of the range judgment unit or the periodic judgment unit that there is a possibility that a periodic change in distance to the object is occurring.

11. A focus control method comprising the steps of: obtaining, from an output signal of an imaging unit, distance distribution information in the front-to-back direction of an object as seen from the imaging unit; obtaining information on a change in distance over time of the object; determining a distance change range of the object based on the distance distribution information and the distance change information; and adjusting a focus position based on the determination result of the range determination unit.

12. A focus control program that causes a computer to execute the following: a distance distribution information acquisition process that acquires distance distribution information in the forward and backward directions of an object as seen from the imaging unit from an output signal of the imaging unit, and acquires information on changes in distance over time of the object; a range determination process that determines the range of change in distance of the object based on the distance distribution information and the distance change information; and a focus position control process that adjusts the focus position based on the determination result of the range determination unit.

13. An endoscopic system comprising: an endoscope including an imaging unit including an imaging optical system that forms an optical image of an object, an imaging element that photoelectrically converts the optical image formed by the imaging optical system to obtain an image signal, an endoscope including a focus adjustment mechanism that moves some of the optical lenses included in the imaging optical system forward and backward in a direction along the optical axis, and a processor, wherein the processor comprises: a distance distribution information acquisition section that acquires distance distribution information in the forward and backward directions of the object as seen from the imaging unit from an output signal of the imaging unit, and also acquires information on changes in distance over time of the object; a range determination section that determines the range of change in distance of the object based on the distance distribution information and the distance change information; and a focus position control section that adjusts the focus position based on the determination result of the range determination section.

14. A focus control device for an optical system for a distal imaging unit of an endoscope insertion part, comprising: a specific object detection unit that, when acquiring distance distribution information in the forward and backward directions of an object in front from the output signal of the imaging unit, determines the image pattern of the object for each position corresponding to each distance indicating a different distance in the distance distribution; and a focus position control unit that adjusts the focus position based on the distance distribution information of the object position classified by the image pattern.

15. A focus control device according to claim 14, further comprising a range determination unit that acquires information on changes in distance over time of the object and determines the range of changes in distance of the object, and the focus position control unit adjusts the focus position based on the determination result of the range determination unit.

16. A focus control device as described in claim 14, characterized in that, when there are multiple positions corresponding to each of the distances, the specific object detection unit determines the image pattern for each position, and when there is any one of the multiple locations that is important for diagnosis, the focus position control unit sets that location as the focus target distance.

17. A focus control device for an optical system for an imaging unit at the tip of an endoscope insertion section, comprising: a specific object detection unit that, when acquiring distance distribution information in the forward and backward directions of an object in front from the output signal of the imaging unit, determines the image pattern of the object for each position corresponding to each distance indicating a different distance in the distance distribution; a range determination unit that acquires information on the change in distance of the object over time and determines the range of change in distance of the object; and a focus position control unit that adjusts the focus position based on the determination result of the range determination unit.

18. A focus control method characterized by the steps of: when obtaining distance distribution information in the forward and backward directions of an object in front from the output signal of an imaging unit, determining an image pattern of the object for each position corresponding to each distance indicating a different distance in the distance distribution; obtaining information on the change in distance of the object over time to determine the range of change in distance of the object; and adjusting the focus position based on the result of determining the range of change in distance of the object.

19. A focus control program that causes a computer to execute the following: an image pattern determination process that, when acquiring distance distribution information in the forward and backward directions of an object in front from the output signal of an imaging unit, determines the image pattern of the object at each position corresponding to each distance that indicates a different distance in the distance distribution; a range determination process that acquires information on the change in distance of the object over time and determines the range of change in distance of the object; and a focus position control process that adjusts the focus position based on the result of the range determination.

20. A focus control method characterized by the steps of: when obtaining distance distribution information in the forward and backward directions of an object in front from the output signal of an imaging unit, determining an image pattern of the object for each position corresponding to each distance indicating a different distance in the distance distribution; obtaining information on the change in distance over time of the object to determine the range of change in distance of the object; and synthesizing the imaging results at each focus position obtained by adjusting the focus position over time based on the determination result of the range of change in distance of the object.

Images