Focus control device, focus control method, focus control program, endoscope system

Abstract

The present application is a focus control device that performs focus position adjustment based on an output signal from an imaging unit (26) provided at a distal end (21a) of an insertion section (21) of an endoscope (20) and having an optical system (26b) that moves in and out in a direction along an optical axis (O) to perform focus adjustment, the focus control device including: a distance distribution information acquisition section (15) that acquires distance distribution information in a front-rear direction of an object being observed by the imaging unit from the output signal of the imaging unit and acquires time-based distance change information of the object; a range determination section (15b) that determines a distance change range of the object based on the distance distribution information and the distance change information; and a focus position control section (16) that performs focus position adjustment based on a determination result of the range determination section.

Description

Technical Field

[0001] This invention relates to a focusing control device, focusing control method, focusing control program, and endoscope system capable of acquiring clear endoscopic images over a wider area, regardless of the shape, state, etc., of the object being observed, while ensuring a state of focus. Background Technology

[0002] In the past, endoscope systems, consisting of an endoscope that captures images of the inside of a subject, a processor that performs various image processing on the image data acquired by the endoscope, a display device that displays the image data processed by the processor as a visually recognizable image, and a storage device that records or stores the image data, have been widely used in medical and industrial fields.

[0003] Furthermore, in the medical field, this type of endoscope system is widely used for various examinations that observe the interior of biological organs. In examinations using a medical endoscope system (hereinafter referred to as endoscopic examinations), for example, while performing operations such as inserting and withdrawing the insertion portion of an endoscope into the lumen of a biological organ along the length of the lumen, an imaging unit located at the tip of the insertion portion captures images. At this time, the endoscopic images acquired by the imaging unit are displayed as dynamic images in real-time, sequentially over time, using a display device. Simultaneously, this dynamic image data can also be stored in a storage device.

[0004] In such an endoscopy, the doctor or other equipment operator observes the real-time endoscopic images displayed on the display device (or, after the examination, a reconstructed endoscopic image based on recorded or stored endoscopic image data) while searching for lesions such as polyps or tumors that have formed on the inner wall of organs as specific objects (hereinafter referred to as specific objects).

[0005] Typically, during endoscopic examinations using endoscopes, the internal structure of organs containing tumors or other specific objects observed is not a uniform planar shape, but rather a complex shape composed of protrusions and depressions. Furthermore, it is well known that tumors and other specific objects often form irregular, uneven shapes on the inner walls of organs.

[0006] Therefore, for example, when the object being observed (such as a tumor) is formed with a concave-convex shape having a distance difference relative to the imaging unit of the endoscope in the front-back direction along the optical axis of the imaging optical system, when focusing is performed with a predetermined point on the surface of the object as the target, a focused state can be ensured within a predetermined range (the so-called depth of field) in the front-back direction of that point. However, sometimes a focused state cannot be ensured in areas that deviate beyond this predetermined range.

[0007] It is generally known that the depth of field of a camera optical system tends to be shallower the shorter the distance from the camera unit to the object being photographed (hereinafter referred to as the shooting distance), and deeper the longer the shooting distance. Therefore, when shooting with the camera optical system close to the object, the depth of field becomes shallower. Consequently, the closer to the object, the narrower the area in focus in the direction in front of and behind the object. Furthermore, the area in front of and behind the object outside the area in focus becomes out of focus (the so-called defocused state), which obstructs observation.

[0008] On the other hand, endoscopy involves observing the interior of organs and other internal structures of living organisms. Therefore, due to factors such as the patient's pulsation, respiration, changes in body position, or peristaltic movements of the organs, the interior of the organ containing the object is constantly moving. Consequently, in endoscopic examinations where the endoscope is positioned inside the organ facing the object, the image may be in focus or out of focus, depending on the timing of the image capture. Therefore, it is difficult to consistently obtain a clear, focused endoscopic image during an endoscopy.

[0009] Therefore, for example, Japanese Patent Publication No. Hei 9-318865 discloses a focus adjustment device that, when photographing a subject that frequently appears in multiple focus detection areas arranged in the camera frame, searches for the moving destination of the subject image in the order of past frequency of appearance, thereby efficiently shortening the search time.

[0010] However, the conventional focus adjustment device disclosed in Japanese Patent Publication No. Hei 9-318865 and the like is a device that determines the movement pattern of the subject image and attempts to search for the moving destination of the subject image.

[0011] However, during endoscopic examinations, the movement within the organs of a living organism, which contains the specific object being observed, is irregular and may not necessarily follow a fixed cycle. Therefore, it is inappropriate to apply conventional methods disclosed in the aforementioned publications to equipment used for endoscopic examinations.

[0012] The purpose of this invention is to provide a focusing control device, focusing control method, focusing control program, and endoscope system that, when performing detailed observation of specific objects such as tumors in medical devices or examination devices (image examination devices, etc.) such as endoscopes, can ensure focus in a wider area and always acquire clear and high-precision endoscopic images regardless of the shape, state, etc. of the observed object. Summary of the Invention

[0013] Solution for solving the problem

[0014] To achieve the above objective, one aspect of the present invention provides a focusing control device that adjusts the focusing position based on an output signal from a camera unit disposed at the front end of an endoscope insertion section and having an optical system that adjusts the focus by moving back and forth along the optical axis. The focusing control device comprises: a distance distribution information acquisition unit that acquires distance distribution information of an object in the front-back direction when observed from the camera unit, and acquires time-varying distance information of the object; a range determination unit that determines the range of distance variation of the object based on the distance distribution information and the distance variation information; and a focusing position control unit that adjusts the focusing position based on the determination result of the range determination unit.

[0015] One aspect of the focusing control method of the present invention is as follows: obtaining distance distribution information of an object in the front-back direction when observed from the camera unit from the output signal of the camera unit, and obtaining distance change information of the object over time; determining the distance change range of the object based on the distance distribution information and the distance change information; and adjusting the focusing position based on the determination result of the range determination unit.

[0016] One aspect of the present invention provides a focus control program that causes a computer to perform the following processes: a distance distribution information acquisition process, which acquires distance distribution information of an object in the front-back direction when observed from the camera unit from the output signal of the camera unit, and acquires distance change information of the object over time; a range determination process, which determines the range of distance change of the object based on the distance distribution information and the distance change information; and a focus position control process, which adjusts the focus position based on the determination result of the range determination unit.

[0017] An endoscope system according to one aspect of the present invention includes an endoscope and a processor. The endoscope includes: a camera unit comprising a camera optical system for imaging an optical image of an object and an imaging element for photoelectric conversion of the optical image image formed by the camera optical system to acquire an image signal; and a focus adjustment mechanism for moving a portion of an optical lens included in the camera optical system back and forth along the optical axis. The processor includes: a distance distribution information acquisition unit for acquiring distance distribution information of the object in the front-back direction when observed from the camera unit, and acquiring time-varying distance information of the object; a range determination unit for determining the range of distance variation of the object based on the distance distribution information and the distance variation information; and a focus position control unit for adjusting the focus position based on the determination result of the range determination unit.

[0018] The second aspect of the focusing control device of the present invention is a focusing control device for an optical system of a front-end camera unit of an endoscope insertion section. The focusing control device includes: a specific object detection unit that, when acquiring distance distribution information of an object in front in the front-back direction from the output signal of the camera unit, determines an image pattern of the object for each position corresponding to each distance representing different distances in the distance distribution; and a focusing position control unit that adjusts the focusing position based on the distance distribution information of the position of the object after classifying the image pattern.

[0019] The third-party focusing control device of the present invention is a focusing control device for an optical system of a front-end camera unit of an endoscope insertion section. The focusing control device includes: a specific object detection unit, which, when acquiring distance distribution information of an object in front in the front-back direction from the output signal of the camera unit, determines the image pattern of the object for each position corresponding to each distance representing different distances in the distance distribution; a range determination unit, which acquires distance change information of the object over time to determine the range of distance change of the object; and a focusing position control unit, which adjusts the focusing position based on the determination result of the range determination unit.

[0020] The second aspect of the focusing control method of the present invention is as follows: when acquiring the distance distribution information of the object in front in the front-back direction from the output signal of the camera unit, the image pattern of the object is determined for each position corresponding to each distance representing different distances in the distance distribution, the distance change information of the object in time is acquired to determine the distance change range of the object, and the focusing position is adjusted based on the determination result of the distance change range of the object.

[0021] The second aspect of the focus control program of the present invention causes the computer to perform the following processes: image pattern determination processing, which, when acquiring distance distribution information of an object in the front direction from the output signal of the camera unit, determines the image pattern of the object for each position corresponding to each distance representing different distances in the distance distribution; range determination processing, which acquires distance change information of the object over time to determine the range of distance change of the object; and focus position control processing, which adjusts the focus position based on the result of the range determination.

[0022] The third-party focusing control method of the present invention is as follows: when obtaining the distance distribution information of the object in front in the front-back direction from the output signal of the camera unit, the image pattern of the object is determined for each position corresponding to each distance representing different distances in the distance distribution, the distance change information of the object over time is obtained to determine the distance change range of the object, and the camera results at each focusing position after multiple focusing position adjustments based on the determination result of the distance change range of the object over time are synthesized.

[0023] According to the present invention, a focusing control device, focusing control method, focusing control program, and endoscope system are provided that, when performing detailed observation of a specific object such as a tumor in a medical device or examination device (image examination device, etc.) such as an endoscope, the focusing state can be ensured over a wider area and clear and high-precision endoscopic images can always be acquired regardless of the shape, state, etc. of the observed object. Attached Figure Description

[0024] Figure 1 This is a structural diagram that schematically illustrates the overall structure of an endoscope system incorporating a focusing control device according to one embodiment of the present invention.

[0025] Figure 2 This is a structural block diagram illustrating the internal structure of an endoscope system incorporating a focusing control device according to an embodiment of the present invention.

[0026] Figure 3 This is an illustration of the inference model obtained as a learning result in the learning device.

[0027] Figure 4 It is a conceptual illustration of the application. Figure 1 The diagram illustrates the structure of the imaging element of the endoscope included in the endoscope system, and explains the principle of the imaging surface phase difference AF based on the imaging element.

[0028] Figure 5 This is a flowchart illustrating the function of an endoscope system incorporating a focusing control device according to one embodiment of the present invention.

[0029] Figure 6 This is a schematic diagram illustrating the situation when using an endoscope to observe a specific object.

[0030] Figure 7 This is a schematic diagram illustrating the phenomenon of pulsation or other abnormalities that occur when observing a specific object using an endoscope.

[0031] Figure 8 This is a flowchart illustrating the determination process for pulsation, etc., in a modified example of the operation of a focusing control device according to an embodiment of the present invention.

[0032] Figure 9 It is shown in Figure 8 The flowchart shows the process of switching focus control after determining and processing the pulsation, etc.

[0033] Figure 10 This is a schematic diagram illustrating the effect of the first focusing control based on the time variation of distance distribution information.

[0034] Figure 11 This is a schematic diagram illustrating the effect of the second focusing control based on the temporal changes in distance distribution information.

[0035] Figure 12 This is a schematic diagram illustrating the effect of the third focusing control based on the temporal changes in distance distribution information. Detailed Implementation

[0036] The present invention will now be described, for example, by way of an illustrated embodiment, which is an application example to an endoscope system. The accompanying drawings used in the following description are schematic. Therefore, in these drawings, each component is shown to a degree that it can be identified on the drawing. Thus, sometimes the dimensional relationships of the components, scale, etc., in the drawings are shown differently according to the component. In the present invention, the quantity, shape, size ratio, and relative positional relationships of each component shown in the drawings are not limited to the illustrated manner.

[0037] First, before describing the detailed structure of a focusing control device according to one embodiment of the present invention, the following uses... Figure 1 , Figure 2 The general structure of an endoscope system including the focusing control device of this embodiment will be described. Figure 1 This is a structural diagram that schematically illustrates the overall structure of an endoscope system incorporating a focusing control device according to one embodiment of the present invention. Figure 2 This is a structural block diagram showing a summary of the internal structure of the endoscope system.

[0038] like Figure 1As shown, the endoscope system 1 is mainly composed of a processor 10, an endoscope 20, a display device 30, a light source device 40, a storage device 50, and external devices 60. Figure 1 The endoscopic system shown illustrates the typical structure of an endoscopic system used in endoscopic examinations of subjects (patients, etc.) where the upper digestive tract (esophagus, stomach, duodenum, etc.) or lower digestive tract (large intestine, etc.) is observed.

[0039] Furthermore, during endoscopic examinations using such endoscopic systems, doctors and other equipment operators observe the endoscopic images displayed on the display device while performing a process known as screening, which involves searching for lesions such as polyps or tumors that have formed on the inner walls of organs.

[0040] However, under normal circumstances, because specific objects such as lesions are very small and their color is similar to that of the surrounding organ walls, reliably identifying and detecting the image area of ​​a specific object from endoscopic images requires skilled technique. Furthermore, the endoscopic procedure itself, such as aligning the tip of the endoscope insertion section (the viewing window of the camera unit) with the desired object to take a picture, requires skilled technique.

[0041] In addition, the use of endoscopes in endoscopic examinations is generally divided into several states, such as the screening state (called the screening state), the detailed observation state (called the detailed observation state), and the transition state (called the approach state) in which the tip of the endoscope is brought closer to the specific object to be observed for detailed observation.

[0042] In the past, when performing endoscopic examinations using medical devices or examination equipment such as endoscopes, in order to always ensure a reliable observation environment and continuously display good images, it was required to perform appropriate focusing control of the camera unit according to the usage status of the endoscope.

[0043] For example, when endoscopes are in a screening state, especially when precise focusing control is not required, the camera unit's camera optical system position setting is mostly set by relying on the characteristics of the camera optical system to achieve a good focus state within a specified camera range. This is called deep focus control.

[0044] In addition, for example, when an endoscope is in a detailed observation state or close to the object, it is required to perform appropriate focus control to set the object as the camera or observation object to a focus state, so as to observe in detail through a clear image.

[0045] Therefore, for example, in detailed observation, in order to display the object being observed as large as possible, there are more cases where the front surface of the camera unit (the front surface of the camera optical system) is brought close to the surface of the object being observed (such as a tumor) to take pictures.

[0046] like Figure 1 As shown, the endoscope 20 is an image inspection device configured to have an insertion part 21, an operation part 22, and a universal cable 23.

[0047] The insertion section 21 is a component that is inserted into a subject such as a biological organism. The insertion section 21 is formed by continuously connecting a front end portion 21a, a bend portion 21b, and a flexible tube portion 21c from the front end side. The insertion section 21 is generally formed into a long and thin tube shape. Inside the insertion section 21, there is a passageway 21d for inserting an endoscopic instrument (not shown), i.e., an instrument insertion channel 21d. This instrument insertion channel 21d is provided such that it extends from the front end to the base end of the insertion section 21. Furthermore, an operating section 22 is connected to the base end side of the insertion section 21.

[0048] The front end portion 21a is a structural unit located at the very front end of the insertion portion 21. Various components, such as a lighting unit 25 and a camera unit 26, are disposed inside the front end portion 21a. Figure 1 Not illustrated in the image; see reference. Figure 2 ).

[0049] Here, the illumination unit 25 is a structural unit that includes optical elements (illumination lens; not shown), etc. The optical elements emit a light beam guided from the light source device 40 described later from the front end of the front end 21a forward to illuminate the observation area of ​​the subject body, including lesions, etc.

[0050] Additionally, the imaging unit 26 is an optical lens (imaging optical system 26a; see reference 26a) that images a specific object (such as a lesion or tumor) inside the subject as the object of observation (imaging). Figure 2 ) and a photoelectric conversion element (image sensor 26b; reference) that generates image information (static image data and dynamic image data, etc.) based on the optical image. Figure 2 The camera unit 26 is an electronic device unit consisting of components such as a focusing lens, a zoom lens, etc. Furthermore, although not shown in the figure, the camera unit 26 is configured to also include a drive mechanism (focus adjustment mechanism, zoom mechanism, etc.) for moving a portion of the optical lenses (e.g., focusing lens, zoom lens, etc.) included in the camera optical system 26a back and forth along the optical axis.

[0051] The bending portion 21b is a tubular portion configured to be actively and freely bent by a bending operation mechanism (not shown) that operates in conjunction with the bending operation member 22b described later. The flexible tubular portion 21c is a flexible tubular member that extends from the front end of the operation portion 22 and is connected to the base end of the bending portion 21b.

[0052] The operating part 22 is connected to the base end of the insertion part 21. The operating part 22 is configured to have an operating part body 22a, a bending operating member 22b, multiple operating members 22c, and a treatment device insertion port 22d, etc.

[0053] The main body 22a of the operating section is generally box-shaped, forming the gripping part for the endoscope 20 to be held by the user (doctor, etc.). Various structural units, such as a bending operating mechanism (not shown), are provided inside the main body 22a. As described above, an insertion part 21 extends from the main body 22a.

[0054] The bending operation member 22b and the plurality of operation members 22c are operation members used to perform various operations on the endoscope 20. These bending operation members 22b and the plurality of operation members 22c are respectively disposed at predetermined positions on the outer surface of the operation part body 22a.

[0055] The treatment device insertion port 22d is located at a predetermined position near the front end of the main body 22a of the operating section. The treatment device insertion port 22d is the base end opening of the treatment device insertion channel 21d of the insertion section 21. The treatment device insertion channel 21d is connected to the front end channel opening (not shown) of the front end 21a at the front end side.

[0056] With this structure, the endoscope treatment instrument (not shown) inserted from the treatment instrument insertion port 22d is configured to protrude outward from the front end channel opening of the front end 21a after the treatment instrument insertion channel 21d is opened.

[0057] The universal cable 23 is a connecting cable used to connect the endoscope 20 to the light source device 40 and the processor 10. Therefore, the universal cable 23 is composed of a tubular member extending from the side of the operation unit body 22a of the operation unit 22. A scope connector 23a is provided at the front end of the universal cable 23. This scope connector 23a is connected to the front panel of the light source device 40.

[0058] An electrical cable 23b extends from the mirror connector 23a. A connector 23c is provided at the front end of the electrical cable 23b. The connector 23c connects to the front panel of the processor 10. Various signal transmission cables, fiber optic cables, etc. (not shown) are inserted into the general-purpose cable 23.

[0059] The light source device 40 is an illumination unit 25 disposed inside the front end portion 21a of the insertion portion 21 of the endoscope 20 (see reference). Figure 2 The device supplies illumination light. Illumination light emitted from the light source device 40 is transmitted to the illumination unit 25 at the front end 21a via an optical fiber cable (not shown) that is connected from the mirror connector 23a through a universal cable 23, an operation section 22, and an insertion section 21. Furthermore, this illumination light is directed towards the observation area in front of the front end 21a through an illumination lens (not shown) included in the illumination unit 25 at the front end 21a.

[0060] The processor 10 is a control device and signal processing device, or circuit unit, that includes a control circuit and signal processing circuit for controlling the endoscope system 1 as a whole. Furthermore, the processor 10 is configured to include the functions of the focusing control device in this embodiment (details are described later; see reference). Figure 2 ).

[0061] The control circuitry included in the processor 10 receives, for example, operation instruction signals from the operation member 22c of the operation unit 22 of the endoscope 20, and outputs various control signals for driving and controlling the camera unit 26, the light source device 40, or the illumination unit 25. Additionally, the signal processing circuitry included in the processor 10 receives, for example, camera signals from the camera unit 26 and performs prescribed image signal processing.

[0062] For this purpose, the processor 10 and the camera unit 26 are electrically connected via a signal transmission cable (not shown). This signal transmission cable is configured to extend from the connector 23c through the electrical cable 23b, the lens connector 23a, the universal cable 23, the operation section 22, and the insertion section 21 to the camera unit at the front end 21a.

[0063] With this structure, control signals output from processor 10 and camera signals output from camera unit 26 are transmitted between camera unit 26 and processor 10 via this signal transmission cable. Furthermore, as a type of signal transmission cable, a composite cable can be used, in which multiple cables are bundled together and covered with an outer shield or tubing.

[0064] The display device 30 is a display device that receives image signals and the like output from the processor 10 and performs endoscopic image display, various information display, etc., in a prescribed manner. Therefore, the display device 30 and the processor 10 are electrically connected using a video cable 24. Furthermore, regarding the configuration of the display device 30, for example, a display device constructed using a general liquid crystal panel is employed.

[0065] Storage device 50 is a storage device that stores (records or stores) image data generated in processor 10 based on image information generated by camera unit 26 through various processing.

[0066] like Figure 1 As shown, the storage device 50 and the processor 10 are integrally formed. Figure 2 As shown, inside the storage device 50, a temporary storage unit 51 is provided separately from the main storage unit (not shown) in the storage device 50. This temporary storage unit 51 is, for example, composed of a semiconductor memory, and functions as a temporary memory area for temporarily storing output signals (image data, etc.) from the camera unit 26 or the image processing unit 11, etc.

[0067] In addition, Figure 1 The illustrated structural example shows a configuration in which the storage device 50 is integrally disposed within the housing of the processor 10, but this configuration is not limited to this. For example, the storage device 50 may also be configured as an external storage device using a housing separate from the processor 10.

[0068] In addition, Figure 2 The example shown illustrates a structure in which the temporary storage unit 51 is disposed inside the storage device 50, but this configuration is not limited to this. For example, the temporary storage unit 51 can also be integrally disposed within the storage control unit 13 described later (see [reference]). Figure 2 (the interior of)

[0069] Furthermore, the processor 10 and the light source device 40 are not limited to, for example, Figure 1 The illustrated structure is designed as a separate structure. For example, the processor 10 and the light source device 40 can also be integrally constructed using a single housing.

[0070] As an example of the structure of the lighting unit 25, it is not limited to the above-described structure (the method of transmitting illumination light from the light source device 40 to the front end 21a via an optical fiber cable or the like). As a structure other than this, the lighting unit 25 can also be configured such that a light-emitting element such as an LED (Light Emitting Diode) as a light source is provided inside the front end 21a, and the power supply to the light source (LED) and its light emission control are performed by a predetermined control circuit included in the processor 10.

[0071] The external device 60 is, for example, a biological information acquisition device that detects and acquires vital signs information in a general manner. The external device 60 is connected to the processor 10. With this structure, various vital sign information acquired by the external device 60 is configured to be output to the pulsation cycle determination unit 15a (described later) within the processor 10. With such a structure, the vital sign information can be used as reference data in the pulsation cycle determination unit 15a when determining the cycle of pulsation, etc.

[0072] The endoscope system 1 constructed as described above has a structure that is substantially the same as that of conventional endoscope systems of the same type. Therefore, illustrations and detailed descriptions of other structural features are omitted.

[0073] In the endoscope system 1 constructed with such a structure, the focusing control device of this embodiment is configured to be included in the processor 10. Therefore, in the endoscope system incorporating the focusing control device of one embodiment of the present invention, the following will use... Figure 2 The internal structure of the processor is described in detail.

[0074] The processor 10 is configured to include an image processing unit 11 (including a depth synthesis processing unit 11a), a display control unit 12 (including a guide display unit 12a), a storage control unit 13, a specific object image detection unit 14, a distance distribution information acquisition and determination unit 15 (including a pulsation cycle determination unit 15a and a pulsation distance range determination unit 15b), a focus control unit 16, an illumination control unit 17, and a storage device 50 (including a temporary storage unit 51).

[0075] The image processing unit 11 is a structural unit or circuit unit that acquires the output signal (mainly image information) from the camera unit 26 and performs various information processing based on the acquired image information. The image processing performed here includes, for example, general image information processing, display image data processing, storage image data processing, and various other image processing.

[0076] Typical image processing includes analog-to-digital (AD) conversion, which receives analog image signals output from the imaging element 26b and converts them into digital image signals. This is the basic image information processing typically performed on image information. Since conventional image processing methods are generally used, detailed descriptions are omitted.

[0077] Image data processing for display is the process of generating image data for general display. For example, image data processing for display may be the process of generating image data representing an endoscope image displayed within a predetermined display area on the display screen of the display device 30. Furthermore, image data processing for storage may be the process of generating image data for recording.

[0078] Various image processing techniques are common image processing techniques that appropriately add to image data for display. Specific examples of various image processing techniques include common image adjustment processing such as brightness adjustment processing and white balance adjustment processing, as well as image enhancement processing such as contour enhancement processing and texture and color enhancement imaging (TXI).

[0079] Furthermore, in this embodiment, the image processing unit 11 is configured to include a depth synthesis processing unit 11a. This depth synthesis processing unit 11a is an image processing circuit that performs image synthesis on the same photographic object based on multiple image data with different focus states. Compared with individually acquired images, the image generated by such image synthesis processing can be displayed as an image with a wide area of ​​focus (an image with a deep depth of field).

[0080] Here, as described above, the depth-synthesized image processing is performed based on multiple image data acquired by the imaging element 26b of the imaging unit 26. Therefore, during endoscopic examination, when the endoscope 20 is set to an operating mode for performing depth-synthesized processing, the processor 10 performs the following control.

[0081] That is, the processor 10 uses the focus control unit 16 (described in detail later) to drive and control the imaging unit 26 (the imaging optical system 26a and the imaging element 26b). In this case, the drive control of the imaging unit 26 is to acquire multiple image data while changing the focus position in stages for the same imaging object.

[0082] The depth synthesis processing unit 11a performs image synthesis processing based on the multiple image data acquired in this way to generate an image with a wide area in focus (an image with a deep depth of field).

[0083] Furthermore, the above examples correspond to situations where a doctor or other equipment operator activates a pre-defined camera switch at a desired time to acquire a desired static image, or where the processor 10 automatically initiates the process when pre-defined conditions are met during an endoscopic examination. In addition, depth synthesis processing can also be applied to typical endoscopic observation images (moving images) during endoscopic examinations.

[0084] Regarding endoscopic images, image data is typically acquired continuously, for example, at 30 frames per second (fps). When performing depth synthesis processing on such dynamic image data, a means of oscillating the focusing lens is used, for example. Here, oscillation refers, for example, to the movement of the focusing lens forward and backward between two predetermined points.

[0085] Specifically, for example, it is the action of moving the focusing lens back and forth between two points: a lens position that focuses on the top of the object and a lens position that focuses on the base of the object.

[0086] Furthermore, camera movements are performed at these two locations to acquire multiple (in this case, two) image data. These multiple image data are then combined into a single image. Thus, for example, for an object, it is possible to generate an image showing the entire object from top to bottom in focus. By performing this operation for each frame, multiple image data consecutively in time can be acquired and reproduced as a dynamic image.

[0087] The display control unit 12 is a structural or circuit unit that controls the display device 30 to display the display image data output from the image processing unit 11 in an appropriate display mode. In addition to controlling the display of the display image data (setting various parameters such as display position, display area, and display size), the display control unit 12 also controls the display of various other information (text information, icon information, etc.) besides image information. Among these other information, besides examination date and time information, patient information, and medical record information, there are also various alarm information (notification information) appropriately displayed during endoscopic examinations.

[0088] In detail, the display control unit 12 performs display control to continuously display endoscopic images in the display area. In addition, when a specific object is detected in the endoscopic image, the display control unit 12 performs display control to add a prescribed display (e.g., frame display) to the image area containing the specific object.

[0089] Furthermore, the display control unit 12 includes a guidance display unit 12a. This guidance display unit 12a is, for example, a structural unit or circuit unit that appropriately displays various alarm information (information information), guidance information, etc., to the equipment operator, such as a doctor, during an endoscopy on the display screen of the display device 30 at a predetermined time.

[0090] The storage control unit 13 is a structural or circuit unit that controls the storage device 50 (including the temporary storage unit 51) to save (record or store) the image data for storage output from the imaging unit 26 or the image processing unit 11 in an appropriate storage manner. The storage control unit 13 performs storage control to permanently store the image data representing the endoscope image acquired by the endoscope 20 in the storage area of ​​the storage device 50.

[0091] The specific object image detection unit 14 is a structural unit or circuit unit that detects an image region containing a specific object from an endoscopic image based on image data acquired by the endoscope 20.

[0092] In detail, the specific object image detection unit 14 detects specific objects (lesions such as tumors) in each frame of endoscopic images that are continuously displayed based on image data output from the image processing unit 11.

[0093] Regarding the detection of specific objects by the specific object image detection unit 14, for example, it performs similar image recognition (pattern matching processing, etc.) based on multiple pre-prepared specific object images (case images, etc.), or it performs data analysis using deep learning or machine learning to detect the desired specific object.

[0094] In addition, the specific object image detection unit 14 may also have the following functions, in addition to being able to determine whether a specific object (such as a tumor or lesion) exists in the endoscopic image (and, if it exists, which area of ​​the image the specific object occupies), it may also be able to use techniques such as pattern determination, similar image determination, and inference model to determine, based on the image, features in the object area that can be displayed in image form and that doctors wish to confirm visually, such as vascular patterns and tissue variations, which are used for diagnosis.

[0095] For example, in recent years, object detection technology using AI (Artificial Intelligence) has been developed. This AI obtains the desired inference results (such as the detection of specific objects such as tumors) by machine learning on endoscopic images obtained by endoscopy.

[0096] Machine learning learns features, time-series information, spatial information, etc., from known input information, and makes inferences based on the learning results, thereby obtaining inferences about unknown matters. That is, in machine learning, a fully learned model (or inference model) is first obtained, which can infer a decidable output result based on specific input information.

[0097] In this context, when generating a fully learned model, a large amount of information about the known relationship between input and output is used as training data to obtain highly reliable inference results. For example, in deep learning, a large amount of training data is used to design the network in a way that yields the desired output for known inputs. The fully learned model obtained through this process can be used independently of the learned network.

[0098] Therefore, by having such an inference model, the processor 10 can accurately and quickly perform various judgments or detections, such as the detection of specific objects in the specific object image detection unit 14, for example, the judgment of the range of camera distances that can be captured in the focusing state during pan-focus control, the judgment of the endoscope usage status in endoscopic examination, and the detection of the surface vascular pattern of a specific object (including the detection of information such as the position of the vascular pattern in the image).

[0099] Here, the inference model is constructed roughly as follows. For example, a large amount of image data corresponding to the input and output is provided as training data to the network (not shown) used to construct the inference model. Here, the input as training data includes image data of endoscopic images acquired sequentially in time during endoscopic examinations, image data of images of specific objects (case images, etc.) captured in the endoscopic images, and other image data suitable for detection or judgment. In this case, annotations such as those surrounding the image region of the specific object by boxes can also be set.

[0100] By learning from a large amount of training data, the network design (not shown) is determined in a way that yields outputs corresponding to the inputs. For example, Figure 3 This is an explanatory diagram of the inference model obtained as a result of learning in a prescribed learning device.

[0101] For example, such as Figure 3 As shown, when an input set of endoscopic images containing a specific object (such as a tumor) (first training data set 201) is provided, additional information such as the information of the specific object captured in the image and the display of the bounding box of the image region surrounding the specific object are obtained together with the confidence information. Thus, an inference model 200 for detecting specific objects from endoscopic images is constructed.

[0102] When endoscopic image data 202 from an endoscopic examination is input into the inference model 200 constructed in this way, the inference model 200 detects image regions containing specific objects in frames captured in the endoscopic image data 202, and outputs image data with a bounding box appended to the image region of the specific object (inference result image 203). In this way, specific objects can be detected based on endoscopic images. Moreover, by temporarily storing the image data of image 203, or displaying the image data of image 203 on a designated display area of ​​the display device 30 as needed, it can be used by the endoscopic user for reference during endoscopic examinations.

[0103] Furthermore, "Deep Learning" is a multi-layered structured version of the "machine learning" process that uses neural networks. A typical example is the "forward propagation neural network," which makes decisions by sending information forward. In its simplest form, it only requires three layers: an input layer with N1 neurons, an intermediate layer with N2 neurons (with parameters given), and an output layer with N3 neurons corresponding to the number of classes to be judged. Moreover, the neurons in the input layer and the intermediate layer, and the intermediate layer and the output layer, are connected by connection weights, and bias values ​​are applied to the intermediate and output layers, making it easy to form logic gates. For simple discrimination, three layers are sufficient, but by using a large number of intermediate layers, it is possible to learn combinations of multiple features during machine learning. In recent years, considering the relationship between learning time, judgment accuracy, and energy consumption, neural networks with 9 to 152 layers have reached a practical level.

[0104] As the network used in machine learning, various well-known networks can be employed. For example, R-CNN (Regions with CNN features) and FCN (Fully Convolutional Networks) can be used. This involves a process called "convolution" that compresses the feature set of the image, performing actions with minimal processing and demonstrating strong capabilities for pattern recognition. Alternatively, "recurrent neural networks" (fully connected recurrent neural networks) can be used to process more complex information, analyze information whose meaning changes according to sequence or order, and enable bidirectional flow of information.

[0105] To implement these technologies, general-purpose computing circuits such as CPUs and FPGAs can be used. However, since neural network processing mostly involves matrix multiplication, GPUs (Graphics Processing Units) or Tensor Processing Units (TPUs), which are dedicated to matrix computation, are sometimes utilized. In recent years, such artificial intelligence (AI)-specific hardware, namely "Neural Network Processing Units (NPUs)," has sometimes been designed to be integrated and embedded with other circuits such as CPUs, thus becoming part of the processing circuitry.

[0106] Furthermore, beyond deep learning, various well-known machine learning methods can also be used to obtain inference models. For example, there are methods called Support Vector Machines (SVMs) and Support Vector Regression (SVR). Here, learning is used to calculate the weights, filter coefficients, and biases of the recognizer. In addition, there are methods using logistic regression. When enabling a machine to make a certain judgment, a human needs to teach the machine the judgment mechanism. In this embodiment, a method of deriving image judgments through machine learning is used. However, rule-based methods employing rules obtained by humans through empirical rules or heuristics for specific judgments can also be applied.

[0107] The distance distribution information acquisition and determination unit 15 is a structural unit or circuit unit that acquires distance distribution information (hereinafter referred to as distance distribution information or depth data) within the camera frame based on the output signal (image information or phase difference information; described later) from the camera element 26b, and acquires information on the temporal distance change of the object in the front-back direction.

[0108] Here, distance distribution information (depth map) within the camera frame refers to information representing the distribution of distances in the front-to-back direction of objects captured within the camera frame, including a specific object being observed.

[0109] Furthermore, the front-back direction of an object refers to the front-back direction (in other words, the depth direction) along the optical axis of the imaging optical system when observing an object containing a specific object from the imaging unit 26 (imaging element 26b) located at the front end 21a of the endoscope 20. It is the forward and backward direction of the endoscope 20, which refers to the direction along the insertion axis of the endoscope 20.

[0110] In this embodiment, the imaging element 26b included in the imaging unit 26 of the endoscope 20 employs elements that include multiple pixels capable of receiving a light beam from the subject and outputting an image signal. Furthermore, some or all of these pixels have a structure that enables phase difference detection of the imaging plane, thereby facilitating automatic focus control (AF control), predefined focus control, and the like. In this embodiment, by applying the imaging element 26b in this manner, image information and phase difference information can be acquired.

[0111] Here, we will briefly describe the structure that enables phase difference detection of the imaging plane using an imaging element. Figure 4 This is a schematic diagram illustrating the principle of camera surface phase difference AF control, which detects the phase difference of the camera surface through the camera element.

[0112] Figure 12The mark 260 shown indicates a pixel of the imaging element 26b. Pixel 260 has a photodiode 262 and a microlens 261. Here, the photodiode 262 is formed, for example, in a configuration that is divided into a right photodiode 262R and a left photodiode 262L.

[0113] Microlens 261 is an optical lens that allows the light beam passing through the imaging optical system 26a to pass through and forms an optical image on the light-receiving surface of photodiode 262. Photodiode 262 is a photoelectric conversion element that converts the optical image formed by microlens 261 into an electrical signal and outputs it.

[0114] Here, the photodiode 262R on the right receives the light beam LL from the left region (in Figure 4 The area shown by the left diagonal shading is imaged onto the light-receiving surface. Additionally, the left photodiode 262L receives the light beam LR from the right area (in...). Figure 4 The area shown by the right diagonal shading is imaged on the illuminated surface.

[0115] Pixel 260, based on this structure, can calculate the distance (parallax) between two images based on the output signals (phase difference information) from the left and right photodiodes 262R and 262L, thereby determining the amount of defocusing of the imaging optical system relative to the object. Furthermore, by adding the output signals of the left and right photodiodes 262R and 262L, an image signal for a single pixel can be processed.

[0116] The distance distribution information acquisition and determination unit 15 receives the phase difference information in the output signal acquired by the imaging element 26b in this manner, and acquires the distance distribution information (depth map) of a specified area (e.g., an area containing a specific object) within the imaging plane. In this case, the distance distribution information acquisition and determination unit 15 functions as a distance distribution information acquisition unit.

[0117] In addition, Figure 4 The structure shown is configured to divide a photodiode into two parts, and to perform pupil segmentation of the microlens by processing signals from each photodiode separately. However, this structure is not limited to this one. For example, a similar pupil segmentation structure can be achieved by using a light-shielding member.

[0118] Furthermore, the distance distribution information acquisition and determination unit 15 acquires information on the temporal distance change of the object in the front-back direction based on multiple image data (hereinafter referred to as time-series image data) continuously output from the camera unit 26 in a time sequence.

[0119] Furthermore, the distance distribution information acquisition and determination unit 15 determines the concavity / convexity range of a specific object based on the acquired distance distribution information (depth data), and determines the required depth of field based on the concavity / convexity range. In this case, the distance distribution information acquisition and determination unit 15 functions as both an object concavity / convexity range determination unit and a depth of field determination unit.

[0120] Here, the range of the protrusion or depression of a specific object refers to, for example, the range of the distance difference relative to the camera unit 26 in the front-to-back direction (depth direction) when the specific object is formed in a shape that protrudes from the inner wall of an organ or the surface of the specific object is formed in a protrusion or depression manner.

[0121] Furthermore, when the distance distribution information acquisition and determination unit 15 functions as an object protrusion / concavity range determination unit, it determines the distance difference between the object's protrusion / concavity and the camera unit 26 in the front-to-back direction. Additionally, when the distance distribution information acquisition and determination unit 15 functions as a depth-of-field determination unit, it determines the depth of field that includes the determined distance difference between the object's protrusion / concavity within the focus range.

[0122] Furthermore, in the above-described structural example, the distance distribution information acquisition and determination unit 15 is configured to acquire distance distribution information (depth map) based on the phase difference information from the imaging element 26b, which has multiple pixels for image signal acquisition and imaging surface phase difference detection, but is not limited to this structure. For example, in a typical imaging element, all pixels provided on the imaging surface are configured to acquire the imaging signal.

[0123] In an imaging unit having an imaging element 26b in such a conventional manner, for example, at least a portion of the optical lenses constituting the imaging optical system 26a, namely the optical lenses that aid in focus control (hereinafter referred to as focusing lenses), are moved slightly forward and backward along the optical axis to change the lens position. This results in a change in the contrast of objects in the image, and thus a change in the focus state. At this time, distance information can be obtained based on the correlation between the lens position and the focus state corresponding to the lens position. Based on the distance information obtained in this way, distance distribution information (depth map) within the image can be obtained. That is, it is also possible to configure the system to obtain distance distribution information (depth map) based on image signals from the imaging element 26b.

[0124] In this way, the distance distribution information acquisition and determination unit 15 acquires a depth map based on the output signal (phase difference information or image information) of the camera element 26b. That is, the endoscopic image generated based on the output signal (image signal) of the camera element 26b and the depth map generated based on the output signal (phase difference information or image information) of the camera element 26b are correlated and correspond to each other.

[0125] In addition, the distance distribution information acquisition and determination unit 15 is configured to also have a pulsation cycle determination unit 15a and a pulsation distance range determination unit 15b.

[0126] The pulsation cycle determination unit 15a is a structural or circuit unit that detects and determines the periodicity of the temporal distance change of the object caused by pulsation, respiration, changes in body position, or peristaltic movements of the organs within a biological organism (hereinafter collectively referred to as pulsation, etc.). The pulsation cycle determination unit 15a detects and determines the periodicity of the temporal distance change of the object based on the distance distribution information and periodic distance change information acquired by the distance distribution information acquisition and determination unit 15.

[0127] The pulsation distance range determination unit 15b is a structural unit or circuit unit that detects and determines the range of temporal distance changes of an object caused by pulsation or the like in the front-back direction based on the distance distribution information and distance change information acquired by the distance distribution information acquisition and determination unit 15.

[0128] Furthermore, the peristaltic movement of the digestive tract typically occurs at a cycle of approximately 3 to 20 times per minute. Additionally, the heartbeat at rest is typically around 50 to 100 beats per minute. On the other hand, the imaging element 26b used in common endoscopes is generally driven at 30 frames per second (fps) or 60 frames per second (fps). Therefore, compared to the speed at which the imaging element 26b acquires one frame of image data, it can be said that the pulsations and other movements within a living organism occur at a very slow cycle.

[0129] The focus control unit 16 is a structural unit or circuit unit that drives and controls the camera optical system 26a of the camera unit 26. The focus control unit 16 drives and controls the camera optical system 26a based on, for example, a set camera operation mode (e.g., depth synthesis camera mode) or the determination result obtained by the determination unit 15 based on the detection result of the image detection unit 14 of a specific object or distance distribution information, and performs a predetermined autofocus adjustment operation (hereinafter referred to as autofocus control) or pan-focus control, etc.

[0130] In addition, the focus control unit 16 functions as a focus position control unit that performs drive control of the imaging optical system 26a to adjust the relative position of the focusing lens and the imaging element on the optical axis and maintain the predetermined position of the focusing lens.

[0131] The lighting control unit 17 is a structural unit or circuit unit that drives and controls the light source device 40 and the lighting unit 25. The lighting control unit 17 controls the light source device 40 or the lighting unit 25 to switch the type of light source (white light, special light, etc.) based on the operation instruction signals from the various operation components 22c provided in the operation unit 22, or the instruction signals based on the detection results of the image detection unit 14 of a specific object and the determination results of the distance distribution information acquisition determination unit 15.

[0132] Here, as for the types of light sources, in addition to white light used to illuminate the subject during normal observation, there are also special lights used to illuminate the subject during image enhancement observations such as narrow band imaging (NBI) or red dichromatic imaging (RDI).

[0133] The above describes the structure of the endoscope system 1 including the focusing control device of this embodiment. Furthermore, the structures omitted from the description are the same as those of conventional endoscope systems.

[0134] Furthermore, all or part of the processor 10 includes hardware. Here, the processor 10 may be composed of, for example, a known structure that includes a central processing unit (CPU), RAM (Random Access Memory), ROM (Read Only Memory), non-volatile memory, non-volatile storage, and a non-transitory computer readable medium, as well as peripheral devices.

[0135] Fixed data such as software programs and data tables to be executed by the CPU are pre-stored in ROM, non-volatile memory, or other non-volatile storage devices. Furthermore, the CPU reads the software program stored in ROM or the like, expands it in RAM, and executes it. In addition, the software program appropriately references various data, thereby realizing the functions of the processor 10 described above.

[0136] Alternatively, the processor 10 can also be constructed from semiconductor chips such as FPGAs (Field Programmable Gate Arrays). Furthermore, the processor 10 can also be constructed from electronic circuits.

[0137] Furthermore, software programs can also be recorded or stored, either entirely or in part, on removable disk media such as floppy disks, CD-ROMs, and DVD-ROMs, card-type storage devices, HDD (Hard Disk Drive) devices, SSD (Solid State Drive) devices, and other non-transitory computer-readable mediums as computer program products.

[0138] The following describes the function of the endoscope system 1, which includes the focusing control device configured as described in this embodiment. Figure 5 This is a flowchart illustrating the function of an endoscope system incorporating a focusing control device according to one embodiment of the present invention. Figure 5 The flowchart shown illustrates the processing sequence that functions when using the endoscope system 1 for endoscopic examination.

[0139] First, the endoscope system 1 is activated and in a state where endoscopic examination can be performed. At this time, the endoscope 20 in the endoscope system 1 is inserted into the organ of the subject (patient, etc.) to be examined. Furthermore, the focusing control setting of the endoscope 20 in its initial state is, for example, set to a pan-focus control setting.

[0140] When the endoscope system 1 of this embodiment is in such a state, Figure 5 In step S1, the specific object image detection unit 14 of the processor 10 performs image detection processing of specific objects such as tumors based on the endoscopic image data sequentially acquired by the camera unit 26, and confirms whether a specific object that should be observed has been detected.

[0141] Here, if a specific object is detected, the process proceeds to step S2. Otherwise, if no specific object is detected, the process proceeds to step S8. Furthermore, the specific object image detection process in step S1 continues to be performed during the endoscopic examination.

[0142] Furthermore, the detection of a specific object in step S1 is performed by the specific object image detection unit 14, but it is not limited to this method. For example, the detection of a specific object can also be considered as a situation where a doctor or other equipment operator identifies the specific object by visually observing the display on the display device 30.

[0143] In this case, confirmation of whether a specific object has been detected can be achieved, for example, by checking the output signal generated when a doctor or other equipment operator manipulates a prescribed operating component. When a doctor or other equipment operator identifies a specific object, in most cases, they will manipulate the endoscope insertion section to face the object and attempt to bring the tip of the endoscope insertion section closer to the object. By detecting this series of actions, it can be determined that the doctor or other equipment operator has identified a specific object.

[0144] Furthermore, in the endoscope system 1 of this embodiment, it is configured to search for a specific object under a generalized focus control setting, and when the specific object is detected, switch to automatic focusing control to automatically adjust the focus with a predetermined focal point as the target. Additionally, when a specific object is detected, in order to observe it, the equipment operator, such as a doctor, may perform operations such as bringing the endoscope 20 close to the specific object or fixing the endoscope 20 at a predetermined position facing the specific object.

[0145] In the next step S2, the processor 10 receives image data (image information and phase difference information) sequentially output from the imaging element 26b of the imaging unit 26, and begins the acquisition processing of image data and depth data for each frame. At the same time, the processor 10 begins temporary storage processing of various data to the temporary storage unit 51 of the storage device 50 via the storage control unit 13.

[0146] Here, the acquired depth data may be, for example, data representing the unevenness or convexity of a specific object. For instance, such as... Figure 6 As shown, specific objects inside the organs of an organism sometimes appear as protrusions from the inner wall of the lumen of the organ or similar vessel. Figure 6 This is a schematic diagram illustrating the situation when using an endoscope to observe a specific object.

[0147] exist Figure 6 In this text, 300 indicates an organ, etc. 301 indicates a specific object protruding from the inner wall of the organ, etc. Additionally, 301a indicates the top of the specific object 301. 301b indicates the base of the specific object 301, specifically the side closest to the endoscope 20. Furthermore, in... Figure 6 In the diagram, O represents the optical axis of the camera optical system 26a, and V represents the field of view of the camera optical system 26a.

[0148] Figure 6The arrangement shown depicts the front end face 21a of the insertion portion 21 of the endoscope 20 being positioned facing a specific object 301 detected on the inner wall surface of an organ or similar structure 300. At this time, in... Figure 6 In the example shown, for example, L1 represents the distance from the front end face (front surface of the imaging optics system 26a) of the endoscope 20 to the top 301a of the object 301. Similarly, for example, L2 represents the distance from the front end face (front surface of the imaging optics system 26a) of the endoscope 20 to the base 301b of the object 301 on the side closest to the endoscope 20. Furthermore, in this case, since distance L1 > distance L2, distance L1 is referred to as the far point distance, and distance L2 is referred to as the near point distance.

[0149] In this case, automatic focusing control is configured to focus on a specific part of the object 301 (e.g., near the top of the head 301a). For example, here... Figure 6 The DOF1 marker indicates depth of field.

[0150] In this situation, when performing autofocus control, the focus point is set such that the top 301a of a specific object 301 and the base 301b near the endoscope 20 are within a depth of field (DOF 1) (i.e., in a focused state). Therefore, the focus point can be set based on depth data of the image area containing the object 301. Figure 6 In the situation shown (depth of field is marked DOF1), for example, the area in focus on the surface region of a specific object 301 is the area marked F1 as a thick solid line.

[0151] return Figure 5 In the next step S3, the processor 10 refers to an internal clock (not shown) to confirm whether a predetermined time has elapsed since the start of the process in step S2. If it is confirmed that the predetermined time has elapsed, the process proceeds to the next step S4. Otherwise, if the predetermined time has not elapsed, the process proceeds to step S8.

[0152] The reason for confirming the elapsed time through step S3 is as follows: By repeatedly performing step S2 within the specified time, multiple image data and depth data can be continuously acquired in a time sequence. Based on the multiple data acquired in this way, the processor 10 performs the specified processing, thereby acquiring information on the temporal distance change of the object and the period of distance change.

[0153] In order to obtain information such as distance changes, after object detection (step S1), it is necessary to fix the position of the endoscope 20 relative to the detected object for a specified time and continue to take pictures. Therefore, if the detection of the object is confirmed through the above-mentioned step S1, and the device user such as a doctor wants to observe the detected object in detail, the user can be given a prescribed instruction (e.g., please fix the endoscope or other guided display).

[0154] In step S4, the processor 10 determines whether a specific object is pulsating or throbbing based on the various data obtained in step S2 above (pulsation determination processing).

[0155] Here, the following uses Figure 7 Briefly describe the situation where pulsation or other similar phenomena occur inside the organs of a living organism. Figure 7 This is a schematic diagram illustrating situations such as pulsation that occur when observing a specific object using an endoscope. Furthermore, in Figure 7 The tags used in Figure 6 The same mark refers to the same thing or phenomenon.

[0156] Figure 7 The diagram shown by the solid line (refer to label [A]) represents the same as... Figure 6 The same situation. Figure 7 The diagram shown by the dashed line (refer to label [B]) indicates that from Figure 6 The condition changes over time due to factors such as pulsation. In other words, in... Figure 7 The image shows a specific object 301 changing at a predetermined period in the direction indicated by arrow S between the states shown in [A] and [B].

[0157] Here, the label DOF1 indicates that in Figure 7 The depth of field when autofocus control is performed with the area near the top 301a of a specific object 301 as the focus point, as shown in [A] (solid line). The focus area at this time is represented by the thick solid line area indicated by mark F1 (and...). Figure 6 same).

[0158] In contrast, Figure 7 In the text, the marker DOF2 indicates that in Figure 7 The depth of field when autofocus control is performed using a specific part of the object 301 (e.g., near the top of the head 301aa) as the focus point under the condition shown in [B] (dashed line). The focus area at this time is represented by the thick dashed line area indicated by mark F2.

[0159] Here, we consider a scenario where an endoscope 20 is fixed in a predetermined position to observe a specific object 301 as its condition changes over time due to pulsation or other factors. In this case, such as Figure 7 As shown, the distance between the front end face of the endoscope 20 and the top 301a of the specific object 301 is different in the situations shown in [A] and [B].

[0160] exist Figure 7 In the examples shown, specifically, for example, in the case shown in [A], the distal distance between the tip surface of the endoscope 20 and the top surface 301a is marked L1a. On the other hand, in the case shown in [B], the distal distance between the tip surface of the endoscope 20 and the top surface 301a is marked L1b.

[0161] At this time, if the distance to the far point L1a is greater than the distance to the far point L1b, then the depth of field DOF1 in the situation shown in [A] is different from the depth of field DOF2 in the situation shown in [B], and the focus areas F1 and F2 are also different.

[0162] Therefore, when observing (photographing) a specific object 301 with the endoscope 20 fixed in a predetermined position inside an organ or other part of a living organism, if the position of the specific object 301 changes due to pulsation or other reasons, the multiple endoscope images acquired continuously along the time sequence are images with different distance distribution information frame by frame and different focus areas centered on the object.

[0163] return Figure 5 If a pulsation or similar symptom is detected in step S4, the process proceeds to step S5. Otherwise, if no pulsation or similar symptom is detected, the process proceeds to step S8.

[0164] In step S5, the processor 10 determines whether the object can be handled by the first focus control based on the depth data and distance change information of the endoscopic image containing the object. Here, the first focus control is based on the object's concavity / convexity range (distance difference in the front-to-back direction) and distance change range (distance difference in the front-to-back direction) to control the focus position setting to obtain a depth of field that covers both DOF1 and DOF2.

[0165] Specifically, for example, according to Figure 7 Information on the convexity / concave range of the specific object 301 shown in [A] (distance information between the top 301a and the base 301b) and Figure 7For the specific object 301 in the situation shown in [B], the concavity and convexity range (distance information between the top 301aa and the base 301bb) is calculated. The distance information of the farthest point (the top 301a in [A]) and the distance information of the closest point (the base 301bb in [B]) are obtained. A depth of field DOF3 (refer to) is then performed to obtain the depth of field covering these two points (farthest and closest points). Figure 7 (Focus position setting)

[0166] Of course, if the terms "farthest point" and "nearest point" mentioned above refer to parts unrelated to diagnosis, they can be excluded from the concavity / convexity range of the specific object 301. Therefore, a technique is needed to determine whether a location is unrelated to diagnosis. For this purpose, techniques such as image-based pattern determination, similar image determination, and inference models can be used to determine features such as vascular patterns and tissue variations used in diagnosis. Furthermore, a structure is considered that equips the specific object detection unit 14 with such functionality.

[0167] In other words, when the focusing control device of the optical system of the front camera unit of the endoscope insertion section obtains the distance distribution information of the object in front in the front-back direction according to the output signal of the camera unit, it determines the image pattern of the object at each distance representing the distance distribution, and obtains the distance change information of the object over time. Based on the distance distribution information of the object's position after being classified by the above image pattern and the distance change information, the focusing position control is performed to adjust the focusing position.

[0168] Alternatively, before determining the distance change in time, the focusing control device of the optical system of the front camera unit of the endoscope insertion section can determine the image pattern of the object for each position corresponding to each distance representing different distance distributions when it obtains the distance distribution information of the object in the front and rear directions from the output signal of the camera unit.

[0169] Here, "corresponding positions" refers to the pattern used to determine the position of an object at 1mm, the pattern used to determine the position of an object at 2mm, and the pattern used to determine the position of an object at 3mm, when the distance distribution is divided into three cases, such as 1mm, 2mm, and 3mm.

[0170] In this case, for example, if there is no pattern that can be used as a reference for diagnosis at the object part at a distance of 3mm, it is possible to prioritize control that focuses on distances of 1mm and 2mm, thus making control easier.

[0171] Patterns are determined for locations at each distance that form a distance distribution. However, sometimes there are multiple locations at each distance, and the pattern at each location must be determined. When there are multiple locations corresponding to the above distances, the pattern at each location is determined. If at least one of the above multiple locations is important for diagnosis, it is set as the focus object distance.

[0172] Based on distance distribution information obtained by classifying object positions according to their diagnostic importance in image patterns, focus position adjustments are made, thereby enabling the production of natural images through unforced control. Locations deemed "not diagnostically important" can also be re-evaluated from other perspectives.

[0173] If it is determined in step S5 that the situation can be addressed through the first focus control, the process proceeds to step S9, where the first focus control is executed. Then, the process proceeds to step S10.

[0174] In step S10, the processor 10 fixes the focus position via the focus control unit 16. Then, the process proceeds to step S8.

[0175] On the other hand, if it is determined in step S5 that it is difficult to deal with the situation through the first focusing control, the process proceeds to step S6.

[0176] In step S6, the processor 10 determines whether the second focus control can address the issue based on the period of pulses, etc. Here, the second focus control is focus control that acquires multiple image data with different focus states for the same object. Furthermore, the second focus control includes so-called depth-synthesized image processing based on the acquired multiple image data.

[0177] For example, if a distance change period of a specific object is detected (e.g., a pulse of about 50-60 times per minute), and if this distance change period is slow enough compared to the frame rate of the camera unit 26 (e.g., 30 frames per second), it is considered that multiple image data can be acquired at appropriate times during one pulse. Therefore, in this case, it can be determined that the second focus control is capable of handling the situation.

[0178] Specifically, for example, if a distance change period of approximately 50-60 times per minute is detected, and the frame rate of the camera unit 26 is, for example, 30 frames per second, then multiple image data can be acquired at appropriate times during a single pulse, etc. For example, the first pair Figure 7 In condition [A], the top of the head 301a is captured by AF camera, the second pair Figure 7 AF imaging was performed on the base 301b in condition [A], the third pair Figure 7[B] The top of the head 301aa was captured by AF camera, the fourth pair Figure 7 AF imaging is performed on the base 301bb under condition [B]. This allows the acquisition of four image data points. Furthermore, depth synthesis processing is performed on these four image data points. The image displayed based on the image data generated in this way can be displayed as an image with a greater depth of field.

[0179] If it is determined in step S6 that the situation can be addressed through the second focus control, the process proceeds to step S7, where the second focus control is executed. Then, the process proceeds to step S8.

[0180] On the other hand, if it is determined in step S6 that it is difficult to deal with the situation through the second focusing control, the process proceeds to step S11.

[0181] In step S11, the processor 10 executes third focus control. Then, it proceeds to step S8. Here, the third focus control is an automatic focus control that follows the movement of a specific object based on the detected distance change period of that object. Furthermore, in cases where it is impossible to follow the movement of the object, it can be switched to normal automatic focus control performed sequentially for each frame.

[0182] In step S8, the processor 10 confirms an instruction indicating the end of the ongoing endoscopic examination. Here, the instruction to end the endoscopic examination may be, for example, a predetermined examination end indication signal generated by operating a predetermined operating member in the operating member 22c on the operating section 22 of the endoscope 20, such as the processor 10 of the endoscope system 1. Alternatively, the intention to end the endoscopic examination may also be determined by confirming the image data of the endoscopic images acquired by the endoscope 20.

[0183] If an inspection end indication is confirmed in step S8, the series of processes ends (end). Otherwise, if no inspection end indication is confirmed, the process returns to step S1 and repeats the subsequent processes.

[0184] As explained above, in the endoscope system 1 including the focusing control device of this embodiment, distance distribution information (depth map) obtained based on image data of each frame continuously acquired along the time sequence is used to detect the temporal distance change information, the period of distance change, and the range of distance change in the front-back direction of a specific object, thereby determining the pulsation of the object, etc. (see reference). Figure 5 Steps S2~S4).

[0185] Furthermore, by appropriately switching focus control to perform imaging actions based on the shape, state, and periodicity of the object's pulsation, endoscopic images can be acquired over a wider area while maintaining focus (see reference). Figure 5 (Steps S5~S11). In this way, clear and high-resolution endoscopic images can always be obtained, thus contributing to the efficiency and precision of endoscopic examinations.

[0186] Next, use Figure 8 The flowchart will be used to describe in detail a variation of the function of the endoscope system 1, which includes the focusing control device of the above embodiment, particularly the determination and processing of pulsation, etc. Figure 8 This is a flowchart illustrating the determination and processing of pulsation, etc., in a modified example of the function of the focusing control device of this embodiment.

[0187] First of all, Figure 8 In step S21, the processor 10 performs object detection processing. This object detection processing is equivalent to... Figure 5 The process of step S1 is as described above. Here, if an object is detected, the process proceeds to the next step, S22. If no object is detected, this process is repeated.

[0188] When an object is detected and the process proceeds to step S22, in step S22, the processor 10 executes automatic focus control (focus adjustment action) with the detected object as the target.

[0189] Typically, when an object is detected, an endoscope is operated so that the detected object is imaged approximately in the central area of ​​the imaging element 26b. Furthermore, when autofocus control is performed, the focal point is usually set based on, for example, the central area of ​​the image frame. Therefore, in this case, for example, it is assumed that the focal point is set near the top of the object's head for autofocus control.

[0190] Next, in step S23, the processor 10 performs a camera operation and temporarily records the acquired image data in the temporary storage unit 51. The image data recorded at this time is then labeled lm1.

[0191] Next, in step S24, the processor 10 records various information when capturing image data lm1, such as focus position information. The focus position information recorded at this time is represented by a marker Z1. This focus position information Z1 is data corresponding to the image data lm1.

[0192] Next, in step S25, the processor 10 performs a process of comparing the previous image data recorded before image data lm1 with the current image data lm1.

[0193] In the next step S26, the processor 10 checks whether there is an image change between the previous image data and the current image data lm1. Here, the image change is considered to be, for example, a change in distance distribution information. In this case, the processor 10 (distance distribution information acquisition and determination unit 15) functions as a change determination unit for the distance change of the object in the front-back direction. If an image change is confirmed, the process proceeds to step S31. Otherwise, if no image change is detected, the process proceeds to step S27.

[0194] In step S27, the processor 10 maintains the focusing lens position fixed based on the currently set focus position information Z1. Then, it returns to the processing in step S21 and repeats the subsequent processing steps.

[0195] On the other hand, when an image change is detected during the processing in step S26 and the process proceeds to step S31, the processor 10 performs object detection processing again in step S31. This object detection processing is largely the same as the processing in step S21 described above.

[0196] Here, if an object is detected, proceed to the next step, S32. Alternatively, if no object is detected, for example, if it is assumed that the object detected last time (in step S21) was missed, or if the object has not been observed, switch to search mode. In this case, switch the focus control setting to pan-focus setting and return to the process described in step S21. Then, repeat the subsequent processes.

[0197] Next, when an object is detected in step S31 and the process proceeds to step S32, in step S32, the processor 10 executes automatic focus control (focus adjustment action) with the detected object as the target. This process is the same as the process in step S22 described above.

[0198] Next, in step S33, the processor 10 performs a camera operation and temporarily records the acquired image data in the temporary storage unit 51. The image data recorded at this time is marked lm2.

[0199] Next, in step S34, the processor 10 records various information when capturing image data lm2, such as focus position information. The information about the focus position (focus point) recorded at this time is represented by a marker Z2. This focus position information Z2 is data corresponding to the image data lm2.

[0200] Next, in step S35, the processor 10 performs a process of comparing the previous image data lm1 with the current image data lm2.

[0201] In the next step S36, the processor 10 checks whether there is an image change between the previous image data lm1 and the current image data lm2. In this case, the processor 10 (distance distribution information acquisition and determination unit 15) functions as a change determination unit for the distance change of the object in the front-back direction. Here, if an image change is confirmed, it is determined that the object may be pulsating, etc., and the process proceeds to step S38. Otherwise, if there is no image change, the process proceeds to step S37.

[0202] In step S37, the processor 10 maintains the focusing lens position fixed based on the currently set focus position information Z2. Then, it returns to the processing in step S31 and repeats the subsequent processing steps.

[0203] On the other hand, in step S38, the processor 10 controls the guide display unit 12a of the display control unit 12 to display the pulsation determination result using the display device 30 or the like. In this case, the display of the pulsation determination result is a way of informing or explaining the current examination status to the equipment operator. Specifically, this could include text displays indicating that the observed object may be pulsating, and thus reminding the operator not to move the endoscope during operation, or icon displays composed of prescribed graphic characters.

[0204] Next, in step S41, the processor 10 performs the object detection process again. This object detection process is largely the same as the processes described in steps S21 and S31 above.

[0205] If an object is detected, proceed to step S42. If no object is detected, switch the focus control setting to pan-focus and return to step S21. Then, repeat the subsequent steps.

[0206] Next, when an object is detected in step S41 and the process proceeds to step S42, in step S42, the processor 10 executes automatic focus control (focus adjustment action) with the detected object as the target. This process is the same as the processes in steps S22 and S32 described above.

[0207] Next, in step S43, the processor 10 performs a camera operation and temporarily records the acquired image data in the temporary storage unit 51. The image data recorded at this time is marked lm3.

[0208] Next, in step S44, the processor 10 performs a process of comparing the image data lm1 with the current image data lm3.

[0209] In the next step S45, the processor 10 confirms whether image data lm3 is similar to image data lm1. If similarity is confirmed, it is determined that the object is pulsating, etc. In this case, the processor 10 (distance distribution information acquisition and determination unit 15) functions as a unit determining the change in distance of the object in the front-back direction. Furthermore, in this way, if it is determined that the object is pulsating, etc., information such as the period of pulsation (period of distance change) and the range of distance change in the front-back direction is further acquired. Then, the process moves to... Figure 9 The processing sequence (focus control switching processing).

[0210] On the other hand, if it is determined in step S45 that image data lm1 and image data lm3 are not similar, the process proceeds to step S46.

[0211] In step S46, the processor 10 determines that the object may be pulsating, etc. (refer to...) Figure 8 Reset the "Yes" option in S36. Then, return to the processing in step S21 and repeat the subsequent processing.

[0212] Figure 9 It is shown in Figure 8 The flowchart shows the process of switching focus control after determining and processing the pulsation, etc.

[0213] exist Figure 9 In step S51, the processor 10 determines whether the object is pulsating, etc. (pulsation determination processing). The processing performed here is similar to... Figure 5 The process in step S4 is the same. Here, if it is determined that the object is pulsating, etc., the process proceeds to step S52. Conversely, if it is determined that the object is not pulsating, etc., the process returns to... Figure 8 Step S21 processing (refer to) Figure 8 , Figure 9 (marked 9B).

[0214] Next, in step S52, the processor 10 executes the first focus control. This first focus control is related to... Figure 5 The same process as step S9.

[0215] In step S53, the processor 10 checks whether a periodicity such as a pulsation is detected. If a periodicity such as a pulsation is detected, the process proceeds to step S58. Otherwise, if a periodicity such as a pulsation is not detected, the process proceeds to step S54.

[0216] In step S54, the processor 10 compares image data lm1 with image data lm2.

[0217] In step S55, the processor 10 checks whether image data lm1 and image data lm2 are similar. If image data lm1 and lm2 are not similar, the process proceeds to step S56. If image data lm1 and lm2 are similar, the process proceeds to step S57.

[0218] In step S56, the processor 10 maintains the focusing lens position fixed based on the currently set focus position information. Then, it returns to... Figure 8 Step S41 processing (refer to) Figure 8 , Figure 9 (marked 9A).

[0219] In step S57, the processor 10 determines the pulsation, etc., of the object (refer to...). Figure 9 Reset the S51 ("Yes") function. Then, return. Figure 8 Step S21 processing (refer to) Figure 8 , Figure 9 (marked 9B).

[0220] On the other hand, when a periodicity such as a pulsation is detected in step S53 above, and the process proceeds to step S58, in step S58, the processor 10 executes either the second focus control or the third focus control. The second focus control is... Figure 5 The process is the same as step S7. Additionally, the third focus control is the same as... Figure 5 The process is the same as step S11. Here, the focus control can be selected and set as needed.

[0221] Here, the effects of the first to third focusing controls are briefly explained using the accompanying diagrams. Figure 10 This is a schematic diagram illustrating the effect of the first focusing control based on the temporal changes in distance distribution information. Figure 10 In the diagram, the vertical axis represents the reciprocal (1 / D) of the distance D based on the distance distribution information. This means that from... Figure 10 The further upwards from the origin 0, the closer the distance represents; the closer to the origin, the farther the distance represents. Figure 10 In the middle, the horizontal axis represents time. From Figure 10 The further to the right the origin is, the longer the elapsed time.

[0222] exist Figure 10 In the diagram, curve [HD] represents the time variation of the distance distribution at the top of the object's head. Curve [BT] represents the time variation of the distance distribution at the top of the object's head. Curve [Focus] (dashed line) represents the time variation of the focal point.

[0223] Figure 10The periods indicated by the markers [A1], [A2], etc., are set as the object being in a certain state. Figure 7 During the period indicated by [A]. Additionally, in Figure 10 The periods indicated by the markers [B1], [B2], [B3]... are set as the object being in a certain state. Figure 7 The period indicated by [B]. That is, the object changes between state [A] and state [B] at a predetermined time period. Such a state is called a state in which pulsation or the like is occurring.

[0224] Here, for example, in Figure 10 During [B1], the depth of field is indicated by the symbol DOF2 when focusing is performed on a specified position near the base [BT]. Additionally, in Figure 10 During [A1], the depth of field is indicated by the symbol DOF1 when focusing is performed using a specified position near the base [BT] as the focal point.

[0225] In this case, when the object is from Figure 10 When the state of [B1] changes to the state of [A1], for example, in Figure 10 The areas shown in [X1] and [X2] change from a focused state to an out-of-focus state, or vice versa. This is one reason why some areas of the image lack sharpness, hindering observation.

[0226] Therefore, in the first focus control, a focus position setting of depth of field DOF3 is performed to obtain both depth of field DOF2 in condition [B] and depth of field DOF1 in condition [A]. This depth of field DOF3 has the depth to ensure the focus state within the desired area (the area of ​​the object to be observed) even if the distance distribution changes over time due to the movement of the object.

[0227] Figure 11 This is a schematic diagram illustrating the effect of the second focusing control based on the temporal changes in distance distribution information. Figure 11 The tags used are Figure 10 Based on. For example... Figure 11 As shown, in the second focus control, for example, as using Figure 7 As explained, first, proceed with... Figure 11 AF imaging of the base section in condition [A2] (refer to reference mark [S1]), then, proceeding... Figure 11 The AF camera at the top of the head during the [A2] condition (refer to mark [S2]), then, proceed... Figure 11 AF imaging of the base section in condition [B3] (refer to mark [S3]), and finally, [the process is as follows]. Figure 11The AF camera at the top of the head captures images in condition [B3] (see reference mark [S4]). In this way, multiple (in this case, four) image data are acquired. Furthermore, depth synthesis image processing is performed on these multiple (four) image data. An image is displayed based on the image data generated as a result.

[0228] Figure 12 This is a schematic diagram illustrating the effect of the third focusing control based on the temporal changes in distance distribution information. Figure 12 The tags used are also Figure 10 Based on. For example... Figure 12 As shown, in the third focus control, the focus point follows the movement of the object. Therefore, the area where the focus state can be obtained changes according to the changes in the object. However, since the frame rate of the camera is sufficiently high relative to the period of the object's movement, the occurrence of the out-of-focus area is considered to be very short-lived. Under such circumstances, it can be expected that good endoscopic images can be obtained even through the third focus control.

[0229] As explained above, the same effect as the above-described embodiment can be obtained according to the above-described modified example.

[0230] Furthermore, in the above-mentioned variant, the processing is set as follows: Figure 8 If the determination result in step S45 is that the object is pulsating, then proceed to... Figure 9 The processing sequence.

[0231] However, alternative processing methods can be used, for example, by setting the following processing sequence: In Figure 8 If the result of step S45 is that a pulsation is occurring, then proceed to... Figure 5 The processing of step S4. In this case, it can be considered that from Figure 8 The processing sequence from step S21 to step S45 is equivalent to Figure 5 The processing of steps S2 and S3.

[0232] Through such a processing sequence, the same effect as in one implementation method and its variations can be obtained.

[0233] This invention is not limited to the embodiments described above, and it is self-evident that various modifications and applications can be implemented without departing from the spirit of the invention. For example, an endoscope was used as an example in this description, but any device that sequentially obtains and judges images (such as an image inspection device) can be used. Furthermore, the invention includes various stages in the above embodiments, and various inventions can be extracted by appropriately combining the multiple constituent elements disclosed. For example, if the problem to be solved by the invention can be solved and the effect of the invention can be obtained even if several constituent elements are deleted from all the constituent elements shown in one of the above embodiments, the structure after deleting the constituent element can be extracted as an invention. Furthermore, the constituent elements of different embodiments can also be appropriately combined. This invention is not limited to its specific embodiments except as defined by the appended claims.

Claims

1. A focusing control device for adjusting the focus position based on an output signal from a camera unit, the camera unit being disposed at the front end of an endoscope insertion section and having an optical system for adjusting the focus by moving back and forth along the optical axis, the focusing control device being characterized by comprising: The distance distribution information acquisition unit acquires distance distribution information of an object in the front-back direction when observed from the camera unit from the output signal of the camera unit, and acquires distance change information of the object over time. The range determination unit determines the range of distance changes of the object based on the distance distribution information and the distance change information; and The focus position control unit adjusts the focus position based on the determination result of the range determination unit.

2. The focusing control device according to claim 1, characterized in that, The distance distribution information acquisition unit acquires the distance distribution information based on the phase difference information or image information in the output signal from the camera unit.

3. The focusing control device according to claim 1, characterized in that, The focus position control unit sets a focus position based on the determination result of the range determination unit to ensure a depth of field that includes the farthest and nearest points within the range of distance variation of the object.

4. The focusing control device according to claim 1, characterized in that, It also includes a period determination unit, which determines the period of distance change of the object based on the distance distribution information and the distance change information. The range determination unit determines the range of distance change of the object based on the determination result of the period determination unit.

5. The focusing control device according to claim 4, characterized in that, The distance distribution information acquisition unit repeatedly acquires the distance distribution information at a period shorter than the period of distance change of the object.

6. The focusing control device according to claim 4, characterized in that, It also includes a change determination unit, which determines the change in distance of the object in the front-back direction. The focusing position control unit predicts the focusing position corresponding to the distance change cycle of the object based on the determination results of the change determination unit and the range determination unit, and performs focusing position tracking control.

7. The focusing control device according to claim 4, characterized in that, It also includes a specific object image detection unit, which detects image regions containing specific objects from images based on image data acquired by the camera unit. When the specific object is detected by the specific object image detection unit, the focus position control unit performs automatic focus control for that specific object.

8. The focusing control device according to claim 4, characterized in that, It also has: A temporary storage unit that temporarily stores multiple output signals sequentially output from the camera unit in a time sequence; and The depth compositing unit performs image compositing processing on the same object based on multiple image data with different focus states. The focus position control unit performs multiple focus controls to acquire multiple image data with different focus states for the same object based on the determination results of the range determination unit and the period determination unit. The camera unit captures and outputs multiple image data for each of the multiple focus controls performed by the focus position control unit. The temporary storage unit temporarily stores the multiple image data. The depth synthesis processing unit performs image synthesis processing based on the multiple image data.

9. The focusing control device according to claim 7, characterized in that, The specific object image detection unit has an inference model, which is constructed using a large amount of specific object image data as training data. The inference model is constructed by setting annotations through the display of bounding boxes around the image region containing the specific object.

10. The focusing control device according to claim 4, characterized in that, It also has a guidance display section. When it is determined, based on the determination result of the range determination unit or the period determination unit, that the object may have undergone periodic distance changes, a guidance display to remind and pay attention is provided.

11. A focusing control method, characterized in that, The distance distribution information of the object in the front-to-back direction under the observation of the camera unit is obtained from the output signal of the camera unit, and the distance change information of the object over time is also obtained. Based on the distance distribution information and the distance change information, the range of distance change of the object is determined. The focusing position is adjusted based on the determination result of the range determination unit.

12. A focus control program that causes a computer to perform the following processes: Distance distribution information acquisition and processing: acquiring distance distribution information of an object in the front-back direction when observed from the camera unit, and acquiring distance change information of the object over time; The range determination process determines the range of distance changes of the object based on the distance distribution information and the distance change information. as well as The focus position control process adjusts the focus position based on the determination result of the range determination unit.

13. An endoscope system comprising an endoscope and a processor, the endoscope comprising: a camera unit including a camera optics system for imaging an optical image of an object and an imaging element for photoelectric conversion of the optical image imaged by the camera optics system to acquire an image signal; and a focus adjustment mechanism for moving a portion of an optical lens included in the camera optics system back and forth in a direction along the optical axis. The processor has: The distance distribution information acquisition unit acquires distance distribution information of an object in the front-back direction when observed from the camera unit from the output signal of the camera unit, and acquires distance change information of the object over time. The range determination unit determines the range of distance changes of the object based on the distance distribution information and the distance change information; and The focus position control unit adjusts the focus position based on the determination result of the range determination unit.

14. A focusing control device, which is a focusing control device for an optical system of a front-end camera unit of an endoscope insertion section, characterized in that it comprises: The specific object detection unit, when acquiring distance distribution information of objects in the foreground in the front-back direction from the output signal of the camera unit, determines the image pattern of the object for each position corresponding to each distance representing different distances in the distance distribution; and The focus position control unit adjusts the focus position based on the distance distribution information of the position of the object after it has been classified by the image pattern.

15. The focusing control device according to claim 14, characterized in that, It also includes a range determination unit, which acquires the distance change information of the object over time to determine the range of the distance change of the object. The focus position control unit adjusts the focus position based on the determination result of the range determination unit.

16. The focusing control device according to claim 14, characterized in that, When there are multiple locations corresponding to each distance, the specific object detection unit determines the pattern of the image at each location. If at least one of the multiple locations is important for diagnosis, the focusing position control unit sets that location as the focusing object distance.

17. A focusing control device, which is a focusing control device for an optical system of a front-end camera unit of an endoscope insertion section, characterized in that it comprises: When the specific object detection unit acquires the distance distribution information of the object in front in the front-back direction from the output signal of the camera unit, the specific object detection unit determines the image pattern of the object for each position corresponding to each distance representing different distances in the distance distribution. The range determination unit acquires information about the temporal distance change of an object to determine the range of distance change of the object; and The focus position control unit adjusts the focus position based on the determination result of the range determination unit.

18. A focusing control method, characterized in that, When acquiring distance distribution information of objects in the foreground in the front-back direction from the output signal of the camera unit, the image pattern of the object is determined for each position corresponding to each distance representing different distances in the distance distribution. The range of distance change of the object is determined by acquiring information about the distance change of the object over time. The focus position is adjusted based on the determination of the distance variation range of the object.

19. A focusing control program, characterized in that, To cause the computer to perform the following processes: Image pattern determination processing: When obtaining distance distribution information of objects in the front and rear directions from the output signal of the camera unit, the image pattern of the object is determined for each position corresponding to each distance in the distance distribution that represents different distances. Range determination processing involves acquiring temporal distance change information of an object to determine the range of distance change of the object; and The focus position control process adjusts the focus position based on the result of the range determination.

20. A focusing control method, characterized in that, When acquiring distance distribution information of objects in the foreground in the front-back direction from the output signal of the camera unit, the image pattern of the object is determined for each position corresponding to each distance representing different distances in the distance distribution. The range of distance change of the object is determined by acquiring information about the distance change of the object over time. The camera images at each focal position are synthesized after the determination result based on the distance change range of the object has been adjusted multiple times over time.

Images