Control device, endoscope system, operation method of control device, and program

Abstract

To provide a control device, an endoscope system, an operation method of the control device, and a program capable of suppressing an image from becoming dark as a whole by brightness control operated due to an image becoming bright by the influence of halation generated when an instrument is attached to a distal end part of an endoscope.SOLUTION: The processor performs brightness control for bringing brightness of an image, which is obtained by imaging the inside of the body by the endoscope in a state in which the inside of the body is irradiated with light, close to target brightness. The brightness control is classified into first brightness control and second brightness control. The first brightness control is performed in a case where no instrument is attached to the distal end portion of the endoscope. The second brightness control is control that is performed in a case where an instrument is attached to the distal end portion of the endoscope and is different from the first brightness control.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a control device, an endoscope system, an operation method of the control device, and a program. [Background technology]

[0002] Patent Document 1 discloses an imaging device including an imaging unit that captures an image of a subject, a comparison unit, a photometric unit, and a condition adjustment unit. In the imaging device described in Patent Document 1, the comparison unit compares the brightness of a first portion of the image captured by the imaging unit, including the center, with the brightness of a second portion of the image that is closer to the edge than the first portion. The photometric unit measures the overall brightness of the captured image by comparing the average brightness of the image captured by the imaging unit with the brightness of high-brightness portions in the captured image in the comparison unit, weighting the brightness of the high-brightness portions more heavily the brighter the first portion is compared with the brightness of the second portion. The condition adjustment unit adjusts the imaging conditions that affect the brightness of the captured image based on the brightness measured by the photometric unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-060237 Summary of the Invention

[0004] One embodiment of the present disclosure provides a control device, an endoscope system, an operating method of the control device, and a program that can suppress the overall darkening of an image due to brightness control that is activated when an instrument is attached to the tip of an endoscope, which causes the image to become brighter due to the effect of halation. [Means for solving the problem]

[0005] A first aspect of the present disclosure is a control device that includes a processor and is used for an endoscope having a tip that irradiates light, wherein the processor performs brightness control to bring the brightness of an image obtained by imaging the inside of the body with the endoscope while light is irradiated inside the body closer to a target brightness, and the brightness control is classified into first brightness control and second brightness control, wherein the first brightness control is performed when no instrument is attached to the tip, and the second brightness control is performed when an instrument is attached to the tip, and is a control that is different from the first brightness control.

[0006] A second aspect of the present disclosure is a control device according to the first aspect, in which first brightness control or second brightness control is performed based on the result of comparing the photometric value of a first image obtained by imaging the inside of the body with an endoscope while light is irradiated inside the body with a target photometric value, and the second brightness control increases the degree to which the brightness of the second image obtained after the first image by imaging the inside of the body with an endoscope while light is irradiated inside the body to a greater extent than the first brightness control.

[0007] A third aspect of the present disclosure is a control device according to the second aspect, in which the brightness of the second image is increased by lengthening the exposure time of the endoscope when imaging is performed to obtain the second image, by increasing the amount of light when imaging is performed to obtain the second image, or by increasing the degree of influence that image processing has on the second image when image processing for brightness adjustment is performed on the second image.

[0008] A fourth aspect of the present disclosure is the control device according to the third aspect, in which the image processing is performed on a specific area of ​​the second image, and the specific area is determined according to the appliance.

[0009] A fifth aspect of the present disclosure is a control device according to any one of the second to fourth aspects, in which brightness control is performed based on multiple photometric values ​​obtained by measuring the first image using multiple photometric methods, and the degree to which the multiple photometric values ​​are used differs between the first brightness control and the second brightness control.

[0010] A sixth aspect of the present disclosure is a control device according to the fifth aspect, in which the multiple photometric values ​​include an average photometric value of the first image, and the degree to which the average photometric value is used in the second brightness control is greater than the degree to which the average photometric value is used in the first brightness control.

[0011] A seventh aspect of the present disclosure is a control device according to any one of the second to sixth aspects, in which brightness control is performed based on the photometric value of the first image, the photometric value is obtained for each divided area into which the first image is divided using a division method that differs between the first brightness control and the second brightness control, and the magnitude of the photometric value used in the first brightness control and the second brightness control is a magnitude that corresponds to the divided area.

[0012] An eighth aspect of the present disclosure is the control device according to the seventh aspect, in which the division method used in the second brightness control is determined depending on the appliance.

[0013] A ninth aspect of the present disclosure is a control device according to any one of the second to eighth aspects, in which brightness control is performed based on the photometric value of the first image, and the photometric value used in the second brightness control is obtained from an area of ​​the first image that is more inward than the photometric value used in the first brightness control.

[0014] A tenth aspect of the present disclosure is a control device according to any one of the first to ninth aspects, in which the endoscope is a variable magnification endoscope, and the second brightness control is enabled when the magnification of the variable magnification endoscope is a first magnification, and disabled when the magnification is a second magnification greater than the first magnification or a third magnification less than the first magnification.

[0015] An eleventh aspect of the present disclosure is a control device according to any one of the first to tenth aspects, in which an enabled state in which the second brightness control is enabled and an disabled state in which the second brightness control is disabled are switched depending on the type of endoscope.

[0016] A twelfth aspect of the present disclosure is a control device according to any one of the first to eleventh aspects, in which the brightness of the image changes gradually when switching from one of the first brightness control and the second brightness control to the other.

[0017] A thirteenth aspect of the present disclosure is a control device according to the twelfth aspect, in which the gradual change in image brightness is achieved by gradually increasing the exposure time of the endoscope when imaging is performed to obtain the image, by gradually increasing the amount of light when imaging is performed to obtain the image, or by gradually increasing the degree of influence that image processing has on the image when image processing for brightness adjustment is performed on the image.

[0018] A fourteenth aspect of the present disclosure is the control device according to any one of the first to thirteenth aspects, in which the content of the second brightness control differs depending on the type of appliance.

[0019] A fifteenth aspect of the present disclosure is the control device according to any one of the first to fourteenth aspects, wherein the appliance is a cylindrical hood having an opening.

[0020] A sixteenth aspect of the present disclosure is the control device according to any one of the first to fifteenth aspects, wherein the device has optical properties that cause halation due to light.

[0021] A seventeenth aspect of the present disclosure is a control device according to any one of the first to sixteenth aspects, in which the target brightness used in the second brightness control is higher than the target brightness used in the first brightness control.

[0022] An eighteenth aspect of the present disclosure is an endoscope system including a control device according to any one of the first to seventeenth aspects and an endoscope.

[0023] A 19th aspect of the present disclosure is an operation method of a control device used for an endoscope having a tip that irradiates light, which includes performing brightness control to bring the brightness of an image obtained by imaging the inside of the body with the endoscope while light is irradiated into the body closer to a target brightness, wherein the brightness control is classified into first brightness control and second brightness control, the first brightness control is performed when no instrument is attached to the tip, and the second brightness control is performed when an instrument is attached to the tip, and is a control different from the first brightness control.

[0024] A twentieth aspect of the present disclosure is a program for causing a computer used with an endoscope having a tip that irradiates light to execute processing, the processing including performing brightness control to bring the brightness of an image obtained by imaging the inside of the body with the endoscope while light is irradiated inside the body closer to a target brightness, the brightness control being classified into first brightness control and second brightness control, the first brightness control being performed when no instrument is attached to the tip, and the second brightness control being performed when an instrument is attached to the tip, and being a control different from the first brightness control. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 1 is a conceptual diagram showing an example of how the endoscope system is used by a doctor. [Figure 2] FIG. 2 is a block diagram showing an example of a hardware configuration of an electrical system of the endoscope system. [Figure 3] 10A and 10B are conceptual diagrams showing an example of a display mode of a moving image when the hood is not attached and an example of a display mode of a moving image when the hood is attached. [Figure 4] FIG. 2 is a block diagram showing an example of main functions of a processor. [Figure 5] FIG. 10 is a conceptual diagram illustrating an example of processing content of a determination unit. [Figure 6] FIG. 4 is a conceptual diagram illustrating an example of processing content of a control unit. [Figure 7A]10 is a flowchart illustrating an example of the flow of a brightness control process. [Figure 7B] This is a continuation of the flowchart shown in FIG. 7A. [Figure 8] FIG. 10 is a conceptual diagram showing a first modified example of the processing content of the control unit. [Figure 9] FIG. 10 is a conceptual diagram showing a second modified example of the processing content of the control unit. [Figure 10] 10 is a conceptual diagram showing an example of how the brightness of a frame is changed in stages from a first brightness to a second brightness. FIG. [Figure 11] 10 is a conceptual diagram showing an example of how the brightness of a frame is monotonically changed from a first brightness to a second brightness. FIG. [Figure 12] FIG. 10 is a conceptual diagram showing a first modified example of the processing content of the determination unit. [Figure 13] FIG. 10 is a conceptual diagram showing a second modified example of the processing content of the determination unit. [Figure 14] FIG. 10 is a conceptual diagram showing a third modified example of the processing content of the determination unit. [Figure 15] FIG. 10 is a conceptual diagram showing a fourth modified example of the processing content of the determination unit. [Figure 16] FIG. 10 is a conceptual diagram showing a fifth modified example of the processing content of the determination unit. [Figure 17] FIG. 10 is a conceptual diagram showing a third modified example of the processing content of the control unit. [Figure 18] FIG. 10 is a conceptual diagram showing a fourth modified example of the processing content of the control unit. [Figure 19] FIG. 10 is a conceptual diagram showing a sixth modified example of the processing content of the determination unit. DETAILED DESCRIPTION OF THE INVENTION

[0026] Hereinafter, exemplary embodiments of a control device, an endoscope system, an operation method of a control device, and a program according to the present disclosure will be described with reference to the accompanying drawings. Note that the present disclosure can also be applied to a program and a computer program product.

[0027] First, the terms used in the following description will be explained.

[0028] CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". GPGPU is an abbreviation for "General-Purpose computing on Graphics Processing Units". APU is an abbreviation for "Accelerated Processing Unit". TPU is an abbreviation for "Tensor Processing Unit". RAM is an abbreviation for "Random Access Memory". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus". EL is an abbreviation for "Electro-Luminescence". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device." I / F is an abbreviation for "Interface." 5G is an abbreviation for "5th Generation Mobile Communication System." IC is an abbreviation for "Integrated Circuit."

[0029] In the following description, a coded processor (hereinafter simply referred to as a "processor") may be a single physical or virtual computing device, or a combination of multiple physical or virtual computing devices. Furthermore, a processor may be a single type of computing device, or a combination of multiple types of computing devices. Examples of computing devices include a CPU, a GPU, a GPGPU, an APU, or a TPU.

[0030] In the following description, a signed memory is a memory such as a RAM in which information is temporarily stored, and is used as a work memory by a processor.

[0031] In the following description, the term "storage" refers to one or more nonvolatile storage devices that store various programs, various parameters, etc. Examples of nonvolatile storage devices include flash memory, magnetic disks, and magnetic tapes. Another example of storage is cloud storage.

[0032] In the following embodiments, the external I / F with a symbol controls the exchange of various information between multiple devices connected to each other. An example of the external I / F is a USB interface. A communication I / F including a communication processor, an antenna, etc. may be applied to the external I / F. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G, Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0033] In the following description, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0034] In the following description, "same shape" refers to an identical shape that includes not only a completely identical shape, but also an error that is generally acceptable in the technical field to which this disclosure pertains and that does not contradict the spirit of this disclosure.

[0035] Fig. 1 shows an example of an embodiment in which an endoscope system 10 is used. As shown in Fig. 1, the endoscope system 10 is used by a doctor 12 in an endoscopic examination or the like. The endoscope system 10 includes an endoscope 14, a display device 16, a control device 18, and a light source device 20.

[0036] The endoscopy system 10 is a modality that allows a doctor 12 to use an endoscope 14 to examine a large intestine 24 contained within the body of a subject 22 (e.g., a patient). Here, a lower endoscopy is shown as an example for examining the large intestine 24, but this is merely one example, and the present disclosure is also applicable to endoscopic examinations that are performed for the purpose of examining hollow organs other than the large intestine 24, such as upper endoscopy.

[0037] The endoscope 14 is used by a doctor 12 and inserted into a large intestine 24 of a subject 22. The endoscope system 10 causes the endoscope 14 inserted into the large intestine 24 to capture images of the inside of the large intestine 24 (for example, an area including a colon wall 26), and performs various medical procedures on the large intestine 24 as necessary.

[0038] The endoscope 14 includes a camera 30 that captures images of the inside of the large intestine 24, and an illumination device 32 that irradiates the inside of the large intestine 24 with light 34 via irradiation ports 32A and 32B. The camera 30 has an objective lens 30A. The objective lens 30A is provided at a tip 36 of the endoscope 14 (at the tip surface of the endoscope 14 in the example shown in FIG. 1). The tip 36 (at the tip surface of the endoscope 14 in the example shown in FIG. 1) is also provided with irradiation ports 32A and 32B. Subject light, which is reflected light obtained by irradiating the light 34 from the irradiation ports 32A and 32B within the large intestine 24 and reflecting it off an area including the intestinal wall 26, enters the objective lens 30A. The subject light that has entered the objective lens 30A is imaged by the camera 30.

[0039] A treatment tool opening 38 is provided at the tip 36 of the endoscope 14 (the tip surface of the endoscope 14 in the example shown in FIG. 1). The treatment tool opening 38 is an opening for allowing a treatment tool (not shown) to be ejected from the tip 36 (the tip surface of the endoscope 14 in the example shown in FIG. 1). Examples of treatment tools that can be ejected from the treatment tool opening 38 include grasping forceps, a puncture needle, a papillotomy knife, a snare, a catheter, a guidewire, a cannula, and / or a puncture needle with a guide sheath. The treatment tool opening 38 is also used as a suction port for sucking blood, internal waste, and the like, and as a delivery port for delivering fluids.

[0040] A light-transmitting hood 40 (in the present embodiment, as an example, a transparent hood) is removably attached to the distal end portion 36. The hood 40 is an example of an "instrument" and a "hood" according to the present disclosure. The hood 40 is an instrument having optical properties that cause halation due to light 34. Although the hood 40 is illustrated here as an example, in addition to the hood 40, an instrument such as an external treatment tool (for example, an instrument having optical properties that cause halation due to light 34) can be removably attached to the distal end portion 36.

[0041] The hood 40 has a tubular shape (here, a cylindrical shape is used as an example). The hood 40 is used to maintain an appropriate distance between the objective lens 30A, the irradiation ports 32A and 32B, and the treatment tool opening 38 and the subject (e.g., an observation target within the intestinal wall 26) and to ensure a good field of view. The hood 40 may also be used to support the operation of a treatment tool. The hood 40 has a circular opening 40A in a front view. When the hood 40 is attached to the distal end portion 36, the objective lens 30A, the irradiation ports 32A and 32B, and the treatment tool opening 38 face the opening 40A. Therefore, the opening 40A falls within the angle of view of the camera 30, and the light 34 emitted from the irradiation ports 32A and 32B is irradiated onto an area including the intestinal wall 26 through the opening 40A, and the treatment tool from the treatment tool opening 38 is inserted and removed through the opening 40A. Furthermore, when a hood 40 is attached to the tip portion 36, light 34 is irradiated onto the area including the intestinal wall 26 from the irradiation ports 32A and 32B via the hood 40. In the example shown in Fig. 1, light 34 is irradiated from the opening 40A, but because the hood 40 is translucent, the light 34 irradiated from the irradiation ports 32A and 32B passes through the wall surface of the hood 40. Furthermore, subject light also passes through the wall surface of the hood 40 and enters the objective lens 30A.

[0042] The control device 18 controls the entire endoscope system 10. For example, the control device 18 is used for the endoscope 14, the display device 16, the light source device 20, etc., and the endoscope 14, the display device 16, the light source device 20, etc. are controlled by the control device 18.

[0043] The display device 16 displays various information on a screen 42 under the control of the control device 18. Examples of the display device 16 include a liquid crystal display and an EL display. Alternatively, a tablet terminal with a display may be used instead of the display device 16 or together with the display device 16.

[0044] The light source device 20 generates light 34 under the control of the control device 18 and supplies the generated light 34 to the illumination device 32. The illumination device 32 has a built-in light guide (not shown), and the light 34 supplied from the light source device 20 passes through the light guide and is emitted from irradiation ports 32A and 32B.

[0045] Under the control of the control device 18, the camera 30 captures an image of an object to be observed inside the intestinal wall 26 to generate a moving image 44. The moving image 44 is composed of a plurality of frames 46 arranged in a time series. The control device 18 acquires the moving image 44 generated by the camera 30 and performs various image processing on the acquired moving image 44. The control device 18 then outputs various information, including the moving image 44 that has undergone image processing, to the display device 16. As a result, the various information, including the moving image 44, is displayed on the screen 42 of the display device 16. The screen 42 displays a plurality of frames 46 arranged in a time series at a default frame rate. Examples of the default frame rate include 15 frames / second, 30 frames / second, and 60 frames / second.

[0046] In the example shown in FIG. 1, a moving image 44 obtained by imaging by the camera 30 with the hood 40 attached to the distal end portion 36 is displayed on the screen 42. The hood 40, including the opening 40A, is included in the angle of view of the camera 30. Therefore, the moving image 44 captures the hood 40 together with the opening 40A, in addition to the intestinal wall 26 (in the example shown in FIG. 1, the intestinal wall 26 including the lesion 48). The moving image 44 displayed on the screen 42 also captures the intestinal wall 26 through the opening 40A. The moving image 44 displayed on the screen 42 also captures the intestinal wall 26 through the wall surface of the hood 40. Therefore, when the hood 40 is attached to the distal end portion 36, the doctor 12 can observe the intestinal wall 26 through the screen 42 from both inside and outside the opening 40A.

[0047] In this embodiment, the endoscopic system 10 is an example of an "endoscopic system" according to the present disclosure. In this embodiment, the endoscope 14 is an example of an "endoscope" according to the present disclosure. In this embodiment, the control device 18 is an example of a "control device" according to the present disclosure. In this embodiment, the light 34 is an example of "light" according to the present disclosure. In this embodiment, the tip portion 36 is an example of a "tip portion" according to the present disclosure. In this embodiment, the camera 30 is an example of a "camera" according to the present disclosure. In this embodiment, the hood 40 is an example of a "hood" according to the present disclosure. In this embodiment, the opening 40A is an example of an "opening" according to the present disclosure. In this embodiment, the frame 46 is an example of an "image" according to the present disclosure.

[0048] Fig. 2 shows an example of the hardware configuration of the electrical system of the endoscope system 10. As shown in Fig. 2, the control device 18 includes a computer 50 and an external I / F 52. The computer 50 includes a processor 54, a memory 56, and a storage 58. The processor 54, the memory 56, the storage 58, and the external I / F 52 are connected to a bus 60. The processor 54 controls the entire control device 18. The memory 56 and the storage 58 are used by the processor 54.

[0049] The external I / F 52 controls the exchange of various information between the processor 54 and one or more devices that exist outside the control device 18 (hereinafter also referred to as "external devices").

[0050] A reception device 62 is connected to the external I / F 52 as one of the external devices. Examples of the reception device 62 include a foot switch, a microphone, a touch panel, a keyboard, and / or a mouse. The processor 54 acquires instructions received by the reception device 62 via the external I / F 52 and executes processing according to the acquired instructions.

[0051] The display device 16 is connected as one of the external devices to the external I / F 52. The processor 54 controls the display device 16 via the external I / F 52, thereby causing the display device 16 to display various information (for example, a moving image 44 that has been subjected to various image processing).

[0052] The light source device 20 is connected to the external I / F 52 as one of the external devices, and the external I / F 52 controls the exchange of various information between the light source device 20 and the processor 54. Under the control of the processor 54, the light source device 20 generates light 34 (see FIG. 1) and supplies it to the illumination device 32. The illumination device 32 irradiates the light 34 (see FIG. 1) supplied from the light source device 20.

[0053] The camera 30 is connected to the external I / F 52 as one of the external devices. The external I / F 52 controls the exchange of various information between the camera 30 and the processor 54. The camera 30 has an image sensor 64 and a variable magnification optical system 66. In other words, the endoscope 14 can be said to be a variable magnification endoscope.

[0054] Examples of the image sensor 64 include a CMOS image sensor or a CCD image sensor. The variable magnification optical system 66 is an optical system that realizes so-called optical zoom and operates under the control of the processor 54. Operation of the variable magnification optical system 66 changes the magnification of the moving image 44 (see FIG. 1). That is, frames 46 (see FIG. 1) are optically zoomed in or out. The image sensor 64 receives subject light incident on the variable magnification optical system 66 and photoelectrically converts the received subject light to generate an electrical signal corresponding to the subject light. The camera 30 includes a signal processing circuit (not shown) connected to the image sensor 64. The signal processing circuit acquires electrical signals from the image sensor 64 and performs various signal processing, including A / D conversion, on the acquired electrical signals to generate frames 46 (see FIG. 1) at a predetermined frame rate (e.g., a predetermined frame rate such as 15 frames / second, 30 frames / second, or 60 frames / second).

[0055] Each time a frame 46 (see FIG. 1) is generated by the camera 30 at a predetermined frame rate, the generated frame 46 is acquired by the processor 54. That is, the processor 54 acquires from the camera 30 a moving image 44 (see FIG. 1) that is configured to include a plurality of frames 46 in time series.

[0056] Although the embodiment described above is an example in which frame 46 is generated by a signal processing circuit of camera 30 performing various signal processes on an electrical signal corresponding to the subject light, this is merely one example. For example, control device 18 may have a signal processing circuit, and the signal processing circuit of control device 18 may generate frame 46 by acquiring an electrical signal corresponding to the subject light from image sensor 64 and performing various signal processes on it.

[0057] In this embodiment, the computer 50 is an example of a "computer" according to the present disclosure. Also, in this embodiment, the processor 54 is an example of a "processor" according to the present disclosure.

[0058] Meanwhile, each of the multiple frames 46 included in the video 44 acquired by the processor 54 from the camera 30 is displayed on a screen 42 (see FIGS. 1 and 3) of the display device 16. For example, if a hood 40 is attached to the tip end 36, the hood 40 will be reflected on the screen 42, as shown in FIG. 3 as an example.

[0059] In conventional endoscope systems, when the hood 40 appears in the frame 46, brightness control is activated to control the brightness in the frame 46 by controlling the exposure time, controlling the amount of light, and / or performing image processing on the frame 46 so that the brightness of the hood 40 appearing in the frame 46 is appropriate, rather than focusing on the intestinal wall 26 as an object of observation. In this case, for example, when the hood 40 is attached to the tip of the endoscope, the frame 46 becomes bright due to the influence of halation that occurs in the hood 40 (i.e., halation caused by the light 34), and this brightness control causes the frame 46 to become dark overall.

[0060] For example, in this case, an object to be observed that is located further away than the hood 40 (such as the inner area of ​​the opening 40A) will not be properly lit, which will hinder the doctor 12 from making an observation. Naturally, the brightness of the frame 46 will differ between when the hood 40 is attached to the distal end portion 36 and when it is not. For example, the frame 46 obtained when the hood 40 is attached to the distal end portion 36 will be darker overall than the frame 46 obtained when the hood 40 is not attached to the distal end portion 36. This is because the hood 40 blocks part of the subject light, even if the hood 40 is translucent.

[0061] The doctor 12 adjusts the intensity of the light 34 and switches the brightness control operation mode by giving instructions to the endoscope system 10 so that the brightness of the subject is appropriate when the hood 40 is attached and when it is not attached to the distal end. However, giving instructions to the endoscope system 10 to adjust the brightness of the subject is time-consuming for the doctor 12.

[0062] In view of these circumstances, in this embodiment, as an example, brightness control processing is performed by the processor 54 of the control device 18, as shown in Fig. 4. Note that Fig. 4 shows an example of the operation mode of the control device 18 for realizing the brightness control processing.

[0063] A brightness control program 68 is stored in the storage 58. In this embodiment, the brightness control program 68 is an example of a "program" according to the present disclosure. The processor 54 reads the brightness control program 68 from the storage 58 and executes the read brightness control program 68 on the memory 56 to perform brightness control processing. The brightness control processing is realized by the processor 54 operating as the determination unit 54A and the control unit 54B in accordance with the brightness control program 68 executed on the memory 56.

[0064] In the brightness control process, whether or not the hood 40 is attached to the tip end portion 36 is determined based on the feature amount of the frame 46 and conditions given to the feature amount of the frame 46 (for example, a time condition and / or a luminance distribution condition included in the frame 46, which is an example of the feature amount of the frame 46). Whether or not the hood 40 is attached to the tip end portion 36 is determined by the determination unit 54A, as shown in FIG. 5. An example of the processing content of the determination unit 54A will be described below with reference to FIG. 5.

[0065] FIG. 5 illustrates an example of the processing performed by the determination unit 54A. As illustrated in FIG. 5, the determination unit 54A determines whether the hood 40 is attached to the distal end portion 36 based on frames 46 obtained by capturing images of the inside of the large intestine 24 using the camera 30 while the inside of the large intestine 24 is illuminated with light 34. To achieve this, the determination unit 54A first extracts a central portion 46A from each of a plurality of time-series frames 46 included in the moving image 44. The central portion 46A is extracted because the central portion 46A is the portion that attracts the most attention from the doctor 12 and because the characteristics of the hood 40 (e.g., the opening 40A) are more likely to be apparent in the central portion 46A than in portions other than the central portion 46A of the frame 46. In the example illustrated in FIG. 5, a rectangular region having a center that coincides with the center of the frame 46 is illustrated as an example of the central portion 46A. The central portion 46A need not be a rectangular region, but may be a region of a geometric shape other than a rectangle, such as a circular region.

[0066] The determination unit 54A extracts the edge 46A1 that appears in the central portion 46A from the central portion 46A. For example, the edge 46A1 is an image region of high-frequency components that has been determined in advance as a high-frequency component that represents the edge 46A1 through testing using an actual device and / or computer simulation, etc.

[0067] The determination unit 54A executes a circle extraction process. The circle extraction process is a process of extracting a circle 46A2 from the edge 46A1 by performing a Hough transform on the edge 46A1 extracted from the central portion 46A. The reason for extracting the circle 46A2 is that the opening 40A is circular. That is, if the shape of the opening 40A (i.e., a circle) is included in the central portion 46A, it can be said that the hood 40 is attached to the tip portion 36.

[0068] The determination unit 54A determines whether the circle 46A2 is extracted from the edge 46A1. If the determination unit 54A determines that the circle 46A2 is not extracted from the edge 46A1, the determination unit 54A determines that the hood 40 is not attached to the tip end portion 36.

[0069] On the other hand, when the determination unit 54A determines that the circle 46A2 has been extracted from the edge 46A1, the determination unit 54A determines whether the extraction of the circle 46A2 is continuing across multiple frames 46. Here, when the determination unit 54A determines that the extraction of the circle 46A2 is not continuing across multiple frames 46 (in other words, when the extraction of the circle 46A2 has been interrupted), the determination unit 54A determines that the hood 40 is not attached to the tip end portion 36. When the determination unit 54A determines that the extraction of the circle 46A2 is continuing across multiple frames 46, the determination unit 54A subsequently determines whether a circle extraction continuation condition is satisfied.

[0070] The circle extraction continuation condition refers to a condition that the circle 46A2 is continuously extracted for a predetermined time (e.g., a time specified in advance within a range of several seconds to several tens of seconds) or more. The concept of the predetermined time includes a predetermined number of frames. One example of the predetermined number of frames is a predetermined number of frames (e.g., 90 frames) within a range of several tens of frames to several hundred frames. If the circle extraction continuation condition is satisfied, the determination unit 54A determines that the hood 40 is attached to the tip portion 36. If the circle extraction continuation condition is not satisfied, the determination unit 54A executes the above-described process (i.e., the series of processes starting from the process of extracting the center portion 46A from the frame 46) again for the next frame 46.

[0071] In the brightness control process, the brightness control for the frame 46 differs depending on whether the determination unit 54A determines that the hood 40 is not attached to the tip end portion 36 or whether the determination unit 54A determines that the hood 40 is attached to the tip end portion 36. However, these controls have in common the fact that they are controls that bring the brightness of the frame 46 closer to a target brightness. The brightness control for the frame 46 is realized by the control unit 54B, as shown in FIG. 6. An example of the processing content of the control unit 54B will be described below with reference to FIG. 6.

[0072] FIG. 6 shows an example of the processing content of the control unit 54B. As shown in FIG. 6, the control unit 54B performs brightness control, which is control for bringing the brightness of the frame 46 closer to a target brightness. The brightness control is categorized into a control when the hood is not attached and a control when the hood is attached. The control unit 54B selectively performs the control when the hood is not attached and the control when the hood is attached. The control when the hood is not attached is a control for controlling the brightness of the frame 46 when the determination unit 54A determines that the hood 40 is not attached to the tip end portion 36. The control when the hood is attached is a control different from the control when the hood is not attached. The control when the hood is not attached and the control when the hood is attached are performed based on the result of comparing the photometric value of the frame 46 with the target photometric value. The degree to which the brightness of a frame 46 obtained temporally after the frame 46 from which the photometric value is obtained (for example, a frame 46 obtained a predetermined number of frames within a range of one frame to several hundred frames after the frame 46 from which the photometric value is obtained) is increased is greater in the hooded control than in the hooded control. Here, the frame 46 from which the photometric value is obtained is an example of a "first image" according to the present disclosure, and the frame 46 obtained temporally after the frame 46 from which the photometric value is obtained is an example of a "second image" according to the present disclosure. Hereinafter, for convenience of explanation, the former frame 46 will be referred to as the first frame, and the latter frame 46 will be referred to as the second frame. Note that in this embodiment, the hooded control and the hooded control are examples of "brightness control" according to the present disclosure. Also, in this embodiment, the hooded control is an example of "first brightness control" according to the present disclosure. Also, in this embodiment, the hooded control is an example of "second brightness control" according to the present disclosure.

[0073] First, the control when the hood is not attached will be described. The control when the hood is not attached is a control that sets the brightness of a frame 46 (e.g., a second frame) obtained by imaging the inside of the large intestine 24 with the camera 30 irradiating the inside of the large intestine 24 with light 34 when the hood 40 is not attached to the distal end portion 36 to a first brightness B1. The first brightness B1 is the ideal brightness when the hood 40 is not attached to the distal end portion 36. The first brightness B1 is determined based on the brightness of the frame 46. Here, the brightness of the frame 46 is an example of a "photometric value" according to the present disclosure.

[0074] In the hood-unattached control, the control unit 54B acquires, from a frame 46 (e.g., a first frame), the luminance of each pixel in a frame 46 obtained by the camera 30 capturing an image of the inside of the large intestine 24 while the hood 40 is not attached to the distal end portion 36 and the inside of the large intestine 24 is illuminated with light 34. The control unit 54B creates a luminance histogram 73, which is a histogram of the luminance acquired from the frame 46. The control unit 54B then calculates an average luminance 70 and a peak luminance 72 based on the luminance histogram 73. The average luminance 70 is a luminance equivalent to the luminance obtained by performing so-called average photometry on the frame 46 (e.g., the first frame). The peak luminance 72 is a luminance equivalent to the luminance obtained by performing so-called peak photometry on the frame 46 (e.g., the first frame). In this embodiment, the average photometry and the peak photometry are examples of the "multiple photometric methods" according to the present disclosure. In this embodiment, the average luminance 70 is an example of the "average photometric value" according to the present disclosure. In this embodiment, the average luminance 70 and the peak luminance 72 are examples of the "plurality of photometric values" according to the present disclosure.

[0075] 6, the control unit 54B calculates the average brightness of the entire frame 46 from the brightness histogram 73 as the average brightness 70. The average brightness 70 is a brightness that represents the overall brightness of the frame 46. Although the average brightness 70 is exemplified here, this is merely an example, and any brightness that represents the overall brightness of the frame 46, such as the median brightness of all pixels in the frame 46, may be used.

[0076] 6, control unit 54B calculates the average brightness of a plurality of pixels representing hood 40 appearing in frame 46 from high brightness region 73A of brightness histogram 73 as peak brightness 72. An example of peak brightness 72 is a brightness that allows identification of the presence of hood 40 in frame 46, assuming that hood 40 appears in frame 46.

[0077] Here, high-brightness region 73A refers to a brightness region (e.g., a region in the top 10% of brightness) in brightness histogram 73 where the presence of hood 40 can be identified if hood 40 is captured in frame 46. High-brightness region 73A is determined in advance as a brightness region where the presence of hood 40 can be identified by testing using an actual device and / or computer simulation, etc.

[0078] Note that, because hood 40 is transparent, an example is given here in which peak luminance 72 is calculated from high luminance region 73A, but the present disclosure is not limited to this. For example, if hood 40 is a color other than transparent, the luminance region in luminance histogram 73 from which the presence of hood 40 can be identified is the luminance region corresponding to the color of hood 40. Therefore, the average luminance of multiple pixels that exhibit the characteristics of hood 40 captured in frame 46 can be calculated from the luminance region in luminance histogram 73 corresponding to the color of hood 40.

[0079] The control unit 54B controls the light source device 20 based on the average luminance 70 and the peak luminance 72 to set the brightness of the frame 46 (e.g., the second frame) to a first brightness B1. The first brightness B1 is determined by adjusting the average luminance 70 with respect to a target average luminance 74 and adjusting the peak luminance 72 with respect to a target peak luminance 76. Here, adjusting the average luminance 70 with respect to the target average luminance 74 means adjusting the average luminance 70 to approach the target average luminance 74 (in other words, adjusting to the target average luminance 74), and adjusting the peak luminance 72 with respect to the target peak luminance 76 means adjusting the peak luminance 72 to approach the target peak luminance 76 (in other words, adjusting to the target peak luminance 76). For example, the control unit 54B controls the light source device 20 to approach the average luminance 70 with respect to the target average luminance 74 and the peak luminance 72 with respect to the target peak luminance 76, thereby setting the brightness of the frame 46 to the first brightness B1.

[0080] For example, the ratio of the degree to which the average brightness 70 approaches the target average brightness 74 (in other words, the degree to which the average brightness 70 is adjusted to the target average brightness 74) to the degree to which the peak brightness 72 approaches the target peak brightness 76 (in other words, the degree to which the peak brightness 72 is adjusted to the target peak brightness 76) is 1:1. 1:1 means that the degree to which the average brightness 70 approaches the target average brightness 74 and the degree to which the peak brightness 72 approaches the target peak brightness 76 are equal (i.e., the current average brightness 70 is increased by 50% and the peak brightness 72 is increased by 50%). In this way, by setting the ratio of the degree to which the average brightness 70 approaches the target average brightness 74 to the degree to which the peak brightness 72 approaches the target peak brightness 76 to 1:1, the shape of the brightness histogram 73 is maintained before and after the brightness adjustment of frame 46. Note that 1:1 is merely an example, and may be 2:1, 1:0, or 0:0 depending on the preference of the doctor 12 or the like.

[0081] The target average luminance 74 may be a luminance specified in advance by the doctor 12 or the like, or may be a luminance determined in advance as an ideal luminance to which the average luminance 70 should be matched through tests using an actual device and / or computer simulations, etc. The same applies to the target peak luminance 76. The target average luminance 74 and / or the target peak luminance 76 may be fixed values, or may be variable values ​​that are changed according to given instructions and / or various conditions.

[0082] Next, the control when the hood is attached will be described. The control when the hood is attached is a control that sets the brightness of a frame 46 (e.g., a second frame) obtained by the camera 30 capturing an image of the inside of the large intestine 24 with the hood 40 attached to the distal end portion 36 and the light 34 irradiating the inside of the large intestine 24 to a second brightness B2. The second brightness B2 is the ideal brightness when the hood 40 is attached to the distal end portion 36. The second brightness B2 is equal to or greater than the first brightness B1. The second brightness B2 is determined based on the luminance of the frame 46.

[0083] Furthermore, the control when the hood is attached is also a control for increasing the average brightness 78 compared to the average brightness 70 controlled by the control when the hood is not attached.

[0084] In the hood-attached control, the control unit 54B acquires, from a frame 46 (e.g., a first frame) obtained by imaging the inside of the large intestine 24 with the camera 30 while the hood 40 is attached to the distal end portion 36 and the inside of the large intestine 24 is illuminated with light 34, the brightness of each pixel in the frame 46. The control unit 54B creates a brightness histogram 81, which is a histogram of the brightness acquired from the frame 46. The control unit 54B then calculates an average brightness 78 and a peak brightness 80 based on the brightness histogram 81. The average brightness 78 has the same meaning as the average brightness 70, and the peak brightness 80 has the same meaning as the peak brightness 72. The calculation of the average brightness 78 and the peak brightness 80 is performed in a manner similar to the calculation of the average brightness 70 and the peak brightness 72. In the example shown in FIG. 6 , the high brightness region 81A in the brightness histogram 81 has the same meaning as the high brightness region 73A in the brightness histogram 73. In this embodiment, the average luminance 78 is an example of an "average photometric value" according to the present disclosure. Also, in this embodiment, the average luminance 78 and the peak luminance 80 are examples of "multiple photometric values" according to the present disclosure.

[0085] The control unit 54B performs image processing for brightness adjustment (hereinafter also simply referred to as "image processing") based on the average luminance 78 and the peak luminance 80, thereby setting the brightness of the frame 46 (e.g., the second frame) to a second brightness B2. An example of the image processing is digital gain adjustment. Specific examples of image processing, in addition to digital gain adjustment, include gamma correction. The second brightness B2 is determined by adjusting the average luminance 78 relative to a target average luminance 82 and adjusting the peak luminance 80 relative to a target peak luminance 84. Here, adjusting the average luminance 78 relative to the target average luminance 82 means adjusting the average luminance 78 to approach the target average luminance 82 (in other words, adjusting the average luminance 78 to the target average luminance 82), and adjusting the peak luminance 80 relative to the target peak luminance 84 means adjusting the peak luminance 80 to approach the target peak luminance 84 (in other words, adjusting the peak luminance 80 to the target peak luminance 84).

[0086] For example, control unit 54B performs image processing to bring average luminance 78 closer to target average luminance 82 and bring peak luminance 80 closer to target peak luminance 84. Control unit 54B also performs image processing so that the ratio of the degree to which average luminance 78 approaches target average luminance 82 relative to the degree to which peak luminance 80 approaches target peak luminance 84 is greater than the ratio of the degree to which average luminance 70 approaches target average luminance 74 relative to the degree to which peak luminance 72 approaches target peak luminance 76. Control unit 54B also performs image processing so that average luminance 78 is higher than the average luminance 70 adjusted in the control performed when the hood is not attached. The image processing performed here is processing to adjust the luminance of each pixel in frame 46.

[0087] The degree to which the average brightness 78 approaches the target average brightness 82 is greater than the degree to which the peak brightness 80 approaches the target peak brightness 84. For example, the ratio of the degree to which the average brightness 78 approaches the target average brightness 82 to the degree to which the peak brightness 80 approaches the target peak brightness 84 is 3:1. 3:1 means that the degree to which the average brightness 78 approaches the target average brightness 82 is three times the degree to which the peak brightness 80 approaches the target peak brightness 84.

[0088] In this way, by setting the ratio of the degree to which the average luminance 78 approaches the target average luminance 82 to the degree to which the peak luminance 80 approaches the target peak luminance 84 to 3:1, the difference between the high luminance region 81A and the other regions in the luminance histogram 81 after the brightness adjustment of the frame 46 is smaller than the difference between the high luminance region 81A and the other regions in the luminance histogram 81 before the brightness adjustment of the frame 46. Note that 3:1 is merely an example, and the ratio may be 2:1 or 1:0 depending on the preference of the physician 12 or the like. In either case, the degree to which the average luminance 70 and the peak luminance 72 are used in the hood-unmounted control differs from the degree to which the average luminance 78 and the peak luminance 80 are used in the hood-mounted control. For example, the degree to which the average luminance 78 is used in the hood-mounted control is greater than the degree to which the average luminance 70 is used in the hood-unmounted control.

[0089] The target average luminance 82 may be a luminance specified in advance by the doctor 12 or the like, or may be a luminance determined in advance as an ideal luminance to which the average luminance 78 should be matched through tests using an actual device and / or computer simulations or the like. The same applies to the target peak luminance 84. The target average luminance 82 and / or the target peak luminance 84 may be fixed values, or may be variable values ​​that are changed according to given instructions and / or various conditions.

[0090] In the present embodiment, the brightness of frame 46 (e.g., the second frame) is increased by increasing the degree of influence of image processing for brightness adjustment on frame 46 (e.g., the second frame) in the hooded control compared to the hooded control (e.g., increasing the digital gain). However, this is merely an example. For example, to increase the brightness of frame 46 (e.g., the second frame), the exposure time of the endoscope 14 (i.e., the exposure time of the image sensor 64) when imaging to obtain frame 46 (e.g., the second frame) may be increased, or the amount of light 34 when imaging to obtain frame 46 (e.g., the second frame) may be increased. Furthermore, adjustment of the exposure time, adjustment of the amount of light 34, and image processing (e.g., adjustment of the digital gain) may be performed sequentially. For example, the exposure time may be allowed up to a certain time (for example, 1 / 30 seconds or 1 / 60 seconds), and if an exposure time longer than the certain time is required, the effect of image blurring becomes significant, so adjustment of the amount of light 34 may be performed. Since adjustment of the digital gain may increase noise, adjustment of the digital gain may be used when adjustment of the exposure time and adjustment of the amount of light 34 does not sufficiently increase the brightness of the frame 46.

[0091] Next, the functions and effects of the parts of the endoscope system 10 according to this embodiment that are related to the present disclosure will be described.

[0092] 7A and 7B show an example of the flow of brightness control processing executed by processor 54. The flowcharts shown in Figures 7A and 7B are an example of an "operation method of a control device" according to the present disclosure. For ease of explanation, the following description will be given on the assumption that the brightness control processing is executed by processor 54 when light 34 is irradiated into large intestine 24 with tip portion 36 inserted into large intestine 24.

[0093] In the brightness control process shown in Fig. 7A, in step ST10, the determination unit 54A determines whether or not one frame of image has been captured by the camera 30. If one frame of image has not been captured by the camera 30 in step ST10, the determination is negative, and the brightness control process proceeds to step ST38 shown in Fig. 7B. If one frame of image has been captured by the camera 30 in step ST10, the determination is positive, and the brightness control process proceeds to step ST12.

[0094] In step ST12, the determination unit 54A acquires a frame 46 from the camera 30 (see FIG. 5). After the process of step ST12 is executed, the brightness control process proceeds to step ST14.

[0095] In step ST14, the determination unit 54A extracts the central portion 46A from the frame 46 (see FIG. 5). After the process of step ST14 is executed, the brightness control process proceeds to step ST16.

[0096] In step ST16, the determination unit 54A extracts an edge 46A1 from the central portion 46A (see FIG. 5). After the process of step ST16 is executed, the brightness control process proceeds to step ST18.

[0097] In step ST18, the determination unit 54A executes a circular shape extraction process using the edge 46A1 (see FIG. 5). After the process of step ST18 is executed, the brightness control process proceeds to step ST20.

[0098] In step ST20, the determination unit 54A determines whether or not the circle 46A2 has been extracted by the execution of the circle extraction process (see FIG. 5). If the circle 46A2 has not been extracted by the execution of the circle extraction process in step ST20, the determination is negative, and the brightness control process proceeds to step ST28. If the circle 46A2 has been extracted by the execution of the circle extraction process in step ST20, the determination is positive, and the brightness control process proceeds to step ST22.

[0099] In step ST22, the determination unit 54A determines whether extraction of the circle 46A2 is continuing across multiple frames 46 in the time series (see FIG. 5). If extraction of the circle 46A2 is not continuing across multiple frames 46 in the time series in step ST22, the determination is negative, and the brightness control process proceeds to step ST28. If extraction of the circle 46A2 is continuing across multiple frames 46 in the time series in step ST22, the determination is positive, and the brightness control process proceeds to step ST24.

[0100] In step ST24, the determination unit 54A determines whether the circular extraction continuation condition is satisfied (see FIG. 5). If the circular extraction continuation condition is not satisfied in step ST24, the determination is negative, and the brightness control process proceeds to step ST10. If the circular extraction continuation condition is satisfied in step ST24, the determination is positive, and the brightness control process proceeds to step ST26.

[0101] In step ST26, the determination unit 54A determines that the hood 40 is attached to the tip end portion 36 (see FIG. 5). After the process of step ST26 is executed, the brightness control process proceeds to step ST30 shown in FIG. 7B.

[0102] In step ST28, the determination unit 54A determines that the hood 40 is not attached to the tip end portion 36 (see FIG. 5). After the process of step ST28 is executed, the brightness control process proceeds to step ST34 shown in FIG. 7B.

[0103] 7B, the control unit 54B calculates the average luminance 78 and the peak luminance 80 from the frame 46 acquired in step ST12 (see FIG. 6). After the process of step ST30 is executed, the brightness control process proceeds to step ST32.

[0104] In step ST32, the control unit 54B performs image processing on the frame 46 acquired in step ST12 based on the average brightness 78 and the peak brightness 80, thereby setting the brightness of the frame 46 to the second brightness B2 (see FIG. 6). After the processing of step ST32 is performed, the brightness control processing proceeds to step ST38.

[0105] In step ST34, the control unit 54B calculates the average luminance 70 and the peak luminance 72 from the frame 46 acquired in step ST12 (see FIG. 6). After the process of step ST34 is executed, the brightness control process proceeds to step ST36.

[0106] In step ST36, the control unit 54B controls the light source device 20 to set the brightness of the frame 46 acquired in step ST12 to the first brightness B1 (see FIG. 6).

[0107] In step ST36, an example has been given in which the brightness of frame 46 is set to first brightness B1 by controlling light source device 20, but this is merely one example, and the brightness of frame 46 may be set to first brightness B1 by performing image processing on frame 46 (for example, processing to adjust the brightness of each pixel of frame 46). After the processing of step ST36 is performed, the brightness control processing proceeds to step ST38.

[0108] In step ST38, the control unit 54B determines whether a condition for terminating the brightness control process is satisfied. One example of the condition for terminating the brightness control process is that an instruction to terminate the brightness control process has been given to the endoscope system 10 (for example, that the instruction to terminate the brightness control process has been accepted by the accepting device 62).

[0109] In step ST38, if the condition for terminating the brightness control process is not satisfied, the determination is negative, and the brightness control process proceeds to step ST10 shown in Fig. 7A. In step ST38, if the condition for terminating the brightness control process is satisfied, the determination is positive, and the brightness control process ends.

[0110] As described above, in the endoscope system 10 according to the present embodiment, brightness control for bringing the brightness of the frame 46 closer to the target brightness is performed using control when the hood 40 is not attached and control when the hood is attached. The control when the hood is not attached is performed when the hood 40 is not attached to the distal end portion 36, and the control when the hood is attached is performed when the hood 40 is attached to the distal end portion 36. The control when the hood is attached is different from the control when the hood is not attached. Therefore, it is possible to prevent the frame 46 (e.g., the second frame) from becoming dark overall due to brightness control (e.g., control when the hood is not attached) that is activated in response to the frame 46 (e.g., the first frame) becoming brighter due to the influence of halation that occurs when the hood 40 is attached to the distal end portion 36.

[0111] Furthermore, in the endoscope system 10 according to this embodiment, the hood-unattached control is performed based on the result of comparing the photometric values ​​(e.g., average luminance 70 and peak luminance 72) of frame 46 (e.g., the first frame) with the target photometric values ​​(e.g., the target average luminance 74 and target peak luminance 76), and the hood-attached control is performed based on the result of comparing the photometric values ​​(e.g., average luminance 78 and peak luminance 80) of frame 46 or the hood-attached control (e.g., the first frame) with the target photometric values ​​(e.g., the target average luminance 82 and target peak luminance 84). The degree of brightness enhancement of frame 46 (e.g., the second frame) is greater in the hood-attached control than in the hood-unattached control. Even when the hood 40 is attached to the distal end 36, if the same control as the hood-unattached control is performed, the brightness of frame 46 will be reduced due to the influence of halation caused by the hood 40, resulting in the overall darkening of frame 46. However, in this embodiment, by increasing the degree to which the brightness of frame 46 (e.g., the second frame) is increased by the control when the hood is attached greater than the degree to which the brightness of frame 46 (e.g., the second frame) is increased by the control when the hood is not attached, it is possible to obtain a frame 46 that is brighter overall than frame 46 (e.g., the second frame) whose overall brightness has been adjusted by brightness control (e.g., control when the hood is not attached) that operates due to the frame 46 (e.g., the first frame) becoming brighter due to the effect of halation that occurs when the hood 40 is attached to the tip 36.

[0112] Furthermore, in the endoscope system 10 according to the present embodiment, the brightness of the frame 46 (e.g., the second frame) is increased by lengthening the exposure time of the endoscope 14 when imaging is performed to obtain the frame 46 (e.g., the second frame), by increasing the amount of light 34 when imaging is performed to obtain the frame 46 (e.g., the second frame), or by increasing the degree of influence that image processing for brightness adjustment has on the frame 46 (e.g., the second frame) when image processing for brightness adjustment is performed on the frame 46 (e.g., the second frame). This makes it possible to obtain a frame 46 that is brighter overall than the frame 46 whose overall brightness has been adjusted by brightness control (e.g., control when the hood is not attached) that operates due to the brightening of the frame 46 (e.g., the first frame) caused by the influence of halation that occurs when the hood 40 is attached to the distal end portion 36.

[0113] Furthermore, in this embodiment, the ratio of the average brightness 70 to the peak brightness 72 used in the control when the hood is not attached is 1:1, whereas the ratio of the average brightness 78 to the peak brightness 80 used in the control when the hood is attached is 3:1. Thus, the ratio of the average brightness 78 used in the control when the hood is attached is greater than the ratio of the average brightness 70 used in the control when the hood is not attached. This makes it possible to prevent the frame 46 (e.g., the second frame) from becoming dark overall due to brightness control (e.g., the control when the hood is not attached) that is activated due to the frame 46 (e.g., the first frame) becoming brighter due to the influence of halation that occurs when the hood 40 is attached to the tip end portion 36.

[0114] Furthermore, in the endoscope system 10 according to the present embodiment, when the hood 40 is not attached to the distal end portion 36, the brightness of the frame 46 is set to a first brightness B1. On the other hand, when the hood 40 is attached to the distal end portion 36, the brightness of the frame 46 is set to a second brightness B2. The first brightness B1 is the ideal brightness when the hood 40 is not attached to the distal end portion 36, and the second brightness B2 is the ideal brightness when the hood 40 is attached to the distal end portion 36. Therefore, the brightness of the frame 46 can be set to the ideal brightness when the hood 40 is not attached to the distal end portion 36 and when the hood 40 is attached to the distal end portion 36.

[0115] Furthermore, in the endoscope system 10 according to this embodiment, whether or not the hood 40 is attached to the distal end portion 36 is determined based on the circle 46A2 extracted from the frame 46 and the circle extraction continuation condition. That is, whether or not the hood 40 is attached to the distal end portion 36 is determined based on whether or not the extraction of the circle 46A2 from the frame 46 has continued for a predetermined time or more across multiple frames 46. Therefore, whether or not the hood 40 is attached to the distal end portion 36 can be determined without determining whether or not the hood 40 is attached to the distal end portion 36 based on the visual inspection by the doctor 12.

[0116] Furthermore, in the endoscope system 10 according to the present embodiment, the circle 46A2 used to determine whether or not the hood 40 is attached to the distal end portion 36 is extracted from the central portion 46A of the frame 46. Therefore, compared to when the circle 46A2 used to determine whether or not the hood 40 is attached to the distal end portion 36 is extracted from the entire frame 46, the calculation load required to extract the circle 46A2 used to determine whether or not the hood 40 is attached to the distal end portion 36 can be reduced.

[0117] Furthermore, in the endoscope system 10 according to the present embodiment, the second brightness B2 of the frame 46 when the hood 40 is attached to the distal end portion 36 is set to be brighter than or equal to the first brightness B1 of the frame 46 when the hood 40 is not attached to the distal end portion 36. Therefore, even when the hood 40 is attached to the distal end portion 36, the brightness of the frame 46 can be set to be at least as bright as when the hood 40 is not attached to the distal end portion 36.

[0118] Furthermore, in the endoscope system 10 according to the present embodiment, the second brightness B2, which is the brightness of the frame 46 when the hood 40 is attached to the distal end portion 36, is determined based on the luminance of the frame 46 (average luminance 78 and peak luminance 80 in the example shown in FIG. 6). Furthermore, the first brightness B1, which is the brightness of the frame 46 when the hood 40 is not attached to the distal end portion 36, is also determined based on the luminance of the frame 46 (average luminance 70 and peak luminance 72 in the example shown in FIG. 6). Therefore, the first brightness B1 and the second brightness B2 can be set to appropriate brightnesses with higher accuracy than when the first brightness B1 and the second brightness B2 are determined based only on elements that are completely unrelated to the luminance of the frame 46 out of all the elements constituting the frame 46.

[0119] Furthermore, in the endoscope system 10 according to the present embodiment, the control to increase the average brightness 78 to set the brightness of the frame 46 to the second brightness B2 when the hood 40 is attached to the distal end portion 36 is performed to increase the average brightness 78 more than the control to increase the average brightness 70 to set the brightness of the frame 46 to the first brightness B1 when the hood 40 is not attached to the distal end portion 36. In other words, the degree to which the average brightness 78 is increased is greater than the degree to which the average brightness 70 is increased. This allows the brightness of the object of observation observed by the physician 12 through the frame 46 to be adjusted to an appropriate brightness when the hood 40 is attached to the distal end portion 36.

[0120] Furthermore, in the endoscope system 10 according to this embodiment, the second brightness B2 is determined by adjusting the average brightness 78 relative to the target average brightness 82 and by adjusting the peak brightness 80 relative to the target peak brightness 84. Therefore, compared to when the second brightness B2 is determined only by adjusting the average brightness 78 relative to the target average brightness 82 or when the second brightness B2 is determined only by adjusting the peak brightness 80 relative to the target peak brightness 84, the second brightness B2 can be set to an appropriate brightness.

[0121] Furthermore, in the endoscope system 10 according to the present embodiment, the adjustment of the average brightness 78 relative to the target average brightness 82 is performed by adjusting the average brightness 78 to the target average brightness 82. The adjustment of the peak brightness 80 relative to the target peak brightness 84 is performed by adjusting the peak brightness 80 to the target peak brightness 84. Here, the degree to which the average brightness 78 is adjusted to the target average brightness 82 is greater than the degree to which the peak brightness 80 is adjusted to the target peak brightness 84. Therefore, it is possible to make the second brightness B2 an appropriate brightness more easily than when the degree to which the peak brightness 80 is adjusted to the target peak brightness 84 is greater than the degree to which the average brightness 78 is adjusted to the target average brightness 82.

[0122] In the above embodiment, the average luminances 70 and 78 and the peak luminances 72 and 80 are calculated using the luminance of all pixels included in the frame 46. However, the present disclosure is not limited to this. For example, as shown in FIG. 8 , the average luminance 70 and the peak luminance 72 used to obtain the first brightness B1 may be determined based on the luminance of a first range 86 of the frame 46, and the average luminance 78 and the peak luminance 80 used to obtain the second brightness B2 may be determined based on the luminance of a second range 90, which is a range closer to the center of the frame 46 than the first range 86 (in other words, a region more inward in the frame 46 than the first range 86). The second range 90 is an example of a "specific region" according to the present disclosure. The second range 90 is a specific region determined based on the hood 40. In other words, the geometric characteristics of the second range 90 are not constant but are determined based on the shape and / or size of the hood 40, etc. This reduces the computational load required to calculate the average brightness 78 and peak brightness 80 compared to when the second brightness B2 is determined based on the brightness within the same range of frame 46 as when the first brightness B1 is determined (in the example shown in Figure 8, the brightness within the first range 86) (in other words, the computational load required to obtain the second brightness B2 can be reduced).

[0123] In the example shown in Figure 8, a closed area corresponding to the opening 40A shown in the frame 46 is shown as an example of the second range 90, but the present disclosure is not limited to this, and the second range 90 may be any range (i.e., a closed area) closer to the center than the first range 86.

[0124] 8, the first range 86 is illustrated as the entire range of the frame 46, but this is merely an example. For example, the first range 86 may be wider than the second range 90 and narrower than the entire range of the frame 46. In this case, the average luminance 70 and the peak luminance 72 are calculated based on the luminance of each pixel included in a closed region narrower than the entire range of the frame 46.

[0125] In the example shown in FIG. 8, the first range 86 has a plurality of first segmented regions 86A. The plurality of first segmented regions 86A are obtained by segmenting the first range 86. In the example shown in FIG. 8, nine (=3×3) segmented regions are shown as an example of the plurality of first segmented regions 86A. Generally, the closer to the center of the frame 46, the more attention the doctor 12 will pay to the plurality of first segmented regions 86A. Therefore, the closer to the center of the frame 46, the greater the first weights 88 assigned to the plurality of first segmented regions 86A. The first brightness B1 is determined based on the first weights 88 assigned to each of the plurality of first segmented regions 86A and the luminance frequency of the first range 86.

[0126] That is, the luminance of each pixel in the first range 86 is adjusted by multiplying it by the corresponding first weight 88 (i.e., the first weight 88 assigned to the first partitioned region 86A to which the pixel belongs), and the average luminance 70 and the peak luminance 72 are calculated based on the adjusted luminance and the frequency of the adjusted luminance. The average luminance 70 and the peak luminance 72 are obtained for each first partitioned region 86A and have sizes corresponding to the first partitioned region 86A. Then, based on the calculated average luminance 70 and peak luminance 72, image processing is performed on the frame 46 to adjust the luminance of each pixel included in the first range 86, thereby adjusting the brightness of the frame 46. As a result, the brightness of the frame 46 is set to the first brightness B1. In this way, the closer to the center of the frame 46, the more importance is assigned to the first brightness B1. Therefore, when the hood 40 is not attached to the distal end portion 36, the frame 46 can be provided with good visibility of the portion that is of high interest to the doctor 12.

[0127] In the example shown in FIG. 8, the second range 90 has a plurality of second segmented regions 90A. The plurality of second segmented regions 90A are obtained by segmenting the frame 46 using a segmentation method different from that used for the hood-unattached control. The segmentation method is determined according to the instrument attached to the distal end portion 36. In this example, since a hood 40 is attached to the distal end portion 36, the segmentation method is determined according to the hood 40. The plurality of second segmented regions 90A are obtained by segmenting the second range 90 using a segmentation method according to the hood 40. In the example shown in FIG. 8, nine (=3×3) segmented regions are shown as an example of the plurality of second segmented regions 90A. Generally, the closer to the center of the frame 46, the more attention the doctor 12 pays to the second segmented regions 90A. Therefore, the second weights 90A assigned to the plurality of second segmented regions 90A are greater the closer to the center of the frame 46. The second brightness B2 is determined based on the second weights 92 assigned to each of the plurality of second segmented regions 90A and the luminance frequency of the second range 90.

[0128] That is, the luminance of each pixel in the second range 90 is adjusted by multiplying it by the corresponding second weight 92 (i.e., the second weight 92 assigned to the second segmented region 90A to which the pixel belongs), and the average luminance 78 and the peak luminance 80 are calculated based on the adjusted luminance and the frequency of the adjusted luminance. The average luminance 78 and the peak luminance 80 are obtained for each second segmented region 90A and have sizes corresponding to the second segmented region 90A. Then, based on the calculated average luminance 78 and peak luminance 80, image processing is performed on the frame 46 to adjust the luminance of each pixel included in the second range 90, thereby adjusting the brightness of the frame 46. As a result, the brightness of the frame 46 is set to the second brightness B2. In this way, the closer to the center of the frame 46, the higher the importance of the second brightness B2 is determined. Therefore, when the hood 40 is attached to the distal end portion 36, the frame 46 can be provided with good visibility of the portion that is of high interest to the doctor 12.

[0129] 8, the degree to which the second weight 92 increases closer to the center of the frame 46 (e.g., the degree to which the second weight 92 increases closer to the center of the second range 90) is greater than the degree to which the first weight 88 increases closer to the center of the frame 46 (e.g., the degree to which the first weight 88 increases closer to the center of the first range 86). In this way, when the hood 40 is attached to the distal end portion 36, the brightness of the frame 46 is determined with a higher importance closer to the center of the frame 46 than when the hood 40 is not attached to the distal end portion 36. As a result, when the hood 40 is attached to the distal end portion 36, it is possible to provide the doctor 12 with a frame 46 that has good visibility of the portion that is attracting the doctor 12's attention.

[0130] In the example shown in Fig. 8, nine segmented regions are shown as an example of the plurality of first segmented regions 86A, but this is merely an example, and the number of first segmented regions 86A may be less than nine, or may be ten or more. In addition, in the example shown in Fig. 8, nine segmented regions are shown as an example of the plurality of second segmented regions 90A, but this is merely an example, and the number of second segmented regions 90A may be less than nine, or may be ten or more.

[0131] In the example shown in Figure 8, an example is given in which the brightness is adjusted using the first weight 88 and the second weight 92, but this is merely one example, and pixel values ​​of a size determined according to the first partition area 86A or pixel values ​​of a size determined according to the second partition area 90A may also be used.

[0132] Fig. 9 shows an example of image processing performed to increase brightness during hood attachment control. As shown in Fig. 9, during hood attachment control, image processing including digital gain increase is performed on the inner region of the opening 40A shown in the frame 46. This makes it possible to increase the visibility of an area that generally attracts a high level of attention from the doctor 12 (e.g., the inner region of the opening 40A) within the entire frame 46 obtained when the hood 40 is attached to the distal end portion 36.

[0133] Furthermore, the control unit 54B may perform digital gain-up on the inner region of the opening 40A shown in the frame 46, thereby setting the second brightness B2 to a brightness that makes the visibility of the inner region of the opening 40A shown in the frame 46 better than the visibility of the outer region of the opening 40A shown in the frame 46. That is, the control unit 54B may set the second brightness B2 to a brightness that is characterized in that the inner region of the opening 40A shown in the frame 46 is brighter than the outer region of the opening 40A shown in the frame 46. In this way, it is possible to improve the visibility of a region that generally receives more attention from the doctor 12 (e.g., the inner region of the opening 40A) than of a region that generally receives less attention from the doctor 12 (e.g., the outer region of the opening 40A).

[0134] 9, in the hood-attached control, image processing is performed that includes gamma correction that sets the gamma value higher than a reference gamma value (for example, a default gamma value or a gamma value designated by the doctor 12, etc.). This makes it possible to improve the overall brightness of the frame 46 obtained when the hood 40 is attached to the distal end portion 36.

[0135] 10 and 11 show examples of switching modes when switching from the first brightness B1 to the second brightness B2. When switching from the first brightness B1 to the second brightness B2, the brightness of the frame 46 may be changed in stages, as shown in Fig. 10. One example of changing the brightness of the frame 46 in stages is changing the brightness of the frame 46 in multiple stages at regular time intervals and by a regular amount.

[0136] Furthermore, when switching from the first brightness B1 to the second brightness B2, the brightness of the frame 46 may be changed monotonically, as shown in Fig. 11. Examples of monotonically changing the brightness of the frame 46 include a form example in which the brightness of the frame 46 is changed linearly, and a form example in which the brightness of the frame 46 is changed exponentially.

[0137] In the examples shown in Figures 10 and 11, an example of a change from the first brightness B1 to the second brightness B2 is shown, but even when changing from the second brightness B2 to the first brightness B1, the brightness of the frame 46 may be changed in stages or may be changed monotonically.

[0138] A gradual or monotonous change in the brightness of frame 46 (e.g., the second frame) can be achieved by gradually increasing the exposure time of endoscope 14, gradually increasing the amount of light 34, or gradually increasing the degree of influence that image processing has on frame 46 when brightness adjustment image processing is performed on frame 46.

[0139] In this way, when switching from one of the first brightness B1 and the second brightness B2 to the other, by changing the brightness of the frame 46 in stages or by changing the brightness of the frame 46 monotonically, it is possible to reduce the visual discomfort that occurs with the change in brightness of the frame 46 compared to when switching directly from one of the first brightness B1 and the second brightness B2 to the other.

[0140] In the above embodiment, an example was given in which whether or not the hood 40 is attached to the tip end portion 36 is determined based on the circle 46A2, which is an example of a feature amount of the center portion 46A of the frame 46, and the circle extraction continuation condition, but the present disclosure is not limited to this. For example, whether or not the hood 40 is attached to the tip end portion 36 may be determined based on a feature amount related to the luminance in the frame 46 and a condition assigned to the feature amount related to the luminance in the frame 46.

[0141] Here, an example of a feature amount related to luminance in frame 46 is a luminance histogram 94, as shown in Fig. 12. An example of a condition given to the feature amount related to luminance in frame 46 is a time condition that the luminance histogram 94 continues for a predetermined time (for example, a predetermined number of frames) or more.

[0142] The luminance histogram 94 is created by the determination unit 54A based on a group of pixels with no motion. The luminance histogram 94 is a luminance histogram of the group of pixels with no motion. Here, the group of pixels with no motion refers to a group of pixels whose luminance does not change between multiple frames 46 (e.g., between two temporally adjacent frames 46) among multiple time-series frames 46 included in the moving image 44. The luminance histogram 94 has a high-luminance region 94A, which is synonymous with the high-luminance regions 73A and 81A described in the above embodiment. The luminance histogram 94 is characterized in that the frequency of the high-luminance region 94A (i.e., the frequency of luminance at which the characteristics of the hood 40 appearing in the frame 46 can be identified) is equal to or greater than a predetermined frequency. An example of the predetermined frequency is a frequency at which the presence of the hood 40 can be identified, determined in advance through testing using an actual device and / or computer simulation.

[0143] When a luminance histogram 94 of the same shape is continuously created for a predetermined time or more, the determination unit 54A determines that the hood 40 is attached to the tip end portion 36. When a luminance histogram 94 of the same shape is not created, or when the creation of a luminance histogram 94 of the same shape stops before the predetermined time, the determination unit 54A determines that the hood 40 is not attached to the tip end portion 36. By doing so, the same effect as in the above embodiment can be obtained.

[0144] 12 illustrates an example in which it is determined that the hood 40 is attached to the tip end portion 36 when a luminance histogram 94 of the same shape is continuously created for a predetermined time or more, but the present disclosure is not limited to this. For example, as shown in FIG. 13, it may be determined that the hood 40 is attached to the tip end portion 36 when the dissimilarity between the overall luminance 96, the central region luminance 98, and the central surrounding luminance 100 satisfies a dissimilarity condition.

[0145] Here, the overall luminance 96 refers to the representative luminance (here, as an example, the average luminance) of all pixels included in the central portion 46A. The central region luminance 98 refers to the representative luminance (here, as an example, the average luminance) of the central region of the central portion 46A. The central surrounding luminance 100 refers to the representative luminance (here, as an example, the average luminance) of the area around the central region of the central portion 46A.

[0146] The dissimilarity condition includes a first condition and a second condition. The first condition is that the ratio of the central region luminance 98 to the overall luminance 96 is equal to or greater than a threshold value TH1. The second condition is that the ratio of the central luminance 100 to the overall luminance 96 is equal to or less than a threshold value TH2. Here, a ratio is used as an example, but a difference may be used as the dissimilarity instead of a ratio.

[0147] The threshold value TH2 is a value less than the threshold value TH1. An example of the threshold value TH1 is a value determined in advance by tests using an actual device and / or computer simulation as a lower limit of the ratio of the central region luminance 98 to the overall luminance 96 when the hood 40, including the opening 40A, is reflected in the central portion 46A. An example of the threshold value TH2 is a value determined in advance by tests using an actual device and / or computer simulation as an upper limit of the ratio of the central area luminance 100 to the overall luminance 96 when the hood 40, including the opening 40A, is reflected in the central portion 46A.

[0148] The threshold value TH2 may be the same as the threshold value TH1. Furthermore, the threshold value TH1 and / or the threshold value TH2 may be variable values ​​that are changed according to given instructions and / or various conditions.

[0149] When a central portion 46A that satisfies both the first condition and the second condition is continuously obtained for a predetermined time or longer, it is determined that the hood 40 is attached to the tip portion 36. When the first condition and / or the second condition is not satisfied, or when acquisition of a central portion 46A that satisfies both the first condition and the second condition ceases for less than the predetermined time, the determination unit 54A determines that the hood 40 is not attached to the tip portion 36. In this way, the same effect as in the above embodiment can be obtained.

[0150] Here, we have given an example in which overall luminance 96, central region luminance 98, and central surrounding luminance 100 are obtained from central portion 46A, which is a part of frame 46, but this is merely one example, and overall luminance 96, central region luminance 98, and central surrounding luminance 100 may be obtained from the entire frame 46, or overall luminance 96, central region luminance 98, and central surrounding luminance 100 may be obtained from a representative region of frame 46.

[0151] 13 illustrates an example in which it is determined that the hood 40 is attached to the tip end portion 36 when a central portion 46A that satisfies both the first condition and the second condition is obtained continuously for a predetermined time or more, but the present disclosure is not limited to this. For example, as shown in FIG. 14, it may be determined that the hood 40 is attached to the tip end portion 36 when a first brightness histogram 102, a second brightness histogram 104, and a third brightness histogram 106 are obtained continuously for a predetermined time or more (i.e., when the time condition is satisfied).

[0152] The first brightness histogram 102 is a histogram of the brightness of each pixel included in the entire central portion 46A of a frame 46 obtained by imaging the inside of the large intestine 24 with the camera 30 while the hood 40 is attached to the distal end portion 36 and light 34 is irradiating the inside of the large intestine 24. The first brightness histogram 102 has a high brightness region 102A that is synonymous with the high brightness region 94A shown in FIG.

[0153] The second brightness histogram 104 is a histogram of the brightness of each pixel contained in the central region of the central portion 46A of the frame 46 obtained by imaging the inside of the large intestine 24 with the camera 30 while the hood 40 is attached to the tip portion 36 and light 34 is irradiated inside the large intestine 24.

[0154] The third luminance histogram 106 is a histogram of the luminance of each pixel included around the central region of the central portion 46A (in other words, the region other than the central region within the central portion 46A) of the frame 46 obtained by imaging the inside of the large intestine 24 with the camera 30 while the hood 40 is attached to the tip portion 36 and light 34 is irradiated inside the large intestine 24. The third luminance histogram 106 has a high luminance region 106A that is synonymous with the high luminance region 94A shown in FIG.

[0155] The determination unit 54A creates a first luminance histogram 102, a second luminance histogram 104, and a third luminance histogram 106 each time a frame 46 is obtained. Then, the determination unit 54A determines whether the first luminance histogram 102, the second luminance histogram 104, and the third luminance histogram 106 of the same shape have been created continuously for a predetermined time or more. Here, if the first luminance histogram 102, the second luminance histogram 104, and the third luminance histogram 106 of the same shape have been created continuously for a predetermined time or more, the determination unit 54A determines that the hood 40 is attached to the tip end portion 36.

[0156] Furthermore, if the first luminance histogram 102 of the same shape, the second luminance histogram 104 of the same shape, and / or the third luminance histogram 106 of the same shape are not created, or if the creation of the first luminance histogram 102 of the same shape, the second luminance histogram 104 of the same shape, and / or the third luminance histogram 106 of the same shape stops before the predetermined time, the determination unit 54A determines that the hood 40 is not attached to the tip end portion 36. By doing so, the same effect as in the above embodiment can be obtained.

[0157] 14 illustrates an example in which a first luminance histogram 102, a second luminance histogram 104, and a third luminance histogram 106 are created, and whether or not the hood 40 is attached to the tip end portion 36 is determined based on the created first luminance histogram 102, the second luminance histogram 104, and the third luminance histogram 106. However, the present disclosure is not limited to this. For example, whether or not the hood 40 is attached to the tip end portion 36 may be determined using a trained model that can distinguish between a frame 46 obtained when the hood 40 is not attached to the tip end portion 36 and a frame 46 obtained when the hood 40 is attached to the tip end portion 36.

[0158] For example, as shown in FIG. 15, an example of a trained model is a determination model 108. The determination model 108 is obtained by performing machine learning using training data 112 on a model 110 (e.g., a neural network) in the learning stage. The training data 112 includes a plurality of data sets. The data sets are data in which example data 114 and correct answer data 116 are associated with each other.

[0159] The example data 114 includes first example data 114A and second example data 114B. The first example data 114A is an image corresponding to a frame 46 obtained by imaging the inside of the large intestine 24 with the camera 30 when the hood 40 is attached to the tip portion 36 and light 34 is irradiating the inside of the large intestine 24. The second example data 114B is an image corresponding to a frame 46 obtained by imaging the inside of the large intestine 24 with the camera 30 when light 34 is irradiating the inside of the large intestine 24 and hood 40 is not attached to the tip portion 36.

[0160] The correct answer data 116 includes a hood-on label 116A and a hood-off label 116B. The hood-on label 116A is a label that can identify that the hood 40 is attached to the tip portion 36. The hood-off label 116B is a label that can identify that the hood 40 is not attached to the tip portion 36.

[0161] Machine learning using the teacher data 112 configured in this manner is performed on the model 110, whereby the model 110 is optimized and a determination model 108 is generated.

[0162] The determination unit 54A inputs the frame 46 into the determination model 108, causing the determination model 108 to output a determination result 108A. The determination result 108A is information that can identify whether or not the hood 40 is attached to the tip end portion 36. The determination unit 54A determines whether or not the hood 40 is attached to the tip end portion 36 by referring to the determination result 108A. In this way, the same effect as in the above embodiment can be obtained.

[0163] 15 illustrates an example of a form in which whether or not the hood 40 is attached to the distal end portion 36 is identified using the determination model 108, but the present disclosure is not limited to this. For example, when specific information that can identify whether or not the hood 40 is attached to the distal end portion 36 is provided from the outside (for example, the doctor 12), whether or not the hood 40 is attached to the distal end portion 36 may be identified based on the specific information provided from the outside.

[0164] 16 shows an example in which hood attachment information 118 or hood non-attachment information 120 is received by the reception device 62. The hood attachment information 118 is information that can identify that the hood 40 is attached to the tip end portion 36. The hood non-attachment information 120 is information that can identify that the hood 40 is not attached to the tip end portion 35.

[0165] When hood attachment information 118 is received by reception device 62, determination unit 54A determines that hood 40 is attached to tip end portion 36. When hood non-attachment information 120 is received by reception device 62, determination unit 54A determines that hood 40 is not attached to tip end portion 36. By doing so, the same effect as in the above embodiment can be obtained.

[0166] FIG. 17 shows a modified example of the processing performed by the control unit 54B. As shown in FIG. 17, when the determination unit 54A determines that the hood 40 is attached to the tip end portion 36, the control unit 54B performs control to make the difficulty level of the hood non-attachment determination condition higher than the difficulty level of the hood attached determination condition. The hood non-attachment determination condition refers to a condition used when the determination unit 54A determines that the hood 40 is not attached to the tip end portion 36. The hood attached determination condition refers to a condition used when the determination unit 54A determines that the hood 40 is attached to the tip end portion 36. Examples of making the difficulty level of the hood non-attachment determination condition higher than the difficulty level of the hood attached determination condition include tightening the time condition used by the determination unit 54A (i.e., lengthening the above-mentioned predetermined time used by the determination unit 54A or increasing the above-mentioned predetermined number of frames used by the determination unit 54A), increasing the threshold value TH1 shown in FIG. 13, increasing the threshold value TH2 shown in FIG. 13, or increasing the predetermined frequency shown in FIGS. 12 and 14.

[0167] By doing so, it is possible to prevent the occurrence of a situation in which the determination unit 54A erroneously determines that the hood 40 is not attached to the tip end portion 36, even though the hood 40 is actually attached to the tip end portion 36.

[0168] In the above-described examples, whether or not the hood 40 is attached to the distal end portion 36 is determined and the brightness is switched from the first brightness B1 to the second brightness B2 regardless of the type of endoscope 14, but the present disclosure is not limited to this. For example, if the type of endoscope 14 is not a type in which the hood 40 is attached to the distal end portion 36, it is possible to prevent whether or not the hood 40 is attached to the distal end portion 36 from being determined and / or prevent the brightness of the frame 46 from being switched from the first brightness B1 to the second brightness B2.

[0169] In the above embodiment, an example is given in which one type of endoscope 14 is used, but this is merely one example, and it is also possible to switch between an enabled state in which the control when the hood is attached is enabled and an disabled state in which the control when the hood is attached is disabled depending on the type of endoscope 14.

[0170] 18 shows an example in which endoscope type information 122 is received by the reception device 62. The endoscope type information 122 refers to information that can identify the type of endoscope 14. Examples of types of endoscope 14 include variable magnification endoscopes with an optical magnification function (in other words, magnifying endoscopes), treatment endoscopes that can use treatment tools, general-purpose endoscopes that are used for general purposes, and bronchial endoscopes. Generally, a hood 40 is used for variable magnification endoscopes and treatment endoscopes, but a hood 40 is not used for general-purpose endoscopes and bronchial endoscopes.

[0171] 18, the endoscope type information 122 is received by the reception device 62, and the control unit 54B determines whether the endoscope 14 used in the endoscope system 10 is an endoscope to which the hood 40 cannot be applied (hereinafter referred to as a "hood-inapplicable endoscope") by referring to the endoscope type information 122 received by the reception device 62. If the control unit 54B determines that the endoscope 14 used in the endoscope system 10 is an endoscope to which the hood 40 cannot be applied, it turns on a flag (hereinafter simply referred to as a "flag") indicating that the endoscope 14 used in the endoscope system 10 is an endoscope to which the hood 40 cannot be applied. If the control unit 54B determines that the endoscope 14 used in the endoscope system 10 is not an endoscope to which the hood 40 cannot be applied, it turns off the flag.

[0172] When the flag is on, the control unit 54B performs non-execution control, and when the flag is off, the control unit 54B does not perform non-execution control. The non-execution control refers to control that does not cause the determination unit 54A to determine whether the hood 40 is attached to the tip end portion 36, and does not cause the control unit 54B to switch the brightness of the frame 46 from the first brightness B1 to the second brightness B2.

[0173] By doing this, even if the type of endoscope 14 used in the endoscope system 10 is not a type that has a hood 40 attached to the tip 36, it is possible to avoid unnecessary processing such as determining whether or not a hood 40 is attached to the tip 36 or switching the brightness of the frame 46 from the first brightness B1 to the second brightness B2.

[0174] Here, an example has been given in which it is determined whether or not the hood 40 is attached to the tip 36, and the brightness of the frame 46 is prevented from being switched from the first brightness B1 to the second brightness B2. However, this is merely one example, and it is also possible to determine whether or not the hood 40 is attached to the tip 36, or to prevent the brightness of the frame 46 from being switched from the first brightness B1 to the second brightness B2.

[0175] FIG. 19 illustrates a modified example of the processing performed by the determination unit 54A. As illustrated in FIG. 19, the determination unit 54A determines whether the brightness of the frame 46 is the first brightness B1, on the condition that it has determined that the hood 40 is attached to the distal end portion 36. If the determination unit 54A determines that the brightness of the frame 46 is the first brightness B1, the brightness of the frame 46 is switched from the first brightness B1 to the second brightness B2. The determination unit 54A then suspends the determination of whether the hood 40 is attached to the distal end portion 36. The determination unit 54A also resumes the determination of whether the hood 40 is attached to the distal end portion 36, on the condition that the magnification has been changed by the variable magnification optical system 66. In other words, the second brightness B2 is maintained from the time the brightness of the frame 46 is switched from the first brightness B1 to the second brightness B2 until the magnification is changed by the variable magnification optical system 66.

[0176] 19 , when the brightness of the frame 46 is switched from the first brightness B1 to the second brightness B2 and the determination of whether the hood 40 is attached to the distal end portion 36 is interrupted, the determination of whether the hood 40 is attached to the distal end portion 36 is resumed on the condition that the magnification has been changed by the variable magnification optical system 66. This prevents the brightness of the frame 46 from becoming inappropriate due to the hood 40 appearing or disappearing in the frame 46 as a result of the magnification being changed by the variable magnification optical system 66. Furthermore, in the example shown in FIG. 19 , the second brightness B2 is maintained from the time the brightness of the frame 46 is switched from the first brightness B1 to the second brightness B2 until the magnification is changed by the variable magnification optical system 66. This prevents frequent switching between the first brightness B1 and the second brightness B2 caused by the determination unit 54A erroneously determining whether the hood 40 is attached to the distal end portion 36.

[0177] 19 illustrates an example in which the second brightness B2 is maintained from the time the brightness of the frame 46 is switched from the first brightness B1 to the second brightness B2 until the magnification is changed by the variable magnification optical system 66. However, this is merely an example, and the second brightness B2 may be maintained from the time the brightness of the frame 46 is switched from the first brightness B1 to the second brightness B2 until a specific condition is satisfied. An example of the specific condition is a condition in which an event occurs in which it is no longer necessary to maintain the second brightness B2. An example of an event in which it is no longer necessary to maintain the second brightness B2 is an event in which the endoscopic examination is completed, or an event in which an instruction to cancel the second brightness B2 is received by the reception device 62.

[0178] The control when the hood is attached may be enabled when the magnification of the variable magnification optical system 66 is a first magnification (i.e., when the magnification is life-size), and disabled when the magnification of the variable magnification optical system 66 is a second magnification that is larger than the first magnification (i.e., when zoomed in), or when the magnification of the variable magnification optical system 66 is a third magnification that is smaller than the first magnification (i.e., when zoomed out). In this way, it is possible to prevent the brightness of the frame 46 from becoming inappropriate due to the hood 40 appearing or disappearing from the frame 46 as a result of the magnification being changed by the variable magnification optical system 66.

[0179] The content of the control when the hood is attached may be varied depending on the type of instrument attached to the distal end portion 36. For example, the ratio of the degree to which the average luminance 78 is made to approach the target average luminance 82 to the degree to which the peak luminance 80 is made to approach the target peak luminance 84 may be varied when the instrument attached to the distal end portion 36 is the hood 40 and when it is an external treatment tool.

[0180] In the above embodiment, no reference is made to the magnitude relationship between the target average brightness 74 and the target average brightness 82, but the target average brightness 82 may be higher than the target average brightness 74. Furthermore, the target peak brightness 84 may be higher than the target peak brightness 76. Furthermore, the degree to which the target average brightness 82 is made higher than the target average brightness 74 may be changed depending on the type of appliance attached to the distal end portion 36, and the degree to which the target peak brightness 84 is made higher than the target peak brightness 76 may also be changed depending on the type of appliance attached to the distal end portion 36.

[0181] In the above embodiment, an example in which the brightness control process is performed by the computer 50 has been described, but the present disclosure is not limited to this, and at least a portion of the processes included in the brightness control process may be performed by a device provided outside the computer 50. Furthermore, at least a portion of the processes included in the brightness control process may be realized by network computing such as cloud computing, fog computing, edge computing, or grid computing.

[0182] In the above embodiment, an example in which the brightness control program 68 is stored in the storage 58 has been described, but the present disclosure is not limited to this. For example, the brightness control program 68 may be stored in a portable, computer-readable, non-transitory storage medium such as an SSD or a USB memory. The brightness control program 68 stored in the non-transitory storage medium is installed in the computer 50 of the endoscope system 10. The processor 54 executes brightness control processing in accordance with the brightness control program 68.

[0183] Alternatively, the brightness control program 68 may be stored in a storage device such as another computer or server connected to the endoscope system 10 via a network, and the brightness control program 68 may be downloaded and installed in the computer 50 in response to a request from the endoscope system 10.

[0184] It is not necessary to store the entire brightness control program 68 in a storage device such as another computer or server device connected to the endoscope system 10, or to store the entire brightness control program 68 in the storage 58; only a portion of the brightness control program 68 may be stored.

[0185] The hardware resources that execute the brightness control process can be various processors, as listed below. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource that executes the brightness control process by executing software, i.e., a program. Examples of processors include dedicated electrical circuits, such as FPGAs, PLDs, or ASICs, which are processors with circuit configurations specifically designed to execute specific processes. Each processor has built-in or connected memory, and executes the brightness control process by using the memory.

[0186] The hardware resource that executes the brightness control process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the brightness control process may be a single processor.

[0187] As an example of a configuration using a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes brightness control processing. Second, there is a configuration in which a processor is used that realizes the functions of the entire system, including multiple hardware resources that execute brightness control processing, on a single IC chip, as typified by SoCs. In this way, the brightness control processing is realized using one or more of the above-mentioned various processors as hardware resources.

[0188] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The brightness control process described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the process.

[0189] The above-described description and illustrations are a detailed explanation of the parts related to the present disclosure and are merely an example of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or elements may be replaced with other parts from the above-described description and illustrations, as long as they do not deviate from the gist of the present disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the parts related to the present disclosure, the above-described description and illustrations omit explanations of common general technical knowledge that do not require particular explanation to enable the implementation of the present disclosure.

[0190] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0191] The following additional notes are provided regarding the above-described embodiments.

[0192] (Addendum) a processor; A control device used for an endoscope having a tip portion that irradiates light, The processor is performing brightness control to bring the brightness of an image obtained by imaging the inside of the body with the endoscope while the light is irradiated inside the body closer to a target brightness; The brightness control is classified into a first brightness control and a second brightness control, the first brightness control is performed when no instrument is attached to the tip end portion, The second brightness control is performed when the instrument is attached to the distal end portion and is different from the first brightness control; The processor is Detecting the instrument attached to the distal end portion based on the feature amount of the image; Switching from the first brightness control to the second brightness control on the condition that the device is detected

[0193] Control device. [Explanation of symbols]

[0194] 10 Endoscopy System 12 Doctors 14 Endoscope 16 Display device 18 Control Device 20 Light source device 22 Subject 24 Large intestine 26 Intestinal wall 30 Camera 30A objective lens 32 Lighting equipment 32A,32B Irradiation port 34 light 36 Tip 38 Opening for treatment instruments 40 Food 40A aperture 42 screens 44 Video 46 frames 46A Central part 46A1 Edge 46A2 Round 48 Lesions 50 Computers 52 External I / F 54 processors 54A Judgment section 54B Control section 56 memory 58 Storage 60 Bus 62 Reception device 64 image sensors 66 Variable Magnification Optical System 68 Brightness Control Program 70,78 average brightness 72,80 Peak Brightness 73,81 Luminance Histogram 73A, 81A, 94A, 102A, 106A High brightness area 74,82 target average brightness 76,84 Target Peak Luminance 86 1st Range 86A 1st division area 88 First Weight 90 Second Range 90A 2nd division area 92 Second Weight 94 Luminance Histogram 96 Overall Brightness 98 Central Area Luminance 100 Center brightness 102 First Luminance Histogram 104 Second Luminance Histogram 106 Third Luminance Histogram 108 Decision Model 108A Judgment result 110 model 112 Teacher Data 114 Example Data 114A First Example Data 114B Second example data 116 Correct data 116A Hood installation label 116B Hood not attached label 118 Hood installation information 120 Hood not attached information 122 Endoscope type information B1 First brightness B2 Second Brightness TH1, TH2 threshold

Claims

1. a processor; A control device used for an endoscope having a tip portion that irradiates light, The processor: performing brightness control to bring the brightness of an image obtained by imaging the inside of the body with the endoscope while the light is irradiated inside the body closer to a target brightness; The brightness control is classified into a first brightness control and a second brightness control, the first brightness control is performed when no instrument is attached to the distal end portion, The second brightness control is performed when the instrument is attached to the distal end portion, and is different from the first brightness control. Control device.

2. the first brightness control or the second brightness control is performed based on a result of comparing a photometric value of a first image obtained by imaging the inside of the body with the endoscope while the light is irradiated into the body with a target photometric value; The degree of brightness increase of a second image obtained later than the first image by imaging the inside of the body with the endoscope while the light is irradiated inside the body is greater in the second brightness control than in the first brightness control. The control device according to claim 1 .

3. The brightness of the second image is increased by lengthening the exposure time of the endoscope when imaging is performed to obtain the second image, by increasing the amount of light when imaging is performed to obtain the second image, or by increasing the degree of influence of image processing for brightness adjustment on the second image when image processing is performed on the second image. The control device according to claim 2 .

4. the image processing is performed on a specific region of the second image; The specific area is determined according to the instrument. The control device according to claim 3 .

5. the brightness control is performed based on a plurality of photometric values ​​obtained by measuring the first image using a plurality of photometric methods; The first brightness control and the second brightness control have different degrees of use of the plurality of photometric values. The control device according to claim 2 .

6. the plurality of photometric values ​​includes an average photometric value of the first image; The degree to which the average photometric value is used in the second brightness control is greater than the degree to which the average photometric value is used in the first brightness control. The control device according to claim 5 .

7. the brightness control is performed based on a photometric value of the first image; the photometric value is obtained for each divided area into which the first image is divided by a division method different between the first brightness control and the second brightness control, The magnitude of the photometric value used in the first brightness control and the second brightness control is a magnitude corresponding to the divided area. The control device according to claim 2 .

8. The division method used in the second brightness control is determined depending on the device. The control device according to claim 7.

9. the brightness control is performed based on a photometric value of the first image; The photometric value used in the second brightness control is obtained from an area of ​​the first image that is more inward than the photometric value used in the first brightness control. The control device according to claim 2 .

10. the endoscope is a variable magnification endoscope, The second brightness control is enabled when the magnification of the variable magnification endoscope is a first magnification, and is disabled when the magnification is a second magnification that is larger than the first magnification or a third magnification that is smaller than the first magnification. The control device according to claim 1 .

11. An enabled state in which the second brightness control is enabled and an disabled state in which the second brightness control is disabled are switched depending on the type of the endoscope. The control device according to claim 1 .

12. When switching from one of the first brightness control and the second brightness control to the other, the brightness of the image changes stepwise. The control device according to claim 1 .

13. The gradual change in brightness of the image is realized by gradually increasing the exposure time of the endoscope when imaging is performed to obtain the image, by gradually increasing the amount of light when imaging is performed to obtain the image, or by gradually increasing the degree of influence of image processing on the image when image processing for brightness adjustment is performed on the image. The control device according to claim 12.

14. The content of the second brightness control varies depending on the type of the fixture. The control device according to claim 1 .

15. The device is a cylindrical hood having an opening. The control device according to claim 1 .

16. The instrument has optical properties that cause halation due to the light. The control device according to claim 1 .

17. The target brightness used in the second brightness control is higher than the target brightness used in the first brightness control. The control device according to claim 1 .

18. A control device according to any one of claims 1 to 17; The endoscope Endoscopy system.

19. 1. A method for operating a control device used for an endoscope having a tip portion that irradiates light, comprising: performing brightness control to bring the brightness of an image obtained by imaging the inside of the body with the endoscope while the light is irradiated inside the body closer to a target brightness; The brightness control is classified into a first brightness control and a second brightness control, the first brightness control is performed when no instrument is attached to the distal end portion, The second brightness control is performed when the instrument is attached to the distal end portion, and is different from the first brightness control. How the control device operates.

20. A program for causing a computer to execute processing for an endoscope having a tip portion that emits light, The process comprises: performing brightness control to bring the brightness of an image obtained by imaging the inside of the body with the endoscope while the light is irradiated inside the body closer to a target brightness; The brightness control is classified into a first brightness control and a second brightness control, the first brightness control is performed when no instrument is attached to the distal end portion, The second brightness control is performed when the instrument is attached to the distal end portion, and is different from the first brightness control. program.

Images