Endoscope system, image generating device, and image generating method

Abstract

This endoscope system comprises: an endoscope that is provided with an illumination unit that illuminates a subject with illumination light and an imaging unit that captures an image of the subject; a control device that generates a captured image by performing, on an imaging signal acquired from the imaging unit, image signal processing that is based on an imaging parameter; and an image generating device. The control device generates a first captured image that is the captured image as obtained by capturing the subject while illuminated with a light amount equal to or less than an upper limit value, and a second captured image that is the captured image as obtained by capturing the subject while illuminated with a light amount equal to or less than the upper limit value and that is brighter than the first captured image. The image generating device generates, from the first captured image and the second captured image, an estimated image estimating an image obtained by capturing the subject while illuminated with a light amount greater than the upper limit value.

Description

Endoscope system, image generating device, and image generating method

[0001] The present invention relates to an endoscope system, an image generating device, and an image generating method.

[0002] The endoscope system has an illumination unit and an imaging unit at its distal end, and the illumination unit irradiates an object with illumination light, and the imaging unit images the object. It is desirable that the captured image be sufficiently bright so that the object can be visually recognized in detail.

[0003] When the amount of illumination light emitted from the illumination unit is increased to make the captured image sufficiently bright, an upper limit of the amount of illumination light is set so that the temperature of the tip does not exceed an allowable range. On the other hand, when the captured image is made sufficiently bright by image signal processing such as gain adjustment, noise caused by the image signal processing may exceed an allowable range.

[0004] Patent Document 1 describes an endoscope system that suppresses the amount of illumination light emitted to an upper limit value except when capturing still images, thereby suppressing a rise in temperature at the tip of the endoscope.

[0005] Japanese Patent Application Publication No. 2020-116147

[0006] However, in the endoscope system described in Patent Document 1, there is still a possibility that the temperature of the tip of the endoscope may exceed the allowable range when capturing a still image.

[0007] In consideration of the above circumstances, the present invention aims to provide an endoscopic system, an image generation device, and an image generation method that can provide sufficiently bright captured images while suppressing temperature rise at the tip of the endoscope, even when capturing still images.

[0008] In order to solve the above problems, the present invention proposes the following means: An endoscopic system according to a first aspect of the present invention includes an endoscope having an illumination unit that irradiates an object with illumination light and an imaging unit that images the object, a control device that performs imaging signal processing on an imaging signal acquired from the imaging unit based on imaging parameters to generate a captured image, and an image generation device, wherein the control device generates a first captured image that is the captured image of the object illuminated with an amount of light equal to or less than an upper limit, and a second captured image that is the captured image of the object illuminated with an amount of light equal to or less than the upper limit and is brighter than the first captured image, and the image generation device generates an estimated image that estimates an image of the object illuminated with an amount of light greater than the upper limit, from the first captured image and the second captured image.

[0009] An image generating device according to a second aspect of the present invention is an image generating device that acquires an imaged image by performing image signal processing based on image capturing parameters on an image capturing signal acquired from an image capturing unit that captures an image of a subject, and acquires a first image captured of the subject illuminated with a light amount equal to or less than an upper limit value, and a second image captured of the subject illuminated with a light amount equal to or less than the upper limit value and that is brighter than the first imaged image, and generates an estimated image from the first imaged image and the second imaged image by estimating an image captured of the subject illuminated with a light amount greater than the upper limit value.

[0010] An image generation method according to a third aspect of the present invention obtains a first captured image generated by performing image capture signal processing on an image capture signal obtained by capturing an image of a subject illuminated with a light amount equal to or less than an upper limit value, and a second captured image generated by performing image capture signal processing on an image capture signal obtained by capturing an image of the subject illuminated with a light amount equal to or less than the upper limit value, the second captured image being brighter than the first captured image, and generates an estimated image from the first captured image and the second captured image by estimating an image of the subject illuminated with a light amount greater than the upper limit value.

[0011] According to the endoscope system, image generating device, and image generating method of the present invention, even when capturing a still image, it is possible to provide a sufficiently bright captured image while suppressing a temperature rise at the tip of the endoscope.

[0012] 1 is a diagram showing an endoscopic system according to a first embodiment. It is a functional block diagram of the endoscopic system. It is a functional block diagram of an imaging control unit 21 of the endoscopic system. It is a diagram showing the hardware configuration of an image generating device of the endoscopic system. It is a functional block diagram of the image generating device. It is a diagram showing the relationship between a first captured image and a second captured image. It is a diagram explaining the generation of a trained model of the endoscopic system. It is a diagram showing teacher data used for generating the trained model. It is a control flowchart of the endoscopic system. It is a diagram showing a modified example of the trained model. It is a diagram showing another modified example of the trained model. It is a diagram showing a moving image generated by the image generating unit. It is a diagram showing captured images generated by an imaging control unit and still images generated by an image generating unit in an endoscopic system according to a second embodiment. It is a control flowchart of the endoscopic system. It is a functional block diagram of an endoscopic system according to a third embodiment. It is a diagram showing the image generating unit of the endoscopic system. It is a diagram showing the relationship between an image captured with first special light and an image captured with second special light. It is a diagram showing a modified example of a trained model of the endoscopic system. It is a diagram explaining the generation of the trained model. It is a diagram showing teacher data used for generating the trained model. It is a diagram showing the image generating unit of an endoscopic system according to a fourth embodiment. It is a diagram explaining the generation of a trained model of the endoscopic system.

[0013] First Embodiment An endoscope system 100 according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 9. FIG.

[0014] [Endoscope System 100] Fig. 1 is a diagram showing an endoscope system 100. The endoscope system (image generation system) 100 includes an endoscope 1, a control device 2, an image generation device 3, and a display device 4. The control device 2 and the image generation device 3 may be integrated into one device (image control device). The display device 4 is a device that displays images generated by the control device 2 and the image generation device 3, various information related to the endoscope system 100, and the like.

[0015] [Endoscope 1] The endoscope 1 is a device for observing and treating the inside of the body of a patient lying on, for example, an operating table T. The endoscope 1 includes an elongated insertion section 10 that is inserted into the body of the patient, an operating section 18 connected to the base end of the insertion section 10, and a universal cord 19 that extends from the operating section 18.

[0016] The insertion section 10 has a distal end section 11, a bendable bending section 12, and a long, flexible flexible tube section 13. The distal end section 11, the bending section 12, and the flexible tube section 13 are connected in this order from the distal end side. The flexible tube section 13 is connected to an operation section 18.

[0017] 2 is a functional block diagram of the endoscope system 100. The distal end portion 11 has an imaging unit 14, an illumination unit 15, and a temperature sensor 17.

[0018] The imaging unit 14 has an optical system 141 and an imaging element 142 such as a CCD image sensor or a CMOS image sensor (see FIG. 3). The imaging unit 14 captures an image of a subject based on imaging parameters P transmitted from the imaging control unit 21 via an imaging control cable 143, and generates an imaging signal. The imaging signal is transmitted to the control device 2 via an imaging signal cable 144.

[0019] The illumination unit (white light illumination unit) 15 irradiates the subject with illumination light (white light) transmitted by the light guide 151. The light guide 151 is connected to the control device 2 through the insertion section 10, the operation section 18, and the universal cord 19. The illumination unit 15 may include a light source such as an LED, an optical element such as a phosphor having a wavelength conversion function, or the like.

[0020] The temperature sensor 17 is a sensor that detects the temperature of the illumination unit 15. The temperature sensor 17 includes, for example, a thermocouple, a thermistor, a temperature resistor, a thermal ferrite, a thermal expansion thermometer, etc. The detected temperature of the illumination unit 15 is acquired by the control device 2.

[0021] The operation unit 18 accepts operations for the endoscope 1. The operation unit 18 has an ankle knob 181 that controls the bending section 12, an air / water supply button 182, a suction button 183, a release button 184, and an observation mode switching button 185. Operations input to the air / water supply button 182, the suction button 183, the release button 184, and the observation mode switching button 185 are acquired by the control device 2. The release button 184 is a push button that is used to input an operation to save an image acquired from the imaging unit 14.

[0022] The universal cord 19 connects the endoscope 1 and the control device 2. The universal cord 19 is a cable through which the imaging control cable 143, the imaging signal cable 144, the light guide 151, etc. are inserted.

[0023] [Control Device 2] The control device 2 is a device that controls the entire endoscope system 100. The control device 2 includes an imaging control unit 21, an illumination control unit 22, a control unit 24, and a recording unit 25.

[0024] 3 is a functional block diagram of the imaging control unit 21. The imaging control unit 21 performs imaging signal processing based on imaging parameters P on the imaging signal acquired from the imaging unit 14 to generate a captured image (captured video) D. Ｗ The imaging control unit 21 includes an imaging drive unit 211, an analog signal processing unit 212, an AD conversion unit 213, and a digital signal processing unit 214.

[0025] The imaging driver 211 drives the imaging element 142 of the imaging unit 14 based on imaging parameters P from the control unit 24. The imaging driver 211 controls the exposure of the imaging element 142.

[0026] The analog signal processing unit 212 performs analog signal processing, including noise removal and amplification, on the imaging signal acquired from the imaging element of the imaging unit 14 based on the imaging parameters P (analog gain, ISO sensitivity, etc.) from the control unit 24.

[0027] Based on an instruction from the control unit 24, the AD conversion unit 213 converts the imaging signal that has undergone analog signal processing by the analog signal processing unit 212 into a digital signal (for example, RAW data).

[0028] The digital signal processing unit 214 generates a captured image by digitally processing the digital signal (e.g., RAW data) converted by the AD conversion unit 213 based on the imaging parameter P from the control unit 24. The digital signal processing unit 214 performs digital signal processing on the captured image to adjust brightness, contrast, etc. as necessary.

[0029] The lighting control unit 22 has a light source such as an LED, and controls the light source based on the imaging parameters P (such as light intensity) from the control unit 24 to control the amount of illumination light transmitted to the lighting unit 15 via the light guide 151.

[0030] The control unit 24 is a programmable processing circuit (computer) having a processor, a program-readable memory, etc. The control unit 24 controls the endoscope system 100 by executing an endoscope control program. The control unit 24 may include a dedicated circuit. The dedicated circuit is a processor separate from the processor of the control unit 24, a logic circuit implemented in an ASIC or FPGA, or a combination thereof.

[0031] The control unit 24 controls the imaging control unit 21 and the illumination control unit 22 by specifying operation parameters including the imaging parameter P to the imaging control unit 21 and the illumination control unit 22 to generate the captured image D. Ｗ The control unit 24 also instructs the image generation device 3 to generate an image and the display device 4 to display an image.

[0032] The recording unit 25 is a non-volatile recording medium that stores the above-mentioned programs and necessary data. The recording unit 25 is configured, for example, with a writable non-volatile memory such as a flexible disk, a magneto-optical disk, a ROM, or a flash memory, a portable medium such as a CD-ROM, or a storage device such as a hard disk built into a computer system. The recording unit 25 may also be a storage device provided on a cloud server connected to the control device 2 via the Internet.

[0033] [Image Generation Device 3] The image generation device 3 generates the captured image (captured video) D output by the imaging control unit 21 of the control device 2. Ｗ Based on this, a new estimated image E Ｗ and the generated estimated image E Ｗ The new estimated image E Ｗ When it is not necessary to generate the image, the image generating device 3 displays the captured image output by the imaging control unit 21 of the control device 2 on the display device 4 .

[0034] 4 is a diagram showing the hardware configuration of the image generation device 3. The image generation device 3 has a processor 301, a memory 302 into which a program can be read, a storage unit 303, and an input / output control unit 304. The image generation device 3 is a computer capable of executing a program. The functions of the image generation device 3 are realized by the processor 301 executing the program. At least some of the functions of the image generation device 3 may be realized by a dedicated logic circuit implemented in an ASIC or FPGA.

[0035] The storage unit 303 is a non-volatile recording medium that stores the above-mentioned programs and necessary data. The storage unit 303 is configured, for example, by a ROM, a hard disk, etc. The programs recorded in the storage unit 303 are read into the memory 302 and executed by the processor 301.

[0036] The input / output control unit 304 is connected to the control device 2, the display device 4, an input device (not shown), and a network device (not shown). Based on the control of the processor 301, the input / output control unit 304 transmits and receives data and control signals to and from the connected devices.

[0037] The image generation device 3 is not limited to an integrated hardware device. For example, the image generation device 3 may be configured by separating some parts into separate hardware devices and connecting the separated hardware devices via a communication line. For example, the image generation device 3 may be a cloud system in which separated storage units 303 are connected via a communication line.

[0038] The image generating device 3 may further include components other than the processor 301, memory 302, storage unit 303, and input / output control unit 304. For example, the image generating device 3 may further include an image calculation unit that performs part or all of the image processing and image recognition processing. By further including the image calculation unit, the image generating device 3 can execute specific image processing and image recognition processing at high speed. For example, the image generating device 3 may further include an estimated image E (described later). Ｗ By further including the inference calculation unit, the image generating device 3 can generate the estimated image E Ｗ The image calculation unit and the inference calculation unit may be mounted on separate hardware devices connected via a communication line.

[0039] 5 is a functional block diagram of the image generation device 3. The image generation device 3 includes an image generation unit 31, an image display unit 32, and an image storage unit 33 as functional blocks.

[0040] The image generation unit 31 generates a first captured image (first white light captured image) D input from the control device 2 based on the trained model M. Ｗ１ and a second captured image (second white light captured image) D Ｗ２ From the estimated image (white light estimated image) E Ｗ Generate.

[0041] FIG. 6 shows the first captured image D Ｗ１ and the second captured image D Ｗ２ 1 is a diagram illustrating the relationship between the first captured image D and the second captured image D. Ｗ１ is a captured image of a subject illuminated with a light amount equal to or less than the upper limit value U. Ｗ２ is a captured image of a subject illuminated with a light amount equal to or less than the upper limit value U, and the first captured image DＷ１ Compared to the first captured image D, the first captured image D is brighter but with increased noise due to image signal processing (analog signal processing by the analog signal processing unit 212 and / or digital signal processing by the digital signal processing unit 214) based on the image capturing parameter P. The "brightness of the captured image" refers to, for example, "brightness," "luminance," and "luminosity." In the following description, the first captured image D Ｗ１ The brightness of the second captured image D is set to "first brightness B1". Ｗ２ The brightness of is also called "second brightness B2".

[0042] First captured image D Ｗ１ and the second captured image D Ｗ２ The first captured image D is an image of a subject illuminated with substantially the same amount of light. Ｗ１ and the second captured image D Ｗ２ However, the images do not necessarily have to be images of a subject illuminated with substantially the same amount of light.

[0043] First captured image D Ｗ１ and the second captured image D Ｗ２ It is desirable that the first captured image D is an image of a subject illuminated with an amount of light that is equal to or less than the upper limit value U and is close to the upper limit value U. By bringing the amount of light closer to the upper limit value U, Ｗ１ and the second captured image D Ｗ２ This results in a brighter, less noisy captured image. Ｗ In particular, the estimation accuracy of the second captured image D Ｗ２ The noise in the estimated image E Ｗ This can efficiently improve the estimation accuracy of the

[0044] Second captured image D Ｗ２ is the first captured image D Ｗ１ Since the period T1 is very short, the first captured image D Ｗ１ and the second captured image D Ｗ２ The subject captured by the first captured image D is in substantially the same state. Ｗ１ and the second captured image D Ｗ２ The order in which the images are taken does not matter.

[0045] First captured image D Ｗ１ and the second captured image D Ｗ２ is the input of the trained model M. The estimated image E output by the trained model M Ｗ In order to improve the estimation accuracy of the first captured image D Ｗ１ and the information contained in the second captured image D Ｗ２ Therefore, it is desirable that the difference between the information contained in the first captured image D Ｗ１ The imaging parameters P for generating the second captured image D Ｗ２ It is desirable that the imaging parameters P that generate the image are more different from the image parameters P that generate the image.

[0046] The image signal processing includes, for example, amplification of an analog signal or amplification of a digital signal. Ｗ２ The gain of the imaging parameter P for generating the first captured image D Ｗ１ is greater than the gain of the imaging parameter P that generates

[0047] Estimated image E Ｗ In order to improve the estimation accuracy of the first captured image D, the amplification process of the analog signal is more suitable than the amplification process of the digital signal. The amplification process of the digital signal is performed after the AD conversion in which some information of the image signal is lost. Ｗ１ and the information contained in the second captured image D Ｗ２ On the other hand, in order to perform diverse and flexible imaging signal processing, amplification processing of a digital signal is more suitable than amplification processing of an analog signal. In the amplification processing of a digital signal, for example, imaging signal processing based on the gains of two types of imaging parameters P is performed on one type of imaging signal obtained from the imaging unit 14 to generate a first captured image D Ｗ１ and the second captured image D Ｗ２ and can be generated.

[0048] Estimated image E Ｗ is the first captured image D Ｗ１ and the second captured image D Ｗ２ The image is generated based on the trained model M from the above, and is an estimate of an image captured of a subject illuminated with a light amount greater than the upper limit value U.

[0049] The image display unit 32 displays the estimated image E generated by the image generation unit 31. Ｗ is output to the display device 4 and displayed on the display device 4.

[0050] The image storage unit 33 stores the estimated image E generated by the image generation unit 31. Ｗ The estimated image E Ｗ If there is no need to record the image, the image storage unit 33 is not necessary.

[0051] [Trained Model M] The trained model M is a machine learning model that learns the relationship between an input image and an output image, and is a machine learning model suitable for image generation, such as a neural network, a simple perceptron, or a multilayer perceptron. The trained model M is generated by prior supervised learning based on training data. The trained model M may be generated by the image generation device 3, or may be generated using another computer with a higher computing power than the image generation device 3.

[0052] 7 is a diagram illustrating the generation of the trained model M. The training data is the first captured image D ＴＷ１ , the second captured image for learning D ＴＷ２ , and the third captured image for learning D ＴＷ３ It is desirable that the training data contains as many images as possible taken under a variety of conditions.

[0053] The trained model M is the first captured image D ＴＷ１ and the second captured image for learning D ＴＷ２ Based on the estimated image E Ｗ The trained model M generates the output estimated image E Ｗ and the third captured image for learning D ＴＷ３ The parameters of the trained model M are trained so that the difference between them becomes small.

[0054] 8 is a diagram showing training data used to generate the trained model M. ＴＷ１ is the first captured image D prepared for learning Ｗ１ Second captured image for learning D ＴＷ２ is the second captured image D prepared for learningＷ２ is.

[0055] Third captured image for learning D ＴＷ３ is an image actually captured of a subject illuminated with a light amount greater than the upper limit value U. Also, the third learning image D ＴＷ３ is the noise generated by the imaging signal processing based on the imaging parameter P (analog signal processing by the analog signal processing unit 212 and / or digital signal processing by the digital signal processing unit 214) in the first learning image D ＴＷ１ For example, the third captured image for learning D ＴＷ３ is the image capturing parameter P other than the light amount when the first captured image D ＴＷ１ The image is substantially the same as that obtained with the imaging parameter P.

[0056] Third captured image for learning D ＴＷ３ is the first captured image for learning D ＴＷ１ and the second captured image for learning D ＴＷ２ Since the period T2 is very short, the first learning image D ＴＷ１ and the second captured image for learning D ＴＷ２ and the third captured image for learning D ＴＷ３ The subject captured by each of the first captured learning images D ＴＷ１ and the second captured image for learning D ＴＷ２ and the third captured image for learning D ＴＷ３ The order in which the first captured learning image D ＴＷ１ , the second captured image for learning D ＴＷ２ and the third captured image for learning D ＴＷ３ may be an image of a stationary subject. In this case, the first captured image for learning D ＴＷ１ and the second captured image for learning D ＴＷ２ and the third captured image for learning D ＴＷ３ The subject captured by the two cameras will be in the same state.

[0057] First captured image for learning D ＴＷ１ and the brightness of the second captured image for learning D ＴＷ２It is desirable that the brightness of the first captured image D is approximately equal to the first brightness B1 and the second brightness B2. Ｗ１ and the second captured image D Ｗ２ This is because it is easy to efficiently learn the relationship between

[0058] Second captured image for learning D ＴＷ２ is the first captured image for learning D ＴＷ１ It is desirable that the image is captured approximately a period T1 after the first captured image D Ｗ１ and the second captured image D Ｗ２ This is because it is easy to efficiently learn the relationship between

[0059] From the supervised learning using the above-mentioned training data, the first captured image D Ｗ１ and the second captured image D Ｗ２ and estimated image E Ｗ A trained model M is generated by previously learning the relationship between the first captured image D and the second captured image D. Ｗ１ and the second captured image D Ｗ２ The estimated image E is output by the trained model M to which Ｗ The noise contained in the first captured image D Ｗ１ The noise level is about the same as that generated by image signal processing.

[0060] [Operation of Endoscopic System 100] Next, the operation (image generating method) of the endoscope system 100 will be described. The explanation will be made along the control flowchart of the endoscope system 100 shown in Fig. 9. When the control unit 24 of the control device 2 detects that the operator has pressed the release button 184, the endoscope system 100 executes step S110.

[0061] <Step S110: Imaging Parameter Adjustment Step> In step S110, the control unit 24 of the control device 2 calculates optimal imaging parameters P (including light intensity and analog gain) for imaging the subject based on the captured image captured by the imaging control unit. A known imaging adjustment technique is used to calculate the optimal imaging parameters P. The endoscope system 100 then executes step S120.

[0062] <Step S120: Light Intensity Determination Step> In step S120, the control unit 24 of the control device 2 determines whether the calculated light intensity exceeds the upper limit value U. If the calculated light intensity exceeds the upper limit value U, the control unit 24 of the control device 2 then executes step S130. If the calculated light intensity is equal to or less than the upper limit value U, the endoscope system 100 then executes step S160.

[0063] The upper limit value U does not have to be a fixed value, but may be a value that fluctuates based on the temperature of the illumination unit 15 acquired by the temperature sensor 17 of the endoscope 1. For example, if the temperature of the illumination unit 15 is lower than a predetermined temperature, the control unit 24 may increase the upper limit value U. Also, if the temperature of the illumination unit 15 is higher than the predetermined temperature, the control unit 24 may decrease the upper limit value U. If the endoscope 1 does not have a temperature sensor 17, the control unit 24 may use a temperature estimated from the integrated light amount obtained by multiplying the light amount by the light emission time.

[0064] <Step S130: First Captured Image Generation Process> In step S130, the control unit 24 of the control device 2 sets the light amount of the imaging parameter P to an upper limit value U or less, and generates the first captured image D Ｗ１ The endoscope system 100 then executes step S140.

[0065] <Step S140: Second Captured Image Generation Process> In step S140, the control unit 24 of the control device 2 changes the imaging parameters P other than the light amount to generate the first captured image D Ｗ１ The second captured image D has increased brightness but with increased noise compared to the first captured image D. Ｗ２ For example, the control unit 24 increases the analog gain of the imaging parameter P to generate the second captured image D Ｗ２ The control unit 24 of the control device 2 generates the first captured image D Ｗ１ After the time period T1 has elapsed since the image was captured, the second captured image D Ｗ２ The endoscope system 100 then executes step S150.

[0066] <Step S150: Estimated Image Generation Step> In step S150, the image generation unit 31 of the image generation device 3 generates an estimated image D based on the trained model M. Ｗ１ and the second captured image D Ｗ２ From this, the estimated image E Ｗ The image storage unit 33 generates the estimated image E Ｗ The endoscope system 100 then executes step S170.

[0067] <Step S160: First Captured Image Generation Process> In step S160, the control unit 24 and the imaging control unit 21 of the control device 2 generate the first captured image D Ｗ１ Since the light amount calculated in step S110 is equal to or less than the upper limit value U, the control unit 24 of the control device 2 sets the light amount to the calculated light amount and generates the first captured image D Ｗ１ The endoscope system 100 then executes step S170.

[0068] <Step S170: Image Display Step> The image display unit 32 of the image generating device 3 displays the estimated image E generated in step S150. Ｗ Or the first captured image D captured in step S160 Ｗ１ is displayed on the display device 4. The endoscope system 100 then executes step S180.

[0069] <Step S180: End Determination Step> In step S180, the control unit 24 of the control device 2 determines whether the release button 184 has been pressed by the surgeon, etc. If the release button 184 has been pressed, the control unit 24 of the control device 2 performs step S110 again.

[0070] According to the endoscope system 100 of this embodiment, even when a still image is acquired, it is possible to provide a sufficiently bright captured image while suppressing a temperature rise in the distal end portion 11 of the endoscope 1. Specifically, the endoscope system 100 captures a first captured image D of a subject illuminated with a light amount equal to or less than the upper limit value U. Ｗ１ and the second captured image D Ｗ２Since only the upper limit value U is used, the amount of illumination light emitted from the illumination unit 15 can be kept below the upper limit value U, and the temperature of the tip portion 11 of the endoscope 1 can be kept below a predetermined temperature. On the other hand, the image generating device 3 of the endoscope system 100 generates a first captured image D Ｗ１ and the second captured image D Ｗ２ Therefore, an estimated image E is obtained by estimating an image of a subject illuminated with a light amount greater than the upper limit value U. Ｗ That is, the endoscope system 100 can estimate the first captured image D while suppressing the amount of illumination light emitted from the illumination unit 15 to an upper limit value U or less. Ｗ１ The image is brighter than the first captured image D Ｗ１ The estimated image E is equivalent to the noise generated by the imaging signal processing. Ｗ can be presented to the user.

[0071] Although the first embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment and includes design modifications within the scope of the present invention. Furthermore, the components shown in the above-described embodiment and modifications can be configured by appropriately combining them.

[0072] (Modification 1) Fig. 10 is a diagram showing a trained model M1 which is a modification of the trained model M. The trained model M1 uses at least a part of the imaging parameters P (e.g., analog gain) as input data. For example, the trained model M1 uses the first captured image D Ｗ１ First imaging parameter P, which is the imaging parameter P when imaging Ｗ１ and the second captured image D Ｗ２ The second imaging parameter P is the imaging parameter P when the Ｗ２ By adding at least a part of the imaging parameters P as an input to the trained model M1, the estimated image E Ｗ The imaging parameter P is also added as an input to the training data used to train the trained model M1.

[0073] 11 is a diagram showing a trained model M2 which is another trained model M. The trained model M2 uses an additional auxiliary captured image as input data. For example, the trained model M2 uses the first captured image D Ｗ１ Immediately after capturing the image, a first auxiliary captured image S is captured with the light amount set to zero without changing the image capturing parameters P other than the light amount. Ｗ１ and the second captured image D Ｗ２ Immediately after capturing the image, a second auxiliary captured image S is captured with the light amount set to zero without changing the image capturing parameters P other than the light amount. Ｗ２ By adding an auxiliary captured image captured with a zero light amount as an input to the trained model M2, an estimated image E that is less susceptible to pattern noise of the image sensor 142 of the image capturing unit 14 can be obtained. Ｗ In particular, when a CMOS image sensor, which is relatively difficult to remove noise by correlated double sampling, is used as the image sensor 142, an estimated image E can be generated by adding an auxiliary captured image captured with the amount of light set to zero as an input. Ｗ can be generated with higher accuracy. The auxiliary captured image is also added as an input to the training data used for training the trained model M2.

[0074] (Modification 3) In the above embodiment, the endoscope system 100 detects a pressing operation on the release button 184 to generate a single estimated image E Ｗ However, the endoscope system 100 generates the estimated image E regardless of whether the release button 184 is pressed (ON). Ｗ are successively generated, and successive estimated images E Ｗ 12 shows a moving image (sequential estimated images E Ｗ The endoscope system 100 executes a series of steps from step S110 to step S170 to generate an estimated image E Ｗ By repeatedly executing a series of steps, a moving image (successive estimated images E ＷIn order to generate smooth moving images, the endoscope system 100 generates estimated images E Ｗ It is desirable to have the processing capability to generate

[0075] The endoscope system 100 generates an estimated image E Ｗ The light amount is adjusted in step S110 each time a moving image is generated. Therefore, the light amount fluctuates during generation of a moving image. It is desirable that the light amount is equal to or less than the upper limit value U and close to the upper limit value U, but it may be set to a value lower than the upper limit value U based on the adjustment result in step S110. Note that the moving image generation sequence shown in FIG. 12 is a moving image generation sequence when the light amount calculated in step S120 exceeds the upper limit value U. If the light amount calculated during generation of a moving image is equal to or less than the upper limit value U, the endoscope system 100 adjusts the first captured image D in step S160. Ｗ１ The generated first captured image D Ｗ１ is one frame of a video.

[0076] The endoscope system 100 captures the first captured image D while suppressing the amount of illumination light emitted from the illumination unit 15 to an upper limit value U or less. Ｗ１ The image is brighter than the first captured image D Ｗ１ The estimated image E is equivalent to the noise generated by the imaging signal processing. Ｗ A continuous video image can be presented to the user.

[0077] (Modification 4) In the above embodiment, the illumination unit irradiates the subject with white light. However, the illumination light irradiated by the illumination unit is not limited to this. For example, the illumination unit may irradiate the subject with special light, which will be described later.

[0078] Second Embodiment A second embodiment of the present invention will be described with reference to Fig. 13 and Fig. 14. An endoscope system 100B according to the second embodiment differs from the endoscope system 100 according to the first embodiment only in the control flow. In the following description, components that are common to those already described will be assigned the same reference numerals, and redundant description will be omitted.

[0079] The endoscope system 100B displays a captured image (moving image) on the display device 4, and generates a captured image (still image) when the control unit 24 of the control device 2 detects that the operator has pressed (ON) the release button 184. FIG. 13 shows a captured image generated by the imaging control unit 21 and a still image (estimated image E) generated by the image generation unit 31. Ｗ 14. When the endoscope system 100B is started up, the endoscope system 100B executes step S210.

[0080] <Step S210: Moving Image Generation Step> In step S210, the control unit 24 of the control device 2 calculates the optimal imaging parameters P (including the amount of light and the analog gain) for capturing an image of the subject, and generates the second captured image D Ｗ２ For example, the control unit 24 of the control device 2 generates the second captured images D at 30 FPS. Ｗ２ As shown in FIG. 13 , the control unit 24 of the control device 2 adjusts the amount of light within a range equal to or less than the upper limit value U. The control unit 24 of the control device 2 adjusts the amount of light within a range equal to or less than the upper limit value U. Ｗ２ are displayed on the display device 4 as captured images (moving images).

[0081] <Step S220: Release Detection Process> In step S220, the control unit 24 of the control device 2 detects a press operation on the release button 184. The control unit 24 of the control device 2 continues step S210 unless a press operation on the release button 184 is detected. If a press operation on the release button 184 is detected, the endoscope system 100B then executes step S230.

[0082] <Step S230: First Captured Image Generation Process> As in step S130 of the first embodiment, the control unit 24 of the control device 2 sets the light amount of the imaging parameter P to be equal to or less than the upper limit value U, and generates the first captured image D Ｗ１ For example, the control unit 24 temporarily reduces the analog gain of the imaging parameter P to generate the first captured image D Ｗ１ The control unit 24 generates a first captured image D Ｗ１When generating the first captured image D, it is desirable to set the amount of light to a value equal to or less than the upper limit value U and as close as possible to the upper limit value U. Ｗ１ After generating the second captured image D Ｗ２ are continuously generated, and the continuously generated second captured images D Ｗ２ as a captured image (moving image) on the display device 4. The endoscope system 100B then executes step S240.

[0083] <Step S240: Estimated Image Generation Step> The image generation unit 31 of the image generation device 3 generates the first captured image D Ｗ１ and the first captured image D Ｗ１ The second captured image D generated before and after the generation of Ｗ２ and the estimated image E based on the trained model M. Ｗ The image storage unit 33 generates the estimated image E Ｗ The endoscope system 100B then executes step S250.

[0084] <Step S250: Image Display Step> The image display unit 32 of the image generating device 3 displays the estimated image E generated in step S240. Ｗ is displayed on the display device 4. The image display unit 32 of the image generating device 3 displays the second captured image D Ｗ２ and estimated image E Ｗ The image display unit 32 of the image generating device 3 may simultaneously display the second captured image D Ｗ２ and the estimated image E Ｗ may be displayed on the display device 4 for only a predetermined period of time. The endoscope system 100 then executes step S260.

[0085] <Step S260: End Determination Step> In step S180, the control unit 24 of the control device 2 determines whether to end the moving image display. If the moving image display is not to be ended, the control unit 24 of the control device 2 performs step S210 again.

[0086] As in the third modification, the endoscope system 100B generates an estimated image E Ｗ are successively generated, and successive estimated images EＷ A moving image having frames may be generated.

[0087] According to the endoscopic system 100B of this embodiment, even when a still image is acquired in the middle of acquiring an imaged video (moving image), a sufficiently bright imaged image (still image) can be provided while suppressing the temperature rise of the tip portion 11 of the endoscope 1.

[0088] Although the second embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment and includes design modifications within the scope of the present invention. Furthermore, the components shown in the above embodiment and modified examples can be configured by appropriately combining them.

[0089] (Modification 5) In the above embodiment, the endoscope system 100B continuously generates the second captured image D Ｗ２ is a captured image (moving image), and a first captured image D is generated in response to a pressing operation of the release button 184. Ｗ１ Using the estimated image E Ｗ However, the manner in which the captured video (moving image) and the still image are generated is not limited to this. Ｗ１ is a captured image (moving image), and a second captured image D is generated in response to a pressing operation of the release button 184. Ｗ２ Using the estimated image E Ｗ may be generated as a captured image (still image).

[0090] Third Embodiment A third embodiment of the present invention will be described with reference to Fig. 15 to Fig. 17. Compared to the endoscope system 100 according to the first embodiment, an endoscope system 100C according to the third embodiment can use two types of observation modes. In the following description, components that are common to those already described will be assigned the same reference numerals, and duplicated description will be omitted.

[0091] 15 is a functional block diagram of an endoscope system 100C. The endoscope system 100C includes an endoscope 1C, a control device 2C, an image generating device 3C, and a display device 4.

[0092] The endoscope 1C is the same as the endoscope 1 of the first embodiment except for the distal end portion 11C. The distal end portion 11C of the endoscope 1C has an imaging unit 14, an illumination unit 15, a special light illumination unit 16, and a temperature sensor 17.

[0093] The special light illumination unit 16 irradiates the subject with special light used in NBI (registered trademark: Narrow Band Imaging) and RDI (registered trademark: Red Dichromatic Imaging). Here, the special light irradiated by the special light illumination unit 16 is light with a wavelength range different from that of white light. The special light used in NBI is light with two narrow wavelength ranges: blue (wavelengths of 390 to 445 nm) and green (wavelengths of 530 to 550 nm). The special light used in RDI is light with three narrow wavelength ranges: red, amber, and green.

[0094] The control device 2C is a device that controls the entire endoscope system 100C. The control device 2C includes an imaging control unit 21C, an illumination control unit 22, a special light illumination control unit 23, a control unit 24C, and a recording unit 25.

[0095] The imaging control unit 21C has the same functions as the imaging control unit 21 of the first embodiment, and further performs imaging signal processing based on the imaging parameters P on imaging signals obtained by imaging a subject illuminated with special light, to generate a special light captured image (special light captured video) D. Ｓ can be generated.

[0096] The special light illumination control unit 23 controls the light source of the special light based on the imaging parameters P (such as the amount of special light) from the control unit 24, and controls the amount of special light transmitted to the special light illumination unit 16.

[0097] The control unit 24C has the same functions as the control unit 24 of the first embodiment, and can switch between two types of observation modes (white light observation mode and special light observation mode) based on an operation input to the observation mode switching button 185. In the white light observation mode, the control unit 24C generates an image (white light image) D from an image signal obtained by capturing an image of a subject illuminated with white light from the illumination unit 15. ＷIn the special light observation mode, the control unit 24C generates a special light captured image D from an image signal obtained by capturing an image of a subject illuminated with special light from the special light illumination unit 16. Ｓ Generate.

[0098] The image generating device 3C includes an image generating unit 31C, an image display unit 32, and an image storage unit 33 as functional blocks.

[0099] 16 is a diagram illustrating the image generation unit 31C. The image generation unit 31C generates the first captured image D input from the control device 2C based on the trained model MC. Ｗ１ , second captured image D Ｗ２ , first special light captured image D Ｓ１ and second special light captured image D Ｓ２ From the estimated image E Ｗ Generate.

[0100] FIG. 17 shows the first special light captured image D Ｓ１ and the second special light captured image D Ｓ２ 1 is a diagram illustrating the relationship between the first special light captured image D and the second special light captured image D. Ｓ１ is a special light captured image of a subject illuminated with special light having an amount of light equal to or less than the upper limit value U. Ｓ２ is a special light captured image of a subject illuminated with special light of an amount equal to or less than the upper limit value U, and the first special light captured image D Ｓ１ In comparison with the image captured with special light, the image is brighter but with increased noise due to imaging signal processing based on the imaging parameter P (analog signal processing by the analog signal processing unit 212 and / or digital signal processing by the digital signal processing unit 214).

[0101] First special light captured image D Ｓ１ and the second special light captured image D Ｓ２ It is desirable that the first special light captured image D is an image of a subject illuminated with an amount of light that is equal to or less than the upper limit value U and is close to the upper limit value U. Ｓ１ and the second special light captured image D Ｓ２ This results in a brighter, less noisy captured image. Ｗ In particular, the estimation accuracy of the second special light captured image DＳ２ The noise in the estimated image E Ｗ This can efficiently improve the estimation accuracy of the

[0102] First special light captured image D Ｓ１ is the first captured image D Ｗ１ The special light image is captured after a period T3 has elapsed since the first captured image D Ｗ１ and first special light captured image D Ｓ１ The subject captured by the first captured image D is in substantially the same state. Ｗ１ and first special light captured image D Ｓ１ The order in which the images are taken does not matter.

[0103] Second special light captured image D Ｓ２ is the second captured image D Ｗ２ The special light image is captured after a period T4 has elapsed since the first captured image D. Since the period T4 is very short, the second captured image D Ｗ２ and the second special light captured image D Ｓ２ The subject captured by the second captured image D is in substantially the same state. Ｗ２ and the second special light captured image D Ｓ２ The order in which the images are taken does not matter.

[0104] The trained model MC is compared with the trained model M of the first embodiment in that the first special light captured image D Ｓ１ and second special light captured image D Ｓ２ The trained model MC further has, as an input, the special light captured image as an input to the training data used for training the trained model MC.

[0105] When the control unit 24C of the control device 2C detects that the operator has pressed the release button 184, the endoscope system 100C generates a first captured image D Ｗ１ , second captured image D Ｗ２ , first special light captured image D Ｓ１ and second special light captured image D Ｓ２ and an estimated image E is generated from these four captured images. Ｗ Generate.

[0106] As in the third modification, the endoscope system 100C generates an estimated image EＷ are successively generated, and successive estimated images E Ｗ In addition, the endoscope system 100C may generate a moving image having frames of the second captured images D Ｗ２ is a captured image (moving image), and a first captured image D Ｗ１ , first special light captured image D Ｓ１ and second special light captured image D Ｓ２ Using the estimated image E Ｗ may be generated as a captured image (still image).

[0107] According to the endoscope system 100C of this embodiment, the amount of illumination light emitted from the illumination unit 15 and the special light illumination unit 16 is kept below the upper limit value U, while the first captured image D Ｗ１ The image is brighter than the first captured image D Ｗ１ The estimated image E is equivalent to the noise generated by the imaging signal processing. Ｗ can be presented to the user.

[0108] The special light captured image obtained from the image signal of the subject illuminated with the special light is added as an input to the trained model MC. Since the two types of captured images obtained by capturing the subject illuminated with the two types of illumination light (white light and special light) have different characteristics, the estimated image E Ｗ This can advantageously improve the estimation accuracy.

[0109] Although the third embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment and includes design modifications within the scope of the present invention. Furthermore, the components shown in the above embodiment and modifications can be configured in any suitable combination.

[0110] 18 is a diagram showing a trained model MC1, which is a trained model MC. The trained model MC1 is a trained model MC1 for a special light estimated image E Ｓ The special light estimated image E is output. Ｓ As shown in FIG. 17, the first special light captured image D Ｓ１The image is brighter than the first special light captured image D Ｓ１ The image is an image with noise that is equivalent to that generated by image capture signal processing in the image. Ｗ and special light estimation image E Ｓ Both may be output.

[0111] 19 is a diagram illustrating the generation of a trained model MC1. ＴＷ１ , learning first special light captured image D ＴＳ１ , the second captured image for learning D ＴＷ２ , learning second special light captured image D ＴＳ２ , and the third special light captured image for learning D ＴＳ３ This is a set of images.

[0112] 20 is a diagram showing training data used to generate the trained model MC1. ＴＳ１ is the first special light captured image D prepared for learning. Ｓ１ Second special light captured image for learning D ＴＳ２ is the second captured special light image D prepared for learning. Ｓ２ is.

[0113] Third special light captured image for learning D ＴＳ３ is an image actually captured of a subject illuminated with special light having a light intensity greater than the upper limit value U. Also, the third special light image for learning D ＴＳ３ is the noise generated by the image signal processing based on the image capturing parameter P (analog signal processing by the analog signal processing unit 212 and / or digital signal processing by the digital signal processing unit 214) in the first special light-captured learning image D ＴＳ１ For example, the third special light captured image D ＴＳ３ is the image capturing parameter P other than the light amount when the first special light captured image for learning D ＴＳ１ The image is substantially the same as that obtained with the imaging parameter P.

[0114] (Variation 7) In the above embodiment, the endoscope 1C has two types of observation modes. If the endoscope 1C has three or more types of observation modes, the trained model MC may be capable of inputting three or more types of captured images.

[0115] (Fourth Embodiment) A fourth embodiment of the present invention will be described with reference to Fig. 21 and Fig. 22. The endoscope system 100D according to the fourth embodiment can use two types of observation modes, similar to the endoscope system 100C according to the third embodiment. In the following description, components that are common to those already described will be assigned the same reference numerals, and duplicate description will be omitted.

[0116] The endoscope system 100D is different from the endoscope system 100C according to the third embodiment in that it includes an image generating unit 31D instead of the image generating unit 31C.

[0117] 21 is a diagram illustrating the image generation unit 31D. The image generation unit 31D generates the first special light captured image D input from the control device 2C based on the trained model MD. Ｓ１ and the second captured image D Ｗ２ Special light estimated image E Ｓ Generate.

[0118] 22 is a diagram illustrating generation of a trained model MD. The training data is a training image captured with first special light D ＴＳ１ , the second captured image for learning D ＴＷ２ , and the third special light captured image for learning D ＴＳ３ It is desirable that the training data contains as many images as possible taken under a variety of conditions.

[0119] The trained model MD is a training image captured with first special light D ＴＳ１ and the second captured image for learning D ＴＷ２ Based on the special light estimation image E Ｓ The trained model MD generates the output special light estimated image E Ｓ and the third special light captured image D for learning ＴＳ３ The parameters of the trained model MD are trained so that the difference between them becomes small.

[0120] Third special light captured image for learning D ＴＳ３ is the first special light captured image for learning D ＴＳ１ and the second captured image for learning D ＴＷ２ The first special light-captured image for learning D is captured after a period T5 has elapsed since the first special light-captured image for learning D was captured. ＴＳ１ and the second captured image for learning D ＴＷ２ and the third special light captured image D for learning ＴＳ３ The subject images captured by the first special light learning image D are in substantially the same state. ＴＳ１ and the second captured image for learning D ＴＷ２ and the third special light captured image D for learning ＴＳ３ The order in which the images are taken does not matter.

[0121] When the control unit 24C of the control device 2C detects that the operator has pressed the release button 184, the endoscope system 100D generates a first special light captured image D Ｓ１ and the second captured image D Ｗ２ and a special light estimated image E is generated from these two captured images. Ｓ Generate.

[0122] As in the third modification, the endoscope system 100D generates a special light estimated image E Ｓ are continuously generated, and continuous special light estimated images E Ｓ In addition, the endoscope system 100D may generate a moving image having frames of the second captured images D Ｗ２ is a captured image (moving image), and a first special light captured image D Ｓ１ Using the special light estimation image E Ｓ may be generated as a captured image (still image).

[0123] According to the endoscope system 100D of this embodiment, the amount of illumination light emitted from the illumination unit 15 and the special light illumination unit 16 is kept below the upper limit value U, and the first special light captured image D Ｓ１ The image is brighter than the first special light captured image D Ｓ１ The special light estimated image E is equivalent to the noise generated by the image signal processing in ＳThe endoscope system 100D requires fewer captured images to generate an estimated image than the endoscope system 100C of the third embodiment. Therefore, the frame rate can be increased when generating a moving image using estimated images as frames, as in the third modification.

[0124] Depending on the type of special light, a special light image may be darker than a white light image. For example, a special light image acquired by NBI may be darker than a white light image. The special light used in NBI is more easily absorbed by the subject's human tissue than white light, and is less likely to be reflected by the tissue. Therefore, when the irradiated light intensity is the same, the light intensity received by the image sensor 142 is weaker when the subject is irradiated with special light than when the subject is irradiated with white light. Therefore, when a special light image is acquired by NBI, it is difficult to provide a sufficiently bright special light image in applications where the white light image and the special light image are used in combination (e.g., parallel display of the white light image and the special light image). However, the endoscope system 100D can provide a first special light image D Ｓ１ and the second captured image D Ｗ２ First special light captured image D Ｓ１ Special light estimated image E is brighter than Ｓ can be generated and provided.

[0125] Although the fourth embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment and includes design modifications within the scope of the present invention. Furthermore, the components shown in the above-described embodiment and modifications can be configured by appropriately combining them.

[0126] (Variation 8) In the above embodiment, the imaging unit 14 that generates the imaging signal is provided in the endoscope 1, and the image generation device 3, 3C generates the estimated image E based on the captured image acquired from the imaging unit 14 of the endoscope 1. However, the source of the imaging signal of the image generation device is not limited to the endoscope 1. The image generation device may generate the estimated image E based on the captured image acquired from another imaging device such as a camera, a video camera, an industrial endoscope, a microscope, a robot with an image recognition function, or a mobile device such as a smartphone, a mobile phone, a smartwatch, a tablet terminal, or a notebook PC.

[0127] (Modification 9) In the above embodiment, LEDs are used as light sources included in the illumination unit 15 and the illumination control unit 22, but the light source is not limited to LEDs. For example, the light source may be a laser light source including a laser diode, an organic electroluminescent (EL) light bulb, a xenon or halogen light bulb, a combination of these, or a combination of these with an optical element such as a phosphor having a wavelength conversion function.

[0128] (Variation 10) In the above embodiment, the imaging parameter P is the light intensity and gain, but the imaging parameter P is not limited to these. The imaging parameter P may also be a spectral distribution, an aperture value (F-number), a shutter speed, a frame rate of a moving image, an optical magnification, a digital magnification, a gradation, a pixel size, a resolution (DPI), etc., in a process in which the imaging control unit 21 generates a captured image (captured video) from an imaging signal captured by the imaging element 142. The imaging parameter P may also include the organ type of the subject (esophagus, stomach, duodenum, etc.). Note that the organ type may be determined from the captured image by a model such as a machine learning model, may be determined from the insertion distance of the distal end portion 11 of the endoscope 1 into the body, or may be input to the image generation device 3 by the surgeon.

[0129] The programs of the embodiments and their variations may be recorded on a computer-readable recording medium, and then loaded and executed by a computer system. The term "computer system" includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" may also include media that dynamically store programs for a short period of time, such as communication lines used when transmitting programs over a network such as the Internet or a telephone line, or media that store programs for a fixed period of time, such as volatile memory within a computer system serving as a server or client. The program may also be designed to implement some of the functions described above, or may be capable of implementing the functions described above in combination with a program already stored in the computer system.

[0130] REFERENCE SIGNS LIST 100 ENDOSCOPY SYSTEM (IMAGE GENERATION SYSTEM) 1 ENDOSCOPY 14 IMAGING UNIT 15 ILLUMINATION UNIT (WHITE LIGHT ILLUMINATION UNIT) 16 SPECIAL LIGHT ILLUMINATION UNIT 17 TEMPERATURE SENSOR 2 CONTROL UNIT 3 IMAGE GENERATION DEVICE 4 DISPLAY UNIT P IMAGING PARAMETER U UPPER LIMIT D Ｗ１ First captured image (first white light captured image) D Ｗ２ Second captured image (second white light captured image) E Ｗ　 Estimated image (white light estimated image) D Ｓ１ First captured image (first special light captured image) D Ｓ２ Second captured image (second special light captured image) E Ｓ　 Estimated image (special light estimated image)

Claims

1. an endoscope having an illumination unit that irradiates an object with illumination light and an imaging unit that images the object; a control device that performs imaging signal processing based on imaging parameters on an imaging signal acquired from the imaging unit to generate a captured image; An image generating device; Equipped with The control device includes: a first captured image obtained by capturing an image of the subject illuminated with an amount of light equal to or less than an upper limit; a second captured image obtained by capturing an image of the subject illuminated with an amount of light equal to or less than the upper limit, the second captured image being brighter than the first captured image; Generate the image generating device generates an estimated image by estimating an image captured of the subject illuminated with an amount of light greater than the upper limit value, from the first captured image and the second captured image. Endoscopy system.

2. A display device for displaying the captured image is further provided. the control device continuously generates the second captured images and displays them on the display device as a moving image, and generates the first captured images as still images; the image generating device generates the estimated image from the first captured image and the second captured image generated before and after generation of the first captured image. The endoscope system according to claim 1 .

3. The image generating device generates the estimated image based on a trained model that has previously been machine-learned to determine a relationship between the first captured image, the second captured image, and the estimated image. The endoscope system according to claim 1 or 2.

4. The trained model is a model that has been machine-learned using the first captured image prepared for learning, the second captured image prepared for learning, and the third captured image prepared for learning as training data, the third captured image is an image obtained by capturing an image of the subject illuminated with a light amount greater than the upper limit value, and a noise generated by the image capture signal processing is an image having a level of noise generated by the image capture signal processing in the first captured image. The endoscope system according to claim 3 .

5. The trained model uses at least a portion of the imaging parameters as input data. The endoscope system according to claim 3 .

6. The noise included in the estimated image is about the same as the noise generated in the first captured image by the imaging signal processing. The endoscope system according to claim 1 .

7. noise generated in the second captured image by the imaging signal processing is greater than noise generated in the first captured image by the imaging signal processing; The endoscope system according to claim 1 .

8. the imaging signal processing includes an amplification process for the imaging signal, a gain of the imaging parameters for generating the second captured image is greater than a gain of the imaging parameters for generating the first captured image; The endoscope system according to claim 7 .

9. the control device acquires a temperature of the illumination unit, and determines the upper limit value of the light amount so that the temperature is equal to or lower than a predetermined temperature. The endoscope system according to claim 1 .

10. The endoscope further includes a special light illumination unit that emits special light having a wavelength range different from that of the illumination light emitted by the illumination unit, the control device is capable of performing image capture signal processing based on the imaging parameters on an image capture signal obtained by capturing an image of the subject illuminated with the special light, to generate a special light captured image; The control device includes: a first special light captured image obtained by capturing an image of the subject illuminated with the special light having a light amount equal to or less than the upper limit; a second special light captured image obtained by capturing an image of the subject illuminated with the special light having a light amount equal to or less than the upper limit value, the second special light captured image being brighter than the first special light captured image; Generate the image generating device generates the estimated image from the first captured image, the second captured image, the first special light captured image, and the second special light captured image. The endoscope system according to claim 1 .

11. the image generating device generates the estimated image based on a trained model that has previously been machine-learned to define relationships between the first captured image, the second captured image, the first special light captured image, and the second special light captured image, and the estimated image. The endoscope system according to claim 10.

12. The endoscope further includes a special light illumination unit that emits special light having a wavelength range different from that of the illumination light emitted by the illumination unit, the first captured image is an image generated by performing image signal processing on an image signal obtained by capturing an image of the subject illuminated with the special light by the special light illumination unit, the second captured image is an image generated by performing image capture signal processing on an image capture signal obtained by capturing an image of the subject illuminated with the illumination light by the illumination unit, the estimated image is an image obtained by estimating, from the first captured image and the second captured image, an image of the subject illuminated with the special light having a light amount greater than the upper limit value. The endoscope system according to claim 1 .

13. The image generating device generates the estimated image based on a trained model that has previously been machine-learned to determine a relationship between the first captured image, the second captured image, and the estimated image. The endoscope system of claim 12.

14. The trained model is a model that has been machine-learned using the first captured image prepared for learning, the second captured image prepared for learning, and the third captured image prepared for learning as training data, the third captured image is an image obtained by capturing an image of the subject illuminated with the special light having a light amount greater than the upper limit value, and a noise generated by the image capture signal processing is approximately the same as a noise generated by the image capture signal processing in the first captured image. The endoscope system according to claim 13.

15. The image generating device continuously generates the estimated images, and generates a moving image including the continuous estimated images as frames and a moving image. The endoscope system according to claim 1, claim 10 or claim 12.

16. An image generating device that acquires a captured image by performing image signal processing based on an imaging parameter on an image signal acquired from an imaging unit that captures an image of a subject, A first captured image of the subject illuminated with an amount of light equal to or less than the upper limit, and a second captured image of the subject illuminated with an amount of light equal to or less than the upper limit, the second captured image being brighter than the first captured image, are obtained. generating an estimated image by estimating an image captured of the subject illuminated with an amount of light greater than the upper limit value from the first captured image and the second captured image; Image generating device.

17. generating the estimated image based on a trained model that has previously been machine-learned to determine a relationship between the first captured image, the second captured image, and the estimated image; 17. The image generating device of claim 16.

18. The trained model is a model that has been machine-learned using the first captured image prepared for learning, the second captured image prepared for learning, and the third captured image prepared for learning as training data, the third captured image is an image obtained by capturing an image of the subject illuminated with a light amount greater than the upper limit value, and a noise generated by the image capture signal processing is an image having a level of noise generated by the image capture signal processing in the first captured image.

18. An image generating device according to claim 17.

19. The trained model uses at least a portion of the imaging parameters as input data.

18. An image generating device according to claim 17.

20. The noise included in the estimated image is about the same as the noise generated in the first captured image by the imaging signal processing.

17. An image generating device according to claim 16.

21. noise generated in the second captured image by the imaging signal processing is greater than noise generated in the first captured image by the imaging signal processing; 17. The image generating device of claim 16.

22. the imaging signal processing includes an amplification process for the imaging signal, a gain of the imaging parameters for generating the second captured image is greater than a gain of the imaging parameters for generating the first captured image; 22. The image generating device of claim 21.

23. acquiring a first captured image generated by performing image signal processing on an image signal obtained by capturing an image of a subject illuminated with an amount of light equal to or less than an upper limit value, and a second captured image generated by performing image signal processing on an image signal obtained by capturing an image of the subject illuminated with an amount of light equal to or less than the upper limit value, the second captured image being brighter than the first captured image; generating an estimated image by estimating an image captured of the subject illuminated with an amount of light greater than the upper limit value from the first captured image and the second captured image; Image generation method.

24. generating the estimated image based on a trained model that has previously been machine-learned to determine a relationship between the first captured image, the second captured image, and the estimated image; 24. The image generating method of claim 23.

25. The trained model is a model that has been machine-learned using the first captured image prepared for learning, the second captured image prepared for learning, and the third captured image prepared for learning as training data, the third captured image is an image of the subject illuminated with a light amount greater than the upper limit value, and the third captured image is an image in which noise caused by the image capture signal processing is approximately the same as noise caused by the image capture signal processing in the first captured image.

25. The image generating method of claim 24.

26. generating a first captured image by capturing an image of a subject illuminated with a first light amount based on a first imaging parameter; generating a second captured image by capturing an image of the subject illuminated with the first light amount based on a second imaging parameter; generating an estimated image based on the first captured image and the second captured image; a processor configured to execute the second captured image has a luminance value greater than that of the first captured image, the estimated image is an image obtained by estimating an image obtained by capturing an image of the subject illuminated with a second amount of light that is greater than the first amount of light. Image generating device.

27. ​​The estimated image is generated based on a trained model that has previously been machine-learned to determine a relationship between the first captured image and the second captured image and the estimated image.

27. An image generating device according to claim 26.

28. The imaging parameters include an amplification process for the imaging signal, The gain of the second imaging parameter is greater than the gain of the first imaging parameter.

27. An image generating device according to claim 26.

29. The second captured image generated using the second imaging parameters contains more noise than the first captured image generated using the first imaging parameters.

29. The image generating device of claim 28.

30. The first light amount is a light amount that causes a temperature of an illumination unit that irradiates illumination light onto the subject to be equal to or lower than a predetermined temperature.

27. An image generating device according to claim 26.

31. The second light amount is an amount of light that causes the temperature of the illumination unit to be higher than a predetermined temperature.

31. The image generating device of claim 30.