Endoscope system, and control device, dontrol method, and control program for endoscope

Abstract

This endoscope system 1 comprises: an illumination light source 311 that emits illumination light; a treatment light source 612 that emits treatment light that causes a photoreactive reagent accumulated in a target site to react; an imaging unit that images return light of the illumination light from a subject and fluorescence generated by irradiating the subject with the treatment light; a light source control unit 44 that controls the operation of the illumination light source 311 and the treatment light source 612; and an imaging control unit 44 that causes the imaging unit to perform imaging in accordance with a specific frame period. When a first mode among a plurality of modes is set, the light source control unit 44 causes the emission of the illumination light from the illumination light source 311 and the emission of the treatment light from the treatment light source 612 to be performed in a time division manner in accordance with the specific frame period in a state in which the number of emissions of the treatment light per unit time is less than the number of emissions of the illumination light per unit time.

Description

Endoscope system, endoscope control device, control method, and control program 【0001】 The present invention relates to an endoscope system, an endoscope control device, a control method, and a control program. 【0002】 In recent years, research has progressed on photoimmunotherapy (PIT), a method in which an antibody drug that binds to the surface of cancer cells is administered to a subject, and then light in a wavelength range suitable for phototherapy (hereinafter referred to as "therapy light") is irradiated to specifically kill only cancer cells (see, for example, Patent Document 1). In this process, the antibody drug is excited by the therapy light and emits fluorescence. The intensity of this fluorescence can then be used, for example, to confirm the condition of the target site before, during, or after the treatment. Therefore, by imaging this fluorescence with an imaging device, it becomes possible to confirm the condition of the target site from the fluorescence intensity captured by the imaging device. 【0003】 International Publication No. 2021 / 038913 【0004】 However, when checking the condition of the target area before or after actual treatment, the treatment light is irradiated using the same mechanism as the treatment itself, which means there is a possibility that the treatment may progress unintentionally. Therefore, there is a need for a technology that can check the condition of the target area by irradiating it with treatment light while simultaneously suppressing the progression of the treatment. 【0005】 The present invention has been made in view of the above, and aims to provide an endoscopic system, an endoscopic control device, a control method, and a control program that can suppress the progression of treatment while confirming the condition of the target site by irradiating it with therapeutic light. 【0006】To solve the above-mentioned problems and achieve the objective, the endoscopic system according to the present invention comprises an illumination light source that emits illumination light to illuminate a subject, a therapeutic light source that emits therapeutic light to react with a photoreactive reagent accumulated in the target area, an imaging unit that images the reflected light of the illumination light from the subject and the fluorescence generated by the irradiation of the subject with the therapeutic light, a light source control unit that controls the operation of the illumination light source and the therapeutic light source, and an imaging control unit that causes the imaging unit to take images in accordance with a specific frame period, wherein when the light source control unit is set to the first mode among a plurality of modes, the number of times the therapeutic light is emitted per unit time is less than the number of times the illumination light is emitted per unit time, and the emission of the illumination light from the illumination light source and the emission of the therapeutic light from the therapeutic light source are performed in a time-division manner in accordance with the specific frame period. 【0007】 The endoscope control device according to the present invention is an endoscope control device to which an illumination light source, a therapeutic light source, and a scope having an imaging unit are each connected, wherein the illumination light source emits illumination light to illuminate a subject, the therapeutic light source emits therapeutic light to react with a photoreactive reagent accumulated in a target area, the imaging unit images the reflected light of the illumination light from the subject and the fluorescence generated by irradiating the subject with the therapeutic light, the endoscope control device comprises a light source control unit that controls the operation of the illumination light source and the therapeutic light source, and an imaging control unit that causes the imaging unit to take images in accordance with a specific frame period, wherein when the light source control unit is set to the first mode among a plurality of modes, the emission of the therapeutic light per unit time is less than the emission of the illumination light per unit time, and the emission of the illumination light from the illumination light source and the emission of the therapeutic light from the therapeutic light source are performed in a time-division manner in accordance with the specific frame period. 【0008】The control method according to the present invention is a control method performed by an endoscope control device, wherein the endoscope control device is connected to an illumination light source that emits illumination light to illuminate a subject, a therapeutic light source that emits therapeutic light to react with a photoreactive reagent accumulated in a target area, and a scope having an imaging unit that images the reflected light of the illumination light from the subject and the fluorescence generated by irradiating the subject with the therapeutic light, and the control method includes the steps of controlling the operation of the illumination light source and the therapeutic light source, and causing the imaging unit to take images in accordance with a specific frame period, wherein in the step of controlling the operation of the illumination light source and the therapeutic light source, when set to a first mode among a plurality of modes, the emission of the therapeutic light per unit time is less than the emission of the illumination light per unit time, and the emission of the illumination light from the illumination light source and the emission of the therapeutic light from the therapeutic light source are performed in a time-division manner in accordance with the specific frame period. 【0009】 The control program according to the present invention is a control program to be executed by the processor of an endoscope control device, wherein the endoscope control device is connected to an illumination light source that emits illumination light to illuminate a subject, a therapeutic light source that emits therapeutic light to react with a photoreactive reagent accumulated in a target area, and a scope having an imaging unit that images the reflected light of the illumination light from the subject and the fluorescence generated by irradiating the subject with the therapeutic light, and the control program causes the processor to execute the steps of controlling the operation of the illumination light source and the therapeutic light source, and causing the imaging unit to take images in accordance with a specific frame period, wherein in the step of controlling the operation of the illumination light source and the therapeutic light source, when set to a first mode among a plurality of modes, the emission of the therapeutic light per unit time is less than the emission of the illumination light per unit time, and the emission of the illumination light from the illumination light source and the emission of the therapeutic light from the therapeutic light source are performed in a time-division manner in accordance with the specific frame period. 【0010】According to the endoscopic system, endoscopic control device, control method, and control program of the present invention, the condition of the target site can be confirmed while suppressing the progression of treatment by irradiating it with light in the wavelength range for phototherapy. 【0011】 Figure 1 is a diagram illustrating the configuration of an endoscope system according to an embodiment. Figure 2 is a diagram illustrating the configuration of an endoscope system according to an embodiment. Figure 3 is a diagram illustrating the tip configuration of the endoscope. Figure 4 is a diagram illustrating an example of the absorption spectrum of an antibody drug. Figure 5 is a diagram illustrating an example of the excitation spectrum and fluorescence spectrum of an antibody drug, the wavelength spectrum of the therapeutic light, and the transmission characteristics of an optical filter. Figure 6 is a diagram illustrating the pre- and post-treatment fluorescence observation mode. Figure 7 is a diagram illustrating the pre- and post-treatment fluorescence observation mode. Figure 8 is a diagram illustrating the white image observation mode. Figure 9 is a diagram illustrating the in-treatment fluorescence observation mode. Figure 10 is a diagram illustrating modification 1 of the embodiment. Figure 11 is a diagram illustrating modification 2 of the embodiment. Figure 12 is a diagram illustrating modification 3 of the embodiment. 【0012】 The embodiments for carrying out the present invention (hereinafter referred to as "embodiments") will be described below with reference to the drawings. However, the present invention is not limited to the embodiments described below. Furthermore, in the drawings, the same parts are denoted by the same reference numerals. 【0013】 [Configuration of the Endoscope System] Figures 1 and 2 show the configuration of the endoscope system 1 according to an embodiment. Figure 3 is a diagram illustrating the tip configuration of the endoscope 2. The endoscope system 1 is used in the medical field and is a system that performs treatment while observing the inside of a subject (in vivo). As shown in Figures 1 and 2, this endoscope system 1 comprises an endoscope 2, a light source device 3, a processing device 4, a display device 5, and a treatment device 6. 【0014】 In this embodiment, the endoscope 2 is a so-called flexible endoscope. Part of this endoscope 2 is inserted into the living body, images are captured within the living body, and an image signal generated by the imaging is output. As shown in Figure 1, the endoscope 2 comprises an insertion section 21, an operating section 22, and a universal cord 23. 【0015】 The insertion portion 21 is a part that is inserted into the body and is at least partially flexible. As shown in Figures 1 and 2, the insertion portion 21 comprises a tip portion 24, a flexible curved portion 25 (Figure 1) composed of multiple curved pieces, and a long, flexible flexible tube portion 26 (Figure 1) connected to the base end of the curved portion 25. The tip portion 24 contains an image sensor 244 (Figure 2) in which multiple pixels, each generating a signal by receiving light and converting it into photoelectricity, are arranged in a two-dimensional manner in horizontal line units. The insertion portion 21 is inserted into the body cavity of the subject and uses the image sensor 244 to image subjects such as biological tissue located in a position where external light cannot reach. 【0016】 The operating unit 22 is connected to the proximal end portion of the insertion section 21. The operating unit 22 receives various operations on the endoscope 2. As shown in Figure 1, the operating unit 22 includes a bending knob 221 for bending the bending section 25 in the vertical and horizontal directions, a treatment instrument insertion section 222 for inserting treatment instruments such as a treatment light irradiation device, biopsy forceps, electrosurgical unit, and examination probe into the body cavity of the subject, and a plurality of switches 223 for operating peripheral equipment such as air supply means, water supply means, and screen display control, in addition to the processing unit 4. Treatment instruments inserted from the treatment instrument insertion section 222 emerge from the opening via the treatment instrument channel (not shown) of the tip section 24 (Figure 3). 【0017】The universal cord 23 incorporates at least a light guide 241 (Figure 2) and a bundled cable 245 (Figure 2) that bundles one or more signal lines. The light guide 241 is made of glass fiber or the like and forms a light guide path for the light emitted by the light source device 3. As shown in Figure 1, the universal cord 23 branches at the end opposite to the end connected to the operation unit 22. At the branched end of the universal cord 23, a connector 231 that can be attached to the light source device 3 and a connector 232 that can be attached to the processing unit 4 are provided. A part of the light guide 241 extends from the end of the connector 231. The universal cord 23 propagates the illumination light emitted from the light source device 3 to the tip 24 via the connector 231 (light guide 241), the operation unit 22, and the flexible tube section 26. The universal cord 23 also transmits the image signal captured by the image sensor 244 provided at the tip 24 to the processing unit 4 via the connector 232. The bundled cable 245 includes signal lines for transmitting image signals, signal lines for transmitting drive signals for driving the image sensor 244, and signal lines for sending and receiving information including unique information about the endoscope 2 (image sensor 244). In this embodiment, it is described as transmitting electrical signals using signal lines, but optical signals may be transmitted, or signals may be transmitted between the endoscope 2 and the processing unit 4 by wireless communication. 【0018】 The exit end of the light guide 241 is inserted into the tip portion 24. As shown in Figures 2 and 3, the tip portion 24 includes an illumination lens 242, a light-gathering optical system 243, and an image sensor 244 (Figure 2) provided at the imaging position of the optical system 243, which receives the light gathered by the optical system 243, converts it into an electrical signal via photoelectricity, and performs predetermined signal processing. The optical system 243 and the image sensor 244 constitute the imaging unit according to the present invention. 【0019】The optical system 243 is composed of one or more lenses and forms an observation image on the light-receiving surface of the image sensor 244. Note that the optical system 243 may have an optical zoom function for changing the angle of view and a focus function for changing the focus. This optical system 243 includes an optical filter 243a (Fig. 2). The optical characteristics of the optical filter 243a will be described in "Characteristics of Therapeutic Light, Antibody Drugs, Fluorescence, and Optical Filters" which will be described later. 【0020】 The image sensor 244 photoelectrically converts the light from the optical system 243 to generate an electrical signal (image signal). This image sensor 244 is formed by arranging a plurality of pixels each having a photodiode that accumulates charges according to the amount of light and a capacitor that converts the charges transferred from the photodiode into a voltage level in a matrix. Then, the image sensor 244 photoelectrically converts the light incident on each pixel through the optical system 243 to generate an electrical signal, and sequentially reads out the electrical signals generated by the pixels arbitrarily set as the readout targets among the plurality of pixels, and outputs them as an image signal. This image sensor 244 is realized using, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. 【0021】 In the following, for the sake of convenience of explanation, an image signal generated by the image sensor 244 imaging the return light of the illumination light (white light) from the subject will be referred to as a white light image. The white light image corresponds to the illumination light image according to the present invention. The return light of the white light is the white light irradiated on the subject and reflected by the subject. Also, in the following, an image signal generated by the image sensor 244 imaging the fluorescence generated by irradiating the subject with therapeutic light will be referred to as a fluorescence image. The fluorescence is the fluorescence emitted when the therapeutic light irradiates the subject and the antibody drug contained in the subject is excited. And there are cases where the white light image and the fluorescence image are collectively referred to as an imaging image. The characteristics of the therapeutic light, antibody drug, and fluorescence will be described in "Characteristics of Therapeutic Light, Antibody Drugs, Fluorescence, and Optical Filters" which will be described later. 【0022】Incidentally, the endoscope 2 has a memory (not shown) that stores an execution program and a control program for the imaging element 244 to execute various operations, and data including the identification information of the endoscope 2. The identification information includes the unique information (ID) of the endoscope 2, the model year, the specification information, the transmission method, and the like. Further, the memory may temporarily store an imaging image or the like generated by the imaging element 244. 【0023】 Here, in the present embodiment, in the endoscope system 1, as an observation mode, either a white image observation mode for observing an image obtained by illuminating with white light or a fluorescence observation mode for observing a fluorescence image obtained by illuminating with treatment light is set. The fluorescence observation mode includes a fluorescence observation mode during treatment and a fluorescence observation mode before and after treatment, which will be described later. Details of these white image observation mode, fluorescence observation mode during treatment, and fluorescence observation mode before and after treatment will be described in "Regarding Observation Modes" described later. 【0024】 As shown in FIG. 2, the light source device 3 includes a light source unit 31, an illumination control unit 32, and a light source driver 33. 【0025】 The light source unit 31 has a white light source 311 that emits illumination light under the control of the illumination control unit 32. The white light source 311 corresponds to the illumination light source according to the present invention. This white light source 311 emits light (white light (illumination light)) having a wavelength band of visible light. This white light source 311 is realized using any light source such as an LED (Light Emitting Diode) light source, a laser light source, a xenon lamp, or a halogen lamp. Further, the white light source 311 may have one or more lenses or the like. Then, the light generated by the white light source 311 is emitted from the tip of the distal end portion 24 toward the subject via the light guide 241 and the illumination lens 242. 【0026】 Here, the processing device 4 (control unit 44) notifies the illumination control unit 32 of the emission timing of white light. Then, the illumination control unit 32 controls the amount of electric power supplied to the white light source 311 and controls the emission timing of the white light from the white light source 311 in response to the notification. 【0027】The light source driver 33, under the control of the lighting control unit 32, supplies current to the light source to be emitted, thereby causing the light source unit 31 to emit light. 【0028】 The processing unit 4 corresponds to the endoscope control device according to the present invention. As shown in Figure 2, the processing unit 4 comprises an image processing unit 41, a synchronization signal generation unit 42, an input unit 43, a control unit 44, and a storage unit 45. 【0029】 The image processing unit 41 performs predetermined image processing on the captured image received from the endoscope 2 to generate a display image. This display image is output to the display device 5. As shown in Figure 2, the image processing unit 41 includes a white light image processing unit 411, a fluorescence image processing unit 412, and a superimposed image generation unit 413. 【0030】 The white light image processing unit 411 performs image processing on the white light image from the captured images received from the endoscope 2. 【0031】 The fluorescence image processing unit 412 performs image processing on the fluorescence image from the image captured by the endoscope 2. 【0032】 Examples of image processing performed by the white light image processing unit 411 and the fluorescence image processing unit 412 include optical black subtraction (clamping), white balance adjustment, demosaicing, color correction matrix processing, gamma correction, YC processing to convert RGB signals into luminance chromatic difference signals (Y, Cb / Cr signals), digital gain adjustment to multiply digital gain, noise reduction, and filtering to enhance structure. 【0033】 The superimposed image generation unit 413 generates a superimposed image by superimposing a fluorescence image onto a white light image. 【0034】 The image processing unit 41 described above is configured using a general-purpose processor such as a CPU (Central Processing Unit) or a dedicated processor such as an ASIC (Application Specific Integrated Circuit) that performs specific functions. 【0035】The synchronization signal generation unit 42 generates a clock signal (synchronization signal) that serves as the reference for the operation of the processing unit 4, and outputs the generated synchronization signal to the light source device 3, the image processing unit 41, the control unit 44, and the endoscope 2. Here, the synchronization signal generated by the synchronization signal generation unit 42 includes a horizontal synchronization signal and a vertical synchronization signal. As a result, the light source device 3, the image processing unit 41, the control unit 44, and the endoscope 2 operate in synchronization with each other based on the generated synchronization signal. 【0036】 The input unit 43 is implemented using a keyboard, mouse, switches, or touch panel, and accepts various operations to instruct the operation of the endoscope system 1. The input unit 43 may also include switches provided on the operation unit 22 or portable terminals such as external tablet computers. 【0037】 The control unit 44 corresponds to the processor according to the present invention. This control unit 44 is configured using a general-purpose processor such as a CPU or a dedicated processor such as an ASIC that performs various arithmetic circuits that execute specific functions. The control unit 44 has the functions of a light source control unit and an imaging control unit according to the present invention. The functions of the control unit 44 as a light source control unit and an imaging control unit will be explained later in the section "About the observation mode". 【0038】 The storage unit 45 stores data including various programs executed by the control unit 44 (including the control program according to the present invention) and various parameters necessary for processing by the control unit 44. The storage unit 45 also stores identification information of the processing unit 4. Here, the identification information includes the processing unit 4's unique information (ID), year of manufacture, and specifications. These various programs can be recorded on computer-readable recording media such as hard disks, flash memory, CD-ROMs, DVD-ROMs, and flexible disks and widely distributed. These various programs can also be obtained by downloading them via a communication network. The communication network referred to here can be implemented by existing public telephone networks, LANs (Local Area Networks), WANs (Wide Area Networks), etc., and can be wired or wireless. 【0039】 The storage unit 45 having the above configuration is implemented using a ROM (Read Only Memory) on which various programs are pre-installed, and a RAM or hard disk that stores calculation parameters and data for each process. 【0040】 In this embodiment, the light source device 3 and the processing device 4 were housed in separate enclosures, but this is not limited to this configuration; they may be housed integrally within the same enclosure. 【0041】 The display device 5 displays the display image received from the processing device 4 (image processing device 41) via a video cable. This display device 5 is configured using a monitor such as a liquid crystal or organic EL (Electro Luminescence). 【0042】 As shown in Figures 1 and 2, the treatment device 6 includes a treatment tool operating section 61 and a flexible treatment tool 62 extending from the treatment tool operating section 61. The treatment tool 62 used in PIT is a treatment light emitting section that emits therapeutic light. 【0043】 Here, the control unit 44 notifies the treatment instrument operation unit 61 (treatment instrument control unit 614) of the timing of emission of the therapeutic light and guide light. The treatment instrument operation unit 61 then controls the emission timing of the therapeutic light and guide light from the treatment instrument 62 in response to this notification. As shown in Figure 2, the treatment instrument operation unit 61 comprises an operation input unit 611, a therapeutic light source 612, a guide light source 613, and a treatment instrument control unit 614. Each light source 612, 613 is realized using a semiconductor laser, an LED, or the like. In the treatment device 6, the light source that emits therapeutic light may be provided in the treatment instrument 62 or in the treatment instrument operation unit 61. 【0044】 The operation input unit 611 is composed of, for example, a switch. The treatment instrument operation unit 61 causes the treatment instrument 62 to emit therapeutic light or guide light in response to input to the operation input unit 611 (for example, pressing a switch). 【0045】The therapeutic light source 612 is composed of a light source and one or more lenses, and emits therapeutic light by driving the light source. The light source provided in this therapeutic light source 612 emits light (therapeutic light) in a wavelength range that excites antibody drugs. In the case of PIT, for example, this therapeutic light is light in a wavelength range of 680 nm or higher, for example, light with a central wavelength of 690 nm. 【0046】 The guide light source 613 is composed of a light source and one or more lenses, and emits guide light when the light source is driven. The light source of this guide light source 613 emits light with a wavelength band shorter than the wavelength band of the light emitted by the therapeutic light source 612, and with a wavelength band shorter than the wavelength band blocked by the optical filter 243a. 【0047】 The illumination optical system of the treatment device 62 emits the therapeutic light and the guide light in such a manner that their irradiation positions are the same. For example, the irradiation area of ​​the therapeutic light and the irradiation area of ​​the guide light are approximately the same. The illumination optical system of the treatment device 62 may also be configured to change the irradiation range of the therapeutic light. For example, it may be configured with an optical system that can change the focal length under the control of the treatment device operation unit 61, or with a DMD (Digital Micromirror Device), etc., which can change the spot diameter of the light irradiated onto the subject and the shape of the irradiation range. 【0048】 The treatment device control unit 614 controls the emission timing of the treatment light and guide light from the treatment light source 612 and the guide light source 613, respectively, in response to notifications of the emission timing of the treatment light and guide light from the control unit 44. The treatment device control unit 614 also emits the treatment light and guide light from the treatment light source 612 and the guide light source 613, respectively, in response to user input to the operation input unit 611. This treatment device control unit 614 is configured using a general-purpose processor such as a CPU or a dedicated processor such as an ASIC that performs various calculation circuits to execute specific functions. 【0049】 [Characteristics of the therapeutic light, antibody drug, fluorescence, and optical filter] Next, the characteristics of the therapeutic light, antibody drug, fluorescence, and optical filter 243a will be described with reference to Figures 4 and 5. 【0050】Figure 4 shows an example of the absorption spectrum of an antibody drug. Specifically, Figure 4 shows the absorption spectrum of IRDye® 700DX as an example of an antibody drug. Note that in Figure 4, the normalized intensity is shown, with the intensity of the maximum peak set to 1. IRDye® 700DX has a first light absorption band with a peak at 690 nm in the wavelength band greater than 650 nm, and a second light absorption band with a peak at 350 nm in the wavelength band less than 450 nm. In particular, the first light absorption band corresponds to the wavelength band that serves as the target for reacting the antibody drug when performing phototherapy. Note that the second light absorption band is sometimes called the Soret band. 【0051】 Figure 5 shows an example of the excitation spectrum and fluorescence spectrum of the antibody drug, the wavelength spectrum of the therapeutic light, and the transmission characteristics of the optical filter 243a. Here, in Figure 5, curve F E This shows the absorption characteristics of IRDye® 700DX (see Figure 4). Curve F L This curve shows the intensity distribution of the laser light irradiated as therapeutic light. F This curve shows the intensity distribution of fluorescence generated when an antibody drug is excited. C This shows the absorption characteristics (transmittance) of the optical filter 243a. Note that in Figure 5, curve F E F L F F This shows the normalized intensity, where the intensity of the maximum peak is normalized to 1. 【0052】 The optical filter 243a has a curve F C It has the absorption characteristics shown. That is, the wavelength band of the optical filter 243a is set considering the tolerance of the central wavelength of the therapeutic light source 612, the spectral width, and the grazing incidence characteristics of the optical system 243 to the optical filter 243a, with 690 nm as the center. For example, the optical filter 243a blocks light in the wavelength band greater than 670 nm and less than 700 nm, and transmits light in other wavelength bands. 【0053】Light entering from the tip 24 is filtered out by the optical filter 243a, which cuts off light within the specified range. For example, due to the characteristics of the optical filter 243a, light in the wavelength range of the therapeutic light is not incident on the image sensor 244, while fluorescence is incident on the image sensor 244. 【0054】 [Regarding Observation Modes] Next, the observation modes will be described. The control unit 44 sets the observation mode of the endoscope system 1 to pre- and post-treatment fluorescence observation mode, white image observation mode, or in-treatment fluorescence observation mode in response to user operations on the input unit 43 by the user. These observation modes correspond to the modes according to the present invention. The pre- and post-treatment fluorescence observation mode corresponds to the first mode according to the present invention. 【0055】 The following describes the pre- and post-treatment fluorescence observation mode, the white image observation mode, and the intra-treatment fluorescence observation mode in that order. Furthermore, in the following description, the image sensor 244 is assumed to be composed of a global shutter type image sensor. 【0056】 [About Pre- and Post-Treatment Fluorescence Observation Mode] First, let me explain the pre- and post-treatment fluorescence observation mode. This mode is used to check the condition of the target area before or after the actual treatment. 【0057】Figures 6 and 7 illustrate the pre- and post-treatment fluorescence observation mode. Specifically, Figure 6(a) shows the imaging control of the image sensor 244, with the vertical axis representing the horizontal lines of the image sensor 244 (the top row represents the uppermost horizontal line (the first horizontal line), and the bottom row represents the lowermost horizontal line (the final line)), and the horizontal axis representing time. The rectangular area is the area that contributes to the generation of the image in one frame. In Figure 6(a), for the sake of explanation, the letters "WLI" are written within the rectangular area if the captured light is white light, and the letters "Fluorescence" are written if the captured light is fluorescent. The letters written in parentheses along with these letters represent the frame number. Figure 6(b) shows the light source control of the therapeutic light source 612, with the state of emitting therapeutic light represented by a thick line (ON). Figure 6(c) shows the light source control of the white light source 311, with the state of emitting white light represented by a thick line (ON). Figure 6(d) shows the update image of the superimposed image that constitutes the display image generated by the image processing unit 41. In Figure 6(d), for the sake of explanation, the words "Superimposed Image" are written within the rectangular area showing the superimposed image. The words written in parentheses with these words represent the frame number of the superimposed image. Figure 6(e) shows the update image of the white light image that constitutes the display image generated by the image processing unit 41. In Figure 6(e), for the sake of explanation, the words "WLI" are written within the rectangular area showing the white light image. The words written in parentheses with these words represent the frame number of the white light image. Figure 7 shows an example of a display image displayed on the display device 5. 【0058】 The control unit (imaging control unit) 44 performs imaging control using a so-called global shutter method, which involves exposing all horizontal lines of the image sensor 244 for the same period (total line exposure period TE) during one frame period, and then reading out signals from all horizontal lines at the same timing during the readout period TO after the total line exposure period TE. The frame period (frame cycle) is, for example, 1 / 60 [s] in the case of the NTSC system and 1 / 50 [s] in the case of the PAL system. 【0059】 Furthermore, the control unit (light source control unit) 44 controls the light source so that the number of times the therapeutic light is emitted per unit time from the therapeutic light source 612 is less than the number of times the white light is emitted per unit time from the white light source 311, and in accordance with the frame period in imaging by the image sensor 244, it performs light source control (notification of the timing of white light emission to the illumination control unit 32, and notification of the timing of therapeutic light and guide light emission to the treatment tool control unit 614). 【0060】 Specifically, in the example shown in Figure 6, the treatment device control unit 614, in response to a notification from the control unit 44, emits therapeutic light from the therapeutic light source 612 at a rate of once every four frames, with the emission time per emission set to be the same as the total line exposure period TE of the image sensor 244, as shown in Figure 6(b). In addition, the illumination control unit 32, in response to a notification from the control unit 44, emits white light from the white light source 311 at a rate of three times every four frames, with the emission time per emission set to be the same as the total line exposure period TE, as shown in Figure 6(c). 【0061】 Although not shown in Figure 6, the treatment device control unit 614, in response to a notification from the control unit 44, emits guide light from the guide light source 613 only while white light is being emitted from the white light source 311. 【0062】 Through the above imaging and light source control, when white light is emitted, the image sensor 244 captures the reflected white light from the subject. This generates a white light image. When therapeutic light is emitted, the image sensor 244 captures the fluorescence generated by irradiating the subject with therapeutic light. This generates a fluorescence image. In other words, the control unit 44 causes the image sensor 244 to capture the reflected white light and the fluorescence in a time-division manner in accordance with the frame period, with the number of frames capturing the reflected white light being greater than the number of frames capturing the fluorescence. 【0063】Then, the image processing unit 41 generates a superimposed image by superimposing the white light image and the fluorescence image, and generates a display image for arranging and displaying the superimposed image and the white light image side by side. As a result, on the display device 5, as shown in FIG. 7, the superimposed image G S and the white light image G W are displayed side by side. In FIG. 7, the white light image G W is displayed larger than the superimposed image G S , but this is not the only case, and the white light image G W may be displayed smaller than the superimposed image G S . 【0064】 Here, the superimposed image generation unit 413 generates the superimposed image G W by superimposing the fluorescence image and the white light image G S which is the immediately preceding or subsequent frame in time series with respect to the fluorescence image. Further, the superimposed image G S is updated every plurality of frames (every 4 frames in (d) of FIG. 6). 【0065】 More specifically, the superimposed image generation unit 413 makes the brightness of the fluorescence image 4 times brighter than the brightness of the white light image G W at a predetermined ratio, for example, at a ratio of the white light image G W , and superimposes this fluorescence image on the white light image G F while emphasizing the fluorescence image R W (region with high fluorescence intensity). At this time, since the fluorescence intensity is used, for example, to judge the treatment effect, for images of the same observation site, the degree of brightness enhancement is the same, and the fluorescence image R F is processed so that the bright and dark parts of the fluorescence can be distinguished without the brightness of the fluorescence image changing due to image processing. 【0066】 Also, the white light image G W is updated every frame as shown in (e) of FIG. 6. That is, the frame rate of display in the superimposed image G S is the white light image G WThe frame rate is lower than that of the display. Note that a white light image is not generated by the image sensor 244 at the time the therapeutic light is emitted, but the white light image G is the frame that is chronologically immediately preceding that timing. W The white light image G is copied. W So, the target area P is affected by the reflected and scattered light of white light. T The tissue in the observation area, including the treatment instrument 62, and the image TR of the instrument are visualized, and the irradiation range R of the guide light is also visualized. G This is depicted. 【0067】 Then, the operator sees the image (white light image G) displayed on the display device 5. W and superimposed image G S By confirming the target site P before or after actually performing the treatment, T Check the status. 【0068】 [White Image Observation Mode] Next, we will explain the white image observation mode. The white image observation mode is an observation mode for searching for the treatment location. 【0069】 Figure 8 is a diagram illustrating the white image observation mode. Specifically, Figures 8(a) to 8(c) correspond to Figures 6(a) to 6(c), respectively. The control unit (imaging control unit) 44 performs imaging control using a global shutter method, as shown in Figure 8(a), similar to the pre- and post-treatment fluorescence observation mode described above. 【0070】 Furthermore, the control unit (light source control unit) 44 prevents the emission of therapeutic light from the therapeutic light source 612 (Figure 8(b)) and continuously emits white light from the white light source 311 (Figure 8(c)) by performing light source control (notifying the lighting control unit 32 of the timing of white light emission, and notifying the treatment tool control unit 614 of the timing of therapeutic light and guide light emission). General white light dimming control (control of the white light emission time, etc.) is also performed. 【0071】 Although not shown in Figure 8, the treatment device control unit 614, in response to a notification from the control unit 44, continuously emits guide light from the guide light source 613, similar to the white light source 311. 【0072】Through the above imaging and light source control, as shown in Figure 8(a), the image sensor 244 captures the reflected white light from the subject in all frames, resulting in a white light image (for example, the white light image G shown in Figure 7). W This generates the white light image. As a result, the display device 5 displays the white light image. 【0073】 The surgeon then searches for the treatment position by checking the white light image displayed on the display device 5. 【0074】 [Fluorescence Observation Mode During Treatment] Next, we will explain the fluorescence observation mode during treatment. The fluorescence observation mode during treatment is an observation mode used to check the condition of the target area while actually performing treatment. 【0075】 Figure 9 is a diagram illustrating the fluorescence observation mode during treatment. Specifically, Figures 9(a) to 9(c) correspond to Figures 6(a) to 6(c), respectively. The control unit (imaging control unit) 44 performs imaging control using a global shutter method, as shown in Figure 9(a), similar to the pre- and post-treatment fluorescence observation mode described above. 【0076】 Furthermore, the control unit (light source control unit) 44 continuously emits therapeutic light from the therapeutic light source 612 (Figure 9(b)), and emits white light from the white light source 311 at a rate of once every two frames for a duration equal to the total line exposure period TE of the image sensor 244 (Figure 9(c)), performing light source control (notifying the illumination control unit 32 of the timing of white light emission, and notifying the treatment tool control unit 614 of the timing of therapeutic light and guide light emission). 【0077】Through the above imaging and light source control, when white light is emitted, the image sensor 244 captures both the reflected white light from the subject and the fluorescence generated by irradiating the subject with therapeutic light. The fluorescence intensity is very weak compared to the reflected white light. Therefore, a white light image is generated. When only therapeutic light is emitted, the image sensor 244 captures the fluorescence generated by irradiating the subject with therapeutic light. This generates a fluorescence image. In other words, the control unit 44 causes the image sensor 244 to alternately capture white light images and fluorescence images in a time-division manner in accordance with the frame period. 【0078】 The image processing unit 41 then generates a superimposed image, similar to the pre- and post-treatment fluorescence observation mode described above, and also generates a display image for displaying the superimposed image and the white light image side by side. As a result, the display device 5 displays the same display image (Figure 7) as the pre- and post-treatment fluorescence observation mode described above. In the in-treatment fluorescence observation mode, the superimposed image and the white light image are updated every two frames. The display image may be a display image showing the superimposed image and the white light image side by side, or it may be a display image showing only the superimposed image. 【0079】 The embodiment described above provides the following effects. The processing device 4 according to this embodiment notifies the white light source 311 and the treatment light source 612 of the timing of emission of white light and treatment light. As a result, each light source 311 and 612 operates in accordance with the notification. The processing device 4 also causes the image sensor 244 to take images in accordance with a specific frame period. Furthermore, when the observation mode of the endoscope system 1 is set to the pre- and post-treatment fluorescence observation mode, the processing device 4 causes the emission of white light from the white light source 311 and the emission of treatment light from the treatment light source 612 to be performed in a time-division manner in accordance with the specific frame period, such that the number of times treatment light is emitted per unit time is less than the number of times white light is emitted per unit time. Therefore, in the pre- and post-treatment fluorescence observation mode, the target area P is irradiated with treatment light. T This allows us to monitor the patient's condition while simultaneously suppressing the progression of the disease. 【0080】 In particular, the imaging unit according to the present invention comprises only one image sensor 244. The control unit 44 causes the image sensor 244 to capture the reflected white light from the subject and the fluorescence generated by irradiating the subject with therapeutic light in a time-resolved manner, in accordance with a specific frame period. This makes it possible to avoid increasing the diameter of the insertion unit 21. 【0081】 The processing apparatus 4 according to this embodiment displays a white light image G on the display device 5. W and superimposed image G S This generates a display image for showing the items side by side. Therefore, the operator can identify the target area P before or after the actual treatment. T The condition can be clearly confirmed. 【0082】 In this embodiment, the processing device 4 sets the duration of a single emission of white light to be the same as the total line exposure period TE when the observation mode of the endoscope system 1 is set to the pre- and post-treatment fluorescence observation mode. Therefore, during the frame period in which fluorescence generated by irradiating the subject with therapeutic light is imaged, the reflected light of the white light is not captured by the image sensor 244, and a good fluorescence image capturing only the fluorescence can be generated. 【0083】 In this embodiment, the processing device 4 sets the duration of a single irradiation of the treatment light to be the same as the total line exposure period TE when the observation mode of the endoscope system 1 is set to the pre- and post-treatment fluorescence observation mode. Therefore, the progression of the treatment can be kept to a minimum. 【0084】 (Other Embodiments) While embodiments for carrying out the present invention have been described so far, the present invention is not limited to the embodiments described above. In the embodiments described above, the imaging unit according to the present invention was equipped with only one image sensor, but it is not limited to this, and a configuration may be adopted that is equipped with multiple image sensors, such as an image sensor for imaging the reflected white light from the subject and an image sensor for imaging the fluorescence generated by irradiating the subject with therapeutic light. 【0085】In the embodiment described above, the configurations of the following modified examples 1 to 3 may also be adopted. Modified examples 1 to 3 will be described in order below. 【0086】 (Modification 1) Figure 10 is a diagram illustrating modification 1 of the embodiment. Specifically, Figures 10(a) to 10(c) correspond to Figures 6(a) to 6(c), respectively. In the embodiment described above, in the pre- and post-treatment fluorescence observation mode, the emission time of the treatment light per shot was set to the same time as the total line exposure period TE of the image sensor 244, but this is not limited to this. For example, as shown in Figure 10(b), the emission time of the treatment light per shot may be set to a time that exceeds the total line exposure period TE. Alternatively, for example, the emission time of the treatment light per shot may be set to a time that is less than the total line exposure period TE. 【0087】 Furthermore, in the embodiment described above, the emission time of white light per flash in the pre- and post-treatment fluorescence observation mode was set to the same time as the total line exposure period TE of the image sensor 244, but this is not limited to this. For example, the emission time of white light per flash may be set to a time less than the total line exposure period TE. 【0088】 Even when configured as in the modified example 1 described above, the same effects as those of the embodiment described above are achieved. 【0089】 (Modification 2) In the embodiment described above, the image sensor 244 was composed of a global shutter type image sensor, but it is not limited to this and may be composed of a rolling shutter type image sensor. 【0090】Figure 11 illustrates a modified example of the embodiment. Specifically, Figure 11(a) shows the imaging control of the image sensor 244, with the vertical axis representing the horizontal lines of the image sensor 244 (the top row represents the uppermost horizontal line (the first horizontal line), and the bottom row represents the lowermost horizontal line (the final line)), and the horizontal axis representing time. The parallelogram region is the region that contributes to the generation of the captured image in one frame. In Figure 11(a), for the sake of explanation, the letters "WLI" are written within the parallelogram region if the captured light is white light, and the letters "Fluorescent" are written if the captured light is fluorescent. The letters written in parentheses along with these letters represent the frame number. Figures 11(b) and 11(c) correspond to Figures 6(b) and 6(c), respectively. 【0091】 The control unit (imaging control unit) 44 performs imaging control using a so-called rolling shutter method, in which exposure of the image sensor 244 for one frame period is sequentially started for each horizontal line, and readout (readout period TO) is performed sequentially for each horizontal line after a predetermined period (so-called shutter speed) has elapsed from the start of exposure. The one frame period (frame period) is, for example, 1 / 60 [s] in the case of the NTSC system and 1 / 50 [s] in the case of the PAL system. 【0092】 Furthermore, the control unit (light source control unit) 44 controls the light source so that the number of times the therapeutic light is emitted per unit time from the therapeutic light source 612 is less than the number of times the white light is emitted per unit time from the white light source 311, and in accordance with the frame period in imaging by the image sensor 244, it performs light source control (notification of the timing of white light emission to the illumination control unit 32, and notification of the timing of therapeutic light and guide light emission to the treatment tool control unit 614). 【0093】Specifically, in the example shown in Figure 11, the treatment instrument control unit 614, in response to a notification from the control unit 44, emits treatment light from the treatment light source 612 at a rate of once every four frames, with the emission time per emission set to be the same as the total line exposure period TE of the image sensor 244, similar to the pre- and post-treatment fluorescence observation mode in the embodiment described above. The illumination control unit 32, in response to a notification from the control unit 44, emits white light from the white light source 311 at a rate of three times every four frames, with the emission time per emission set to be the same as the total line exposure period TE, similar to the pre- and post-treatment fluorescence observation mode in the embodiment described above. 【0094】 Although not shown in Figure 11, the treatment device control unit 614, in response to a notification from the control unit 44, emits guide light from the guide light source 613 only while white light is being emitted from the white light source 311. 【0095】 Through the above imaging and light source control, when white light is emitted, the image sensor 244 captures the reflected white light from the subject. This generates a white light image. When therapeutic light is emitted, the image sensor 244 captures the fluorescence generated when the subject is irradiated with therapeutic light. This generates a fluorescence image. In other words, the control unit 44 causes the image sensor 244 to capture the reflected white light and the fluorescence in a time-division manner in accordance with the frame period, with the number of frames capturing the reflected white light from the subject being greater than the number of frames capturing the fluorescence generated when the subject is irradiated with therapeutic light. 【0096】 The image processing unit 41 then generates a superimposed image, similar to the pre- and post-treatment fluorescence observation mode in the embodiment described above, and also generates a display image for displaying the superimposed image and the white light image side by side. As a result, the display device 5 displays a display image (Figure 7) similar to the pre- and post-treatment fluorescence observation mode in the embodiment described above. 【0097】 Even when configured as in the modified example 2 described above, the same effects as those of the embodiment described above are achieved. 【0098】(Modification 3) Figure 12 is a diagram illustrating modification 3 of the embodiment. Specifically, Figures 12(a) to 12(c) correspond to Figures 11(a) to 11(c), respectively. In modification 2 described above, the time per irradiation of the treatment light in the pre- and post-treatment fluorescence observation mode was set to the same time as the total line exposure period TE of the image sensor 244, but this is not limited to this. For example, as shown in Figure 12(b), the time per irradiation of the treatment light may be set to a time that exceeds the total line exposure period TE. Alternatively, for example, the time per irradiation of the treatment light may be set to a time that is less than the total line exposure period TE. 【0099】 Furthermore, in the modified example 2 described above, the emission time of white light per flash in the pre- and post-treatment fluorescence observation mode was set to the same time as the total line exposure period TE of the image sensor 244, but this is not limited to this. For example, the emission time of white light per flash may be set to a time less than the total line exposure period TE. 【0100】 Even when configured as in the modified example 3 described above, the same effects as those of the embodiment and modified example 2 described above are achieved. 【0101】 1 Endoscope system 2 Endoscope 3 Light source device 4 Processing device 5 Display device 6 Treatment device 21 Insertion section 22 Operation section 23 Universal cord 24 Tip section 25 Bending section 26 Flexible tube section 31 Light source section 32 Illumination control section 33 Light source driver 41 Image processing section 42 Synchronization signal generation section 43 Input section 44 Control section 45 Storage section 61 Treatment instrument operation section 62 Treatment instrument 221 Bending knob 222 Treatment instrument insertion section 223 Switch 231, 232 Connector 241 Light guide 242 Illumination lens 243 Optical system 243a Optical filter 244 Image sensor 245 Cable 311 White light source 411 White light image processing section 412 Fluorescence image processing section 413 Superimposed image generation section 611 Operation input section 612 Treatment light source 613 Guide light source 614 Treatment device control unit F C , F E , F F, F L curve G W White light image G S Superimposed image P T Target area: R F Fluorescence image R G Irradiation area TE Total line exposure period TO Readout period TR Image

Claims

1. An endoscopic system comprising: an illumination light source that emits illumination light to illuminate a subject; a therapeutic light source that emits therapeutic light to react with a photoreactive reagent accumulated in a target area; an imaging unit that images the reflected light of the illumination light from the subject and the fluorescence generated by irradiating the subject with the therapeutic light; a light source control unit that controls the operation of the illumination light source and the therapeutic light source; and an imaging control unit that causes the imaging unit to take images in accordance with a specific frame period, wherein, when the light source control unit is set to a first mode among a plurality of modes, the emission of therapeutic light per unit time is less than the emission of illumination light per unit time, and the emission of illumination light from the illumination light source and the emission of therapeutic light from the therapeutic light source are performed in time division in accordance with the specific frame period.

2. The endoscope system according to claim 1, wherein the imaging unit comprises only one image sensor, and the imaging control unit causes the image sensor to capture the reflected light of the illumination light and the fluorescence in a time-division manner in accordance with the specific frame period.

3. The endoscope system according to claim 1, further comprising an image processing unit that processes an illumination light image obtained by the imaging unit imaging the reflected light of the illumination light and a fluorescence image obtained by the imaging unit imaging the fluorescence, wherein the image processing unit generates a superimposed image by superimposing the illumination light image and the fluorescence image.

4. The endoscope system according to claim 3, wherein the image processing unit generates the superimposed image by superimposing the illumination light image, which is a frame immediately preceding or following the fluorescence image in chronological order, onto the fluorescence image.

5. The endoscope system according to claim 3, further comprising a display device for displaying at least one of the illumination light image and the superimposed image.

6. The endoscope system according to claim 5, wherein the image processing unit generates a display image for displaying the illumination light image and the superimposed image side by side on the display device.

7. The endoscopic system according to claim 1, wherein the light source control unit and the imaging control unit control the operation of the illumination light source, the therapeutic light source, and the imaging unit based on the same synchronization signal.

8. The endoscope system according to claim 1, wherein the imaging unit is a global shutter type in which a plurality of pixels are arranged two-dimensionally in units of horizontal lines, and includes an image sensor having a full line exposure period in which all of the horizontal lines of the effective pixel area are exposed simultaneously, and when set to the first mode, the time of one emission of the illumination light is the same as the full line exposure period.

9. The endoscope system according to claim 1, wherein the imaging unit is a global shutter type in which a plurality of pixels are arranged two-dimensionally in units of horizontal lines, and includes an image sensor having a full-line exposure period in which all of the horizontal lines in the effective pixel area are exposed simultaneously, and when set to the first mode, the time of one emission of the therapeutic light is the same as the full-line exposure period.

10. The endoscope system according to claim 1, wherein the imaging unit is a rolling shutter type in which a plurality of pixels are arranged two-dimensionally in units of horizontal lines, and includes an image sensor having a full line exposure period in which all of the horizontal lines of the effective pixel area are exposed simultaneously, and when set to the first mode, the time of one emission of the illumination light is the same as the full line exposure period.

11. The endoscope system according to claim 1, wherein the imaging unit is a rolling shutter type in which a plurality of pixels are arranged two-dimensionally in units of horizontal lines, and includes an image sensor having a full line exposure period in which all of the horizontal lines in the effective pixel area are exposed simultaneously, and when set to the first mode, the time of one emission of the therapeutic light is the same as the full line exposure period.

12. An endoscope control device to which an illumination light source, a therapeutic light source, and a scope having an imaging unit are each connected, wherein the illumination light source emits illumination light to illuminate a subject, the therapeutic light source emits therapeutic light to react with a photoreactive reagent accumulated in a target area, the imaging unit images the reflected light of the illumination light from the subject and the fluorescence generated by irradiating the subject with the therapeutic light, the endoscope control device comprises a light source control unit that controls the operation of the illumination light source and the therapeutic light source, and an imaging control unit that causes the imaging unit to take images in accordance with a specific frame period, wherein when the light source control unit is set to a first mode among a plurality of modes, the endoscope control device causes the emission of illumination light from the illumination light source and the emission of therapeutic light from the therapeutic light source to be performed in a time-division manner in accordance with the specific frame period, such that the number of times the therapeutic light is emitted per unit time is less than the number of times the illumination light is emitted per unit time.

13. A control method performed by an endoscope control device, wherein the endoscope control device is connected to an illumination light source that emits illumination light to illuminate a subject, a therapeutic light source that emits therapeutic light to react with a photoreactive reagent accumulated in a target area, and a scope having an imaging unit that images the reflected light of the illumination light from the subject and the fluorescence generated by irradiating the subject with the therapeutic light, the control method includes the steps of controlling the operation of the illumination light source and the therapeutic light source, and causing the imaging unit to take images in accordance with a specific frame period, wherein, when set to a first mode among a plurality of modes, the control method causes the emission of the illumination light from the illumination light source and the emission of the therapeutic light from the therapeutic light source to be performed in a time-division manner in accordance with the specific frame period, such that the number of times the therapeutic light is emitted per unit time is less than the number of times the illumination light is emitted per unit time.

14. A control program to be executed by the processor of an endoscope control device, wherein the endoscope control device is connected to an illumination light source that emits illumination light to illuminate a subject, a therapeutic light source that emits therapeutic light to react with a photoreactive reagent accumulated in a target area, and a scope having an imaging unit that images the reflected light of the illumination light from the subject and the fluorescence generated by irradiating the subject with the therapeutic light, and the control program causes the processor to execute the steps of controlling the operation of the illumination light source and the therapeutic light source, and causing the imaging unit to take images in accordance with a specific frame period, wherein in the step of controlling the operation of the illumination light source and the therapeutic light source, when set to a first mode among a plurality of modes, the control program causes the emission of the therapeutic light from the illumination light source and the emission of the therapeutic light from the therapeutic light source to be performed in a time-division manner in accordance with the specific frame period, such that the number of times the therapeutic light is emitted per unit time is less than the number of times the illumination light is emitted per unit time.

Images