Light treatment device, computer program product, and recording medium

Abstract

The light treatment device of the present application has: a treatment light emitting device that emits treatment light that causes a medicament to react; a tissue structure image acquisition section that acquires a tissue structure image obtained by irradiating narrowband light to an irradiation position of the treatment light; a fluorescent light image acquisition section that acquires a fluorescent light image obtained by irradiating excitation light to the irradiation position of the treatment light; a boundary region determination section that determines a boundary region in which a tissue structure has changed using the tissue structure image; a fluorescent light intensity change calculation section that calculates a size of a change in fluorescent light intensity of the boundary region; and a display image generation section that generates a display image for displaying the size of the change in fluorescent light intensity.

Description

Technical Field

[0001] This invention relates to phototherapy devices, phototherapy methods, and phototherapy procedures. Background Technology

[0002] In recent years, research has been conducted on photoimmunotherapy (PIT), which involves antibody agents binding to cancer cells. Irradiation with near-infrared light activates the antibody agents, thereby destroying the cancer cells and treating cancer (see, for example, Patent Document 1 and Non-Patent Document 1). The antibody agents irradiated with near-infrared light absorb the light energy, undergo molecular vibrations, and generate heat. This heat destroys the cancer cells. At this point, the antibody agents are excited and emit fluorescence. The intensity of this fluorescence is used as an indicator of treatment efficacy.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2017-71654

[0006] Non-patent literature

[0007] Non-patent literature 1T. Nagaya, et al., Cancer Science. 2018; 109: 1902-1908 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] However, the light source on the surface of cancer cells is uneven, so even when irradiated with the same amount of light, the amount of reaction within the tissue is sometimes uneven. Therefore, the rate of reaction also becomes uneven, with some areas reacting rapidly and others reacting slowly. In appropriate light irradiation, it is necessary to estimate the appropriate amount of light for each area with different reaction rates and set the light intensity for each area.

[0010] However, Patent Document 1 and Non-Patent Document 1 evaluate the therapeutic effect by measuring the reduction in fluorescence across the entire illuminated area, which sometimes fails to adequately assess the therapeutic effect. Since fluorescence images are typically blurred, localized changes are easily obscured in existing methods that observe the reduction in fluorescence across the entire illuminated area. In particular, the fluorescence reduction in reacted areas is sometimes obscured by the fluorescence of unreacted areas with high fluorescence intensity.

[0011] The present invention was made in view of the above circumstances, and its object is to provide a phototherapy device, a phototherapy method, and a phototherapy procedure capable of appropriately irradiating a treatment area with light.

[0012] Methods for solving problems

[0013] To address the aforementioned issues and achieve the objectives, the phototherapy apparatus of the present invention comprises: a therapeutic light emitting device that emits therapeutic light that causes a drug to react; a tissue structure image acquisition unit that acquires a tissue structure image obtained by a narrow band of light irradiating an irradiation position of the therapeutic light; a fluorescence image acquisition unit that acquires a fluorescence image obtained by excitation light irradiating an irradiation position of the therapeutic light; a boundary region determination unit that uses the tissue structure image to determine a boundary region where the tissue structure has changed; a fluorescence intensity change calculation unit that calculates the magnitude of the change in fluorescence intensity in the boundary region; and a display image generation unit that generates a display image for displaying the magnitude of the change in fluorescence intensity.

[0014] Furthermore, in the phototherapy device of the present invention, the boundary region determination unit detects the temporal changes in the tissue structure image and determines the region where the tissue structure has changed as the boundary region based on the amount of the temporal change.

[0015] Furthermore, in the phototherapy device of the present invention, the boundary region determination unit compares the value of the tissue structure image with a preset threshold, thereby determining the region where the tissue structure has changed as the boundary region.

[0016] Furthermore, in the phototherapy device of the present invention, the boundary region determination unit uses a feature quantity pre-calculated through machine learning to determine the region where the tissue structure has changed as the boundary region.

[0017] Furthermore, in the phototherapy apparatus of the present invention described above, the tissue structure image acquisition unit acquires a tissue structure image obtained by the narrowband light in the wavelength range of 380 nm or higher and 440 nm or lower.

[0018] Furthermore, in the above-described invention, the phototherapy device of the present invention further includes a fluorescence intensity normalization unit, which normalizes the fluorescence intensity calculated by the fluorescence intensity change calculation unit using the light intensity of the returned light of narrowband light in the wavelength band of 440 nm to 490 nm.

[0019] Furthermore, in the phototherapy apparatus of the present invention described above, the tissue structure image acquisition unit acquires a tissue structure image obtained by the narrowband light in the wavelength range of 490 nm or higher and 590 nm or lower.

[0020] Furthermore, in the phototherapy apparatus of the present invention described above, the tissue structure image acquisition unit acquires a tissue structure image obtained by the narrowband light in the wavelength range of 590 nm or higher and 620 nm or lower.

[0021] Furthermore, in the phototherapy apparatus of the present invention described above, the tissue structure image acquisition unit acquires a tissue structure image obtained by the narrowband light in the wavelength range of 620 nm or higher and 780 nm or lower.

[0022] In addition, in the above-described invention, the phototherapy device of the present invention further includes a control unit that uses the multiplication value of light irradiation intensity and irradiation time as a set irradiation light amount to control the emission of the treatment light to the target area of ​​the treatment light.

[0023] Furthermore, the phototherapy method of the present invention is used to confirm the therapeutic effect after irradiating the treatment site with therapeutic light that will cause a drug reaction. The phototherapy method includes the following steps: a tissue structure image acquisition step, acquiring a tissue structure image obtained by irradiating the irradiation position of the therapeutic light with narrow band light; a fluorescence image acquisition step, acquiring a fluorescence image obtained by irradiating the irradiation position of the therapeutic light with excitation light; a boundary region determination step, using the tissue structure image to determine the boundary region where the tissue structure has changed; a fluorescence intensity change calculation step, calculating the magnitude of the change in fluorescence intensity in the boundary region; and a display image generation step, generating a display image for displaying the magnitude of the change in fluorescence intensity.

[0024] Furthermore, the phototherapy procedure of the present invention enables the phototherapy device to perform the following steps, wherein the phototherapy device generates information to confirm the therapeutic effect after the therapeutic light that will cause the drug to react is irradiated onto the treatment site, the steps including: a tissue structure image acquisition step, acquiring a tissue structure image obtained by a narrow band of light irradiating the irradiation position of the therapeutic light; a fluorescence image acquisition step, acquiring a fluorescence image obtained by excitation light irradiating the irradiation position of the therapeutic light; a boundary region determination step, using the tissue structure image to determine the boundary region where the tissue structure has changed; a fluorescence intensity change calculation step, calculating the magnitude of the change in fluorescence intensity in the boundary region; and a display image generation step, generating a display image for displaying the magnitude of the change in fluorescence intensity.

[0025] Invention Effects

[0026] According to the present invention, it is possible to appropriately apply light irradiation to the treatment area. Attached Figure Description

[0027] Figure 1 This is a diagram showing a schematic structure of the endoscope system according to Embodiment 1 of the present invention.

[0028] Figure 2 This is a block diagram illustrating the schematic structure of an endoscope system according to Embodiment 1 of the present invention.

[0029] Figure 3 This is a diagram illustrating the front end structure of the endoscope according to Embodiment 1 of the present invention.

[0030] Figure 4 This is a diagram illustrating the structure of the imaging optical system of the endoscope according to Embodiment 1 of the present invention.

[0031] Figure 5 This is a diagram used to illustrate an example of the wavelength of light used as narrowband light.

[0032] Figure 6 This is a diagram illustrating an example of a treatment process using the endoscope of Embodiment 1 of the present invention.

[0033] Figure 7 This is a flowchart illustrating an example of the processing of the processing apparatus according to Embodiment 1 of the present invention.

[0034] Figure 8 It is a diagram illustrating the regions defined by boundary area determination.

[0035] Figure 9 This is a diagram illustrating an example of the fluorescence intensity transition when the reaction proceeds at a slow rate.

[0036] Figure 10 This is a diagram illustrating an example of the fluorescence intensity transition when the reaction proceeds at a rapid rate.

[0037] Figure 11 This is a block diagram illustrating the schematic structure of an endoscope system according to a modified embodiment 1 of the present invention.

[0038] Figure 12 This is a diagram illustrating the structure of the imaging optical system of an endoscope according to a modified embodiment 1 of the present invention.

[0039] Figure 13 This is a block diagram illustrating the schematic structure of the endoscope system according to Embodiment 2 of the present invention.

[0040] Figure 14 This is a diagram illustrating the structure of the imaging optical system of the endoscope according to Embodiment 2 of the present invention.

[0041] Figure 15 It is a diagram that schematically shows the image obtained by the first imaging element.

[0042] Figure 16 This is a schematic diagram showing an image obtained by a third camera element.

[0043] Figure 17 It is used to illustrate the passage Figure 15 The image shown is the same as Figure 16The diagram shows the boundary region defined by the sum of the images shown.

[0044] Figure 18 This is a block diagram illustrating the schematic structure of the endoscope system according to Embodiment 3 of the present invention.

[0045] Figure 19 This is a diagram illustrating the structure of the imaging optical system of the endoscope according to Embodiment 3 of the present invention.

[0046] Figure 20 This is a diagram illustrating the structure of the imaging optical system of the endoscope according to Embodiment 4 of the present invention. Detailed Implementation

[0047] Hereinafter, embodiments for carrying out the present invention (hereinafter referred to as "implementations") will be described. In these embodiments, as an example of a system including the phototherapy apparatus of the present invention, a medical endoscope system for capturing and displaying images of a patient or other subject body will be described. However, the present invention is not limited to this embodiment. Furthermore, in the accompanying drawings, the same reference numerals will be used to refer to the same parts.

[0048] (Implementation Method 1)

[0049] Figure 1 This is a diagram showing a schematic structure of the endoscope system according to Embodiment 1 of the present invention. Figure 2 This is a block diagram showing the general structure of the endoscope system of Embodiment 1. Figure 3 This is a diagram illustrating the front end structure of the endoscope in Embodiment 1.

[0050] Figure 1 and Figure 2 The endoscope system 1 shown includes: an endoscope 2 that takes internal images of the subject by inserting its tip into the subject's body; a light source device 3 that generates illumination light emitted from the tip of the endoscope 2; a processing device 4 that performs prescribed signal processing on the image signals obtained by the endoscope 2 and uniformly controls the overall operation of the endoscope system 1; a display device 5 that displays the internal images generated by the signal processing of the processing device 4; and a treatment device 6.

[0051] The endoscope 2 has: an insertion part 21, which is in a flexible, elongated shape; an operation part 22, which is connected to the base end of the insertion part 21 and receives various operation signals; and a universal cable 23, which extends from the operation part 22 in a direction different from the direction of the insertion part 21, and has various cables built in for connection with the light source device 3 and the processing device 4.

[0052] The insertion part 21 has: a front end 24, which houses an imaging element 244 formed by arranging pixels in a two-dimensional shape, the pixels generating signals by receiving light and performing photoelectric conversion; a flexible bending part 25 composed of multiple bending blocks; and a long, flexible tube part 26 connected to the base end of the bending part 25, which is flexible. The insertion part 21 is inserted into the body cavity of the subject, and the imaging element 244 captures images of biological tissues or other subjects located in areas inaccessible to external light.

[0053] The operating unit 22 includes: a bending knob 221 that bends the bending portion 25 in the up-down and left-right directions; a treatment instrument insertion portion 222 that inserts treatment instruments such as a therapeutic light irradiation device, biopsy forceps, electrosurgical scalpel, and examination probe into the body cavity of the subject; and multiple switches 223, which are operation input portions for inputting operation indication signals into the treatment device 4 and peripheral equipment such as the air supply unit, water supply unit, and screen display control. Treatment instruments inserted through the treatment instrument insertion portion 222 protrude from the opening via the treatment instrument channel (not shown) at the front end 24 (see reference). Figure 3 ).

[0054] The universal cable 23 has at least a built-in light guide 241 and a bundled cable 245 that aggregates one or more signal lines. The universal cable 23 branches at an end opposite to the side connected to the operating section 22. At the branch end of the universal cable 23 are a connector 231 that is detachable from the light source device 3 and a connector 232 that is detachable from the processing device 4. A portion of the light guide 241 extends from the end of the connector 231. The universal cable 23 propagates illumination light emitted from the light source device 3 to the front end portion 24 via the connector 231 (light guide 241), the operating section 22, and the flexible tube portion 26. Additionally, the universal cable 23 transmits image signals captured by the imaging element 244 located at the front end portion 24 to the processing device 4 via the connector 232. The bundled cable 245 includes signal lines for transmitting imaging signals, signal lines for transmitting drive signals for driving the imaging element 244, and signal lines for transmitting and receiving information including inherent information related to the endoscope 2 (imaging element 244). In addition, in this embodiment, an electrical signal is transmitted using a signal line for explanation, but an optical signal can also be transmitted, and a signal can also be transmitted between the endoscope 2 and the processing device 4 via wireless communication.

[0055] The front end portion 24 includes: a light guide 241, which is made of glass fiber or the like, and forms a light guide path for the light emitted by the light source device 3; an illumination lens 242, which is disposed at the front end of the light guide 241; an optical system 243 for focusing light; and an imaging element 244, which is disposed at the imaging position of the optical system 243, receives the light focused by the optical system 243 and converts it into an electrical signal to perform the prescribed signal processing.

[0056] The optical system 243 is constructed using one or more lenses. The optical system 243 forms an image on the light-receiving surface of the imaging element 244. The optical system 243 may have an optical zoom function that changes the viewing angle and a focusing function that changes the focal point.

[0057] The imaging element 244 performs photoelectric conversion on light from the optical system 243 to generate an electrical signal (image signal). Specifically, the imaging element 244 has two imaging elements (a first imaging element 244a and a second imaging element 244b). The first imaging element 244a and the second imaging element 244b are formed by arranging multiple pixels in a matrix. Each of these pixels has a photodiode that stores a charge corresponding to the amount of light, a capacitor that converts the charge transferred from the photodiode into a voltage level, etc. In the first imaging element 244a and the second imaging element 244b, each pixel performs photoelectric conversion on light incident via the optical system 243 to generate an electrical signal. The electrical signal generated by any pixel selected as the readout object is read out sequentially, and it is output as an image signal. The first imaging element 244a and the second imaging element 244b can be implemented, for example, using a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0058] Figure 4 This diagram illustrates the structure of the imaging optical system of the endoscope according to Embodiment 1. The optical system 243 and the imaging element 244 are disposed inside the front end portion 24.

[0059] The optical system 243 includes an objective lens 243a, a beam splitter 243b, and a cutoff filter 243c, all composed of one or more optical elements. The cutoff filter 243c blocks light in the excitation wavelength range. This excitation light corresponds to the wavelength range used to excite the antibody drug in PIT. In addition to the aforementioned optical elements, the optical system 243 may also include lenses, etc. Alternatively, a beam splitter such as a semi-reflective mirror may be used instead of the beam splitter 243b.

[0060] Light from the subject is incident on the beam splitter 243b via the objective lens 243a. Here, it is preferable that the distance from the light passing / reflecting position in the beam splitter 243b to the light-receiving surface of each imaging element (first imaging element 244a and second imaging element 244b) is the same.

[0061] Beam splitter 243b bends the optical path of light with wavelengths greater than that of the excitation light, while allowing light with wavelengths less than that of the excitation light to pass through. In other words, beam splitter 243b bends the optical path of the excitation light and its fluorescence that excites the subject. The light passing through beam splitter 243b is incident on the first imaging element 244a. On the other hand, the excitation light and the excitation light in the fluorescence, whose optical paths have been bent by beam splitter 243b, are cut off by cutoff filter 243c, and the fluorescence is incident on the second imaging element 244b.

[0062] Here, the transmittance of the excitation light of the cutoff filter 243c is set to, for example, 0.1% or less. By making the transmittance of the excitation light of the cutoff filter 243c 0.1% or less, fluorescence can be selectively captured during excitation light illumination.

[0063] In addition, the first imaging element 244a corresponds to the tissue structure image acquisition unit, and the cut-off filter and the second imaging element 244b correspond to the fluorescence image acquisition unit.

[0064] Additionally, the endoscope 2 has a memory (not shown) that stores execution programs and control programs for causing the imaging element 244 to perform various actions, as well as data containing the endoscope 2's identification information. The identification information includes the endoscope 2's inherent information (ID), year, specifications, and transmission method, etc. Furthermore, the memory can also temporarily store image data generated by the imaging element 244.

[0065] The structure of the light source device 3 will be described. The light source device 3 includes a light source unit 31, an illumination control unit 32, and a light source driver 33. Under the control of the illumination control unit 32, the light source unit 31 sequentially switches the illumination light and emits the illumination light onto the subject (examined object).

[0066] The light source unit 31 is constructed using a light source, one or more lenses, etc., and emits light (illumination light) by driving the light source. The light generated by the light source unit 31 is emitted from the front end of the front end 24 towards the subject via the light guide 241. The light source unit 31 has a white light source 311, a narrow band light source 312, and an excitation light source 313.

[0067] The white light source 311 emits light (white light) in the visible light spectrum. The white light source 311 can be implemented using any light source such as an LED light source, a laser light source, a xenon lamp, or a halogen lamp.

[0068] The narrowband light source 312 emits light consisting of a portion of the wavelengths or bands in the visible light region (narrowband light). Figure 5 This diagram illustrates an example of a wavelength band used as narrowband light. Narrowband light refers to light in the band between 380 nm and 440 nm. V Light in the wavelength band above 440nm and below 490nmB Light in the wavelength band above 490nm and below 590nm G Light in the wavelength band above 590nm and below 620nm A And light in the wavelength band above 620nm and below 780nm R Light is any light source or a combination of any portion thereof. Examples of narrowband light include light used in NBI (Narrow Band Imaging) observations, consisting of wavelengths from 380 nm to 440 nm centered at 415 nm, and wavelengths from 490 nm to 590 nm centered at 540 nm. The narrowband light source 312 is implemented using LED light sources, laser light sources, etc.

[0069] Additionally, when using antibody agents to stimulate PIT, for example, near-infrared light with a center wavelength of 690 nm... P .

[0070] Here, by irradiating light in the wavelength range of 380nm to 440nm and obtaining its scattered and reflected light, blood vessels in the mucosal surface can be depicted with high contrast. Furthermore, by irradiating light in the wavelength ranges of 490nm to 590nm, 590nm to 620nm, or 620nm to 780nm and obtaining its scattered and reflected light, deeper blood vessels in the mucosal surface can be depicted with high contrast.

[0071] In addition, light in the band above 440 nm and below 490 nm is used, for example, as reference light for generating images to correct fluorescence intensity, in addition to vascular mapping.

[0072] In addition, when using light in the wavelength range of 620nm and above and 780nm and below, the optical system 243 is configured to replace the beam splitter 243b with a semi-reflective mirror, or the optical system 243 directly uses the electrical signal generated by the second imaging element 244b.

[0073] The excitation light source 313 emits excitation light to excite the object to be excited (e.g., an antibody drug if it is a PIT). The excitation light source 313 can be implemented using a light source such as an LED light source or a laser light source. When stimulating the antibody drug of the PIT, for example, near-infrared light is used... P .

[0074] The lighting control unit 32 controls the electrical power supplied to the light source unit 31 based on the control signal (dimming signal) from the processing device 4, and controls the light source and the timing of the light source's drive.

[0075] Under the control of the lighting control unit 32, the light source driver 33 supplies current to the light source of the light-emitting object, thereby causing the light source unit 31 to emit light.

[0076] The structure of the processing device 4 will be described. The processing device 4 includes an image processing unit 41, a synchronization signal generation unit 42, an input unit 43, a control unit 44, and a storage unit 45.

[0077] The image processing unit 41 receives image data of various colors of illumination light captured by the imaging element 244 from the endoscope 2. When receiving analog image data from the endoscope 2, the image processing unit 41 performs A / D conversion to generate a digital imaging signal. Furthermore, when receiving image data as a light signal from the endoscope 2, the image processing unit 41 performs photoelectric conversion to generate digital image data.

[0078] The image processing unit 41 performs prescribed image processing on the image data received from the endoscope 2 to generate an image and output it to the display device 5, or sets a boundary region determined based on the image, or calculates the time change of fluorescence intensity. The image processing unit 41 includes a boundary region determination unit 411, a fluorescence intensity change calculation unit 412, and a display image generation unit 413.

[0079] The boundary region determination unit 411 determines the boundary between the changed tissue structure and the unchanged or minimally changed tissue structure based on the image (tissue structure image) formed by narrow-band light generated from the imaging signal generated by the first imaging element 244a. By determining the boundary, the boundary region determination unit 411 determines the boundary regions of the changed tissue structure and the unchanged or minimally changed tissue structure.

[0080] The fluorescence intensity change calculation unit 412 calculates the time change of fluorescence intensity for each boundary region based on the second image, which is generated by the second imaging element 244b based on the fluorescence image.

[0081] The display image generation unit 413 generates images by performing prescribed image processing. In addition to images based on white light and narrowband light, the images include images representing boundaries determined by the boundary region determination unit 411, images corresponding to changes in fluorescence intensity calculated by the fluorescence intensity change calculation unit, and images that impart visual information about the fluorescence intensity itself. Here, the prescribed image processing includes simulcasting, grayscale correction, and color correction. Simulcasting is the process of simulcasting image data for each of the RGB color components. Grayscale correction is the process of correcting the grayscale of the image data. Color correction is the process of correcting the color tone of the image data. Furthermore, the display image generation unit 413 can also adjust the gain based on the brightness of the image.

[0082] The image processing unit 41 is configured using a general-purpose processor such as a CPU (Central Processing Unit) or a special-purpose processor such as an ASIC (Application Specific Integrated Circuit) that performs specific functions. Alternatively, the image processing unit 41 may be configured to have a frame memory that stores R image data, G image data, and B image data.

[0083] The synchronization signal generation unit 42 generates a clock signal (synchronization signal) that serves as a reference for the operation of the processing device 4, and outputs the generated synchronization signal to the light source device 3, the image processing unit 41, the control unit 44, and the endoscope 2. Here, the synchronization signal generated by the synchronization signal generation unit 42 includes a horizontal synchronization signal and a vertical synchronization signal.

[0084] Therefore, the light source device 3, the image processing unit 41, the control unit 44, and the endoscope 2 operate synchronously with each other based on the generated synchronization signal.

[0085] The input unit 43 uses a keyboard, mouse, switch, or touch panel to receive various signals, such as motion indication signals that instruct the endoscope system 1 to move. Alternatively, the input unit 43 may include a switch located on the operation unit 22, or an external portable terminal such as a tablet computer.

[0086] The control unit 44 performs drive control of each structural component including the imaging element 244 and the light source device 3, as well as input / output control of information for each structural component. The control unit 44 refers to control information data (e.g., readout timing) stored in the storage unit 45 for image control and sends it as a drive signal to the imaging element 244 via a predetermined signal line included in the cable 245. Alternatively, it switches between a normal viewing mode and a fluorescence viewing mode. In the normal viewing mode, it observes the image obtained under white light illumination, and in the fluorescence viewing mode, it calculates the fluorescence intensity of the excitation object. The control unit 44 is configured using a general-purpose processor such as a CPU, or a dedicated processor such as an ASIC that performs specific functions using various arithmetic circuits.

[0087] The storage unit 45 stores various programs used to operate the endoscope system 1, as well as data including various parameters required for the operation of the endoscope system 1. Additionally, the storage unit 45 stores identification information for the processing device 4. This identification information includes the processing device 4's unique ID, year, and specifications.

[0088] In addition, the storage unit 45 stores various programs including image acquisition processing programs for executing the image acquisition processing method of the processing device 4. These programs can also be widely distributed by recording on computer-readable recording media such as hard disks, flash memory, CD-ROMs, DVD-ROMs, and floppy disks. Furthermore, these programs can also be obtained by downloading via a communication network. The communication network mentioned here can be, for example, existing public landline networks, LANs (Local Area Networks), WANs (Wide Area Networks), etc., and can be wired or wireless.

[0089] The storage unit 45 with the above structure is implemented using ROM (Read Only Memory) pre-installed with various programs, RAM, hard disk, etc., which store the operation parameters and data of each process.

[0090] The display device 5 displays a display image corresponding to the image signal received from the processing device 4 (image processing unit 41) via the image cable. The display device 5 is configured using a monitor such as a liquid crystal or an organic EL (electroluminescence) display.

[0091] The treatment device 6 includes a treatment device operation unit 61 and a flexible treatment device 62 extending from the treatment device operation unit 61. The treatment device 62 used in PIT emits light for treatment (hereinafter referred to as therapeutic light). The treatment device operation unit 61 controls the emission of the therapeutic light from the treatment device 62. The treatment device operation unit 61 includes an operation input unit 611. The operation input unit 611 is, for example, a switch. The treatment device operation unit 61 causes the treatment device 62 to emit the therapeutic light by inputting to the operation input unit 611 (e.g., pressing a switch). Furthermore, in the treatment device 6, the light source emitting the therapeutic light can be provided in the treatment device 62 or in the treatment device operation unit 61. The light source is implemented using a semiconductor laser, LED, etc. For example, in the case of PIT, the therapeutic light is light in the wavelength range of 680nm or higher, for example, light with a center wavelength of 690nm (e.g.). Figure 5 The light L shown P ).

[0092] Here, the illumination optical system of the treatment device 62 can change the irradiation range of the therapeutic light. For example, under the control of the treatment device operation unit 61, it can be composed of an optical system or DMD (Digital Micromirror Device) that can change the focal length, the diameter of the light spot irradiating the subject, and the shape of the irradiation range.

[0093] Next, refer to Figure 6 and Figure 7 The procedure for treatment using endoscopy 2 is explained. Figure 6 This is a diagram illustrating an example of a treatment process using the endoscope of Embodiment 1 of the present invention. Figure 6 This diagram illustrates an example of the implementation of PIT, in which the insertion part 21 is inserted into the gastric ST for treatment.

[0094] First, the surgeon inserts the insertion part 21 into the gastric ST (refer to...). Figure 6 (a) At this time, the surgeon illuminates the light source device 3 with white light and explores the treatment location while observing the white light image of the gastric ST displayed on the display device 5. Here, treatment is performed on tumors B1 and B2, which are the targets of treatment. The surgeon observes the white light image and determines the area containing tumors B1 and B2 as the irradiation area.

[0095] The surgeon positions the anterior end 24 toward tumor B1, allowing the treatment instrument 62 to protrude from the tip of the endoscope 2 and irradiate the tumor B1 with therapeutic light (see reference). Figure 6 (b)). By irradiating with therapeutic light, the antibody agent that binds to tumor B1 reacts, thus treating tumor B1.

[0096] Then, the surgeon positions the tip 24 toward the tumor B2, causing the treatment instrument 62 to protrude from the tip of the endoscope 2 and irradiate the tumor B2 with therapeutic light (see reference). Figure 6 (c)). By irradiating with therapeutic light, the antibody agent that binds to tumor B2 reacts, thus treating tumor B2.

[0097] Then, the surgeon positions the anterior end 24 toward tumor B1 and irradiates excitation light onto tumor B1 from the tip of endoscope 2 (see reference). Figure 6 (d) The surgeon confirms the therapeutic effect on tumor B1 by observing the fluorescence intensity. The confirmation of the therapeutic effect is made by the surgeon using the images described later.

[0098] Additionally, the surgeon positions the anterior end 24 towards tumor B2 and irradiates excitation light onto tumor B2 from the tip of endoscope 2 (see reference). Figure 6 (e)). The surgeon confirmed the treatment effect on tumor B2 by observing the fluorescence intensity.

[0099] The surgeon may repeatedly apply additional therapeutic light as needed and confirm the therapeutic effect.

[0100] Next, refer to Figure 7 The processing in processing device 4 will be explained. Figure 7 This is a flowchart illustrating an example of the processing by the processing apparatus of Embodiment 1. Figure 6 same, Figure 7 This shows an example of the process for implementing PIT.

[0101] First, before the treatment light is applied, a narrow-band light is irradiated from the front end 24 toward the treatment position to obtain a tissue structure image before treatment (step S101: tissue structure image acquisition process). Here, in the processing device 4, a tissue structure image is generated based on the imaging signal generated by the first imaging element 244a.

[0102] Then, the light source device 3 emits excitation light to detect the fluorescence of the antibody drug (step S102: fluorescence detection process). Through the emission of excitation light, the endoscope 2 illuminates the subject, exciting the antibody drug before treatment and causing it to emit fluorescence. At this time, the processing device 4 acquires the imaging signal (fluorescence image) generated by the second imaging element 244b.

[0103] Then, through the operator's manipulation, therapeutic light is irradiated from the treatment instrument 62 onto the antibody drug that binds to the cancer cells, causing the drug to react (step S103: drug reaction process). In this drug reaction process, the treatment is performed as follows: the antibody drug is activated and destroyed by irradiation with near-infrared light, which is used as the therapeutic light.

[0104] Then, narrow-band light is irradiated from the front end 24 toward the treatment position to obtain a tissue structure image after treatment (step S104: tissue structure image acquisition process). In step S104, the processing device 4 also generates a tissue structure image based on the imaging signal generated by the first imaging element 244a, just like in step S101.

[0105] Then, the light source device 3 emits excitation light to detect the fluorescence of the antibody drug (step S105: fluorescence detection process). In step S105, the processing device 4 also acquires the imaging signal (fluorescence image) generated by the second imaging element 244b in the same way as in step S102.

[0106] Additionally, the boundary region determination unit 411 uses the tissue structure image obtained in step S101 and the tissue structure image obtained in step S103 to detect the boundary between the region with a fast reaction rate and the region with a slow reaction rate, thereby determining the boundary region (step S106: boundary region determination process). Furthermore, the boundary region determination process can be performed before the fluorescence detection process or simultaneously with the fluorescence detection process.

[0107] Here, the determination process performed by the boundary region determination unit 411 will be described. The boundary region determination unit 411 determines the boundary region, for example, by any of the determination processes in determination process 1 and determination process 2 below. In addition, known methods other than determination process 1 and determination process 2 may also be used to determine the boundary region.

[0108] [Decision Processing 1]

[0109] The boundary region determination unit 411 detects the temporal changes of two tissue structure images acquired at different times. Based on the amount of this temporal change, it determines the region whose boundary is the outer edge of the area containing the tissue structure change as a boundary region. For example, the boundary region determination unit 411 compares the value (brightness value) of the tissue structure image with a preset threshold, extracts the area where the tissue structure change has occurred, and determines it as a boundary region whose boundary is the outer edge of the extracted area. This threshold can be a preset brightness value for a normal state (a state without tumors) or a brightness value from a tissue structure image acquired before treatment.

[0110] [Decision Processing 2]

[0111] The boundary region determination unit 411 uses feature quantities pre-calculated through machine learning to determine the regions where tissue structure has changed as boundary regions. The boundary region determination unit 411 calculates feature quantities of the acquired tissue structure image and uses the calculated feature quantities and the learning model to determine the boundary regions.

[0112] Figure 8 This diagram illustrates the regions defined by boundary region determination. The boundary region determination unit 411 compares the tissue structure image, identifies regions with large tissue changes as regions with fast reaction speeds, and regions with small tissue changes as regions with slow reaction speeds, detects their boundaries, and determines the boundary regions. For example, the boundary region determination unit 411 sets the first region ROI1 as a region with slow reaction speed and the second region ROI2 as a region with fast reaction speed.

[0113] Figure 9 This is a diagram illustrating an example of the fluorescence intensity transition when the reaction proceeds at a slow rate. Figure 10 This is a diagram illustrating an example of fluorescence intensity transitions when the reaction proceeds rapidly. In regions with slow reaction rates (e.g., the first region ROI1), the fluorescence intensity Q1 based on the antibody agent decays slowly and remains high over time (see reference). Figure 9 On the other hand, in regions with fast reaction rates (e.g., the second region ROI2), the decay rate of antibody-based fluorescence intensity Q2 is large (see reference). Figure 10 ).

[0114] Furthermore, the fluorescence intensity change calculation unit 412 uses the fluorescence images obtained in step S102 and step S105 to calculate the fluorescence intensity change (step S107: fluorescence intensity change calculation process). The fluorescence intensity change calculation unit 412 calculates the fluorescence intensity change (the difference between the fluorescence intensity before and after treatment) for each region determined by the boundary region determination unit 411. Alternatively, known methods such as pattern matching can be used to align the images before and after treatment.

[0115] Subsequently, the display image generation unit 413 generates an image to be displayed on the display device 5 (step S108). The display image generation unit 413 generates an image that visually represents the change in fluorescence intensity. For example, the display image generation unit 413 generates an image obtained by overlaying visual information corresponding to the change in fluorescence intensity onto the tissue structure image, or generates an image obtained by overlaying visual information corresponding to the time change (fluorescence intensity change) of fluorescence intensity in each boundary region together with the boundary line of the boundary region (e.g., the first region ROI1) onto the tissue structure image, or generates an image that shows the time change (e.g., reference) of fluorescence intensity in each boundary region. Figure 9 , Figure 10 The image is displayed alongside the image. As visual information corresponding to fluorescence intensity, for example, the color of areas with small changes in fluorescence intensity is set to a visually easily recognizable color (a hue easily recognizable to humans, color intensity, etc.). By displaying the image, for example, it is possible to visually identify the differences in fluorescence intensity changes between distinct boundary regions (e.g., the first region ROI1 and the second region ROI2). Furthermore, the image generation unit 413 can also generate images consisting solely of tissue structures, white light images, and fluorescence intensity images (intensity maps).

[0116] The control unit 44 causes the display device 5 to display the image generated in step S108 (step S109: display process). By displaying the image on the display device 5, the surgeon can confirm the treatment effect. The surgeon confirms the treatment effect with reference to the image, determines whether to apply additional treatment light, or determines the area to be irradiated (e.g., the first region ROI1). The surgeon operates the input unit 43 to input the determination result.

[0117] After the input unit 43 receives the judgment result, the control unit 44 determines whether additional irradiation with therapeutic light should be performed (step S110). If the control unit 44 determines that additional irradiation with therapeutic light is not required based on the input judgment result (step S110: No), the process ends. Conversely, if the control unit 44 determines that additional irradiation with therapeutic light should be performed (step S110: Yes), the process proceeds to step S111.

[0118] When additional irradiation is performed, for example in an illumination optics system, control is exercised to ensure that the shape of the irradiated area matches the boundary region, or the surgeon adjusts the spot diameter to irradiate the therapeutic area.

[0119] The control unit 44 determines whether the amount of light already irradiated in the area where additional therapeutic light is applied is within the permissible range (step S111). Here, the permissible range is a preset light amount, at least with an upper limit. This upper limit is a value set to prevent tissue damage due to excessive irradiation. The control unit 44 determines, for example, whether the amount of light already irradiated on the target area (cumulative light amount value) exceeds the upper limit.

[0120] If the control unit 44 determines that the amount of light after irradiation is below the allowable range (upper limit) (step S111: Yes), it moves to step S112. If the control unit 44 determines that the amount of light after irradiation exceeds the allowable range (upper limit) (step S111: No), it moves to step S113.

[0121] In step S112, the control unit 44 sets the irradiation area for additional irradiation. After setting the irradiation area, the control unit 44 returns to step S103 to repeat the process.

[0122] In step S113, the control unit 44 outputs an alarm indicating that the amount of irradiated light exceeds the permissible range. This alarm can be displayed as text on the display device 5, or it can be configured to emit sound, light, or a combination of these. After the display device 5 displays the alarm, the control unit 44 terminates the process.

[0123] In Embodiment 1 described above, tissue structure images are acquired using narrowband light. Based on changes in the tissue before and after treatment, regions with different reaction rates (boundary regions) are divided, and the changes in fluorescence intensity in each region are calculated. At this time, by displaying the boundary regions or showing the changes in fluorescence intensity in each boundary region, the surgeon can determine whether additional treatment light irradiation is needed for each region. According to Embodiment 1, additional treatment light irradiation can be applied to regions, thus enabling appropriate light irradiation of the treatment area.

[0124] Furthermore, in Embodiment 1 described above, after confirming the treatment effect through fluorescence, when performing additional irradiation, the cumulative light intensity of the treatment light targeting the area is compared with an allowable range. If the cumulative light intensity exceeds the allowable range, an alarm indicating that the cumulative light intensity exceeds the allowable range is output. According to Embodiment 1, tissue damage due to excessive irradiation by treatment light can be suppressed.

[0125] Alternatively, in Embodiment 1 described above, a multi-band image sensor can be used to construct the first imaging element 244a, acquiring multiple light bands that are different from each other individually. For example, by acquiring scattered and returned light in the band of light above 380 nm and below 440 nm, and scattered and returned light in the band of light above 490 nm and below 590 nm, the multi-band image sensor can generate individual narrowband light images. This allows for the acquisition of vascular images at different depths from the mucosal surface, enabling the determination of boundary regions with higher precision by utilizing changes in blood vessels and tissues at each depth.

[0126] (A variation of Implementation Method 1)

[0127] Next, refer to Figure 11 and Figure 12 A variation of Implementation 1 will be described. Figure 11 This is a block diagram illustrating the schematic structure of an endoscope system according to a modified embodiment 1 of the present invention. The endoscope system 1A of this modified embodiment has an endoscope 2A instead of the endoscope 2 of the endoscope system 1 of embodiment 1. The structure other than the endoscope 2A is the same, therefore description is omitted.

[0128] Endoscope 2A has a front end portion 24A instead of the front end portion 24 of endoscope 2. The structure other than the front end portion 24A is the same as that of endoscope 2, so the description is omitted.

[0129] The front end 24A includes: a light guide 241; an illumination lens 242; an optical system 243A for focusing light; and an imaging element 244A, which is disposed at the imaging position of the optical system 243A, receives the light focused by the optical system 243A and converts it into an electrical signal to perform the prescribed signal processing.

[0130] Figure 12 This is a diagram illustrating the structure of the imaging optical system of an endoscope according to a modified embodiment 1 of the present invention. The optical system 243A and the imaging element 244A are disposed inside the front end portion 24A.

[0131] The optical system 243A includes an objective lens 2430, a first lens 2431 composed of one or more optical elements, a second lens 2432 composed of one or more optical elements, a third lens 2433 composed of one or more optical elements, a cutoff filter 2434, and a fourth lens 2435 composed of one or more optical elements. The cutoff filter 2434 blocks light in the wavelength range of the excitation light. This excitation light corresponds to the wavelength range used to excite the antibody drug in PIT. The second lens 2432 and the fourth lens 2435 image the observed image onto different and non-overlapping positions of the imaging element 244A.

[0132] Here, the transmittance of the excitation light from the cutoff filter 2434 is set to 0.1% or less. By making the transmittance of the excitation light 0.1% or less, fluorescence can be selectively captured, for example, during excitation light illumination.

[0133] The image sensor 244A performs photoelectric conversion on light from the optical system 243A to generate an electrical signal (image signal). Specifically, the image sensor 244A arranges multiple pixels in a matrix, each having a photodiode that stores a charge corresponding to the amount of light, and a capacitor that converts the charge transferred from the photodiode into a voltage level. Each pixel performs photoelectric conversion on light from the optical system 243A to generate an electrical signal, which is then output as an image signal. The image sensor 244A can be implemented using, for example, a CCD image sensor or a CMOS image sensor.

[0134] Light L3 and L4 from the subject are incident on the first lens 2431 and the third lens 2433 respectively through the objective lens 2430. Light L3 incident on the first lens 2431 is imaged through the second lens 2432. Light L4 incident on the third lens 2433 is imaged through the cutoff filter 2434 and the fourth lens 2435.

[0135] The second lens 2432 forms an observation image in the first imaging section 244c of the imaging element 244A. The fourth lens 2435 forms an observation image in the second imaging section 244d of the imaging element 244A. The first imaging section 244c and the second imaging section 244d are formed by dividing the light-receiving area of ​​the imaging element into two.

[0136] In the case of PIT implementation, the processing unit 4 follows Figure 7 The process is executed. At this time, the first camera element 244a is replaced with the first camera unit 244c, and the second camera element 244b is replaced with the second camera unit 244d.

[0137] In the modified example described above, similar to Embodiment 1, a tissue structure image is acquired using narrowband light. Based on the changes in the tissue before and after treatment, regions with different reaction rates (boundary regions) are divided. The changes in fluorescence intensity of each region are calculated. By displaying the boundary regions or displaying the changes in fluorescence intensity of each boundary region, the surgeon can determine whether additional irradiation with treatment light is needed for each region. According to this modified example, additional irradiation with treatment light can be applied to the region, thus enabling appropriate light irradiation of the treatment area.

[0138] (Implementation Method 2)

[0139] Next, refer to Figure 13 and Figure 14 Implementation method 2 will be described. Figure 13This is a block diagram illustrating the schematic structure of an endoscope system according to Embodiment 2 of the present invention. The endoscope system 1B of Embodiment 2 replaces the endoscope 2 and processing device 4 of the endoscope system 1 of Embodiment 1 with an endoscope 2B and a processing device 4A. The structure is identical except for the endoscope 2A and the processing device 4A, therefore description is omitted.

[0140] Endoscope 2B has a front end portion 24B instead of the front end portion 24 of endoscope 2. The structure other than the front end portion 24B is the same as that of endoscope 2, so the description is omitted.

[0141] The front end 24B includes: a light guide 241; an illumination lens 242; an optical system 243B for focusing light; and an imaging element 244B, which is disposed at the imaging position of the optical system 243B, receives the light focused by the optical system 243B and converts it into an electrical signal to perform the prescribed signal processing.

[0142] Figure 14 This is a diagram illustrating the structure of the imaging optical system of the endoscope according to Embodiment 2 of the present invention. The optical system 243B and the imaging element 244B are disposed inside the front end portion 24B.

[0143] The optical system 243B includes an objective lens 243a, a beam splitter 243b (hereinafter referred to as the first beam splitter 243b), a cutoff filter 243c, and a second beam splitter 243d. The cutoff filter 243c cuts off light in the excitation wavelength range. The second beam splitter 243d bends the optical path of light in the blue component wavelength range, for example, light in the wavelength range below 490 nm, while allowing light in other components (such as the green and red components) to pass through. In addition to the optical elements described above, the optical system 243B may also include lenses, etc.

[0144] Light from the subject is incident on the first beam splitter 243b via the objective lens 243a. The first beam splitter 243b bends the light path of light (light L2) with wavelengths longer than the fluorescence emitted by the subject, while allowing light (light L1) with wavelengths shorter than the fluorescence to pass through. The light (light L1) after passing through the first beam splitter 243b is then incident on the second beam splitter 243d. On the other hand, the excitation light after its path is bent by the first beam splitter 243b and the excitation light in the fluorescence (light L2) are cut off by the cutoff filter 243c, and the fluorescence is incident on the second imaging element 244b.

[0145] The second beam splitter 243d directs the return light (light L) containing narrowband light in the wavelength range of 440nm to 490nm. 12 The light path is bent, and the light of color components other than the blue component (such as components with wavelengths longer than 490 nm) is reduced (light L). 11 The light passing through the second beam splitter 243d (light L) passes through. 11The light is incident on the first imaging element 244a. On the other hand, the light (light L) whose optical path is bent by the second beam splitter 243d is incident on the first imaging element 244a. 12 The light is incident on the third camera element 244e.

[0146] The image sensor 244B performs photoelectric conversion on the light from the optical system 243B to generate an electrical signal (image signal). Specifically, the image sensor 244B has three image sensors (a first image sensor 244a, a second image sensor 244b, and a third image sensor 244e). The first image sensor 244a to the third image sensor 244e are implemented, for example, using a CCD image sensor or a CMOS image sensor.

[0147] The structure of the processing device 4A will be described. The processing device 4A includes an image processing unit 41A, a synchronization signal generation unit 42, an input unit 43, a control unit 44, and a storage unit 45.

[0148] The image processing unit 41A receives image data of various colors of illumination light captured by the imaging element 244B from the endoscope 2B. The image processing unit 41A performs prescribed image processing on the image data received from the endoscope 2B to generate an image and output it to the display device 5, or sets a boundary region determined based on the image, or calculates the time change of fluorescence intensity. The image processing unit 41A includes a boundary region determination unit 411, a fluorescence intensity change calculation unit 412, a display image generation unit 413, a specific region intensity calculation unit 414, and a fluorescence intensity normalization unit 415.

[0149] In this embodiment 2, the display image generation unit 413 generates a white light image based on the electrical signals generated by the first imaging element 244a and the third imaging element 244e.

[0150] The specific region intensity calculation unit 414 calculates the light intensity of a specific wavelength band. In this embodiment 2, the light intensity of the blue component band (light L) is calculated. 12 The intensity of the blue component is calculated by the intensity calculation unit 414 in a specific area based on the electrical signal generated by the third imaging element 244e.

[0151] The fluorescence intensity normalization unit 415 normalizes the intensity change by dividing the intensity change calculated by the fluorescence intensity change calculation unit 412 by the light intensity of the blue component calculated by the specific region intensity calculation unit 414.

[0152] When processing unit 4A implements PIT, according to Figure 7The process is executed. During the fluorescence detection step (step S105), in addition to the excitation light, a narrowband light of 440 nm to 490 nm is irradiated onto the subject. Therefore, the specific region intensity calculation unit 414 calculates the intensity of the returned light from the narrowband light of 440 nm to 490 nm. Alternatively, the narrowband light can be irradiated at a different time than in the fluorescence detection step.

[0153] In addition, in the fluorescence intensity change calculation process (step S107), the fluorescence intensity change after normalization by the fluorescence intensity normalization unit 415 is calculated. Furthermore, in the boundary region determination process (step S106), the boundary region determination unit 411 can determine the boundary region based on the electrical signal generated by the first imaging element 244a, or based on the electrical signal generated by the third imaging element 244e, or based on the electrical signals generated by both the first imaging element 244a and the third imaging element 244e.

[0154] Here, refer to Figures 15-17 The determination of the boundary area is explained. Figure 15 It is a diagram that schematically shows the image obtained by the first imaging element. Figure 16 It is a schematic diagram showing the image obtained by the third imaging element.

[0155] The image obtained by the first imaging element 244a is based on an image formed by light in a band other than the fluorescent and blue components. Furthermore, the image obtained by the third imaging element 244e is based on an image formed by light in a band containing the blue component. For example, suppose the image obtained by the first imaging element 244a... Figure 15 The image shown was obtained by the third camera element 244e. Figure 16 The image shown. Additionally... Figure 15 and Figure 16 The X and Y axes shown are labeled to indicate the relative positional relationships of the images. Figure 15 as well as Figure 16 The images shown are based on different wavelengths of light (wavelengths containing the blue component, wavelengths containing the blue component, and wavelengths other than fluorescence), depicting different tissue structures. Specifically, they depict blood vessels at varying depths from the tissue surface. Figure 15 and Figure 16 In the process, the image of the tissue structure is depicted in the light detection regions R1 and R2.

[0156] The boundary region determination unit 411 determines the boundary region based on the image obtained by the first imaging element 244a (e.g., Figure 15 The image shown (hereinafter sometimes referred to as the first image) and the image obtained by the third imaging element 244e (e.g.) Figure 16The image shown (hereinafter sometimes referred to as the second image) identifies boundary regions with different degrees of change in tissue structure. Figure 17 It is used to illustrate the passage Figure 15 The image shown and Figure 16 The boundary region is defined by summing the images shown. The boundary region determination unit 411 synthesizes the first image and the second image, performs contour extraction of the synthesized image, etc., and uses the extracted contour as the boundary region. Figure 17 In the diagram, the dashed line R3 is defined as the boundary region.

[0157] In Embodiment 2 described above, similar to Embodiment 1, a tissue structure image is acquired using narrowband light. Based on the changes in tissue before and after treatment, regions with different reaction rates (boundary regions) are divided. The changes in fluorescence intensity of each region are calculated, and the changes in fluorescence intensity of the boundary regions or each boundary region are displayed. This allows the surgeon to determine whether additional treatment light irradiation is needed for each region. According to Embodiment 2, additional treatment light irradiation can be applied to a region, thus enabling appropriate light irradiation of the treatment area.

[0158] Furthermore, in this embodiment 2, the fluorescence intensity variation is normalized. Therefore, by displaying the normalized fluorescence intensity variation, the surgeon can appropriately control the fluorescence intensity variation regardless of the distance between the endoscope 2B (front end 24B) and the subject. Additionally, the narrow band obtained for normalization is not limited to the band of 440 nm to 490 nm, and can be set to other bands. Here, the light in the band of 440 nm to 490 nm does not have the contribution of absorption caused by blood components, and the scattered light from biological tissue becomes dominant. Therefore, the intensity of the scattered light from the tissue depends only on the distance, thus it is suitable for eliminating distance-related variations in fluorescence intensity caused by division operations, etc.

[0159] (Implementation Method 3)

[0160] Next, refer to Figure 18 and Figure 19 Implementation method 3 will be described. Figure 18 This is a block diagram illustrating the schematic structure of an endoscope system according to Embodiment 3 of the present invention. The endoscope system 1C of Embodiment 3 has a processing device 4A instead of the processing device 4 of the endoscope system 1 of Embodiment 1. Furthermore, the front end portion 24 has the same optical system 243 and imaging element 244 as in Embodiment 1; however, the first imaging element 244a is composed of a multi-band image sensor, and electrical signals are generated separately for each color component.

[0161] Figure 19This is a diagram illustrating the structure of the imaging optical system of the endoscope according to Embodiment 3 of the present invention. The light reflected or scattered from the subject is, for example, narrowband light L containing a red component with a center wavelength of 660 nm. R Light with an amber component centered at 590nm A The green component of light with a center wavelength of 525nm G The blue component of light with a central wavelength of 480nm B The violet component of light with a central wavelength of 380nm V Contains excitation light (e.g.) Figure 5 The light L shown P (and fluorescence excited by excitation light) T Note that light L T After the excitation light is cut off by the cutoff filter 243c, it enters the second imaging element 244b.

[0162] The light L after passing through beam splitter 243b R L A L G L B L V Each light source passes through a filter and is individually incident on the first imaging element 244a. The first imaging element 244a focuses the light L... R L A L G L B L V Each signal is generated by photoelectric conversion separately.

[0163] In this embodiment 3, the specific area intensity calculation unit 414 uses the light (light L) in the electrical signal generated by the first imaging element 244a based on the blue component. B The generated electrical signal is used to calculate the light intensity.

[0164] When processing unit 4A implements PIT, according to Figure 7 The process is executed. At this time, in the fluorescence intensity change calculation step (step S107), the fluorescence intensity change after normalization by the fluorescence intensity normalization unit 415 is calculated. Furthermore, in the boundary region determination step (step S106), the boundary region determination unit 411 can determine the boundary region using an electrical signal based on the blue component of the electrical signal generated by the first imaging element 244a, or using an electrical signal based on light components other than the blue component, or based on electrical signals of all color components generated by the first imaging element 244a. Here, the electrical signals of all color components are equivalent to electrical signals generated by multiple filters in a multi-band image sensor that receive or transmit light at different wavelengths.

[0165] In Embodiment 3 described above, similar to Embodiment 1, a tissue structure image is acquired using narrowband light. Based on the changes in tissue before and after treatment, regions with different reaction rates (boundary regions) are divided. The changes in fluorescence intensity of each region are calculated. By displaying the boundary regions or displaying the changes in fluorescence intensity of each boundary region, the surgeon can determine whether additional treatment light irradiation is needed for each region. According to this Embodiment 3, additional treatment light irradiation can be applied to the region, thus enabling appropriate light irradiation of the treatment area.

[0166] In addition, in Embodiment 3, an example was described in which the first imaging element 244a generates an electrical signal separately for each color component. However, it is also possible to generate an electrical signal based on light equivalent to the return light of narrowband light in the band of 440nm and above and 490nm and an electrical signal based on light components other than the return light.

[0167] (Implementation Method 4)

[0168] Next, refer to Figure 20 Implementation method 4 will be described. Figure 20 This is a block diagram showing the schematic structure of the endoscope system according to Embodiment 4 of the present invention. The endoscope system 1D of Embodiment 4 has the same structure as the endoscope system 1 of Embodiment 1. In the endoscope system 1D, the processing device 4 is electrically connected to the treatment device 6, and the emission of therapeutic light from the treatment device 62 is controlled by the control unit 44.

[0169] In the case of PIT implementation, the processing unit 4 follows Figure 7 The process is executed. During the treatment light irradiation, the control unit 44 controls the irradiation range, timing, and duration of the treatment light. Specifically, the control unit 44, for example, sets the light intensity (output value) and irradiation time of a preset amount of irradiated light for the irradiation range set by the surgeon. The control unit 44 initiates treatment light irradiation control by pressing the switch of the operation input unit 611. Furthermore, during additional irradiation, the control unit 44 sets the shape of the irradiation range of the treatment light emitted from the treatment device 62 according to the boundary area of ​​the object, and initiates treatment light irradiation control by pressing the switch of the operation input unit 611. In addition, the control unit 44 can also determine whether the cumulative amount of irradiated light in the irradiated object area exceeds a preset upper limit value, and output an alarm if it does.

[0170] In Embodiment 4 described above, similar to Embodiment 1, a tissue structure image is acquired using narrowband light. Based on the changes in tissue before and after treatment, regions with different reaction rates (boundary regions) are divided. The changes in fluorescence intensity of each region are calculated. By displaying the boundary regions or displaying the changes in fluorescence intensity of each boundary region, the surgeon can determine whether additional treatment light irradiation is needed for each region. According to this Embodiment 4, additional treatment light irradiation can be applied to the region, thus enabling appropriate light irradiation of the treatment area.

[0171] In addition, in this embodiment 4, the control unit 44 controls the emission of the treatment light emitted by the treatment device 62, so that the surgeon can irradiate the treatment light to the appropriate area without adjusting the irradiation range of the treatment light to match the boundary area.

[0172] Furthermore, in the above embodiments, the excitation light and the treatment light can be the same wavelength (same center wavelength) or different wavelengths (center wavelength). In addition, when the excitation light and the treatment light are interchangeable, the treatment light (excitation light) can be irradiated by the treatment device 62 or the excitation light source 313, or it can be configured without either the excitation light source 313 or the treatment device 62.

[0173] Furthermore, while the above embodiment describes an example where the light source device 3 and the processing device 4 are separate, it is also possible to configure the light source device 3 and the processing device 4 as an integrated unit. Also, while the embodiment describes an example where therapeutic light is irradiated by a treatment device, it is also possible to configure the light source device 3 to emit therapeutic light.

[0174] Furthermore, in the above embodiments, the endoscope system of the present invention is described as an endoscope system 1 that uses a flexible endoscope 2, the object of observation of which is biological tissue in the body of the patient. However, it can also be applied to endoscope systems that use a structure in which a camera is connected to the eyepiece of an optical endoscope such as a rigid endoscope, an industrial endoscope for observing the properties of the material, a fiber endoscope, or an optical viewing tube.

[0175] (Additional items)

[0176] A phototherapy method includes the following steps:

[0177] Insert the tip of the endoscope into the treatment area;

[0178] Irradiate the treatment area with therapeutic light to cause a reaction of the drug that is bound to the treatment area;

[0179] Using a tissue structure image obtained by irradiating the treatment site with narrow-band light, areas where tissue structure has changed are identified as boundary regions.

[0180] Calculate the change in fluorescence intensity in the boundary region;

[0181] The decision to apply additional therapeutic light is based on the change in fluorescence intensity.

[0182] The therapeutic light is applied to the area requiring additional irradiation; and

[0183] Calculate the change in fluorescence intensity in the boundary region after the additional irradiation.

[0184] Industrial utilization potential

[0185] As described above, the phototherapy device, phototherapy method, and phototherapy procedure of the present invention are useful for properly applying light irradiation to the treatment area.

[0186] Explanation of reference numerals in the attached figures

[0187] 1. 1A-1D Endoscopic System

[0188] 2. Endoscopes 2A and 2B

[0189] 3 Light Source Device

[0190] 4. 4A processing device

[0191] 5 display devices

[0192] 6. Handling equipment and devices

[0193] 21 Insertion section

[0194] 22 Operations Department

[0195] 23 General Purpose Cable

[0196] 24 front end

[0197] 25 bends

[0198] 26 Flexible tube section

[0199] 31 Light Source Department

[0200] 32 Lighting Control Department

[0201] 33 Light Source Driver

[0202] 41 Image Processing Department

[0203] 42 Synchronization Signal Generation Unit

[0204] 43 Input Section

[0205] 44 Control Department

[0206] 45 Storage Unit

[0207] 61 Handling Equipment Operation Section

[0208] 62 treatment equipment

[0209] 241 optical guide

[0210] 242 Illumination Lens

[0211] 243, 243A optical systems

[0212] 243a and 2430 objectives

[0213] 243b Beam Spectroscope (First Beam Spectroscope)

[0214] 243c and 2434 cutoff filters

[0215] 243d Second Beam Spectroscope

[0216] 244, 244A, 244B camera elements

[0217] 244a First camera element

[0218] 244b Second camera element

[0219] 244c First Camera Department

[0220] 244d Second Camera Department

[0221] 244e Third Camera Element

[0222] 311 White Light Source

[0223] 312 Narrowband Light Source

[0224] 313 excitation light source

[0225] 411 Boundary Area Determination Department

[0226] 412 Fluorescence Intensity Change Calculation Unit

[0227] 413 Display Image Generation Unit

[0228] 414 Specific Area Intensity Calculation Department

[0229] 415 fluorescence intensity normalization part

[0230] 2431 First Lens

[0231] 2432 Second Lens

[0232] 2433 Third Lens

[0233] 2435 Fourth Lens

Claims

1. A phototherapy device, comprising: A therapeutic light emitting device that emits therapeutic light that causes a drug to react; The tissue structure image acquisition unit acquires a tissue structure image obtained by narrowband light irradiating the irradiation position of the treatment light; A fluorescence image acquisition unit acquires a fluorescence image obtained by excitation light irradiating the irradiation position of the therapeutic light; The boundary region determination unit uses the tissue structure image to determine the boundary regions where the tissue structure has changed, so as to divide regions with different reaction rates. A fluorescence intensity change calculation unit calculates the magnitude of the change in fluorescence intensity in the boundary region; as well as The display image generation unit generates a display image to show the magnitude of the change in fluorescence intensity.

2. The phototherapy device according to claim 1, wherein, The boundary region determination unit detects the temporal changes in the tissue structure image and determines the region where the tissue structure has changed as the boundary region based on the amount of the temporal change.

3. The phototherapy device according to claim 2, wherein, The boundary region determination unit compares the value of the tissue structure image with a preset threshold, thereby determining the area where the tissue structure has changed as the boundary region.

4. The phototherapy device according to claim 1, wherein, The boundary region determination unit uses feature quantities pre-calculated through machine learning to determine the regions where the tissue structure has changed as the boundary regions.

5. The phototherapy device according to claim 1, wherein, The tissue structure image acquisition unit acquires tissue structure images obtained by the narrowband light in the wavelength range of 380nm and above and 440nm and below.

6. The phototherapy device according to claim 1, wherein, The phototherapy device also has a fluorescence intensity normalization unit, which normalizes the fluorescence intensity calculated by the fluorescence intensity change calculation unit by using the light intensity of the returned light of narrowband light in the band above 440nm and below 490nm.

7. The phototherapy device according to claim 1, wherein, The tissue structure image acquisition unit acquires tissue structure images obtained by the narrowband light in the wavelength range of 490 nm to 590 nm.

8. The phototherapy device according to claim 1, wherein, The tissue structure image acquisition unit acquires tissue structure images obtained by the narrowband light in the wavelength range of 590nm and above and 620nm and below.

9. The phototherapy device according to claim 1, wherein, The tissue structure image acquisition unit acquires tissue structure images obtained by the narrowband light in the wavelength range of 620 nm to 780 nm.

10. The phototherapy device according to claim 1, wherein, The phototherapy device also includes a control unit that uses the multiplication value of light intensity and irradiation time as the set irradiation light amount to control the emission of the treatment light to the target area of ​​the treatment light.

11. A computer program product comprising a computer program that, when executed by a processor, performs the following steps for confirming the therapeutic effect after irradiating a treatment site with therapeutic light that will cause a pharmaceutical reaction: The tissue structure image acquisition step involves acquiring a tissue structure image obtained by irradiating a narrow band of light onto the irradiation location of the treatment light; The fluorescence image acquisition step involves acquiring a fluorescence image obtained by excitation light irradiating the irradiation position of the therapeutic light; The boundary region determination step uses the tissue structure image to determine the boundary regions where the tissue structure has changed. The fluorescence intensity change calculation step involves calculating the magnitude of the fluorescence intensity change in the boundary region. as well as The display image generation step generates a display image to show the magnitude of the change in fluorescence intensity.

12. A computer-readable storage medium storing a computer program that, when executed by a processor, performs the following steps to generate information for confirming the therapeutic effect after irradiating a treatment site with therapeutic light that will cause a pharmaceutical reaction, the steps including: The tissue structure image acquisition step involves acquiring a tissue structure image obtained by irradiating a narrow band of light onto the irradiation location of the treatment light; The fluorescence image acquisition step involves acquiring a fluorescence image obtained by excitation light irradiating the irradiation position of the therapeutic light; The boundary region determination step uses the tissue structure image to determine the boundary regions where the tissue structure has changed, in order to divide regions with different reaction rates. The fluorescence intensity change calculation step involves calculating the magnitude of the fluorescence intensity change in the boundary region. as well as The display image generation step generates a display image to show the magnitude of the change in fluorescence intensity.

Images