IMAGE PROCESSING APPARATUS, PHOTOIMMUNOTREATMENT SYSTEM, IMAGE PROCESSING METHOD, AND IMAGE PROCESSING PROGRAM

Abstract

This image processing device comprises: a fluorescent image acquisition unit for acquiring a fluorescent image obtained using therapeutic light which causes a drug to react; a gradation range setting unit for setting the gradation range of an image on the basis of the distribution of fluorescence strength in the fluorescent image; and a gradation expansion image generation unit for generating a gradation expansion image by distributing gradation in accordance with the gradation range set by the gradation range setting unit.

Description

[Technical field]

[0001] The present invention relates to an image processing device, a photoimmunotherapy system, an image processing method, and an image processing program. [Background technology]

[0002] In recent years, research has been conducted into photoimmunotherapy (PIT), which treats cancer by specifically binding an antibody drug to the proteins of cancer cells and activating the antibody drug by irradiating it with near-infrared light, a therapeutic light, to destroy the cancer cells. When irradiated with near-infrared light, the antibody drug causes the cancer cells to expand and induces cell death. In this process, the antibody drug is excited to emit fluorescence. The intensity of this fluorescence is used as an indicator of the effectiveness of the treatment.

[0003] Also, as a technique for evaluating treatment based on the intensity of fluorescence, a technique is known in which indocyanine green (ICG) is introduced into the blood and the subcutaneous blood circulation is observed by imaging the fluorescence of this ICG (see, for example, Patent Document 1). In Patent Document 1, the contrast and dynamic range are compressed in the observed image to express the overall gradation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-135253 A Summary of the Invention [Problem to be solved by the invention]

[0005] In PIT, the completion of treatment is judged by the decrease in observed fluorescence intensity. When treatment approaches completion, the fluorescence intensity decreases, but when the fluorescence intensity is low, it becomes difficult to visually recognize the change in fluorescence in the image. In order to accurately judge the progress and completion of treatment, it is necessary to accurately grasp the change in fluorescence.

[0006] The present invention has been made in consideration of the above, and aims to provide an image processing device, a photoimmunotherapy system, an image processing method, and an image processing program that can accurately grasp changes in fluorescence intensity. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, the image processing device of the present invention includes a fluorescence image acquisition unit that acquires a fluorescence image obtained by using therapeutic light that reacts with a drug, a tone range setting unit that sets a tone range of the image based on the distribution of fluorescence intensity of the fluorescence image, and a gradation-expanded image generation unit that generates a gradation-expanded image by assigning gradations in accordance with the gradation range set by the gradation range setting unit.

[0008] In the image processing device of the present invention, in the above-mentioned invention, the fluorescence image acquisition unit acquires an initial fluorescence image at the start of irradiation of the therapeutic light, and further comprises a difference image generation unit which generates a difference image representing the difference in fluorescence intensity between the initial fluorescence image and the fluorescence image obtained at the time of irradiation of the therapeutic light, and the gradation range setting unit sets a gradation range for the difference.

[0009] In the image processing device according to the present invention, in the above-mentioned invention, the gradation range setting section performs clipping processing to set a gradation range in accordance with a condition that is preset for a maximum value or a minimum value of the fluorescence intensity of the fluorescence image.

[0010] In the image processing device according to the present invention, in the above-mentioned invention, the gradation range setting section performs clipping processing to set a gradation range in accordance with a condition that is preset for a maximum value or a minimum value of the difference of the difference image.

[0011] In the image processing device according to the present invention, in the above-mentioned invention, the gradation range setting section sets the gradation range in accordance with a ratio that is set in advance with respect to the maximum value or the minimum value.

[0012] In the image processing device according to the present invention, in the above-mentioned invention, the gradation range setting section sets the gradation range based on a target attenuation value that is set for the fluorescent image.

[0013] In addition, in the image processing device according to the present invention, in the above-mentioned invention, the image processing device further includes a control unit that sets a region of interest in the fluorescence image for which the gradation range setting unit sets a gradation range.

[0014] Furthermore, the photoimmunotherapy system of the present invention includes a therapeutic light emitting unit that emits therapeutic light that induces a drug reaction, a fluorescent image acquiring unit that acquires a fluorescent image obtained by the therapeutic light, a gradation range setting unit that sets a gradation range of the fluorescent image based on the distribution of fluorescent intensity, and a gradation-expanded image generating unit that generates a gradation-expanded image by assigning gradations in accordance with the gradation range set by the gradation range setting unit.

[0015] Moreover, the image processing method of the present invention includes a fluorescence image acquiring step in which a fluorescence image acquiring unit acquires a fluorescence image obtained by using therapeutic light that causes a drug to react, a gradation range setting step in which a gradation range setting unit sets a gradation range of the image based on the distribution of fluorescence intensity of the fluorescence image, and a gradation expanded image generating step in which a gradation expanded image generating unit generates a gradation expanded image by assigning gradations in accordance with the gradation range set by the gradation range setting unit.

[0016] The image processing program of the present invention causes a computer to execute a fluorescence image acquiring step of acquiring a fluorescence image obtained by using therapeutic light that causes a drug to react, a tone range setting step of setting a tone range of the image based on the distribution of fluorescence intensity of the fluorescence image, and a tone-expanded image generating step of generating a tone-expanded image by assigning tone in accordance with the tone range set in the tone range setting step. Effect of the Invention

[0017] The present invention has the effect of enabling changes in fluorescence intensity to be accurately grasped. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an endoscope system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing a schematic configuration of the endoscope system according to the first embodiment of the present invention. As shown in FIG. [Diagram 3] FIG. 3 is a diagram illustrating a tip configuration of the endoscope according to the first embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing an example of a flow of treatment using the endoscope according to the first embodiment of the present invention. [Diagram 5] FIG. 5 is a flowchart illustrating an example of processing by the processing device according to the first embodiment of the present invention. [Figure 6] FIG. 6 is a diagram showing an example of an observation image including a fluorescent image. [Figure 7] FIG. 7 is a diagram showing a fluorescent image showing the amount of fluorescent light in the observation image shown in FIG. [Figure 8] FIG. 8 is a diagram showing the relationship between the fluorescence intensity and the occurrence frequency in a region of interest. [Figure 9] FIG. 9 is a diagram (part 1) showing an example of the fluorescence intensity distribution for pixels of a fluorescence image. [Figure 10] FIG. 10 is a diagram (part 2) showing an example of the fluorescence intensity distribution for pixels of a fluorescence image. [Figure 11] FIG. 11 is a diagram showing an example of a gradation expanded image obtained by performing gradation expansion processing on the fluorescence image shown in FIG. [Figure 12] FIG. 12 is a diagram (part 1) showing another display example of a gradation expanded image obtained by performing gradation expansion processing on the fluorescence image shown in FIG. [Figure 13] FIG. 13 is a diagram (part 2) showing another display example of a gradation expanded image obtained by performing gradation expansion processing on the fluorescence image shown in FIG. [Figure 14] FIG. 14 is a diagram (part 1) for explaining the gradation expansion process according to the first embodiment of the present invention. [Figure 15] FIG. 15 is a diagram illustrating a clipping process according to a modification of the first embodiment of the present invention. In FIG. [Figure 16] FIG. 16 is a block diagram showing a schematic configuration of an endoscope system according to the second embodiment of the present invention. As shown in FIG. [Figure 17] FIG. 17 is a flowchart illustrating an example of processing by the processing device according to the second embodiment of the present invention. [Figure 18] FIG. 18 is a diagram showing an example of the amount of fluorescence of corresponding pixels acquired at different times. [Figure 19] FIG. 19 is a diagram showing an example of a difference image. [Figure 20] FIG. 20 is a diagram (part 1) showing an example of the distribution of fluorescence amount differences for pixels of a difference image and an example of the gradation range. [Figure 21] FIG. 21 is a diagram (part 2) showing an example of the distribution of fluorescence amount differences for pixels of a difference image and an example of the gradation range. [Figure 22] FIG. 22 is a diagram showing an example of a gradation expanded image obtained by performing gradation expansion processing on the difference image shown in FIG. [Figure 23] FIG. 23 is a diagram showing an example of the amount of fluorescence of corresponding pixels acquired at different times. [Figure 24] FIG. 24 is a diagram showing an example of a difference image. [Diagram 25] FIG. 25 is a diagram (part 1) showing an example of the distribution of fluorescence amount differences for pixels of a difference image and an example of the gradation range. [Figure 26] FIG. 26 is a diagram (part 2) showing an example of the distribution of fluorescence amount differences for pixels of a difference image and an example of the gradation range. [Figure 27] FIG. 27 is a diagram showing an example of a gradation expanded image obtained by performing gradation expansion processing on the difference image shown in FIG. [Figure 28] FIG. 28 is a flowchart illustrating an example of processing by the processing device according to the third embodiment of the present invention. [Figure 29] FIG. 29 is a diagram showing an example of the amount of fluorescence of corresponding pixels acquired at different times. [Diagram 30] FIG. 30 is a diagram (part 1) showing an example of the distribution of fluorescence amount differences for pixels of a fluorescence image. [Diagram 31] FIG. 31 is a diagram (part 2) showing an example of the distribution of fluorescence amount differences for pixels of a fluorescence image. [Diagram 32] FIG. 32 is a diagram illustrating an example of setting the minimum value of the gradation range. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] A mode for carrying out the present invention (hereinafter referred to as "embodiment") will be described below. In the embodiment, a medical endoscope system that captures and displays images inside a subject such as a patient will be described as an example of a system including an image processing device, a photoimmunotherapy system, and a fluorescence endoscope according to the present invention. The present invention is not limited to the embodiment. Furthermore, in the description of the drawings, the same parts are described with the same reference numerals.

[0020] (Embodiment 1) Fig. 1 is a diagram showing a schematic configuration of an endoscope system according to a first embodiment of the present invention. Fig. 2 is a block diagram showing a schematic configuration of an endoscope system according to the first embodiment. Fig. 3 is a diagram for explaining a tip configuration of an endoscope according to the first embodiment.

[0021] The endoscopic system 1 shown in Figures 1 and 2 includes an endoscope 2 that captures in-vivo images of a subject by inserting a tip portion into the subject, a light source device 3 that generates illumination light to be emitted from the tip of the endoscope 2, a processing device 4 that performs predetermined signal processing on the imaging signal captured by the endoscope 2 and generally controls the operation of the entire endoscopic system 1, a display device 5 that displays the in-vivo images generated by the signal processing of the processing device 4, and a treatment device 6.

[0022] The endoscope 2 comprises an insertion portion 21 having a flexible, elongated shape, an operation portion 22 connected to the base end side of the insertion portion 21 and accepting input of various operation signals, and a universal cord 23 extending from the operation portion 22 in a direction different from the direction in which the insertion portion 21 extends and incorporating various cables connected to the light source device 3 and the processing device 4.

[0023] The insertion section 21 has a tip section 24 incorporating an image sensor 244 in which pixels that receive light and generate signals by photoelectric conversion are arranged two-dimensionally, a freely bendable bending section 25 composed of a plurality of bending pieces, and a long, flexible tube section 26 that is connected to the base end side of the bending section 25. The insertion section 21 is inserted into a body cavity of a subject, and captures an image of a subject such as biological tissue in a position where external light does not reach, using the image sensor 244.

[0024] The operation unit 22 has a bending knob 221 for bending the bending portion 25 in the up-down and left-right directions, a treatment tool insertion portion 222 for inserting treatment tools such as a therapeutic light irradiation device, a biopsy forceps, an electric scalpel, and an examination probe into the body cavity of the subject, and a plurality of switches 223 which are an operation input portion for inputting operation instruction signals for peripheral devices such as an air supply means, a water supply means, a screen display control, etc. in addition to the processing device 4. The treatment tool inserted from the treatment tool insertion portion 222 passes through a treatment tool channel (not shown) of the tip portion 24 and emerges from the opening (see FIG. 3).

[0025] The universal cord 23 incorporates at least a light guide 241 and a cable assembly 245 that bundles one or a plurality of signal lines. The universal cord 23 branches at an end opposite to the end connected to the operation unit 22. At the branched end of the universal cord 23, a connector 231 that can be detached from the light source device 3 and a connector 232 that can be detached from the processing device 4 are provided. A part of the light guide 241 extends from the end of the connector 231. The universal cord 23 transmits illumination light emitted from the light source device 3 to the tip portion 24 via the connector 231 (light guide 241), the operation unit 22, and the flexible tube portion 26. The universal cord 23 also transmits an image signal captured by an image sensor 244 provided at the tip portion 24 to the processing device 4 via the connector 232. The cable assembly 245 includes a signal line for transmitting an imaging signal, a signal line for transmitting a drive signal for driving the imaging element 244, and a signal line for transmitting and receiving information including unique information related to the endoscope 2 (imaging element 244). Note that in this embodiment, the signal line is described as transmitting an electrical signal, but the signal line may be used to transmit an optical signal, or may be used to transmit a signal between the endoscope 2 and the processing device 4 by wireless communication.

[0026] The tip portion 24 includes a light guide 241 constructed using glass fiber or the like and forming a light guide path for the light emitted by the light source device 3, an illumination lens 242 provided at the tip of the light guide 241, an optical system 243 for collecting light, and an image sensor 244 provided at the imaging position of the optical system 243, which receives the light collected by the optical system 243, photoelectrically converts it into an electrical signal, and performs predetermined signal processing.

[0027] The optical system 243 is configured using one or more lenses. The optical system 243 forms an observation image on the light receiving surface of the image sensor 244. The optical system 243 may have an optical zoom function for changing the angle of view and a focus function for changing the focus.

[0028] The imaging element 244 photoelectrically converts light from the optical system 243 to generate an electric signal (image signal). The imaging element 244 is formed by arranging a plurality of pixels in a matrix, each of which has a photodiode that accumulates an electric charge according to the amount of light, a capacitor that converts the electric charge transferred from the photodiode to a voltage level, and the like. The imaging element 244 photoelectrically converts light incident on each pixel via the optical system 243 to generate an electric signal, sequentially reads out the electric signals generated by pixels arbitrarily set as readout targets among the plurality of pixels, and outputs them as an image signal. The imaging element 244 is realized by using, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0029] The endoscope 2 has a memory (not shown) that stores execution programs and control programs for the image sensor 244 to execute various operations, and data including identification information of the endoscope 2. The identification information includes unique information (ID), year of manufacture, specification information, transmission method, etc. of the endoscope 2. The memory may also temporarily store image data generated by the image sensor 244.

[0030] The endoscope 2 has a memory (not shown) that stores execution programs and control programs for the image sensor 244 to execute various operations, and data including identification information of the endoscope 2. The identification information includes unique information (ID), year of manufacture, specification information, transmission method, etc. of the endoscope 2. The memory may also temporarily store image data generated by the image sensor 244.

[0031] The following describes the configuration of the light source device 3. The light source device 3 includes a light source section 31, an illumination control section 32, and a light source driver 33. Under the control of the illumination control section 32, the light source section 31 sequentially switches between and emits illumination light with different exposure amounts to a subject (subject).

[0032] The light source unit 31 is configured using a light source and one or more lenses, and emits light (illumination light) by driving the light source. The light generated by the light source unit 31 is emitted toward the subject from the tip of the tip unit 24 via the light guide 241. The light source unit 31 has a white light source 311.

[0033] The white light source 311 emits light (white light) having a wide visible wavelength band. The white light source 311 is realized using any of light sources such as an LED light source, a laser light source, a xenon lamp, and a halogen lamp.

[0034] The illumination control unit 32 controls the amount of power supplied to the light source unit 31 based on a control signal (dimming signal) from the processing device 4, and also controls the light source to be caused to emit light and the drive timing of the light source.

[0035] Under the control of the illumination control unit 32, the light source driver 33 supplies a current to the light source that is the target of light emission, thereby causing the light source unit 31 to emit light.

[0036] The following describes the configuration of the processing device 4. The processing device 4 includes an image processing unit 41, a synchronization signal generating unit 42, an input unit 43, a control unit 44, and a storage unit 45.

[0037] The image processing unit 41 receives image data of the illumination light of each color captured by the imaging element 244 from the endoscope 2. When the image processing unit 41 receives analog image data from the endoscope 2, it performs A / D conversion to generate a digital imaging signal. When the image processing unit 41 receives image data as an optical signal from the endoscope 2, it performs photoelectric conversion to generate digital image data.

[0038] The image processing unit 41 performs predetermined image processing on the image data received from the endoscope 2 to generate an image and outputs it to the display device 5, sets an enhancement area determined based on the image, calculates the change in fluorescence intensity over time, etc. The image processing unit 41 has a white light image generation unit 411, a fluorescence image generation unit 412, a gradation range setting unit 413, and a gradation expanded image generation unit 414.

[0039] The white light image generator 411 generates a white light image based on an image formed by white light.

[0040] The fluorescent light image generating section 412 generates a fluorescent light image based on an image formed by the fluorescent light.

[0041] The gradation range setting section 413 sets the range of the image for which gradation setting is to be performed (for example, the range of brightness, etc.) based on the fluorescence intensity.

[0042] The gradation expanded image generating unit 414 generates a gradation expanded image by allocating gradations according to the gradation range set by the gradation range setting unit 413. The gradation expanded image generating unit 414 generates a gradation expanded image by expanding the gradations of, for example, a portion of the fluorescent image.

[0043] The white light image generating section 411, the fluorescent image generating section 412, and the gradation expanded image generating section 414 generate images by performing predetermined image processing. Here, the predetermined image processing is a synchronization process, a gradation correction process, a color correction process, and the like. The synchronization process is a process of synchronizing image data of each color component of RGB. The gradation correction process is a process of performing gradation correction on the image data. The color correction process is a process of performing color correction on the image data. Note that the white light image generating section 411, the fluorescent image generating section 412, and the gradation expanded image generating section 414 may adjust gain according to the brightness of the image.

[0044] The image processing unit 41 is configured using a general-purpose processor such as a CPU (Central Processing Unit) or a dedicated processor such as various arithmetic circuits that execute specific functions such as an ASIC (Application Specific Integrated Circuit). The image processing unit 41 may be configured to have a frame memory that holds R image data, G image data, and B image data.

[0045] The synchronization signal generation unit 42 generates a clock signal (synchronization signal) that serves as a reference for the operation of the processing device 4, and outputs the generated synchronization signal to the light source device 3, the image processing unit 41, the control unit 44, and the endoscope 2. Here, the synchronization signal generated by the synchronization signal generation unit 42 includes a horizontal synchronization signal and a vertical synchronization signal. Therefore, the light source device 3, the image processing unit 41, the control unit 44, and the endoscope 2 operate in synchronization with one another based on the generated synchronization signal.

[0046] The input unit 43 is realized using a keyboard, a mouse, a switch, and a touch panel, and receives input of various signals such as an operation instruction signal that instructs the operation of the endoscope system 1. The input unit 43 may include a switch provided on the operation unit 22 or a portable terminal such as an external tablet computer.

[0047] The control unit 44 controls the drive of each component including the image sensor 244 and the light source device 3, and controls input and output of information to each component. The control unit 44 refers to control information data (e.g., read timing, etc.) for image capture control stored in the storage unit 45, and transmits the control information data as a drive signal to the image sensor 244 via a predetermined signal line included in the cable assembly 245. The control unit 44 may also switch modes depending on the light to be observed. For example, the control unit 44 switches between a normal observation mode in which an image obtained by illumination with white light is observed, and a fluorescent observation mode in which a fluorescent image obtained by illumination with therapeutic light is observed. The control unit 44 is configured using a general-purpose processor such as a CPU or a dedicated processor such as various arithmetic circuits that execute specific functions, such as an ASIC.

[0048] The storage unit 45 stores various programs for operating the endoscope system 1, and data including various parameters necessary for the operation of the endoscope system 1. The storage unit 45 also stores identification information of the processing device 4. Here, the identification information includes unique information (ID), year and specification information, etc. of the processing device 4.

[0049] The storage unit 45 also stores various programs including an image acquisition processing program for executing the image acquisition processing method of the processing device 4. The various programs can be recorded on computer-readable recording media such as a hard disk, a flash memory, a CD-ROM, a DVD-ROM, or a flexible disk, and widely distributed. The above-mentioned various programs can also be acquired by downloading them via a communication network. The communication network referred to here is realized by, for example, an existing public line network, a LAN (Local Area Network), a WAN (Wide Area Network), or the like, and may be wired or wireless.

[0050] The storage unit 45 having the above configuration is realized using a ROM (Read Only Memory) in which various programs and the like are pre-installed, and a RAM, a hard disk, and the like for storing calculation parameters and data for each process.

[0051] The display device 5 displays an image for display corresponding to an image signal received from the processing device 4 (image processing unit 41) via a video cable. The display device 5 is configured using a monitor such as a liquid crystal or organic EL (Electro Luminescence) monitor.

[0052] The treatment device 6 has a treatment tool operation section 61 and a flexible treatment tool 62 extending from the treatment tool operation section 61. The treatment tool 62 used in PIT is a treatment light emitting section that emits light for treatment (hereinafter referred to as treatment light). The treatment tool operation section 61 controls the emission of treatment light from the treatment tool 62. The treatment tool operation section 61 has an operation input section 611. The operation input section 611 is composed of, for example, a switch or the like. The treatment tool operation section 61 causes the treatment tool 62 to emit treatment light by inputting to the operation input section 611 (for example, pressing a switch). In the treatment device 6, a light source that emits treatment light may be provided in the treatment tool 62 or in the treatment tool operation section 61. The light source is realized by using a semiconductor laser, an LED, or the like. For example, in the case of PIT, the treatment light is light in a wavelength band of 680 nm or more, for example, light having a central wavelength of 690 nm. Here, the illumination optical system of the treatment tool 62 may be configured to change the irradiation range of the therapeutic light. For example, under the control of the treatment tool operation unit 61, it may be configured with an optical system with a changeable focal length or a DMD (Digital Micromirror Device), and the spot diameter of the light irradiated on the subject and the shape of the irradiation range can be changed.

[0053] Next, the flow of treatment using the endoscope 2 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the flow of treatment using the endoscope according to the first embodiment of the present invention. Fig. 4 is a diagram showing an example of performing PIT, in which the insertion portion 21 is inserted into the stomach ST to perform treatment.

[0054] First, the surgeon inserts the insertion portion 21 into the stomach ST (see (a) of FIG. 4). At this time, the surgeon causes the light source device 3 to irradiate white light, and searches for the treatment position while observing the white light image of the inside of the stomach ST displayed by the display device 5. In this case, tumors B1 and B2 are treated as treatment targets. At this time, an antibody drug is administered to the tumors B1 and B2, which are the treatment target sites. The antibody drug may be administered using the endoscope 2 or other equipment, or the drug may be swallowed by the patient. The operator observes the white light image and determines the region including the tumors B1 and B2 as the irradiation region. If necessary, the operator may irradiate the irradiation region with excitation light or the like.

[0055] The surgeon aims the tip 24 at the tumor B1, protrudes the treatment tool 62 from the tip of the endoscope 2, and irradiates the tumor B1 with therapeutic light (see FIG. 4(b)). The therapeutic light causes the antibody drug bound to the tumor B1 to react, and the tumor B1 is treated.

[0056] The surgeon then aims the tip 24 at the tumor B2, protrudes the treatment tool 62 from the tip of the endoscope 2, and irradiates the tumor B2 with therapeutic light (see FIG. 4(c)). The therapeutic light causes the antibody drug bound to the tumor B2 to react, and the tumor B2 is treated.

[0057] The surgeon then aims the tip 24 at the tumor B1 and irradiates the tumor B1 with therapeutic light and excitation light from the tip of the endoscope 2 (see FIG. 4(d)). The surgeon confirms the therapeutic effect on the tumor B1 by obtaining a post-treatment fluorescent image. The therapeutic effect is confirmed by the surgeon, for example, by observing images, which will be described later. In this embodiment, an example in which fluorescence obtained by therapeutic light is observed will be described.

[0058] The surgeon also aims the tip 24 at the tumor B2 and irradiates the tumor B2 with therapeutic light from the tip of the endoscope 2 (see FIG. 4(e)). The surgeon confirms the therapeutic effect on the tumor B2 by acquiring a post-treatment fluorescent image.

[0059] If necessary, the surgeon repeats the additional irradiation of the therapeutic light and the confirmation of the therapeutic effect.

[0060] Next, the processing in the processing device 4 will be described with reference to Fig. 5. Fig. 5 is a flow chart showing an example of the processing in the processing device according to one embodiment of the present invention. Fig. 5 shows the processing in PIT when white light is irradiated to search for an object (here, an area including cancer cells bound with an antibody drug), the object is irradiated with therapeutic light, and the therapeutic effect is confirmed from the fluorescence intensity. The surgeon searches for the treatment area while observing the white light image displayed on the display device 5. Note that the processing device 4 may change the observation mode according to the light to be observed.

[0061] First, the operator operates the treatment tool 62 to irradiate the antibody drug bound to the cancer cells with therapeutic light, causing the drug to react (step S101: drug reaction step). In this drug reaction step, the antibody drug is activated by irradiation with near-infrared light, which is the therapeutic light, to destroy the cancer cells.

[0062] The controller 44 may set the observation mode to the fluorescence observation mode by irradiating the therapeutic light. In this case, the controller 44 determines whether to emit the therapeutic light, for example, in response to an operation signal received from the operation input unit 611 or a command to start irradiation of the therapeutic light input to the input unit 43 by the surgeon.

[0063] During irradiation of the therapeutic light, the endoscope 2 detects the fluorescence generated by the therapeutic light (step S102: fluorescence detection step). The therapeutic light excites the antibody drug in the subject, causing it to emit fluorescence. The fluorescence image generator 412 generates a fluorescence image or a white light image containing the fluorescence based on the imaging signal.

[0064] FIG. 6 is a diagram showing an example of an observation image including a fluorescent image. FIG. 7 is a diagram showing a fluorescent image showing the amount of fluorescence in the observation image shown in FIG. 6. FIG. 6 shows an example of an observation image obtained when white light and therapeutic light are irradiated. Observation image W1 shown in FIG. 6 shows reflected light illuminated by white light, as well as fluorescence emitted by an antibody drug when irradiated with therapeutic light. Fluorescence image F1 shown in FIG. 7 shows an image formed by fluorescence. Note that a fluorescent image can be generated by separating only the fluorescence using a prism or the like in the image sensor 244, and receiving the white light and the fluorescence at different positions in the image sensor 244, or by passing only the fluorescence using a filter or the like.

[0065] Here, the observation image W1 and the fluorescence image F1 are images of the same region of the subject. A region of interest R1 is set in the observation image W1 and the fluorescence image F1. The position of the region of interest may be set in advance with respect to the image, may be arbitrarily set by the surgeon through input to the input unit 43, or may be set by the control unit 44 by detecting a characteristic portion of the image. The control unit 44 adjusts and tracks the position of the region of interest depending on images acquired at different times and corresponding positions between the white light image and the fluorescence image. At this time, the control unit 44 detects corresponding positions between the images by a known method such as pattern matching.

[0066] After that, the gradation range setting unit 413 generates a histogram showing the occurrence frequency of the fluorescence intensity (step S103: histogram generating step). FIG. 8 is a diagram showing the relationship between the fluorescence intensity and the occurrence frequency in the region of interest. FIG. 8 is a histogram showing the occurrence frequency of the fluorescence intensity. The gradation range setting unit 413 generates, for example, the histogram shown in FIG. 8. In this histogram, the luminance of the fluorescence image in the region of interest is used for the fluorescence intensity. Note that the luminance may be divided at preset intervals, for example, 0 to 255 may be divided into 10 ranges. Specifically, for example, it is divided into 0 to 25, 26 to 51, 52 to 76, 77 to 102, 103 to 127, 128 to 153, 154 to 178, 179 to 204, 205 to 229, and 230 to 255, and each pixel is assigned according to the luminance.

[0067] After generating the histogram, the gradation range setting unit 413 sets the gradation range (step S104: gradation range setting step). The gradation range setting unit 413 sets the gradation range based on a preset occurrence frequency threshold F TH The above fluorescence intensity is set as the gradation change range. For example, in FIG. 8, the fluorescence intensity P L From the fluorescence intensity P H The gradation change range is then set by gradation range setting section 413 to the range up to the gradation change range in the fluorescence intensity distribution of each pixel of the fluorescence image.

[0068] 9 and 10 are diagrams showing an example of the fluorescence intensity distribution for the pixels of a fluorescence image. In the fluorescence intensity distribution before the gradation change shown in FIG. 9, the gradation is set from the minimum fluorescence intensity (e.g., 0) to the maximum fluorescence intensity (e.g., the upper limit value of brightness (255)). In contrast, in the fluorescence intensity distribution after the gradation change shown in FIG. 10, the minimum fluorescence intensity P L Maximum fluorescence intensity P H The gradation is set for the minimum fluorescence intensity P L The brightness of 0, the maximum fluorescence intensity P H The brightness of is set to 255, and the gradation change range R PThe gradation expansion ratio in this case corresponds to the value obtained by dividing the maximum gradation value (here, 255) by the maximum luminance value of the fluorescent image before the gradation change.

[0069] After the gradation range is set, the gradation expanded image generating unit 414 generates a gradation expanded image in which brightness is allocated to the fluorescence intensity based on the gradation range set in step S104 (step S105).

[0070] Thereafter, the control unit 44 causes the display device 5 to display the generated gradation-expanded image (step S106: display step). Fig. 11 is a diagram showing an example of a gradation-expanded image obtained by performing gradation expansion processing on the fluorescence image shown in Fig. 7. In the gradation-expanded image T1 shown in Fig. 11, the fluorescence is more clearly expressed than in the fluorescence image shown in Fig. 7.

[0071] Fig. 12 and Fig. 13 are diagrams showing other display examples of a gradation expanded image obtained by performing gradation expansion processing on the fluorescence image shown in Fig. 7. In addition to the gradation expanded image shown in Fig. 11, the gradation expanded image generating unit 414 may generate a gradation expanded image T2 (see Fig. 12) in which the gradation is inverted, or a gradation expanded image T3 (see Fig. 13) showing a color map of the amount of fluorescence to which colors according to the intensity of fluorescence are assigned.

[0072] In the first embodiment described above, a range for changing the gradation is set based on the distribution of the fluorescence intensity, and gradation is allocated within the range from the minimum fluorescence intensity to the maximum fluorescence intensity within the set gradation change range, thereby generating a gradation-expanded image that clearly expresses the fluorescence and displaying it on the display device 5. According to the first embodiment, gradation is set within a limited range of the majority fluorescence intensity according to the distribution of the fluorescence intensity of the fluorescence image, and a gradation-expanded image is generated, so that changes in the fluorescence intensity can be accurately grasped.

[0073] In PIT, the amount of fluorescence from the reagent decreases as the therapeutic light is irradiated, and the progress of treatment is understood using the amount of fluorescence. In this case, if the same dynamic range is used to display the initial high fluorescence intensity state to the low fluorescence intensity state just before the end of treatment, it becomes difficult to recognize partial differences in intensity when the fluorescence intensity is low. Therefore, by selecting the range of the fluorescence amount (fluorescence intensity) to be displayed according to the decrease in the amount of fluorescence, allocating the dynamic range to be displayed, and expanding the gradation in the low fluorescence intensity area, it becomes easier to recognize partial differences in fluorescence intensity in areas where the fluorescence intensity has decreased as treatment progresses.

[0074] 14 is a diagram for explaining the gradation expansion process according to the first embodiment of the present invention. The gradation expansion process is repeated according to the fluorescence intensity, and the maximum attenuation value P TH By setting the gradation to the range of the majority of the fluorescence intensities according to the distribution of the fluorescence intensities in the fluorescence image, it is possible to accurately grasp the change in the fluorescence intensity regardless of the magnitude of the fluorescence intensity.

[0075] (Modification of the first embodiment) Next, a modified example of the first embodiment will be described. The endoscope system according to this modified example is the same as the endoscope system 1 according to the first embodiment, and therefore the description will be omitted. In the modified example, clipping processing is performed to adjust the range of the amount of fluorescence for which gradation setting is performed according to the value of the difference in the amount of fluorescence. Fig. 15 is a diagram for explaining the clipping processing according to the modified example of the first embodiment of the present invention.

[0076] The gradation range setting unit 413 sets the range of the amount of fluorescence for which gradation setting is performed according to a preset condition. In this modification, the minimum value of the gradation setting range is a preset minimum value P MIN The maximum value of the gradation setting range is set to the maximum fluorescent light amount P H The maximum value P obtained by multiplying MAXIn this modified example, the ratio is set to, for example, 0.8 (80%). This ratio can be set arbitrarily. The minimum value of the gradation setting range may also be multiplied by a preset ratio with respect to the minimum value of the amount of fluorescence.

[0077] According to this modification, it is possible to obtain the same effects as those of the first embodiment, and also to express in detail the change in fluorescence in a part of the region where the amount of fluorescence is detected.

[0078] (Embodiment 2) Next, a second embodiment will be described. Fig. 16 is a block diagram showing a schematic configuration of an endoscope system according to the second embodiment of the present invention. The endoscope system 1A according to the second embodiment includes a processing device 4A instead of the processing device 4 of the endoscope system 1 according to the first embodiment. The configuration other than the processing device 4A is the same as that of the endoscope system 1, so a description thereof will be omitted.

[0079] The configuration of the processing device 4 A will be described. The processing device 4 A includes an image processing unit 41 A, a synchronization signal generating unit 42, an input unit 43, a control unit 44, and a storage unit 45.

[0080] The image processing section 41A has a white light image generating section 411 , a fluorescent light image generating section 412 , a gradation range setting section 413 , a gradation expanded image generating section 414 , and a difference image generating section 415 .

[0081] The difference image generating section 415 generates a difference image showing the difference in the amount of fluorescence between two fluorescence images captured at different times. Here, the difference image is an image obtained by taking the difference in the amount of fluorescence between pixels that correspond to each other in positions on the subject in the two fluorescence images.

[0082] Next, the processing in the processing device 4A will be described with reference to Fig. 17. Fig. 17 is a flowchart showing an example of the processing in the processing device according to the second embodiment.

[0083] First, before the start of treatment, the treatment light is irradiated and the fluorescence before treatment is detected (step S201: initial fluorescence detection step). This step obtains a fluorescence image before the start of treatment. This fluorescence image before the start of treatment is stored in the storage unit 45 as a reference fluorescence image.

[0084] Thereafter, the operator operates the treatment tool 62 to irradiate the antibody drug bound to the cancer cells with therapeutic light, causing the drug to react (step S202: drug reaction step). Through the drug reaction process, the treatment tool 62 irradiates the subject with therapeutic light, and treatment is performed to destroy the cancer cells.

[0085] The endoscope 2 detects the fluorescence generated by the therapeutic light (step S203: fluorescence detection step). The therapeutic light excites the antibody drug in the subject, causing it to emit fluorescence. The fluorescent light image generating unit 412 generates a fluorescent light image or a white light image containing the fluorescent light based on the imaging signal.

[0086] Thereafter, the difference image generating unit 415 generates a difference image between the fluorescence image before the start of treatment (reference fluorescence image) acquired in step S201 and the fluorescence image acquired in step S203 (step S204: difference image generating step).

[0087] Fig. 18 is a diagram showing an example of the fluorescence amount of corresponding pixels acquired at different times. Fluorescence amount curve L1 shown in Fig. 18 shows the fluorescence amount of each pixel (e.g., a pixel located in a region of interest) in the reference fluorescence image. Fluorescence amount curve L2 shows the fluorescence amount of a pixel of a fluorescence image acquired at a time different from that of the reference fluorescence image, which corresponds to the pixel position of the reference fluorescence amount. The fluorescence amount here corresponds to the luminance value of the fluorescence image. In fluorescence amount curve L2 shown in Fig. 18, the fluorescence amount has decreased in an area where a reaction has progressed due to the therapeutic light.

[0088] The difference image generating section 415 calculates the difference between the amount of fluorescence in the reference fluorescent image and the amount of fluorescence in the fluorescent image, and generates a difference image expressed by this difference. FIG. 19 is a diagram showing an example of a difference image. The difference image D1 shown in FIG. 19 is an image that expresses the difference (amount of change in fluorescence) between the amount of fluorescence in the reference fluorescent image and the amount of fluorescence in the fluorescent image. When the difference between the amount of fluorescence in the reference fluorescent image and the amount of fluorescence in the fluorescent image is small, it is difficult to visually distinguish the difference. To clearly express the difference, a gradation is set in a range equivalent to the maximum difference (maximum difference range). For example, in FIG. 18, the minimum value of the fluorescence amount difference (zero difference in this case) to the maximum value PD H Range up to R D 1 is the maximum difference range (maximum difference range R D 1).

[0089] After generating the difference image, the gradation range setting unit 413 sets the gradation range (step S205: gradation range setting step). Fig. 20 and Fig. 21 are diagrams showing an example of the distribution of the fluorescence amount difference for the pixels of the difference image and the gradation range. The gradation range setting unit 413 sets the maximum difference range R D 20, the gradation range setting unit 413 changes the upper limit setting value of the fluorescence amount difference that is set in advance based on the maximum difference range R D 1 is set as the range of gradation change. By changing the gradation range, the fluorescence amount difference changes from the minimum value to the maximum value PD H Brightness is allocated up to (see Figure 21).

[0090] After the gradation range is set, the gradation expanded image generating unit 414 generates a gradation expanded image in which brightness is allocated to the fluorescence intensity based on the gradation range set in step S205 (step S206).

[0091] Thereafter, the control unit 44 causes the display device 5 to display the generated gradation expanded image (step S207: display step). FIG. 22 is a diagram showing an example of a gradation expanded image obtained by performing gradation expansion processing on the difference image shown in FIG. 19. In the gradation expanded image T4 shown in FIG. 22, the difference in the amount of fluorescence is more clearly expressed compared to the difference image shown in FIG. 19. Note that, in addition to the gradation expanded image shown in FIG. 22, a gradation expanded image with inverted gradation (see FIG. 12) or a gradation expanded image showing a color map of the amount of fluorescence to which colors according to the intensity of fluorescence are assigned (see FIG. 13) may be generated.

[0092] The above-mentioned gradation change process is also executed when the treatment progresses and the fluorescence amount has sufficiently decreased compared to the amount of fluorescence before the treatment. Fig. 23 is a diagram showing an example of the fluorescence amount of a corresponding pixel acquired at a different time. The fluorescence amount curve L3 shown in Fig. 23 shows the fluorescence amount of a pixel corresponding to the pixel position of the reference fluorescence amount, which is a pixel of a fluorescence image acquired at a time when the treatment has progressed, which is different from the reference fluorescence image.

[0093] The difference image generating section 415 calculates the difference between the amount of fluorescence in the reference fluorescent image and the amount of fluorescence in the fluorescent image, and generates a difference image expressed by this difference. Fig. 24 is a diagram showing an example of the difference image. The difference image D2 shown in Fig. 24 is an image that expresses the difference (amount of change in fluorescence) between the amount of fluorescence in the reference fluorescent image and the amount of fluorescence in the fluorescent image.

[0094] 25 and 26 are diagrams showing an example of the distribution of the fluorescence amount difference for pixels of the difference image and an example of the gradation range. D 2, the gradation range is changed by changing the upper limit setting value of the fluorescence amount difference that is set in advance. For example, the gradation range setting unit 413 changes the maximum difference range R D 2 is set as the range of gradation change. By changing the gradation range, the fluorescence amount difference changes from the minimum value (zero in this case) to the maximum value PD H Brightness is allocated up to (see Figure 26).

[0095] After setting the gradation range, the gradation expanded image generating unit 414 generates a gradation expanded image in which brightness is allocated to the fluorescence intensity based on the set gradation range. Thereafter, the control unit 44 causes the display device 5 to display the generated gradation expanded image. Fig. 27 is a diagram showing an example of a gradation expanded image obtained by performing gradation expansion processing on the difference image shown in Fig. 24. In the gradation expanded image T5 shown in Fig. 27, the difference in the amount of fluorescence is more clearly expressed compared to the difference image shown in Fig. 24.

[0096] In the second embodiment described above, a range for changing gradation is set based on the difference in fluorescence intensity, and gradation is allocated within the set gradation change range, thereby generating a gradation-expanded image that clearly expresses the fluorescence and displaying it on the display device 5. According to the second embodiment, gradation is set within a limited range of the majority fluorescence intensity according to the distribution of the fluorescence intensity of the fluorescence image, and a gradation-expanded image is generated, so that changes in the fluorescence intensity can be accurately grasped.

[0097] Furthermore, in the second embodiment, a gradation range is set for the difference in fluorescence intensity, and the gradation of the difference image expressing the difference is changed and displayed, so that the surgeon can directly visually confirm the change in the amount of fluorescence. According to the second embodiment, the surgeon can more accurately grasp the change in the amount of fluorescence.

[0098] It should be noted that the clipping process of the above-mentioned modified example can be adopted in the second embodiment. In this case, the gradation setting range is determined by the preset minimum and maximum values ​​or the ratio to the difference.

[0099] (Embodiment 3) Next, a third embodiment will be described. The endoscope system according to the third embodiment is the same as the endoscope system 1A according to the second embodiment, and therefore the description will be omitted. In the third embodiment, the storage unit 45 stores an attenuation target image of the amount of fluorescence. This attenuation target image corresponds to a fluorescence image for which treatment is determined to be complete. FIG. 28 is a flowchart showing an example of processing by a processing device according to the third embodiment.

[0100] First, before starting treatment, the therapeutic light is irradiated and the fluorescence before treatment is detected in the same manner as in step S201 of FIG. 17 (step S301: initial fluorescence detection step).

[0101] Thereafter, by the operation of the surgeon, the treatment tool 62 irradiates the antibody drug bound to the cancer cells with therapeutic light, causing the drug to react (step S302: drug reaction step). The drug reaction process provides a treatment that destroys the cancer cells.

[0102] The endoscope 2 detects the fluorescence generated by the therapeutic light (step S303: fluorescence detection step). The therapeutic light excites the antibody drug in the subject, causing it to emit fluorescence. The fluorescent light image generating unit 412 generates a fluorescent light image or a white light image containing the fluorescent light based on the imaging signal.

[0103] Thereafter, the control unit 44 refers to the storage unit 45, reads out the attenuation target image, and calculates the maximum value of the attenuation target difference value between the attenuation target image and the reference fluorescent image (step S304: attenuation target value calculation step).

[0104] Fig. 29 is a diagram showing an example of the fluorescence amount of corresponding pixels acquired at different times. Fluorescence amount curve L1 shown in Fig. 29 shows the fluorescence amount of each pixel (e.g., a pixel located in a region of interest) in the reference fluorescence image. Fluorescence amount curve L2 shows the fluorescence amount of a pixel of a fluorescence image acquired at a time different from the reference fluorescence image, which corresponds to the pixel position of the reference fluorescence amount during treatment. Fluorescence amount curve L4 shows the fluorescence amount in the attenuation target image. The difference between fluorescence amount curve L1 and fluorescence amount curve L4 is the attenuation target difference value, and its maximum value is the maximum attenuation difference PD H It becomes.

[0105] Thereafter, the difference image generating unit 415 generates a difference image between the fluorescence image before the start of treatment (reference fluorescence image) acquired in step S201 and the fluorescence image acquired in step S203 (step S305: difference image generating step).

[0106] After generating the difference image, the gradation range setting unit 413 sets the gradation range (step S306: gradation range setting step). Fig. 30 and Fig. 31 are diagrams showing an example of the distribution of the fluorescence amount difference for the pixels of the fluorescence image and the gradation range. The gradation range setting unit 413 sets the maximum difference range R D 1, the gradation range is changed by changing a preset upper limit setting value of the fluorescence amount difference (see FIG. 30). For example, in the case of a fluorescence image showing the fluorescence amount curve L2, the gradation range setting unit 413 calculates the difference between the fluorescence amount curve L1 and the fluorescence amount curve L4, and sets the maximum value PD H The range in which the fluorescence amount difference is maximized is set as the gradation change range (see FIG. 31). By changing the gradation range, the fluorescence amount difference is increased from the minimum value (zero in this case) to the maximum value PD H Brightness is allocated up to

[0107] After the gradation range is set, the gradation expanded image generating unit 414 generates a gradation expanded image in which brightness is allocated to the fluorescence intensity based on the gradation range set in step S306 (step S307).

[0108] After that, the control unit 44 causes the display device 5 to display the generated gradation expanded image (Step S308: display step).

[0109] In the third embodiment described above, a range for changing gradation is set based on the difference in fluorescence intensity, and gradation is allocated within the set gradation change range, thereby generating a gradation-expanded image that clearly expresses the fluorescence and displaying it on the display device 5. According to the third embodiment, gradation is set within a limited range of the majority fluorescence intensity according to the distribution of the fluorescence intensity of the fluorescence image, and a gradation-expanded image is generated, so that changes in the fluorescence intensity can be accurately grasped.

[0110] Furthermore, in the third embodiment, the gradation range is set using the fluorescence intensity of the attenuation target, and the gradation of the difference image expressing the difference is changed and displayed, so that the change in the amount of fluorescence that determines the completion of treatment can be confirmed more reliably. According to the third embodiment, the operator can more accurately grasp the change in the amount of fluorescence.

[0111] In the above-mentioned third embodiment, an example has been described in which the gradation range is set with the minimum value of the fluorescence amount set to zero in the gradation setting process, but the surgeon may set the minimum value for the difference image or the fluorescence image. FIG. 32 is a diagram for explaining an example of setting the minimum value of the gradation range. In the setting screen Q1 shown in FIG. 32, the level of the fluorescence amount (or luminance value) at the position of the cursor C1 is displayed. The surgeon can place the cursor C1 at the position to be set as the minimum value, and while checking the level of the fluorescence amount (or luminance value) at that position, set that level as the minimum value. This setting of the minimum value can also be applied to the first and second embodiments.

[0112] In the above-described embodiment, the therapeutic light is used for both treatment and excitation of the antibody drug, but the light source device may be provided with an excitation light source that emits excitation light to excite the antibody drug.

[0113] In the above embodiment, the light source device 3 is separate from the processing device 4, but the light source device 3 and the processing device 4 may be integrated together. In the above embodiment, the therapeutic light is irradiated by a treatment tool, but the light source device 3 may emit the therapeutic light.

[0114] In addition, in the above-mentioned embodiment, the endoscopic system of the present invention has been described as an endoscopic system 1 using a flexible endoscope 2 whose observation target is biological tissue inside a subject, etc., but the present invention can also be applied to an endoscopic system using a rigid endoscope, an industrial endoscope for observing material properties, a fiberscope, an optical endoscope such as an optical scope with a camera head connected to the eyepiece. [Industrial Applicability]

[0115] INDUSTRIAL APPLICABILITY As described above, the image processing device, photoimmunotherapy system, image processing method and image processing program according to the present invention are useful for accurately grasping changes in fluorescence intensity. [Explanation of symbols]

[0116] 1. 1A Endoscope System 2 Endoscope 3 Light source device 4, 4A Processing Equipment 5 Display device 6 Treatment device 21 Insertion section 22 Control section 23 Universal Code 24 Tip 25 Curved section 26 Flexible tube section 31 Light source section 32 Lighting control section 33 Light source driver 41 Image processing section 42 Synchronization signal generator 43 Input section 44 Control section 45 Storage section 61 Treatment tool operation unit 62 Treatment tools 311 White light source 411 White light image generation unit 412 Fluorescence Image Generation Unit 413 Tone range setting section 414 Gradation expansion image generation unit 415 Differential Image Generation Unit

Claims

1. a fluorescence image acquisition unit for acquiring a fluorescence image obtained by irradiating the therapeutic light with a drug; a gradation range setting unit that sets a gradation range of the image based on a distribution of the fluorescent intensity of the fluorescent image; a gradation expansion image generating unit that generates a gradation expansion image by allocating gradations according to the gradation range set by the gradation range setting unit; Equipped with the fluorescence image acquisition unit acquires an initial fluorescence image at the start of irradiation of the therapeutic light; a difference image generating unit that generates a difference image representing a difference in fluorescence intensity between the initial fluorescence image and the fluorescence image obtained upon irradiation with the therapeutic light, the gradation range setting unit sets a gradation range for the difference; Image processing device.

2. the gradation range setting unit performs clipping processing to set a gradation range in accordance with a condition that is preset for a maximum value or a minimum value of a difference of the difference image; The image processing device according to claim 1 .

3. a therapeutic light emitting unit that emits therapeutic light to cause a drug reaction; a fluorescence image acquisition unit for acquiring an initial fluorescence image at the start of irradiation of the therapeutic light and a fluorescence image during the therapeutic light irradiation; a difference image generating unit that generates a difference image representing a difference in fluorescence intensity between the initial fluorescence image and the fluorescence image obtained upon irradiation with the therapeutic light; a gradation range setting unit that sets a gradation range of the image based on a distribution of the fluorescent intensity of the fluorescent image; a gradation expansion image generating unit that generates a gradation expansion image by allocating gradations according to the gradation range set by the gradation range setting unit; Equipped with the gradation range setting unit sets a gradation range for the difference; Photoimmunotherapy system.

4. a fluorescence image acquiring step in which a fluorescence image acquiring unit acquires an initial fluorescence image at the start of irradiation of the therapeutic light causing a drug reaction, and a fluorescence image during the irradiation of the therapeutic light; a difference image generating step in which a difference image generating unit generates a difference image representing a difference in fluorescence intensity between the initial fluorescence image and the fluorescence image obtained upon irradiation with the therapeutic light; a gradation range setting step in which a gradation range setting unit sets a gradation range of the image for the difference; a gradation-expanded image generating step in which a gradation-expanded image generating unit generates a gradation-expanded image by allocating gradations according to the gradation range set by the gradation range setting unit; An image processing method comprising:

5. a fluorescence image acquisition step of acquiring an initial fluorescence image at the start of irradiation of the therapeutic light for causing a drug reaction, and a fluorescence image during the irradiation of the therapeutic light; a difference image generating step in which a difference image generating unit generates a difference image representing a difference in fluorescence intensity between the initial fluorescence image and the fluorescence image obtained upon irradiation with the therapeutic light; a tone range setting step of setting a tone range of an image for the difference; a gradation-expanded image generating step of generating a gradation-expanded image by allocating gradations according to the gradation range set in the gradation range setting step; An image processing program that causes a computer to execute the above.

6. a fluorescence image acquisition unit for acquiring an initial fluorescence image at the start of irradiation of the therapeutic light for causing a drug reaction, and a fluorescence image during the irradiation of the therapeutic light; a difference image generating unit that generates a difference image representing a difference in fluorescence intensity between the initial fluorescence image and the fluorescence image obtained upon irradiation with the therapeutic light; a gradation range setting unit that sets a gradation range having a maximum value of a difference between the amount of fluorescence in the initial fluorescence image and the amount of fluorescence in an attenuation target image corresponding to a fluorescence image judged to be at the completion of treatment; a gradation expansion image generating unit that generates a gradation expansion image by allocating gradations according to the gradation range set by the gradation range setting unit; An image processing device comprising:

7. a therapeutic light emitting unit that emits therapeutic light to cause a drug reaction; a fluorescence image acquisition unit for acquiring an initial fluorescence image at the start of the therapeutic light irradiation and a fluorescence image during the therapeutic light irradiation; a difference image generating unit that generates a difference image representing a difference in fluorescence intensity between the initial fluorescence image and the fluorescence image obtained upon irradiation with the therapeutic light; a gradation range setting unit that sets a gradation range having a maximum value of a difference between the amount of fluorescence in the initial fluorescence image and the amount of fluorescence in an attenuation target image corresponding to a fluorescence image judged to be at the completion of treatment; a gradation expansion image generating unit that generates a gradation expansion image by allocating gradations according to the gradation range set by the gradation range setting unit; A photoimmunotherapy system comprising:

8. a fluorescence image acquiring step in which a fluorescence image acquiring unit acquires an initial fluorescence image at the start of irradiation of the therapeutic light causing a drug reaction, and a fluorescence image during the irradiation of the therapeutic light; a difference image generating step in which a difference image generating unit generates a difference image representing a difference in fluorescence intensity between the initial fluorescence image and the fluorescence image obtained upon irradiation with the therapeutic light; a gradation range setting step in which a gradation range setting unit sets a gradation range having a maximum value of a difference between a fluorescence amount in the initial fluorescence image and a fluorescence amount in an attenuation target image corresponding to a fluorescence image judged to have completed treatment; a gradation-expanded image generating step in which a gradation-expanded image generating unit generates a gradation-expanded image by allocating gradations according to the gradation range set by the gradation range setting unit; An image processing method comprising:

9. a fluorescence image acquisition step of acquiring an initial fluorescence image at the start of irradiation of the therapeutic light for causing a drug reaction, and a fluorescence image during the irradiation of the therapeutic light; a difference image generating step of generating a difference image representing a difference in fluorescence intensity between the initial fluorescence image and the fluorescence image obtained upon irradiation with the therapeutic light; a gradation range setting step for setting a gradation range having a maximum value of a difference between the amount of fluorescence in the initial fluorescence image and the amount of fluorescence in an attenuation target image corresponding to a fluorescence image judged to be at the completion of treatment; a gradation-expanded image generating step of generating a gradation-expanded image by allocating gradations according to the gradation range set in the gradation range setting step; An image processing program that causes a computer to execute the above.

Images