Learning device and medical image processing device

Abstract

The learning device 3 includes: a training image acquisition unit 321, which acquires a training image that is a pair of a first training image and a second training image, the first training image captures light from a sample irradiated with light of a first band, and the second training image captures light from a sample irradiated with light of a second band different from the first band; a singular area designation unit 322, which designates a singular area within the second training image; a first feature data extraction unit 323, which extracts feature data of a singular corresponding area at a pixel position corresponding to the singular area within the first training image; and a singular corresponding area learning unit 324, which performs machine learning on the singular corresponding area based on the feature data and generates a learning model.

Description

Technical Field

[0001] The present disclosure relates to a learning device and a medical image processing device. Background Art

[0002] Conventionally, there is known a rigid endoscope system which administers a fluorescent substance such as indocyanine green into a living body, emits excitation light to excite the fluorescent substance to an observation target, and performs fluorescent observation of a lesion site where the fluorescent substance is accumulated (for example, see Patent Document 1).

[0003] In the rigid endoscope system described in Patent Document 1, the following first captured image and second captured image are acquired, and a superimposed image is generated by superimposing corresponding pixels of the first captured image and the second captured image.

[0004] The first captured image is an image captured in such a manner that ordinary light as white light is emitted toward the observation target and the ordinary light reflected by the observation target is captured by the imaging element.

[0005] The second captured image is an image captured in the following manner: That is, excitation light that excites a fluorescent substance such as indocyanine green is emitted to an observation target, and fluorescence from the observation target excited by the excitation light is captured by a high-sensitivity imaging element.

[0006] Reference List

[0007] Patent Literature

[0008] Patent document 1: JP 2015-29841 A. Summary of the invention

[0009] Technical issues

[0010] However, in the rigid endoscope system described in Patent Document 1, in order to observe the lesion site (the area where fluorescence is emitted from the observation target (hereinafter, referred to as the fluorescence area)), it is necessary to administer a fluorescent substance such as indocyanine green into the living body. For example, when the fluorescence area can be estimated from the first captured image without administering the drug, convenience can be improved.

[0011] The present disclosure has been provided in view of the foregoing, and is intended to provide a learning device and a medical image processing device capable of improving convenience.

[0012] Solution to the problem

[0013] In order to solve the above problems and achieve the purpose, the learning device according to the present disclosure includes: a training image acquisition unit, configured to acquire a training image of a pair of a first training image and a second training image, wherein the first training image is acquired by capturing light from an object irradiated with light of a first band, and the second training image is acquired by capturing light from an object irradiated with light of a second band different from the first band; a singular area designation unit, configured to designate a singular area in the second training image; a first feature data extraction unit, configured to extract feature data of a singular corresponding area at a pixel position corresponding to the singular area in the first training image; and a singular corresponding area learning unit, configured to generate a learning model by performing machine learning on the singular corresponding area based on the feature data.

[0014] In addition, in the above-mentioned learning device according to the present disclosure, the object emits fluorescence by being irradiated with excitation light of a second band, a second training image is acquired by capturing the fluorescence from the object irradiated with the excitation light, and the singular area designation unit is configured to designate an area in which the intensity of the component of the fluorescence in the second training image is equal to or higher than a specific threshold as a singular area.

[0015] Furthermore, in the above-mentioned learning device according to the present disclosure, the singular region specifying unit is configured to specify a region in the second training image where the pixel level is equal to or higher than a certain threshold as the singular region.

[0016] In addition, in the above-mentioned learning device according to the present disclosure, the singular area designation unit is configured to designate each of a first singular area having a pixel level within a first range and a second singular area having a pixel level within a second range higher than the first range as a singular area, the first feature data extraction unit is configured to extract feature data of each of a first singular corresponding area and a second singular corresponding area, the first singular corresponding area is a singular corresponding area in a first training image and is located at a pixel position corresponding to the first singular area, the second singular corresponding area is a singular corresponding area in the first training image and is located at a pixel position corresponding to the second singular area, and the singular corresponding area learning unit is configured to perform machine learning on each of the first singular corresponding area and the second singular corresponding area based on the feature data.

[0017] In addition, in the above-mentioned learning device according to the present disclosure, the feature data includes feature data related to at least one of color and brightness, and the singular corresponding area learning unit is configured to: when machine learning is performed on the singular corresponding area and a learning model is generated, the weight of the feature data related to the blue component is lower than the weight of the feature data related to another color component.

[0018] Furthermore, in the above-mentioned learning device according to the present disclosure, the light of the first wavelength band is emitted in the first period of the alternately repeated first period and second period, and the light of the second wavelength band is emitted in the second period.

[0019] According to the present disclosure, a medical image processing device includes: a captured image acquisition unit, configured to acquire a captured image acquired by capturing light from an observation target irradiated with light of a first wavelength band; a second feature data extraction unit, configured to extract feature data of each area of ​​the captured image; and a singular corresponding area designation unit, configured to designate a singular corresponding area in the captured image based on the feature data by using a learning model constructed by machine learning, wherein the learning model is generated by performing machine learning on the singular corresponding area based on the feature data of the singular corresponding area at a pixel position in the first training image corresponding to the singular area in the second training image using a training image paired with a first training image and a second training image, wherein the first training image is acquired by capturing light from an object irradiated with light of a first wavelength band, and the second training image is acquired by capturing light from an object irradiated with light of a second wavelength band different from the first wavelength band.

[0020] In addition, in the above-mentioned medical image processing device according to the present disclosure, the object emits fluorescence by being irradiated with excitation light of a second band, a second training image is acquired by capturing the fluorescence from the object irradiated with the excitation light, and the singular area is an area where the intensity of the fluorescence component in the second training image is equal to or higher than a specific threshold.

[0021] Furthermore, in the above-mentioned medical image processing apparatus according to the present disclosure, the singular region is a region in the second training image where the pixel level is equal to or greater than a specific threshold.

[0022] In addition, in the above-mentioned medical image processing device according to the present disclosure, the singular region includes: a first singular region whose pixel level is within a first range; and a second singular region whose pixel level is within a second range higher than the first range, and based on the feature data of the first singular corresponding region and the second singular corresponding region, a learning model is generated by performing machine learning on each of the first singular corresponding region and the second singular corresponding region, the first singular corresponding region is a singular corresponding region in the first training image and is located at a pixel position corresponding to the first singular region, and the second singular corresponding region is a singular corresponding region in the first training image and is located at a pixel position corresponding to the second singular region.

[0023] In addition, in the above-mentioned medical image processing device according to the present disclosure, the feature data includes feature data related to at least one of color and brightness, and a learning model is generated by performing machine learning on singular corresponding areas in a state where the weight of the feature data related to the blue component is lower than the weight of the feature data related to another color component.

[0024] Furthermore, in the above-mentioned medical image processing apparatus according to the present disclosure, light of the first wavelength band is emitted in the first period of the alternately repeated first period and second period; and light of the second wavelength band is emitted in the second period.

[0025] In addition, the above-mentioned medical image processing apparatus according to the present disclosure further includes: a display controller configured to generate a display image in which the singular corresponding region is displayed in a manner distinguishable from another region in the captured image.

[0026] Advantageous Effects of the Invention

[0027] The learning device and the medical image processing device according to the present disclosure can improve convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a diagram showing a configuration of a learning system according to an embodiment.

[0029] Figure 2 is a diagram showing a configuration of a training image generating device.

[0030] Figure 3 is a diagram showing the configuration of a camera head and a control device.

[0031] Figure 4 is a flowchart showing the operation of the training image generating apparatus.

[0032] Figure 5 is a diagram for describing the operation of the training image generating apparatus.

[0033] Figure 6 is a diagram for describing the operation of the training image generating apparatus.

[0034] Figure 7 is a diagram showing the configuration of a learning device.

[0035] Figure 8 is a flowchart showing the operation of the learning device.

[0036] Fig. 9 is a diagram showing a configuration of a medical observation apparatus.

[0037] Fig.10 is a diagram showing the configuration of a camera head and a control device.

[0038] Fig.11 is a flowchart showing the operation of the medical observation device.

[0039] Fig.12 is a diagram for describing the operation of the medical observation apparatus. DETAILED DESCRIPTION

[0040] Hereinafter, a mode for carrying out the present disclosure (hereinafter, an embodiment) will be described with reference to the accompanying drawings. Note that the present disclosure is not limited to the embodiments described below. In addition, in the accompanying drawings, the same reference numerals are assigned to the same parts.

[0041] [1. Schematic configuration of learning system]

[0042] Figure 1 is a diagram showing the configuration of a learning system 1 according to the present embodiment.

[0043] The learning system 1 is a system for generating a learning model for estimating a fluorescent region (singular region) that emits fluorescent light when near-infrared excitation light is emitted to an observation target. Figure 1 As shown, the learning system 1 includes a training image generating device 2 and a learning device 3. Then, these training image generating device 2 and the learning device 3 are connected via a network NE ( Figure 1 ) performs wired or wireless communication.

[0044] Note that although Figure 1 Only one training image generating device 2 is shown in FIG. 1 , but the number of training image generating devices 2 is not limited to one and may be plural.

[0045] [2. Configuration of training image generation device]

[0046] First, the configuration of the training image generating device 2 will be described.

[0047] Figure 2 is a diagram showing the configuration of the training image generating device 2 .

[0048] The training image generating device 2 is a device for generating a training image for machine learning in the learning device 3. Figure 2 As shown, the training image generating device 2 includes an insertion unit 21, a light source device 22, a light guide 23, a camera head 24, a first transmission cable 25, a display device 26, a second transmission cable 27, a control device 28 and a third transmission cable 29.

[0049] The insertion unit 21 is a rigid endoscope. That is, the insertion unit 21 has an elongated shape that is rigid as a whole or partially flexible and partially rigid, and is inserted into a living body. An optical system is provided in the insertion unit 21, which includes one or more lenses and converges light from inside a living body (object).

[0050] One end of the light guide 23 is connected to the light source device 22, and under the control of the control device 28, light emitted to the inside of the biological body is supplied therefrom to one end of the light guide 23. Figure 2 As shown, the light source device 22 includes a first light source 221 and a second light source 222 .

[0051] The first light source 221 emits ordinary light of a first wavelength band. In this embodiment, the first light source 221 includes a light emitting diode (LED) that emits white light.

[0052] The second light source 222 emits excitation light of a second wavelength band different from the first wavelength band. In the present embodiment, a semiconductor laser emitting near-infrared excitation light of a near-infrared wavelength band is included. The near-infrared excitation light is excitation light that excites a fluorescent substance such as indocyanine green. In addition, when excited by the near-infrared excitation light, the fluorescent substance such as indocyanine green emits fluorescence having a central wavelength on the longer wavelength side of the central wavelength of the wavelength band of the near-infrared excitation light. Note that the wavelength band of the near-infrared excitation light and the wavelength band of the fluorescence can be set to partially overlap, or can be set to not overlap at all.

[0053] Then, in the light source device 22, the first light source 221 is driven in the first period between the first period and the second period that are alternately repeated under the control of the control device 28. That is, the light source device 22 emits ordinary light (white light) in the first period. In addition, in the light source device 22, the second light source 222 is driven in the second period under the control of the control device 28. That is, the light source device 22 emits near-infrared excitation light in the second period.

[0054] Note that although the light source device 22 is configured separately from the control device 28 in the present embodiment, this is not a limitation, and a configuration in which it is provided inside the control device 28 may be employed.

[0055] One end of the light guide 23 is detachably connected to the light source device 22, and the other end thereof is detachably connected to the insertion unit 21. Then, the light guide 23 transmits the light (ordinary light or near-infrared excitation light) provided from the light source device 22 from one end to the other end, and provides the light to the insertion unit 21. In the case of emitting ordinary light (white light) into the interior of the biological body, the ordinary light reflected inside the biological body is converged by the optical system in the insertion unit 21. Note that, for the sake of convenience of description, the ordinary light converged by the optical system in the insertion unit 21 is referred to as the first object image below. In addition, in the case of emitting near-infrared excitation light into the interior of the biological body, the near-infrared excitation light reflected inside the biological body and the fluorescence emitted from the excited fluorescent substance such as indocyanine green accumulated in the lesion part in the biological body are converged by the optical system in the insertion unit 21. Note that, for the sake of convenience of description, the fluorescence that passes through the excitation light cutoff filter 242a (described later) after the optical system in the insertion unit 21 converges the near-infrared excitation light and the fluorescence is referred to as the second object image below.

[0056] The camera head 24 is detachably connected to the base end (eyepiece 211 ( Figure 2 )). Then, under the control of the control device 28, the camera head 24 captures the first object image (ordinary light) and the second object image (fluorescence), and outputs an image signal (RAW signal) acquired by imaging. The image signal is, for example, an image signal of 4K or larger.

[0057] Note that the detailed configuration of the camera head 24 will be described in “2-1. Configuration of Camera Head” described later.

[0058] One end of the first transmission cable 25 is connected to the connector CN1 ( Figure 2 ) is detachably connected to the control device 28, and its other end is connected via the connector CN2 ( Figure 2 ) is detachably connected to the camera head 24. Then, the first transmission cable 25 transmits the image signal and the like output from the camera head 24 to the control device 28, and also transmits the control signal, synchronization signal, clock, power and the like output from the control device 28 to the camera head 24.

[0059] Note that when the image signal and the like are transmitted from the camera head 24 to the control device 28 via the first transmission cable 25, the image signal and the like may be transmitted as an optical signal or may be transmitted as an electrical signal. The same applies to the transmission of a control signal, a synchronization signal, and a clock from the control device 28 to the camera head 24 via the first transmission cable 25.

[0060] The display device 26 includes a display using liquid crystal, organic electroluminescence (EL), or the like, and displays an image based on a video signal from the control device 28 under the control of the control device 28 .

[0061] One end of the second transmission cable 27 is detachably connected to the display device 26, and the other end thereof is detachably connected to the control device 28. Then, the second transmission cable 27 transmits the video signal processed by the control device 28 to the display device 26.

[0062] The control device 28 includes a central processing unit (CPU), a field programmable gate array (FPGA), etc., and comprehensively controls the operations of the light source device 22 , the camera head 24 , and the display device 26 .

[0063] Note that the detailed configuration of the control device 28 will be described in “2-2. Configuration of Control Device”.

[0064] One end of the third transmission cable 29 is detachably connected to the light source device 22, and the other end thereof is detachably connected to the control device 28. Then, the third transmission cable 29 transmits a control signal from the control device 28 to the light source device 22.

[0065] [2-1. Camera head configuration]

[0066] Next, the configuration of the camera head 24 will be described.

[0067] Figure 3 2 is a diagram showing the configuration of the camera head 24 and the control device 28 .

[0068] Note that for ease of description, Figure 3 The connectors CN1 and CN2 between the control device 28 and the camera head 24 and the first transmission cable 25, the connector between the control device 28 and the display device 26 and the second transmission cable 27, and the connector between the control device 28 and the light source device 22 and the third transmission cable 29 are omitted.

[0069] like Figure 3 As shown, the camera head 24 includes a lens unit 241 , an imaging unit 242 , and a first communication unit 243 .

[0070] The lens unit 241 includes one or more lenses, and forms a first object image (ordinary light) and a second object image (fluorescent light) on an imaging surface of the imaging unit 242 (imaging element 242 b ).

[0071] The imaging unit 242 captures the interior of the organism under the control of the control device 28. Figure 3 As shown, the imaging unit 242 includes an excitation light cutoff filter 242a, an imaging element 242b, and a signal processing unit 242 c .

[0072] The excitation light cutoff filter 242a is disposed between the lens unit 241 and the imaging element 242b, and includes a band-stop filter that removes a specific wavelength band. That is, the excitation light cutoff filter 242a is arranged on the optical path of the near-infrared excitation light reflected from the inside of the biological body to the imaging element 242b. Note that, for ease of description, the wavelength band cutoff (removed) by the excitation light cutoff filter 242a is referred to as the cutoff frequency band, the wavelength band on the short wavelength side of the cutoff frequency band and passing through the excitation light cutoff filter 242a is referred to as the short-wave side transmission frequency band, and the wavelength band on the long wavelength side of the cutoff frequency band and passing through the excitation light cutoff filter 242a is referred to as the long-wave side transmission frequency band.

[0073] Here, the cutoff band includes at least a portion of the band of the near-infrared excitation light. In the present embodiment, the cutoff band includes the entire band of the near-infrared excitation light. In addition, the long-wave side transmission band includes the entire band of the fluorescence. In addition, the short-wave side transmission band includes the entire band of ordinary light (white light).

[0074] That is, the excitation light cutoff filter 242a transmits the first object image (normal light (white light)) from the lens unit 241 toward the imaging element 242b. On the other hand, for the near-infrared excitation light and fluorescence from the lens unit 241 toward the imaging element 242b, the excitation light cutoff filter 242a cuts off (removes) the near-infrared excitation light and transmits the fluorescence (second object image).

[0075] The imaging element 242b includes a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS), etc., which receives light passing through the excitation light cutoff filter 242a and converts it into an electrical signal (analog signal).

[0076] Here, the imaging surface (light receiving surface) of the imaging element 242b is provided with a color filter 242d ( Figure 3 ), wherein three filter groups grouped according to wavelength bands of transmitted light (red (R), green (G), and blue (B)) are arranged in a predetermined format (such as a Bayer array).

[0077] Specifically, the color filter 242d includes an R filter group that mainly transmits light in the R band, a B filter group that mainly transmits light in the B band, a first G filter group that mainly transmits light in the G band (configured in the same column as the R filter group), and a second G filter group that mainly transmits light in the G band (configured in the same column as the B filter group). Note that for ease of description, the first G filter group and the second G filter group are collectively referred to as the G filter group below.

[0078] Here, each of the R filter group, the G filter group, and the B filter group also transmits fluorescence. In addition, the imaging element 242b has sensitivity not only to light in the R, G, and B wavelength bands but also to light in the wavelength band of fluorescence.

[0079] Then, the imaging element 242b performs imaging in each first period and second period that are alternately repeated in synchronization with the light emission timing of the light source device 22 under the control of the control device 28. Hereinafter, for the sake of convenience of description, the image generated by capturing the first object image (ordinary light) by the imaging element 242b in the first period is referred to as the first training image, and the image generated by capturing the second object image (fluorescence) by the imaging element 242b in the second period is referred to as the second training image. In addition, the first training image and the second training image are collectively referred to as training images.

[0080] Signal processing unit 242 c Signal processing is performed on the training image (analog signal) generated by the imaging element 242 b and the training image (RAW signal (digital signal)) is output.

[0081] The first communication unit 243 functions as a transmitter that transmits the training image (RAW signal (digital signal)) output from the imaging unit 242 to the control device 28 through the first transmission cable 25. The first communication unit 243 includes, for example, a high-speed serial interface that performs communication of the training image with the control device 28 through the first transmission cable 25 at a transmission rate equal to or higher than 1 Gbps.

[0082] [2-2. Configuration of control device]

[0083] Next, we will refer to Figure 3 The configuration of the control device 28 is described.

[0084] like Figure 3 As shown, the control device 28 includes a second communication unit 281 , a memory 282 , an image generation unit 283 , a control unit 284 , an input unit 285 , an output unit 286 , a storage unit 287 , and a third communication unit 288 .

[0085] The second communication unit 281 functions as a receiver that receives the training image (RAW signal (digital signal)) output from the camera head 24 (first communication unit 243) through the first transmission cable 25. The second communication unit 281 includes, for example, a high-speed serial interface that performs communication of the training image with the first communication unit 243 at a transmission rate equal to or higher than 1 Gbps.

[0086] The memory 282 includes, for example, a dynamic random access memory (DRAM), etc. The memory 282 can temporarily store a plurality of frames of training images sequentially output from the camera head 24 (first communication unit 243).

[0087] The image generation unit 283 processes the training images (RAW signals (digital signals)) sequentially output from the camera head 24 (first communication unit 243) and received by the second communication unit 281 under the control of the control unit 284. Figure 3 As shown, the image generating unit 283 includes a memory controller 283a, a first image processing unit 283b, a second image processing unit 283c and a display controller 283d.

[0088] The memory controller 283a controls writing and reading of training images into and from the memory 282. More specifically, the memory controller 283a sequentially writes the training images (first training image and second training image) sequentially output from the camera head 24 (first communication unit 243) and received by the second communication unit 281 into the memory 282. In addition, the memory controller 283a reads the first training image from the memory 282 at a specific timing, and causes the first image processing unit 283b to input the read first training image. In addition, the memory controller 283a reads the second training image from the memory 282 at a specific timing, and causes the second image processing unit 283c to input the read second training image.

[0089] The first image processing unit 283 b performs first image processing on the input first training image (RAW signal (digital signal)).

[0090] Examples of the first image processing include optical black subtraction processing, white balance adjustment processing, demosaicing processing, color correction processing, gamma correction processing, and conversion of an RGB signal (first training image) into a luminance signal and color difference signals (Y and C). B / CR signal) of YC processing.

[0091] Second image processing unit 283 c A second image processing different from the first image processing is performed on the input second training image (RAW signal (digital signal)).

[0092] Examples of the second image processing include a process of generating only a luminance signal (Y signal) from an input second training image (RAW signal (digital signal)).

[0093] The display controller 283d generates a video signal for displaying at least one of the first training image on which the first image processing unit 283b performs the first image processing and the second training image on which the second image processing unit 283c performs the second image processing. Then, the display controller 283d outputs the video signal to the display device 26 through the second transmission cable 27.

[0094] The control unit 284 includes, for example, a CPU, an FPGA, etc., and controls the operation of the light source device 22, the camera head 24 and the display device 26 by outputting control signals via the first to third transmission cables 25, 27 and 29, and controls the operation of the control device 28 as a whole.

[0095] Note that a part of the functions of the control unit 284 will be described in “3. Operation of training image generating device” described later.

[0096] The input unit 285 includes an operation device such as a mouse, a keyboard, and a touch panel, and receives a user operation of a user such as a doctor. Then, the input unit 285 outputs an operation signal corresponding to the user operation to the control unit 284.

[0097] The output unit 286 includes a speaker, a printer, and the like, and outputs various information.

[0098] The storage unit 287 stores programs executed by the control unit 284 , information necessary for processing by the control unit 284 , and the like.

[0099] The third communication unit 288 transmits / receives information to / from the learning device 3 via the network NE under the control of the control unit 284 .

[0100] [3. Operation of Training Image Generating Device]

[0101] Next, the operation of the above-mentioned training image generating device 2 will be described.

[0102] Figure 4 : is a flowchart showing the operation of the training image generating device 2 . Figure 5 and Figure 6 is a diagram for describing the operation of the training image generating device 2. Specifically, Figure 5 is a diagram showing a first training image WLI of one frame after performing a first image process. Figure 6 is a diagram showing a second training image IR of one frame after performing the second image processing. Note that Figure 6 The second training image IR shown is represented in grayscale, and as it approaches black, the intensity (corresponding to the brightness value) of the component of the captured fluorescence is higher (the brightness value is larger).

[0103] First, the control unit 284 performs time-division driving of the first light source 221 and the second light source 222 (step S1A). Specifically, in step S1A, based on the synchronization signal, the control unit 284 causes the first light source 221 to emit ordinary light (white light) in the first time period and the second light source 222 to emit near-infrared excitation light in the second time period in the alternatingly repeated first time period and second time period.

[0104] After step S1A, the control unit 284 synchronizes the light emission timing of the first light source 221 and the second light source 222 based on the synchronization signal, so that the imaging element 242b captures the first object image and the second object image in the first time period and the second time period, respectively (steps S1B to S1D). That is, in the case of the first time period (step S1B: Yes), in other words, in the case of emitting ordinary light (white light) into the interior of the organism, the imaging element 242b captures the first object image (ordinary light) and generates a first training image (step S1C). On the other hand, in the case of the second time period (step S1B: No), in other words, in the case of emitting near-infrared excitation light into the interior of the organism, the imaging element 242b captures the second object image (fluorescence) and generates a second training image (step S1D).

[0105] After steps S1C and S1D, the memory controller 283 a controls writing and reading of the training image into and from the memory 282 based on the synchronization signal (step S1E).

[0106] After step S1E, the first image processing unit 283b and the second image processing unit 283 c The following processing is performed (step S1F).

[0107] That is, the first image processing unit 283b sequentially performs the first image processing on the first training images sequentially read from the memory 282 by the memory controller 283a. Then, for example, the first image processing unit 283b outputs Figure 5 On the other hand, the second image processing unit 283 c The second image processing is sequentially performed on the second training images sequentially read from the memory 282 by the memory controller 283a. Then, for example, the second image processing unit 283 c Output Figure 6 Note that since the first training image and the second training image are images captured by the same imaging element 242b, Figure 5 The first training image WLI shown is Figure 6Comparison of the second training image IR shown in FIG. 1 shows that the first training image and the second training image have the same image size, that is, the same pixel position in the first training image and the second training image is a pixel obtained by imaging the same position of the same object.

[0108] After step S1F, the control unit 284 controls the operation of the third communication unit 288 and transmits the images received from the first image processing unit 283b and the second image processing unit 283 in step S1F to the third image processing unit 288. c The output training images of the first training image and the second training image pair are sequentially sent to the learning device 3 (step S1G).

[0109] Then, the control unit 284 returns to step S1A.

[0110] [4. Configuration of learning device]

[0111] Next, the configuration of the learning device 3 will be described.

[0112] Figure 7 is a diagram showing the configuration of the learning device 3 .

[0113] The learning device 3 is, for example, a server device, and is a part that generates a learning model by using the training image generated by the training image generating device 2. Figure 7 As shown, the learning device 3 includes a communication unit 31 , a control unit 32 and a storage unit 33 .

[0114] The communication unit 31 transmits / receives information to / from the training image generating device 2 (third communication unit 288 ) via the network NE under the control of the control unit 32 .

[0115] The control unit 32 includes, for example, a CPU, an FPGA, etc., and controls the overall operation of the learning device 3. The control unit 32 includes a training image acquisition unit 321, a singular region specifying unit 322, a first feature data extraction unit 323, and a singular corresponding region learning unit 324.

[0116] Note that the functions of the training image acquisition unit 321, the singular region specifying unit 322, the first feature data extraction unit 323, and the singular corresponding region learning unit 324 will be described in “5. Operation of the learning device”.

[0117] The storage unit 33 stores programs executed by the control unit 32 , information necessary for processing by the control unit 32 , information generated by the processing, and the like.

[0118] [5. Learning device operation]

[0119] Next, the operation of the above-mentioned learning device 3 will be described.

[0120] Figure 8 is a flowchart showing the operation of the learning device 3 .

[0121] First, the training image acquisition unit 321 sequentially acquires the training images (the first training image and the second training image) transmitted from the training image generation device 2 (the third communication unit 288 ) via the communication unit 31 (step S2A).

[0122] After step S2A, the singular region specifying unit 322 specifies the fluorescence region (singular region) in the second training image (step S2B).

[0123] Specifically, the singular region specifying unit 322 specifies a region in the second training image whose pixel level is equal to or higher than a certain threshold as a fluorescent region.

[0124] Here, examples of pixel levels include brightness values ​​corresponding to Y signals (brightness signals) and RGB values ​​(pixel values). In this embodiment, brightness values ​​are used as pixel levels. Figure 6 In the second training image IR shown in FIG. 1 , an area Ar whose brightness value is equal to or greater than a specific threshold is designated as a fluorescent area. In addition, in this embodiment, the singular area designation unit 322 designates a first fluorescent area (first singular area) Ar1 ( Figure 6 The gray portion shown in FIG. 1 and the second fluorescent region (second singular region) Ar2 ( Figure 6 Each of the black portions shown in FIG. 4 is designated as a fluorescent area Ar.

[0125] After step S2B, the first feature data extraction unit 323 extracts feature data of each of the fluorescence corresponding region (singular corresponding region) and the non-corresponding region in the first training image paired with the second training image specifying the fluorescence region in step S2B (step S2C).

[0126] Here, in the first training image, the fluorescence corresponding area is an area at a pixel position corresponding to the fluorescence area of ​​the second training image (the same pixel position as the fluorescence area). In addition, the non-corresponding area is an area other than the fluorescence corresponding area in the first training image. In this embodiment, the fluorescence corresponding area includes a first fluorescence corresponding area at a pixel position corresponding to the first fluorescence area Ar1 of the second training image and a second fluorescence corresponding area at a pixel position corresponding to the second fluorescence area Ar2 of the second training image in the first training image. That is, the first feature data extraction unit 323 extracts feature data of each of the first fluorescence corresponding area, the second fluorescence corresponding area, and the non-corresponding area.

[0127] In addition, as feature data of the first fluorescence corresponding region, the following extraction methods (1) to (3) can be exemplified.

[0128] (1) Feature data is extracted for each pixel included in the first fluorescence corresponding region.

[0129] (2) A plurality of pixels included in the first fluorescence corresponding region are grouped together, and feature data is extracted for each group.

[0130] (3) Extracting feature data of the entire region corresponding to the first fluorescence.

[0131] Note that the method for extracting the feature data of the second fluorescence corresponding area and the feature data of the non-corresponding area is also similar to the above.

[0132] Furthermore, examples of feature data include feature data related to resolution, edge, color, brightness, noise, contrast, histogram, and the like.

[0133] After step S2C, the singular corresponding region learning unit 324 generates a learning model by performing machine learning on the first fluorescence corresponding region and the second fluorescence corresponding region based on the feature data of the first fluorescence corresponding region, the feature data of the second fluorescence corresponding region, and the feature data of the non-corresponding region (step S2D). That is, by using the learning model, it is possible to determine whether the region having the feature data is any of the first fluorescence corresponding region, the second fluorescence corresponding region, and the non-corresponding region based on the feature data.

[0134] Here, examples of machine learning include machine learning (deep learning) using convolutional neural networks. That is, in machine learning, as the number of training images paired with the first training image and the second training image increases, a learning model that can more accurately determine the first fluorescence corresponding area, the second fluorescence corresponding area, and the non-corresponding area can be generated.

[0135] In this embodiment, when generating a learning model by performing machine learning on the first fluorescence corresponding region and the second fluorescence corresponding region, the singular corresponding region learning unit 324 sets the weight of the feature data related to the blue component to be lower than the weight of the feature data related to the other red and green components. For example, the feature data related to the blue component is not used for machine learning.

[0136] Then, the singular corresponding region learning unit 324 stores the generated learning model in the storage unit 33 .

[0137] [6. Configuration of medical observation equipment]

[0138] Next, the medical observation device 4 that estimates the fluorescence area by using the learning model generated by the learning device 3 will be described.

[0139] Fig. 9 is a diagram showing the configuration of the medical observation apparatus 4 . Fig.10 is a diagram showing the configuration of the camera head 44 and the control device 48 .

[0140] like Fig. 9 or Fig.10 As shown, the medical observation apparatus 4 has a substantially similar structure to the training image generating apparatus 2. Note that, in the medical observation apparatus 4, the same configurations as those of the training image generating apparatus 2 are assigned the same reference numerals.

[0141] Hereinafter, among the configurations of the medical observation apparatus 4 , the configurations different from the training image generation apparatus 2 will be mainly described.

[0142] In the medical observation device 4, the light source device 42 has a configuration corresponding to the light source device 22 in the training image generation device 2. Fig. 9 or Fig.10 As shown, the light source device 42 only includes the first light source 221. That is, unlike the light source device 22, the light source device 42 does not include the second light source 222.

[0143] Then, in the light source device 42 , under the control of the control device 48 , the first light source 221 is driven and only normal light (white light) is emitted.

[0144] Note that although the light source device 42 is configured separately from the control device 48 in the present embodiment, this is not a limitation, and a configuration in which it is provided inside the control device 48 may be employed.

[0145] In the medical observation device 4, the camera head 44 has a configuration corresponding to the camera head 24 in the training image generation device 2. Fig.10 As shown, since the light source device 42 does not include the second light source 222, the camera head 44 has the feature of omitting the excitation light cutoff filter 242 from the camera head 24. a Configuration.

[0146] Then, the camera head 44 (imaging element 242b) performs imaging in a specific frame period under the control of the control device 48. Hereinafter, in order to distinguish from the first training image, the image generated by capturing the first object image (ordinary light) by the camera head 44 (imaging element 242b) will be referred to as a captured image.

[0147] The control device 48 corresponds to the medical image processing device according to the present disclosure. In the medical observation device 4, the control device 48 has a configuration corresponding to the control device 28 in the training image generation device 2. In the control device 48, as shown in FIG. Fig.10As shown, an image generating unit 483 and a control unit 484 are used instead of the image generating unit 283 and the control unit 284 with respect to the control device 28 .

[0148] The image generation unit 483 processes the captured images (RAW signals (digital signals)) sequentially output from the camera head 44 (first communication unit 243) and received by the second communication unit 281 under the control of the control unit 484. Fig.10 As shown, the image generating unit 483 includes a memory controller 483a, an image processing unit 483b, and a second feature data extracting unit 483 c , a singular corresponding region specifying unit 483d and a display controller 483 e .

[0149] Note that the memory controller 483a, the image processing unit 483b, the second feature data extraction unit 483 will be described in “7. Operation of the medical observation apparatus” described later. c , singular corresponding area specifying unit 483d, display controller 483 e function.

[0150] The control unit 484 includes, for example, a CPU, an FPGA, etc., and controls the operation of the light source device 42, the camera head 44 and the display device 26 by outputting control signals via the first to third transmission cables 25, 27 and 29, and controls the operation of the control device 48 as a whole.

[0151] Note that a part of the functions of the control unit 484 will be described in “7. Operation of medical observation apparatus” described later.

[0152] [7. Operation of medical observation device]

[0153] Next, the operation of the above-mentioned medical observation apparatus 4 will be described.

[0154] Fig.11 : is a flowchart showing the operation of the medical observation device 4 . Fig.12 4 is a diagram for describing the operation of the medical observation device 4. Specifically, Fig.12 is a diagram showing a display image WLI' generated in step S3G. Fig.12 For the convenience of description, it is assumed that in step S3B Figure 5 The first training image WLI shown is the same captured image.

[0155] Note that it is assumed that the control unit 484 controls the operation of the third communication unit 288 , receives the learning model from the learning device 3 , and stores the learning model in the storage unit 287 before performing the operation of the medical observation device 4 described below.

[0156] First, the control unit 484 drives the light source device 42 (first light source 221) (step S3A). Thus, ordinary light (white light) is emitted into the interior of the biological body (observation target).

[0157] After step S3A, the control unit 484 generates a captured image by causing the imaging element 242 b to capture a first object image (normal light) in a specific frame period (step S3B).

[0158] After step S3B, the memory controller 483a controls writing and reading of the captured images into and from the memory 282 (step S3C). Specifically, the memory controller 483a sequentially writes the captured images sequentially output from the camera head 44 (first communication unit 243) and received by the second communication unit 281 into the memory 282. In addition, the memory controller 483a reads the captured images from the memory 282 at a specific timing, and causes the image processing unit 483b to input the read captured images.

[0159] After step S3C, the image processing unit 483b sequentially performs the above-described first image processing on each captured image sequentially read from the memory 282 by the memory controller 483a (step S3D).

[0160] After step S3D, with respect to the captured images sequentially output from the image processing unit 483b in step S3D, the second feature data extraction unit 483c extracts feature data of each region of the captured images (step S3E).

[0161] Here, as feature data, the following extraction methods (4) and (5) can be exemplified.

[0162] (4) Feature data is extracted for each pixel included in the captured image.

[0163] (5) A plurality of pixels included in a captured image are regarded as a group (region), and feature data is extracted for each group.

[0164] In addition, the feature data extracted by the second feature data extraction unit 483 c is the same type of feature data as the feature data extracted by the first feature data extraction unit 323 .

[0165] After step S3E, based on the feature data extracted in step S3E, the singular corresponding region specifying unit 483d specifies the first fluorescence corresponding region and the second fluorescence corresponding region in the captured image by using the learning model stored in the storage unit 287 (step S3F).

[0166] After step S3F, the display controller 483 eA display image is generated (step S3G) in which the first fluorescence corresponding area and the second fluorescence corresponding area specified in step S3F are displayed in a manner distinguished from other areas in the captured image. Figure 5 The first training image WLI is shown in the same captured image as Fig.12 As shown, a display image WLI' is generated in the captured image to distinguish the first fluorescence corresponding area Ar1' and the second fluorescence corresponding area Ar2' (fluorescence corresponding area Ar') from other areas. As a method of distinguishing, for example, a single color (such as green) can be applied to the fluorescence corresponding area Ar' in the captured image. In addition, as described above, the second fluorescence area Ar2 is an area with a brightness value greater than that of the first fluorescence area Ar1. Therefore, preferably, a color darker than the color of the first fluorescence corresponding area Ar1' (for ease of description, in Fig.12 ) is applied to the second fluorescence corresponding area Ar2' (for ease of description, Fig.12 in black).

[0167] Then, the display controller 483e generates a video signal corresponding to the display image WLI', and outputs it to the display device 26 through the second transmission cable 27. Therefore, the display device 26 displays the display image WLI'.

[0168] According to the present embodiment described above, the following effects are achieved.

[0169] The learning device 3 according to the present embodiment uses a training image that is a pair of a first training image and a second training image, and based on feature data of a fluorescence corresponding area at a pixel position in the first training image corresponding to a fluorescence area in the second training image, performs machine learning on the fluorescence corresponding area to generate a learning model.

[0170] Then, the control device 48 according to this embodiment obtains a captured image obtained by capturing a first object image (ordinary light) from inside a biological body (observation target), and based on the feature data of each area in the captured image, specifies the fluorescence corresponding area in the captured image using the above-mentioned learning model.

[0171] That is, since the fluorescent area can be estimated using the learning model, it is not necessary to administer a fluorescent substance such as indocyanine green to the living body. Therefore, convenience can be improved.

[0172] In addition, in the medical observation device 4, since it is not necessary to emit near-infrared excitation light into the living body, the second light source 222 and the excitation light cut filter 242a can be omitted. Therefore, the configuration can be simplified and miniaturized.

[0173] In addition, in the learning device 3 according to the present embodiment, the fluorescence region is divided into two stages of the first fluorescence region and the second fluorescence region according to the magnitude of the brightness value. The same applies to the fluorescence corresponding region corresponding to the fluorescence region. That is, the fluorescence corresponding region specified by the control device 48 is also divided into the first fluorescence corresponding region and the second fluorescence corresponding region. Therefore, the user such as a doctor can easily recognize the portion where the intensity of the estimated fluorescence component is high and the portion where the intensity of the estimated fluorescence component is low from the displayed image.

[0174] Incidentally, when specifying a lesion site such as cancer by image recognition, it is not necessary to consider feature data of the blue component.

[0175] In the learning device 3 according to the present embodiment, when generating a learning model by performing machine learning on the fluorescence corresponding area, the weight of the feature data related to the blue component is made lower than the weight of the feature data related to other color components. For example, the feature data related to the blue component is not used for machine learning. Therefore, since machine learning can be performed without considering unnecessary feature data, the processing load can be reduced.

[0176] (Other embodiments)

[0177] Although the modes for carrying out the present disclosure have been described above, the present disclosure is not limited to the above-mentioned embodiments.

[0178] Although the fluorescent area is divided into two stages, the first fluorescent area and the second fluorescent area, according to the magnitude of the brightness value in the above embodiment, this is not a limitation. There may be only one fluorescent area, or the fluorescent area may be divided into three stages or more according to the magnitude of the brightness value. The same applies to the fluorescent corresponding area corresponding to the fluorescent area.

[0179] In the above embodiment, the light of the first wavelength band is ordinary light (white light), and the light of the second wavelength band is near-infrared excitation light. However, this is not a limitation. As long as the first wavelength band and the second wavelength band are different, other lights can be used. At this time, the first wavelength band and the second wavelength band can be partially overlapping frequency bands, or can be completely non-overlapping frequency bands.

[0180] For example, narrow band light used in so-called narrow band imaging (NBI) may be used as the light of the second wavelength band. In this case, the light of the first wavelength band may be ordinary light (white light) or other light.

[0181] Incidentally, photodynamic diagnosis (PDD) has been conventionally known as one of cancer diagnosis methods for detecting cancer cells.

[0182] In photodynamic diagnosis, for example, a photosensitive substance such as 5-aminolevulinic acid (hereinafter, referred to as 5-ALA) is used. 5-ALA is a natural amino acid originally included in the organisms of animals and plants. This 5-ALA enters cells after in vivo administration and is biosynthesized into protoporphyrin in mitochondria. Protoporphyrin is excessively accumulated in cancer cells. In addition, protoporphyrin excessively accumulated in cancer cells is photoactive. Therefore, when excited by excitation light (such as blue visible light in the 375nm to 445nm band), protoporphyrin emits fluorescence (such as red fluorescence in the 600nm to 740nm band). As described above, the cancer diagnosis method of making cancer cells fluoresce by using a photosensitive substance is called photodynamic diagnosis.

[0183] Then, in the above embodiment, the excitation light for exciting protoporphyrin (such as blue visible light in the wavelength band of 375nm to 445nm) can be used as the light of the second wavelength band. At this time, the light of the first wavelength band can be ordinary light (white light) or other light.

[0184] Although in the above-described embodiment, the first training image and the second training image are generated by a single imaging element 242b, this is not a limitation. For example, a configuration may be adopted in which the first object image and the second object image are separated and captured by two imaging elements, respectively, and the first training image and the second training image are generated by two imaging elements, respectively. In this case, the learning device 3 needs to identify the correspondence relationship between the pixels of the first training image and the second training image.

[0185] Although in the above-described embodiment, the training image generating device 2 and the learning device 3 are communicably connected to each other via the network NE, this is not a limitation. The training image generating device 2 and the learning device 3 may also configure one device.

[0186] Although in the above-described embodiment, the medical image processing device according to the present disclosure is mounted on the medical observation device 4 whose insertion unit 21 includes a rigid endoscope, this is not a limitation. For example, the medical image processing device according to the present disclosure may be mounted on a medical observation device whose insertion unit 21 has a soft endoscope. In addition, the medical image processing device according to the present disclosure may be mounted on a medical observation device such as a surgical microscope (for example, see Japanese Patent Application Publication No. 2016-42981) that magnifies and observes a predetermined field of view area of ​​the inside of an object (inside a living body) or the surface of an object (surface of a living body).

[0187] In the above-described embodiment, a part of the configuration of the camera head 44 or a part of the configuration of the control device 48 may be provided in the connector CN1 or the connector CN2, for example.

[0188] Note that the following configurations also belong to the technical scope of the present disclosure.

[0189] (1) A learning device comprising:

[0190] a training image acquisition unit configured to acquire a training image that is a pair of a first training image and a second training image, wherein the first training image is acquired by capturing light from an object irradiated with light of a first wavelength band, and the second training image is acquired by capturing light from an object irradiated with light of a second wavelength band different from the first wavelength band;

[0191] a singular region specifying unit configured to specify a singular region in the second training image;

[0192] A first feature data extraction unit is configured to extract feature data of a singular corresponding area at a pixel position corresponding to the singular area in the first training image; and

[0193] The singular corresponding region learning unit is configured to generate a learning model by performing machine learning on the singular corresponding regions based on the feature data.

[0194] (2) A learning device according to (1), wherein:

[0195] The object emits fluorescence by being irradiated with excitation light of a second wavelength band;

[0196] acquiring a second training image by capturing fluorescence from an object illuminated with excitation light; and

[0197] The singular region specifying unit is configured to specify, as the singular region, a region in which the intensity of the component of the fluorescence in the second training image is equal to or higher than a certain threshold.

[0198] (3) The learning device according to (1) or (2), wherein the singular region specifying unit is configured to specify a region in the second training image where the pixel level is equal to or higher than a specific threshold as the singular region.

[0199] (4) A learning device according to (3), wherein:

[0200] The singular region designation unit is configured to designate each of a first singular region with a pixel level within a first range and a second singular region with a pixel level within a second range higher than the first range as a singular region;

[0201] The first feature data extraction unit is configured to extract feature data of each of a first singular corresponding region, which is a singular corresponding region in the first training image and is located at a pixel position corresponding to the first singular region, and a second singular corresponding region, which is a singular corresponding region in the first training image and is located at a pixel position corresponding to the second singular region; and

[0202] The singular corresponding region learning unit is configured to perform machine learning on each of the first singular corresponding region and the second singular corresponding region based on the feature data.

[0203] (5) A learning device according to any one of (1) to (4), wherein:

[0204] The characteristic data includes characteristic data related to at least one of color and brightness; and

[0205] The singular corresponding region learning unit is configured to make the weight of feature data related to the blue component lower than the weight of feature data related to another color component when performing machine learning on the singular corresponding region and generating a learning model.

[0206] (6) The learning device according to any one of (1) to (5), wherein:

[0207] emitting light of a first wavelength band in a first period of time in the alternately repeated first and second periods of time; and

[0208] Light of a second wavelength band is emitted in a second period of time.

[0209] (7) A medical image processing device comprising:

[0210] a captured image acquisition unit configured to acquire a captured image acquired by capturing light from an observation target irradiated with light of a first wavelength band;

[0211] a second feature data extraction unit configured to extract feature data of each area of ​​the captured image; and

[0212] A singular corresponding region specifying unit is configured to specify a singular corresponding region in a captured image based on feature data by using a learning model constructed by machine learning; wherein,

[0213] A learning model is generated by performing machine learning on singular corresponding areas based on feature data of singular corresponding areas at pixel positions in the first training image corresponding to singular areas in the second training image using a training image that is a pair of a first training image and a second training image, wherein the first training image is acquired by capturing light from an object illuminated with light of a first wavelength band, and the second training image is acquired by capturing light from an object illuminated with light of a second wavelength band different from the first wavelength band.

[0214] (8) A medical image processing device according to (7), wherein:

[0215] The object emits fluorescence by being irradiated with excitation light of a second wavelength band;

[0216] acquiring a second training image by capturing fluorescence from an object illuminated with excitation light; and

[0217] The singular region is a region in which the intensity of the fluorescent component in the second training image is equal to or higher than a certain threshold.

[0218] (9) A medical image processing apparatus according to (7) or (8), wherein the singular region is a region in the second training image where a pixel level is equal to or greater than a specific threshold.

[0219] (10) A medical image processing device according to (9), wherein:

[0220] The singular region includes: a first singular region in which the pixel level is within a first range; and

[0221] a second singular region having a pixel level within a second range higher than the first range; and

[0222] Based on feature data of a first singular corresponding region and a second singular corresponding region, a learning model is generated by performing machine learning on each of the first singular corresponding region and the second singular corresponding region, wherein the first singular corresponding region is a singular corresponding region in the first training image and is located at a pixel position corresponding to the first singular region, and the second singular corresponding region is a singular corresponding region in the first training image and is located at a pixel position corresponding to the second singular region.

[0223] (11) A medical image processing device according to any one of (7) to (10), wherein:

[0224] The characteristic data includes characteristic data related to at least one of color and brightness; and

[0225] A learning model is generated by performing machine learning on the singular corresponding region in a state where the weight of feature data related to the blue component is made lower than the weight of feature data related to another color component.

[0226] (12) A medical image processing device according to any one of (7) to (11), wherein:

[0227] emitting light of a first wavelength band in a first period of time in the alternately repeated first and second periods of time; and

[0228] Light of a second wavelength band is emitted in a second period of time.

[0229] (13) The medical image processing device according to any one of (7) to (12), further includes: a display controller configured to generate a display image in which the singular corresponding area is displayed in a manner distinguishable from another area in the captured image.

[0230] Reference Mark List

[0231] 1 Learning system

[0232] 2 Training Image Generation Device

[0233] 3 Learning Devices

[0234] 4 Medical observation device

[0235] 21 Insertion unit

[0236] 22, 42 Light source device

[0237] 23 Light guide

[0238] 24, 44 camera head

[0239] 25 First Transmission Cable

[0240] 26 Display device

[0241] 27 Second transmission cable

[0242] 28, 48 Control device

[0243] 29 Third transmission cable

[0244] 31 Communication unit

[0245] 32 Control Unit

[0246] 33 Storage Units

[0247] 211 Eyepiece

[0248] 221 First Light Source

[0249] 222 Second Light Source

[0250] 241 lens unit

[0251] 242 Imaging Unit

[0252] 242a Excitation light cutoff filter

[0253] 242b Imaging element

[0254] 242c Signal Processing Unit

[0255] 242d Color Filter

[0256] 243 First communication unit

[0257] 281 Second communication unit

[0258] 282 Memory

[0259] 283 Image Generation Unit

[0260] 283a Memory Controller

[0261] 283b First image processing unit

[0262] 283c Second image processing unit

[0263] 283d Display Controller

[0264] 284, 484 control unit

[0265] 285 Input Unit

[0266] 286 Output Unit

[0267] 287 storage units

[0268] 288 Third Communication Unit

[0269] 321 Training image acquisition unit

[0270] 322 Singular Area Designation Unit

[0271] 323 First feature data extraction unit

[0272] 324 Singular Correspondence Region Learning Unit

[0273] 483 Image Generation Unit

[0274] 483a Memory Controller

[0275] 483b Image Processing Unit

[0276] 483c Second feature data extraction unit

[0277] 483d Singular Correspondence Area Designation Unit

[0278] 483e Display Controller

[0279] Ar fluorescence area

[0280] Ar' fluorescence corresponding area

[0281] Ar1 The first fluorescent area

[0282] Ar1' first fluorescence corresponding area

[0283] Ar2 Second fluorescence area

[0284] Ar2' second fluorescence corresponding area

[0285] CN1, CN2 Connectors

[0286] IR second training image

[0287] NE Network

[0288] WLI first training image

[0289] WLI' displays the image.

Claims

1. A learning device, comprising: a training image acquisition unit configured to acquire a training image that is a pair of a first training image and a second training image, wherein the first training image is acquired by capturing light from an object illuminated with light of a first wavelength band, and the second training image is acquired by capturing light from the object illuminated with light of a second wavelength band different from the first wavelength band; a singular region specifying unit, configured to specify a singular region in the second training image; A first feature data extraction unit is configured to extract feature data of a singular corresponding area at a pixel position corresponding to the singular area in the first training image; and a singular corresponding region learning unit configured to generate a learning model by performing machine learning on the singular corresponding region based on the feature data, The singular region designating unit is configured to designate a region in the second training image whose pixel level is equal to or higher than a specific threshold as the singular region, and wherein the singular region designation unit is configured to designate each of a first singular region in which the pixel level is within a first range and a second singular region in which the pixel level is within a second range higher than the first range as the singular region; The first feature data extraction unit is configured to extract feature data of each of a first singular corresponding region that is a singular corresponding region in the first training image and is located at a pixel position corresponding to the first singular region, and a second singular corresponding region that is a singular corresponding region in the first training image and is located at a pixel position corresponding to the second singular region; and The singular corresponding region learning unit is configured to perform machine learning on each of the first singular corresponding region and the second singular corresponding region based on the feature data.

2. The learning device according to claim 1, wherein: The object emits fluorescence by being irradiated with the excitation light of the second wavelength band; acquiring the second training image by capturing the fluorescence from the object illuminated with the excitation light; and The singular region specifying unit is configured to specify, as the singular region, a region in the second training image where the intensity of the component of the fluorescence is equal to or higher than a specific threshold.

3. The learning device according to claim 1, wherein: The characteristic data includes characteristic data related to at least one of color and brightness; and The singular corresponding region learning unit is configured to make the weight of the feature data related to the blue component lower than the weight of the feature data related to another color component when performing machine learning on the singular corresponding region and generating a learning model.

4. The learning device according to claim 1, wherein: emitting light of the first wavelength band in the first period of time in the alternatingly repeated first period of time and second period of time; and The light of the second wavelength band is emitted in the second period.

5. A medical image processing device, comprising: a captured image acquisition unit configured to acquire a captured image acquired by capturing light from an observation target irradiated with light of a first wavelength band; a second feature data extraction unit configured to extract feature data of each area of ​​the captured image; as well as a singular corresponding region specifying unit configured to specify a singular corresponding region in the captured image based on the feature data by using a learning model constructed by machine learning; wherein, The learning model is generated by performing machine learning on singular corresponding regions based on feature data of the singular corresponding regions at pixel positions in the first training image corresponding to singular regions in the second training image using training images that are paired with a first training image and a second training image, wherein the first training image is acquired by capturing light from an object illuminated with light of a first wavelength band, and the second training image is acquired by capturing light from the object illuminated with light of a second wavelength band different from the first wavelength band, The singular region is a region in the second training image where the pixel level is equal to or greater than a specific threshold, and The singular region includes: a first singular region in which the pixel level is within a first range; and a second singular region where the pixel level is within a second range higher than the first range; and The learning model is generated by performing machine learning on each of the first singular corresponding region and the second singular corresponding region based on feature data of the first singular corresponding region, wherein the first singular corresponding region is a singular corresponding region in the first training image and is located at a pixel position corresponding to the first singular region, and the second singular corresponding region is a singular corresponding region in the first training image and is located at a pixel position corresponding to the second singular region.

6. The medical image processing apparatus according to claim 5, wherein: The object emits fluorescence by being irradiated with the excitation light of the second wavelength band; acquiring the second training image by capturing the fluorescence from the object illuminated with the excitation light; and The singular region is a region in the second training image where the intensity of the fluorescence component is equal to or higher than a specific threshold.

7. The medical image processing apparatus according to claim 5, wherein: The characteristic data includes characteristic data related to at least one of color and brightness; and The learning model is generated by performing machine learning on the singular corresponding region in a state where the weight of the feature data related to the blue component is made lower than the weight of the feature data related to another color component.

8. The medical image processing apparatus according to claim 5, wherein: emitting light of the first wavelength band in the first period of time in the alternatingly repeated first period of time and second period of time; and The light of the second wavelength band is emitted in the second period.

9. The medical image processing apparatus according to claim 5, further comprising: A display controller is configured to generate a display image in which the singular corresponding region is displayed in a manner distinguishable from another region in the captured image.

Images