Processor device, medical image processing device, medical image processing system, and endoscope system

Abstract

Provided are a processor device (14), a medical image processing device (17), a medical image processing system (18), and an endoscopic system (10) with which it is possible to easily discriminate the type of an endoscopic image.　The medial image processing system (10) is provided with the processor device (14) and the medical image processing device (17). The processor device (14) generates an identification-information-attached medical image generated by having a part of data constituting a medical image be identification information indicating the type of the medical image. The medical image processing device acquires the identification-information-attached medical image, identifies the type of the medical image, and performs image processing corresponding to the type of the medical image. The endoscopic system (10) is provided with a light source, an endoscope, and the medical image processing system (18).

Description

[Technical Field]

[0001] The present invention relates to a processor device, a medical image processing device, a medical image processing system, and an endoscope system. [Background technology]

[0002] In the medical field, diagnosis using an endoscopic system including a light source device, an endoscope, and a processor device is widely performed. In diagnosis using an endoscopic system, various diagnostic support information regarding the surface structure or mucosal surface of the object to be observed may be obtained using an image (hereinafter referred to as an endoscopic image) obtained by photographing the object with an endoscope using image enhanced endoscopy (IEE) that utilizes a specially designed illumination light.

[0003] In diagnosis using IEE, appropriate diagnosis may be possible by acquiring multiple types of endoscopic images obtained using multiple types of illumination light, etc., and comparing or superimposing these endoscopic images in detail. For example, an endoscopic system is known that acquires a normal image signal using white light illumination and a special image signal using special light illumination with a spectrum different from that of white light, thereby preventing overlooking of lesions, etc., and enabling highly accurate determination of the severity or progression of a disease during endoscopic examination (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-065685 Summary of the Invention [Problem to be solved by the invention]

[0005] Appropriate diagnostic support information can be obtained by performing different image processing on endoscopic images based on normal image signals using white light and endoscopic images based on special image signals using special light. In other words, when the type of endoscopic image is set according to the type of illumination light used to acquire the endoscopic image, it is preferable to perform image processing on the endoscopic image selected according to the type of endoscopic image.

[0006] The type of illumination light is determined based on a signal transmitted from the processor device to the light source device, and the processor device stores information about both the endoscopic image and its type. Therefore, when transmitting the endoscopic image from the processor device to an external device and processing the image there, the type of endoscopic image must also be transmitted from the processor device to the external device at the same time. To ensure the correspondence between the endoscopic image and the type of endoscopic image is not lost, if the type of endoscopic image is recorded in the header or other portion of the information container storing the endoscopic image and transmitted, it cannot be transmitted as a general video signal such as DVI (Digital Visual Interface) and is often not receivable by a general-purpose personal computer (hereinafter referred to as a PC). Furthermore, transmitting the image over a separate signal line without losing the correspondence is often difficult for a PC.

[0007] On the other hand, since image processing using PCs is widely used in various ways, there has been a demand for a simpler way to classify and handle endoscopic images using a PC.

[0008] An object of the present invention is to provide a processor device, a medical image processing device, a medical image processing system, and an endoscope system that can easily distinguish the type of an endoscopic image. [Means for solving the problem]

[0009] The present invention is a processor device comprising a first processor that acquires multiple types of medical images with different shooting conditions, and changes part of the data that constitutes the medical image depending on the type of medical image, or changes the data for at least one type of medical image and does not change it for another type of medical image, thereby generating a medical image with identification information in which part of the data that constitutes the medical image is identification information that indicates the type of medical image.

[0010] The data constituting the medical image is preferably data constituting a predefined region of the medical image.

[0011] The data constituting the medical image is preferably pixel values.

[0012] The plurality of types of medical images preferably include display images for displaying on a display and analysis images for performing analysis relating to diagnostic information.

[0013] It is preferable that the first processor uses the identification information for the analysis image by changing part of the data constituting the analysis image, and uses the identification information for the display image without changing the data constituting the display image in the part that corresponds to the data used as identification information in the analysis image.

[0014] It is preferable that the first processor uses the identification information for the display image by changing part of the data constituting the display image, and uses the identification information for the analysis image without changing the data constituting the analysis image in the part corresponding to the data used as identification information in the display image.

[0015] The photographing condition is preferably the spectrum of the illumination light.

[0016] The present invention also provides a medical image processing device that includes a second processor that acquires multiple types of medical images with identification information, in which part of the data constituting the medical image is used as identification information, recognizes the type of medical image with identification information based on the identification information, and controls the display of the medical image with identification information on a display based on the type of medical image with identification information.

[0017] The plurality of types of medical images with identification information preferably include a display image for displaying on a display and an analysis image for performing an analysis related to diagnostic information.

[0018] It is preferable that the second processor displays the display image on the main screen of the display, and determines whether or not to display the analysis image on a sub-screen of the display based on the type of medical image with identification information, and displays the medical image with identification information that it has decided to display on the sub-screen of the display.

[0019] It is preferable that the second processor performs image processing set for each type of identification-information-added medical image on the identification-information-added medical image based on the type of identification-information-added medical image.

[0020] It is preferable that the second processor performs image processing for display on the image for display when the information-attached medical image is an image for display, and performs image processing for analysis on the image for analysis when the identification information-attached medical image is an image for analysis.

[0021] The second processor preferably performs the analytical image processing using an analytical model based on machine learning.

[0022] The second processor preferably creates an analysis result image showing the result of the analytical image processing, and generates a superimposed image by superimposing the analysis result image on the display image.

[0023] Moreover, a medical image processing system of the present invention includes a processor device and a medical image processing device, and a second processor acquires a plurality of types of medical images with identification information generated by the first processor.

[0024] Furthermore, the medical image processing system of the present invention includes a processor device and a medical image processing device, and the processor device acquires an analysis result image that indicates the result of the analytical image processing created by the second processor.

[0025] The processor device preferably superimposes the analysis result image on the display image.

[0026] Preferably, the processor device adjusts the frame rate of the identification-added medical images, and the medical image processing device acquires the identification-added medical images whose frame rate has been adjusted.

[0027] Preferably, the processor device or the medical image processing device adjusts the frame rate of the images for display on the display.

[0028] The present invention also provides an endoscopic system that includes a plurality of light sources that emit light in different wavelength bands, an endoscope that captures an image of a subject illuminated by the illumination light emitted by the plurality of light sources, and a medical image processing system, and the processor device is equipped with a light source processor that controls the emission of each of a plurality of types of illumination light that have different combinations of light intensity ratios from the plurality of light sources. [Effects of the Invention]

[0029] According to the present invention, the type of endoscopic image can be easily determined. [Brief explanation of the drawings]

[0030] [Figure 1] FIG. 1 is an external view of an endoscope system. [Figure 2] FIG. 2 is a block diagram showing the functions of the endoscope system. [Figure 3] FIG. 2 is an explanatory diagram illustrating four color LEDs included in the light source unit. [Figure 4] 1 is a graph showing the spectra of violet light V, blue light B, green light G, and red light R. [Figure 5]10 is a graph showing the spectrum of the first illumination light. [Figure 6] FIG. 2 is an explanatory diagram illustrating the types and order of endoscopic images captured by the endoscopic system. [Figure 7] FIG. 10 is an image diagram showing an endoscopic image with identification information. [Figure 8] FIG. 10 is an image diagram showing an endoscopic image including an observation target portion and a mask portion. [Figure 9] Figure 9(A) is an image diagram showing an endoscopic image with first recognition information having first identification information, and Figure 9(B) is an image diagram showing an endoscopic image with second recognition information having second identification information. [Figure 10] 1 is an explanatory diagram illustrating the types and identification information of endoscopic images captured by an endoscopic system. FIG. [Figure 11] 1 is an explanatory diagram illustrating the types of endoscopic images captured by the endoscopic system, the order of capturing images, and identification information. [Figure 12] FIG. 10 is an explanatory diagram illustrating a case where identification information is added to an analysis image. [Figure 13] FIG. 10 is an explanatory diagram illustrating a case where identification information is added to a display image. [Figure 14] FIG. 2 is a block diagram showing the functions of the medical image processing apparatus. [Figure 15] 1 is an explanatory diagram illustrating various images and a processing flow in a medical image processing apparatus. [Figure 16] FIG. 10 is an image diagram showing an analysis image displayed on a sub-screen of a display. [Figure 17] FIG. 10 is a diagram showing an image when no image is displayed on the sub-screen of the display. [Figure 18] FIG. 10 is a diagram showing an image when a past image is displayed on a sub-screen of a display. [Figure 19] 10 is an explanatory diagram illustrating a function in which a frame rate conversion unit adjusts the frame rate by duplicating an image for display and an image for analysis. FIG. [Figure 20] 10 is an explanatory diagram illustrating a function of a frame rate conversion unit that copies a display image and adjusts the frame rate. FIG. [Figure 21] 10 is an explanatory diagram illustrating an endoscopic image with third identification information generated by adding identification information based on the type of endoscopic image to a complementary frame image. FIG. [Figure 22] 10 is an explanatory diagram illustrating a third identification-information-added endoscopic image generated by adding identification information based on the type of endoscopic image and information on the original image to a complementary frame image. FIG. [Figure 23] 10 is an explanatory diagram illustrating an endoscopic image with third identification information generated by adding identification information based on the type of endoscope and the imaging sequence to a display image, an analysis image, and a complementary frame image. FIG. [Figure 24] 1 is a flowchart showing a series of steps of endoscopic image processing in a medical image processing system and an endoscope system. [Figure 25] FIG. 10 is an explanatory diagram illustrating a case where a medical image processing apparatus is included in a diagnosis support apparatus. [Figure 26] FIG. 10 is an explanatory diagram illustrating a case where a medical image processing apparatus is included in a medical service support apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0031] 1, the endoscopic system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, a keyboard 16, and a medical image processing device 17. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14. The processor device 14 is connected to the medical image processing device 17. The medical image processing device 17 receives endoscopic images with identification information attached from the processor device 14 and performs various image processing including image analysis using machine learning and the like. In this embodiment, the medical images are endoscopic images.

[0032] The endoscope 12 has an insertion section 12a that is inserted into the body of a subject having an observation target, an operation section 12b provided at the base end of the insertion section 12a, and a bending section 12c and a tip section 12d provided at the tip side of the insertion section 12a. The bending section 12c is bent by operating an angle knob 12e (see FIG. 2) of the operation section 12b. The tip section 12d is directed in a desired direction by the bending operation of the bending section 12c.

[0033] The operation unit 12b has an angle knob 12e, a zoom operation unit 12f for changing the imaging magnification, and a mode changeover switch 12g used for switching the observation mode. Note that the switching operation of the observation mode or the zoom operation may be performed or instructed using the keyboard 16, a foot switch (not shown), or the like, in addition to the mode changeover switch 12g or the zoom operation unit 12f.

[0034] The endoscope system 10 has three observation modes: a normal observation mode, a special observation mode, and a diagnostic assistance observation mode. The normal observation mode is a mode in which a normal image, which is an image with natural coloring obtained by capturing an image of an observation target using white light as illumination light, is displayed on the display 15. The special observation modes include a first special observation mode. The first special observation mode is a mode in which a first image in which surface information such as superficial blood vessels is emphasized is displayed on the display 15.

[0035] The diagnostic assistance observation mode is a mode in which a superimposed image, in which an analysis result image showing the results of image analysis is superimposed on a normal image, is displayed on the display 15. The results of the image analysis are diagnostic assistance information for assisting doctors and others in making diagnoses, and are obtained by image analysis using the first image. Therefore, the analysis result image includes diagnostic assistance information related to a lesion or the like obtained by image analysis using the first image. In the diagnostic assistance observation mode, if a lesion or the like is detected by image analysis using the first image, a superimposed image, in which an analysis result image showing diagnostic assistance information such as the location of the lesion is superimposed on the normal image, is displayed on the display 15. The diagnostic assistance observation mode is selected when the endoscope system 10 is started up.

[0036] The processor device 14 is electrically connected to a display 15 and a keyboard 16. The display 15 displays the normal image, the first image, the superimposed image, and / or information accompanying these images. The keyboard 16 functions as a user interface that accepts input operations such as function settings. An external recording unit (not shown) that records images, image information, etc. may be connected to the processor device 14.

[0037] As shown in FIG. 2, the light source device 13 emits illumination light to illuminate an object to be observed and includes a light source unit 20 and a light source processor 21 that controls the light source unit 20. The light source unit 20 is configured, for example, with a semiconductor light source such as a multicolored light-emitting diode (LED), a combination of a laser diode and a phosphor, or a xenon lamp or halogen light source. The light source unit 20 also includes an optical filter or the like for adjusting the wavelength band of the light emitted by the LED or the like. The light source processor 21 controls the amount of illumination light by turning on / off each LED or adjusting the drive current or drive voltage of each LED or the like. The light source processor 21 also controls the wavelength band of the illumination light by changing the optical filter or the like.

[0038] As shown in FIG. 3, in this embodiment, the light source unit 20 has four color LEDs: a V-LED (Violet Light Emitting Diode) 20a, a B-LED (Blue Light Emitting Diode) 20b, a G-LED (Green Light Emitting Diode) 20c, and an R-LED (Red Light Emitting Diode) 20d.

[0039] As shown in FIG. 4, the V-LED 20a emits violet light V with a central wavelength of 410±10 nm and a wavelength range of 380 to 420 nm. The B-LED 20b emits blue light B with a central wavelength of 450±10 nm and a wavelength range of 420 to 500 nm. The G-LED 20c emits green light G with a wavelength range of 480 to 600 nm. The R-LED 20d emits red light R with a central wavelength of 620 to 630 nm and a wavelength range of 600 to 650 nm.

[0040] The light source processor 21 controls the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d. In the normal observation mode, the light source processor 21 controls each of the LEDs 20a to 20d so that the LEDs 20a to 20d emit normal light in which the light intensity ratio between the purple light V, the blue light B, the green light G, and the red light R is Vc:Bc:Gc:Rc.

[0041] When the special observation mode is selected, the light source processor 21 controls the LEDs 20a to 20d to emit first illumination light in which the light intensity ratio of the purple light V, blue light B, green light G, and red light R is Vs1:Bs1:Gs1:Rs1. It is preferable that the first illumination light emphasizes superficial blood vessels. Therefore, it is preferable that the intensity of the purple light V is greater than the intensity of the blue light B. For example, as shown in FIG. 5, the ratio of the intensity Vs1 of the purple light V to the intensity Bs1 of the blue light B is set to 4:1.

[0042] In this specification, a combination of light intensity ratios includes a case where the ratio of at least one semiconductor light source is 0 (zero). Therefore, it also includes a case where one or more of the semiconductor light sources are not lit. For example, when only one of the semiconductor light sources is lit and the other three are not lit, such as when the light intensity ratio combination among purple light V, blue light B, green light G, and red light R is 1:0:0:0, this also has a light intensity ratio and is one of the combinations of light intensity ratios.

[0043] As described above, the combinations of light intensity ratios of violet light V, blue light B, green light G, and red light R, i.e., the types of illumination light, emitted in the normal observation mode and the special observation mode are different from each other. In the diagnosis assistance observation mode, different types of illumination light are automatically switched and emitted. Note that an observation mode using a different type of illumination light having a combination of light intensity ratios different from the illumination light used in these observation modes may also be used.

[0044] When the diagnostic assistance observation mode is set, the light source processor 21 switches between emitting specific types of illumination light. Specifically, it alternates between a normal light period in which normal light is continuously emitted and a first illumination light period in which first illumination light is continuously emitted. Specifically, after a normal light period in which normal light is emitted is performed for a predetermined number of frames, a first illumination light period in which first illumination light is emitted is performed for a predetermined number of frames. After that, the normal light period begins again, and a set of the normal light period and the first illumination light period is repeated.

[0045] Note that the term "frame" refers to a unit for controlling the image sensor 45 (see FIG. 2) that captures an image of an object to be observed, and for example, "one frame" refers to a period that includes at least an exposure period in which the image sensor 45 is exposed to light from the object to be observed and a readout period in which an image signal is read out. In this embodiment, various periods such as the normal light period or the first period are defined in accordance with the "frame" that is the unit of imaging.

[0046] As shown in Figure 6, in the diagnostic assistance observation mode, a normal light period in which normal light is emitted is performed for three frames, followed by a first illumination light period in which first illumination light is emitted for one frame. The normal light period then begins again, and a set of normal light and first illumination light periods is repeated for four frames. Therefore, three normal images 71 are captured consecutively during the three-frame normal light period, and then one first image 72 is captured during the first illumination light period. Note that the first image 72 is shown shaded in Figure 6. After that, the normal light period resumes, and this pattern is repeated.

[0047] Light emitted from each of the LEDs 20a to 20e is incident on a light guide 41 via an optical path coupling unit (not shown) formed of a mirror, lens, etc. The light guide 41 is built into the endoscope 12 and a universal cord (a cord that connects the endoscope 12 with the light source device 13 and the processor device 14). The light guide 41 propagates light from the optical path coupling unit to the tip 12d of the endoscope 12.

[0048] An illumination optical system 30a and an imaging optical system 30b are provided at the distal end 12d of the endoscope 12. The illumination optical system 30a has an illumination lens 42, and illumination light propagated by a light guide 41 is irradiated onto the observation object via the illumination lens 42. The imaging optical system 30b has an objective lens 43, a zoom lens 44, and an imaging sensor 45. Various types of light, such as reflected light, scattered light, and fluorescence from the observation object, are incident on the imaging sensor 45 via the objective lens 43 and the zoom lens 44. As a result, an image of the observation object is formed on the imaging sensor 45. The zoom lens 44 is freely movable between the telephoto end and the wide-angle end by operating the zoom operation unit 12f, thereby enlarging or reducing the image of the observation object formed on the imaging sensor 45.

[0049] The image sensor 45 is a color image sensor in which each pixel is provided with an R (red), G (green), or B (blue) color filter. It captures an image of the object to be observed and outputs image signals of each of the RGB colors. The image sensor 45 may be a charge-coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor. Instead of the image sensor 45 equipped with primary color filters, a complementary image sensor equipped with complementary color filters of C (cyan), M (magenta), Y (yellow), and G (green) may be used. When a complementary image sensor is used, four CMYG image signals are output. Therefore, by converting the four CMYG image signals into three RGB image signals using complementary-to-primary color conversion, an RGB image signal similar to that of the image sensor 45 can be obtained. Alternatively, a monochrome sensor without color filters may be used instead of the image sensor 45.

[0050] The imaging sensor 45 is driven and controlled by an imaging control unit (not shown). The central control unit 59 (see FIG. 3) controls the light emission of the light source unit 20 through the light source processor 21 in synchronization with the imaging control unit, thereby controlling the imaging sensor 45 to capture an image of an observation target illuminated with normal light in the normal observation mode. As a result, Bc image signals are output from the B pixels of the imaging sensor 45, Gc image signals are output from the G pixels, and Rc image signals are output from the R pixels. In the special observation mode or the diagnostic assistance observation mode, the central control unit 59 (see FIG. 3) controls the light emission of the light source unit 20 to control the imaging sensor 45 to capture an image of an observation target illuminated with special light. As a result, in the first special observation mode, Bs1 image signals are output from the B pixels of the imaging sensor 45, Gs1 image signals are output from the G pixels, and Rs1 image signals are output from the R pixels.

[0051] A CDS / AGC (Correlated Double Sampling / Automatic Gain Control) circuit 46 performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal obtained from the image sensor 45. The image signal passed through the CDS / AGC circuit 46 is converted into a digital image signal by an A / D (Analog / Digital) converter 47. The digital image signal after A / D conversion is input to the processor device 14.

[0052] The processor device 14 stores programs related to image processing and other processes in a program memory (not shown). In the processor device 14, a central control unit 59, which is composed of an image processor as a first processor, executes the programs in the program memory, thereby realizing the functions of an image acquisition unit 51, a DSP (Digital Signal Processor) 52, a noise reduction unit 53, a memory 54, a signal processing unit 55, an image processing unit 56, a display control unit 57, a video signal generation unit 58, and the central control unit 59. The image processing unit 56 includes an identification information assignment unit 61 and a frame rate conversion unit 62, and these functions are also realized by the central control unit 59, which is also composed of an image processor, executing the programs in the program memory. The central control unit 59 also receives information from the endoscope 12 and the light source device 13, and controls the various units of the processor device 14, as well as the endoscope 12 or the light source device 13, based on the received information. The central control unit 59 also receives information, such as instructions, from the keyboard 16.

[0053] The image acquisition unit 51, which is a medical image acquisition unit, acquires digital image signals of endoscopic images input from the endoscope 12. The image acquisition unit 51 acquires image signals for each frame that capture an image of an observation target illuminated by each illumination light. The type of illumination light, i.e., the spectrum of the illumination light, is one of the imaging conditions. The image acquisition unit 51 acquires multiple types of endoscopic images that differ from each other in imaging conditions such as the spectrum of the illumination light.

[0054] The imaging conditions include the spectrum of the illumination light, i.e., the light intensity ratio of each LED 20a to 20d, as well as the observation distance from the object to be observed and the zoom magnification of the endoscope 12. The light intensity ratio is acquired from the central control unit 59. The observation distance includes, for example, a non-magnified observation distance where the observation distance is long, and a magnified observation distance where the observation distance is short, and is acquired based on the exposure amount obtained from the endoscopic image. The observation distance may also be acquired by frequency analysis of the image. The zoom magnification includes, for example, non-magnified observation for non-magnified observation, and low to high magnifications that enable magnified observation, and can be acquired based on the change operation of the zoom operation unit 12f. In this embodiment, the spectrum of the illumination light is used as the imaging condition.

[0055] The acquired image signal is transmitted to the DSP 52. The DSP 52 performs digital signal processing such as color correction on the received image signal. The noise reduction unit 53 performs noise reduction processing, such as a moving average method or a median filter method, on the image signal that has been subjected to color correction and other processing by the DSP 52. The noise-reduced image signal is stored in the memory 54.

[0056] The signal processing unit 55 acquires the noise-reduced image signal from the memory 54. Then, the acquired image signal is subjected to signal processing such as color conversion processing, color enhancement processing, and structure enhancement processing as necessary, to generate a color endoscopic image showing the object of observation.

[0057] In the normal observation mode or the diagnostic assistance observation mode, the signal processing unit 55 performs image processing for the normal observation mode, such as color conversion processing, color enhancement processing, and structure enhancement processing, on the input image signal for the normal image after noise reduction for one frame. The image signal that has been subjected to this image processing for the normal observation mode is input to the image processing unit 56 as a normal image.

[0058] In the special observation mode or the diagnostic assistance observation mode, the image signals for the first image after noise reduction for one frame input in the first special observation mode are subjected to image processing for the first special observation mode, such as color conversion processing, color enhancement processing, and structure enhancement processing. The image signals subjected to the image processing for the first special observation mode are input to the image processing unit 56 as the first image.

[0059] The endoscopic image generated by the signal processing unit 55 is a normal observation image when the observation mode is normal observation mode, and a special observation image including the first image when the observation mode is special observation mode, and the contents of the color conversion processing, color enhancement processing, and structure enhancement processing differ depending on the observation mode. In normal observation mode, the signal processing unit 55 generates a normal observation image by performing the various signal processing described above to make the observation target appear in natural colors. In special observation mode, the signal processing unit 55 generates a special observation image including the first image by performing the various signal processing described above to, for example, emphasize the blood vessels of the observation target.

[0060] The semiconductor light source includes a V-LED 20a that emits violet light V (first narrowband light) having a wavelength band of 410±10 nm at a center wavelength and a wavelength range of 420 to 500 nm, and a B-LED 20b that emits blue light B (second narrowband light) having a wavelength band of 450±10 nm at a center wavelength and a wavelength range of 380 to 420 nm. Therefore, in the first image, which is a special observation image generated by the signal processing unit 55, blood vessels or blood located relatively shallow in the observation target relative to the surface of the mucosa (so-called superficial blood vessels) appear in a magenta color (e.g., brown). Therefore, in the first image, the blood vessels or bleeding (blood) in the observation target are emphasized by their different colors compared to the mucosa, which is displayed in a pinkish color.

[0061] The image processing unit 56 performs various types of image processing. The image processing unit 56 includes an identification information assigning unit 61 and a frame rate conversion unit 62. The identification information assigning unit 61 modifies a portion of the data constituting the acquired endoscopic image, or modifies at least one type of endoscopic image but not another type of endoscopic image, thereby generating an identification-information-attached medical image in which part of the data constituting the endoscopic image is identification information indicating the type of endoscopic image. The frame rate conversion unit 62 converts the frame rate for the display 15 or medical image processing device 17, if necessary, when sending the identification-information-attached medical image to these devices. Note that in this embodiment, since the medical image is an endoscopic image, an identification-information-attached endoscopic image is generated as an identification-attached medical image.

[0062] The data constituting an endoscopic image refers to the data of the image itself, not data other than the image, such as the header portion of an information container that stores the endoscopic image. The data constituting an endoscopic image is preferably image file data that can be handled on a general-purpose PC. As long as it is data constituting an endoscopic image, there are no restrictions on the data format or representation method, and pixel values, frequency distributions, values ​​calculated using these, etc. can be used.

[0063] The identification information assigning unit 61 identifies the type of endoscopic image from information regarding the light emission of the light source unit 20, which the central control unit 59 controls through the light source processor 21 in synchronization with the imaging control unit, and changes a portion of the data constituting the image itself of the acquired endoscopic image according to the identified type of endoscopic image. Alternatively, depending on the identified type of endoscopic image, the identification information assigning unit 61 changes a portion of the data constituting the image itself of the acquired endoscopic image for one type of endoscopic image, but does not change the data constituting the image itself of the acquired endoscopic image for another type of endoscopic image. The identification information assigning unit 61 generates an endoscopic image with identification information by changing or not changing the data constituting the image for part of the endoscopic image. Therefore, endoscopic images with identification information include endoscopic images in which the data constituting the image has been changed, and endoscopic images in which the data constituting the endoscopic image has not been changed and are left as is.

[0064] The type of endoscopic image is identified by recognizing the identification information in the endoscopic image with identification information. If the identification information is a pixel value, correspondence information is prepared that associates the pixel position and the changed pixel value when changing the pixel value in the endoscopic image with the type of endoscopic image, and this correspondence information is used to check the type of endoscopic image corresponding to the pixel that is identification information among the pixels that make up the endoscopic image with identification information. This makes it possible to identify the type of endoscopic image that the endoscopic image with identification information is. Therefore, there is no need to use data other than the image itself, such as the header portion, to identify the type of endoscopic image.

[0065] The data constituting the endoscopic image is preferably the pixel values ​​of the pixels constituting the endoscopic image. In this case, the identification information assigning unit 61 generates the endoscopic image with identification information by changing the pixel values ​​of a predetermined portion of the pixels constituting the endoscopic image in accordance with the type of endoscopic image identified. In addition, in some cases, in order to distinguish the type of endoscopic image with changed pixel values ​​from the type of endoscopic image with changed pixel values, the endoscopic image with identification information is defined as an endoscopic image with identification information for another type of endoscopic image with unchanged pixel values.

[0066] It is preferable that the pixel values ​​to be changed are changed so as not to affect the use of the endoscopic image for observation, diagnosis, etc. Either color information or brightness information can be used for the pixel values. Furthermore, the pixel values ​​can be changed so that the user cannot see the changed pixels, or the pixel values ​​can be changed in a manner that allows the user to see the changed pixels but does not affect the visibility of the observation target, etc., displayed in the endoscopic image.

[0067] In cases where the user cannot see the pixels whose pixel values ​​have been changed, methods include changing some of the pixels in the endoscopic image to specific pixel values ​​that do not affect the user's visibility, changing to dummy pixel values, or applying a digital watermark to the endoscopic image. Note that digital watermarks can also be applied to data other than pixel values ​​that make up the endoscopic image.

[0068] When changing some pixels of an endoscopic image to pixels with specific pixel values ​​that do not affect the user's visibility, changes can be made, such as increasing or decreasing, at least one or more of the red, green, and blue values ​​that make up the pixel values ​​at some positions in the endoscopic image, or brightness information. Depending on the type of endoscopic image, the red, green, blue, or brightness may be changed accordingly, or the pixel values ​​may be increased or decreased accordingly. Any method can be used as long as the pixels with changed pixel values ​​are not visible to the user when viewed. In this case, the type of endoscopic image can be identified by, for example, comparing the changed pixel values ​​with the surrounding pixel values.

[0069] Even when some pixel values ​​of an endoscopic image are replaced with dummy pixel values, the replacement with the dummy pixel values ​​can be performed without affecting the user's visibility. The dummy pixel values ​​are, for example, pixel values ​​determined in advance according to the type of endoscopic image. These dummy pixel values ​​are replaced with pixel values ​​at predetermined positions in the endoscopic image. In this case, the type of endoscopic image can be identified by acquiring the dummy pixel values ​​at the replaced positions.

[0070] When applying a digital watermark to an endoscopic image, a known digital watermarking technique can be used. For example, watermark information including the type of endoscopic image can be embedded in the endoscopic image, and the type of endoscopic image can be identified when obtaining the watermark information or restoring the watermarked endoscopic image.

[0071] 7, an endoscopic image with identification information 82 is generated by changing part of the data constituting the endoscopic image to identification information 81, which is a preset pixel value. In the endoscopic image with identification information 82, the identification information 81 is assigned to an area of ​​the endoscopic image with identification information 82 that does not show the observation target, and is assigned in a manner that allows the user to view it but does not affect the visibility of the observation target, etc. shown in the endoscopic image.

[0072] The data constituting the endoscopic image is preferably data constituting a predetermined region of the endoscopic image. Therefore, the identification information of the endoscopic image with identification information is preferably located in a predetermined region of the endoscopic image. Examples of the predetermined region include a masked portion of the endoscopic image that does not show the observation target, or the edge of the region that shows the observation target. As shown in FIG. 8, in this specification, the endoscopic image 83 refers to the entire image displayed on the display 15, including the observation target portion 83a and the masked portion 83b. In FIG. 8, the observation target portion 83a is indicated by a dashed line, and the masked portion 83b is indicated by diagonal lines. In the case of FIG. 7, the identification information 81 is obtained by changing a portion of the masked portion 83b of the endoscopic image to a predetermined pixel value while keeping it visible to the user.

[0073] As shown in FIG. 9 , the identification information assigning unit 61 assigns different pixel values ​​to the endoscopic image 83 depending on the type of endoscopic image by replacing them with pixel values ​​at similar positions in the endoscopic image 83, and assigns the modified pixel values ​​as identification information. In FIG. 9(A), first identification information 81a is assigned, and in FIG. 9(B), second identification information 81b is assigned. Therefore, the endoscopic image 82 with identification information of FIG. 9(A) and the endoscopic image 82 with identification information of FIG. 9(B) are different types of endoscopic images, and the type of each endoscopic image of the identification information-added endoscopic image 82 can be identified solely from the image data of the identification information-added endoscopic image 82, without relying on information other than the image data, such as a header portion, or other information synchronously transmitted from, for example, the central control unit 59 or the light-source processor 21. Furthermore, by making the first identification information 81a and the second identification information 81b different colors that can be distinguished by the user, the user can correctly identify the type of the endoscopic image at a glance of the identification information-added endoscopic image 82.

[0074] The multiple types of endoscopic images preferably include display images to be displayed on the display 15 and analysis images for performing analysis related to the diagnostic support information. By including two types of endoscopic images, i.e., display images and analysis images, the images to be displayed on the display 15 can be used, and the images to be subjected to image analysis can be of a type that is difficult for the user to see but that provides good analysis results when subjected to image analysis by machine learning or the like. In this case, the analysis images can be prevented from being displayed on the display 15.

[0075] In this embodiment, in the diagnosis assistance observation mode, two types of endoscopic images, a normal image and a first image, which are different in type and spectrum of illumination light, are automatically acquired. Therefore, the normal image is used as the image for display, and the first image is used as the image for analysis.

[0076] As shown in FIG. 10, in this embodiment, a pattern of acquiring three frames of a normal image 71 followed by one frame of a first image 72 is repeated (see FIG. 6). The identification information assigning unit 61 assigns identification information 81 to each acquired endoscopic image to generate an endoscopic image 82 with identification information attached. Therefore, the identification information assigning unit 61 assigns first identification information 81a to the normal image 71 to identify it as a normal image by changing pixels in a predetermined area of ​​the mask portion of the endoscopic image to predetermined pixel values, thereby generating an endoscopic image 82a with first identification information attached. Similarly, identification information 81b to identify the first image 72 as the first image is assigned, thereby generating an endoscopic image 82b with second identification information attached. Note that in FIG. 10, the normal image 71 is shown with the normal image 71 and the display 15, indicating that the normal image 71 is displayed on the display 15. The first image 72 is not displayed on the display 15 and is therefore shown as is. In addition, in the figure, different shading of the identification information 81 indicates that it is different identification information 81.

[0077] 10, as described above, the normal image 71 and the first image 72 appear to have different colors when viewed by humans, and therefore the difference in appearance between the two endoscopic images is indicated by hatching the first image 72. Also, the identification information 81 attached to the endoscopic image 82 with identification information attached is shown enlarged. In the figure, to avoid cluttering the diagram, only some parts may be labeled with reference numerals.

[0078] Furthermore, the identification information 81 may include two or more pieces of information. For example, the identification information 81 may include information regarding the order in which the endoscopic images were captured in addition to the type of endoscopic image. As shown in FIG. 11, similar to the case of FIG. 10, a pattern of acquiring three frames of the normal image 71 and then acquiring one frame of the first image 72 is repeated (see FIG. 6). The identification information assigning unit 61 assigns identification information 81 to each of the acquired endoscopic images to generate an endoscopic image 82 with identification information. Here, the identification information assigning unit 61 assigns first identification information 81 (A-1) to the first frame of the normal image 71 to identify that it is a normal image and was captured first by changing pixels in a predetermined region of the mask portion of the endoscopic image to a predetermined pixel value, thereby generating an endoscopic image 82 (A-1) with first identification information. The first identification information 81 (A-1) indicates that the image is a normal image and was captured first.

[0079] In the frame following the normal image 71, first identification information 81 (A-2) is assigned to identify that the image is a normal image and was captured second, by changing the pixels in a predetermined area of ​​the mask portion of the endoscopic image to predetermined pixel values, and an endoscopic image 82 (A-2) with first identification information is generated. The first identification information 81 (A-2) indicates the identification information (A-2) indicating that the image is a normal image and was captured second. Similarly, FIG. 11 shows that an endoscopic image 82 with first identification information assigned to it is generated, to which first identification information 81 (A-3) to first identification information (A-7) are assigned.

[0080] Similarly to the normal image 71, the first frame of the first image 72 is given second identification information 81(B-1) for identifying that it is the first image and was captured first by changing the pixels in a predetermined area of ​​the masked portion of the endoscopic image to predetermined pixel values, thereby generating an endoscopic image 82(B-1) with second identification information. The second identification information 81(B-1) indicates the identification information (B-1) that indicates that it is the first image and was captured first.

[0081] In the frame next to the first image 72, second identification information 81 (B-2) for identifying that it is the first image and was captured second is assigned by changing the pixels in a predetermined area of ​​the mask portion of the endoscopic image to a predetermined pixel value, and an endoscopic image 82 (B-2) with second identification information is generated. The second identification information 81 (B-2) indicates the identification information (B-2) indicating that it is the first image and was captured second. Figure 11 shows that an endoscopic image 82 with second identification information assigned to it is generated.

[0082] It is preferable that the identification information 81 includes information about the order in which the endoscopic images were taken in addition to the type of endoscopic image, since this information can be easily obtained from only the image data of the endoscopic image.

[0083] Furthermore, the identification information assigning unit 61 may assign identification information to the analysis image by changing a part of the data constituting the analysis image, and assign identification information to the display image by not changing the part of the data constituting the display image that corresponds to the data that has been assigned as identification information in the analysis image. Similarly, the identification information assigning unit 61 may assign identification information to the display image by changing a part of the data constituting the display image, and assign identification information to the analysis image by not changing the part of the data constituting the analysis image that corresponds to the data that has been assigned as identification information in the display image.

[0084] As shown in Fig. 12, for the first image 72, which is an image for analysis, identification information 81b is assigned by changing part of the data constituting the first image 72 to generate an endoscopic image 82b with second identification information, and for the normal image 71, which is an image for display, the data constituting the normal image 71 in the part corresponding to the data set as identification information 81b in the first image 72 is left unchanged and used as identification information 81a to generate an endoscopic image 82a with first identification information. Note that in Fig. 12, the identification information 81b is shown without shading, which indicates that no changes have been made to the data constituting the original endoscopic image by the identification information assigning unit 61. The same applies to the following Fig. 13.

[0085] As shown in Figure 13, in a normal image 71, which is an image for display, identification information 81a is assigned by changing part of the data constituting the normal image 71 to generate an endoscopic image 82a with first identification information, and in a first image 72, which is an image for analysis, a second endoscopic image 82b with identification information is generated by leaving the data constituting the first image 72 in the part corresponding to the data set as identification information 81a in the normal image 71 unchanged and setting it as identification information 81b.

[0086] As described above, when the identification information assigning unit 61 assigns identification information 81 to an image for analysis or an image for display, the normal image, which is the image for display, and the first image, which is the image for analysis, can be distinguished from each other by whether or not a portion of the data constituting the image has been changed. Therefore, for example, if there are two types of endoscopic images, it is sufficient to change a portion of the data in one type of endoscopic image, thereby reducing the effort required to assign identification information 81. Furthermore, when identification information 81 is assigned by changing a portion of the data only in the identification image, the image data of the image for display is not changed at all. Therefore, when the image for display is displayed on the display 15, etc., the identification information 81 does not affect the user's visibility, which is preferable.

[0087] The endoscopic image 82 with identification information is sent from the processor device 14 to the medical image processing device 17. The medical image processing device 17 receives the endoscopic image 82 with identification information sent from the processor device 14 and controls the display of the endoscopic image 82 with identification information on the display 15 based on the type of the medical image 82 with identification information. The medical image processing device 17 also performs display processing or analysis processing depending on the type of the endoscopic image 82 with identification information. After the analysis processing, it sends an analysis result image that displays the results of the analysis to the processor device 14. It also generates a superimposed image by superimposing the analysis result image on the endoscopic image with identification information 82, and displays the superimposed image on the display 15.

[0088] The medical image processing device 17 is a general-purpose PC equipped with a processor, and various functions are realized by installing software. Like the processor device 14, the medical image processing device 17 also has programs related to image analysis and other processes stored in a program memory. In the medical image processing device 17, a central control unit, which is comprised of an image processor (second processor), etc., executes programs stored in the program memory to realize the functions of an identification-added medical image acquisition unit 91, an identification-added medical image recognition unit 92, an identification-added medical image processing unit 93, and a display control unit 94 (see FIG. 14). The identification-added medical image processing unit 93 includes a display image processing unit 95, an image analysis unit 96, an analysis result creation unit 97, an image superposition unit 98, and a frame rate conversion unit 99 (see FIG. 14). These functions are also realized by the central control unit, which is comprised of an image processor, executing programs stored in the program memory. The central control unit also receives information from the processor device 14, etc., and controls each unit of the medical image processing device 17 based on the received information. It is also connected to a user interface such as a keyboard (not shown) and receives information such as instructions from the user interface.

[0089] The medical image processing device 17 is also connected to a display 15, which displays various images generated by the medical image processing device 17. Various devices may be connected to the medical image processing device 17. Examples of the various devices include a user interface such as a keyboard for issuing instructions, and storage for saving data such as images.

[0090] As shown in FIG. 14 , the medical image processing device 17 includes an identification-added medical image acquisition unit 91, an identification-added medical image recognition unit 92, an identification-added medical image processing unit 93, and a display control unit 94. The identification-added medical image acquisition unit 91 acquires multiple types of identification-added endoscopic images 82 sent from the processor device 14. The acquired images are sent to the identification-added medical image recognition unit 92. In the identification-added endoscopic images 82, part of the data constituting the endoscopic image is identification information 81. The identification-added medical image recognition unit 92 recognizes the type of the identification-added endoscopic image 82 based on the identification information 81 assigned to the identification-added endoscopic image 82. The identification-added medical image processing unit 93 controls display on the display 15 based on the type of the identification-added endoscopic image 82, and also performs image processing set for each type of identification-added endoscopic image 82 on the identification-added endoscopic image 82.

[0091] The identification information-added medical image recognition unit 92 recognizes the type of the identification information-added endoscopic image 82 based on the identification information of the identification information-added endoscopic image 82. The type of the identification information-added endoscopic image 82 is the same as the type of the endoscopic image that was the source of the identification information-added endoscopic image 82. Recognition is performed based on the content of the identification information 81. The identification information-added medical image recognition unit 92 is previously provided with correspondence information in which the content of the identification information 81 corresponds to the type of endoscopic image. The type of endoscopic image 82 with identification information is identified based on the correspondence information and the content of the identification information 81 held by the identification information-added endoscopic image 82. The identification information 81 and the content of the identification information 81 are the same as the identification information 81 added by the identification information adding unit 61 in the processor device 14, as described above.

[0092] As shown in FIG. 14, the identification-information-added medical image processing unit 93 includes a display image processing unit 95 , an image analysis unit 96 , an analysis result generation unit 97 , an image superimposition unit 98 , and a frame rate conversion unit 99 .

[0093] The image processing performed by the identification-information-attached medical image processing unit 93 includes image processing for display and image processing for analysis. The multiple types of identification-information-attached endoscopic images 82 preferably include display images for display on the display 15 and analysis images for performing analysis related to the diagnostic support information. Furthermore, when the identification-information-attached endoscopic images 82 are display images, the identification-information-attached medical image processing unit 93 preferably performs image processing for display on the display images, and when the identification-information-attached endoscopic images 82 are analysis images, it preferably performs image processing for analysis on the analysis images.

[0094] When the type of the identification-added endoscopic image 82 is a display image, which is a type of endoscopic image to be displayed on the display 15, the display image processing unit 95 performs display image processing on this image. It is preferable that the display image processing be different for each type of identification-added endoscopic image 82. The display image processing performed by the display image processing unit 95 generates an image suitable for display on the display 15.

[0095] When the type of the identification-added endoscopic image 82 is an analysis image, which is a type of endoscopic image for performing analysis related to diagnostic support information, the image analysis unit 96 performs analytical image processing on this image. It is preferable that the analytical image processing differs for each type of identification-added endoscopic image 82 and also differs for each analysis content. Diagnostic support information can be obtained by the image analysis processing performed by the image analysis unit 96. The diagnostic support information is presented to the user as an analysis result image or the like showing this information.

[0096] 15, specifically, in this embodiment, the image for display is the normal image 71 and the image for analysis is the first image 72, and therefore there are two types of endoscopic images 82 with identification information: a first endoscopic image 82a with identification information 81a added to the normal image 71, and a second endoscopic image 82b with identification information 81b added to the first image 72. The identification information 81 of these endoscopic images 82 with identification information is read by an identification-information-added medical image recognition unit 92, and the type of endoscopic image is identified. Note that when the type of identification information or endoscopic images with identification information is not distinguished, they are referred to as, for example, identification information 81 or endoscopic images with identification information 82, etc.

[0097] After the type of each of the identification-information-added endoscopic images 82 has been specified, the first identification-information-added endoscopic image 82a and the second identification-information-added endoscopic image 82b are processed according to different flows.

[0098] The endoscopic image 82a with first identification information is sent to a display image processing unit 95 and an image analysis unit 96. The display image processing unit 95 performs image processing for displaying the endoscopic image 82a with first identification information on the display 15. The image analysis unit 96 performs analysis of the endoscopic image 82a with first identification information when necessary. The endoscopic image 82b with second identification information is sent to the display image processing unit 95 and the image analysis unit 96. When the endoscopic image 82b with second identification information is to be displayed on the display, the display image processing unit 95 performs image processing for displaying the endoscopic image 82b with second identification information on the display 15. The image analysis unit 96 performs analysis of the endoscopic image 82b with second identification information when necessary. In FIG. 15 , the flow of the endoscopic image 82b with second identification information is indicated by a dashed line to distinguish between the flow of the endoscopic image 82a with first identification information and the flow of the endoscopic image 82b with second identification information.

[0099] In this embodiment, the medical image processing unit 93 with identification information performs image processing for display in the display image processing unit 95 since the endoscopic image 82a with first identification information is an image for display, but does not perform image analysis; and performs image processing for analysis in the image analysis unit 96 since the endoscopic image 82b with second identification information is an image for analysis, but does not display the endoscopic image 82b with second identification information on the display 15, so does not perform image processing for display.

[0100] The image analysis unit 96 performs analytical image processing for computer-aided diagnosis (CAD) on the identification-added endoscopic image 82. The analytical image processing can be any known analytical image processing. The analytical image processing based on the endoscopic image outputs diagnostic support information such as various feature quantities such as oxygen saturation, detection of blood vessel position or lesion position, or estimation of the stage of the lesion.

[0101] It is preferable that the image analysis unit 96 performs the analytical image processing using an analytical model based on machine learning. The analytical model based on machine learning preferably uses a convolutional neural network that outputs good results in image analysis. It is also preferable that the analytical model be different for each type of identification-added endoscopic image 82. This is because the content of analytical image processing that can output good results differs for each type of identification-added endoscopic image 82. For the same reason, it is preferable that the analytical model be different for each analysis content. Therefore, it is preferable that the image analysis unit 96 is provided with multiple analytical models and uses an appropriate analytical model depending on the type of endoscopic image. It is also preferable that the multiple analytical models generate different diagnostic support information as analysis results.

[0102] In this embodiment, the image analysis unit 96 performs image processing for analysis on the endoscopic image 82b with second identification information. Because the endoscopic image 82b with second identification information is the first image 72, surface features and the like are emphasized, and favorable results can be obtained using an analysis model that distinguishes between neoplastic and non-neoplastic polyps. Therefore, the image analysis unit 96 analyzes the endoscopic image 82b with second identification information using an analysis model that detects neoplastic polyps to generate analysis results. Note that this analysis model distinguishes between neoplastic and non-neoplastic polyps, and even if a non-neoplastic polyp is present, it does not notify or warn the user if it is not a neoplastic polyp.

[0103] Preferably, the identification-added medical image processing unit 93 generates an analysis result image showing the results of the analytical image processing and superimposes the analysis result image on the display image to generate the analysis image. Specifically, the analysis result generating unit 97 acquires the analysis result from the image analysis unit 96 and generates the analysis result in a form that can notify the user of the analysis result from the image analysis unit 96, such as in the form of sound or image. In this embodiment, the presence or absence of a neoplastic polyp is notified to the user by displaying a frame and a color of the frame on the display 15 around the edge of the area of ​​the endoscopic image where the observation target is captured. If a neoplastic polyp is detected, a red frame is displayed, and if no neoplastic polyp is detected, a green frame is displayed. In this embodiment, since no neoplastic polyp is detected, the analysis result generating unit 97 generates an analysis result image 101 with a green frame around the edge of the area where the observation target is captured as the analysis result.

[0104] The image superimposing unit 98 acquires an endoscopic image on which the analysis result image 101 is to be superimposed from the display image processing unit 95. The endoscopic image on which the analysis result image 101 is superimposed is preferably an image for display, and is therefore a normal image 71 obtained by performing display processing on the first identification information-added endoscopic image 82a. The image superimposing unit 98 also acquires the analysis result image 101 from the analysis result creating unit 97. Then, the image superimposing unit 98 generates a superimposed image 102 by superimposing the analysis result image 101 on the normal image 71 acquired from the display image processing unit 95. The generated superimposed image 102 is transmitted to the display control unit 94.

[0105] The display control unit 94 acquires three types of images from the display image processing unit 95 and the image superimposing unit 98 and performs control to display them on the display 15. From the display image processing unit 95, it acquires a normal image 71 based on the endoscopic image 82a with first identification information, and a first image 72 based on the endoscopic image 82b with second identification information. From the image superimposing unit 98, it acquires a superimposed image 102. Therefore, the display control unit 94 performs control to display the normal image 71, the first image 72, and / or the superimposed image 102 on the display 15 in accordance with an instruction. As described above, by connecting the medical image processing device 17 to the display 15, it is possible to display one or more of these images in a preset layout.

[0106] Furthermore, the analysis result image 101 created by the analysis result creation unit 97 may be sent to the processor device 14. This allows the processor device 14 to superimpose the analysis result image 101 on various endoscopic images. The superimposed image can also be displayed on a display 15 connected to the processor device 14. In this way, sending the analysis result image 101 to the processor device 14 is desirable because it increases the availability of the analysis result image.

[0107] In the above embodiment, the first image 72, which is the analysis image, is not displayed on the display 15. However, depending on the type of analysis image, it may be displayed on the display 15. The analysis image to be displayed is preferably an endoscopic image of a type for which diagnostics using endoscopic images are established and observable. For example, an analysis image such as the first image 72, which is an endoscopic image obtained by special light observation using narrow-band blue light, is displayed on the display 15 because diagnostics using this image are established and viewing it is useful for doctors to make diagnoses. Examples of types of analysis images that are preferably displayed include an analysis image using narrow-band blue light, an endoscopic image that has been subjected to color enhancement processing or structure enhancement processing, or an analysis image that displays biological information such as oxygen saturation.

[0108] On the other hand, analysis images that are difficult to observe because diagnostics using that type of endoscopic image have not yet been established as analysis images may not be displayed on the display 15. For example, an endoscopic image illuminated only with violet light V is an analysis image that is useful for analyzing oxygen saturation levels, etc., but may not be useful for diagnosis even if displayed on the display 15. Therefore, the identification-information-added medical image processing unit 93 may control the display of the identification-information-added endoscopic image 82 on the display 15 based on the type of the identification-information-added endoscopic image 82.

[0109] When the analysis image is displayed on the display 15, it is preferable to display it, for example, on a sub-screen of the display 15. In this case, the display 15 has a main screen and a sub-screen. Therefore, the identification-information-added medical image processing unit 93 displays the display image on the main screen of the display 15. Furthermore, it is preferable to determine whether or not to display the analysis image on the sub-screen of the display 15 based on the type of the identification-information-added endoscopic image 82, and to display the identification-information-added endoscopic image 82 that has been decided to be displayed on the sub-screen of the display 15.

[0110] As shown in FIG. 16 , in this embodiment, the display 15 has one main screen 201 and two sub-screens: a first sub-screen 202 and a second sub-screen 203. The display 15 also has a patient information display screen 204 that displays patient information. The main screen 201 displays a normal image 71, which is an image for display. The first sub-screen 202 displays, for example, an analysis result image 101 created by the analysis result creation unit 97. In FIG. 16 , the analysis result image is an image that displays, in map format, the discrimination results of the regions of interest, which are the analysis results. The discrimination results displayed in map format are distinguished by, for example, the color of the regions of interest on the map. The first sub-screen 202 also has an analysis result text display screen 205 that displays the analysis results in text. The analysis result text display screen 205 displays the analysis results in text, for example, by displaying "NON-NEOPLASTIC" or "HYPERPLASTIC."

[0111] The second sub-screen 203 displays, for example, an endoscopic image, among the images for analysis, that the analysis result creation unit 97 used to create the analysis result image 101. In this embodiment, the second sub-screen 203 displays the first image 72, which is an image for analysis. The first image 72 is an endoscopic image obtained by special light observation using narrow-band blue light, and therefore displaying it on the second sub-screen 203 is useful for diagnosis by a doctor or the like.

[0112] As shown in Fig. 17, when the display 15 has a sub-screen, depending on the type of analysis image, the analysis image is not displayed on the second sub-screen 203. Also, as shown in Fig. 18, when the analysis image is not displayed on the second sub-screen 203, a previous image 205, which is an endoscopic image acquired in the past, may be displayed for the subject of the display image displayed on the main screen 201. This is because comparing past and present images of the same subject can be useful for diagnosis, etc.

[0113] As described above, in the medical image processing system 18 including the processor device 14 and the medical image processing device 17, the identification-information-attached medical image acquisition unit 91 of the medical image processing device 17 acquires the identification-information-attached endoscopic image 82 generated by the identification information assignment unit 61 of the processor device 14. The identification-information-attached endoscopic image 82 is generated by using identification information 81 indicating the type of endoscopic image as part of the data constituting the endoscopic image. This makes it easy to identify the type of endoscopic image. For example, because the type of endoscopic image is included in the data of the endoscopic image itself, it can be easily identified on a general-purpose PC. Furthermore, when different image processing is performed in CAD or the like depending on the type of endoscopic image, the type of endoscopic image can be automatically identified and the image processing corresponding to the type of endoscopic image can be performed automatically and consecutively. Therefore, when the type of endoscopic image to be acquired and the image processing associated with it are set for each observation mode, the burden on the user is reduced compared to manually switching between acquiring a specific type of endoscopic image and the image processing associated with it.

[0114] The processor device 14, medical image processing device 17, and medical image processing system 18 are useful in IEE when automatically switching the type of illumination light. That is, when the difference in the spectrum of illumination light is used as an imaging condition and associated with the type of endoscopic image, multiple display images and diagnostic support information can be automatically obtained by setting the type of illumination light to automatically switch. Furthermore, since the display image and the analysis results can be processed by multiple devices, the processor device 14 and the medical image processing device 17, images can be created or processed with a high degree of freedom in a preferred form for displaying on the display 15 or a preferred form for creating a medical record or examination report.

[0115] Furthermore, when the endoscopic image with identification information 82 includes multiple types of identification information 81, other information can be obtained in addition to the type of endoscopic image, such as the spectrum of the illumination light. Examples of other information include information on the imaging order.

[0116] It is preferable that the processor device 14 and the medical image processing device 17 include a frame rate conversion unit 62 or 99 that adjusts the frame rate of the endoscopic image. In the processor device 14, the frame rate conversion unit 62 adjusts the frame rate of the image to be transmitted in order to send the image to the medical image processing device 17 or to display an image such as a display image on the display 15. Similarly, in the medical image processing device 17, the frame rate conversion unit 99 adjusts the frame rate of the image to be transmitted when necessary, and adjusts the frame rate of the image to be transmitted in order to display an image such as a display image on the display 15.

[0117] When transmitting the endoscopic image 82 with identification information including the image for display and the image for analysis from the processor device 14 to the medical image processing device 17, it is preferable to adjust the frames from which the image for display and the image for analysis were acquired by complementing them with the image for display and the image for analysis, respectively, and transmit the endoscopic image 82 with identification information at a frame rate suitable for processing by the medical image processing device 17. When complementing with the image for display and the image for analysis, for example, a complemented frame image is created by duplicating the frame from which the image for display and the image for analysis were acquired, and this complemented frame image can be used as the image for display or the image for analysis.

[0118] The frame rate conversion unit 62 in the processor device 14 creates a complementary frame image 73 for the frame of the image for display and the image for analysis. The endoscopic image 82a with first identification information generated from the normal image 71, the endoscopic image 82b with second identification information generated from the first image 72, and the complementary frame image 73 copied from the endoscopic image 82a with first identification information and the endoscopic image 82b with second identification information are combined to form a frame rate of 60 frames per second (60 fps), and then sent to the medical image processing device 17. This allows the medical image processing device 17 to acquire video at a frame rate adjusted to a constant rate.

[0119] 19, for example, in the processor device 14, the first identification-added endoscopic image 82a, which is an image for display, is acquired at 30 fps per second, and the second identification-added endoscopic image 82b, which is an image for analysis, is acquired at 15 fps per second. The second identification-added endoscopic image 82b is indicated by diagonal lines. To achieve a combined frame rate of 60 fps for the first identification-added endoscopic image 82a, the second identification-added endoscopic image 82b, and the complementary frame image 73, the frame rate conversion unit 62 copies 10 fps from the first identification-added endoscopic image 82a and 5 fps from the second identification-added endoscopic image 82b out of the 15 fps of the complementary frame image 73 to generate the complementary frame image 73. In other words, at 30 fps for the first identification-added endoscopic image 82a, one frame is copied for every three frames to generate the 10 fps complementary frame image 73. Similarly, at 15 fps for the second identification-added endoscopic image 82b, one frame is duplicated for every five frames to create a 5-fps complementary frame image 73. The complementary frame image 73 is indicated by a dotted line. When duplicating, the image of the frame immediately preceding the duplication timing can be duplicated. In this way, if the first identification-added endoscopic image 82a is acquired at 30 fps and the second identification-added endoscopic image 82b is acquired at 15 fps, the frame rate conversion unit 62 can add 15 fps to the complementary frame image 73 to make it 60 fps.

[0120] In the processor device 14, the image acquisition unit 51 acquires a display image and an analysis image, and when the analysis image is not displayed on the display 15, the frame in which the analysis image was acquired is complemented with the display image, thereby making it possible to display an endoscopic image that is easy to view. The same is true for the medical image processing device 17, and it is preferable to adjust the frame rate for the image to be displayed on the display 15.

[0121] 20, for example, in the processor device 14, an endoscopic image 82a with first identification information, which is an image for display, is acquired at 39 frames per second (39 fps), and an endoscopic image 82b with second identification information, which is an image for analysis, is acquired at 13 frames per second (13 fps). When the image for analysis is not displayed on the display 15, the frame rate conversion unit 62 creates a complementary frame image 73 for each of the frames of the image for display, and displays the endoscopic image 82a with first identification information and the complementary frame image 73 together on the display 15 at 60 frames per second (60 fps). This makes it possible to improve the visibility of the image displayed on the display 15. The frame rate conversion unit 99 in the medical image processing device 17 also functions in a similar manner.

[0122] In the image processing unit 56, the identification information assigning unit 61 and the frame rate conversion unit 62 may perform frame rate conversion by the frame rate conversion unit 62, and then the identification information assigning unit 61 may assign the identification information 81. In this case, the frame rate conversion unit 62 generates the complementary frame image 73, and then the identification information assigning unit 61 assigns the identification information 81.

[0123] The identification information 81 may be assigned to the complementary frame image 73. In this case, as shown in Fig. 21, an endoscopic image 82c with third identification information is generated by assigning identification information 81c indicating that the complementary frame image 73 is the complementary frame image 73 to the complementary frame image 73. The identification information 81c can be different from the identification information indicating the type of endoscopic image. This allows the medical image processing device 17 to easily recognize that the image is the complementary frame image 73 from the image data.

[0124] Furthermore, the identification information 81c can include information regarding the type of the endoscopic image from which the image is copied. As shown in Fig. 22, in the case of a complementary frame image 73 obtained by copying a normal image 71, an endoscopic image 82(C-1) with third identification information to which identification information 81(C-1) is added may be generated, and in the case of a complementary frame image 73 obtained by copying a first image 72, an endoscopic image 82(C-2) with third identification information to which identification information 81(C-2) is added may be generated. This allows the medical image processing device 17 to easily determine from the image data not only that the image is a complementary frame image 73, but also what the original image from which the complementary frame image 73 is copied.

[0125] 23, an endoscopic image 82 (A-3) with first identification information is generated based on the third captured image of the normal image 71, and an endoscopic image 82 (A3-C1) with third identification information is generated as an interpolated frame image 73 obtained by duplicating this normal image 71. Similarly, an endoscopic image 82 (Bn) with second identification information is generated based on the nth captured image of the first image 72, and an endoscopic image 82 (Bn-Cm) with third identification information is generated as an interpolated frame image 73 obtained by duplicating this first image 72. The mth interpolated frame image 73 is an endoscopic image 82 (Bn-Cm) with third identification information is generated as an interpolated frame image 73 obtained by duplicating this first image 72. This allows the medical image processing device 17 to easily determine from the image data not only that the interpolated frame image 73 is a complementary frame image 73, but also the source of the complementary frame image 73 and the imaging order.

[0126] In the medical image processing device 17, the complementary frame image 73 can be identified, and therefore the complementary frame image 73 can be considered as one type of endoscopic image. Therefore, the medical image processing device 17 can perform an image processing method corresponding to the complementary frame image 73. Examples of the image processing method for the complementary frame image 73 include performing image processing on an endoscopic image of the same type as the original image of the complementary frame image 73, or not performing image processing on the complementary frame image 73.

[0127] Furthermore, when a high frame rate is set by adjusting the frame rate conversion unit 62, the endoscopic image with the first identification information 82a, the endoscopic image with the second identification information 82b, or the endoscopic image with the third identification information 82c which is the complementary frame image 73 may be identified, and when the endoscopic image with the third identification information 82c is present at a certain rate or more, the speed of image processing may be adjusted in image processing for the endoscopic image with the first identification information 82a or the endoscopic image with the second identification information 82b from the viewpoint of image processing speed, etc. In this way, by assigning the identification information 81c to the complementary frame image 73 as well, the medical image processing device 17 can grasp information about the frame rate from information only about the image data without acquiring information about the frame rate, and this can be utilized for adjusting the speed of image processing, etc.

[0128] Next, a series of steps for determining the type of endoscopic image will be described with reference to the flowchart shown in Fig. 24. An image of an observation target is captured using an endoscope. A normal image 71, which is an image for display, and a first image 72, which is an image for analysis, are acquired with a predetermined frame pattern (step ST110). The normal image 71 is displayed on the display 15 after the frame rate has been adjusted. In the processor device 14, the identification information assigning unit 61 assigns identification information 81 to each of the normal image 71 and the first image 72 (step ST120).

[0129] The first identification-added endoscopic image 82a and the second identification-added endoscopic image 82b, to which the identification information 81 has been assigned, are acquired by the identification-added medical image acquisition unit 91 of the medical image processing device 17 (step ST130). The display image processing unit 95 performs image processing for display on the first identification-added endoscopic image 82a, which is an image for display. The image analysis unit 96 performs image analysis for obtaining diagnostic support information on the second identification-added endoscopic image 82b, which is an image for analysis, using an analysis model based on machine learning (step ST140). The analysis result creation unit 97 generates an analysis result image 101 that displays the results of the image analysis. The image superimposition unit 98 superimposes the analysis result image 101 on the normal image 71, which is the first identification-added endoscopic image 82a that has been subjected to image processing for display, to generate a superimposed image 102 (step ST150). The superimposed image 102 is displayed on the display 15 (step ST160).

[0130] In the above embodiment, the present invention is applied to the processing of endoscopic images, but the present invention can also be applied to processor devices, medical image processing devices, medical image processing systems, etc. that process medical images other than endoscopic images.

[0131] 25 , part or all of the image processing unit 56 and / or central control unit 59 of the endoscope system 10 can be provided in a diagnosis support device 610 that communicates with, for example, the processor device 14 and cooperates with the endoscope system 10. Similarly, part or all of the medical image processing device 17 of the endoscope system 10 can be provided in a diagnosis support device 610 that communicates with, for example, the medical image processing device 17 and cooperates with the endoscope system 10.

[0132] 25 , part or all of the image processing unit 56 and / or central control unit 59 of the endoscope system 10 can be provided in a diagnosis support device 610 that acquires images captured by the endoscope 12, for example, directly from the endoscope system 10 or indirectly from a PACS (Picture Archiving and Communication Systems) 22. Similarly, part or all of the medical image processing device 17 of the endoscope system 10 can be provided in a diagnosis support device 610 that acquires images captured by the endoscope 12, for example, directly from the endoscope system 10 or indirectly from a PACS (Picture Archiving and Communication Systems) 22.

[0133] Furthermore, as shown in FIG. 26, a medical service support device 630 connected to various inspection devices including the endoscopic system 10, such as a first inspection device 621, a second inspection device 622, ..., an Nth inspection device 623, via a network 626, can be provided with part or all of the image processing unit 56 and / or central control unit 59 of the endoscopic system 10, or part or all of the medical image processing device 17.

[0134] In the above embodiment, the hardware structures of the processing units that perform various processes, such as the light source processor, the image processors that are the first and second processors, the central control unit 59, image acquisition unit 51, DSP 52, noise reduction unit 53, memory 54, signal processing unit 55, image processing unit 56, display control unit 57, and video signal generation unit 58 included in the processor device 14, and the identification-added medical image acquisition unit 91, identification-added medical image recognition unit 92, identification-added medical image processing unit 93, and display control unit 94 included in the medical image processing device, are various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture, and a dedicated electrical circuit, which is a processor having a circuit configuration specifically designed to perform various processes.

[0135] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor. Examples of multiple processing units configured with a single processor include, first, a configuration in which one processor is configured with a combination of one or more CPUs and software, as typified by client or server computers, and this processor functions as multiple processing units. Second, a configuration in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip, as typified by a System on Chip (SoC). In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0136] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit in the form of a combination of circuit elements such as semiconductor elements. [Explanation of symbols]

[0137] 10 Endoscopy System 12 Endoscopy 12a Insertion part 12b Operation section 12c curved section 12d Tip 12e Angle Knob 12f Zoom control 12g mode switch 13 Light source device 14 Processor unit 15 Display 16 keyboards 17 Medical image processing equipment 18 Medical Image Processing System 20 Light source section 20a V-LED 20b B-LED 20c G-LED 20d R-LED 21 Light Source Processor 22 PACS 30a illumination optical system 30b Imaging optical system 41 Light Guide 42 Lighting lens 43 Objective Lens 44 Zoom Lens 45 imaging sensor 46 CDS / AGC circuit 47 A / D converter 51 Image acquisition unit 52 DSP 53 Noise reduction section 54 memory 55 Signal Processing Unit 56 Image processing section 57, 94 Display control unit 58 Video signal generator 59 Central Control Unit 61 Identification information assignment unit 62, 99 Frame rate conversion section 71 Normal Images 72 Image 1 73 complementary frame images 81 Identification Information 81a First identification information 81b Secondary identification information 82 Endoscopic images with identification information 82a Endoscopic image with first identification information 82b Endoscopic image with second identification information 83 Endoscopic images 83a Observation area 83b Mask part 91 Medical image acquisition unit with identification information 92 Medical image recognition unit with identification information 93 Medical image processing unit with identification information 95 Display image processing unit 96 Image Analysis Unit 97 Analysis Results Creation Department 98 Image superimposition section 101 Analysis result image 102 Superimposed Images 201 Main Screen 202 1st sub-screen 203 Second sub-screen 204 Patient information display screen 205 Analysis result text display screen 206 Past Images 610 Diagnostic support device 621 First Inspection Device 622 Second Inspection Device 623 Nth Inspection Device 626 Network 630 Medical Business Support Devices ST110~ST160 Steps

Claims

1. A processor device including a first processor, The first processor Acquire multiple types of medical images under different shooting conditions, a part of the data constituting the medical image is changed depending on the type of the medical image, or the part of the data constituting the medical image is changed for at least one type of the medical image and not changed for another type of the medical image, thereby generating a medical image with identification information in which part of the data constituting the medical image is used as identification information indicating the type of the medical image; the data constituting the medical image is data constituting a predetermined region of the medical image, The predetermined region of the medical image is a mask in the medical image that does not show the object of observation. A processor device that is a part or an edge of the area in which the observed object is captured.

2. 2. The processor device according to claim 1, wherein the data constituting the medical image is pixel values.

3. The plurality of types of medical images include display images to be displayed on a display and images to be displayed as diagnostic information. and an analysis image for performing analysis relating to the analysis.

4. 4. The processor device according to claim 3, wherein the first processor sets the identification information for the analysis image by changing a portion of the data constituting the analysis image, and sets the identification information for the display image without changing the data constituting the display image in a portion corresponding to the data set as the identification information in the analysis image.

5. The processor device according to claim 3, wherein the first processor sets the identification information for the display image by changing a portion of the data constituting the display image, and sets the identification information for the analysis image without changing the data constituting the analysis image in a portion corresponding to the data set as the identification information in the display image.

6. 2. The processor device according to claim 1, wherein the imaging condition is a spectrum of illumination light.

7. 1. A medical imaging device comprising a second processor, The second processor Acquire multiple types of medical images with identification information, where part of the data constituting the medical images is identification information; Recognizing the type of the medical image with identification information based on the identification information; Controlling the display of the identification information-added medical image on a display based on the type of the identification information-added medical image; the identification information is data constituting a predetermined region of the medical image; A medical image processing device, wherein the predetermined region of the medical image is a mask portion of the medical image that does not show the object of observation or an edge portion of a region that shows the object of observation.

8. The medical image processing apparatus according to claim 7 , wherein the plurality of types of medical images with identification information include a display image to be displayed on the display and an analysis image to be used for analysis of diagnostic information.

9. 9. The medical image processing device according to claim 8, wherein the second processor displays the display image on a main screen of the display, and determines whether or not to display the analysis image on a sub-screen of the display based on the type of the medical image with identification information, and displays the medical image with identification information that has been determined to be displayed on the sub-screen of the display.

10. The medical image processing apparatus according to claim 7 or 8, wherein the second processor performs image processing set for each type of the identification-information-added medical image on the identification-information-added medical image based on the type of the identification-information-added medical image.

11. The medical image processing device according to claim 8, wherein the second processor performs image processing for display on the display image when the medical image with identification information is the image for display, and performs image processing for analysis on the image for analysis when the medical image with identification information is the image for analysis.

12. The medical image processing apparatus according to claim 11 , wherein the second processor performs the analytical image processing using an analytical model based on machine learning.

13. The medical image processing apparatus according to claim 11 or 12, wherein the second processor creates an analysis result image showing a result of the analysis image processing, and generates a superimposed image by superimposing the analysis result image on the display image.

14. The processor device of claim 1; and the medical image processing device according to any one of claims 7 to 9, The second processor acquires the plurality of types of medical images with the identification information generated by the first processor.

15. The processor device of claim 1; and the medical image processing device according to claim 13, The processor device is a medical image processing system that acquires the analysis result image that indicates the result of the analysis image processing created by the second processor.

16. 16. The method according to claim 15, wherein the processor device superimposes the analysis result image on the display image. Medical image processing system.

17. the processor device adjusts the frame rate of the identification information-added medical image; The medical image processing device acquires the medical image with identification information, the frame rate of which has been adjusted.

15. The medical image processing system according to claim 14.

18. The processor device or the medical image processing device processes an image to be displayed on a display. The medical image processing system of claim 14, wherein the frame rate of the image is adjusted.

19. a plurality of light sources that emit light in different wavelength bands; an endoscope for photographing an object illuminated by illumination light emitted from the plurality of light sources; and the medical image processing system of claim 14, The processor device is an endoscope system including a light source processor that controls the emission of each of a plurality of types of illumination light having different combinations of light intensity ratios of the plurality of light sources.

Images