Image processing device, endoscope system, and image processing program

Abstract

To provide an image processing device that prevents misunderstanding between a result of performed recognition processing and a result of not performed recognition processing, and to provide an endoscope system, and an image processing program.SOLUTION: An image processing device includes: a recognition part 72 for performing recognition processing of recognizing a subject using an image in which the subject is filmed; a determination part 73 for determining an operation to a subject; a setting part 74 for setting the recognition part 72 to be enabled or disabled using a determination result by the determination part 73; and a notification part 73 for making notification of a state whether the recognition part 72 is enabled or disabled.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an image processing device, an endoscope system, and an image processing program that perform subject recognition processing using an image captured by an endoscope. [Background technology]

[0002] In the medical field, an endoscope system including a light source device, an endoscope, and a processor device is widely used. The light source device generates illumination light. The endoscope captures an image of a subject using an image sensor. The processor device generates images and performs other image processing.

[0003] An endoscope system may have additional functions in addition to photographing a subject for observation. For example, an endoscope system having an "input mode" in which the position and direction of a treatment tool are operated by a touch panel is known (Patent Document 1). In the endoscope system of Patent Document 1, the state of the input mode is displayed on a screen. Also, an endoscope system having a function of displaying an image showing the state of curvature of the tip of the endoscope and a character string notifying that calibration has been completed on a screen to notify an abnormality in the system is known (Patent Document 2). In addition, an endoscope system that displays the on / off state of a foot switch on a screen (Patent Document 3), an endoscope system that displays a mark indicating that air is being sent on a screen (Patent Document 4), and an endoscope system that displays the on / off state of recording on a screen (Patent Document 5) are known.

[0004] Also, there are known endoscope systems that detect treatment tools and use the results. For example, there is known an endoscope system that uses the results of treatment tool detection to determine the image correction strength (Patent Document 6). There is also known an endoscope system that stops the autofocus function when a treatment tool is detected (Patent Document 7).

[0005] Additionally, in recent years, an endoscope system has become known that assists diagnosis by calculating biological function information using an image of a subject (Patent Document 8). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2008-245840 A [Patent Document 2] JP 2007-330405 A [Patent Document 3] JP 2005-305574 A [Patent Document 4] Japanese Patent Application Publication No. 11-10313 [Patent Document 5] Japanese Patent Application Publication No. 63-259614 [Patent Document 6] JP 2015-96237 A [Patent Document 7] JP 2013-150658 A [Patent Document 8] JP 2018-51364 A Summary of the Invention [Problem to be solved by the invention]

[0007] When obtaining information to assist diagnosis (hereinafter referred to as diagnostic assistance information) by recognizing a subject or a part of a subject having a specific characteristic using medical images such as images taken using an endoscope (hereinafter referred to as endoscopic images), it is necessary to use medical images that are taken under specific conditions and on which recognition processing can function. This is because if a medical image on which recognition processing cannot function is used, even if a result of the recognition processing is obtained, the result of the recognition processing may be inaccurate.

[0008] However, it is not always possible to obtain medical images that can be used for the recognition process. For this reason, the result of an inaccurate recognition process or diagnostic support information calculated using the result of an inaccurate recognition process may hinder the observation or diagnosis of the subject. For example, in an endoscope system, the accuracy of the recognition process may decrease if a treatment tool is captured in an endoscope image.

[0009] In order to avoid providing inaccurate results of the recognition process as described above, it is possible to consider not performing the recognition process when a medical image on which the recognition process can function cannot be obtained. However, if the recognition process is simply not performed, the doctor or the like who is the user may mistakenly recognize the result of the recognition process or the result of not performing the recognition process. For example, in the case of using a device or the like that performs a recognition process to detect a candidate for a lesion, if the recognition process is simply not performed, the doctor or the like who uses the device may mistakenly recognize that the result of the recognition process has not detected a candidate for a lesion. In other words, the doctor or the like may not be able to distinguish whether the result of the recognition process has not been displayed because the recognition process has not been performed, or the result of the recognition process has been performed and there was no target to recognize, so the result of the recognition process has not been displayed.

[0010] In addition, when surgery or the like is started on the subject, or when detailed observation is started after spraying a dye, the recognition process or the display of the recognition process result may be automatically disabled so as not to interfere with the surgeon's operation. However, in this case, as in the above, the doctor or the like may misinterpret the result of the recognition process or the result of not performing the recognition process. In addition, if the recognition function is automatically disabled even though the doctor or the like has not explicitly disabled it, there is a problem that the doctor or the like may lose concentration.

[0011] An object of the present invention is to provide an image processing device, an endoscope system, and an image processing program that prevent misrecognition between the results of recognition processing and the results of not performing recognition processing. [Means for solving the problem]

[0012] The image processing device of the present invention comprises a recognition unit that performs a recognition process to recognize a subject using an image of the subject, a discrimination unit that discriminates an operation performed on the subject, a setting unit that sets the recognition unit to be enabled or disabled using the discrimination result of the discrimination unit, and a notification unit that notifies the user of the enabled or disabled state of the recognition unit.

[0013] Preferably, recognizing a subject means detecting whether or not a part of the subject has a particular characteristic, distinguishing the type or degree of progression of a part of the subject having a particular characteristic, or obtaining biofunctional information about part or all of the subject.

[0014] The recognition unit preferably includes an artificial intelligence. The notification is preferably a notification of whether the recognition process using the artificial intelligence is valid or invalid. The recognition unit is preferably capable of at least one of detecting the presence or absence of a part of the subject having a specific characteristic and distinguishing the type or degree of progress of the part of the subject having the specific characteristic.

[0015] The recognition unit is preferably capable of detecting the presence or absence of a part of the subject having a specific characteristic, and distinguishing the type or degree of progress of the part of the subject having the specific characteristic. When the discrimination unit determines that the operation on the subject has not been performed or that the subject has been washed, it is preferable to enable both detection and discrimination.

[0016] It is preferable that the setting unit independently sets the detection and discrimination of the recognition unit to be enabled or disabled. When the discrimination unit discriminates bleeding from the subject, it is preferable that it performs detection and does not perform discrimination. It is preferable that the setting unit collectively sets the detection and discrimination to be enabled or disabled. When the discrimination unit discriminates that a dye has been sprayed on the subject, it is preferable that it performs discrimination without performing detection. It is preferable that the notification unit collectively displays on the monitor information indicating the enabled or disabled state of the recognition unit, the result of discrimination by the discrimination unit, the type of operation discriminated by the discrimination unit, and the image.

[0017] It is preferable that the discrimination section discriminates the presence or absence of a treatment tool or the type of the treatment tool.

[0018] The operation to be discriminated by the discrimination unit is preferably an operation that reduces the accuracy of the recognition process or an operation that requires the user to concentrate on the operation. The operation to be discriminated by the discrimination unit is preferably using a treatment tool, changing the color of the subject, or spraying a liquid on the subject. The operation preferably includes collection of biological tissue, a procedure related to endoscopic resection or treatment, spraying a liquid on the subject, or washing of the subject.

[0019] It is preferable that the type of operation determined by the determination unit is notified. It is preferable that the recognition unit is set to be invalid to not execute the recognition process or not notify the result of the recognition process.

[0020] It is preferable that the setting unit sets the recognition unit to be invalid when an operation is performed on the subject, and sets the recognition unit to be valid when no operation is performed on the subject. It is preferable that the setting unit sets the recognition unit to be invalid when a specific operation is performed on the subject.

[0021] It is preferable that the notification unit not only notifies whether the recognition unit is enabled or disabled, but also notifies the result of the discrimination by the discrimination unit. It is preferable that the notification unit notifies the enabled or disabled state of the recognition unit in an area different from the area where the image is displayed. When the recognition unit performs multiple types of recognition processing, it is preferable that the setting unit sets enabled or disabled for each recognition processing.

[0022] The endoscopic system of the present invention comprises the image processing device of the present invention described above, a light source device that generates illumination light, an endoscope that photographs a subject illuminated by the illumination light, and a processor device that controls the light source device and the endoscope to generate an image, and the image processing device acquires an image from the processor device.

[0023] The image processing program of the present invention causes a computer to execute a function of performing a recognition process to recognize a subject using an image of the subject, a function of determining an operation performed on the subject, a function of setting the recognition unit to be enabled or disabled using the determination result of the determination unit, and a function of notifying the user of the enabled or disabled state of the recognition unit. Effect of the Invention

[0024] According to the present invention, it is possible to provide an endoscope system that prevents misidentification between the results of having performed recognition processing and the results of having not performed recognition processing. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is an external view of an endoscope system. [Diagram 2] FIG. 1 is a block diagram of an endoscope system. [Diagram 3] FIG. 2 is a block diagram of an image processing unit. [Figure 4] 4 is a flowchart showing an operation mode of the endoscope system. [Diagram 5] This is a display example when the recognition unit is enabled. [Figure 6] 13 is a display example when the recognition unit is disabled. [Figure 7] 13 is a display example of the second embodiment. [Figure 8] 1 is a table showing the operating states of detection and discrimination. [Figure 9] 13 is a display example of a modified example. [Figure 10] 13 is a display example of operation contents. [Figure 11] 13 is a display example of operation contents. [Figure 12] 13 is a display example of operation contents. [Figure 13] FIG. 13 is a block diagram of a processor device for notifying the validity or ineffectiveness of a recognition unit by a method other than display on a monitor. [Figure 14] FIG. 1 is an explanatory diagram illustrating a medical image processing apparatus. [Figure 15] FIG. 1 is an explanatory diagram showing a diagnosis support device. [Figure 16] FIG. 1 is an explanatory diagram showing a medical service support device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] [First embodiment] As shown in Fig. 1, an endoscope system 10 (endoscopic device) includes an endoscope 12, a light source device 14, a processor device 16, a monitor 18, and a console 19. The endoscope 12 captures an image of a subject. The light source device 14 generates illumination light. The processor device 16 performs system control of the endoscope system 10 and image processing. The monitor 18 is a display unit that displays images and the like on the endoscope 12. The console 19 is an input device that inputs settings and the like to the processor device 16 and the like.

[0027] The endoscope 12 has an insertion section 12a to be inserted into the subject, an operation section 12b provided at the base end of the insertion section 12a, a bending section 12c provided at the tip side of the insertion section 12a, and a tip section 12d. The bending section 12c is bent by operating an angle knob 12e of the operation section 12b. As a result, the tip section 12d faces a desired direction. In addition to the angle knob 12e, the operation section 12b is provided with a treatment tool insertion port 12f, a zoom operation section 13a, and a water supply button 13b. The treatment tool insertion port 12f is an entrance through which a treatment tool such as a biopsy forceps, a snare, or an electric scalpel is inserted. The treatment tool inserted into the treatment tool insertion port 12f protrudes from the tip section 12d. The subject can be photographed by enlarging or reducing the size by operating the zoom operation section 13a. Furthermore, by operating the water supply button 13b, liquid such as water stored in the water supply tank 17 is sprayed from the tip portion 12d.

[0028] As shown in FIG. 2, the light source device 14 includes a light source section 20 that emits illumination light, and a light source control section 22 that controls the operation of the light source section 20.

[0029] The light source unit 20 emits illumination light for illuminating the subject. The illumination light includes excitation light used to emit the illumination light. The light source unit 20 includes a light source such as a laser diode (hereinafter referred to as LD), an LED (Light Emitting Diode), a xenon lamp, or a halogen lamp, and emits at least white illumination light or excitation light used to emit white illumination light. The white light includes so-called pseudo-white light that is substantially equivalent to white light in photographing the subject using the endoscope 12. The light source unit 20 includes, as necessary, a phosphor that emits light when irradiated with excitation light, or an optical filter that adjusts the wavelength band, spectral spectrum, or light amount of the illumination light or excitation light. In addition, the light source unit 20 can emit light having a specific wavelength band required for photographing an image used to calculate biological information such as oxygen saturation of hemoglobin contained in the subject.

[0030] In this embodiment, the light source unit 20 has four color LEDs, namely, V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d. The V-LED 20a emits purple light VL having a central wavelength of 405 nm and a wavelength band of 380 to 420 nm. The B-LED 20b emits blue light BL having a central wavelength of 460 nm and a wavelength band of 420 to 500 nm. The G-LED 20c emits green light GL having a wavelength band of 480 to 600 nm. The R-LED 20d emits red light RL having a central wavelength of 620 to 630 nm and a wavelength band of 600 to 650 nm. The central wavelengths of the V-LED 20a and the B-LED 20b have a width of about ±20 nm, preferably about ±5 nm to about ±10 nm.

[0031] The light source control unit 22 controls the timing of turning on, off or blocking each light source constituting the light source unit 20, and the amount of light emitted. As a result, the light source unit 20 can emit a plurality of types of illumination light with different spectral spectra. In this embodiment, the light source control unit 22 adjusts the spectral spectrum of the illumination light by inputting independent control signals to each of the LEDs 20a to 20d for turning on or off, the amount of light emitted when turned on, and inserting or removing an optical filter. As a result, the light source unit 20 emits white light. The light source unit 20 can emit illumination light consisting of at least narrowband purple light. The term "narrowband" refers to a substantially single wavelength band in relation to the characteristics of the subject and / or the spectral characteristics of the color filter of the image sensor 48. For example, when the wavelength band is, for example, about ±20 nm or less (preferably about ±10 nm or less) based on the central wavelength, this light is narrowband.

[0032] An illumination optical system 30a and an imaging optical system 30b are provided at the tip portion 12d of the endoscope 12. The illumination optical system 30a has an illumination lens 45, through which illumination light is emitted toward a subject.

[0033] The photographing optical system 30b has an objective lens 46, a zoom lens 47, and an image sensor 48. The image sensor 48 photographs the subject using reflected light of the illumination light returning from the subject (including reflected light, scattered light, fluorescence emitted by the subject, or fluorescence caused by a drug administered to the subject, etc.) via the objective lens 46 and the zoom lens 47. The zoom lens 47 moves by operating the zoom operation unit 13a, and enlarges or reduces the subject image.

[0034] The image sensor 48 has a color filter of one color among a plurality of color filters for each pixel. In this embodiment, the image sensor 48 is a color sensor having primary color filters. Specifically, the image sensor 48 has an R pixel having a red color filter (R filter), a G pixel having a green color filter (G filter), and a B pixel having a blue color filter (B filter).

[0035] As the image sensor 48, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor can be used. Although the image sensor 48 of this embodiment is a primary color sensor, a complementary color sensor can also be used. The complementary color sensor has, for example, a cyan pixel provided with a cyan color filter, a magenta pixel provided with a magenta color filter, a yellow pixel provided with a yellow color filter, and a green pixel provided with a green color filter. When a complementary color sensor is used, an image obtained from the pixels of each color can be converted into an image similar to an image obtained by a primary color sensor by performing complementary color-primary color conversion. The same applies to a primary color or complementary color sensor having one or more types of pixels having characteristics other than those described above, such as a W pixel (a white pixel that receives light in almost the entire wavelength band). Although the image sensor 48 of this embodiment is a color sensor, a monochrome sensor without a color filter may also be used.

[0036] The processor device 16 includes a control unit 52, an image acquisition unit 54, an image processing unit 61, a notification unit 65, and a display control unit 66 (see FIG. 2).

[0037] The control unit 52 performs overall control of the endoscope system 10, such as synchronous control of the illumination light irradiation timing and the image capturing timing. When various settings are input using the console 19 or the like, the control unit 52 inputs the settings to each unit of the endoscope system 10, such as the light source control unit 22, the image sensor 48, or the image processing unit 61.

[0038] The image acquisition unit 54 acquires an image of a subject. More specifically, the image acquisition unit 54 acquires an image of a subject using pixels of each color, that is, a RAW image, from the image sensor 48. The RAW image is an image before demosaicing is performed. If the image is an image before demosaicing is performed, the RAW image also includes an image obtained by performing any processing, such as noise reduction processing, on the image acquired from the image sensor 48.

[0039] The image acquisition unit 54 includes a DSP (Digital Signal Processor) 56, a noise reduction unit 58, and a conversion unit 59 in order to perform various processes on the acquired RAW image as necessary to generate an endoscopic image.

[0040] The DSP 56 includes, for example, an offset processing unit, a defect correction processing unit, a demosaic processing unit, an interpolation processing unit, a linear matrix processing unit, and a YC conversion processing unit (none of which are shown in the figure). The DSP 56 uses these units to perform various processes on a RAW image or an image generated using the RAW image.

[0041] The offset processing unit performs offset processing on the RAW image. The offset processing is a process for reducing dark current components from the RAW image and setting an accurate zero level. The offset processing may be referred to as clamp processing. The defect correction processing unit performs defect correction processing on the RAW image. The defect correction processing is a process for correcting or generating pixel values ​​of RAW pixels corresponding to defective pixels of the image sensor 48 when the image sensor 48 includes pixels (defective pixels) having defects due to the manufacturing process or changes over time. The demosaic processing unit performs demosaic processing on RAW images of each color corresponding to a color filter of each color. The demosaic processing is a process for generating pixel values ​​missing in the RAW image due to the arrangement of the color filters by interpolation. The linear matrix processing unit performs linear matrix processing on an endoscopic image generated by assigning one or more RAW images to each RGB color channel. The linear matrix processing is a process for improving the color reproducibility of the endoscopic image. The YC conversion processing unit is a process for converting an endoscopic image generated by assigning one or more RAW images to each RGB color channel into an endoscopic image having a luminance channel Y, a color difference channel Cb, and a color difference channel Cr.

[0042] The noise reduction unit 58 performs noise reduction processing on the endoscopic image having the luminance channel Y, the color difference channel Cb, and the color difference channel Cr, for example, using a moving average method or a median filter method, etc. The conversion unit 59 reconverts the luminance channel Y, the color difference channel Cb, and the color difference channel Cr after the noise reduction processing into an endoscopic image having BGR color channels again.

[0043] The image processing unit 61 performs necessary image processing on the endoscopic image output by the image acquisition unit 54. The image processing unit 61 also performs recognition processing to recognize a subject or a part of a subject having a specific feature using the endoscopic image. Specifically, as shown in Fig. 3, the image processing unit 61 includes an image generation unit 71, a recognition unit 72, a discrimination unit 73, and a setting unit 74.

[0044] The image generating unit 71 acquires an endoscopic image from the image acquiring unit 54 and generates an endoscopic image to be used for display on the monitor 18, etc. For example, the image generating unit 71 acquires from the image acquiring unit 54 a B image obtained by photographing a subject using B pixels, a G image obtained by photographing a subject using G pixels, and an R image obtained by photographing a subject using R pixels, and generates an endoscopic image for display using all or a part of these.

[0045] Furthermore, when generating an endoscopic image for display, the image generating unit 71 performs necessary image processing on the endoscopic image acquired from the image acquiring unit 54 or on an image generated using the endoscopic image acquired from the image acquiring unit 54. The image processing performed by the image generating unit 71 is, for example, an enhancement process for enhancing a subject or a part of a subject. Enhancement means making it possible to obtain information on a specific part by distinguishing it from other tissues or structures. For example, an enhancement process includes processing such as surrounding a part having a specific characteristic with a frame, showing the outline, or changing the color or brightness relative to other parts (for example, normal mucosa, etc.). "Making it possible to obtain information" includes making it possible to recognize the position, shape, color or brightness, and / or size (range), etc. of a specific part, as well as making it possible to obtain biological function information (for example, oxygen saturation or blood vessel density, etc.) on the specific part.

[0046] The recognition unit 72 performs a recognition process to recognize the subject using an image of the subject. "Recognizing the subject" refers to detecting the presence or absence of a part of the subject having a specific characteristic (including the case of detecting the whole of the subject), distinguishing the type or degree of progress of the part of the subject having a specific characteristic (including distinguishing the whole of the subject), and / or obtaining (calculating, etc.) biological function information for a part or the whole of the subject. The part of the subject having a specific characteristic is, for example, a lesion, a candidate for a lesion, or a treatment mark that is a trace of a treatment (hereinafter, referred to as a lesion, etc.). That is, the recognition unit 72 can recognize the presence or absence of a lesion, etc. through the recognition process. Furthermore, the recognition unit 72 can recognize the type or degree of progress of the lesion, etc. through the recognition process. Recognizing the type of the lesion, etc. refers to distinguishing the type of the lesion, etc., such as adenoma, hyperplastic polyp, or cancer, when the lesion, etc. is a polyp. Recognizing the degree of progression of a lesion or the like means, for example, differentiating the stage of cancer if the lesion or the like is cancer, or the NICE (Narrow Band Imaging (NBI) International Colorectal Endoscopic Classification) classification or the JNET (The Japan NBI Expert Team) classification, etc. In this embodiment, the image of the subject is an endoscopic image acquired from the image acquisition unit 54, or an endoscopic image generated by the image generation unit 71. Also, in this embodiment, the recognition unit 72 detects a lesion or the like by recognition processing.

[0047] The recognition unit 72 is, for example, an artificial intelligence (AI) having a learning function. Specifically, the recognition unit 72 is an AI that has been trained using a machine learning algorithm such as a neural network (NN), a convolutional neural network (CNN), adaboost, or a random forest. In addition, since the recognition unit 72 has been trained to perform recognition processing using a specific image, even if the recognition unit 72 can perform recognition processing using other images, the accuracy of the recognition processing result may be low. In this embodiment, the recognition unit 72 is an AI that has been trained to detect lesions, etc. using endoscopic images. "Having a learning function" means being able to learn, and includes having already learned. Note that the recognition unit 72 can be configured to calculate feature amounts from an image and perform detection, etc. using the calculated feature amounts, instead of being configured as an AI.

[0048] In this embodiment, the recognition unit 72 includes a detection unit 76 and a discrimination unit 77. The detection unit 76 is an AI that has learned to detect lesions and the like using endoscopic images. The discrimination unit 77 is an AI that has learned to discriminate the degree of progression of lesions and the like using endoscopic images.

[0049] The recognition unit 72 inputs the result of the recognition process to the image generation unit 71 or the display control unit 66 according to the contents of the recognition process. When the recognition unit 72 inputs the result of the recognition process to the image generation unit 71, the image generation unit 71 generates an endoscopic image for display that reflects the result of the recognition process. When the recognition unit 72 inputs the result of the recognition process to the display control unit 66, the display control unit 66 displays the result of the recognition process on the screen of the monitor 18 together with the endoscopic image acquired from the image generation unit 71. The recognition unit 72 inputs the result of the recognition process to the image generation unit 71, for example, when changing the color of the endoscopic image for display using the value of the biological function information that is the result of the recognition process. The recognition unit 72 inputs the result of the recognition process to the display control unit 66, for example, when indicating the position of a lesion or the like that is the result of the recognition process by displaying a frame superimposed on the endoscopic image. In this embodiment, the recognition unit 72 inputs information on the position of the detected lesion or the like that is the result of the recognition process to the image generation unit 71. Then, the image generating unit 71 generates an endoscopic image for display in which the portion having the lesion or the like is emphasized.

[0050] The discrimination unit 73 discriminates an operation on the subject. An operation on the subject refers to an act of giving some (any) action to the subject. For example, the collection of biological tissue for use in a biopsy, or a procedure related to surgery or treatment such as endoscopic mucosal resection (EMR), the spraying of a liquid on the subject, or the washing of the subject are operations on the subject. The spraying of a liquid on the subject refers to administering a liquid medicine or spraying water on the subject. The spraying of water refers to pouring water on a part or the whole of the subject. The administration of a medicine includes spraying a dye such as indigo carmine on the subject, as well as intravenously injecting a fluorescent medicine such as indocyanine green (ICG) into the subject. The washing of the subject refers to washing away residue, remaining liquid, or substances that hinder observation such as blood attached to the subject with water or air, and exposing the subject to be observed to the endoscope 12. Note that an action that may have an effect on the subject is included in the "operation on the subject" even if the action does not have an effect on the subject as a result. For example, the action of blowing air to clean the subject is considered an operation on the subject even if the subject is not cleaned as a result. Also, an "operation on the subject" includes a series of related actions before and after the action on the subject. For example, the action of protruding forceps from endoscope 12 when performing a biopsy is a series of actions related to the biopsy, even before tissue is collected, and is included in the operation on the subject.

[0051] The determination of an operation on a subject means detecting that an operation is performed on the subject or detecting that an operation has been performed on the subject. The determination of an operation is performed, for example, by detecting the use of a treatment tool, detecting treatment scars (including detection of surgical scars, detection of bleeding caused by treatment, detection of tissue or structure whose color has changed due to spraying of a dye, etc.), or detecting the operation of the water supply button 13b, etc. The determination unit 73 can detect the use of a treatment tool by using an endoscopic image or setting information, etc., to detect the presence or absence of a treatment tool or the type (shape, etc.) of the treatment tool. The determination unit 73 can detect the treatment tool, etc. (including spraying of water) by analyzing an endoscopic image. The determination unit 73 can also detect the operation of the water supply button 13b, etc. by acquiring a signal or setting information from the control unit 52.

[0052] Determining an operation on a subject includes determining whether or not an operation has been performed on the subject, and determining the type of operation on the subject. Determining whether or not an operation has been performed includes detecting an "operation on a subject" to be detected, regardless of the type of operation. Determining the type of operation on a subject means distinguishing the content of the operation (e.g., whether it is a biopsy, administration of medicine, etc.) and determining the operation on the subject. In this embodiment, the determination unit 73 determines whether or not an operation has been performed on the subject.

[0053] The setting unit 74 sets the recognition unit 72 to be enabled or disabled using the determination result of the determination unit 73. The determination result of the determination unit 73 is the presence or absence of an operation on the subject and / or the type of operation on the subject. "Setting the recognition unit 72 to be enabled" means enabling the recognition process performed by the recognition unit 72, setting the recognition unit 72 to execute the recognition process, setting the recognition unit 72 to notify the result of the recognition process, or setting the recognition unit 72 to output the result of the recognition process. "Setting the recognition unit 72 to be disabled" means disabling the recognition process performed by the recognition unit 72, setting the recognition unit 72 not to execute the recognition process, setting the recognition unit 72 not to notify the result of the recognition process, or setting the recognition unit 72 not to output the result of the recognition process. In other words, "setting the recognition unit 72 to be enabled or disabled" means enabling or disabling the recognition process performed by the recognition unit 72, or enabling or disabling the notification of the result of the recognition process (or output of the recognition process result). "Notification of the result of the recognition processing" means making the result of the recognition processing available for a doctor or the like to recognize. "Output of the result of the recognition processing" means that the recognition unit 72 passes the result of the recognition processing to another processing unit such as the image generation unit 71. In this embodiment, the setting unit 74 sets the recognition processing itself performed by the recognition unit 72 to be enabled or disabled.

[0054] When the discrimination unit 73 discriminates whether or not an operation has been performed on the subject, the setting unit 74 sets the recognition unit 72 to be invalid when an operation has been performed on the subject, and sets the recognition unit 72 to be valid when no operation has been performed on the subject. This is because when an operation has been performed on the subject, there is a possibility that the accuracy of the recognition process will decrease, or it is a situation in which the doctor or the like needs to concentrate on the operation.

[0055] When the discrimination unit 73 discriminates the type of operation on the subject, the setting unit 74 disables the recognition unit 72 if there is a specific operation on the subject. The specific operation is an operation that reduces the accuracy of the recognition process performed by the recognition unit 72, or an operation that requires the doctor or the like to concentrate on the operation. Therefore, when the discrimination unit 73 discriminates the type of operation on the subject, the setting unit 74 can set the recognition unit 72 to be active depending on the type of operation. Therefore, it is possible to continue appropriate diagnostic support according to the situation.

[0056] Furthermore, when the recognition unit 72 performs a plurality of types of recognition processing, the setting unit 74 can set each recognition processing to be enabled or disabled. For example, when setting the recognition unit 72 to be enabled or disabled, the setting unit 74 can set the detection unit 76 to be enabled or disabled, and can set the discrimination unit 77 to be enabled or disabled independently of the setting of the detection unit 76 to be enabled or disabled. Of course, the setting unit 74 can set the entire recognition unit 72 to be enabled or disabled, including both the detection unit 76 and the discrimination unit 77. In this embodiment, the setting unit 74 sets the entire recognition unit 72, i.e., the detection unit 76 and the discrimination unit 77, to be enabled or disabled together.

[0057] The notification unit 65, for example, obtains a discrimination result from the discrimination unit 73 (or obtains information related to the setting of the recognition unit 72 from the setting unit 74) and notifies the valid or invalid state of the recognition unit 72. "Notifying" the valid or invalid state of the recognition unit 72 means making the valid or invalid state of the recognition unit 72 known to a user, such as a doctor. For example, the notification unit 65 can notify the valid or invalid state of the recognition unit 72 by displaying a message (character string), a character, a figure, and / or a symbol (including a mark, an icon, an indicator, etc.) on the screen of the monitor 18, not displaying these, or changing these displays. In addition, the notification unit 65 can notify the valid or invalid state of the recognition unit 72 by turning on, off, or blinking a lamp, sound (including voice), vibration of a member having a vibration function, or changes to these. Of course, the notification unit 65 can notify the enabled or disabled state of the recognition unit 72 by combining a character string or the like with lighting of a lamp or the like. In this embodiment, the notification unit 65 displays on the screen of the monitor 18 whether the recognition unit 72 is enabled or disabled.

[0058] The display control unit 66 converts the endoscopic image output by the image processing unit 61 into a format suitable for display and outputs the converted image to the monitor 18. As a result, the monitor 18 displays the endoscopic image. In this embodiment, the notification unit 65 inputs information indicating the enabled or disabled state of the recognition unit 72. As a result, the display control unit 66 displays the information on the screen of the monitor 18.

[0059] The endoscope system 10 configured as above operates as follows to notify the valid or invalid state of the recognition unit 72. As shown in Fig. 4, when an object is photographed using the endoscope 12 (step S111), the image acquisition unit 54 acquires an endoscopic image. Then, the image generation unit 71 performs necessary image processing to generate an endoscopic image for display (step S112).

[0060] Thereafter, or in parallel with the endoscopic image for display, the discrimination unit 73 discriminates an operation on the subject, for example, by using the endoscopic image for display (step S113). If there is no operation on the subject (step S113: YES), the setting unit 74 sets the recognition unit 72 to be enabled (step S114). For this reason, the recognition unit 72 executes a recognition process, for example, on the endoscopic image for display (step S115). In this embodiment, the recognition unit 72 includes a detection unit 76 and a discrimination unit 77, and the setting unit 74 sets these to be enabled or disabled collectively, so that if there is no operation on the subject, the recognition unit 72 executes recognition processes for both detection and discrimination of lesions, etc. After that, the recognition unit 72 outputs the recognition result to the image generation unit 71, and the image generation unit 71 uses the recognition result to generate an endoscopic image for display in which, for example, lesions, etc. are emphasized, and outputs the image to the display control unit 66. As a result, the monitor 18 displays the endoscopic image on the screen. The notification unit 65 also notifies the active state of the recognition unit 72 (step S116). Here, since the setting unit 74 sets the recognition unit 72 to be active, the notification unit 65 displays a character string or the like indicating that the recognition unit 72 is active on the screen of the monitor 18. For example, as shown in FIG. 5, the notification unit 65 displays 122 "AI: ON" on the screen of the monitor 18 displaying an endoscopic image 121. This notifies the active state of the recognition unit 72. Note that the endoscopic image 121 is an image in which no operation has been performed on the subject and no lesion or the like is present.

[0061] On the other hand, when the discrimination unit 73 detects an operation on the subject (step S113: NO), the setting unit 74 sets the recognition unit 72 to be disabled (step S117). Therefore, the recognition unit 72 does not perform either detection or discrimination, and the image generation unit 71 outputs the generated endoscopic image for display to the display control unit 66. As a result, the monitor 18 displays the endoscopic image on the screen. In addition, the notification unit 65 notifies the disabled state of the recognition unit 72 (step S118). For example, as shown in FIG. 6, the notification unit 65 displays a display 127 of "AI: OFF" on the screen of the monitor 18 displaying the endoscopic image 126. This notifies the disabled state of the recognition unit 72. Note that the endoscopic image 126 includes a biopsy forceps 128. Therefore, the recognition unit 72 is disabled.

[0062] As described above, in the endoscope system 10, the discrimination unit 73 discriminates an operation on the subject, and the setting unit 74 sets the recognition unit 72 to be enabled or disabled using the discrimination result of the discrimination unit 73. Therefore, since the endoscope system 10 enables or disables the recognition unit 72 depending on the observation situation, the user, such as a doctor, does not have to manually set the recognition unit 72 to be enabled or disabled during the observation, which is convenient. However, if the recognition unit 72 is only automatically set to be enabled or disabled, the doctor may misinterpret the result of the recognition process or the result of not performing the recognition process. For example, if there is a preconceived notion that the recognition unit 72 is enabled, when the recognition unit 72 is automatically disabled and the result of the recognition process is no longer displayed, the doctor may misinterpret that the result of the recognition unit 72 performing the recognition process indicates that there is no lesion, even if there is a lesion. Therefore, in the endoscope system 10, the notification unit 65 notifies the user of the enabled or disabled state of the recognition unit 72. Therefore, by looking at the "AI: ON" display 122 or the "AI: OFF" display 127, it is clear whether the recognition unit 72 is performing recognition processing or not, so that the above-mentioned misrecognition can be prevented. As a result, misdiagnosis can be prevented.

[0063] [Second embodiment] When the setting unit 74 sets each recognition process to be enabled or disabled, the notification unit 65 can notify the enabled or disabled state of each recognition process. For example, when the recognition unit 72 has the detection unit 76 and the discrimination unit 77, and the setting unit 74 sets each of the detection unit 76 and the discrimination unit 77 to be enabled or disabled, as shown in Fig. 7, the notification unit 65 can display an indication 223 indicating the enabled or disabled state of the detection unit 76 and an indication 224 indicating the enabled or disabled state of the discrimination unit 77 in a display field 222 indicating the enabled or disabled state of the recognition unit 72, and can notify these states. For example, the indication 223 of "detection: ON" is an indication indicating that the detection unit 76 is enabled, and the indication of "detection: OFF" (not shown) is an indication indicating that the detection unit 76 is disabled. Similarly, the display 224 of "Discrimination: ON" indicates that the discriminating unit 77 is enabled, and the display of "Discrimination: OFF" (not shown) indicates that the discriminating unit 77 is disabled.

[0064] As described above, when the setting unit 74 sets each recognition process to be enabled or disabled, the recognition process appropriate for the observation situation can be enabled, and the other recognition processes can be disabled. This allows the endoscope system 10 to provide optimal diagnostic support. Furthermore, the notification unit 65 notifies the enabled or disabled state of each recognition process, so that the doctor or the like can accurately grasp whether each recognition process is enabled or disabled. This prevents misidentification of the result of a recognition process or the result of not performing a recognition process, and as a result, prevents misdiagnosis.

[0065] In the above second embodiment, an example is shown in which both the detection unit 76 and the classification unit 77 are enabled. However, as shown in FIG. 8, depending on the discrimination situation of the discrimination unit 73, it may be appropriate to enable only the detection unit 76, it may be appropriate to enable only the classification unit 77, or it may be appropriate to disable both the detection unit 76 and the classification unit 77.

[0066] "Discrimination situation 1" is a situation in which it is appropriate to activate both the detection unit 76 and the discrimination unit 77, such as when there is no operation on the subject, or when the subject is washed with water, etc. These are situations in which it is highly likely that detection and discrimination of lesions, etc. are required.

[0067] "Discrimination situation 2" is a situation in which it is appropriate to enable the detection unit 76 and disable the discrimination unit 77. For example, when performing a biopsy or surgery, bleeding from the subject is the discrimination target (detection target) of the discrimination unit 73, that is, detection of bleeding is necessary to confirm hemostasis, but discrimination is not necessary because surgery has already begun. The discrimination unit 73 can discriminate the above situation, for example, by the type of operation on the subject.

[0068] "Discrimination situation 3" is a situation in which it is appropriate to disable the detection unit 76 and enable the discrimination unit 77. For example, this is a case where a lesion or the like has already been found, so there is no need to detect the lesion or the like again, and more detailed diagnostic support is desired from the endoscope system 10 in diagnosing the detected lesion or the like. Specifically, this is a case where a dye is sprayed on the subject and the degree of progression of the lesion or the like is desired to be discriminated. The discrimination unit 73 can discriminate the above situation, for example, by the type of operation on the subject.

[0069] "Discrimination status 4" is a situation in which it is appropriate to disable both the detection unit 76 and the discrimination unit 77. For example, this is the case when a biopsy or surgery is being performed after detection and discrimination of a lesion or the like has been completed and it is necessary to concentrate on the operation. The discrimination unit 73 can distinguish the above situation depending on the type of operation. When the discrimination unit 73 detects bleeding from the subject, for example, the discrimination status becomes 3.

[0070] In the second embodiment, the validity or invalidity of the detection unit 76 and the discrimination unit 77 is represented by "ON" or "OFF", but the validity or invalidity of the detection unit 76 and the discrimination unit 77 can be represented by other indications. For example, instead of the indication of "OFF", an indication such as "not supported" may be used. Even if the detection unit 76 or the discrimination unit 77 is in an enabled state, if the accuracy (precision or reliability) of the recognition process is low, an indication such as "Unsupported" (not supported) can be used instead of the indication of "ON". For example, in the discrimination situation 3, if the discrimination unit 77 is not compatible with discrimination of a subject on which a dye agent has been sprayed, the discrimination accuracy decreases, so that "detection: OFF" and "discrimination: Unsupported" can be displayed. Changes in these display modes can be made in the same manner in the first embodiment.

[0071] [Third embodiment] In the above first and second embodiments, the notification unit 65 notifies the valid or invalid state of the recognition unit 72, but the notification unit 65 can further notify the presence or absence of an operation or the type of operation determined by the determination unit 73. That is, the notification unit 65 can notify the determination result of the determination unit 73 in addition to the valid or invalid state of the recognition unit 72.

[0072] For example, when the discrimination unit 73 detects an operation on the subject, the notification unit 65 can notify the detection of the operation on the subject by a display 301 of "Actions" in addition to notifying whether the recognition unit 72 is enabled or disabled, as shown in Fig. 9. Note that the endoscopic image 310 is an endoscopic image in which a dye agent 311 is sprayed on the subject.

[0073] As described above, when the notification unit 65 notifies the presence or absence of an operation on the subject, the doctor or the like can recognize the cause of the validity or invalidity of the recognition unit 72. As a result, it is possible to more reliably prevent misidentification of the result of the recognition process or the result of not performing the recognition process, and to prevent misdiagnosis.

[0074] Furthermore, the notification unit 65 can notify the type of operation determined by the discrimination unit 73. For example, as shown in Fig. 10, the notification unit 65 can notify the type of operation determined by the discrimination unit 73 by displaying a list of "Biopsy" representing a biopsy, "Chromo" representing the spraying of a dye, and "Resection" representing an endoscopic resection in an operation type display field 313 and adding a check mark ("R") to the type of operation determined by the discrimination unit 73 ("Chromo" in Fig. 10).

[0075] As described above, when the discrimination unit 73 discriminates the type of operation, if the notification unit 65 notifies the type of operation discriminated by the discrimination unit 73, the doctor or the like can recognize in detail the cause of the validity or invalidity of the recognition unit 72. As a result, it is possible to more reliably prevent misidentification of the result of the recognition process or the result of not performing the recognition process, and to prevent misdiagnosis.

[0076] The notification of the presence or absence of an operation on a subject or the type of operation in the above-mentioned third embodiment can also be applied to the case where each recognition process is set to be enabled or disabled and notified as in the second embodiment. In this case, the display field 222 (see FIG. 7) showing the enabled or disabled state of the recognition unit 72 and the "Actions" display 301 or the operation type display field 313 may be displayed on the monitor 18.

[0077] In the third embodiment, the presence or absence of an operation on the subject or the type of the operation can be notified by a display other than a character or a character string. For example, as shown in Fig. 11, an icon 331 indicating a biopsy, an icon 332 indicating the spraying of a dye agent, and an icon 333 indicating an endoscopic resection are displayed in a list in an operation type display field 313, and the type of operation determined by the determination unit 73 can be notified by coloring the icon corresponding to the type of operation determined by the determination unit 73. In Fig. 11, since the endoscopic image 321 includes the forceps 128 for biopsy, the icon 331 indicating the biopsy is colored.

[0078] When the type of operation is shown by an icon as described above, only icons corresponding to the type of operation determined by the determination unit 73 can be displayed in the operation type display field 313, as shown in Fig. 12. The same is true when the type of operation on the subject is displayed as a character string in the operation type display field 313 (see Fig. 10), and only character strings or the like indicating the type of operation determined by the determination unit 73 can be displayed in the operation type display field 313.

[0079] In the first, second, and third embodiments, the setting unit 74 sets the recognition process performed by the recognition unit 72 to be enabled or disabled, but instead, the setting unit 74 can set the recognition unit 72 to be enabled or disabled by setting the notification of the result of the recognition process to be enabled or disabled. In this case, when setting the notification of the result of the recognition process to be disabled, the setting unit 74 stops outputting the recognition result to the image generation unit 71, etc., while leaving the detection unit 76 and the discrimination unit 77 set to be enabled, so that the recognition result is not known to the doctor, etc. As a result, the recognition unit 72 can be set to be enabled or disabled more simply and quickly than when the recognition process itself is set to be enabled or disabled.

[0080] In the first, second, and third embodiments, the notification unit 65 notifies the enabled or disabled state of the recognition unit 72 by displaying on the screen of the monitor 18. However, when the enabled or disabled state of the recognition unit 72 is notified in a manner other than displaying on the screen of the monitor 18, the processor device 16 may include a notification device 171 as shown in FIG. 13. The notification device 171 may be a speaker that emits sound or voice, an indicator constituted by a light-emitting element such as an LED, or a vibration element such as a motor or a piezoelectric element. In addition, the notification unit 65 may use an element or the like provided in the endoscope system 10 as the notification device 171. The notification device 171 may be provided in a device other than the processor device 16, i.e., in the endoscope 12 or the light source device 14.

[0081] 14, the recognition unit 72, the discrimination unit 73, the setting unit 74, and / or the notification unit 65 can be provided in a medical image processing device 901 that communicates with the processor device 16 and cooperates with the endoscope system 10. As shown in FIG. 15, the recognition unit 72, the discrimination unit 73, the setting unit 74, and / or the notification unit 65 can be provided in a diagnosis support device 911 that acquires a RAW image captured by the endoscope 12 directly from the endoscope system 10 (including a device that does not have the notification unit 65, etc.) or indirectly from a PACS (Picture Archiving and Communication Systems) 910. As shown in FIG. 16, the recognition unit 72, the discrimination unit 73, the setting unit 74, and / or the notification unit 65 can be provided in a medical work support device 930 that is connected to various inspection devices, such as a first inspection device 921, a second inspection device 922, ..., a Kth inspection device 923, including the endoscope system 10, via a network 926.

[0082] That is, the present invention includes a medical image processing device and an operation method thereof, the medical image processing device and an operation method thereof include an image acquisition unit that acquires an image of a subject, a recognition unit that performs a recognition process to recognize the subject using the image, a discrimination unit that discriminates an operation on the subject, a setting unit that sets the recognition unit to be enabled or disabled using the discrimination result of the discrimination unit, and a notification unit that notifies the enabled or disabled state of the recognition unit. The present invention also includes a diagnosis support device and an operation method thereof, the medical image processing device and an operation method thereof include an image acquisition unit that acquires an image of a subject, a recognition unit that performs a recognition process to recognize the subject using the image, a discrimination unit that discriminates an operation on the subject, a setting unit that sets the recognition unit to be enabled or disabled using the discrimination result of the discrimination unit, and a notification unit that notifies the enabled or disabled state of the recognition unit. Similarly, the present invention includes a medical service support device and an operation method thereof, the medical image processing device and an operation method thereof include an image acquisition unit that acquires an image of a subject, a recognition unit that performs a recognition process to recognize the subject using the image, a discrimination unit that discriminates an operation on the subject, a setting unit that sets the recognition unit to be enabled or disabled using the discrimination result of the discrimination unit, and a notification unit that notifies the enabled or disabled state of the recognition unit.

[0083] The present invention also includes an operation method for an endoscope system including the steps of: an image acquisition unit acquiring an image of a subject; a recognition unit performing a recognition process to recognize the subject using the image; a setting unit determining an operation on the subject, and setting the recognition unit to be enabled or disabled using a determination result of the determination unit; and a notification unit notifying the enabled or disabled state of the recognition unit. The present invention also includes a processor device and an operation method thereof including an image acquisition unit acquiring an image of a subject, a recognition unit performing a recognition process to recognize the subject using the image, a determination unit determining an operation on the subject, a setting unit setting the recognition unit to be enabled or disabled using a determination result of the determination unit, and a notification unit notifying the enabled or disabled state of the recognition unit.

[0084] It should be noted that a capsule endoscope can be used as the endoscope 12. In this case, the light source device 14 and a part of the processor device 16 can be mounted on the capsule endoscope.

[0085] In the above embodiment, the hardware structure of the processing unit that executes various processes such as the recognition unit 72, the discrimination unit 73, the setting unit 74, or the notification unit 65 is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a GPU (Graphical Processing Unit), a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing various processes.

[0086] One processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same type or different types (for example, multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU, etc.). Also, multiple processing units may be configured with one processor. As an example of configuring multiple processing units with one processor, first, as represented by a computer such as a client or a server, there is a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a system on chip (SoC), there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0087] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit in the form of a combination of circuit elements such as semiconductor elements.

[0088] The present invention can be used in systems or devices that acquire medical images (including videos) other than endoscopic images, in addition to endoscope systems, processor devices, and other related devices that acquire endoscopic images, etc. For example, the present invention can be applied to ultrasound examination devices, X-ray imaging devices (including CT (Computed Tomography) examination devices and mammography devices, etc.), MRI (magnetic resonance imaging) devices, etc. [Explanation of symbols]

[0089] 10 Endoscope System 12 Endoscopy 12a Insertion part 12b Operation section 12c Curved section 12d Tip 12e Angle knob 12f Treatment tool insertion port 13a Zoom operation section 13b Water supply button 14 Light source device 16 Processor Unit 17 Water tank 18 Monitors 19 Console 20 Light source section 22 Light source control unit 30a illumination optical system 30b Shooting optical system 45 Lighting Lens 46 Objective Lens 47 Zoom Lens 48 Image Sensor 52 Control section 54 Image acquisition unit 56 DSP 58 Noise reduction section 59 Conversion section 61 Image processing section 65 Notification Department 66 Display control unit 71 Image Generation Unit 72 Recognition part 73 Discrimination part 74 Setting section 76 Detection unit 77 Identification Department 121, 126, 310, 321 Endoscopic images 122, 127, 223, 224, 301 display 128 Forceps 171 Notification Device 222 Display field 311 Dyeing agents 313 Type display field 331, 332, 333 Icons 901 Medical image processing device 910 PACS 911 Diagnostic Support Device 921 First Inspection Device 922 Second Inspection Device 923 Kth Inspection Device 926 Network 930 Medical Business Support Equipment S111~S118 Operation steps

Claims

1. A recognition unit that performs a recognition process to recognize a subject using an image of the subject; A determination unit that determines an operation performed on the subject; a setting unit that sets the recognition unit to be valid or invalid using a determination result of the determination unit; a notification unit that notifies a valid or invalid state of the recognition unit; Equipped with The image processing device captures an endoscopic image of the subject.

2. The image processing device of claim 1, wherein recognizing the subject means detecting whether or not there is a part of the subject that has a specific characteristic, identifying the type or degree of progression of the part of the subject that has a specific characteristic, or obtaining biofunctional information about part or all of the subject.

3. The image processing device according to claim 1 , wherein the recognition unit includes an artificial intelligence.

4. 4. The image processing apparatus according to claim 3, wherein the notification is a notification as to whether the recognition process using the artificial intelligence is valid or invalid.

5. 5. The image processing device according to claim 1, wherein the recognition unit is capable of at least one of detecting the presence or absence of a part of the subject having a specific characteristic, and identifying the type or degree of progression of the part of the subject having the specific characteristic.

6. The image processing device according to claim 5 , wherein the recognition unit is capable of detecting the presence or absence of a part of the subject having a specific characteristic, and distinguishing the type or degree of progression of the part of the subject having the specific characteristic.

7. The image processing device according to claim 6 , wherein the discrimination section validates both the detection and the discrimination when it determines that the operation on the subject has not been performed or that the subject has been cleaned.

8. 8. The image processing apparatus according to claim 6, wherein the setting section independently sets the detection and the discrimination of the recognition section to be valid or invalid.

9. 9. The image processing apparatus according to claim 6, wherein the discrimination section performs the detection and does not perform the discrimination when the discrimination target is bleeding from the subject.

10. 8. The image processing apparatus according to claim 6, wherein the setting section sets the detection and the discrimination to be valid or invalid together.

11. 9. The image processing apparatus according to claim 8, wherein the discrimination section, when discriminating that a pigment has been sprayed onto the subject, performs the discrimination without performing the detection.

12. An image processing device as claimed in any one of claims 1 to 11, wherein the notification unit displays information indicating the enabled or disabled state of the recognition unit, the result of the determination by the determination unit, the type of operation determined by the determination unit, and the image all at once on a monitor.

13. 13. The image processing device according to claim 1, wherein the determination unit determines the presence or absence of a treatment tool or the type of the treatment tool.

14. 13. The image processing device according to claim 1, wherein the operation discriminated by the discrimination unit is an operation that reduces the accuracy of the recognition process or an operation that requires a user to concentrate on the operation.

15. 13. The image processing device according to claim 1, wherein the operation discriminated by the discriminator is using a treatment tool, changing a color of the subject, or spraying a liquid onto the subject.

16. 13. The image processing apparatus according to claim 1, wherein the operation includes taking a biological tissue, performing an endoscopic resection or treatment, spraying a liquid onto a subject, or washing the subject.

17. 17. The image processing apparatus according to claim 1, further comprising: a notification unit that notifies the type of the operation determined by the determination unit.

18. 18. The image processing apparatus according to claim 1, wherein the recognition unit is set to be invalid by not executing the recognition process or not notifying the result of the recognition process.

19. 18. The image processing device according to claim 1, wherein the setting unit sets the recognition unit to be disabled when an operation is performed on the subject, and sets the recognition unit to be enabled when no operation is performed on the subject.

20. The image processing device according to claim 1 , wherein the setting unit sets the recognition unit to be invalid when a specific operation is performed on the subject.

21. 21. The image processing apparatus according to claim 1, wherein the notification unit notifies a result of the discrimination made by the discrimination unit in addition to notifying whether the recognition unit is valid or invalid.

22. 22. The image processing apparatus according to claim 1, wherein the notification unit notifies the valid or invalid state of the recognition unit in an area different from an area in which the image is displayed.

23. 23. The image processing apparatus according to claim 1, wherein when the recognition unit performs a plurality of types of recognition processing, the setting unit sets each of the recognition processing to be valid or invalid.

24. An image processing apparatus according to any one of claims 1 to 23, A light source device that generates illumination light; an endoscope for capturing an image of the subject illuminated with the illumination light; a processor device that controls the light source device and the endoscope to generate the image, The endoscope system includes an image processing device that acquires the image from the processor device.

25. A function of performing a recognition process for recognizing a subject by using an image of the subject in a recognition unit; A function of determining an operation performed on the subject by a determination unit; a function of setting the recognition unit to be valid or invalid using a determination result of the determination unit; A function of notifying a valid or invalid state of the recognition unit; on the computer, The image processing program is an endoscopic image of the photographed subject.

Images