Endoscope system

By determining the operation status of the subject in the endoscopic system, setting the validity of the recognition unit and reporting the status, the problem of inaccurate identification processing caused by the inability to obtain the medical image that performs functions is solved, preventing misidentification and improving diagnosis accuracy.

CN112566538BActive Publication Date: 2024-11-12FUJIFILM CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980053740.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-08-17
Filing Date
2019-08-16
Publication Date
2024-11-12
Estimated Expiration
2039-08-16

AI Technical Summary

Technical Problem

In the endoscopic system, when the medical images that perform functions cannot be obtained, the results of the recognition processing are inaccurate, which leads to the doctor to misunderstand the results of the other processing or fail to perform the identification processing, affecting diagnosis and observation.

Method used

An endoscope system is designed, including an image acquisition, recognition, discrimination, setting and reporting unit. By determining the operating state of the subject, the validity of the recognition unit is set, and the state of knowledge processing is reported to prevent Misconception.

Benefits of technology

It effectively prevents the misidentification of the identification and processing results, ensures that the doctor correctly recognizes the status of the identification and processing, and improves the accuracy of diagnosis and operation safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112566538B_ABST
    Figure CN112566538B_ABST
Patent Text Reader

Abstract

The present invention provides an endoscope system that prevents misidentification of results of recognition processing and results of non-recognition processing. The endoscope system (10) comprises: an image acquisition unit (54) that acquires an image obtained by photographing a subject; a recognition unit (72) that performs recognition processing to recognize the subject using the image; a determination unit (73) that determines an operation on the subject; a setting unit (74) that uses the determination result of the determination unit (73) to set the recognition unit (72) to be valid or invalid; and a notification unit (65) that reports the valid or invalid state of the recognition unit (72).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an endoscope system that performs recognition processing of a subject using an image captured by an endoscope. Background Art

[0002] In the medical field, endoscope systems including a light source device, an endoscope, and a processor device are becoming popular. The light source device generates illumination light. The endoscope uses an image sensor to capture a subject. The processor device generates an image and performs other image processing.

[0003] An endoscope system not only photographs a subject for observation, but sometimes also has additional functions. For example, an endoscope system having an "input mode" for operating the position and orientation of a treatment instrument using a touch panel is known (Patent Document 1). In the endoscope system of Patent Document 1, the state of the above-mentioned input mode is displayed on the screen. In addition, an endoscope system having the following functions is known: an image showing the bending state of the distal end of the endoscope, a character string informing that calibration has been completed, and a notification of system abnormalities are displayed on the screen (Patent Document 2). In addition, an endoscope system that displays the on or off state of a foot switch on the screen is known (Patent Document 3), an endoscope system that displays a mark indicating that air is being supplied on the screen (Patent Document 4), and an endoscope system that displays the on or off state of a video recording on the screen (Patent Document 5).

[0004] In addition, there are known endoscope systems that detect treatment instruments and use the detection results. For example, there are known endoscope systems that use the detection results of treatment instruments to determine the correction strength of images (Patent Document 6). There are known endoscope systems that stop the autofocus function when a treatment instrument is detected (Patent Document 7).

[0005] In addition, in recent years, there has been known an endoscope system that assists diagnosis by obtaining biological function information through calculation or the like using an image obtained by photographing a subject (Patent Document 8).

[0006] Previous technical literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Publication No. 2008-245840

[0009] Patent Document 2: Japanese Patent Application Publication No. 2007-330405

[0010] Patent Document 3: Japanese Patent Application Publication No. 2005-073799

[0011] Patent Document 4: Japanese Patent Application Laid-Open No. 3-277340

[0012] Patent Document 5: Japanese Patent Application Laid-Open No. 63-259614

[0013] Patent Document 6: Japanese Patent Application Publication No. 2015-96237

[0014] Patent Document 7: Japanese Patent Application Publication No. 2013-150658

[0015] Patent Document 8: Japanese Patent Application Publication No. 2018-51364 Summary of the invention

[0016] Technical issues to be solved by the invention

[0017] In the case of obtaining information to assist diagnosis (hereinafter referred to as auxiliary diagnosis information) by recognizing an object or a part of an object having specific features using a medical image such as an image captured by an endoscope (hereinafter referred to as an endoscopic image), it is necessary to use a medical image that enables recognition processing to function by being captured under specific conditions. This is because when a medical image is used in which recognition processing cannot function, even if a result of recognition processing is obtained, the result of recognition processing is sometimes inaccurate.

[0018] However, it is not always possible to obtain medical images in which the recognition process can function. Therefore, sometimes inaccurate recognition process results or diagnostic auxiliary information obtained by calculation using inaccurate recognition process results may hinder the observation or diagnosis of the subject. For example, in an endoscope system, when a treatment instrument is reflected in the endoscopic image, the accuracy of the recognition process may be reduced.

[0019] In order to avoid providing inaccurate results of recognition processing as described above, it is considered that recognition processing is not performed when a medical image on which recognition processing can function cannot be obtained. However, when recognition processing is not performed at all, doctors and the like who are users may mistakenly recognize the result of recognition processing or the result of no recognition processing. For example, in the case where a device that performs recognition processing to detect candidate lesions is being used, when recognition processing is not performed at all, doctors and the like who use the device may sometimes mistakenly believe that the result of recognition processing is that the candidate lesion was not detected. That is, sometimes doctors and the like cannot determine whether the result of recognition processing is not displayed because recognition processing is not performed, or whether the result of recognition processing is not displayed because the result of recognition processing is that there is no object to be recognized.

[0020] In addition, when a surgery on a subject is started, or when a pigment is dispersed and a close observation is started, in order not to hinder the operation of the operator, the recognition process or the display of the result of the recognition process is sometimes automatically disabled. However, in this case, as in the above, doctors sometimes mistake the result of the recognition process or the result of the non-recognition process. In addition, even if the doctor does not explicitly disable the recognition function, there is also a problem that the doctor's concentration is sometimes reduced when the recognition function is automatically disabled.

[0021] An object of the present invention is to provide an endoscope system that prevents at least misidentification of a result that has been subjected to a recognition process and a result that has not been subjected to a recognition process.

[0022] Means for solving technical problems

[0023] The endoscope system of the present invention comprises: an image acquisition unit, which acquires an image obtained by photographing a subject; a recognition unit, which performs recognition processing to recognize the subject using the image; a discrimination unit, which discriminates the operation on the subject; a setting unit, which uses the discrimination result of the discrimination unit to set the discrimination unit to be valid or invalid; and a notification unit, which reports the valid or invalid state of the discrimination unit.

[0024] Preferably, the determination unit determines whether or not there is an operation on the subject.

[0025] Preferably, the determination unit determines the type of operation on the subject.

[0026] Preferably, the operation determined by the determination unit is use of a treatment tool.

[0027] Preferably, the operation determined by the determination unit is the spreading of liquid on the subject.

[0028] Preferably, the setting unit sets the recognition unit to be invalid when the subject is operated, and sets the recognition unit to be valid when the subject is not operated.

[0029] Preferably, when a specific operation is performed on the subject, the setting unit sets the recognition unit to be invalid.

[0030] Preferably, the setting unit sets the recognition processing performed by the recognition unit to be effective or ineffective.

[0031] Preferably, the setting unit sets notification of the result of the recognition process to be effective or ineffective.

[0032] Preferably, the notification unit not only notifies the valid or invalid state of the identification unit but also notifies the determination result of the determination unit.

[0033] Preferably, the recognition unit recognizes whether the subject has a lesion or a candidate lesion, or the type or development degree of the lesion or the candidate lesion of the subject.

[0034] Preferably, when the recognition unit performs a plurality of recognition processes, the setting unit sets validity or invalidity for each recognition process.

[0035] Effects of the Invention

[0036] According to the present invention, it is possible to provide an endoscope system that prevents misidentification of a result that has been subjected to recognition processing and a result that has not been subjected to recognition processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is an external view of the endoscope system.

[0038] Figure 2 is a block diagram of an endoscope system.

[0039] Figure 3 This is a block diagram of the image processing unit.

[0040] Figure 4 This is a flowchart showing the operation mode of the endoscope system.

[0041] Figure 5 This is an example of the display when the recognition unit is valid.

[0042] Figure 6 This is an example of display when the recognition unit is invalid.

[0043] Figure 7 This is a display example of the second embodiment.

[0044] Figure 8 It is a list of action states for detection and identification.

[0045] Fig. 9 This is a display example of the third embodiment.

[0046] Fig.10 This is an example of the display of operation contents.

[0047] Fig.11 This is an example of the display of operation contents.

[0048] Fig.12 This is an example of the display of operation contents.

[0049] Fig.13 A block diagram of a processor device for reporting the validity or invalidity of another part by a method other than displaying it on a monitor.

[0050] Fig.14 This is an explanatory diagram of a medical image processing device.

[0051] Fig.15 It is an explanatory diagram showing a diagnosis support device.

[0052] Fig.16This is an explanatory diagram showing a medical work support device. DETAILED DESCRIPTION

[0053] [First embodiment]

[0054] like Figure 1 As shown, the endoscope system 10 (endoscope device) includes an endoscope 12, a light source device 14, a processor device 16, a monitor 18, and a console 19. The endoscope 12 photographs a subject. The light source device 14 generates illumination light. The processor device 16 performs system control and image processing of the endoscope system 10. The monitor 18 is a display unit that displays images photographed by the endoscope 12. The console 19 is an input device for inputting settings to the processor device 16.

[0055] The endoscope 12 includes: an insertion portion 12a inserted into the subject; an operating portion 12b provided at the base end of the insertion portion 12a; a bending portion 12c provided at the tip end side of the insertion portion 12a; and a tip end 12d. The bending portion 12c is bent by operating the angle button 12e of the operating portion 12b. As a result, the tip end 12d faces the desired direction. In addition, the operating portion 12b is provided with a treatment instrument insertion port 12f, a zoom operating portion 13a, and a water supply button 13b in addition to the angle button 12e. The treatment instrument insertion port 12f is an entrance for inserting treatment instruments such as biopsy forceps, a snare, or an electric scalpel. The treatment instrument inserted into the treatment instrument insertion port 12f protrudes from the tip end 12d. By operating the zoom operating portion 13a, the subject can be enlarged or reduced for photography. In addition, by operating the water supply button 13b, liquid such as water stored in the water supply tank 17 is ejected from the top end portion 12d.

[0056] like Figure 2 As shown, the light source device 14 includes a light source unit 20 that emits illumination light and a light source control unit 22 that controls the operation of the light source unit 20 .

[0057] The light source unit 20 emits illumination light for illuminating the subject. The emission of illumination light includes emission of excitation light for emitting illumination light. The light source unit 20 includes, for example, a light source such as a laser diode (hereinafter referred to as LD), an LED (Light Emitting Diode), a xenon lamp, or a halogen lamp, and emits at least white illumination light or excitation light for emitting white illumination light. The white color includes so-called pseudo-white color that is substantially equivalent to white in the photography of the subject using the endoscope 12. The light source unit 20 includes, as necessary, a fluorescent body that emits light when irradiated with excitation light, or an optical filter that adjusts the wavelength band, spectral spectrum, or light amount of the illumination light or excitation light. In addition, the light source unit 20 can emit light having a specific wavelength band required for capturing an image for calculating biological information such as the oxygen saturation of hemoglobin contained in the subject.

[0058] In the present embodiment, the light source unit 20 has four-color LEDs, namely, V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d. V-LED 20a emits purple light VL with a central wavelength of 405nm and a wavelength band of 380 to 420nm. B-LED 20b emits blue light BL with a central wavelength of 460nm and a wavelength band of 420 to 500nm. G-LED 20c emits green light GL with a wavelength band of 480 to 600nm. R-LED 20d emits red light RL with a central wavelength of 620 to 630nm and a wavelength band of 600 to 650nm. In addition, the central wavelengths of V-LED 20a and B-LED 20b have a width of about ±20nm, preferably about ±5nm to about ±10nm.

[0059] The light source control unit 22 controls the timing of lighting up or extinguishing or shielding each light source constituting the light source unit 20, as well as the amount of light emitted, etc. As a result, the light source unit 20 can emit a variety of illumination lights with different spectral spectra. In the present embodiment, the light source control unit 22 controls the lighting up or extinguishing of each LED 20a to 20d, the amount of light emitted when lit, the insertion and removal of the filter, etc., by inputting independent control signals, and adjusts the spectral spectrum of the illumination light. Thus, the light source unit 20 emits white light. In addition, the light source unit 20 can at least emit illumination light consisting of narrow-band purple light. The so-called "narrow band" refers to a substantially single wavelength band in relation to the characteristics of the subject and / or the spectral characteristics of the color filter possessed by the image sensor 48. For example, based on the central wavelength, when the wavelength band is, for example, about ±20nm or less (preferably about ±10nm or less), the light is a narrow band.

[0060] An illumination optical system 30a and an imaging optical system 30b are provided in the distal end portion 12d of the endoscope 12. Illumination light from the light source unit 20 is transmitted to the illumination optical system 30a through the light guide 41. The illumination optical system 30a has an illumination lens 45 through which illumination light is emitted toward a subject.

[0061] The photographic optical system 30b includes an objective lens 46, a zoom lens 47, and an image sensor 48. The image sensor 48 photographs the subject using reflected light of illumination light returned from the subject via the objective lens 46 and the zoom lens 47 (in addition to reflected light, scattered light, fluorescence emitted by the subject, or fluorescence caused by a drug administered to the subject, etc.). The zoom lens 47 moves by operating the zoom operation unit 13a to enlarge or reduce the subject image.

[0062] The image sensor 48 has a color filter of one color among a plurality of color filters in each pixel. In the present embodiment, the image sensor 48 is a color sensor having a color filter of a primary color system. Specifically, the image sensor 48 has an R pixel having a red color filter (R color filter), a G pixel having a green color filter (G color filter), and a B pixel having a blue color filter (B color filter).

[0063] In addition, as the image sensor 48, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor can be used. In addition, the image sensor 48 of the present embodiment is a primary color system color sensor, but a complementary color system color sensor can also be used. The complementary color system color sensor has, for example, a cyan pixel provided with a cyan filter, a magenta pixel provided with a magenta filter, a yellow pixel provided with a yellow filter, and a green pixel provided with a green filter. In the case of using a complementary color system color sensor, if the image obtained from the pixels of the above-mentioned colors is subjected to a complementary color-primary color conversion, it can be converted into an image that is the same as the image obtained by the primary color system color sensor. The same is true in the case of pixels having one or more W pixels (white pixels that receive light of approximately the entire wavelength band) and other characteristics other than the above in the primary color system or complementary color system sensor. In addition, the image sensor 48 of the present embodiment is a color sensor, but a monochrome sensor without a color filter can also be used.

[0064] The processor device 16 includes a control unit 52, an image acquisition unit 54, an image processing unit 61, a notification unit 65, and a display control unit 66 (see Figure 2 ).

[0065] The control unit 52 performs overall control of the endoscope system 10, such as synchronous control of the illumination light irradiation timing and the imaging timing. When inputting various settings using the console 19 or the like, the control unit 52 inputs the settings to various parts of the endoscope system 10, such as the light source control unit 22, the image sensor 48, or the image processing unit 61.

[0066] The image acquisition unit 54 acquires an image obtained by photographing a subject. More specifically, the image acquisition unit 54 acquires an image obtained by photographing a subject using pixels of each color, that is, a RAW image, from the image sensor 48. In addition, the RAW image is an image before de-mosaicing is performed. If it is an image before de-mosaicing, an image obtained by performing any processing such as noise reduction processing on the image acquired from the image sensor 48 is also included in the RAW image.

[0067] The image acquisition unit 54 includes a DSP (Digital Signal Processor) 56 , a noise reduction unit 58 , and a conversion unit 59 in order to generate an endoscopic image by performing various processes on the acquired RAW image as necessary.

[0068] The DSP 56 includes, for example, an offset processing unit, a defect correction processing unit, a demosaic processing unit, an interpolation processing unit, a linear matrix processing unit, and a YC conversion processing unit (none of which are shown in the figure). The DSP 56 uses these to perform various processes on the RAW image or an image generated using the RAW image.

[0069] The offset processing unit performs offset processing on the RAW image. Offset processing is a process of reducing the dark current component from the RAW image and setting an accurate 0 level. Offset processing is sometimes called clamping processing. The defect correction processing unit performs defect correction processing on the RAW image. Defect correction processing is a process of correcting or generating pixel values ​​of RAW pixels corresponding to defective pixels of the image sensor 48 when the image sensor 48 contains pixels (defective pixels) having defects caused by the manufacturing process or changes over time. The demosaicing processing unit performs demosaicing processing on the RAW images of each color corresponding to the color filters of each color. Demosaicing processing is a process of generating pixel values ​​that are missing in the RAW image due to the arrangement of the color filters by interpolation. The linear matrix processing unit performs linear matrix processing on the endoscopic image generated by assigning one or more RAW images to channels of each RGB color. Linear matrix processing is a process of improving the color reproducibility of the endoscopic image. The YC conversion processing unit is a process for converting an endoscopic image generated by assigning one or more RAW images to channels of respective RGB colors into an endoscopic image having a luminance channel Y, a color difference channel Cb, and a color difference channel Cr.

[0070] The noise reduction unit 58 performs noise reduction processing on the endoscopic image having the brightness channel Y, the color difference channel Cb, and the color difference channel Cr, for example, using a moving average method or a median filter method, etc. The conversion unit 59 converts the brightness channel Y, the color difference channel Cb, and the color difference channel Cr after the noise reduction processing into an endoscopic image having channels of RGB colors again.

[0071] The image processing unit 61 performs necessary image processing on the endoscopic image output by the image acquisition unit 54. In addition, the image processing unit 61 uses the endoscopic image to perform recognition processing for recognizing a subject or a part of a subject having specific features. Specifically, Figure 3 As shown, the image processing unit 61 includes an image generating unit 71 , a recognizing unit 72 , a determining unit 73 , a setting unit 74 , and the like.

[0072] The image generation unit 71 acquires an endoscopic image from the image acquisition unit 54, and generates an endoscopic image used for display on the monitor 18. For example, the image generation unit 71 acquires a B image obtained by photographing a subject using B pixels, a G image obtained by photographing a subject using G pixels, and an R image obtained by photographing a subject using R pixels from the image acquisition unit 54, and generates an endoscopic image for display using all or part of them.

[0073] In addition, when generating an endoscopic image for display, the image generating unit 71 performs necessary image processing on the endoscopic image acquired from the image acquiring unit 54 or an image generated using the endoscopic image acquired from the image acquiring unit 54. The image processing performed by the image generating unit 71 is, for example, an emphasis processing for emphasizing the subject or a part of the subject. Emphasis refers to the ability to obtain information about a specific part in contrast to other tissues or structures. For example, an emphasis processing includes enclosing a part having specific features with a frame and showing an outline, or changing the color or brightness relative to other parts (such as normal mucosa, etc.). "Information about a specific part" includes, in addition to being able to identify the position, shape, color or brightness, and / or size (range), etc. of a specific part, being able to know biological function information (such as oxygen saturation or blood vessel density, etc.) for a specific part.

[0074] The recognition unit 72 performs recognition processing to recognize the subject using an image obtained by photographing the subject. "Recognizing the subject" means detecting the presence or absence of a portion of the subject having specific characteristics (including detecting the entire subject), identifying the type or development level of the portion of the subject having specific characteristics (including identification for the entire subject), and / or obtaining (performing calculations, etc.) biological function information about a portion or all of the subject. The portion of the subject having specific characteristics is, for example, a lesion, a candidate lesion, or a treatment scar as a mark of treatment (hereinafter referred to as a lesion, etc.). That is, the recognition unit 72 can recognize the presence or absence of a lesion, etc. through recognition processing. In addition, the recognition unit 72 can recognize the type or development level of a lesion, etc. through recognition processing. Recognition of the type of a lesion, etc. means, for example, when the lesion, etc. is a polyp, distinguishing the type of adenoma, hyperplastic polyp, or cancer, etc. Identification of the degree of development of a lesion or the like refers to, for example, if the lesion or the like is cancer, identification of the stage of cancer, and NICE (NBI (Narrow Band Imaging) International Colorectal Endoscopic Classification) classification or JNET (The Japan NBI Expert Team) classification, etc. In the present embodiment, the image obtained by photographing the subject is an endoscopic image acquired from the image acquisition unit 54, or an endoscopic image generated by the image generation unit 71. In addition, in the present embodiment, the recognition unit 72 detects lesions or the like through recognition processing.

[0075] The recognition unit 72 is, for example, an artificial intelligence (AI) with a learning function. Specifically, the recognition unit 72 is an AI that learns using a machine learning algorithm such as a neural network (NN), a convolutional neural network (CNN), an Ada-Boost algorithm (adaboost), and a random forest (RandomForest). In addition, since the recognition unit 72 performs learning for performing recognition processing using a specific image, even if recognition processing using other images is possible, the accuracy of the result of the recognition processing is sometimes low. In the present embodiment, the recognition unit 72 is an AI that has performed learning to detect lesions using endoscopic images, etc. "Having a learning function" means being able to learn, including having completed learning. In addition, the recognition unit 72 may be replaced by an AI, and a structure in which a feature quantity is calculated based on an image, and the calculated feature quantity is used for detection, etc.

[0076] In this embodiment, the recognition unit 72 includes a detection unit 76 and a discrimination unit 77. The detection unit 76 is an AI that has learned to detect lesions, etc. using endoscopic images. The discrimination unit 77 is an AI that has learned to discriminate the degree of development of lesions, etc. using endoscopic images.

[0077] The recognition unit 72 inputs the result of the recognition process to the image generation unit 71 or the display control unit 66 according to the content of the recognition process. When the recognition unit 72 inputs the result of the recognition process to the image generation unit 71, the image generation unit 71 generates an endoscopic image for display reflecting the result of the recognition process. When the recognition unit 72 inputs the result of the recognition process to the display control unit 66, the display control unit 66 displays the result of the recognition process together with the endoscopic image acquired from the image generation unit 71 on the screen of the monitor 18. The recognition unit 72 inputs the result of the recognition process to the image generation unit 71, for example, in a case where the color of the endoscopic image for display is changed using the value of the biological function information as the result of the recognition process. The recognition unit 72 inputs the result of the recognition process to the display control unit 66, for example, in a case where the position of a lesion or the like as the result of the recognition process is shown by displaying a frame superimposed on the endoscopic image. In the present embodiment, the recognition unit 72 inputs information such as the position of the lesion or the like detected as the result of the recognition process to the image generation unit 71. Then, the image generating unit 71 generates an endoscopic image for display in which a portion where a lesion or the like is present is subjected to emphasis processing.

[0078] The discrimination unit 73 discriminates the operation on the subject. The operation on the subject refers to the act of applying a certain (arbitrary) action to the subject. For example, the collection of living tissue for biopsy or the treatment involved in surgery or treatment such as endoscopic mucosal resection (EMR), the spreading of liquid on the subject, or the cleaning of the subject are operations on the subject. The so-called spreading of liquid on the subject refers to the administration of a liquid medicine or water on the subject. Spreading water refers to sprinkling water on a part or all of the subject. In addition, the administration of medicine refers to, in addition to spreading a pigment such as indigo carmine on the subject, also including the intravenous injection of a fluorescent medicine such as indocyanine green (ICG) into the subject. Cleaning the subject refers to exposing the subject to be observed to the endoscope 12 by flushing away residues, residual liquid, blood, and other substances that hinder observation attached to the subject by water or air. Furthermore, even if no action is taken on the subject as a result, an action that may take effect on the subject is also included in the “operation on the subject”. For example, the action of spraying air to clean the subject is equivalent to the operation on the subject even if the subject is not cleaned as a result. In addition, the “operation on the subject” includes a series of actions related to the action on the subject before and after. For example, the action of protruding the forceps from the endoscope 12 during biopsy is a series of actions related to the biopsy even before the tissue is collected, and is included in the operation on the subject.

[0079] Determining the operation on the object refers to detecting the situation in which the object is operated, or detecting the situation in which the object is operated. The operation is determined, for example, by detecting the use of treatment instruments, detecting treatment scars (in addition to detecting surgical marks, it also includes detecting bleeding caused by treatment, detecting tissues or structures whose colors have changed due to the spreading of pigments, etc.), or detecting the operation of the water supply button 13b, etc. Regarding the use of treatment instruments, the determination unit 73 can use endoscopic images or setting information to detect the presence or absence of treatment instruments or the type (shape, etc.) of treatment instruments. The determination unit 73 can detect the above-mentioned treatment instruments, etc. (including spreading water) by analyzing endoscopic images. In addition, the determination unit 73 can detect the operation of the water supply button 13b, etc. by obtaining signals or setting information from the control unit 52.

[0080] Determining the operation on the subject includes determining whether there is an operation on the subject and determining the type of operation on the subject. Determining whether there is an operation includes detecting "the operation on the subject" of the detection object regardless of the type of operation. In addition, determining the type of operation on the subject means distinguishing the content of the operation (for example, whether it is a biopsy or the injection of a drug, etc.) to determine the operation on the subject. In this embodiment, the determination unit 73 determines whether there is an operation on the subject.

[0081] The setting unit 74 sets the recognition unit 72 to be effective or ineffective using the judgment result of the judgment unit 73. The judgment result of the judgment unit 73 is the presence or absence of an operation on the subject and / or the type of operation on the subject. "Setting the recognition unit 72 to be effective" means making the recognition processing performed by the recognition unit 72 effective, setting the recognition unit 72 to perform the recognition processing, setting to report the result of the recognition processing, or setting the recognition unit 72 to output the result of the recognition processing. "Setting the recognition unit 72 to be ineffective" means making the recognition processing performed by the recognition unit 72 ineffective, setting the recognition unit 72 not to perform the recognition processing, setting not to report the result of the recognition processing, or setting not to output the result of the recognition processing. That is, "setting the recognition unit 72 to be effective or ineffective" means making the recognition processing performed by the recognition unit 72 effective or ineffective, or making the reporting of the result of the recognition processing (or the output of the result of the recognition processing) effective or ineffective. "Notification of the result of the recognition processing" means a state in which a doctor or the like can recognize the result of the recognition processing. “Output of the result of the recognition process” means that the recognition unit 72 transmits the result of the recognition process to other processing units such as the image generation unit 71. In the present embodiment, the setting unit 74 sets the recognition process itself performed by the recognition unit 72 to be effective or ineffective.

[0082] When the determination unit 73 determines whether or not there is an operation on the subject, the setting unit 74 sets the recognition unit 72 to be invalid if there is an operation on the subject, and sets the recognition unit 72 to be valid if there is no operation on the subject. This is because if there is an operation on the subject, there is a possibility that the accuracy of the recognition process is reduced, or the doctor or the like needs to concentrate on the operation.

[0083] When the determination unit 73 determines the type of operation on the subject, if a specific operation is performed on the subject, the setting unit 74 disables the recognition unit 72. The specific operation refers to an operation that reduces the accuracy of the recognition processing performed by the recognition unit 72, or an operation that requires the doctor or the like to concentrate on the operation. Therefore, when the determination unit 73 determines the type of operation on the subject, the setting unit 74 can set the recognition unit 72 to be effective according to the type of operation. Therefore, appropriate diagnosis assistance can be continued according to the situation.

[0084] In addition, when the recognition unit 72 performs a plurality of recognition processes, the setting unit 74 may set each recognition process to be valid or invalid. For example, when the setting unit 74 sets the recognition unit 72 to be valid or invalid, the detection unit 76 may be set to be valid or invalid, and the identification unit 77 may be set to be valid or invalid regardless of the valid or invalid setting of the detection unit 76. Of course, the setting unit 74 may unify both the detection unit 76 and the identification unit 77 and set the entire recognition unit 72 to be valid or invalid. In the present embodiment, the setting unit 74 unifies the entire recognition unit 72, that is, the detection unit 76 and the identification unit 77 to be valid or invalid.

[0085] The notification unit 65 obtains the discrimination result from the discrimination unit 73 (or obtains information related to the setting of the identification unit 72 from the setting unit 74), and reports the valid or invalid state of the identification unit 72. The "notification" about the valid or invalid state of the identification unit 72 means that the doctor or the like as the user is in a state where he or she can know the valid or invalid state of the identification unit 72. For example, the notification unit 65 can report the valid or invalid state of the identification unit 72 by displaying a message (character string), text, graphics and / or signs (including displays such as marks, icons or indicators) on the screen of the monitor 18, not displaying them, or changing their display. In addition, the notification unit 65 can report the valid or invalid state of the identification unit 72 by lighting, extinguishing or flashing a light, sound (including sound), vibration of a component with a vibration function, or changes thereof. Of course, the notification unit 65 can report the valid or invalid state of the identification unit 72 by combining a character string with the lighting of a light. In the present embodiment, the notification unit 65 displays the valid or invalid state of the recognition unit 72 on the screen of the monitor 18 .

[0086] The display control unit 66 converts the endoscopic image output by the image processing unit 61 into a format suitable for display and outputs it to the monitor 18. Thus, the endoscopic image is displayed on the monitor 18. In addition, in the present embodiment, the notification unit 65 inputs information indicating the valid or invalid state of the recognition unit 72 to the display control unit 66. Therefore, the display control unit 66 displays the information on the screen of the monitor 18.

[0087] The endoscope system 10 configured as described above operates as follows with respect to the notification of the valid or invalid state of the recognition unit 72. Figure 4 As shown, when an object is photographed using the endoscope 12 (step S111), the image acquisition unit 54 acquires an endoscopic image. Then, the image generation unit 71 performs necessary image processing to generate an endoscopic image for display (step S112).

[0088] Thereafter, or in parallel with the generation of the endoscopic image for display, the discrimination unit 73, for example, uses the endoscopic image for display to discriminate the operation on the subject (step S113). When there is no operation on the subject (step S113: YES), the setting unit 74 sets the recognition unit 72 to be effective (step S114). Therefore, the recognition unit 72, for example, performs recognition processing on the endoscopic image for display (step S115). In the present embodiment, the recognition unit 72 has a detection unit 76 and a discrimination unit 77, and the setting unit 74 uniformly sets them to be effective or invalid, so that when there is no operation on the subject, the recognition unit 72 performs recognition processing of both detection and discrimination of lesions, etc. Thereafter, the recognition unit 72 outputs the recognition result to the image generation unit 71, and the image generation unit 71 uses the recognition result to generate an endoscopic image for display, for example, in which lesions, etc. are emphasized, and outputs it to the display control unit 66. As a result, the monitor 18 displays the endoscopic image on the screen. In addition, the notification unit 65 notifies the effective state of the recognition unit 72 (step S116). Here, since the setting unit 74 sets the recognition unit 72 to be effective, the notification unit 65 displays a character string indicating the effective state of the recognition unit 72 on the screen of the monitor 18. For example, Figure 5 As shown, the notification unit 65 displays "AI: ON" 122 on the screen of the monitor 18 displaying the endoscopic image 121. This notifies the identification unit 72 of the effective state. The endoscopic image 121 is an image in which no operation is performed on the subject and no lesions or the like are present.

[0089] On the other hand, when the discrimination unit 73 detects an operation on the subject (step S113: NO), the setting unit 74 sets the recognition unit 72 to be invalid (step S117). Therefore, the recognition unit 72 does not perform any of the detection and discrimination, and the image generation unit 71 outputs the generated endoscopic image for display to the display control unit 66. As a result, the monitor 18 displays the endoscopic image on the screen. In addition, the notification unit 65 notifies the invalid state of the recognition unit 72 (step S118). For example, if Figure 6 As shown, the notification unit 65 displays a display 127 of "AI: invalid" ("AI: OFF") on the screen of the monitor 18 displaying the endoscopic image 126. Thus, the invalid state of the identification unit 72 is notified. In addition, the biopsy forceps 128 are present in the endoscopic image 126. Therefore, the identification unit 72 becomes invalid.

[0090] As described above, in the endoscope system 10, the discriminating unit 73 discriminates the operation on the object, and the setting unit 74 sets the recognition unit 72 to be effective or ineffective using the discriminating result of the discriminating unit 73. Therefore, since the endoscope system 10 makes the recognition unit 72 effective or ineffective according to the observation situation, it is convenient for the doctor or the like who is the user to set the recognition unit 72 to be effective or ineffective manually during the observation. However, by only automatically setting the effectiveness or ineffectiveness of the recognition unit 72, the doctor or the like may sometimes misunderstand the result of the recognition processing or the result of not performing the recognition processing. For example, in the case where there is a prejudice that the recognition unit 72 is effective, when the recognition unit 72 automatically becomes ineffective and the result of the recognition processing cannot be displayed, even if there is a lesion, the doctor or the like may sometimes mistakenly believe that the result of the recognition processing performed by the recognition unit 72 indicates that there is no lesion, etc. Therefore, in the endoscope system 10, the notification unit 65 notifies the effective or ineffective state of the recognition unit 72. Therefore, if the display 122 of "AI: valid" or the display 127 of "AI: invalid" can be seen, it can be clearly seen whether the recognition unit 72 is performing the recognition process, so that the above-mentioned misrecognition can be prevented. As a result, misdiagnosis can be prevented.

[0091] [Second embodiment]

[0092] When the setting unit 74 sets each recognition process to be effective or ineffective, the notification unit 65 can notify the effective or ineffective state of each recognition process. For example, when the recognition unit 72 has a detection unit 76 and an identification unit 77, and the setting unit 74 sets the detection unit 76 and the identification unit 77 to be effective or ineffective, respectively, Figure 7 As shown, the notification unit 65 displays a display 223 indicating the valid or invalid state of the detection unit 76 and a display 224 indicating the valid or invalid state of the identification unit 77 in a display column 222 indicating the valid or invalid state of the recognition unit 72, and can notify their states. For example, the display 223 of "Detection: Valid" is a display indicating that the detection unit 76 is valid, and the display of "Detection: Invalid" (not shown) is a display indicating that the detection unit 76 is invalid. Similarly, the display 224 of "Identification: Valid" is a display indicating that the identification unit 77 is valid, and the display of "Identification: Invalid" (not shown) is a display indicating that the identification unit 77 is invalid.

[0093] As described above, when the setting unit 74 sets each recognition process to be effective or ineffective, the recognition process suitable for the observation condition can be effective, and the other recognition processes can be ineffective. Therefore, the endoscope system 10 can perform the best diagnosis assistance. In addition, since the notification unit 65 notifies the effective or ineffective state of each recognition process, the doctor or the like can correctly grasp the effectiveness or ineffectiveness of each recognition process. Therefore, it is possible to prevent the result of the recognition process from being misunderstood or the result of not performing the recognition process, and as a result, it is possible to prevent misdiagnosis.

[0094] In the second embodiment described above, an example is shown in which both the detection unit 76 and the identification unit 77 are effective. Figure 8 As shown, depending on the determination status of the determination unit 73, it is sometimes appropriate to enable only the detection unit 76, sometimes to enable only the identification unit 77, or sometimes to disable both the detection unit 76 and the identification unit 77.

[0095] "Discrimination situation 1" is a situation where it is appropriate to enable both the detection unit 76 and the identification unit 77, for example, when there is no operation on the subject, or when the subject is washed with water, etc. These are situations where there is a high possibility that detection and identification of lesions are required.

[0096] "Discrimination situation 2" is a situation in which it is appropriate to enable the detection unit 76 and disable the identification unit 77. For example, it is a situation in which bleeding from the subject during a biopsy or surgery is set as the discrimination target (detection target) of the discrimination unit 73, that is, although bleeding needs to be detected to confirm hemostasis, such a situation does not need to be identified because the surgery has already been performed. The discrimination unit 73 can distinguish the above situation according to the type of operation on the subject, for example.

[0097] "Discrimination condition 3" is a condition in which it is appropriate to invalidate the detection unit 76 and to validate the identification unit 77. For example, it is a condition in which there is no need to re-detect the lesion, etc. because the lesion, etc. has already been found, and in the diagnosis of the found lesion, etc., it is desired to receive more detailed diagnostic assistance from the endoscope system 10. Specifically, it is a condition in which a pigment is to be spread on the subject and the degree of development of the lesion, etc. is to be identified. The discrimination unit 73 can discriminate the above condition, for example, according to the type of operation on the subject.

[0098] "Discrimination condition 4" is a condition in which it is appropriate to disable both the detection unit 76 and the identification unit 77. For example, a biopsy or surgery that requires focused operation is performed in order to complete the detection and identification of a lesion. The discrimination unit 73 can distinguish the above condition according to the type of operation. When the discrimination unit 73 detects bleeding from the subject, for example, it becomes discrimination condition 2.

[0099] In addition, in the above-mentioned second embodiment, the validity or invalidity of the detection unit 76 and the identification unit 77 is indicated by "valid" or "invalid", but the validity or invalidity of the detection unit 76 and the identification unit 77 can also be indicated by other displays. For example, it is also possible to use a display such as "not supported" instead of the display of "invalid". In addition, even if the detection unit 76 or the identification unit 77 is in a valid state, in the case where the accuracy (accuracy or reliability) of the recognition processing is low, it is also possible to replace the display of "valid" and display "Unsupported" (not supported). For example, in the judgment condition 3, in the case where the identification unit 77 does not support the identification of the subject with the pigment dispersed, the accuracy of the identification is reduced, so it is possible to display "detection: invalid" and "identification: Unsupported". These changes in display methods can also be carried out in the same way as for the first embodiment.

[0100] [Third Embodiment]

[0101] In the first and second embodiments, the notification unit 65 notifies the valid or invalid state of the recognition unit 72, but the notification unit 65 may also notify the presence or absence of an operation or the type of operation determined by the determination unit 73. That is, the notification unit 65 may notify the determination result of the determination unit 73 in addition to notifying the valid or invalid state of the recognition unit 72.

[0102] For example, when the determination unit 73 detects an operation on the subject, Fig. 9 As shown, the notification unit 65 can also notify the detection of the operation on the subject by displaying "Actions" 301, etc., in addition to the display 127 of "AI: invalid" etc. to notify the validity or invalidity of the identification unit 72. In addition, the endoscopic image 310 is an endoscopic image in which a pigment 311 is spread on the subject.

[0103] As described above, when the notification unit 65 notifies whether there is an operation on the subject, the doctor or the like can recognize the reason for the effectiveness or ineffectiveness of the recognition unit 72. As a result, it is possible to more reliably prevent the result of the recognition processing from being misunderstood or the result of not performing the recognition processing, and prevent misdiagnosis.

[0104] In addition, the notification unit 65 may notify the type of operation determined by the determination unit 73. Fig.10 As shown in FIG. 1 , the notification unit 65 displays at a glance “Biopsy” indicating biopsy, “Chromo” indicating the spread of the pigment, and “Resection” indicating endoscopic resection in the operation type display column 313, and displays the type of operation determined by the determination unit 73 (in the Fig.10 By adding a check mark ("check mark") or the like to "chroma" in the figure, the type of operation determined by the determination unit 73 can be informed.

[0105] As described above, when the discriminating unit 73 discriminates the type of operation, when the notifying unit 65 notifies the type of operation discriminated by the discriminating unit 73, the doctor or the like can recognize in detail the reason why the discriminating unit 72 is effective or ineffective. As a result, it is possible to more reliably prevent the result of the discriminating process from being misunderstood or the result of not performing the discriminating process, and prevent misdiagnosis.

[0106] In addition, the notification of the presence or absence of an operation on the subject or the type of operation in the third embodiment described above can also be applied to the case where the validity or invalidity setting and notification are performed for each recognition process as in the second embodiment. In this case, the display bar 222 (see FIG. 22 ) indicating the validity or invalidity of the recognition unit 72 is displayed on the monitor 18. Figure 7 ) and the display 301 of “Operation” or the type of operation display column 313.

[0107] In addition, in the third embodiment, the presence or absence of an operation on the subject or the type of operation may be notified by a display other than text or character strings. Fig.11 As shown, in the operation type display column 313, an icon 331 indicating biopsy, an icon 332 indicating the spreading of a pigment, and an icon 333 indicating endoscopic resection are displayed at a glance, and the type of operation determined by the determination unit 73 can be notified by adding a color to the icon corresponding to the type of operation determined by the determination unit 73. Fig.11 In FIG. 3 , since the biopsy forceps 128 are present in the endoscopic image 321 , the icon 331 indicating the biopsy is colored.

[0108] As described above, when the type of operation is indicated by an icon, Fig.12 As shown, only icons corresponding to the types of operations identified by the identification unit 73 may be displayed in the type display column 313 of the operation. Fig.10 ) Similarly, in the operation type display column 313, only a character string indicating the type of operation determined by the determination unit 73 may be displayed.

[0109] In the first embodiment, the second embodiment, and the third embodiment, the setting unit 74 sets the recognition processing performed by the recognition unit 72 to be effective or ineffective, but instead, the setting unit 74 can set the effectiveness or ineffectiveness of the recognition unit 72 by setting the notification of the result of the recognition processing to be effective or ineffective. In this case, when the notification of the result of the recognition processing is set to be ineffective, the setting unit 74, while setting the detection unit 76 and the identification unit 77 to be effective, for example, stops outputting the recognition result to the image generation unit 71, etc., and stays in a state where the doctor or the like cannot obtain the recognition result. As a result, compared with the case of setting the effectiveness or ineffectiveness of the recognition processing itself, the effectiveness or ineffectiveness of the recognition unit 72 can be set simply and quickly.

[0110] In the first embodiment, the second embodiment and the third embodiment, the notification unit 65 notifies the valid or invalid state of the knowledge unit 72 by displaying it on the screen of the monitor 18. However, in the case where the valid or invalid state of the knowledge unit 72 is not notified by displaying it on the screen of the monitor 18, such as Fig.13 As shown in FIG. 1 , the processor device 16 may include a notification device 171. The notification device 171 is a speaker that emits sound or voice, an indicator composed of a light-emitting element such as an LED, or a vibration element such as a motor or a piezoelectric element. In addition, the notification unit 65 may use an element or the like included in the endoscope system 10 as the notification device 171. In addition, the notification device 171 may be provided in a part other than the processor device 16, that is, in the endoscope 12 or the light source device 14.

[0111] In addition, if Fig.14 As shown, the recognition unit 72, the determination unit 73, the setting unit 74, and / or the notification unit 65 may be provided in, for example, a medical image processing device 901 that communicates with the processor device 16 and cooperates with the endoscope system 10. Fig.15 As shown, the recognition unit 72, the determination unit 73, the setting unit 74, and / or the notification unit 65 may be provided in, for example, a diagnosis support device 911, which acquires the RAW image captured by the endoscope 12 directly from the endoscope system 10 (including an endoscope system without the notification unit 65, etc.) or indirectly from a PACS (Picture Archiving and Communication Systems) 910. Fig.16 As shown, an identification unit 72, a determination unit 73, a setting unit 74, and / or a notification unit 65 can be set in a medical business support device 930 that includes an endoscope system 10 and is connected to various inspection devices such as a first inspection device 921, a second inspection device 922, ..., a Kth inspection device 923 via a network 926.

[0112] That is, the present invention includes a medical image processing device and a working method thereof, wherein the medical image processing device comprises: an image acquisition unit that acquires an image obtained by photographing a subject; a recognition unit that performs recognition processing for recognizing the subject using the image; a discrimination unit that discriminates an operation on the subject; a setting unit that uses the discrimination result of the discrimination unit to set the recognition unit to be valid or invalid; and a notification unit that reports the valid or invalid state of the recognition unit. In addition, the present invention includes a diagnosis support device and a working method thereof, wherein the diagnosis support device comprises: an image acquisition unit that acquires an image obtained by photographing a subject; a recognition unit that performs recognition processing for recognizing the subject using the image; a discrimination unit that discriminates an operation on the subject; a setting unit that uses the discrimination result of the discrimination unit to set the recognition unit to be valid or invalid; and a notification unit that reports the valid or invalid state of the recognition unit. Similarly, the present invention includes a medical service support device and an operating method thereof, wherein the medical service support device includes: an image acquisition unit that acquires an image obtained by photographing a subject; a recognition unit that performs recognition processing to recognize the subject using the image; a determination unit that determines an operation on the subject; a setting unit that uses the determination result of the determination unit to set the recognition unit to be valid or invalid; and a notification unit that reports the valid or invalid state of the recognition unit.

[0113] In addition, the present invention includes an operation method of an endoscope system, the operation method of the endoscope system including the following steps: a step in which an image acquisition unit acquires an image obtained by photographing a subject; a step in which a recognition unit performs recognition processing using the image to recognize the subject; a step in which a discrimination unit discriminates an operation on the subject; a step in which a setting unit sets the recognition unit to be valid or invalid using a discrimination result of the discrimination unit; and a step in which a notification unit reports the valid or invalid state of the recognition unit. In addition, the present invention includes a processor device and an operation method thereof, the processor device including: an image acquisition unit that acquires an image obtained by photographing a subject; a recognition unit that performs recognition processing using the image to recognize the subject; a discrimination unit that discriminates an operation on the subject; a setting unit that sets the recognition unit to be valid or invalid using a discrimination result of the discrimination unit; and a notification unit that reports the valid or invalid state of the recognition unit.

[0114] In addition, a capsule endoscope may be used as the endoscope 12. In this case, the light source device 14 and part of the processor device 16 may be mounted in the capsule endoscope.

[0115] In the above-mentioned embodiment, the hardware structure of the processing unit that performs various processes such as the identification unit 72, the determination unit 73, the setting unit 74, or the notification unit 65 is various processors as shown below. The various processors include a general-purpose processor that executes software (programs) and functions as various processing units, namely a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a processor that can change the circuit structure after manufacturing FPGA (Field Programmable Gate Array), namely a programmable logic device (PLD), and a processor that has a circuit structure specially designed to perform various processes, namely a dedicated circuit.

[0116] A processing unit can be composed of one of these various processors, or a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, a CPU and FPGA, or a combination of a CPU and GPU, etc.). In addition, multiple processing units can be composed of one processor. As an example of using one processor to constitute multiple processing units, first, there is a form such as represented by a computer such as a client or server, which uses a combination of one or more CPUs and software to constitute a processor, and the processor functions as multiple processing units. Secondly, there is a form such as represented by a system on chip (System OnChip: SoC), which uses a processor that implements the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. In this way, various processing units are composed of one or more various processors as a hardware structure.

[0117] Furthermore, more specifically, the hardware structure of these various processors is a circuit in the form of a combination of circuit elements such as semiconductor elements.

[0118] In addition, in addition to endoscope systems, processor devices, and other related devices that acquire endoscope images, the present invention can also be used in systems or devices that acquire medical images (including dynamic images) other than endoscope images. For example, the present invention can be applied to ultrasonic inspection devices, X-ray imaging devices (including CT (Computed Tomography) inspection devices and mammography devices, etc.), MRI (magnetic resonance imaging) devices, etc.

[0119] Explanation of symbols

[0120] 10 Endoscopic system

[0121] 12 Endoscope

[0122] 12a Insertion section

[0123] 12b Operation section

[0124] 12c Bend

[0125] 12d Top end

[0126] 12e Angle button

[0127] 12f Disposal instrument insertion port

[0128] 13a Zoom operation section

[0129] 13b Water supply button

[0130] 14 Light source device

[0131] 16 Processor Device

[0132] 17 Water supply tank

[0133] 18 Monitor

[0134] 19 Console

[0135] 20 Light source

[0136] 22 Light source control unit

[0137] 30a Illumination optical system

[0138] 30b Photographic optical system

[0139] 45 Lighting lens

[0140] 46 Objective lens

[0141] 47 Zoom lens

[0142] 48 Image Sensor

[0143] 52 Control Department

[0144] 54 Image acquisition unit

[0145] 56 DSP

[0146] 58 Noise Reduction Unit

[0147] 59 Transformation

[0148] 61 Image Processing Unit

[0149] 65 Reporting Department

[0150] 66 Display control unit

[0151] 71 Image Generation Unit

[0152] 72 Identification

[0153] 73 Discrimination

[0154] 74 Setting Department

[0155] 76 Testing Department

[0156] 77 Identification Department

[0157] 121, 126, 310, 321 Endoscopic images

[0158] 122, 127, 223, 224, 301 display

[0159] 128 Pliers

[0160] 171 Notification equipment

[0161] 222 Display bar

[0162] 311 Pigment

[0163] 313 Category Display Column

[0164] 331, 332, 333 icons

[0165] 901 Medical Image Processing Device

[0166] 910 PACS

[0167] 911 Diagnostic Support Device

[0168] 921 First Inspection Device

[0169] 922 Second Inspection Device

[0170] 923 Kth Inspection Device

[0171] 926 Network

[0172] 930 Medical business support device

[0173] S111~S118 Action steps

Claims

1. An endoscope system, in, have: An image acquisition unit that acquires an image obtained by photographing a subject; a recognition unit that performs recognition processing for recognizing the subject using the image; a determination unit configured to determine an operation on the object; a setting unit that uses the determination result of the determination unit to set the recognition unit to be valid or invalid; as well as a notification unit for notifying the valid or invalid state of the identification unit, The determination unit determines whether or not there is an operation on the subject, and if there is an operation on the subject, the setting unit sets the recognition unit to be invalid, and if there is no operation on the subject, the setting unit sets the recognition unit to be valid, or, The determination unit determines the type of operation on the object, and when a specific operation is performed on the object, the setting unit sets the recognition unit to be invalid. The setting unit sets the recognition processing performed by the recognition unit to be effective or ineffective, or The setting unit sets notification of the result of the recognition process to be effective or ineffective.

2. The endoscope system according to claim 1, in, The operation determined by the determination unit is the use of a treatment tool.

3. The endoscope system according to claim 1, in, The operation determined by the determination unit is the spreading of liquid on the object.

4. The endoscope system according to any one of claims 1 to 3, in, The notification unit not only notifies the valid or invalid state of the recognition unit, but also notifies the determination result of the determination unit.

5. The endoscope system according to any one of claims 1 to 3, in, The recognition unit recognizes whether the subject has a lesion or a candidate lesion, or the type or development degree of the lesion or the candidate lesion of the subject.

6. The endoscope system according to any one of claims 1 to 3, in, When the recognition unit performs a plurality of recognition processes, the setting unit sets validity or invalidity for each of the recognition processes.

7. The endoscope system according to claim 1, in, The display of the valid or invalid state of the recognition unit is displayed together with the image used for observation by the endoscope system.

8. The endoscope system according to claim 1, in, The display of the invalid state of the recognition unit is displayed together with the image used for observation by the endoscope system.

9. The endoscope system according to claim 1, in, The operation is an operation of reducing the accuracy of the recognition process.

10. The endoscope system according to claim 1, in, The procedures include those associated with the harvesting or surgical removal of living tissue.

11. The endoscope system according to claim 1, in, The recognition unit includes artificial intelligence.

12. The endoscope system according to claim 11, in, The notification is a notification of whether the recognition processing based on the utilization of the artificial intelligence is effective or invalid.

13. The endoscope system according to claim 1, in, The operation is an operation by a user to change the color of the object.

14. The endoscope system according to claim 1, in, The operation is the spreading of a pigment agent or a fluorescent agent toward the object.

15. The endoscope system according to claim 14, in, The recognition unit includes artificial intelligence.

16. The endoscope system according to claim 15, in, The notification is a notification of whether the recognition processing based on the use of the artificial intelligence is effective or invalid.

17. The endoscope system according to claim 5, in, The recognition process further obtains biological function information related to a part or all of the subject. Identification of the type of lesion or candidate lesion When the lesion or candidate lesion is a polyp, the type of lesion or candidate lesion is determined to be adenoma, hyperplastic polyp or cancer.

18. The endoscope system according to claim 17, in, The identification of the degree of development of the lesion or the candidate lesion is determined by the stage of cancer, or the NICE classification or the JNET classification when the lesion or the candidate lesion is cancer.