IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, NAVIGATION METHOD,
The image processing device optimizes image acquisition conditions for both display and analysis in endoscopes, enhancing surgical support by ensuring images are both easily viewable and analyzable, thus improving surgical precision and diagnosis accuracy.
Patent Information
- Application Number
- JP2022513820
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-04-09
- Publication Date
- 2024-12-09
- Estimated Expiration
- 2040-04-09
AI Technical Summary
Existing image processing technologies for endoscopes do not optimize image acquisition conditions for both display and analysis, leading to inadequate support for surgeons during surgeries and examinations, as images optimized for display may lack necessary information for accurate analysis.
An image processing device that acquires multiple types of images with different conditions, including those optimized for display and analysis, and adaptively changes these conditions based on image analysis results to enhance both visibility and analyzability.
Provides surgeons with highly accurate analysis results and effective support by ensuring images are optimized for both display visibility and analysis performance, allowing for precise navigation and diagnosis.
Smart Images

Figure 0007596365000001 
Figure 0007596365000002 
Figure 0007596365000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an image processing device, an image processing method, a navigation method, and an endoscope system for performing navigation when observing an image. [Background technology]
[0002] Conventionally, navigation technologies that support various tasks have been developed using image processing technology. For example, in the medical field, image processing technology has made it possible to use insertion support that supports the insertion of an endoscope and diagnosis support for estimating the condition of a disease. For example, computer aided diagnosis (CAD) has been developed that provides support information such as providing a quantitative judgment scale, identifying fine structures to be noted during diagnosis, and estimating the condition of a disease through image analysis. In image processing devices that realize such CAD and insertion support, etc., efforts have been made to provide appropriate support to the surgeon.
[0003] For example, Japanese Patent Publication No. 2019-42156 discloses a technology that makes it easy to confirm the analysis results by making it possible to display two analysis results for first and second medical images so that their position or range (size), etc. can be compared.
[0004] Incidentally, real-time medical images acquired by an endoscope or the like are not only processed for image analysis but also displayed on a monitor or the like, so that extremely useful image information of the affected area or the like can be provided to the surgeon during surgery, examination, etc. However, in JP 2019-42156 A, the acquired images are not optimized for visual inspection during surgery, examination, etc., and optimal support is not necessarily provided to the surgeon.
[0005] An object of the present invention is to provide an image processing device, an image processing method, a navigation method, and an endoscope system that can provide extremely effective support to an operator by optimizing image acquisition conditions. DISCLOSURE OF THEINVENTION [Means for solving the problem]
[0006] An image processing device according to one aspect of the present invention acquires a first image based on acquisition conditions for display at a frame rate according to the acquisition conditions for display for acquiring an image suitable for display, and acquires a second image based on acquisition conditions for analysis for acquiring an image suitable for image analysis at a frame rate lower than the frame rate according to the acquisition conditions for display and the first image. Or the third image a first acquisition condition including the display acquisition condition, a second acquisition condition including the display acquisition condition and the analysis acquisition condition, and a second acquisition condition including the display acquisition condition. and The second acquisition condition above The acquisition conditions for the above analysis are Different acquisition conditions for analysis and an analysis acquisition condition in which a wavelength of illumination light for an image pickup target is different from the analysis acquisition condition of the second acquisition condition. an acquisition condition designation unit that sets a third acquisition condition including the first image acquired by the image acquisition unit; from No. 3 an image analysis unit that performs image analysis on an image; a support information generation unit that generates support information based on a result of the image analysis by the image analysis unit; and a support information generation unit that controls switching from the first acquisition condition to the second acquisition condition in accordance with the result of the image analysis and controls switching from the second acquisition condition to the third acquisition condition based on the image analysis by the image analysis unit. result Based on the accuracy of the second acquisition condition, the second image acquired according to the acquisition condition for analysis included in the second acquisition condition is subjected to image analysis. result The accuracy of the first acquisition condition is higher than that of the second acquisition condition. 3 Image analysis on images result and a control unit for increasing the accuracy of the measurement.
[0007] An image processing method according to one aspect of the present invention is an image processing method in an image processing device including an acquisition condition designation unit, an image analysis unit, a support information generation unit, and a control unit, wherein the acquisition condition designation unit acquires a first image based on acquisition conditions for display at a frame rate according to the acquisition conditions for display for acquiring an image suitable for display, and acquires a second image based on acquisition conditions for analysis for acquiring an image suitable for image analysis at a frame rate lower than the frame rate according to the acquisition conditions for display and the first image. Or the third image a first acquisition condition including the display acquisition condition, a second acquisition condition including the display acquisition condition and the analysis acquisition condition, and a second acquisition condition including the display acquisition condition. and The second acquisition condition above The acquisition conditions for the above analysis are Different acquisition conditions for analysis and an analysis acquisition condition in which a wavelength of illumination light for an image pickup target is different from the analysis acquisition condition of the second acquisition condition. and a third acquisition condition including the first image acquired by the image acquisition unit. from No. 3 An image analysis is performed on the image to obtain an image analysis result, the support information generation unit generates support information based on the image analysis result, and the control unit controls switching from the first acquisition condition to the second acquisition condition in accordance with the image analysis result, and controls switching from the second acquisition condition to the third acquisition condition in accordance with the image analysis result in the image analysis unit. result Based on the accuracy of the second acquisition condition, the second image acquired according to the acquisition condition for analysis included in the second acquisition condition is subjected to image analysis. result The accuracy of the first acquisition condition is higher than that of the second acquisition condition. 3 Image analysis on images result Improve the accuracy of.
[0008] A navigation method according to one aspect of the present invention is a navigation method using an image processing device including an acquisition condition designation unit, an image analysis unit, and a control unit, in which the acquisition condition designation unit acquires a first image based on acquisition conditions for display at a frame rate according to the acquisition conditions for display for acquiring an image suitable for display, and acquires a second image based on acquisition conditions for analysis for acquiring an image suitable for image analysis at a frame rate lower than the frame rate according to the acquisition conditions for display and the first image. Or the third image a first acquisition condition including the acquisition condition for displaying is set for an image acquisition unit capable of acquiring a mixture of the above, and a second acquisition condition including the acquisition condition for displaying and the acquisition condition for analysis is set for the image acquisition unit; and The second acquisition condition above The acquisition conditions for the above analysis are Different acquisition conditions for analysis and an analysis acquisition condition in which a wavelength of illumination light for an image pickup target is different from the analysis acquisition condition of the second acquisition condition. a third acquisition condition including a third acquisition condition, and the image analysis unit analyzes the first image acquired by the image acquisition unit. from No. 3 An image analysis is performed on the image to obtain an image analysis result, and the control unit controls switching from the first acquisition condition to the second acquisition condition in accordance with the image analysis result, and controls switching from the second acquisition condition to the third acquisition condition in accordance with the image analysis result in the image analysis unit. result Based on the accuracy of the second acquisition condition, the second image acquired according to the acquisition condition for analysis included in the second acquisition condition is subjected to image analysis. result The accuracy of the first acquisition condition is higher than that of the second acquisition condition. 3 Image analysis on images result Improve the accuracy of.
[0009] An endoscope system according to one aspect of the present invention acquires a first image based on acquisition conditions for display at a frame rate according to the acquisition conditions for display for acquiring an image suitable for display, and acquires a second image based on acquisition conditions for analysis for acquiring an image suitable for image analysis at a frame rate lower than the frame rate according to the acquisition conditions for display and the first image. Or the third imageand an endoscope having an illumination unit and an imaging unit capable of acquiring the first image based on at least one of the display acquisition conditions and the analysis acquisition conditions. from No. 3 a video processor for acquiring an image; a first acquisition condition including the acquisition condition for display; a second acquisition condition including the acquisition condition for display and an acquisition condition for analysis; and The second acquisition condition above The acquisition conditions for the above analysis are Different acquisition conditions for analysis and an analysis acquisition condition in which a wavelength of illumination light for an image pickup target is different from the analysis acquisition condition of the second acquisition condition. an acquisition condition designation unit that sets a third acquisition condition including the first acquisition condition acquired by the video processor; from No. 3 an image analysis unit that performs image analysis on an image; a support information generation unit that generates support information based on a result of the image analysis by the image analysis unit; and a support information generation unit that controls switching from the first acquisition condition to the second acquisition condition in accordance with the result of the image analysis and controls switching from the second acquisition condition to the third acquisition condition based on the image analysis by the image analysis unit. result Based on the accuracy of the second acquisition condition, the second image acquired according to the acquisition condition for analysis included in the second acquisition condition is subjected to image analysis. result The accuracy of the first acquisition condition is higher than that of the second acquisition condition. 3 Image analysis on images result and a control unit for improving the accuracy of the image processing device. [Brief description of the drawings]
[0010] [Figure 1] 1 is a block diagram showing a configuration of an endoscope system including an image processing device according to a first embodiment of the present invention. [Diagram 2] 1 is a diagram for explaining the needs for an image for display and an image for analysis. [Diagram 3] FIG. 2 is an explanatory diagram showing an example of a usage form of the endoscope system of FIG. [Figure 4] 3A to 3C are diagrams for explaining the relationship between WLI light and NBI light irradiated from the endoscope according to the first embodiment and blood vessels in the mucous membrane of a subject. [Diagram 5] FIG. 2 is a diagram for explaining the relationship between DRI light and blood vessels in the mucosa of a subject. [Figure 6] 2 is an explanatory diagram showing an example of a captured image acquired by a video processor 3. FIG. [Figure 7] FIG. 2 is an explanatory diagram showing an example of an image output to a monitor 5. [Figure 8] 4 is an explanatory diagram showing an example of an image supplied to an image analysis unit 32. FIG. [Figure 9] 4 is a table for explaining an example of image processing by the image processing unit 12 based on the acquisition conditions for display and the acquisition conditions for analysis. [Figure 10] 4 is a flowchart for explaining the operation of the first embodiment. [Figure 11] FIG. 11 is an explanatory diagram for explaining an image acquired in a specific use case. [Figure 12] 13 is a table for explaining an example of an acquisition condition I3 based on a determination by a determination unit 34. [Figure 13] FIG. 13 is an explanatory diagram for explaining a support display. [Figure 14] 11 is a diagram for explaining the priority order of acquisition conditions for a plurality of requests. [Figure 15] 10 is a flowchart showing an operation flow employed in the second embodiment. [Figure 16] FIG. 13 is a block diagram showing a third embodiment. BEST MODE FOR CARRYING OUT THEINVENTION
[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
[0012] (First embodiment) FIG. 1 is a block diagram showing the configuration of an endoscope system including an image processing device according to a first embodiment of the present invention.
[0013] For example, in an endoscope, if an image used for image analysis for navigation and an image for display displayed on a monitor are obtained by separate imaging devices, the endoscope tip becomes large. For this reason, generally, an image for display is used as an image for image analysis for navigation. However, the image for display is obtained under acquisition conditions suitable for display, and may lack information necessary for image analysis. In addition, the image for image analysis may have poor visibility, and it is not preferable to use the image for image analysis as the image for display. For the above reasons, it has been difficult to provide support based on a highly accurate analysis result while displaying an easy-to-see endoscopic image. In this specification, a highly accurate analysis result is an analysis result that enables more effective support for the surgeon, and not only means an accurate analysis result, but also means that a type of analysis result necessary for support is obtained from various analysis results.
[0014] Therefore, in this embodiment, it is possible to obtain a plurality of types of images with different acquisition conditions, including an image for image display with excellent visibility and an image with excellent analyticity, thereby enabling extremely effective support to the surgeon. Note that an image with excellent analyticity is an image that allows for obtaining highly accurate analysis results.
[0015] Furthermore, in this embodiment, it is also possible to adaptively change the image acquisition conditions in order to acquire images with better analytical properties while maintaining an image display with excellent visibility. Note that, although an endoscope system is used as an example for explanation in Fig. 1, the present invention is not limited to this, and can be applied to various devices for performing various tasks involving observation.
[0016] FIG. 2 is a diagram for explaining the needs for images to be displayed and images to be analyzed.
[0017] An image for display (image for display) is an image that allows a person to obtain necessary information by visually recognizing the image displayed on a screen. On the other hand, an image for analysis (image for analysis) is an image that is the subject of analysis in a navigation device. Considering the quality of information processing between people and computers, the characteristics suitable for an image for display and an image for analysis are mutually different.
[0018] As shown in Fig. 2, it is preferable that the display image is an image with excellent visibility that contains only useful information so that it is easy for people to recognize. For example, it is preferable that the image quality of the display image is an image with little noise, that has been subjected to gamma processing that is close to the characteristics of the human eye, and that has the frequency band that the user wants to see emphasized.
[0019] On the other hand, since the image for analysis is processed by a computer or the like, the more information contained in the image information for analysis, the more useful the analysis result (the more accurate the analysis result) is obtained. For example, even if the image information for analysis has image quality in which areas other than the target area stand out, the adverse effect on the analysis result is small. In addition, when noise reduction, gamma processing, image enhancement processing, etc. are applied to the image, information necessary for analysis may be lost, so it is better not to apply these image processing to the image for analysis.
[0020] Furthermore, in situations involving special light observation such as Narrow Band Imaging (NBI), which is effective for observing blood vessels in mucous membranes, taking into account human image recognition capabilities, it is better to limit the image displayed on the monitor screen to only one type of special light observation image, or at least to a display in which a special light observation image is superimposed on a normal light observation image.
[0021] On the other hand, even if multiple types of special light observation image signals are continuously input to the navigation device, there is no adverse effect on the image analysis processing; rather, the multiple types of image information increase the likelihood of obtaining useful analysis results.
[0022] In addition, while it is preferable for the frame rate of the display image to be 30 FPS or higher for human visual recognition, it is possible to obtain useful information from the analysis image even if the frame rate is relatively low, for example, 1 FPS or less.
[0023] (composition) Fig. 3 is an explanatory diagram showing an example of a usage form of the endoscope system of Fig. 1. With reference to Fig. 3, an example of a usage form of the endoscope system will be described.
[0024] 3 shows an example of performing a procedure on the abdominal cavity of a subject P using an endoscope system 1. The endoscope system 1 is an example of a laparoscopic surgery system. The endoscope system 1 mainly includes an endoscope 2 (lapaoscope) that captures an image of the inside of the body cavity of the subject P and outputs an image signal, a video processor 3 that is connected to the endoscope 2 and controls the drive of the endoscope 2, acquires an image signal related to the subject captured by the endoscope 2, and performs predetermined image processing on the image signal, a light source device 4 that is provided inside the video processor 3 and supplies a predetermined illumination light for irradiating the subject, a monitor 5 that displays an observation image corresponding to the image signal, and a navigation device 30 that is connected to the video processor 3 and is an image processing device for performing diagnostic support and the like.
[0025] 3 shows a state in which an endoscope 2 and a treatment tool 7 are inserted into the abdomen of a subject P via a trocar. The endoscope 2 is connected to a video processor 3 via a universal cord. The video processor 3 has a built-in light source device 4, and is configured so that the inside of the abdominal cavity is illuminated by the light source device 4. The endoscope 2 is driven by the video processor 3 to capture images of the inside of the abdominal cavity of the subject P. The captured images acquired by the endoscope 2 are subjected to signal processing by the video processor 3 and then supplied to a navigation device 30.
[0026] The navigation device 30 provides the input captured image for display on the monitor 5, and generates support information by performing an analysis process on the captured image. The navigation device 30 supports the surgeon by outputting the generated support information to the monitor 5 and displaying it as necessary.
[0027] In this embodiment, the navigation device 30 gives instructions to the video processor 3 to set image acquisition conditions including at least one of the imaging conditions for imaging by the endoscope 2 and the image processing conditions for image processing by the video processor 3, thereby acquiring images for image display with excellent visibility and at the same time acquiring images that are useful for image analysis for assistance.
[0028] (Endoscopy) 1, various types of endoscopes such as a digestive endoscope or a laparoscope can be used as the endoscope 2. The endoscope 2 has a long and thin insertion section that is inserted into a body cavity of a subject, and an operation section that is disposed on the base end side of the insertion section and is held and operated by an operator. A universal cord extends from the base end of the operation section, and the endoscope 2 is detachably connected to a video processor 3 including a light source device 4 by this universal cord.
[0029] An imaging device 20 is disposed, for example, at the tip of the insertion portion. The imaging device 20 includes an optical system 21, an imaging element 22, and an illumination unit 23. The illumination unit 23 generates illumination light under the control of the light source device 4, and irradiates the generated illumination light onto the subject. The illumination unit 23 may have a predetermined light source (not shown), such as an LED (light-emitting diode). In this embodiment, the illumination unit 23 may have a plurality of light sources, such as a light source that generates white light for normal observation, a light source that generates narrowband light for narrowband observation, and a light source that generates infrared light of a predetermined wavelength. The illumination unit 23 has various irradiation modes, and is controlled by the light source device 4 to switch the wavelength of the illumination light, control the irradiation intensity, and control the temporal pattern of irradiation.
[0030] Although FIG. 1 shows an example in which the illumination unit 23 is provided within the imaging device 20, the light source device 4 may generate illumination light, which may be guided to the tip of the endoscope 2 via a light guide (not shown) to irradiate the subject.
[0031] The optical system 21 may include lenses and apertures (not shown) for zooming and focusing, and may also include a zoom (variable magnification) mechanism, focusing and aperture mechanisms (not shown) for driving these lenses. Illumination light from the illumination unit 23 is irradiated onto the subject, and return light from the subject passes through the optical system 21 and is guided to the imaging surface of the image sensor 22.
[0032] The imaging element 22 is configured with a CCD, a CMOS sensor, or the like, and obtains a captured image (imaging signal) of the subject by photoelectrically converting the subject optical image from the optical system 21. The imaging device 20 outputs the obtained captured image to the video processor 3.
[0033] The video processor 3 includes a control unit 11 that controls each unit of the video processor 3, the imaging device 20, and the light source device 4. The control unit 11 and each unit in the control unit 11 may be configured by a processor using a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), or the like, may operate according to a program stored in a memory (not shown) to control each unit, or may realize some or all of the functions by a hardware electronic circuit.
[0034] (Light source device) The light source device 4 controls the illumination unit 23 to generate white light and various lights for special observation. For example, the light source device 4 may cause the illumination unit 23 to generate white light, NBI (Narrow Band Imaging) light, DRI (Dual Red Imaging) light, and excitation light for AFI (Auto Fluorescence Imaging) (hereinafter, AFI light). White light is used as illumination light (hereinafter, WLI light) for so-called WLI (white light imaging) observation (normal observation), NBI light is used for narrow band light observation, DRI light is used for long wavelength narrow band light observation, and AIF light is used for fluorescence observation.
[0035] The illumination unit 23 may be configured with a plurality of types of LEDs, laser diodes, xenon lamps, etc., and generate these illumination lights, or may generate these illumination lights using white light, an NBI filter, a DRI filter, an AFI filter, etc. The illumination unit 23 can increase or decrease the amount of light, thereby changing the exposure value during imaging by the imaging device 20, and enabling exposure control that eliminates the effects of saturation and low luminance noise. The NBI light may be blue light with a wavelength λ=415 nm and green light with a wavelength λ=540 nm.
[0036] (Video Processor) The control unit 11 of the video processor 3 includes an image processing unit 12, an imaging parameter setting unit 13, an image processing parameter setting unit 14, and a display control unit 15. The imaging parameter setting unit 13 controls the light source device 4 to set the state of the illumination light generated by the illumination unit 23. The imaging parameter setting unit 13 also controls the imaging device 20 to set the state of the optical system defined by the optical system 21 and the driving state of the imaging element 22.
[0037] That is, the imaging parameter setting unit 13 can set imaging conditions including optical conditions during imaging by the imaging device 20 and driving conditions of the imaging element 23. For example, by setting the imaging parameter setting unit 13, it is possible to generate NBI light, DRI light, AFI light, etc. as illumination light and control the wavelength, intensity, etc. of the generated illumination light. In addition, by setting the imaging parameter setting unit 13, the imaging device 20 can output imaging signals in various modes, and can control, for example, the frame rate, the number of pixels, pixel addition, change of readout area, sensitivity switching, discriminative output of color signals, etc.
[0038] The image signal output from the image sensor 22 is sometimes called RAW data, which may be used as original data prior to image processing.
[0039] (Image processing unit) The image processing unit 12 is provided with captured images (moving images and still images) captured by the imaging device 20, and performs predetermined signal processing on the captured captured images, such as color adjustment processing, matrix conversion processing, noise removal processing, image synthesis, adaptive processing, and various other types of signal processing. The image processing parameter setting unit 14 is configured to set processing parameters for image processing in the image processing unit 12.
[0040] The visibility of the captured image can be improved by the image processing of the image processing unit 12. The analysis characteristics of the image analysis process for the captured image can also be improved by the image processing of the image processing unit 12. The image processing unit 12 can also convert so-called RAW data from the imaging element into data in a specific format.
[0041] The display control unit 15 is provided with the captured image that has been signal-processed by the image processing unit 12. The display control unit 15 converts the captured image acquired by the imaging device 20 into an observation image that can be processed by the monitor 5, and outputs the image.
[0042] The video processor 3 is also provided with an operation unit 16. The operation unit 16 may be configured with, for example, various buttons, dials, or a touch panel, and accepts user operations and outputs operation signals based on the user operations to the control unit 11. The operation unit 16 may be configured to be hands-free and to accept gesture input, voice input, or the like and generate operation signals. The control unit 11 is capable of controlling each unit in response to the operation signals.
[0043] In this embodiment, the settings made by the imaging parameter setting section 13 and the image processing parameter setting section 14 are controlled by the navigation device 30.
[0044] (Navigation device) The navigation device 30 includes a control unit 31, an image analysis unit 32, an acquisition condition storage unit 33, a determination unit 34, an acquisition condition designation unit 35, and an assistance information generation unit 36. The control unit 31 may be configured with a processor using a CPU, FPGA, or the like, may operate according to a program stored in a memory (not shown) to control each unit, or may realize some or all of the functions with a hardware electronic circuit. The entire navigation device 30 or each component of the navigation device 30 may also be configured with a processor using a CPU, FPGA, or the like, may operate according to a program stored in a memory (not shown) to control each unit, or may realize some or all of the functions with a hardware electronic circuit.
[0045] The acquisition condition storage unit 33 stores acquisition conditions for determining the setting contents of the imaging parameter setting unit 13 and the image processing parameter setting unit 14 of the video processor 3. For example, the acquisition condition storage unit 33 may store information on the type and setting of the illumination light that the light source device 4 causes the illumination unit 23 to emit (hereinafter referred to as light source setting information), information on the driving of the optical system 21 (hereinafter referred to as optical system setting information), and information on the driving of the image sensor 22 (hereinafter referred to as imaging setting information). Furthermore, the acquisition condition storage unit 33 may store information for determining the image processing contents of the image processing unit 12 (hereinafter referred to as image processing setting information).
[0046] Furthermore, the light source setting information, optical system setting information, imaging setting information, and image processing setting information (hereinafter, also referred to as acquisition condition setting information) may be stored as a set in the acquisition condition storage unit 33. For example, acquisition condition setting information in an initial state, acquisition condition setting information in a predetermined observation mode, acquisition condition setting information corresponding to predetermined analysis conditions, etc. may be stored in advance.
[0047] The acquisition condition designation unit 35 is controlled by the control unit 31 to designate the acquisition condition setting information read from the acquisition condition storage unit 33 to the imaging parameter setting unit 13 and the image processing parameter setting unit 14. In response to the designation by the acquisition condition designation unit 35, the observation mode in the endoscope 2, the type of illumination light, control related to imaging, image processing in the video processor 3, and the like are performed. Note that the acquisition condition designation unit 35 may be configured to generate acquisition condition setting information that is not stored in the acquisition condition storage unit 33 under the control of the control unit 31 and output it to the video processor 3. Also, the acquisition condition storage unit 33 may be omitted, and the acquisition condition designation unit 35 may generate acquisition condition setting information as necessary.
[0048] For example, the acquisition condition designation unit 35 designates light source setting information, whereby the light source device 4 designates which illumination light, such as WLI light, NBI light, DRI light, or AFI light, is to be used.
[0049] (WLI light, NBI light, DRI light, AFI light) Here, the WLI light, NBI light, DRI, and AFI light employed in this embodiment will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is a diagram for explaining the relationship between the WLI light and NBI light irradiated from the endoscope according to the first embodiment and blood vessels in the mucosa of the subject, and Fig. 5 is a diagram for explaining the relationship between the DRI light and blood vessels in the mucosa of the subject.
[0050] By irradiating the mucosal surface with WLI light (white light), blood vessels and other features present in the mucosa can be reproduced on a monitor in colors that appear natural to humans (doctors). On the other hand, when WLI light (white light) is used, the capillaries on the mucosal surface and the fine patterns of the mucosa cannot necessarily be reproduced clearly enough for humans to recognize.
[0051] In this embodiment, Narrow Band Imaging (NBI) light with two narrow wavelengths (blue light: 390 to 445 nm (415 nm in this embodiment) / green light: 530 to 550 nm (540 nm in this embodiment)) that are easily absorbed by hemoglobin in blood may be used to observe the mucosa.
[0052] 4, the capillaries 64 in the mucosal surface layer 61 are clearly visualized by absorbing the blue light (415 nm) of the NBI light, and the blood vessels 65 in the layer 62 slightly deeper than the surface layer are visualized by the green light (540 nm). This results in the capillaries in the mucosal surface layer 61 and the fine mucosal pattern being displayed in an emphasized manner.
[0053] As described above, in this embodiment, the wavelength of the NBI light, which is narrowband light, may be set to a different wavelength to perform special light observation.
[0054] On the other hand, in this embodiment, DRI (Dual Red Imaging) light using light narrowed to two long wavelengths (600 nm / 630 nm) is adopted, and the DRI light is irradiated onto the subject, thereby highlighting blood vessels 66 or blood flow information from the deep mucosal layer to the submucosal layer (layer 63 in FIG. 5), which are difficult to see using normal light observation.
[0055] Furthermore, in this embodiment, it is also possible to perform so-called auto fluorescence imaging (AFI), in which a specific excitation light for fluorescence observation is irradiated onto the subject and tumorous lesions and normal mucosa are highlighted in different colors.
[0056] In addition to the light source control, the optical system 21 and the image sensor 22 can be controlled by the acquisition condition setting information, and for example, the exposure time of the image sensor can be changed by setting the acquisition conditions. The effects of saturation and low luminance noise can also be eliminated by the exposure control.
[0057] (An example of how to acquire multiple types of images) In this embodiment, the acquisition condition designation unit 35 may generate a mixture of acquisition condition setting information that specifies acquisition conditions for display, which are conditions for acquiring an image for display with excellent visibility (hereinafter referred to as acquisition condition setting information for display), and acquisition condition setting information that specifies acquisition conditions for analysis, which are conditions for acquiring an image for analysis with excellent analytical performance in image analysis processing (hereinafter referred to as acquisition condition setting information for analysis). For example, only acquisition condition setting information for display may be output during a predetermined first period, and a mixture of acquisition condition setting information for display and acquisition condition setting information for analysis may be output during a predetermined second period.
[0058] When the display acquisition condition setting information is provided to the video processor 3, the video processor 3 controls at least one of the light source device 4 (illumination unit 23), the optical system 21, the image sensor 22, and the image processing unit 12 based on the display acquisition condition setting information so as to output an image for display with excellent visibility. Furthermore, when the display acquisition condition setting information and the analysis acquisition condition setting information are input in a mixed manner, the video processor 3 controls at least one of the light source device 4 (illumination unit 23), the optical system 21, the image sensor 22, and the image processing unit 12 based on the display acquisition condition setting information and the analysis acquisition condition setting information so as to output an image for display with excellent visibility and an image with excellent analyticity. Thus, the display acquisition conditions are imaging and lighting conditions for making the wavelength of the light source close to natural light (daylight) so that it feels natural when a doctor searches for an affected area with natural light or illuminates the affected area (mainly the surface) to observe it, performing image processing on the imaging results with an emphasis on visibility, and setting the frame rate and other factors with an emphasis on continuity. The analysis acquisition conditions are imaging and lighting conditions that increase the amount of effective information for image judgment rather than the doctor's visibility, making the wavelength of the light source reach not only the surface of the affected area but also the inside of the affected area, performing image processing on the imaging results with an emphasis on the amount of effective information for analysis, and setting the frame rate and other factors with an emphasis on analyticity so that it is easier to judge specific patterns and image features rather than continuity.
[0059] Figures 6 to 8 are explanatory diagrams showing examples of an image acquired by the video processor 3, an image output to the monitor 5, or an image supplied to the image analysis unit 32 when a mixture of display acquisition condition setting information and analysis acquisition condition setting information is input to the video processor 3.
[0060] FIG. 6 shows a series of frames obtained by imaging with the imaging element 22. In FIG. <raw>Fig. 6 shows an image captured at a high frame rate (e.g., 30 FPS or more) using a high-intensity WLI light as illumination light. <raw>shows an image captured at a low frame rate (e.g., about 1 FPS) using NBI light as illumination light (narrow band light observation). <raw>1 shows an image captured at a low frame rate (for example, 1 FPS) using low-intensity WLI light as illumination light.
[0061] WLI <raw>The frames are used to generate images for display. <raw>Frame, low light WLI <raw>The frames are used to generate images for analysis. <raw>The frame may be used to generate an image for analysis. Although not shown in FIG. 6, the DRI light may be used as an illumination light to capture an image for image analysis. <raw>Frames may be acquired at a low frame rate (about 1 FPS), or images may be acquired using excitation light for AFI observation. In this way, the conditions for acquiring the imaged results are changed at the same position where the object is imaged, so that useful information can be acquired with a simple configuration and without complex operations. For example, it is expected that a display image with excellent visibility can be obtained from an image captured at a high frame rate (e.g., 30 FPS or more) using a high-intensity WLI light as illumination light, and the light source setting conditions, optical system setting conditions, imaging setting conditions, etc. for obtaining such an image become the display acquisition conditions. Also, for example, NBI <raw>Images obtained by special light observation using a frame or the like are expected to have excellent analytic properties, and the light source setting conditions, optical system setting conditions, imaging setting conditions, etc. for obtaining such images are acquisition conditions for analysis. In addition, image processing conditions for obtaining images with excellent visibility are acquisition conditions for display, and image processing conditions for obtaining images with excellent analytic properties are acquisition conditions for analysis.
[0062] Fig. 9 is a table showing specific examples of display acquisition conditions and analysis acquisition conditions for image processing, and is intended to explain an example of conditions that can be realized by the image processing of the image processing unit 12. As shown in Fig. 9, the video processor 3 performs gamma processing that matches the characteristics of the human eye according to the display acquisition conditions, for example, regarding gamma processing among the image processing performed on the imaging signal. Furthermore, the video processor 3 does not perform gamma processing that is unnecessary for the analysis processing according to the analysis acquisition conditions. Similarly, according to the display acquisition conditions and the analysis acquisition conditions, the video processor 3 distinguishes between image processing for obtaining an image for display and image processing for obtaining an image for analysis, as shown in Fig. 9, regarding other image processing such as white balance, color correction, noise reduction, and image enhancement.
[0063] The image processing unit 12 of the video processor 3 performs signal processing according to the display acquisition condition setting information, thereby obtaining a WLI <raw>The navigation device 30 obtains a WLI image for display having excellent visibility from the captured images. The navigation device 30 outputs the WLI image having excellent visibility from the captured images from the video processor 3 to the monitor 5 as an image for display.
[0064] This is shown in Fig. 7, where the control unit 31 of the navigation device 30 extracts the WLI image from the images output from the video processor 3 and outputs it to the monitor 5. In the example of Fig. 7, the WLI image from the series of frames in Fig. 6 is <raw>The WLI image obtained by image processing of the WLI image is extracted and supplied to the monitor 5. Note that the image is captured so that the frame rate of the WLI image supplied to the monitor 5 is, for example, 30 FPS or higher. Since the image for visual confirmation is viewed while performing the operation, efforts are made to minimize the loss of image frames per time, but this is not always the case if, for example, the image does not change.
[0065] In this way, the captured image obtained by the imaging device 20 of the endoscope 2 is displayed on the display screen of the monitor 5. The image displayed on the monitor 5 is a WLI image with excellent visibility, and the surgeon can confirm the image within the field of view of the imaging device 20 on the display screen of the monitor 5 as an easy-to-view image.
[0066] A WLI image with excellent visibility may lack information useful for image analysis for navigation due to signal processing in the image processing unit 12. Therefore, as shown in Fig. 9, the video processor 3 stops most image processing for the image for analysis according to the acquisition conditions for analysis and adds information useful for image analysis. In this way, it becomes possible to output an image for analysis useful for image analysis according to the information on the acquisition conditions for analysis.
[0067] For example, it is difficult to distinguish the capillaries and mucosal fine patterns in the above-mentioned mucosal surface layer using a WLI image, but it is relatively easy to distinguish them by image analysis using an NBI image obtained by imaging using an NBI light or the like. Therefore, the control unit 31 is configured to provide all of the output images of the video processor 3, including the NBI image, to the image analysis unit 32 to perform image analysis. FIG. 8 shows the images supplied to the image analysis unit 32. Note that the control unit 31 may provide only the images excluding the WLI image among the output images of the video processor 3 to the image analysis unit 32.
[0068] The image analysis unit 32 performs various image analyses to support the surgeon. The image analysis unit 32 performs image analysis on the captured image input from the video processor 3 to obtain an image analysis result. For example, the image analysis unit 32 obtains an image analysis result regarding the traveling direction of the insertion part of the endoscope 2, or an image analysis result regarding the differentiation result of a lesion. The image analysis result of the image analysis unit 32 is provided to the support information generation unit 36.
[0069] The support information generating unit 36 generates support information based on the image analysis result of the image analyzing unit 32. For example, when the direction in which the insertion portion should be inserted is obtained from the image analysis result, the support information generating unit 36 generates support information indicating the insertion direction. Also, for example, when the image analysis result indicates a differentiation result of a lesion, the support information generating unit 36 generates support information for presenting the differentiation result to the surgeon. The support information generating unit 36 may generate support display data such as an image (support image) or text (support text) to be displayed on the monitor 5 as the support information. Also, the support information generating unit 36 may generate audio data to be output as audio from a speaker (not shown).
[0070] (Changes in acquisition conditions) Furthermore, in this embodiment, the navigation device 30 is capable of changing the image acquisition conditions based on the image analysis results including the characteristics of the image used for analysis and various information acquired from the image. The determination unit 34 determines whether or not the image acquisition conditions should be changed and how the image acquisition conditions should be changed. For example, when the determination unit 34 determines that a sufficient analysis result cannot be obtained from the image analysis results or that a more detailed image analysis is required, the determination unit 34 instructs the acquisition condition designation unit 35 to change the acquisition conditions to those required for performing the desired image analysis.
[0071] For example, the determination unit 34 may determine a change to a specific acquisition condition based on a specific criterion. For example, the determination unit 34 may determine the acquisition condition to be changed by comparing values included in the image analysis result, such as contrast information and histogram information, acquired from the image used for analysis, with a predetermined reference value. The determination unit 34 may also determine whether or not the image used for analysis contains a specific image feature or pattern by pattern matching or the like, and determine the acquisition condition to be set based on the result.
[0072] In addition, the judgment unit 34 may be configured to instruct the acquisition condition designation unit 35 to change the acquisition conditions required to obtain the desired analysis results, depending not only on the image analysis results but also on the observation mode, the contents of the procedure, etc.
[0073] (action) Next, the operation of the embodiment configured as above will be described with reference to Figures 10 to 14. Figure 10 is a flowchart for explaining the operation of the first embodiment, and Figure 11 is an explanatory diagram for explaining an image acquired in a specific use case.
[0074] The example of FIG. 11 shows a usage scene similar to that of FIG. 3, in which an endoscope 2 (rigid endoscope) is inserted into a body cavity to observe internal tissues and organs.
[0075] For example, immediately after power-on, the acquisition condition designation unit 35 of the navigation device 30 reads out display acquisition condition setting information in the initial setting from the acquisition condition storage unit 33, and supplies it to the video processor 3. The display acquisition condition setting information enables the setting of acquisition conditions for acquiring an image for display, and the imaging parameter setting unit 13 in the control unit 11 of the video processor 3 sets parameters of the light source device 4, the optical system 21, and the imaging element 22 based on the display acquisition condition setting information.
[0076] As a result, normal observation is performed in step S1 in Fig. 10. Note that acquisition conditions I1 in Fig. 10 are acquisition conditions corresponding to, for example, acquisition condition setting information for display in the initial setting, and are predetermined conditions. In accordance with acquisition conditions I1, for example, the light source device 4 emits a high amount of WLI light from the illumination unit 23, and the control unit 11 drives the image sensor 22 at a high frame rate (for example, 30 FPS or higher), thereby obtaining WLI light from the image sensor 20. <raw>The captured image is output.
[0077] Furthermore, the image processing parameter setting unit 14 in the control unit 11 sets image processing parameters for the image processing unit 12 based on the display acquisition condition setting information. As a result, the image processing unit 12 performs gamma processing, white balance processing, color correction, noise reduction processing, image enhancement processing, and the like, suited to the characteristics of the human eye, on the captured image from the imaging device 20, as shown in Fig. 9, for example, and generates a WLI image suitable for display.
[0078] The WLI image acquired by the image processing unit 12 is supplied to the navigation device 30. The control unit 31 outputs the input WLI image to the monitor 5 as a display image. In this way, the WLI image with excellent visibility is displayed on the display screen of the monitor 5. The surgeon can reliably observe the internal tissues, organs, etc. in the body cavity by the WLI image with good visibility on the display screen of the monitor 5.
[0079] In the example of FIG. 10, the control unit 31 judges in step S2 whether or not it is a specific timing to change from the acquisition condition I1 to the acquisition condition I2. Support by the navigation device 30 is not necessarily required for the entire period from the start to the end of the surgery or examination. Considering the processing amount of image analysis in the navigation device 30, it is considered preferable to perform support by the navigation device 30 only when support is required. Therefore, the control unit 31 judges the timing of transition from the acquisition condition I1 based on the acquisition condition setting information for display to the acquisition condition I2 including the acquisition condition setting information for analysis when an instruction is given by the surgeon or when a predetermined medical scene is reached. The acquisition condition I2 is a predetermined condition. The acquisition conditions I1 and I2 can be set to appropriate contents by user setting.
[0080] When the control unit 31 determines that the specific timing has been reached in response to, for example, an operation by the surgeon, the control unit 31 shifts the process to step S3 and instructs the acquisition condition designation unit 35 to shift to the acquisition condition I2. Note that when the control unit 31 determines that the specific timing has not been reached, the control unit 31 shifts the process to step S4.
[0081] In step S3, the acquisition condition designation unit 35 reads out acquisition condition setting information including acquisition condition setting information for display and acquisition condition setting information for analysis, and outputs the information to the video processor 3, thereby moving to acquisition condition I2. That is, acquisition condition I2 is a condition for acquiring not only an image for display but also an image for analysis by using acquisition condition setting information for display and acquisition condition setting information for analysis.
[0082] In this case, the light source device 4, the optical system 21 and the image sensor 22 are controlled by the image capture parameter setting unit 13 and the image processing parameter setting unit 14 to perform WLI at a frame rate of, for example, 30 FPS or more. <raw>For example, the imaging device 20 may acquire a WLI image as shown in FIG. <raw>、WLI <raw>、NBI <raw>、WLI <raw>, Low light WLI <raw>、WLI <raw>In the example of FIG. 11, four frames out of a series of six frames are WLI frames acquired based on the display acquisition condition setting information. <raw>The two frames are NBI frames acquired based on the acquisition condition setting information for analysis. <raw>, Low Light WLI <raw>It is a frame.
[0083] The image processing parameter setting unit 14 controls the image processing unit 12 based on the display acquisition condition setting information and the analysis acquisition condition setting information. <raw>For the frame, signal processing is performed based on the display acquisition condition setting information to acquire a WLI image. <raw>, Low light WLI <raw>For the frame, for example, no signal processing for display is performed based on the analysis acquisition condition setting information. <raw>Frame, low light WLI <raw>The image processor 12 converts the frames into an NBI image and a low-light WLI image, respectively, and outputs these images to the navigation device 30.
[0084] 11, the control unit 31 of the navigation device 30 outputs the WLI image to the monitor 5 as a display image, and outputs the NBI image and the low-light-intensity WLI image to the image analysis unit 32. The WLI image is also provided to the image analysis unit 32. The image analysis unit 32 performs image analysis using the WLI image, NBI image, and low-light-intensity WLI image to obtain a predetermined analysis result. For example, when performing diagnostic support, the image analysis unit 32 obtains a desired analysis result such as the presence or absence of a lesion candidate and differentiation of the lesion.
[0085] The images analyzed in the image analysis unit 32 include images obtained by special light observation such as NBI images suitable for analysis, and since no image processing involving information loss has been performed, the images have a sufficient amount of information for image analysis, and highly accurate analysis results can be obtained in the image analysis unit 32. This amount of information is information possessed by each pixel for deriving something from the image, or information that prominently shows changes in the arrangement of pixels, and is assumed to be the amount of information necessary to identify the characteristics of the object possessed by each image to be analyzed, such as contrast, spatial frequency, gradation characteristics, color changes, and the distinguishability of differences in wavelength.
[0086] In this embodiment, step S4 and a judgment in step S5 are performed following step S2 or S3, and if the judgment in step S5 is NO, the process proceeds to step S7. In step S7, it is judged whether or not support display is necessary. For example, when a lesion candidate is found by the image analysis result of image analysis unit 32, control unit 31 judges that support display is necessary and causes support information generation unit 36 to generate support information. Support information generation unit 36 generates support information based on the analysis result of image analysis unit 32.
[0087] For example, as support information when a lesion candidate is found, the support information generating unit 36 may generate display data for displaying a mark (support display) indicating the position of the lesion candidate on the display image displayed on the display screen of the monitor 5. The control unit 31 provides the display data generated by the support information generating unit 36 to the monitor 5. In this way, a mark indicating the position of the lesion candidate is displayed on the display image (image observed by the endoscope 2) displayed on the monitor 5 (step S8).
[0088] In this embodiment, the WLI image with excellent visibility is displayed on the monitor 5, making it easy to check the affected area, and the image analysis for support is performed using the NBI image suitable for image analysis, making it possible to obtain highly accurate analysis results, and providing extremely effective support to the surgeon. In addition, the analysis image is acquired only when support is required, making it possible to display the display image in high quality without unnecessarily lowering the frame rate, and preventing an unnecessarily large amount of image analysis processing. In addition, the display image and the analysis image are acquired based on the imaging signal of the imaging device 20, so there is no need to place multiple imaging devices at the tip of the endoscope insertion part, and the tip part is not enlarged, and the amount of information to be processed does not increase significantly, requiring high-performance hardware.
[0089] (Adaptively changing acquisition conditions) Furthermore, in this embodiment, by setting the acquisition condition I3 that changes depending on the situation, it is possible to perform an even more accurate analysis. In step S4, the determination unit 34 determines whether or not the acquisition condition should be changed to obtain an even more accurate analysis result, and the acquisition condition if it is changed, based on the image for analysis and the image analysis result of the image analysis unit 32. The determination unit 34 determines whether or not an even more accurate analysis result can be obtained (step S5), and if so, causes the acquisition condition designation unit 35 to set the acquisition condition I3 for that purpose (step S6). If the determination unit 34 determines that an even more accurate analysis result cannot be obtained, the process proceeds to step S7.
[0090] In step S6, the acquisition condition designation unit 35 reads out acquisition condition setting information for display and acquisition condition setting information for analysis from the acquisition condition storage unit 33 in accordance with the determination result of the determination unit 34, and outputs them as acquisition conditions I3 to the imaging parameter setting unit 13 and the image processing parameter setting unit 14. That is, the acquisition conditions I3 adaptively changed in accordance with the output of the video processor 3 are fed back to the video processor 3. Note that the acquisition condition designation unit 35 may generate and output acquisition condition setting information for display and acquisition condition setting information for analysis in accordance with the determination result of the determination unit 34, rather than the information stored in the acquisition condition storage unit 33.
[0091] 12 is a chart for explaining an example of the acquisition condition I3 based on the judgment by the judgment unit 34. The status column in FIG. 12 indicates information obtained from the analysis result of the image analysis unit 32, and the feedback content indicates the acquisition condition I3 specified by the acquisition condition specification unit 35 based on the judgment result of the judgment unit 34.
[0092] Even when a display image acquired based on acquisition condition I1 is output, image analysis unit 32 can perform image analysis using this display image (WLI image). When moving from step S2 to step S4, determination unit 34 performs determination using the analysis result of image analysis unit 32 on the WLI image. For example, image analysis unit 32 obtains blood vessel information related to the mucosa from the analysis result of the WLI image. When determining that many blood vessels are visible in the mucosa surface layer, determination unit 34 sets acquisition conditions for acquiring an analysis image such as an NBI image using long-wavelength illumination light as acquisition condition I3.
[0093] A display image (short-wavelength image) using short-wavelength illumination light makes it easy to confirm microvessels in the tissue surface. Therefore, if many microvessels are visible, it is determined that there is a possibility that some kind of malignant tumor is lurking based on the vascular information in the mucosal surface layer, and in order to more clearly grasp the microvessel structure in the mucosal surface layer, acquisition condition I3 is set to acquire an NBI image or the like as an analysis image.
[0094] Furthermore, for example, when an analysis image (WLI image, NBI image, etc.) based on acquisition condition I2 is acquired, and the analysis result based on the analysis image indicates that there is little information on the microvessels in the surface layer of the mucosa, the judgment unit 34 sets acquisition information I3 for acquiring a DRI image by DRI special light observation with a short wavelength so that vascular information from deeper layers of the mucosa (for example, vascular information from the deeper layers of the mucosa to the submucosa) can be obtained.
[0095] Also, for example, the judgment unit 34 increases or decreases the frame rate of the display image depending on the magnitude of the image movement of the affected area of the subject in the image analyzed by the image analysis unit 32, and sets acquisition information I3 for increasing or decreasing the image type of the analysis image.
[0096] Also, for example, the determination unit 34 sets acquisition information I3 for changing the luminance of the analysis image according to the luminance information of the periphery of the affected area of the subject in the image analyzed by the image analysis unit 32. For example, when the image of the periphery of the affected area of the subject is dark, the determination unit 34 sets acquisition information I3 for brightening the luminance of the analysis image, and when the image of the periphery of the affected area of the subject is bright, the determination unit 34 sets acquisition information I3 for darkening the luminance of the analysis image. Note that such control is possible by appropriately correcting the light quantity of the light source or the exposure time related to the imaging element. In this way, the support display in step S8 is performed using the image acquired based on the acquisition condition I3.
[0097] In the flowchart of FIG. 10, an example is shown in which the acquisition condition I3 is changed only once, but the determination unit 34 may be configured to repeatedly change the setting contents of the acquisition condition I3 as necessary.
[0098] 10, an example has been described in which only the display acquisition condition setting information for acquiring the display image is output in a predetermined first period, and the display acquisition condition setting information and the analysis acquisition condition setting information are mixed to acquire the display image and the analysis image in a predetermined second period corresponding to the operation of the surgeon, etc. For example, the surgeon may set the first period to be the period during the movement of the insertion portion of the endoscope 2 to the observation target site, and the second period to be the time when the distal end of the endoscope 2 reaches the observation target site and starts detecting the lesion candidate.
[0099] In addition, the acquisition condition I1 is generated using only the display acquisition condition setting information for acquiring the display image, but the display acquisition condition and the analysis acquisition condition may be set at all times after the power is turned on. For example, the acquisition condition I1 is generated using the WLI <raw>For example, one NBI for each given number of frames. <raw>Set the frame capture and WLI <raw>The WLI image based on the above was used as the display image, and the WLI image and NBI <raw>and an NBI image based on the above may be used as an image for analysis. In this case, a high-quality image can be displayed by using a WLI image with a relatively high frame rate, and it is also possible to perform an analysis required for support with a sufficiently small processing load on the navigation device 30. Then, based on the analysis result or by the operation of the surgeon, an acquisition condition I2 that increases the ratio of acquisition of an image for analysis can be set, thereby making it possible to perform a highly accurate analysis according to the support requested by the surgeon.
[0100] That is, such image acquisition control proceeds as if it were background processing without requiring any particular attention from the surgeon. Therefore, for example, accurate navigation is possible without the surgeon having to focus on judging whether or not it is the timing to perform special observation using NBI light or the like, and effective support can be instantly provided to the surgeon. Fig. 13 is an explanatory diagram for explaining the support display. Fig. 13 shows how the endoscope 2 is inserted into the body cavity P to observe the internal tissues and organs. The arrows indicate the illumination light emitted from the tip of the rigid scope and its reflected light, and the reflected light is incident on the imaging device 20 of the endoscope 2.
[0101] (Image for display under acquisition condition I1) The display image (Im1) obtained under acquisition condition I1 was obtained by white light observation and is close to the result seen under natural light to which people are accustomed. That is, in this example, acquisition condition I1 is a condition for obtaining a display image that emphasizes visibility. However, in imaging under this acquisition condition I1, the reflected component from the object surface prevails and the information on the inside of the tissue is relatively reduced, so even if there is some abnormality in the area surrounded by the dashed line, it may be difficult to find it.
[0102] (Image for analysis under acquisition condition I2) The image for analysis under acquisition condition I2 is an image (Im2) acquired under imaging conditions and image processing conditions including observation light conditions that enable observation of the inside of tissue, and therefore can detect abnormalities inside the tissue that do not appear on the surface of the body tissue. In Fig. 13, the detected lesions in the image are indicated by hatching. As explained in Figs. 4 and 5, the use of the image under special light observation (image acquired under acquisition condition I2) allows for more accurate analysis results to be obtained compared to the image under normal white light observation (image acquired under acquisition condition I1).
[0103] (Image for analysis under acquisition condition I3) The image for analysis under the acquisition condition I3 is an image (Im3) obtained under an acquisition condition changed from the acquisition condition I2 in order to obtain an analysis result with higher accuracy. In this case, as shown by the hatching in Fig. 13, the shape of the lesion is clearer than that of the image Im2. As a result, the analysis result using the image (Im3) is often more accurate than the analysis result using the image (Im2).
[0104] (Support display) The support information generating unit 36 generates support information based on the analysis results with higher accuracy. In the example of FIG. 13, the support information is display data showing the shape of the lesion. The control unit 31 displays a display based on the support information by superimposing it on the display image (Im1). Furthermore, the support information generating unit 36 may generate display data as support information for displaying a text display such as "lesion found" near the position of the dashed line. In this way, the observer can confirm a display showing the presence of the lesion detected by the image analyzing unit 32 on the display image that is natural to the human eye, and the observer can take measures such as re-examining this part using another method.
[0105] The support display method by the support information generating unit 36 can be variously improved and customized. For example, while an example of displaying a support display based on an analysis image based on acquisition condition I3 has been described in Fig. 13, a support display based on an analysis image based on acquisition condition I2 may be displayed. Furthermore, the support information generating unit 36 may display the analysis image as is as the support display, or may display a composite image based on the analysis result.
[0106] (Priority of Acquisition Conditions) When the determination unit 34 changes the acquisition conditions depending on the situation, it may be necessary to consider multiple requests (acquisition conditions). Fig. 14 is a table for explaining the priority order for such multiple requests.
[0107] For example, assume that the WLI image or NBI image used for analysis by the image analysis unit 32 has few detected microvessels, the surrounding image is dark, and there is a large movement in the image. In this case, the determination unit 34 generates an acquisition condition I3 that makes it possible to acquire a bright image, such as a DRI image using a long wavelength, as the image for analysis, without reducing the frame rate of the image for display, if possible.
[0108] However, there are cases where it is not possible to satisfy all requirements. Thus, the determination unit 34 assigns priorities to each requirement (condition) and determines the acquisition condition I3. For example, the determination unit 34 sets a condition of not lowering the frame rate of the display image as a priority order 1. The determination unit 34 also sets a condition of acquiring a DRI image or the like using a long wavelength as an analysis image as a priority order 2. The determination unit 34 also sets a condition of acquiring a brighter image as a priority order 3.
[0109] The determination unit 34 takes such priority into consideration and instructs the acquisition condition designation unit 35 to generate the acquisition condition I3. For example, the acquisition condition designation unit 35 <raw>The acquisition condition specification unit 35 generates display acquisition condition setting information for maintaining the frame rate at 30 FPS or more. In addition, for example, the acquisition condition specification unit 35 generates a DRI image for generating an analysis image. <raw>The acquisition condition setting information for analysis is generated for acquiring frame-based DRI images at 2 FPS. In addition, for example, the acquisition condition designation unit 35 does not support requests of priority order 3, taking into consideration the limitation of the maximum frame rate that can be captured.
[0110] In this way, the acquisition conditions generated by prioritizing requests to the video processor 3 are fed back, so images useful for both display and analysis can be efficiently acquired in the video processor 3. Also, images according to the acquisition conditions can be reliably acquired in various types of endoscopes and video processors with different performance and functions.
[0111] In this manner, in this embodiment, it is possible to obtain an image for image display with excellent visibility and an image with excellent analyticity, and it is possible to provide extremely effective support for various tasks while maintaining an image display with excellent visibility. In addition, it is also possible to adaptively change the image acquisition conditions, and it is possible to provide appropriate support according to the situation.
[0112] 1 shows an example in which the video processor 3 and the navigation device 30 are configured separately, it is clear that the navigation device 30 may be built into the video processor 3. In addition, the endoscope system is not limited to a laparoscopic surgery system, and may be applied to an endoscope system using a normal flexible endoscope.
[0113] Furthermore, the analysis by the image analysis unit 32, the judgment by the judgment unit 34, the generation of the acquisition condition setting information by the acquisition condition designation unit 35, and the like in the navigation device 30 may be realized by an AI (artificial intelligence) device.
[0114] Second embodiment Fig. 15 is a flowchart showing an operation flow adopted in the second embodiment. The hardware configuration in this embodiment is the same as that in Fig. 1, and therefore a description thereof will be omitted.
[0115] In the first embodiment, an example was described in which the acquisition condition I3 is adaptively set when a more accurate analysis result is obtained than the acquisition conditions I1 and I2. In this embodiment, when the acquisition condition I1 is changed to the acquisition condition I2, it is determined which of these analysis results is more accurate. As described above, the analysis result being more accurate means that a more suitable analysis result is obtained for support, and includes, for example, a case in which the amount of information obtained from the image is increased. In this embodiment, when it is determined that a more accurate analysis result can be obtained by changing the conditions, a further change of the same type as the change content of the acquisition condition is made, and when this is not the case, a change of a different type from the change content of the acquisition condition is made, thereby enabling the setting of the optimal acquisition condition.
[0116] An example of a further change of the same kind as the change in the acquisition conditions is, for example, a change to change the wavelength of the NBI light when the acquisition conditions I1 for acquiring a normal light observation image are changed to acquisition conditions I2 for acquiring an NBI image. Also, an example of a change of a different kind from the change in the acquisition conditions is, for example, a change in the acquisition conditions to acquire a DRI image instead of an NBI image when the acquisition conditions I1 for acquiring a normal light observation image are changed to acquisition conditions I2 for acquiring an NBI image.
[0117] For example, an acquisition condition I3 in which a more accurate analysis result is obtained for a combination of acquisition conditions I1 and I2, and an acquisition condition I3 in which the accuracy of the analysis result is reduced, may be registered in advance in the acquisition condition storage unit 33. In this case, the determination unit 34 may instruct the acquisition condition designation unit 35 which of the storage contents in the acquisition condition storage unit 33 should be read out depending on the determination result as to whether the analysis result has become more accurate or has become less accurate.
[0118] In step S11 of Fig. 15, an image is captured based on a predetermined acquisition condition I1. For example, as shown in Fig. 13, an examination inside a body cavity of a subject is started, and an image is captured by the endoscope 2 under the control of the control unit 11 of the video processor 3, and the captured image is supplied to the monitor 5 via the navigation device 30. For example, display acquisition condition setting information is adopted as the acquisition condition I1, and the WLI <raw>The image processor 12 acquires a WLI <raw>The WLI image based on the frames is output to the navigation device 30, and the control unit 31 supplies the WLI image to the monitor 5 to display it on the screen. In this way, the WLI image with excellent visibility is displayed on the display screen of the monitor 5.
[0119] The video processor 3 temporarily records the WLI image acquired based on the acquisition condition I1 as a captured image Im1 in a recording device (not shown) (step S12). The image analysis unit 32 of the navigation device 30 obtains an analysis result by image analysis of the WLI image acquired based on the acquisition condition I1.
[0120] In step S13, the control unit 31 determines whether or not an instruction to change the acquisition conditions has been issued. As in the first embodiment, for example, an instruction to change the acquisition conditions can be issued by an operator, or the determination unit 34 can issue an instruction to change the acquisition conditions based on the analysis result of the image analysis unit 32.
[0121] When an instruction to change the acquisition conditions is issued, the control unit 31 causes the acquisition condition designation unit 35 to generate predetermined acquisition conditions I2. The acquisition condition designation unit 35 may read information on the acquisition conditions I2 from the acquisition condition storage unit 33. Here, the acquisition conditions I2 are conditions for acquiring WLI images at a predetermined frame rate or higher and, for example, NBI images. As a result, for example, as shown in FIG. 6, the WLI <raw>Frame and NBI <raw>An acquired image including a frame is acquired by the endoscope 2 (step S14). The image processing unit 12 generates a WLI image and an NBI image based on the captured image from the imaging device 20, and outputs the WLI image and the NBI image to the navigation device 30.
[0122] The image analysis unit 32 obtains an analysis result by image analysis of the WLI image and the NBI image acquired based on the acquisition condition I2. The support information generation unit 36 generates support information based on the analysis result. The video processor 3 temporarily records the WLI image and the NBI image acquired based on the acquisition condition I2 as the captured image Im2 in a recording device (not shown) (step S15).
[0123] In step S16, the determination unit 34 determines whether images based on the acquisition conditions I1 and I2 have been acquired for the same observation site. For example, the determination unit 34 can determine whether the images are based on the same observation site based on the analysis result of the image analysis unit 32.
[0124] If the determination unit 34 determines that the images based on the acquisition conditions I1 and I2 are of the same observation site, the determination unit 34 determines whether the amount of information (hereinafter, the part written as the amount of information refers to the amount of information expressing the characteristics of the object contained in the image to be used for some kind of support or assistance) has increased in the next step S17. That is, the determination unit 34 compares the amount of information of the image Im1 based on the acquisition condition I1 obtained by irradiating a certain area of the subject with WLI light with the amount of information of the image Im2 based on the acquisition condition I2 obtained by irradiating the same area with WLI light and NBI light. The determination unit 34 determines which of the information amounts of the image Im1 (the amount of information required to obtain effective support) and the information amount of the image Im2 (the amount of information required to obtain effective support) has a relatively larger amount of information.
[0125] If the amount of information of the image based on acquisition condition I2 is greater than that of the image based on acquisition condition I1, the determination unit 34 determines that a more effective image can be acquired using the same type of acquisition condition, and in step S18 instructs the acquisition condition designation unit 35 to set the same type of acquisition condition I3. Note that in the figure, the portion described as image conditions with the same type of changes indicates that if an image with a sufficient amount of information is obtained, no further changes such as image acquisition and processing need be made.
[0126] The acquisition condition designation unit 35 changes the information to acquire an image by NBI light of a wavelength band different from the wavelength band designated in the acquisition condition I2 as, for example, an acquisition condition I3 of the same type as the acquisition condition I2. <raw>NBI based on NBI light of different wavelengths from the previous frame <raw>An acquired image including a frame is acquired by the endoscope 2. The image processing unit 12 generates a WLI image and an NBI image based on the captured image from the imaging device 20, and outputs the WLI image and the NBI image to the navigation device 30.
[0127] The image analysis unit 32 obtains an analysis result by image analysis of the WLI image and the NBI image acquired based on the acquisition condition I2. The support information generation unit 36 generates support information based on the analysis result. In addition, the video processor 3 temporarily records the WLI image and the NBI image acquired based on the acquisition condition I3 as the captured image Im3 in a recording device (not shown) (step S19).
[0128] On the other hand, if the determination unit 34 determines in step S17 that the amount of information has not increased, the process proceeds to step S20 to determine whether the amount of information has decreased. That is, the determination unit 34 determines whether the amount of information of an image Im2 obtained under acquisition conditions I2 by irradiating a certain region of a subject with WLI light is lower than the amount of information of an image Im1 obtained under acquisition conditions I1 by irradiating the same region with WLI light.
[0129] If the amount of information of an image based on acquisition condition I2 is lower than that of an image based on acquisition condition I1, the judgment unit 34 judges that a valid image can be acquired using acquisition conditions different from acquisition condition I2, and in step S21 instructs the acquisition condition designation unit 35 to set different acquisition conditions I3.
[0130] For example, the acquisition condition designation unit 35 changes the acquisition condition I3 different from the acquisition condition I2 to information for acquiring an image using DRI light instead of the NBI light specified in the acquisition condition I2. Note that the acquisition condition designation unit 35 may change the acquisition condition I3 different from the acquisition condition I2 to information for acquiring an image using DRI light and AFI light, including NBI light of a wavelength band different from the wavelength band of the NBI light specified by the acquisition condition I2. Furthermore, the acquisition condition designation unit 35 may change the frame rate of the image sensor 22 and various image processing by the image processing unit 12.
[0131] The image analysis unit 32 obtains an analysis result by performing image analysis on each image acquired based on the acquisition condition I2 and a different acquisition condition I3. The support information generation unit 36 generates support information based on the analysis result. In addition, the video processor 3 temporarily records each image acquired based on the acquisition condition I2 and the different acquisition condition I3 as an image Im4 in a recording device (not shown) (step S22).
[0132] If the judgment is NO in steps S16 and S20 or if the processing of step S22 is completed, the control unit 31 proceeds to the next step S23, and if the images acquired based on acquisition conditions I1 to I3 are of the same observation site, the control unit 31 superimposes a display of support information from the support information generation unit 36 based on the image analysis results of images Im2 to Im4 on the image Im1 displayed on the monitor 5.
[0133] In FIG. 15, the setting of the acquisition condition I3 by steps S18 and S21 is performed only once in any of the steps, but steps S16 to S22 may be repeated until the amount of information is neither increased nor decreased. However, such repetition may take a very long time, and the conditions cannot be determined quickly, so the process may be terminated in a specific situation. The better of the two conditions may be used. In this way, by executing an image processing method including an imaging step of capturing images under a plurality of different imaging conditions to obtain imaging results, and an imaging condition changing step of changing the third imaging condition based on the difference in the amount of information obtained in the comparison result by a comparison step of comparing the plurality of imaging results under the different imaging conditions, it is possible to present highly accurate support information using an image captured under good conditions.
[0134] In this way, the present embodiment can also provide the same effects as the first embodiment.
[0135] (Third embodiment) FIG. 16 is a block diagram showing the third embodiment.
[0136] The endoscopic system according to the third embodiment can be applied to many endoscopic systems, including a system using an endoscope for inspection such as a colonoscope and a system using an endoscope for surgical operations such as a laparoscope. FIG. 16 illustrates an endoscopic system 1 that is intended as a laparoscopic surgery system.
[0137] As shown in FIG. 16, the system mainly comprises an endoscope 2 (lapascope) that captures images of the inside of a body cavity of a subject P and outputs an image capture signal, a video processor 3 that is connected to the endoscope 2 and controls the operation of the endoscope 2, as well as acquiring an image capture signal relating to the subject captured by the endoscope 2 and performing predetermined image processing on the image capture signal, a light source device 4 that is provided within the video processor 3 and supplies a predetermined illumination light for irradiating the subject, a monitor 5 that displays an observation image corresponding to the image capture signal, and a navigation device 30 connected to the video processor 3. Although the type of endoscope 2 is different in a system using an inspection endoscope, the other components are similar to the example shown in FIG. 16.
[0138] Furthermore, the configurations of each component in the endoscopic system 1 of the third embodiment, namely, the endoscope 2, the video processor 3, the light source device 4, the monitor (display) 5, and the navigation device 30, are the same as those of the first embodiment, so detailed explanations will be omitted here.
[0139] In the endoscope system 1 of the third embodiment, for example, in the case of a system using an inspection endoscope, the navigation device 30 outputs to the monitor (display) 5 an image in which a lesion site is marked with high accuracy.
[0140] Specifically, in the case of a navigation device 30 in an endoscope examination system using a colonoscope, as shown in FIG. 16, based on each piece of complete image information (image information for display+image information for analysis) provided from the video processor 3 as described above, an image in which an area thought to be a lesion site is marked with high accuracy is output to a monitor (display) 5 and provided to the surgeon as navigation information.
[0141] On the other hand, in the case of a system using a surgical endoscope, a navigation device 30 outputs to a monitor (display) 5 an image presenting information useful for the procedure.
[0142] Specifically, in the case of a navigation device 30 in a surgical endoscopic system using a laparoscope, as shown in FIG. 16, based on each piece of complete image information (image information for display+image information for analysis) provided by the video processor 3 as described above, information such as the position of the tumor, the resection area, the position of major blood vessels, etc. is output to a monitor (display) 5 and provided to the surgeon as navigation information.
[0143] Like the endoscopic system of the third embodiment, in an endoscopic system 1 using various endoscopes, the present invention provides image information for analysis for the navigation device 30 in addition to image information for display as image information provided to the navigation device 30 from the video processor 3 as described above, and recognition processing is performed in the navigation device 30 using image information without any loss, so that useful navigation information (support information) can be provided to the surgeon in any endoscopic system 1.
[0144] Furthermore, as described above, the endoscopic system of this third embodiment has been given as an example of an endoscopic system for inspection and an endoscopic system for surgical operations, but the endoscopic system of this third embodiment is not limited to these and may be applied to endoscopic systems using other types of endoscopes.
[0145] Furthermore, among the techniques described herein, many of the controls and functions described mainly in the flowcharts can be set by a program, and the above-mentioned controls and functions can be realized by having a computer read and execute the program. The program can be recorded or stored in whole or in part as a computer program product on a portable medium such as a flexible disk, a CD-ROM, or a non-volatile memory, or on a storage medium such as a hard disk or a volatile memory, and can be distributed or provided at the time of product shipment or via a portable medium or a communication line. A user can easily realize the image processing device of this embodiment by downloading the program via a communication network and installing it on a computer, or by installing it on a computer from a recording medium.
[0146] The present invention is not limited to the above-mentioned embodiments as it is, and the components can be modified and embodied in the implementation stage without departing from the gist of the invention. In addition, various inventions can be formed by appropriate combinations of the multiple components disclosed in the above-mentioned embodiments. For example, some of the components shown in the embodiments may be deleted. Furthermore, components across different embodiments may be appropriately combined. Here, the medical application example has been described, but it goes without saying that the invention can be applied to consumer, industrial, and industrial equipment. For example, a navigation device can be rewritten as a device that detects abnormalities in the industrial and security fields as well as in the medical field, and support information can be rewritten as information that encourages awareness. In industrial applications, it can be used to assist in judging the quality of things flowing on a factory line or during work using an in-process camera, or as a guide to induce awareness when monitoring with a wearable camera or a robot camera, and similarly, it can be used to judge obstacles with an in-vehicle camera. Consumer cameras can also have various guide applications. Observation by switching light sources and image processing is also known in microscopes, and the application of the present application is effective.< / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw> < / raw>
Claims
1. an acquisition condition designation unit that sets, for an image acquisition unit that is capable of acquiring a first image based on display acquisition conditions at a frame rate according to the display acquisition conditions for acquiring an image suitable for display, and acquiring a mixture of the first image and a second image or a third image based on analysis acquisition conditions for acquiring an image suitable for image analysis at a frame rate lower than the frame rate according to the display acquisition conditions, a first acquisition condition including the display acquisition conditions, a second acquisition condition including the display acquisition conditions and the analysis acquisition conditions, and a third acquisition condition including the analysis acquisition conditions different from the display acquisition conditions and the analysis acquisition conditions of the second acquisition conditions, wherein a wavelength of illumination light for an imaged object is different from the analysis acquisition conditions of the second acquisition conditions; an image analysis unit that performs image analysis on the first to third images acquired by the image acquisition unit; a support information generating unit that generates support information based on a result of the image analysis by the image analyzing unit; a control unit that controls switching from the first acquisition condition to the second acquisition condition according to the image analysis result, and controls switching from the second acquisition condition to the third acquisition condition based on the accuracy of the image analysis result in the image analysis unit, thereby increasing the accuracy of the image analysis result for the third image acquired according to the acquisition condition for analysis included in the third acquisition condition, more than the accuracy of the image analysis result for the second image acquired according to the acquisition condition for analysis included in the second acquisition condition; 13. An image processing apparatus comprising:
2. The control unit controls switching from the second acquisition condition to the first acquisition condition at a predetermined timing.
2. The image processing device according to claim 1,
3. The image acquisition unit sets at least one of a first parameter for controlling at least one of an optical system and an image sensor of the imaging device, a second parameter for controlling illumination light for an imaging target of the imaging device, and a third parameter for signal processing of the first to third images, based on at least one of the display acquisition conditions and the analysis acquisition conditions.
2. The image processing device according to claim 1,
4. An image processing method in an image processing device including an acquisition condition designation unit, an image analysis unit, a support information generation unit, and a control unit, comprising: the acquisition condition designation unit acquires a first image based on the display acquisition conditions at a frame rate according to the display acquisition conditions for acquiring an image suitable for display, and acquires a mixture of the first image and a second image or a third image based on the analysis acquisition conditions for acquiring an image suitable for image analysis at a frame rate lower than the frame rate according to the display acquisition conditions; and sets a first acquisition condition including the display acquisition conditions, a second acquisition condition including the display acquisition conditions and the analysis acquisition conditions, and a third acquisition condition including the analysis acquisition conditions different from the display acquisition conditions and the analysis acquisition conditions of the second acquisition conditions, the third acquisition condition including the analysis acquisition conditions having a wavelength of illumination light for an imaged object different from the analysis acquisition conditions of the second acquisition conditions; The image analysis unit performs image analysis on the first to third images acquired by the image acquisition unit to obtain image analysis results; the support information generation unit generates support information based on the image analysis result, The control unit controls switching from the first acquisition condition to the second acquisition condition according to the image analysis result, and controls switching from the second acquisition condition to the third acquisition condition based on the accuracy of the image analysis result in the image analysis unit, thereby making the accuracy of the image analysis result for the third image acquired according to the acquisition condition for analysis included in the third acquisition condition higher than the accuracy of the image analysis result for the second image acquired according to the acquisition condition for analysis included in the second acquisition condition.
13. An image processing method comprising:
5. the display acquisition conditions include information that enables acquisition of an image suitable for the display at a rate equal to or higher than a predetermined frame rate; 2. The image processing device according to claim 1,
6. the analysis acquisition conditions include information that enables limiting the wavelength band of the illumination light to a predetermined band; 4. The image processing device according to claim 3.
7. the display acquisition condition includes information for causing an image processing unit that performs signal processing on the first image acquired by the image acquisition unit to perform signal processing for display; The analysis acquisition condition includes information for not performing signal processing for display on the second image or the third image acquired by the image acquisition unit with respect to the image processing unit.
2. The image processing device according to claim 1,
8. The control unit determines the third acquisition condition based on a comparison between accuracy of an image analysis result of the image analysis unit for the first image acquired based on the first acquisition condition and accuracy of an image analysis result of the image analysis unit for the second image acquired based on the second acquisition condition.
2. The image processing device according to claim 1,
9. The first and second acquisition conditions are specified conditions, The third acquisition condition is an adaptively changing condition.
9. The image processing device according to claim 8,
10. The control unit determines the third acquisition condition based on a comparison between a value included in the image analysis result of the image analysis unit and a predetermined reference value.
9. The image processing device according to claim 8,
11. An acquisition condition storage unit that stores the first and second acquisition conditions and stores the third acquisition condition when the accuracy of the image analysis result by the image analysis unit corresponding to the second acquisition condition is higher than the accuracy of the image analysis result by the image analysis unit corresponding to the first acquisition condition, and the third acquisition condition when the accuracy of the image analysis result by the image analysis unit corresponding to the second acquisition condition is lower than the accuracy of the image analysis result by the image analysis unit corresponding to the first acquisition condition. The control unit determines the third acquisition condition to be read from the acquisition condition storage unit depending on whether the accuracy of the image analysis result has increased or decreased.
2. The image processing device according to claim 1,
12. A navigation method using an image processing device including an acquisition condition designation unit, an image analysis unit, and a control unit, comprising: The acquisition condition specification unit is a first image based on the display acquisition conditions at a frame rate according to the display acquisition conditions for acquiring an image suitable for display, and a second image or a third image based on the analysis acquisition conditions for acquiring an image suitable for image analysis at a frame rate lower than the frame rate according to the display acquisition conditions, the first image being acquired based on the display acquisition conditions; setting second acquisition conditions including the acquisition conditions for display and the acquisition conditions for analysis for the image acquisition unit; setting a third acquisition condition including an analysis acquisition condition different from the display acquisition condition and the analysis acquisition condition of the second acquisition condition, the third acquisition condition including an analysis acquisition condition in which a wavelength of illumination light for an image pickup target is different from the analysis acquisition condition of the second acquisition condition; The image analysis unit performing image analysis on the first to third images acquired by the image acquisition unit to obtain image analysis results; The control unit controls switching from the first acquisition condition to the second acquisition condition according to the image analysis result, and controls switching from the second acquisition condition to the third acquisition condition based on the accuracy of the image analysis result in the image analysis unit, thereby making the accuracy of the image analysis result for the third image acquired according to the acquisition condition for analysis included in the third acquisition condition higher than the accuracy of the image analysis result for the second image acquired according to the acquisition condition for analysis included in the second acquisition condition. A navigation method comprising:
13. an endoscope having an illumination unit and an imaging unit capable of acquiring a first image based on display acquisition conditions at a frame rate according to the display acquisition conditions for acquiring an image suitable for display, and acquiring a mixture of the first image and a second image or a third image based on analysis acquisition conditions for acquiring an image suitable for image analysis at a frame rate lower than the frame rate according to the display acquisition conditions; a video processor that causes the endoscope to acquire the first to third images based on at least one of the acquisition conditions for display and the acquisition conditions for analysis; an acquisition condition designation unit that sets first acquisition conditions including the acquisition conditions for display, second acquisition conditions including the acquisition conditions for display and the acquisition conditions for analysis, and third acquisition conditions including the acquisition conditions for analysis that are different from the acquisition conditions for display and the acquisition conditions for analysis of the second acquisition conditions, and in which a wavelength of illumination light for an image capture target is different from the acquisition conditions for analysis of the second acquisition conditions; an image analysis unit that performs image analysis on the first to third images acquired by the video processor; a support information generation unit that generates support information based on an image analysis result of the image analysis unit; and a control unit that controls switching from the first acquisition condition to the second acquisition condition according to the image analysis result and controls switching from the second acquisition condition to the third acquisition condition based on an accuracy of the image analysis result in the image analysis unit, thereby increasing accuracy of the image analysis result for the third image acquired according to the acquisition conditions for analysis included in the third acquisition condition compared to accuracy of the image analysis result for the second image acquired according to the acquisition conditions for analysis included in the second acquisition condition; An endoscope system comprising:
Citation Information
Patent Citations
Electronic endoscope apparatus, endoscopic image generation method, and electronic endoscope system
JP2013000466A
Image processing device, image processing method, image processing program, and endoscope system
JP2016019569A
Medical image processing apparatus, endoscope apparatus, diagnosis support apparatus, and medical service support apparatus
JP2019042156A
Endoscope device, and measurement method
WO2012161244A1
Information processing device, assist system, and information processing method
WO2018163644A1
