Medical camera system, medical equipment and determination method

Abstract

To determine a peculiar structure or state in the inside of a subject such as a posterior nasal nerve.SOLUTION: A medical camera system (100) includes: a camera module (102) that acquires an image; and a control unit (104) that is operably connected to the camera module (102) and receives and processes an output image. The control unit (104) detects at least one risk structure in the acquired image, detects existence of at least one treatment device (200) and / or a guide laser beam in the acquired image, estimates, in real time, thermal diffusion in a horizontal direction generated during actuation of the treatment device (200) on the basis of setting and / or operation status of the treatment device (200), evaluates a distance or position of a predetermined isothermal line with respect to the detected risk structure, and causes at least one predetermined response when the distance between the detected risk structure and the isothermal line falls below a predetermined threshold.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a medical camera system for capturing an image, processing the image, and determining a specific structure or state in a subject, a medical facility including such a medical camera system, and a determination method.

Background Art

[0002] In certain applications in the medical field, it is beneficial to be able to automatically and real-time locate a specific structure or state within a subject of a medical procedure in order to facilitate the work of medical professionals by making the corresponding information available to medical professionals such as surgeons performing the medical procedure through a suitable display device.

[0003] One specific application can be in Ear-Nose-Throat medicine (ENT), specifically, the identification of the Posterior Nasal Nerve (PNN) during an ENT procedure.

[0004] One of the symptoms that frequently occur in the field of otolaryngology is Chronic Rhinosinusitis (CRS), which is characterized by inflammation of the nasal cavity and paranasal sinuses that can last for at least 12 weeks or more. This disease is estimated to affect approximately 14% of the world's population, that is, hundreds of millions of people. In the United States alone, for example, approximately 16 million adults suffer from this symptom, and it is estimated that adults have a higher prevalence. The symptoms that characterize CRS are nasal congestion, facial pain or pressure, nasal discharge, and a decrease in the sense of smell. For the diagnosis of chronic rhinosinusitis, a specialized ENT doctor, who is often board-certified in otolaryngology, needs to appropriately visualize the inside of the nasal passages and paranasal sinuses. Typically, an ENT doctor recommends and performs a nasal endoscopy.

[0005] However, accurate identification of suspicious areas and nerves can be difficult based on optical information alone, and thus, additional imaging examinations such as Computed Tomography (CT) imaging or Magnetic Resonance Imaging (MRI) may be performed to detect any structural abnormalities or chronic inflammation of the paranasal sinuses.

[0006] In particular, during this past year, interest in the treatment of the posterior nasal nerves has been increasing, and conventional surgical methods are increasingly shifting to less invasive treatment with alternative methods. Nevertheless, several challenges remain in the field of treatment area detection.

[0007] The treatment of the posterior nasal nerves (PNN) in chronic rhinosinusitis (CRS) is essential in that it can help relieve the pain of patients suffering from this condition and improve their overall quality of life. Despite the progress of imaging and endoscopic examination techniques, the diagnosis of chronic rhinosinusitis (CRS) remains a difficult task for ENT doctors. In particular, the detection of the posterior nasal nerves remains difficult and often requires treatment at multiple sites or regions, with the risk of over-treatment and unnecessary damage.

[0008] There are several methods for identifying the PNN, including imaging, nasal endoscopy (flexible or rigid), nerve stimulation, and fluoroscopy, but these techniques are rarely employed during the procedure and can be somewhat cumbersome. Using alternative methods such as Radio Frequency - High Frequency (RF - HF), ablation, cryoablation, laser therapy, or other approaches to quickly locate and treat the position of the PNN remains a challenge during procedures in the clinic.

[0009] In particular, ENT doctors often face challenges when identifying and treating delicate anatomical structures such as the posterior nasal nerves. Current procedures may lack the accuracy and real-time assistance that can lead to suboptimal treatment outcomes.

Prior Art Documents

Patent Documents

[0010]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Summary of the Invention

Problems to be Solved by the Invention

[0011] Accordingly, an object of the present invention is to provide a medical camera system, medical equipment, and determination method capable of determining a specific structure or state in a subject such as the posterior nasal nerve.

Means for Solving the Problems

[0012] For this purpose and to achieve the above-specified purpose, a medical camera system for capturing an image, processing the image, and determining a specific structure or state within a subject includes a camera module configured to acquire an image, and a control unit operably coupled to the camera module and configured to receive and process an output image. The control unit is configured to detect at least one risk structure within the acquired image, detect the presence of at least one treatment device and / or a guiding laser beam within the acquired image, estimate in real time a lateral heat diffusion occurring during operation of at least one treatment device based on the settings and / or operating status of at least one treatment device, evaluate the distance or position of a predetermined isotherm relative to at least one detected risk structure, and be configured to cause at least one predetermined response when the distance between at least one detected risk structure and the isotherm falls below a predetermined threshold.

[0013] In this specification, the acquisition of an image by the camera module and the subsequent processing of the image by the control unit may be performed at a given frame rate on the order of several tens of Hz, and other operating modes, such as a freeze frame feature triggered by an operator, may also be provided to enable the acquired image to be examined over a long period of time.

[0014] In the above introduction, the identification of the posterior nasal nerve (PNN) during ENT procedures is specifically mentioned. However, it is clear that the medical camera system according to the present invention may also be used to capture and process images to determine specific structures or states within a subject in different medical fields and in the context of different treatment and diagnostic procedures. In particular, the system according to the present invention enables accurate targeting of anatomical regions, for example, accurate identification of nerves, and performance of diagnostics and treatments, such as laser therapy or ablation, with minimal energy consumption, thereby reducing the thermal stress on the subject's tissue within the targeted area of the procedure. This integrated approach ensures the efficiency and success of medical interventions.

[0015] In particular, for the detection of at least one risk structure in the acquired image and the detection of the presence of at least one treatment device and / or the guiding laser beam in the acquired image, it should be noted that the control unit may rely on different techniques and algorithms, in particular, artificial intelligence (AI) methods such as artificial neural networks. By employing suitable training data, such neural networks can identify risk structures in the acquired image with high reliability as well as high accuracy and resolution. Then, in order to evaluate the distance or position of a predetermined isotherm with respect to the risk structure, by estimating the lateral heat diffusion generated by at least one treatment device, it becomes possible to immediately provide feedback to medical experts about potentially dangerous or harmful scenarios where the risk structure may be exposed to excessive heat and damage the corresponding tissue.

[0016] In particular, the camera module may be configured to capture images via multiple optical modalities. Preferably, the first modality refers to standard white light and / or the second modality refers to near-infrared light. By relying on both white light and infrared light, it becomes possible to read out images with an increased amount of information regarding the structures to be detected and identified, in order to further improve the accuracy and reliability of the medical camera system according to the present invention. As a result, the images output by the camera module may thus comprise information from both the standard white light and the near-infrared light spectrum.

[0017] In order to enable the medical camera system according to the present invention to be employed in standard medical procedures and environments, this medical camera system may further comprise an endoscope in which the endoscope camera head part or the camera module is provided together. This makes it possible to improve the handling of the medical camera system according to the present invention for skilled medical experts who are accustomed to using endoscopes and endoscope camera head parts.

[0018] In order to enable information to be output to medical professionals using the medical camera system according to the present invention, the medical camera system may further comprise a warning system that gives visual and / or audible warnings, and at least one predetermined response may comprise a control unit that causes the warning system to output a warning. Such a warning system that gives visual and / or audible warnings may be embodied in a known manner by sounds and visual effects that notify medical professionals about the corresponding state of the risk structure in the acquired image with respect to a predetermined isotherm. In one example, when the isotherm enters the vicinity of a predetermined location of the corresponding risk structure, an audible or visual signal may be provided. The warning system may also be integrated with the display unit of the medical camera system according to the present invention, and the acquired image may be presented to the medical professional using the medical camera system by, for example, an overlay that displays additional information derived by the control unit.

[0019] In addition or alternatively, at least one predetermined response may comprise a control unit that causes at least one treatment device to stop. In this way, when a state is detected in which the isotherm discussed above approaches the identified risk structure too closely, further introduction of heat into the subject through the operation of the treatment device may be automatically and immediately stopped in order to prevent structural damage to the tissue of the risk structure. The embodiments discussed above with respect to the warning system and automatic stop of at least one treatment device may also be combined such that, for example, when the isotherm reaches a first threshold distance from the risk structure, visual and / or audible warnings are output to the medical professional handling the medical camera system, while when the isotherm then moves beyond a second threshold distance that is shorter than the first threshold distance, automatic stop of at least one treatment device is triggered as an automatic emergency measure.

[0020] In order to estimate the lateral heat diffusion occurring during the operation of at least one treatment device, at least one setting and / or operating status of the treatment device to be evaluated by the control unit may include at least one of the energy level and / or waveform provided by the energy supply of the treatment device, the energy applied by the treatment device, and the operating status of the treatment device. Thus, from these readily available information and data, the control unit can estimate the lateral heat diffusion based on a predetermined algorithm and / or artificial intelligence method, which enables an accurate estimation of the lateral heat diffusion.

[0021] Furthermore, the control unit may also be configured to detect the tissue type and / or tissue structure within the subject in order to optimize the estimation of heat diffusion. For example, certain tissue types and / or tissue structures such as adipose tissue, arteries, veins, etc. may exhibit different thermal conductivities from other tissue types and / or tissue structures, which may be taken into account by the control unit during the estimation of heat diffusion.

[0022] As already briefly described above, the medical camera system according to the present invention may further include a display unit, and the control unit may further be configured to provide the acquired image together with an enhanced visualization overlay to the display unit. Such an enhanced visualization overlay may include any information that may be useful to a medical professional handling the medical camera system, such as highlighting of identified risk structures and / or treatment devices, drawing of estimated isotherms within the image, output of warnings, and / or color-coding of the current status regarding the distance between the risk structure and the isotherm.

[0023] Also, as already described above, the control unit of the medical camera system according to the present invention may be configured to employ artificial intelligence, in particular, to generate an extended visualization overlay and / or to detect tissue types and / or structures, as well as risk structures and treatment devices within the acquired images. For this purpose, in particular, known techniques in artificial intelligence and machine learning may be employed, in the context of image processing and pattern recognition, and in that case, a vast existing available dataset of medical images and videos may be used as training data.

[0024] The medical camera system according to the present invention may further comprise a graphical user interface for adjusting, in particular, colors, shapes, and other visual elements to display patient information and imaging data on the display unit of the system. In a highly integrated embodiment, the medical camera system may comprise, for example, a touch screen display that displays the acquired images together with additional extended information, and may also serve as a graphical user touch interface, whereby a medical professional handling the medical camera system may be able to adjust certain parameters of the display, such as the magnification factor, color scheme, and general information output settings.

[0025] The present invention also relates to a medical facility comprising a medical camera system of the type described above and at least one treatment device, wherein the control unit of the treatment device is integrated with or operably coupled to the control unit of the medical camera system. By integrating at least one treatment device with the medical camera system and their control units, it is possible to immediately provide the data necessary for the corresponding control unit to actually perform the estimation. In particular, the estimation of lateral heat diffusion based on the settings and / or operating status of the treatment device becomes even easier. Also, different techniques regarding feedback between the medical camera system and the treatment device, such as automatically stopping the treatment device when an undesirable condition involving the detected risk structure and the estimated isotherm is discovered, are facilitated.

[0026] In particular, at least one treatment device of the medical equipment according to the present invention may comprise at least one of a monopolar or bipolar HF applicator for ablation, a monopolar or bipolar cutting device, a cutting device using ultrasonic energy, a cutting device using a combination of bipolar HF and ultrasonic energy, and a laser fiber. All devices of this type, during their use, dissipate heat into the subject in a predictable manner, such that the control unit of the medical camera system can perform a reliable estimation of the lateral heat dissipation of the heat generated by the treatment device during use.

[0027] In certain embodiments of the medical equipment according to the present invention, the control unit of the medical camera system may in particular be configured to adjust the energy output of at least one treatment device, based on an energy minimization scheme, whereby the treatment performed by a medical professional using the medical equipment is minimized, energy is conserved, and the introduction of heat into the tissue of the subject is reduced.

[0028] As already discussed previously, the medical camera system and the medical equipment according to the present invention may in particular be used for endoscopic applications, in particular for diagnostic ENT applications.

[0029] Furthermore, the present application also relates to a determination method executed by a control unit of a medical camera system according to the present invention. In this determination method, the control unit of the medical camera system includes steps of: acquiring an image by a camera module; detecting at least one risk structure in the acquired image; detecting the presence of at least one treatment device and / or a guiding laser beam in the acquired image; estimating in real time the lateral heat diffusion occurring during the operation of at least one treatment device based on the setting and / or operation status of at least one treatment device; evaluating the distance or position of a predetermined isotherm with respect to at least one detected risk structure; and causing at least one predetermined response when the distance between at least one detected risk structure and the isotherm is below a predetermined threshold value.

[0030] It should be further pointed out that the isotherm may be predefined, for example, to be about 40°C. It should also be pointed out that the features described in the context of a specific embodiment of the medical camera system and medical equipment according to the present invention can also be understood as related to the method steps of the corresponding embodiment of the determination method according to the present invention. For example, in a similar way as described above in the context of the medical camera system according to the present invention, the determination method according to the present invention may further include outputting visual and / or audible warnings as a predetermined response, detecting the tissue type and / or tissue structure within the subject, providing an image acquired together with an extended visualization overlay, etc.

[0031] Further features and advantages of the present invention will become more apparent through the following description of its embodiments when considered in conjunction with the accompanying drawings. In particular, the following drawings are shown.

Brief Description of the Drawings

[0032]

Figure 1

Figure 2

DETAILED DESCRIPTION OF THE INVENTION

[0033] In Figure 1, the medical equipment according to the present invention is shown in a schematic diagram and is generally represented by reference numeral 10. The medical equipment 10 includes a medical camera system 100 and a treatment device 200.

[0034] The medical camera system 100 includes a camera module 102, and the camera module 102 is configured to capture images via a plurality of optical modalities such as standard white light and near-infrared light at one or more selectable frame rates. The images acquired by the camera module 102 are output to a control unit 104, and the control unit may be formed by a known device such as a microprocessor having a memory unit useful for storing software code and the data necessary to perform the method steps discussed later.

[0035] The medical camera system 100 further includes a display unit 106 and a warning system 108 for visual and / or audible warnings. The display unit 106 and the warning system 108 may be integrated with each other, and may also provide a graphical user interface 110. The graphical user interface 110 may be used, for example, to input input commands for adjusting the displayed colors, shapes, and other visual elements. Therefore, the display unit 106, the warning system 108, and the graphical user interface 110 may be formed by a single touch screen unit further including a speaker if desired.

[0036] Furthermore, it should be noted that the camera module 102 may be provided in the endoscope camera head portion or the endoscope in order to facilitate use during a medical procedure for capturing an image inside a subject.

[0037] On the other hand, the treatment device 200 also includes a control unit 204 in addition to the operation section 202, and the operation section may be suitable for ablation, cutting, providing a laser beam, or similar applications. The control unit 204 of the treatment device can be operably connected to or integrated with the control unit 104 of the medical camera system 100 so as to exchange data and commands between the medical camera system 100 and the treatment device 200. In particular, the control unit 104 of the medical camera system 100 may receive data regarding the energy level and waveform provided by the energy source 206 of the treatment device 200, and / or the energy applied by the treatment device 200, and the operating status of the treatment device 200 from the control unit 204 of the treatment device 200. Based on this data, the control unit 104 of the camera system 100 estimates the lateral heat diffusion occurring during the operation of the treatment device 200 according to the determination method described below with reference to FIG. 2, and can evaluate the distance or position of a predetermined isotherm as a characteristic of the lateral heat diffusion with respect to at least one detected risk structure.

[0038] According to the flowchart shown in FIG. 2, in step S1, the camera module 102 of the medical camera system 100 acquires an image of a region of interest of the subject, for example, at a certain frame rate (repetition rate) or in response to a request by a medical expert handling the equipment. The image is transferred to the control unit 104 operably coupled to the camera module 102 adapted to process the image, for example, using artificial intelligence techniques and / or dedicated algorithms.

[0039] In this context, in step S2, at least one risk structure is detected within the acquired image. For example, a nerve such as the posterior nasal nerve may be identified. At the same time, the presence of the treatment device 200 is detected within the image acquired in step S3, based on, for example, pattern recognition using artificial intelligence (AI) and / or other suitable algorithms.

[0040] Based on the data provided by the control unit 204 of the treatment device 200 regarding the settings and / or operating status of the treatment device 200, the control unit 104 of the medical camera system 100, in step S4, estimates in real time the lateral heat diffusion generated by the treatment device 200, including, for example, deriving at least one isotherm representing 40°C. This step S4 may be involved in the detection and identification of the tissue type and / or tissue structure within the subject in order to optimize the estimation of the heat diffusion.

[0041] Based on these estimated values, in step S5, the distance or position of a predetermined isotherm with respect to at least one detected risk structure is evaluated. If it is detected that the evaluated distance between at least one risk structure and the isotherm is below a predetermined threshold, in step S6, at least one predetermined response is triggered, such as, for example, outputting a visual warning and / or an audible warning. Thereafter, this medical method is repeated starting from step S1.

[0042] Furthermore, the medical method according to the present invention, as described by the above steps S1 to S6, is implemented in the context of a visual output of a captured image with an extended information overlay on the display unit 106 of a medical camera system, and it should be noted that in the medical camera system, both the acquired image and additional information derived from the acquired image and from other sources are displayed for the medical professional handling the equipment according to the present invention. In this medical camera system, the output of a warning by the warning system 108 may also be included on the display unit 106, for example, by a warning sign, a flashing light, or similar means that require the attention of the medical professional regarding the current state, in particular, the distance between the risk structure and the isotherm.

Explanation of Reference Numerals

[0043] 10 Medical equipment 100 Medical camera system 102 Camera module 104 Control unit 106 Display unit 108 Warning system 110 Graphical user interface 200 Treatment device 202 Operation classification 204 Control unit 206 Energy source

Claims

1. A medical camera system for capturing an image and processing the image to determine a specific structure or state within a subject, comprising: a camera module configured to acquire an image; and a control unit operably coupled to the camera module and configured to receive and process an output image; wherein the control unit is configured to detect at least one risk structure in the acquired image; detect the presence of at least one treatment device and / or a guiding laser beam in the acquired image; estimate in real time a lateral heat diffusion occurring during operation of the at least one treatment device based on the setting and / or operating status of the at least one treatment device; evaluate a distance or position of a predetermined isotherm with respect to the at least one detected risk structure; and cause at least one predetermined response when the distance between the at least one detected risk structure and the isotherm falls below a predetermined threshold. A medical camera system configured as such.

2. The camera module is configured to capture an image via a plurality of optical modalities, wherein a first modality of the plurality of optical modalities refers to standard white light, and a second modality of the plurality of optical modalities refers to near-infrared light. The medical camera system according to claim 1.

3. The medical camera system according to claim 1, further comprising an endoscope camera head portion or an endoscope provided with the camera module.

4. The medical camera system according to claim 1, further comprising a warning system configured to provide a visual warning and / or an audible warning, wherein the at least one predetermined response includes causing the control unit to output a warning to the warning system.

5. The medical camera system according to claim 1, wherein the at least one predetermined response includes causing the control unit to cause the at least one treatment device to stop.

6. The setting and / or operating status of the at least one treatment device includes at least one of an energy level and / or waveform provided by an energy source of the treatment device, energy applied by the treatment device, and the operating status of the treatment device. The medical camera system according to claim 1.

7. ​ The medical camera system according to claim 1, wherein the control unit is further configured to detect a tissue type and / or tissue structure in the subject to optimize the estimation of the heat diffusion.

8. Further comprising a display unit, The medical camera system according to claim 1, wherein the control unit is further configured to provide the acquired image together with an augmented visualization overlay to the display unit.

9. The medical camera system according to claim 8, wherein the control unit is configured to employ artificial intelligence to create the augmented visualization overlay and / or to detect the type and / or structure of the tissue.

10. The medical camera system according to claim 8, further comprising a graphical user interface for adjusting colors, shapes, and other visual elements to display patient information and imaging data.

11. A medical facility, The medical camera system according to claim 1, At least one treatment device, Comprising, A medical facility, wherein a control unit of the treatment device is integrated with or operably coupled to a control unit of the medical camera system.

12. The medical facility according to claim 11, wherein the at least one treatment device comprises at least one of a monopolar or bipolar HF applicator for ablation, a monopolar or bipolar cutting device, a cutting device using ultrasonic energy, a cutting device using a combination of bipolar HF and ultrasonic energy, and a laser fiber.

13. The medical facility according to claim 11, wherein the control unit of the medical camera system is configured to adjust the energy output of the at least one treatment device based on an energy utilization minimization scheme.

14. A determination method executed by a control unit of a medical camera system, The control unit, Obtaining an image by the camera module, Detecting at least one risk structure in the acquired image, Detecting the presence of at least one treatment device and / or a guiding laser beam in the acquired image, estimating in real time a lateral heat diffusion occurring during operation of the at least one treatment device based on a setting and / or an operating status of the at least one treatment device; evaluating a distance or a position of a predetermined isotherm with respect to the at least one detected risk structure; causing at least one predetermined response when the distance between the at least one detected risk structure and the isotherm falls below a predetermined threshold; A determination method for performing the above steps.

Images