DEVICE AND METHOD FOR IMAGE-BASED USER SUPPORT

Description

[0001] The invention relates to a device and a method for image-guided support of a user of a (medical) instrument that is designed to apply a treatment medium to a tissue surface.

[0002] Instruments that allow a user to apply a treatment medium, such as low-temperature plasma, freehand and over a small area to a tissue surface are generally known. In these instruments, tissue surfaces such as skin, mucous membranes, or wounds are treated by the user applying a treatment medium, such as cold plasma (non-thermal plasma) or low-energy argon plasma, to the tissue surface.

[0003] This treatment of the tissue surface results in no or only a minimal thermal effect within the tissue, thus preventing coagulation or carbonization. The local temperature on the tissue surface remains below the denaturation temperature of proteins (approximately 60-70°C). The treated tissue surface does not change visually, meaning the user cannot discern which areas of the tissue surface have already been treated. Furthermore, the user cannot determine the dosage or exposure time of individual areas within the treatment zone.

[0004] Therefore, the user must remember as precisely as possible in which areas the treatment medium was applied and for how long.

[0005] This places increased demands on the user's concentration. Furthermore, the assessment of the duration for which the treatment medium was applied to a specific area is subject to the user's subjective perception. Additionally, the lack of visible effects on the tissue may tempt the user to repeatedly apply the treatment to already treated areas, thus resulting in an overdose.

[0006] However, it can also happen that the user does not treat certain areas at all or only for too short a time, resulting in an underdose.

[0007] This can lead to significant variations in treatment quality and the evenness with which the treatment medium is applied to the tissue surface being treated. Furthermore, treatment results are difficult to reproduce and cannot be objectively verified and / or documented.

[0008] There are approaches that attempt to improve the uniformity with which the treatment medium is applied to the tissue surface using newly developed, large-area plasma sources. For example, German patent DE 10 2014 220 488 A1 proposes a large-area plasma source for cold atmospheric pressure plasma, which can treat a relatively large area (such as approximately 100 cm²). Particularly when treating areas that are not simple shapes (such as rectangular, square, elliptical, or circular) but rather complex and / or irregular, it can happen that areas outside the intended treatment area are unintentionally treated.

[0009] Other known approaches propose automating the application of the treatment medium. For example, German patent DE 10 2010 011 643 A1 describes a plasma source that is automatically moved relative to the tissue surface by a motion device controlled by a control unit. However, this directly interferes with the treatment process. Furthermore, complete automation of the application process is development-intensive and therefore costly.

[0010] In "In the article "Concept for Improved Handling Ensures Effective Contactless Plasma Treatment of Patients with kINPen® MED", Applied Sciences, Vol. 10, No. 17, Art. No. 17, Jan. 2020, doi: 10.3390 / app10176133, a sensor system is proposed that determines the distance of the instrument to the treatment area and illuminates the area where the plasma is expected to be applied. However, this information is only displayed to the user for the current time. The user must therefore still remember which areas the treatment medium was applied to and for how long.

[0011] Furthermore, WO 2011 / 044248 A2 and US 2011 / 082451 A1 disclose a system in which, during an ablation operation, the surgical area is monitored using a video camera, and the quality of the lesions created in the tissue during the ablation is evaluated by a processor. Color markers are then superimposed onto the live images of the surgical area according to the determined quality.

[0012] German patent DE 10 2015 100 927 A1 describes an assistance device for providing imaging support to a surgeon during a surgical procedure. The assistance device comprises a camera and an image processing unit, the image processing unit being configured to recognize the medical instrument being used without the need for markers attached to the instrument. Furthermore, EP 3 718 496 A1 describes a device in which energy is supplied to a probe in contact with tissue in a body cavity to ablate the tissue. During the energy input, signals are received from a position sensor in the probe, indicating the probe's position within the cavity. These signals are processed to derive 3D position coordinate points corresponding to the probe's position at a series of time points when the energy was applied.RU 2 582 213 C1 describes a laser surgical system consisting of a CO2 laser with pilot beam guidance, a video monitoring device, a surgical field, a device for adapting the laser beam to a video monitoring device, and a scanning system connected to the optical output of the device for the CO2 laser and the optical input of the adaptation device. Furthermore, WO 2017 / 160808 A1 describes a device for determining the orientation of a distal end of a medical instrument, such as the electrode-tissue alignment of a radiofrequency ablation catheter.

[0013] EP 3 346 807 B1 describes an information processing device with an application position calculation section which is configured to calculate the application position of plasma on a processing target object based on captured image data of the processing target object when plasma is applied to the processing target object from an atmospheric pressure plasma generation device.

[0014] Starting from this, the object of the invention is to provide a device and a method that make it possible to indicate to the user of an instrument with which a treatment medium can be applied to a tissue surface freehand which part of the tissue to be treated has already been treated.

[0015] This problem is solved by the device for image-based support of a user according to claim 1:

[0016] The device according to the invention comprises an instrument, an image acquisition device, an evaluation device, and a display device. The instrument is configured to apply a treatment medium at variable locations within a treatment area of ​​a tissue surface. The image acquisition device is configured to detect the treatment area. The evaluation device is configured to determine a treatment path of the treatment medium relative to the treatment area from the data of the image acquisition device and to calculate a spatially resolved dosage for the treatment area based on this treatment path. Furthermore, the display device is configured to visually represent the treatment path and / or the spatially resolved dosage to the user in relation to the tissue.

[0017] This allows the user to visually see which areas of the tissue surface have already been treated. Specifically, it shows the dosage received by each area. This enables the user to decide which areas still require treatment to achieve the most even distribution of the medium. This reduces the risk of overdosing or underdosing individual areas.

[0018] This can lead to an increase in the uniformity with which the treatment medium was applied to the tissue surface, making the result of the application evaluable and reproducible.

[0019] A special feature of the device according to the invention lies in the evaluation unit, which determines the treatment path of the treatment medium relative to the area being treated. This makes it possible to determine a spatially resolved dosage for the area to be treated based on the treatment path. The treatment path and / or the spatially resolved dosage are optically displayed by the display unit in relation to the area to be treated.

[0020] A spatially resolved dosage can be a dosage value assigned to a specific location within the area to be treated. This dosage value can represent binary states (e.g., treated or untreated) or continuous states (e.g., percentage values ​​relative to a predetermined target dosage). The treatment trace can be a temporal progression of the position and / or size (such as diameter or radius) of the application area. The application area is preferably the area onto which the treatment medium is applied to the tissue surface by the instrument at any given time. This area can be, for example, elliptical or circular. The evaluation device can preferably be a computer or processor.

[0021] The playback device can include a loudspeaker unit that acoustically represents the spatially resolved dosage. For example, a tone with variable characteristics (frequency (pitch), volume, or a variable tone sequence dependent on the dosage) can be emitted. The characteristics can depend on whether the treatment medium is encountering previously treated or untreated tissue. Furthermore, the change in tone characteristics can be used to indicate that the treatment medium has remained on the tissue for too long.

[0022] The applicable treatment medium is a plasma, in particular a low-thermal or non-thermal plasma. The plasma preferably does not produce a macroscopically visible thermal effect on the tissue surface. When treated with such a plasma, the local temperature on the tissue surface can remain below the denaturation temperature of proteins, i.e., below a temperature limit of 40–70°C, preferably below 60–70°C. A non-thermal plasma is a plasma that is not in thermal equilibrium, meaning that, for example, the temperatures of the contained particles (ions, electrons, or neutral particles) differ to a considerable extent.In a low-thermal plasma, the plasma is also not in thermal equilibrium, meaning that the temperatures of the particles (ions, electrons, or neutral particles) contained within it also differ. However, the extent to which the temperatures of the particles differ is less than in a non-thermal plasma. Furthermore, the treatment medium can be a low-energy argon plasma, which does not cause any visible changes to the treated tissue surface. This is true at least if the plasma does not remain in contact with the same area of ​​the tissue for a maximum time exceeding, for example, a few seconds.

[0023] It is preferred that the image acquisition device be configured to capture at least part of the area to be treated in an image sequence comprising a large number of individual images. The treatment medium can thus be detected and tracked within the individual images of the sequence. The image sequence can also be a continuous stream of individual images, with each current individual image being understood as a live image of the captured area.

[0024] The evaluation unit is designed to detect the treatment medium and / or the instrument head in the image sequence and preferably track it across multiple frames. This allows the position of the treatment medium relative to the tissue surface area being treated to be determined. If the treatment medium is obscured by the instrument head in the current frame, the position of the instrument head can be used to estimate the position of the treatment medium. Image recognition algorithms from the field of machine learning, such as neural networks, can be used to detect the instrument head. Support-Vector-Machines or the like.

[0025] Preferably, the evaluation unit is configured to determine the position and / or size of the applied treatment medium in a current frame of the image sequence. This enables the simplest and most robust possible tracking of the treatment medium within the image sequence. The position of the treatment medium can, for example, be the center point of the application area depicted in the frame, where the treatment medium comes into contact with the tissue surface. The application area can be, for example, an ellipse or a circle. The size of the applied treatment area can be the radius, diameter, or area of ​​the region where the treatment medium comes into contact with the tissue surface.

[0026] In particular, the evaluation unit can be configured to determine the position, orientation, and / or dimensions of the instrument head in the current frame of the image sequence. Additional information, such as the position, orientation, and / or dimensions of the head, allows the position of the treatment medium to be determined or estimated if the instrument obscures it.

[0027] In a simplified embodiment, only the position of the head can be used as a measure for the position of the treatment medium. This allows even treatment media that are difficult to detect to be identified and their position determined, such as low-light treatment media and / or treatment media whose contact surface on the tissue surface is irregularly shaped. Furthermore, this reduces the computing power required for implementation.

[0028] In a preferred embodiment, the evaluation unit is configured to recognize the treatment medium in the data from the image acquisition unit based on a color and / or shape characteristic. Preferably, the evaluation unit is configured to recognize the treatment medium based on at least one of the following characteristics: a color change range, a brightness value range, and a saturation value range. By using a color and / or shape characteristic to recognize the treatment medium, the treatment medium can be detected in the simplest and most robust way possible.

[0029] In particular, the evaluation unit can be configured to recognize the instrument head in the image acquisition unit's data based on a color and / or shape characteristic or on a marker attached to the head. This allows the instrument head to be identified using particularly simple and computationally inefficient methods, from which the head's position relative to the area to be treated can be determined.

[0030] Preferably, the evaluation unit is configured to detect the treatment medium in the current single image using a search routine within a defined search area. The search area for the treatment medium can be modified based on the position, orientation, and / or dimensions of the instrument head. This makes it possible, for example, to restrict the search area for the treatment medium to a smaller area immediately adjacent to the tip of the instrument head.

[0031] In a preferred embodiment, the evaluation device is configured to determine a predicted position of the treatment medium in the current single image based on a position and / or speed of the treatment medium from a previous single image.

[0032] Preferably, the search area in which the search routine detects the treatment medium in the current single image is modified based on the predicted position. For example, by means of a Alpha-Beta-Filters or one Kalman FiltersA state estimate for the position (a predicted position) of the treatment medium is determined for the current frame. The search area of ​​the search routine can be defined around the predicted position. The size of the search area can be set to a predetermined value, for example. Alternatively, the size of the search area can be changed based on the predicted position, the speed, and / or a covariance value for the predicted position. For example, the search area can be reduced with a high covariance value and increased with a low covariance value. This improves the tracking of the treatment medium.

[0033] In particular, the search area defined by the predicted position of the treatment medium in the current frame can be used to filter out false positives. The search routine can still search the entire current frame for the treatment medium. However, detections outside the search area can be filtered out as false positives. These filtered-out false positives can then be discarded directly. This reduces the number of false positives.

[0034] Additionally or alternatively, the evaluation unit can be configured so that the area to be treated can be defined by the user before the treatment medium is applied. For example, before beginning application, the user can trace the area to be treated with the instrument within the field of view of the image acquisition unit, without applying any treatment medium. The evaluation unit can be configured to detect and track at least the head of the instrument in order to define the desired treatment area. The treatment area defined in this way can have any shape and complexity, as it can be entered freely by the user. Alternatively, the treatment area can also be defined by the user entering information, for example, on a touchscreen.When the user applies treatment medium to the tissue surface, the evaluation unit can filter out certain detections of the treatment medium that lie outside the area to be treated as false positives. This can help reduce false positives that are introduced, for example, by light reflections from metallic clamps outside the treatment area.

[0035] Preferably, the evaluation unit is configured to recognize characteristic image features of the area to be treated or reference markers applied to the tissue surface in the image acquisition unit and to accurately assign the treatment trace to the area to be treated. Image recognition algorithms from the field of machine learning, such as neural networks, can also be used to recognize the characteristic image features. Convolutional-Neural-Networks, Support-Vector-Machines or the like. The reference marker can, for example, be a colored dot or a dot with a particularly detectable pattern, which is placed in the area to be treated.

[0036] In a preferred embodiment, the playback device has a ( Augmented-Reality-,AR glasses are worn, which are designed to project the treatment path and / or the spatially resolved dosage of the area to be treated accurately into the field of vision of the glasses.

[0037] The evaluation unit is communicatively connected to the instrument's supply unit, and is configured to control the application of the treatment medium by the instrument depending on the local dosage. This allows for the automated prevention of overdosing in individual areas.

[0038] The evaluation unit is communicatively connected to the instrument's delivery unit, and is configured to determine the spatially resolved dosage using measurement information from the delivery unit. This measurement information can be transmitted from the delivery unit to the evaluation unit, thereby enabling a more precise determination of the spatially resolved dosage.

[0039] In a specific embodiment, the instrument can be part of a treatment unit comprising the instrument and a supply unit that provides the instrument with operating resources. Operating resources can be, for example, fluids such as gases or liquids and / or an electrical voltage, in particular a high-frequency alternating voltage. For this purpose, the supply unit can include a controllable (high-frequency, RF) generator. The generator can provide the power required by the instrument to apply the treatment medium. Furthermore, the generator can include a control unit that is, for example, communicatively connected to the evaluation unit. The communication between the control unit and the evaluation unit can be bidirectional.For example, information about the treatment path and / or the spatially resolved dosage and / or measurement data, such as ignition detection, current, voltage, power, and / or resistance, can be transmitted from the control unit to the evaluation unit. Similarly, other components of the supply system, such as a gas supply, can be equipped with a control unit and communicate with the evaluation unit to exchange and / or influence data on gas flow or gas pressure, for example. The components of the supply system can be separate devices that can also operate independently of each other.

[0040] Preferably, the evaluation device is configured to divide the area to be treated into a multitude of resolution elements, for each of which a dosage is determined based on the treatment trace. A resolution element can, for example, be a pixel.

[0041] Furthermore, the problem is solved by the method for image-guided user support according to claim 14: The method according to the invention comprises detecting an area to be treated in which a treatment medium can be applied at variable locations using an instrument. The method also comprises determining a treatment path of the treatment medium relative to the area. Based on the treatment path, a spatially resolved dosage for the area to be treated is determined. In addition, the method comprises displaying the treatment path and / or the spatially resolved dosage in relation to the area to be treated.

[0042] All features and advantages described in relation to the device according to the invention are also applicable to the method according to the invention.

[0043] Further details of advantageous embodiments or specific aspects of the invention will become apparent from the drawings in the description and from the claims. These show: Figure 1 a schematic view of an exemplary embodiment of a device for image-based user support, Figure 2 a schematic view of another embodiment of a device for image-based user support, Figure 3 a schematic view of a further embodiment of the device according to the invention for image-based support of a user, Figures 4a to 4c an example of an area to be treated with examples of corresponding live images of the area in which the treatment path is shown, Figure 5an example of a live image of an area to be treated, Figure 6 a flowchart for an embodiment of the inventive method for image-based user support, Figure 7 Another example of a live image of an area to be treated with a predicted position for the treatment medium.

[0044] In Figure 1 Figure 1 illustrates a device that serves to assist a user of a (medical) instrument 2 by means of imaging.

[0045] The device 1 comprises an instrument 2 with which a user can apply a treatment medium P to a treatment area B of a patient. The instrument 2 can be moved freely in space by the user. The movement of the instrument can follow virtually any path in space x, y, z and can also be pivoted in any direction. By moving and / or rotating the instrument 2, the user can thus vary the location where the treatment medium P comes into contact with a tissue surface in the treatment area B.

[0046] At a distal end 7 of the instrument 2, the instrument 2 has a head 6 for generating a treatment medium, e.g., plasma, in particular non-equilibrium plasma (cold plasma). The plasma exits the distal end 7 of the instrument 2 and comes into contact with the tissue surface. The distal end 7 of the instrument 2 is the end that is closer to the tissue surface when a treatment medium is applied. Conversely, the proximal end 8 of the instrument 2 is the end that is farther from the tissue surface when a treatment medium is applied. The tissue surface can be, for example, skin, mucous membrane, or wounds in the skin or mucous membrane. Electrical power and a suitable gas, e.g., argon, are supplied to the instrument 2 by a power supply unit 9 with an RF generator.

[0047] In this embodiment, the area B to be treated is partially captured by an image acquisition device 3 within its field of view (FoV). The image acquisition device can, for example, be a (video) camera. Preferably, the image acquisition device 3 can be configured to generate a sequence of many individual images, and it can also generate a current individual image (a live image) of the area B to be treated. The image acquisition device 3 can be fixed relative to the area B to be treated or movable (for example, handheld). If the image acquisition device is movable relative to the area B, the field of view (FoV) can shift relative to the area B during treatment.

[0048] The image acquisition device 3 is communicatively connected to an evaluation device 4. The evaluation device 4 is configured to receive and evaluate data from the image acquisition device 3. Within the evaluation device 4, a treatment track T of the treatment medium P can be determined from the data of the image acquisition device 3 in relation to the area B to be treated. Based on the treatment track T, a spatially resolved dosage D for the area to be treated can then be determined within the evaluation device 4. If the image acquisition device 3 is positioned so that it can move relative to the area B, characteristic features 11 of the area B to be treated can be recognized in the individual frames of the image sequence within the evaluation device 4, and the movements of the image acquisition device 3 can be compensated for.

[0049] The evaluation unit 4 is also communicatively connected to a display unit 5. In this embodiment, the display unit 5 is designed as a display unit 5a, for example, in the form of a screen. In this example, the display unit 5 is configured to visually represent the treatment trace T virtually on the image of the tissue for the user. The display unit 5 can change the transparency or color intensity of the displayed treatment trace T according to the local dosage D. This makes the otherwise traceless effect of the medium on the tissue visible as a trace.

[0050] In Figure 1 The treatment line is displayed in a live image of the area to be treated, accurately positioned. In the live image in Figure 1The characteristic features 11 are also depicted. Furthermore, the evaluation unit 4 is configured to recognize the characteristic features 11 in the individual frames of the image sequence and to calculate the treatment track T relative to the position of the characteristic features for a current individual frame (live image) of the image sequence and to display it in this frame.

[0051] Additionally or alternatively, a reference marker 12 can be affixed to the tissue surface in the area B to be treated. The reference marker 12 can, for example, have an optical pattern that can be easily detected by the evaluation device 4. In a preferred embodiment, two or more reference markers are affixed to the tissue surface to enable unambiguous determination of the spatial orientation of each individual image.

[0052] In Figure 2Figure 5b illustrates a further embodiment of the device according to the invention. In this example, instead of the display device 5a, the treatment track T and the spatially resolved dosage are projected directly into the area B to be treated by a projection device 5b. The projection device 5b can, for example, contain a laser or light-emitting diode as a light source, as well as other optical components such as lenses and mirrors to display the treatment track T accordingly.

[0053] Alternatively or additionally, the playback device 5 can also include augmented reality (AR) glasses 5c, which can also be communicatively connected to the evaluation device 4, for example, wired or wirelessly. The AR glasses 5c are configured to project the spatially resolved dosage of the area to be treated accurately into the field of vision of the glasses.

[0054] In Figure 3is illustrated a further embodiment of the device 1 according to the invention. The example from Figure 3 essentially corresponds to the embodiment shown in Figure 1 , with the difference that the communicative connection between the evaluation unit 4 and the supply unit 9, which includes an RF generator and a gas supply and feeds the instrument 2, is shown.

[0055] This communication link allows the evaluation unit 4 to access measurement and control data from the supply unit 9 and use this data to calculate the dosage. The measurement or control data could, for example, be ignition detection, power, output power, current, voltage, or resistance for the RF generator, or gas flow or gas pressure in the case of the gas supply unit.

[0056] In one possible embodiment, the evaluation unit 4 can also store values ​​for an ideal dosage and / or limit values ​​for a minimum and / or maximum dosage of the tissue. These can either be fixed or individually defined by the user via a suitable input device, for example, tailored to the patient or the procedure to be performed.

[0057] In this embodiment, the evaluation unit 4 can be configured to modify the application of the treatment medium P depending on the determined local dosage for the area where the head 6 of the instrument 2 is currently located. For example, the application of the treatment medium by the instrument can be reduced if the currently treated area is already close to the stored desired or maximum dosage, i.e., if, for example, 70%, 80%, or 90% of the desired or maximum dosage has already been reached. The application of the treatment medium can also be completely switched off if the area to be treated has already been treated with the desired or maximum dosage. If the head of the instrument is subsequently moved over a tissue area where the desired or maximum dosage has not yet been reached, the application of the treatment medium is increased again.permitted. This can be achieved, for example, by the evaluation unit 4 providing information about the treatment path T or the local dosage D to a control unit of the RF generator in the supply unit 9. It is also possible for the evaluation unit 4 to send a signal to the control unit to throttle or increase the power, or to switch off or on the power supplied to the instrument 2 by the RF generator of the supply unit 9.

[0058] The spatially resolved dosage can be calculated from the residence time of the treatment medium P at the individual sites in the area B to be treated and the set treatment parameters, such as a treatment mode and / or treatment power. In a simplified embodiment, only the residence time can be used to calculate the spatially resolved dosage, for example, if it can be assumed that a predetermined treatment mode and / or a predetermined treatment power is used for the procedure.

[0059] In the Figures 4a to 4c An example of an area B to be treated is illustrated with a corresponding live image.

[0060] On the left side is in the Figures 4a to 4c The area to be treated, B, is shown in each case, with the plasma P being applied to the tissue surface, G, using instrument 2. On the right side of the Figures 4a to 4cEach entry includes a corresponding live image.

[0061] In the live image in Figure 4a The evaluation unit 4 determined the current position of plasma P in relation to the area B to be treated. Furthermore, the treatment path T is shown in the live image, i.e., the temporal progression of the position of plasma P within the area B to be treated.

[0062] Alternatively or additionally, the dosage D can be displayed in the live image instead of the treatment track T. This is achieved, for example, by varying the color or transparency of the displayed treatment track T in the live image depending on the local dosage D.

[0063] In Figure 4b The example of the area to be treated, B, is from Figure 4a shown. Compared to Figure 4aInstrument 2 was moved further, thus changing the location where instrument 2 applies the treatment medium P. Figure 4b The treatment medium P is obscured by the head 6 of the instrument 2. Therefore, the evaluation unit 4 cannot directly locate the treatment medium P in the current frame of the image sequence. In this case, the evaluation unit 4 determines the position and orientation of the head 6 of the instrument 2 relative to the area B to be treated.

[0064] For this purpose, the evaluation unit 4 determines, for example, a principal axis A of the head 6 of the instrument 2, and a distal direction of the head 6 is determined based on the principal axis A. Based on the position and orientation of the head 6, the evaluation unit 4 determines the position of the concealed treatment medium P. In one embodiment, the evaluation unit 4 can use the distal end 7 of the head 6 of the instrument 2 as the current position of the treatment medium P on the tissue surface.

[0065] In a further embodiment, the evaluation device can be determined by extrapolating the position of the head 6 along the orientation of the instrument 2 defined by the principal axis A of the instrument head 6. This allows for precise determination of the treatment medium P even if it is obscured in the current single image.

[0066] In Figure 4cIs the area to be treated B from Figures 4a and 4b The image shows the instrument 2 being moved by the user, thus changing the current location where the instrument applies the treatment medium P.

[0067] In Figure 5 Figure 1 schematically illustrates an enlarged section of a live image. The area B to be treated, depicted in the live image, is divided into a multitude of resolution elements C11 to Cmn. A dosage D is determined for each resolution element, calculated based on the treatment path T. Figure 5For example, no treatment medium P has been applied to dissolution element C11 because the treatment track T does not pass through it. The dosage D11 of dissolution element C11 is therefore 0. In this example, the treatment track T partially passes through dissolution element C12. For dissolution element C12, the evaluation unit 4 can thus determine a measure of the dosage D12 based on two factors: firstly, the duration for which the treatment medium P is located within dissolution element C12; and secondly, the evaluation unit 4 can also incorporate into the measure of dosage D12 the proportion of the dissolution cell's area to which the treatment medium P was applied. This can be particularly advantageous for the accuracy of the dosage D12 in a marginal region of the treatment medium P.For example, if the position of the treatment medium P is determined as the center point of the detected treatment medium P in the current frame, and a boundary of the treatment medium P is drawn around this center point in the current frame, the boundary of the treatment medium P can run between several resolution cells, so that, particularly in the edge region, a portion of the area of ​​some resolution cells is treated by the treatment medium P. A combination of the two approaches is advantageous, taking into account both the duration of the treatment medium P's presence in the resolution element and the treated area fraction. Figure 5The evaluation unit 4 can determine the current position 13 of the treatment medium P, for example, as the center point of the at least substantially circular area in which the treatment medium P is currently applied. The evaluation unit 4 can also, for example, determine the diameter 14 of this area as the size of the currently applied treatment medium.

[0068] The evaluation unit can also use four measured values ​​from the control of instrument 2, such as ignition detection, current, voltage, power, or resistance, to improve the accuracy of the dosage determined from the position profile of the treatment medium P (the treatment track T). A link between the measured values ​​and the treatment track can be achieved via the time information.

[0069] In Figure 6Figure 1 shows a flowchart for an example of the inventive method for image-based support of a user, carried out by the evaluation unit 4 and the other components of the device 2.

[0070] In process step V1, the area B to be treated is detected, in which treatment medium P can be applied at manually adjustable locations using instrument 2. For example, area B can be detected in an image sequence comprising a large number of individual images. For the further processing steps of the tracking loop ( Tracking Loop ) a current single image is now used, for example (process step V11).

[0071] In process step V2, a treatment track T of the treatment medium P can now be determined relative to area B. For this purpose, it can first be determined in process step V20 whether the field of view (FoV) of the image acquisition device 3 has shifted relative to area B in the current frame. A shift of the field of view (FoV) can occur, for example, due to movement, such as translation and / or rotation, of the image acquisition device 3 relative to the tissue surface, or vice versa. The shift can be determined by comparing the position and location of characteristic features 11 in the current frame with the position and location of the characteristic features 11 in a previous frame. In this way, any shifts that occur can be quantitatively recorded and compensated for.Subsequently, the treatment medium P can be searched for in the current single image, and the position 13 and / or the size 14 of the currently applied treatment medium P can be determined (procedure step V21). If the treatment medium P cannot be found in the current single image (query in procedure step V22), the head 6 of the instrument 2 can be searched for, and the position, orientation, and / or size of the head 6 can be determined (procedure step V23).

[0072] If the treatment medium P is found in the current frame, i.e., if a current position 13 and / or size 14 of the treatment medium P is determined for the current frame, the treatment track T can be determined or updated in process step V25. For this purpose, the determined current position 13 and the size of the treatment medium P can be stored for each corresponding frame, so that a treatment track T in the image sequence results from the totality of the positions 13 and sizes 14 of the frames. The spatially resolved dosage D can now be determined by evaluating the spatial and preferably the temporal progression of the position and size of the treatment medium (the motion track) (still process step V25).If the position of the treatment medium P is not directly found in a current single image, but the position of the head 6 of the instrument 2 has been determined (query in procedure step V24), the position of the treatment medium P can be determined (estimated) based on the position, orientation, and / or size of the head 6 of the instrument 2. If the position of the head 6 of the instrument 2 is also not found, the procedure continues with procedure step V11, in which a new, current single image is now used as the data basis for substeps V21 to V25 of procedure step V2.

[0073] It is also possible to reverse the order of steps V21 / V22 and V23 / V24. This allows the position, orientation, and / or size of the head 6 to be determined first. Based on these parameters, the search area in which the search routine looks for the treatment medium P in the current single image can then be modified. Preferably, the search area is now limited to a region adjacent to the distal end 7 of the head 6. This eliminates the need to search the entire current single image for the treatment medium P.

[0074] Furthermore, it is possible for process steps V21 and V23 to be executed in parallel.

[0075] In a simplified embodiment, instead of the position of the treatment medium P (substeps V21 and V22), only the position of the head 6 (substep V23) can be used as a measure for the position of the treatment medium P. In this example, therefore, only substeps V23 to V25 of process step V2 are performed.

[0076] The treatment track T will preferably be updated in process step V25 according to the estimated position of the treatment medium P.

[0077] Based on the treatment trace T, a spatially resolved dosage of the area B to be treated can be determined in process step V3.

[0078] In process step V4, the treatment track T and / or the spatially resolved dosage D can now be displayed in relation to the area B to be treated by superimposing the treatment track onto the live image.

[0079] After process step V4, process step V11 follows again, in which a new, current single image is used as the data basis for the process steps of the tracking loop.

[0080] In Figure 7 Figure 1 schematically illustrates an enlarged section of a live image. In this embodiment, the evaluation unit is configured to determine a predicted position Pos*_t of the treatment medium P for the current frame. The predicted position Pos*_t can be determined based on the previous position Pos_t-1 and / or the previous velocity V_t-1 of the treatment medium P. In this example, the search area S, within which the search routine detects the treatment medium in the current frame, is defined around the predicted position Pos*_t. The position Pos_t of the treatment medium P detected in the current frame lies within the search area S.

[0081] Alternatively, the defined search area S can also be used to filter out detections fp, in which the treatment medium P is detected outside the search area S, for example due to light reflections, as false-positive misclassifications.

[0082] It is also possible to adjust the size of the search area S depending on the velocity V_t-1. If the velocity V_t-1 is high, the search area S is chosen to be larger. Conversely, if the velocity V_t-1 is low, the search area S can be made smaller.

[0083] The device 1 described above for image-based user support works as follows:

[0084] To perform a treatment, the user can apply plasma to the tissue surface in the desired dosage freehand using instrument 2, such as a plasma applicator. During the treatment, the user can, for example, look at a display device 5a, which shows the area of ​​the tissue surface to be treated, and / or wear AR glasses 5c. Additionally, during the treatment, the treatment path T and / or the dosage D can be projected onto the tissue surface in the area to be treated by a projection device 5b, making it visible to the user.

[0085] During the treatment, the user is shown which areas of the treatment zone have already been treated with plasma. Ideally, the user is also shown the dosage received by each area. This allows the user to apply plasma to untreated areas or areas with an insufficient dosage.

[0086] For example, in the case of treatment of a cervicalen intraepithelialen Neoplasie (CIN) A treatment area of ​​the mucosal surface is viewed with a video colposcope. In this case, the evaluation unit can be connected to the video colposcope, which is usually already in place, so that it can analyze the (video) data from the colposcope. Similarly, for other procedures, for example, the camera of an endoscope or laparoscope can be used.

[0087] Treatment of CIN can, for example, involve applying non-thermal plasma to the mucosal surface. The video colposcope data can then be used by the evaluation unit to determine the temporal progression of the plasma's position (treatment trace) and, from this, calculate the spatially resolved dosage. The treatment trace or spatially resolved dosage can then be displayed on the video colposcope screen and / or on other monitors.

[0088] The device 1 according to the invention serves to provide image-guided support to a user of an instrument 2. The instrument 2 is configured to apply a treatment medium to variable locations within a treatment area of ​​a tissue surface. The device also includes an image acquisition unit configured to capture the treatment area. The device 1 includes an evaluation unit 4 configured to determine a treatment path of the treatment medium relative to the treatment area from the data of the image acquisition unit and to calculate a spatially resolved dosage for the treatment area based on the treatment path. The device 1 also includes a display unit 5 configured to visually represent the treatment path and / or the spatially resolved dosage to the user in relation to the tissue.The device 1 according to the invention makes it possible to visually indicate to the user which areas of the tissue surface to be treated have already been treated and / or with what dosage the corresponding treated areas were treated. Reference symbol:

[0089] 1 Device for imaging support of a user 2 Instrument 3 Image acquisition device (camera) 4 Evaluation device 5 Playback device 5a Screen 5b Projection device 5c (AR) glasses 6 Instrument head 7 Distal end of instrument 8 Proximal end of instrument 9 Power supply unit (RF generator, gas supply) 10 Treatment device 11 Characteristic features of the area 12 Reference marker 13 Position of the treatment medium 14 Size (diameter) of the treatment medium A Main axis of the instrument B Area to be treated C11, ..., Cmn Resolution elements D Dosage fpf False positive misattribution FoV Viewing area P Treatment medium (non-thermal plasma) Pos_t Position of the treatment medium in a current frame Pos_t-1 Position of the treatment medium in a previous frame V_t-1 Velocity of the treatment medium in a previous frame Pos*_t-predicated position of the treatment medium in a current single imageTreatment track Search area V1 Procedure step of capturing the area to be treated V2 Procedure step of determining the treatment track V3 Procedure step of determining the spatially resolved dosage V4 Procedure step of displaying the treatment track and / or the dosage V11 Sub-procedure of using a current single image V20 Sub-procedure of determining the possible change in the field of view (FoV) in relation to area B V21 Sub-procedure of searching for the treatment medium (determining the position and / or size of the treatment medium) V22 Query step to check if the treatment medium has been found V23 Sub-procedure of searching for the instrument head (determining the position, orientation and / or size of the head) V24 Query step to check if the instrument head has been found V25 Sub-procedure of determining or updating the treatment track xX-axis of the instrument yY-axis of the instrumentzZ-axis of the instrument

Claims

1. Device (1) for image based support of a user comprising: - an instrument (2) that is configured to apply a treatment medium (P) at alterable locations in an area (B) to be treated, wherein the treatment medium (P) that can be applied is a plasma; - an image capture device (3) that is configured to capture the area (B) to be treated; - an evaluation device (4) that is configured to determine a treatment trace (T) of the treatment medium (P) relative to the area (B) to be treated from data of the image capture device (3) and to determine a spatially resolved dosage (D) for the area (B) to be treated based on the treatment trace (T); as well as - a representation device (5, 5a, 5b, 5c) that is configured to optically display the treatment trace (T) and / or the spatially resolved dosage (D) in relation to the area (B) to be treated for the user, wherein the evaluation device (4) is configured to recognize the treatment medium (P) and / or a head (6) of the instrument (2) in an image sequence, characterized in that the evaluation device (4) is communica-tively connected with a supply unit (9) of the instrument (2), whereby the evaluation device (4) is configured to determine the spatially resolved dosage (D) in addition based on measurement parameters that are transmitted by the supply unit (9) to the evaluation device (4).

2. Device (1) according to claim 1, characterized in that the treatment medium (P) that can be applied is a low- or non-thermal plasma.

3. Device (1) according to claim 1 or 2, characterized in that the image capture device (3) is configured to capture the area (B) to be treated by an image sequence comprising a multiplicity of individual images.

4. Device (1) according to any of the preceding claims characterized in that the evaluation device (4) is configured to trace the treatment medium (P) and / or a head (6) of the instrument (2) over multiple individual images in the image sequence.

5. Device (1) according to claim 3 or 4, characterized in that the evaluation device (4) is configured to determine a position and / or a size of the applied treatment medium (P) in a current individual image of the image sequence.

6. Device (1) according to any of the claims 3 to 5, characterized in that the evaluation device (4) is configured to determine a position, an orientation and / or a dimension of the head (6) of the instrument (2) in the current individual image of the image sequence.

7. Device (1) according to any of the preceding claims, characterized in that the evaluation device (4) is configured to recognize the treatment medium (P) in data of the image capture device (3) based on a color and / or shape feature or based on a marker attached on the head (6).

8. Device (1) according to any of the preceding claims, characterized in that the evaluation device (4) is configured to recognize the head (6) in data of the image capture device (3) based on a color and / or shape feature or based on a marker attached on the head (6).

9. Device (1) according to any of the preceding claims, characterized in that the evaluation device (4) is configured to recognize the treatment medium (P) in a search area (S) in the current individual image by means of a search routine, whereby the search area for the treatment medium (P) is modified based on the position, the orientation and / or the dimension of the head (6).

10. Device (1) according to any of the preceding claims, particularly according to claim 9, characterized in that the evaluation device (4) is configured to determine a predicted position (Pos*_t) of the treatment medium (P) in the current individual image based on a position (Pos_t-1) and / or velocity (V_t-1) of the treatment medium (P) from a previous individual image, wherein preferably the search area (S), in which the search routine recognizes the treatment medium (P) in the current individual image, is modified based on the predicted position (Pos*_t) and / or the velocity (V_t-1).

11. Device (1) according to any of the preceding claims, characterized in that the evaluation device (4) is configured so that the area (B) to be treated can be defined by the user prior to the application of treatment medium (P).

12. Device (1) according to any of the preceding claims, characterized in that the evaluation device (4) is configured to recognize characteristic image features of the area (B) to be treated in data of the image capture device (3) or to recognize reference markers applied in the area (B) of the tissue surface and to assign the treatment trace (T) to the area (B) to be treated in a spatially accurate manner.

13. Device (1) according to any of the preceding claims, characterized in that the evaluation device (4) is configured to control the application of the treatment medium (P) by the instrument (2) depending on the local dosage.

14. Method for image based support of a user, comprising: - Capturing (V1) by means of an image capturing device (3) an image sequence of an area (B) to be treated in which a treatment medium (P) can be applied at alterable locations, wherein the treatment medium (P) that can be applied is a plasma; - Recognizing by means of an evaluation device (4) the treatment medium (P) and / or a head (6) of the instrument (2) in the image sequence of the area (B) to be treated, - Determining (V2) by means of the evaluation device (4) a treatment trace (T) of the treatment medium (P) relative to the area (B) to be treated; - Determining (V3) by means of the evaluation device (4) a spatially resolved dosage (D) for the area (B) to be treated based on the treatment trace (T) and measurement parameters that are transmitted by a supply unit (9) of the instrument (2) to the evaluation device (4) via a communication connection; - Representing (V4) by means of a representation device (5, 5a, 5b, 5c) the treatment trace and / or the spatially resolved dosage (D) with reference to the area (B) to be treated.