System and method for visual support of a user of a medical instrument

Abstract

FIELD: medicine.SUBSTANCE: system serves to provide visual support to the user of the tool. In this case, the instrument is designed with the ability to influence the working environment on the variable areas of the tissue surface area to be affected. In addition, the system includes an image recording device configured to record the area to be affected. The system includes an evaluation device configured to determine a trace of exposure to the working environment in comparison to the area to be exposed, based on data from the image recording device, and to determine a spatially referenced dose for the area to be exposed based on the trace of exposure. The system also includes a playback device configured to visually display to the user a trace of exposure and / or a spatially referenced dose applied to the tissue.EFFECT: visual indication to the user of which areas of the tissue surface area to be treated have already been treated and / or what the dose of treatment was on the corresponding areas.18 cl, 7 dwg

Description

[0001] The present invention relates to a system, as well as to a method for supporting a user of a medical instrument, configured to act with a working medium on a tissue surface.

[0002] Commonly known are instruments that allow the user to manually apply a working medium, such as low-temperature plasma, to small areas of the tissue surface to be treated. In this method, the user exposes tissue surfaces, such as skin, mucous membranes, or wounds, to a working medium such as cold (non-thermal, low-energy) argon plasma.

[0003] This type of exposure to the tissue surface results in little or no thermal effect, resulting in no coagulation or carbonization of the tissue. The local temperature on the tissue surface remains below the protein denaturation temperature (approximately 60-70°C). The exposed tissue surface does not change visually, so the user cannot determine which area of ​​the tissue has already been exposed to the working medium. The user also cannot determine the dose or duration of exposure for individual sections of the exposed area.

[0004] Therefore, the user must remember as accurately as possible which areas and for how long were exposed to the working environment.

[0005] This places increased demands on the user's concentration. Furthermore, assessing the duration of exposure to the working environment on a given area depends on the user's subjective sensations. Furthermore, due to the lack of visible results on the tissue, the user may be tempted to repeatedly "go over" the previously exposed area, leading to overdose.

[0006] Similarly, however, it may happen that the user does not treat certain areas at all or only treats them for too short a time, resulting in an insufficient dosage.

[0007] This means that the quality and uniformity of the working environment's impact on the exposed fabric surface can sometimes vary greatly. This also makes it difficult to reproduce the results of exposures that cannot be objectively confirmed and / or documented.

[0008] A number of technical solutions are associated with attempts to improve the uniformity of the working medium's effect on the tissue surface using newly developed large-scale plasma sources. For example, publication DE 102014220488 A1 proposes a large-scale source of low-temperature ("cold") atmospheric pressure plasma, which can be used to affect a relatively large area (e.g., an area of ​​approximately 100 cm 2). In particular, in the case of impact areas that are not simple (e.g. rectangular, square, elliptical or circular) but rather complex and / or irregular in shape, there may be unintended effects on areas that are outside the impact area.

[0009] Other known solutions propose automating the exposure to the working medium. For example, publication DE 102010011643 A1 describes a plasma source that automatically moves across the tissue surface using a drive unit controlled by a control device. However, this directly impacts the exposure process. Furthermore, fully automating this process requires significant design and engineering effort and, therefore, financial investment.

[0010] The publication "An Enhanced Surgical Concept Enables Effective, Contactless Plasma Treatment of Patients Using the kINPen® MED Cold Plasma Jet Source," Applied Sciences, Volume 10, Issue 17, Article 17, January 2020, CIOC: 10.3390 / Suppl. 10176133, proposes a sensor system that determines the distance from the instrument to the treatment area and illuminates the area to be treated with plasma. However, the user can only visualize this process at a given moment in time. Therefore, even in this case, the user must remember which areas were exposed to the working environment and for how long.

[0011] Furthermore, publication WO 2011 / 044248 A2 describes a system in which the surgical field is monitored using a video camera during ablation surgery, with the quality of tissue lesions formed during ablation assessed by a processor. Based on the determined quality, colored markers are added to the real-time images of the surgical field.

[0012] DE 102015100927 A1 describes auxiliary equipment for visual support of the operator during surgical intervention. The auxiliary equipment comprises a camera and an image processing unit, wherein the image processing unit is configured to recognize the medical instrument being used without the need for markers applied to the instrument.

[0013] Based on the above, the objective of the invention is to create a system and method that make it possible to show the user of an instrument with which he can manually perform the action of a working medium on the surface of a fabric, which area of ​​the fabric to be affected has already been subjected to the latter.

[0014] This problem is solved in the user visualization support system described in paragraph 1 of the invention formula.

[0015] The system for visualizing the user of a medical instrument proposed in the invention comprises a medical instrument, an image recording device, an evaluation device, and a playback device. The instrument is configured to act upon the surface of a tissue at variable locations of the tissue surface area to be acted upon. The image recording device is configured to photograph the tissue surface area to be acted upon. The evaluation device is configured to determine the impact trace of the working medium relative to the area to be acted upon based on the data from the image recording device, and to determine a spatially referenced (spatially resolved) dose, i.e., one determined in relation to a location in space (with spatial resolution), for the area to be acted upon based on this impact trace.In addition, the reproduction device is configured to visually display to the user a trace of exposure and / or a spatially linked dose in relation to the area of ​​the tissue surface to be exposed.

[0016] This allows the user to visually see which areas or locations of the tissue surface to be treated have already been exposed. Specifically, it is possible to display the dose received by each area. This allows the user to decide which areas of the treatment area to target with the working medium to achieve the most uniform distribution possible. This reduces the risk of overdosing or underdosing in certain areas of the tissue surface to be treated.

[0017] This can help to improve the uniformity of the impact of the working environment on the fabric surface, which will make it possible to evaluate and reproduce the result of this impact.

[0018] A distinctive feature of the system proposed in the invention is an evaluation device that determines the exposure trace of the working environment relative to the area being treated. This allows for the determination of a spatially referenced dose for the affected area based on this exposure trace. The exposure trace and / or spatially referenced dose are visually displayed by a playback device relative to the affected area.

[0019] The spatially referenced dose may in this case be a dosage value assigned to a region within the affected area. This dosage value may be in a binary state (e.g., exposed or unexposed) or continuous (e.g., percentage values ​​relative to a specified dose). The exposure trace may represent the change in position and / or size (e.g., diameter or radius) of the exposed region over time. The exposed region is preferably the region where the working fluid is currently acting on the tissue surface via the instrument. This region may have, for example, an elliptical or circular shape. The evaluation device is preferably a computer or processor device.

[0020] The playback device may contain an acoustic unit that can be used to acoustically represent a spatially referenced dose. For example, this may involve the output of a sound with a variable characteristic (frequency / pitch, volume, or tone sequence that changes depending on the dosage). This characteristic may depend on whether the working medium is affecting previously exposed or unexposed tissue. Furthermore, a change in the sound characteristic can be used to indicate whether the tissue has been exposed to the working medium for too long.

[0021] The working medium is preferably plasma, particularly low-temperature or non-thermal plasma. The plasma preferably does not cause a thermal effect on the tissue surface that is noticeable to the naked eye. When exposed to such plasma, the local temperature on the tissue surface can remain below the protein denaturation temperature, i.e., below the temperature limit of 40-70°C, preferably below 60-70°C. Non-thermal plasma is plasma that is not in thermal equilibrium, as a result of which, for example, the temperatures of the particles contained therein (ions, electrons, or neutral particles) differ significantly. Low-temperature plasma is also not in thermal equilibrium, therefore the temperatures of the particles contained therein (ions, electrons, or neutral particles) also vary. However, the degree of difference in the temperatures of the particles contained therein is smaller compared to non-thermal plasma.Furthermore, the working medium can be low-energy argon plasma, which causes no visible changes to the surface of the exposed tissue. This is the case, at least, when the plasma is applied to the same tissue area for no longer than the maximum time, such as a few seconds.

[0022] The image recording device is preferably configured to at least partially capture the affected area as an image sequence comprising individual images. In this way, the affected area of ​​the working environment can be detected and tracked in individual images of this sequence. The image sequence can also be a continuous stream of individual images, with the current individual image being understood as an image of the area being recorded, obtained in real time.

[0023] In a preferred embodiment of the invention, the evaluation device is configured to recognize, in a sequence of images and preferably also track, in individual images of this sequence, the area of ​​impact of the working medium and / or the tool head. This allows for the determination of the position of the area of ​​impact of the working medium relative to the tissue surface area to be treated. If the area of ​​impact of the working medium is obscured by the tool head in the current individual image, the position of the tool head can be used to estimate the position of this area. Image recognition algorithms from the field of machine learning, such as neural networks, support vector machines, etc., can be used to recognize the tool head.

[0024] The evaluation device is preferably configured to determine the position and / or size of the area of ​​impact of the working medium in the current individual image of the image sequence. This ensures the simplest and most reliable tracking of the area of ​​impact of the working medium in the image sequence. The position of the area of ​​impact of the working medium can be determined, for example, by the center point of the area of ​​impact on the fabric surface shown in the individual image. The impact area can be, for example, an ellipse or a circle. The size of the impact area can be the radius, diameter, or area of ​​the area where the working medium comes into contact with the fabric surface.

[0025] The evaluation device may be configured, in particular, to determine the position, orientation, and / or size of the tool head in the current individual image of the image sequence. Additional information, such as the position, orientation, and / or size of the head, may be used to determine or estimate the position of the area of ​​influence of the working environment if the tool obscures this area.

[0026] In a simplified embodiment of the invention, only the position of the head can be used as a measure of the position of the working fluid's impact site, which also makes it possible to detect (and determine the position of their impact site) working fluids that are difficult to detect, such as, for example, low-luminosity working fluids and / or working fluids whose impact site on the fabric surface has an irregular shape. This also reduces the computing power required for implementation.

[0027] In a preferred embodiment of the invention, the evaluation device is configured to recognize the area of ​​influence of the working environment in the image recording device data based on a color and / or shape feature. The evaluation device is preferably configured to recognize the area of ​​influence of the working environment based on at least one of the following features: a chromaticity range, a luminance range, and a saturation range. By using color and / or shape to recognize the area of ​​influence of the working environment, this area can be detected in the simplest and most reliable manner.

[0028] In particular, the evaluation device can be configured to recognize a tool head in image recording device data based on a color and / or shape feature, or based on a marker provided on the head. This enables the detection of the tool head using particularly simple and computationally inexpensive methods, which can determine the position of the head relative to the area being treated.

[0029] The evaluation device is preferably configured to recognize the work environment's impact area in a search field on the current individual image using a search program. The work environment's impact area search field can be adjusted depending on the position, orientation, and / or size of the tool head. This allows the work environment's impact area search field to be limited, for example, to a smaller area (directly) adjacent to the top of the tool head.

[0030] In a preferred embodiment of the invention, the evaluation device is configured to determine the estimated position of the area of ​​influence of the working medium in the current individual image based on the position of the area of ​​influence and / or the speed of the working medium in the previous individual image.

[0031] The search field in which the search program recognizes the work environment's influence area in the current individual image preferably varies depending on the estimated position. For example, using an alpha-beta filter or a Kalman filter, an estimate of the position (estimated position) of the work environment's influence area in the current individual image can be obtained. The search field in the search program can be defined around the estimated position. The size of the search field can be set, for example, to a specified value. Alternatively, the size of the search field can also be changed, for example, based on the estimated position and / or velocity and / or covariance value for the estimated position. For example, the search field can be reduced when the covariance value is high and, conversely, increased when the covariance value is low. This can improve the quality of tracking the work environment's influence area.

[0032] In particular, a search field defined based on the estimated location of the work environment's impact site in the current individual image can be used to exclude false positive (erroneous) matches. The search program can continue searching for the work environment's impact site throughout the entire current individual image. However, detected locations outside the search field can be excluded as false positive matches. For example, sorted false positive matches can be excluded immediately. This helps reduce the number of false positive matches.

[0033] The evaluation device may be configured, as an additional or alternative option, to determine the area of ​​the tissue surface to be affected by the working medium before the user initiates the action. For example, before initiating the action with the instrument, the user may track the area to be affected in the field of view of the image recording device without initiating the action with the working medium. The evaluation device may be configured to detect and track at least the head of the instrument to thereby determine the desired area to be affected. The area to be affected determined in this way may have any shape and complexity, since the user can manually specify it during the action. Alternatively, the area to be affected may also be determined by user input, for example, from a touchscreen.If the user now applies the working medium to the fabric surface, locations detected by the evaluation device outside the exposed area can be excluded as false positive matches. This can help reduce the number of false positive matches that arise, for example, due to light reflection from metal clamps outside the exposed area.

[0034] The evaluation device is preferably configured to recognize characteristic features of the image of the area of ​​the fabric surface to be treated or control markers applied to the fabric surface in the area to be treated in the image recording device data, and to precisely spatially associate the impact trace with the area to be treated. Machine learning image recognition algorithms, such as neural networks, in particular convolutional neural networks, support vector machines, etc., can also be used to recognize characteristic features of the image. The control marker can be, for example, a colored icon or an icon with a particularly easily recognizable pattern, which is applied to the area to be treated.

[0035] In one preferred embodiment of the invention, the playback device includes augmented reality (AR) glasses capable of spatially accurately projecting into their field of view a trace of exposure and / or a spatially referenced dose of the area being exposed.

[0036] In another preferred embodiment of the invention, the evaluation device is communicatively connected to the instrument's power supply, and the evaluation device is configured to control the instrument's exposure to the working medium based on the local dose. This automatically prevents overdosing in certain areas.

[0037] The evaluation device is preferably communicatively connected to the instrument's power supply unit, wherein the evaluation device is configured to determine the spatially referenced dose additionally based on the power supply unit's measurement information, i.e., based on the measured parameters transmitted to the evaluation device from the power supply unit. Transmission of measurement information from the power supply unit to the evaluation device enables more accurate determination of the spatially referenced dose.

[0038] In one specific embodiment of the invention, the tool may be part of an impact device that includes a tool and a power supply that supplies the tool with working means. The working means may be, for example, a fluid, such as a gas or liquid, and / or electrical voltage, in particular high-frequency alternating voltage. For this purpose, the power supply may include an adjustable (high-frequency) generator. The generator may provide the power required by the tool to impact the working medium. Furthermore, the generator may include a control unit connected, for example, to an evaluation device with data transfer capability. Communication between the control unit and the evaluation device may be bidirectional.For example, information about the exposure trace and / or spatially referenced dose and / or measurement information such as ignition detection, current, voltage, power, and / or resistance data can be transmitted from the control unit to the evaluation device. Similarly, other components of the power supply device, such as the gas supply device, can be equipped with a control unit and connected to the evaluation device with data transfer capability, such as for exchanging and / or controlling gas flow or pressure data. The power supply device components can also be separate devices that can operate even in the absence of all other components.

[0039] The evaluation device is preferably configured to divide the exposed area into multiple resolution elements, the dose for each of which is determined based on the exposure trace. For example, a resolution element may be a pixel.

[0040] The objective of the invention is also solved in the method of visualizing user support, characterized in paragraph 18 of the claims.

[0041] The method for visualizing support for a user of a medical instrument proposed in the invention includes capturing an image (photographing) of a tissue surface area exposed to a working medium at variable locations using the medical instrument. The method also includes determining a trace of the working medium's effect relative to the exposed tissue surface area. Based on this trace of effect, a spatially referenced dose is determined for the exposed tissue surface area. Furthermore, the method includes displaying the trace of effect and / or the spatially referenced dose relative to the exposed tissue surface area.

[0042] All features and advantages described in relation to the system proposed in the invention also apply to the method proposed in the invention.

[0043] Further details and particulars of the preferred embodiments of the invention are presented in the drawings, in the description, and in the dependent claims. The drawings show:

[0044] Fig. 1 is a schematic representation of a user visualization support system in one example of the invention,

[0045] Fig. 2 is a schematic representation of a user visualization support system in another embodiment of the invention,

[0046] Fig. 3 is a schematic representation of a user visualization support system in another embodiment of the invention,

[0047] Fig. 4A-4B - an example of an affected area with examples of corresponding real-time images of this area showing the impact trace,

[0048] Fig. 5 is an example of an image of the affected area obtained in real time,

[0049] Fig. 6 is a flow chart of operations in a method for visualizing user support, implemented in accordance with one example of the invention,

[0050] Fig. 7 is another example of a real-time image of an area exposed to the working environment, with the calculated position of this area.

[0051] Fig. 1 shows a system 1 intended for visual support of a user of a (medical) instrument 2.

[0052] System 1 includes a tool 2, which allows the user to apply the working medium P to the target area B of the patient. Tool 2 is freely movable by the user. It can move along virtually any trajectory in x, y, and z space, and can also be rotated in any direction. By moving and / or rotating tool 2, the user can thus change the location at which the working medium P hits the tissue surface in the target area B.

[0053] The instrument 2 comprises a head 6 at its distal end 7 for generating a working medium, such as plasma, in particular nonequilibrium (cold) plasma. The plasma exits the distal end 7 of the instrument 2 and hits the tissue surface. The distal end 7 of the instrument 2 is the end that is closer to the tissue surface when exposed to the working medium. In contrast, the proximal end 8 of the instrument 2 is the end that is further from the tissue surface when exposed to the working medium. The tissue surface may be, for example, skin, mucous membrane, or wounds on the skin or mucous membrane. Electrical power and an appropriate gas, such as argon, are supplied to the instrument 2 from a power supply unit 9 containing an HF generator.

[0054] The area B to be treated in this embodiment of the invention is partially within the field of view (FoV) of the image recording device 3. The image recording device may be, for example, a (video) camera. The image recording device 3 is preferably configured to create an image sequence consisting of a plurality of individual images, wherein the image recording device 3 can also create a current individual image (an image obtained in real time) of the area B to be treated. The image recording device 3 can be positioned stationary with respect to the area B to be treated or can be moved (e.g. manually). If the image recording device is positioned movably with respect to the area B, then the FoV can shift with respect to the area B during the treatment.

[0055] The image recording device 3 is communicatively connected to the evaluation device 4. The evaluation device 4 is configured to receive and evaluate the data of the image recording device 3. In the evaluation device 4, an exposure trace T of the working medium P relative to the exposed region B can be determined based on the data of the image recording device 3. Based on the exposure trace T, a spatially referenced dose D for the exposed region can then be determined in the evaluation device 4. If the image recording device 3 is arranged in a movable manner relative to the region B, then the evaluation device 4 can, for example, recognize characteristic features 11 of the exposed region B in individual images of the image sequence and compensate for the movements of the image recording device 3.

[0056] Furthermore, the evaluation device 4 is communicatively connected to the playback device 5. In this embodiment, the playback device 5 is configured as a display device 5a, such as a screen. In this example, the playback device 5 is configured to visually and virtually display the impact trace T on the tissue image to the user. The playback device 5 can vary the transparency or color intensity of the displayed impact trace T in accordance with the local dose D. Thus, the impact of the environment on the tissue, which is not noticeable in itself, is visualized as a trace.

[0057] The impact trace on the real-time image shown in Fig. 1 is displayed with precise spatial reference. On the real-time image shown in Fig. 1, characteristic features 11 are also marked. The evaluation device 4 is also configured to detect characteristic features 11 on individual images of the image sequence and calculate the impact trace T relative to the position of these characteristic features for the current individual image (obtained in real time) of the image sequence, with subsequent display of the impact trace on this image.

[0058] As an addition or alternative, a control marker 12 may be affixed to the surface of the fabric in the exposed region B. For example, the control marker 12 may comprise an optical pattern that is particularly easily recognized by the evaluation device 4. In a preferred embodiment of the invention, two or more control markers are affixed to the surface of the fabric to ensure unambiguous determination of the spatial orientation of each individual image.

[0059] Figure 2 shows a system proposed in another embodiment of the invention. Instead of the display device 5a, in this example, the exposure trace T and the spatially referenced dose are projected directly onto the exposed region B using a projection device 5b. The projection device 5b may comprise, for example, a laser or light-emitting diode as a light source, as well as other optical components, such as lenses and mirrors, for appropriately displaying the exposure trace T.

[0060] As an addition or alternative, the playback device 5 may include augmented reality (AR) glasses 5, which may also be connected to the evaluation device 4 for data transmission (e.g., via a wired or wireless connection). The AR glasses 5 are configured to spatially accurately project a spatially referenced dose of the area being exposed into their field of view.

[0061] Figure 3 shows a system 1 proposed in another embodiment of the invention. The example in Figure 3 generally corresponds to the example in Figure 1, with the difference that the evaluation device 4 is communicatively connected to the power supply device 9, which contains, among other things, an RF generator and a gas supply device and supplies power to the instrument 2.

[0062] Via this data connection, the evaluation device 4 can access the measurement and control data of the power supply device 9 and use them to calculate the dose. For example, for an RF generator, the measurement or control data may include ignition detection, power, output power, current, voltage, or resistance data, or, in the case of a gas supply device, gas flow rate or pressure.

[0063] In one possible embodiment of the invention, the evaluation device 4 may also store ideal dosage values ​​and / or threshold values ​​for the minimum and / or maximum tissue dose. These values ​​may either be preset or set by the user using the appropriate input device for each individual case, for example, for a specific patient or intervention.

[0064] The evaluation device 4 in this embodiment of the invention can be configured to vary the effect of the working medium P depending on the determined local dose for the area where the head 6 of the instrument 2 is currently located. For example, the effect of the working medium by means of the instrument can be reduced if the magnitude of this effect on the area exposed to it is already close to a predetermined desired or maximum dose, that is, for example, if 70%, 80%, or 90% of the desired or maximum dose has already been reached. The effect of the working medium can also be completely stopped if the area exposed to it has already been exposed to the desired or maximum dose. If the head of the instrument is then moved over an area of ​​tissue for which the desired or maximum dose has not yet been reached, then the effect of the working medium is again increased or resolved.This can be achieved, for example, by providing the evaluation device 4 with information about the impact trace T or the local dose D to the control unit of the HF generator in the power supply device 9. The evaluation device 4 can also send a signal to the control unit to regulate or increase the power, as well as to turn on or off the energy supply to the instrument 2 via the HF generator of the power supply device 9.

[0065] A spatially referenced dose can be calculated based on the duration of exposure of individual sections of the exposed region B to the working environment P and the established exposure parameters, such as the mode and / or power. In a simplified embodiment of the invention, only the exposure duration can also be used to calculate the spatially referenced dose, for example, when the use of a predetermined mode and / or predetermined exposure power for the intervention can be considered permissible.

[0066] Fig. 4A-4B show an example of the affected area B with an image associated therewith obtained in real time.

[0067] The left side of Figs. 4A-4B shows the region B in which the surface of tissue G is exposed to plasma P by means of instrument 2. The right side of Figs. 4A-4B shows the corresponding image obtained in real time for each of these cases.

[0068] In the image obtained in real time and shown in Fig. 4A, the current position of the plasma impact site P in relation to the target area B was determined by the evaluation device 4. In addition, the image obtained in real time displays the impact trace T, that is, the trajectory of the change over time of the position of the plasma impact site P in the target area B.

[0069] As an alternative or in addition, instead of the exposure trace T, the dose D can also be displayed on the real-time image. This is achieved, for example, by changing the color or transparency of the exposure trace T displayed on the real-time image depending on the local dose D.

[0070] Fig. 4B shows an example of the region B to be affected, as shown in Fig. 4A. Compared to Fig. 4A, tool 2 has been moved further, i.e., the location of the action of tool 2 on the working medium P has changed. The area of ​​action of the working medium P is obscured in Fig. 4B by head 6 of tool 2. This makes it impossible for evaluation device 4 to directly locate the area of ​​action of the working medium P on the current individual image of the image sequence. In this case, evaluation device 4 determines the position and orientation of head 6 of tool 2 relative to the region B to be affected.

[0071] For this, the evaluation device 4 determines, for example, the main axis A of the head 6 of the instrument 2, and based on the main axis A, the distal direction of the head 6 is determined. Using the position and orientation of the head 6, the evaluation device 4 determines the position of the hidden area of ​​influence of the working medium P. In one example of the embodiment of the invention, the evaluation device 4 can use the distal end 7 of the head 6 of the instrument 2 as the current position of the area of ​​influence of the working medium P on the tissue surface.

[0072] In another embodiment of the invention, the evaluation device can determine the position of the area of ​​influence of the working medium P by extrapolating the position of the head 6 of the tool 2 along the main axis A of this head, which determines its orientation. Thanks to this, even in the case of a hidden area of ​​influence of the working medium P, an accurate determination of this area in the current individual image can be achieved.

[0073] Fig. 4B shows the affected area B shown in Fig. 4A and 4B, where the tool 2 has been moved by the user, resulting in a change in the current location of the impact of the tool 2 by the working medium P.

[0074] Fig. 5 schematically shows an enlarged fragment of an image obtained in real time. The exposed region B shown in this image is divided into a plurality of resolution elements C11-Cmn. For each resolution element, a dose D is determined, which is calculated based on the impact trace T. For example, in Fig. 5, the resolution element C11 was not exposed to the working medium P, since the impact trace T does not pass through this element. Thus, the dose D11 for the resolution element C11 is 0. In this example, the impact trace T partially passes through the resolution element C12. Therefore, on the one hand, for the resolution element C12, the evaluation device 4 can determine the dose D12 value based, firstly, on the duration of the period of exposure to the working medium P within the element C12. On the other hand, the evaluation device 4 can also include in the measurement of the dose D12 the proportion of the area of ​​the resolution cell that was exposed to the working medium P.This may be particularly useful for accurately determining the dose D12 at the periphery of the area of ​​exposure to the working environment P. For example, if in the current individual image the position of the area of ​​exposure to the working environment P is defined as the central point of this detected area, and the boundary of the area of ​​exposure to the working environment P in the current individual image is drawn around a certain central point, then this boundary may pass between several resolution cells, so that, in particular, a part of the area in the boundary region of some resolution cells is exposed to the action of the working environment P. A preferred option is a combination of both these possibilities, including both the time of exposure of the working environment P to the resolution element and the part of the area exposed. In Fig.5 the evaluation device 4 can determine the current position 13 of the area of ​​influence of the working environment P, for example, as the central point of at least a substantially circular area, which is currently influenced by the working environment P. The evaluation device 4 can also determine, for example, the diameter 14 of this area of ​​the current influence of the working environment as its size.

[0075] Furthermore, the evaluation device 4 can use measurement data received from the control unit of the instrument 2, such as ignition detection, current, voltage, power, or resistance data, to improve the accuracy of determining the trajectory of the time-dependent change in the position of the working medium exposure site P (exposure trace T) and the dose obtained from it. The relationship between the measurement data and the exposure trace can be established using a time reference.

[0076] Fig. 6 shows a flow chart of operations in a method for visualizing user support, implemented in accordance with one example of implementing the invention using the evaluation device 4 and other components of the system 1.

[0077] In step V1 of the method, the target area B is registered, which can be manipulated using tool 2 with the working medium P at manually adjustable locations. For example, area B can be registered as an image sequence consisting of multiple individual images. Further tracking of the steps of the manipulation cycle is performed after this, for example, using the current individual image (step V11).

[0078] In step V2, the trace T of the influence of the working medium P relative to the region B can now be determined. For this purpose, it can first be determined in step V20 whether the field of view FoV of the image recording device 3 has shifted relative to the region B in the current individual image. The shift of the field of view FoV can occur, for example, due to the movement, translational and / or rotational, of the image recording device 3 relative to the tissue surface or vice versa. The shift can be determined by comparing the position and orientation of the characteristic features 11 in the current individual image with the position and orientation of the characteristic features 11 in the previous individual image. This makes it possible to quantitatively record and compensate for the resulting shifts. After this, a search for the current area of ​​influence of the working medium P can be performed in the current individual image, determining its position 13 and / or size 14 (step V21).If the area of ​​influence of the working environment P cannot be found on the current individual image (request at step V22), then a search for the head 6 of the tool 2 can be performed, determining its position, orientation and / or size (step V23).

[0079] If the area of ​​influence of the working medium P has been found in the current individual image, i.e., the current position 13 and / or the size 14 of the area of ​​influence of the working medium P have been determined for this image, then in step V25 the trace T of influence can be determined or updated. For this purpose, for example, the obtained data on the current position 13 and the size 14 of the area of ​​influence of the working medium P can be stored for each corresponding individual image, so that the trace T of influence is obtained from the set of positions 13 and sizes 14 of this area in the individual images in the image sequence. Now the spatially referenced dose D can be determined by evaluating the spatial and, preferably, temporal change (motion trace) of the position and the size of the area of ​​influence of the working medium (continuation of step V25).If the position of the area of ​​influence of the working medium P is not detected directly in the current individual image, but the position of the head 6 of the tool 2 has been determined for this purpose (request in step V24), then the position of the area of ​​influence of the working medium P can be determined (estimated) on the basis of the position, orientation and / or size of the head 6 of the tool 2. If the position of the head 6 of the tool 2 is also not detected, then the operation continues to step V11, in which the new current individual image is now used as the basis for substeps V21-V25 of step V2.

[0080] The order of steps V21 / V22 and V23 / V24 can also be changed. First, the position, orientation, and / or size of head 6 can be determined. Based on these parameters, the search field in which the search program searches for the area affected by the working medium P in the current individual image can now be changed. The search field is now preferably limited to the area adjacent to the distal end 7 of head 6. This eliminates the need to scan the entire current individual image when searching for the area affected by the working medium P.

[0081] Also, it is possible to execute steps V21 and V23 in parallel.

[0082] In a simplified embodiment of the invention, instead of the position of the working medium impact area P (substeps V21 and V22), only the position of head 6 (substep V23) can be used as a measure of this position. Thus, in this example, only substeps V23-V25 of step V2 are performed.

[0083] The impact trace T is preferably updated in step V25 according to the estimated position of the impact area of ​​the working medium P.

[0084] Based on the exposure trace T in step V3, the spatially referenced dose for the exposure area B can be determined.

[0085] In step V4, the exposure trace T and / or spatially referenced dose D can now be displayed with respect to the exposed region B by superimposing the exposure trace on the real-time image.

[0086] Step V4 is followed again by step V11, which uses the new current single image as the basis for the tracking loop steps.

[0087] Fig. 7 schematically shows an enlarged fragment of an image obtained in real time. In this embodiment of the invention, the evaluation device is configured to determine the estimated position Pos*_t of the area of ​​influence of the working medium P for the current individual image. The estimated position Pos*_t can be determined based on the previous position Pos_t-1 of this area and / or the previous speed V_t-1 of the working medium P. In this example, the search field S, in which the search program recognizes the area of ​​influence of the working medium in the current individual image, is set around the estimated position Pos*_t. The position Pos_t of the area of ​​influence of the working medium P, recorded in the current individual image, is located within the search field S.

[0088] Alternatively, the specified search field S may also be used to exclude as false positive matches locations fp (areas of influence of the working environment P) detected, for example due to light reflection, outside the search field S.

[0089] It is also possible to adapt the search field size S depending on the speed V_t-1. If the speed V_t-1 is high, a larger search field S is selected. Conversely, if the speed V_t-1 is low, the search field S can also be reduced.

[0090] The user visualization support system 1 described above operates as follows.

[0091] To perform the operation, the user may use tool 2, such as a plasma applicator, to manually apply plasma at a required dosage to the tissue surface area to be treated. During this application, the user may, for example, look at display device 5a, which displays the tissue surface area to be treated, and / or wear AR glasses 5c. Furthermore, during the application, the application trace T and / or dose D may be projected onto the tissue surface area to be treated in a manner discernible to the user using projection device 5b.

[0092] During treatment, the user will now be shown which areas of the treatment area have already been exposed to plasma. Additionally, the dose received by individual areas of the treatment area is now displayed. This allows the user to apply plasma to areas that have not yet been treated or that have received too little.

[0093] For example, in the case of treatment for cervical intraepithelial neoplasia (CIN), the area of ​​the mucosal surface to be treated is visualized using a video colposcope. In this case, the evaluation device can be connected to an existing video colposcope, allowing for the evaluation of the latter's (video) data. Similarly, for other interventions, an endoscope or laparoscope camera can be used.

[0094] Treatment of CIN can be accomplished, for example, by applying non-thermal plasma to the mucosal surface. In this case, video colposcope data can be used by the evaluation device to determine the temporal change in the position of the plasma exposure site (the exposure trace) and, based on this, to determine a spatially referenced dose. The exposure trace or spatially referenced dose can be displayed accordingly on the video colposcope screen and / or other displays.

[0095] The system 1 proposed in the invention is intended to provide visualization support to the user of the instrument 2. The instrument 2 is configured to act on variable areas of the tissue surface area to be acted upon by the working medium. The system also includes an image recording device configured to record images of the area being acted upon. The system 1 includes an evaluation device 4 configured to determine the impact trace of the working medium relative to the area to be acted upon, based on the data of the image recording device, and to determine the spatially referenced dose for the area being acted upon, based on this impact trace. The system 1 also includes a playback device 5 configured to visually display to the user the impact trace and / or the spatially referenced dose relative to the tissue.The system 1 proposed in the invention allows for a visual indication to the user of which areas of the tissue surface area to be treated have already been exposed and / or what the dose of this exposure was.

[0096] Reference Designations

[0097] 1 - User visualization support system

[0098] 2 - tool

[0099] 3 - Image recording device (camera)

[0100] 4 - evaluation device

[0101] 5 - playback device 5a screen

[0102] 5b - projection device

[0103] 5c - augmented reality glasses

[0104] 6 - Tool head

[0105] 7 - distal end of the instrument

[0106] 8 - proximal end of the instrument

[0107] 9 - power supply unit (HF generator, gas supply)

[0108] 10 - Impact device

[0109] 11 - characteristic features of the area

[0110] 12 - control marker

[0111] 13 - position of the area of ​​influence of the working environment

[0112] 14 - size (diameter) of the area affected by the working environment A main axis of the tool

[0113] B - area to be affected

[0114] C11, Cmn - permission elements

[0115] D - dose fp false positive (erroneous) match

[0116] FoV - field of view

[0117] P - working environment (non-thermal plasma)

[0118] Pos_t - the position of the area of ​​influence of the working environment on the current individual image

[0119] Pos_t-1position of the area of ​​influence of the working environment on the previous separate image

[0120] V_t-1 - the speed of the working environment in the previous separate image

[0121] Pos*_t - calculated position of the area of ​​influence of the working environment on the current individual image

[0122] T - trace of impact

[0123] S - search field

[0124] V1 - step of registering the area to be affected

[0125] V2 - Impact trace determination step

[0126] V3 - step for determining the spatially-related dose

[0127] V4 - step for displaying the trace of exposure and / or dose

[0128] V11 - Step of using the current single image

[0129] V20 - step to detect possible changes in the FoV field of view relative to area B

[0130] V21 - substep of searching for the area of ​​influence of the working environment (determining the position and / or size of the area of ​​influence of the working environment)

[0131] V22 - query step whether the area of ​​influence of the working environment is found

[0132] V23 - Tool head search substep (determine the position, orientation and / or size of the head)

[0133] V24 - query step whether the tool head is found

[0134] V25 - Substep of determining or updating the impact trace

[0135] x - X-axis of the tool

[0136] y - Y-axis of the tool

[0137] z - Z-axis of the tool.

Claims

1. A system (1) for visual support of a user of a medical instrument, including: - a medical instrument (2) designed with the ability to act with a working medium (P) on the surface of the tissue in variable locations of the area (B) of the tissue surface to be acted upon, - an image recording device (3) designed with the possibility of capturing the area (B) to be affected, - an evaluation device (4) configured to determine a trace (T) of exposure to the working environment (P) relative to the area to be exposed (B) based on data from an image recording device (3) and to determine a spatially referenced dose (D) for the area to be exposed (B) based on the trace (T) of exposure, and - a playback device (5, 5a, 5b, 5c) designed with the ability to visually display to the user a trace (T) of exposure and / or a spatially linked dose (D) in relation to the area (B) to be exposed.

2. The system (1) according to paragraph 1, characterized in that the acting working medium (P) is plasma.

3. The system (1) according to paragraph 2, characterized in that the plasma is a low-temperature or non-thermal plasma.

4. A system (1) according to one of the preceding paragraphs, characterized in that the image recording device (3) is designed with the possibility of at least partially capturing the area (B) to be affected in the form of a sequence of images, including individual images.

5. The system (1) according to one of the previous paragraphs, characterized in that the evaluation device (4) is designed with the ability to recognize in a sequence of images the area of ​​impact of the working environment (P) and / or the head (6) of the tool (2).

6. The system (1) according to paragraph 5, characterized in that the evaluation device (4) is designed with the ability to track the area of ​​influence of the working environment (P) and / or the head (6) of the tool (2) in individual images of the image sequence.

7. The system (1) according to one of paragraphs 4-6, characterized in that the evaluation device (4) is designed with the possibility of determining the position and / or size of the area of ​​influence of the working environment (P) on the current individual image of the image sequence.

8. The system (1) according to one of paragraphs 4-7, characterized in that the evaluation device (4) is configured to determine the position, orientation and / or size of the head (6) of the tool (2) on the current individual image of the image sequence.

9. The system (1) according to one of the previous paragraphs, characterized in that the evaluation device (4) is designed with the ability to recognize the area of ​​influence of the working environment (P) in the data of the image recording device (3) based on the color and / or shape feature.

10. The system (1) according to claim 9, characterized in that the color and / or shape feature includes at least one of the following features: a color range, a brightness range, and a saturation range.

11. The system (1) according to one of the preceding paragraphs, characterized in that the evaluation device (4) is designed with the ability to recognize the head (6) in the data of the image recording device (3) based on a color and / or shape feature or based on a marker provided on the head (6).

12. The system (1) according to one of paragraphs 5-11, characterized in that the evaluation device (4) is designed with the ability to recognize the area of ​​influence of the working environment (P) in the search field (S) on the current individual image using a search program, wherein the search field for the area of ​​influence of the working environment (P) changes depending on the position, orientation and / or size of the head.

13. The system (1) according to claim 12, characterized in that the evaluation device (4) is configured to determine the calculated position (Pos*_t) of the area of ​​influence of the working environment (P) on the current individual image based on the position (Pos_t-1) of the area of ​​influence and / or the speed (V_t-1) of the working environment (P) on the previous individual image, wherein the search field (S) in which the search program recognizes the area of ​​influence of the working environment (P) on the current individual image changes depending on the calculated position (Pos*_t) and / or the speed (V_t-1).

14. The system (1) according to one of the previous paragraphs, characterized in that the evaluation device (4) is designed with the possibility of determining the area (B) of the surface of the fabric subject to exposure to the working medium (P), before this exposure is carried out by the user.

15. The system (1) according to one of the preceding paragraphs, characterized in that the evaluation device (4) is designed with the ability to recognize in the data of the image recording device (3) characteristic features of the image of the area (B) of the surface of the fabric to be affected or control markers applied to the surface of the fabric in its area (B) to be affected, and the precise spatial reference of the trace (T) of the impact to the area (B) to be affected.

16. The system (1) according to one of the previous paragraphs, characterized in that the evaluation device (4) is connected with the possibility of transmitting data to the power supply device (9) of the instrument (2), and the evaluation device (4) is designed with the possibility of controlling the effect of the working environment (P) by means of the instrument (2) depending on the local dose.

17. The system (1) according to one of the preceding paragraphs, characterized in that the evaluation device (4) is connected with the possibility of transmitting data to the power supply device (9) of the instrument (2), wherein the evaluation device (4) is configured to determine the spatially linked dose (D) additionally on the basis of the measured parameters transmitted to the evaluation device (4) from the power supply device (9).

18. A method for visualizing support for a user of a medical instrument, including: - registration (V1) of an image of the area (B) of the tissue surface subject to impact in variable places by the working environment (P) using a medical instrument (2), - determination (V2) of the trace (T) of the impact of the working environment (P) in relation to the area to be affected (B), - determination (V3) for the affected area (B) of a spatially referenced dose (D) based on the exposure trace (T), - display (V4) of the exposure trace and / or spatially referenced dose relative to the exposed area (B).

Images