System and method for breathing guidance

Abstract

Disclosed is a computer-implemented method for breathing guidance, the method comprising providing first breathing target data; outputting breathing instructions to a user, the breathing instructions based on the first breathing target data; providing observed user breathing data, the observed user breathing data based on measurement data representative of a user's breathing; using a model to determine second breathing target data based on a deviation of the observed user breathing data from the first breathing target data; and outputting updated breathing instructions to a user, the updated breathing instructions based on the second breathing target data.

Description

[0001] Brainlab AG

[0002] Attorney’s File: B19286WO

[0003] SYSTEM AND METHOD FOR BREATHING GUIDANCE

[0004] FIELD OF THE INVENTION

[0005] The present invention relates to a computer-implemented method for breathing guidance, a corresponding system, and a corresponding computer program product and computer-readable medium.

[0006] TECHNICAL BACKGROUND

[0007] In the medical field, a patient following a specific breathing pattern to their best ability can have a significant impact on the outcome of a medical procedure. When breathing, the arrangement of the patient’s anatomical features changes relative to each other and to the medical system. This can, for example, impact medical imaging and radiation treatment procedures. Breathing can be automatically monitored and to some degree procedures can be adjusted to the patient’s breathing. However, it is still desirable for a patient to breathe in such a manner that is suited to the medical procedure. To that end medical personnel can provide instructions to the patient to breathe appropriately. Adequate breathing control is, however, still challenging.

[0008] Thus, an object underlying the subject-matter of the present disclosure is to address at least some of the above challenges, in particular, to provide a method and system for breathing guidance, for example that allows for improved breathing for a medical procedure.

[0009] The present invention can be used for different procedures where controlled breathing is carried out, e.g. in connection with a system for image-guided radiotherapy such as VERO® and ExacTrac®, both products of Brainlab AG.

[0010] Aspects of the present invention, examples and exemplary steps and their embodiments are disclosed in the following. Different exemplary features of the invention can be combined in accordance with the invention wherever technically expedient and feasible.

[0011] EXEMPLARY SHORT DESCRIPTION OF THE INVENTION In the following, a short description of the specific features of the present invention is given which shall not be understood to limit the invention only to the features or a combination of the features described in this section.

[0012] The present invention provides a computer-implemented method for breathing guidance, the method comprising providing first breathing target data; outputting breathing instructions to a user, the breathing instructions based on the first breathing target data; providing observed user breathing data, the observed user breathing data based on measurement data representative of a user’s breathing; using a model to determine second breathing target data based on a deviation of the observed user breathing data from the first breathing target data; and outputting updated breathing instructions to a user, the updated breathing instructions based on the second breathing target data.

[0013] GENERAL DESCRIPTION OF THE INVENTION

[0014] In this section, a description of the general features of the present invention is given for example by referring to possible embodiments of the invention.

[0015] The present invention provides a method, system, computer program product, and computer- readable medium according to the independent claims. Preferred embodiments are provided in the dependent claims.

[0016] The present disclosure provides a computer-implemented method for breathing guidance, the method comprising providing first breathing target data; outputting breathing instructions to a user, the breathing instructions based on the first breathing target data; providing observed user breathing data, the observed user breathing data based on measurement data representative of a user’s breathing; using a model to determine second breathing target data based on a deviation of the observed user breathing data from the first breathing target data; and outputting updated breathing instructions to a user, the updated breathing instructions based on the second breathing target data.

[0017] As will be understood from the discussions below, the method of the present disclosure allows for addressing the above challenges. It provides a method and system for breathing guidance, for example that allows for improved breathing for a medical procedure.

[0018] Specifically, first breathing target data and corresponding instructions can be used as a starting point used as a reference. The user, for example a patient, tries following the instructions and user breathing data may be observed and used, together with the reference, i.e., the first breathing target data, to determine second breathing target data and corresponding updated breathing instructions. This technique can be carried out in different manners, as also described in more detail below. For example, the first breathing target data may be associated with breathing instructions that are the goal for a certain medical procedure. Such breathing instructions may be difficult to follow. The method of the present disclosure can provide updated breathing instructions that are closer to what the user could reproduce when breathing, thereby allowing the user to more easily follow along the current breathing instructions. Then, difficulty could be increased, e.g. incrementally in several iterations of the method described above while monitoring the user’s progress until breathing is within an acceptable range for the medical procedure. Instead of starting with difficult to follow breathing instructions, it would also be possible to immediately start with easier to follow instructions and increase difficulty, e.g. incrementally in iterations of the method described above.

[0019] When breathing instructions are closer to a user’s actual breathing and within the range of their breathing capabilities, the user will be able to follow the breathing instructions along much better and progress from their current level of capability in a controlled and quantifiable manner. Since controlled breathing can negatively affected by stress, it is additionally beneficial if there is no need to demand too much of a user in terms of breathing instruction difficulty, as failing to follow too difficult instructions may increase both physical and mental stress, thereby decreasing the user’s performance.

[0020] Irrespective of how the method is carried out, since the user’s actual breathing is taken into account for updating breathing instructions, the method allows for breathing guidance that can be provided suitably for users with different breathing capabilities and strike a balance between the user’s capabilities and the medial procedure’s requirements. This allows for overall better outcomes of following the breathing instructions and of the associated medical procedures.

[0021] According to the present disclosure, “breathing target data” may comprise, for example, one more individual target values of parameters characterizing breathing and / or a target function, forexample a function of time, that characterizes breathing, for example a target breathing curve. Breathing data may, for example, be a progression of breathing amplitude values over time “Target” in this context indicates that the breathing target data underlies the breathing instructions, which indicate to a user how to breathe. That is, following the breathing instructions associated with breathing target data, represent a target for the user.

[0022] The first breathing target data may be representative of a breathing pattern, forexample a breathing pattern for a medical procedure. This may be a generic breathing pattern for a / the medical procedure or a breathing pattern for the medical procedure adjusted for the individual case, for example based on a treatment plan. The breathing target data may be empirically or semi- empirically obtained breathing target data. For example, the first breathing target data may be based in part on a model, which may be a purely theoretical model or may in part be based on empirical breathing data, e.g. observed for a plurality of users.

[0023] The first breathing target data may be user-specific, for example based on data characterizing the user, such as observed breathing data for the user, age, gender, conditions, of the user. For example, a breathing pattern for a medical procedure as described above, may be adapted based on the data characterizing the user.

[0024] When carrying out the method iteratively, i.e. in several iterations, in some of the iterations the second breathing target data obtained at the end of one iteration may be used as the first breathing target data for a subsequent, e.g. the next, iteration.

[0025] From the introductory comments, it will be apparent that the first breathing target data may, but does not have to, correspond to breathing data representing the goal for a given individual medical procedure, such as specified in a treatment plan. Indeed, the first breathing target data may, at least initially when iteratively carrying out of the method, deviate from said goal. Particularly in the case where difficulty is gradually increased, the first breathing target data may deviate from said goal.

[0026] The method may comprise determining the first breathing target data and / or retrieving it, e.g. from a data storage. The latter may be the case when using generic breathing target data, e.g. for a medical procedure.

[0027] In the present disclosure, outputting breathing instructions to a user may comprise, for example, displaying a representation of the breathing target data on a display device. In an example, a breathing curve may be rendered. In another example, the visualization may comprising computer game like visual elements. This will be explained more detail below. Alternatively or in addition, audio output and / or haptic output may be provided, for example to indicate certain landmarks and / or phases in a breathing curve, like breath hold period, and / or start / stop breath hold, or the like.

[0028] The breathing instructions or updated breathing instructions being based on the first or second breathing target data, respectively, may comprise that breathing characteristics represented by the first or second breathing target data are transformed into instructions. As mentioned above, breathing target data may, for example, be a progression of breathing amplitude values over time. Accordingly, this progression can be visualized, with one dimension representing amplitude and one representing time, and / or transferred into a audio and / or haptic output that is time-dependent and varies its output over time based on the amplitude values’ progression over time. These are mere examples, other types of transforming breathing target data into instructions are possible.

[0029] The second breathing target data may also be referred to as updated or adapted breathing target data. The updated breathing instructions may also be referred to as adapted breathing instructions

[0030] The method of the present disclosure comprises providing observed user breathing data, the observed user breathing data based on measurement data representative of a user’s breathing, particularly measurement data acquired while the breathing instructions and / or updated breathing instructions are output to the user. The method may comprise acquiring the measurement data and / or receiving the measurement data. Known methods for acquiring user breathing data may be used. Observed user breathing data may for example be based on measurement data comprising image data, such as provided by optical cameras and / or medical imaging devices, and / or distance measurement data, such as provided by optical surface scanning, and / or acceleration data, such as provided by acceleration detection at a user’s chest.

[0031] In particular, the user breathing data may be breathing data observed while the breathing instructions are output to the user and optionally while the updated breathing instructions are output to the user.

[0032] A deviation between the observed user breathing data and breathing target data represented by the breathing instructions may be indicative of how well the user’s breathing matches a target breathing pattern. In particular, when the user breathing data is breathing data observed while the breathing instructions are output to the user breathing instructions are output to the user, the user breathing data can then be seen as an indicator for how well the user is following the breathing instructions.

[0033] As explained above, updated breathing instructions are based on the second breathing target data and the second breathing target data are determined based on a deviation of the observed user breathing data from the first breathing target data. In view of the above considerations, it will be understood that this allows for taking into account, when generating the updated breathing instructions, how well the user’s breathing is currently matching a target breathing pattern, in particular, how well the user is following the breathing instructions. As such, the updated breathing instructions can strike a balance between the overall goal for the user’s breathing and the user’s capabilities. When carrying out the method iteratively, the user may also be guided towards more desirable breathing patterns in accordance with their capability. As briefly mentioned above, a model is to determine second breathing target data based on a deviation of the observed user breathing data from the first breathing target data. Which model to use depends on the application at hand. Even a relatively simple model will bring about the abovedescribed benefits.

[0034] For example, a model may employ rule-based decision making, which may involve applying rules to the values of parameters characterizing the breathing determined from the observed user breathing data. Applying rules may comprise determining whether the values are within a predetermined range of values and / or exceed a predetermined threshold value and / or remain below a predetermined threshold value. This may be done for individual values and / or for a mean value over a given time interval, for example. The values of the range and / or the threshold values may be determined from the breathing target data. The rules to be applied may be specified by the model. For example, it may thus be determined that observed values are outside a predetermined range of values in a predetermined time. For example, the user may not be able to maintain breath hold over an entire breath hold period as specified by the first breathing target data. In that case, the observed values will be out of range towards the end of the breath hold period. The model may be used to determine that and / or how much the observed values are out of range and based thereon determine updated second breathing target data, such as second breathing data having a breath hold period that is shortened, for example shortened by an amount of time in correlation with how much or for how long the observed values are out of range. An example as described above can be particularly useful when carrying out the method iteratively, e.g. as a control loop. The corrections made between iterations may, for example, be empirically determined, e.g. based on historic data possibly including data from previous iterations.

[0035] This is a basic description merely given as an explanatory example and non-limiting.

[0036] Other models may be used, including Support Vector Machines, SVM, recurrent neural networks, RNNs, e.g. with long short-term memory, LSTM, and / or models with reinforcement learning. It should be understood that the skilled person will understand how to employ such models correctly for the application at hand.

[0037] In the following, breathing and exemplary parameters that may characterize breathing will be explained to give some further context. Breathing during medical procedures may, for example, comprise one or more cycles, each cycle comprising a breathing phase, a pre-breath hold phase and a breath hold phase. Often, medical procedural steps are carried out during the breath hold phase. Parameters that characterize the breathing may comprise, for each phase, a duration, a frequency (e.g. frequency of breathing in or out), and an amplitude. The frequency during an ideal breath hold phase scenario (when no breathing in / out occurs) may be considered to be zero. It may be particularly advantageous for breathing target data to comprise a frequency for a pre-breath hold preparation breathing phase. This may allow for better preparation for the breath hold.

[0038] Generally, planning data for medical procedures comprise goals for the parameters of the breath hold phase. Similarly, the breathing target data according to the present disclosure may comprise target values for duration and / or amplitude of the breath hold phase. For example, the target values may be selected taking into account the goals comprised in the planning data.

[0039] According to the present disclosure, the model may be configured to output second breathing target data such that the second breathing target data has a different difficulty level compared to the first breathing target data, according to on one or more predefined difficulty metrics. The difficulty metrics may pertain to a total breath hold time, which may for example be accomplished within a single breath hold period or accumulatively over multiple breath hold periods, to how many breath holds are required to accomplish a target total breath hold time, duration of one or more individual breath hold periods, and / or breath hold consistency.

[0040] Total breath hold time may refer to the accumulated breath hold duration that can be achieved over a given period, such as over a treatment session. A target would be that this total breath hold time is at least as high as an envisaged treatment time. The respective duration of an individual breath hold period may be the length of time that a user can remain in breath hold. Breath hold consistency may refer to how good the breathing level, e.g. amplitude, can be kept. For example, this may be determined based on maximum deviations from a mean amplitude during breath hold or rate of decrease or increase of the amplitude from beginning of breath hold.

[0041] This may allow for step-wise guidance of a user towards breathing requirements for a given application, such as specified in a treatment plan.

[0042] When the method is carried out iteratively, the difficulty level may but does not have to change for each iteration. This may allow for stabilizing a certain difficulty level before advancing to a higher difficulty level, for example.

[0043] The model may be configured such that a change in difficulty level between the first breathing target data and the second breathing target data depends on the observed user breathing data, particularly on the deviation of the observed user breathing data from the first breathing target data. The different difficulty level may be automatically selected, for example, based on a current difficulty level and a deviation of the observed user breathing data from the first breathing target data. In case the deviation of the observed user breathing data exceeds a first threshold a difficulty level below the current difficult level may be selected. Alternatively or in addition, in case the deviation of the observed user breathing data does not exceed a second threshold a difficulty level above the current difficult level may be selected. The second threshold may be smaller than or equal to the first threshold. Optionally, the difficulty level may remain unchanged in case the deviation of the observed user breathing data does not exceed a third threshold, in particular the third threshold being smaller than the second threshold, and / or in case no increase in difficulty is required. No increase in difficulty is required, for example, when the breathing requirements of a given application, e.g. medical procedure, are met by the user’s breathing. This may be the case when the updated breathing target data corresponds to breathing target data specified by the breathing requirements and a deviation of the user breathing data is in an acceptable range.

[0044] The method may be repeated until, via an overall increase in difficulty level, the user meets the breathing requirements. The overall increase may be brought about by one or more iterations in which the difficulty level is increased. This does not preclude temporary decreases, e.g. a decrease in one or more iterations, in difficulty level in the course of the overall increase.

[0045] Thus, a user may be guided towards the breathing requirements iteratively and via changes in difficulty level with goals set for the respective iterations that may be more achievable than the final difficulty level. This can be seen as a tool enabling a user to incrementally improve their breathing with attainable goals.

[0046] As briefly mentioned above, the model may be a ML model. Particularly, it may be a ML model trained to output, based on inputs comprising breathing target data and observed user breathing data, second breathing target data. As an example, the ML model may comprise a Support Vector Machine, RNNs with LSTM or Hybrid Models, e.g. where Reinforcement Learning is added, as mentioned above.

[0047] The model may be a model trained on past breathing data, in particular based on a previous session and optionally a current session, in particular by reinforcement learning.

[0048] Training data may comprise breathing data, such as breathing curves, and optionally training data may also comprise breathing target levels, and / or patient specific properties like gender, age, conditions. The training may be done offline and could additionally be done online, i.e., the parameters of the model may be refined concurrently with outputting the breathing guidance to the user.

[0049] According to the present disclosure, the above-described deviation may be determined based on only a subset of the breathing data, the subset being associated with only a part of the breathing cycle, in particular only with the breath hold period of a breathing cycle or only for a preparatory period prior to the breath hold period and the subsequent breath hold period. The updated breathing instructions may updated for the part of the breathing cycle associated with the subset of the breathing data from which the deviation is determined.

[0050] Some parts of the breathing cycle may be more relevant than others. For example, a breathing after breath hold may be less relevant than the preparation before the breath hold and / or the breath hold itself. The preparation before breath hold may be decisive for how well a user will be able to carry out the breath hold. As such, when basing the deviation on such more relevant parts if the breathing cycle, updates to the breathing instructions will be more targeted towards suitably guiding the user in these relevant parts, making the method more efficient and accurate.

[0051] The model may be configured to adjust breathing target data such that, if a deviation is determined in the breath hold period, to update breathing instructions for the pre breath hold part of the cycle. This can account for different users being able to carry out breath hold better after different breathing patterns in the pre breath hold part of the cycle. For example, some users may gain more through longer breaths, others through consistent shorter / medium breaths in the pre breath hold part of the breathing cycle.

[0052] According to the present disclosure, the first and / or second breathing target data may comprise a target level and / or a target time and / or a target breathing curve.

[0053] In this context, a target level and / or target time and / or target breathing curve during a given iteration. While the target level and / or time and / or breathing curve may, in some instances and / or iterations correspond to specified target levels and / or times and / or breathing curves as specified in a treatment plan, e.g. a radiation treatment plan, they may also differ from said specified levels during some or all iterations. However, if multiple iterations are performed, an overall approach of the target level and / or target time and / or target breathing curve towards those specified in the treatment plan may be aimed at. This may be taken into account in the step of providing second breathing target data and corresponding updated breathing instructions. This does not preclude temporarily moving away from the specified target levels and / or times and / or breathing curves as specified in a treatment plan, e.g. during one or more iterations. In this context, the considerations outlined in the context of the context of temporarily decreasing difficulty level similarly apply. The target level may be based on an amplitude of a breathing curve or breathing signal, for example, e.g. a maximum amplitude and / or a mean altitude in a certain time period. The target level may be relevant for a user being able to bring the anatomical features into a required position for a step of a medical procedure, such as image acquisition and / or irradiation. For example, this may increase the distance between anatomical structures to be irradiated and spared from irradiation and / or alleviate obstructions in an image acquisition.

[0054] The target time may be a target breath hold time and / or a target length of breaths in preparation to breath hold and / or an overall time over which the breathing instructions are provided. The breath hold time may be relevant for a user being able to hold their breath for a given step of the medical procedure, such as an irradiation and / or image acquisition step. The target length in preparation of the breath hold may be a relevant indicator for a subsequent breath hold being successful. The overall time may, for example, be relevant for a user being or becoming able to follow breathing instructions throughout the entirety of a medical procedure.

[0055] According to the present disclosure, the observed user breathing data may comprise an observed level and / or an observed time and / or an observed breathing curve. In particular, where the breathing target data comprises a target level, the observed user breathing data may comprise and observed level. Where the breathing target data comprises a target time, the observed user breathing data may comprise an observed time, particularly the corresponding time, e.g. breath hold time, length of breaths in preparation to breath hold, and / or overall time, respectively. When the breathing target data comprises a breathing curve, the observed user breathing data may comprise a breathing curve.

[0056] If observed user breathing data comprise at least a shared subset of parameters with the breathing target data, inputs may be easier to process by the model, in particular, deviations may be easier to quantify.

[0057] According to the present disclosure, additional inputs may be provided to the model, comprising at least one of: age, gender, conditions, historical observed user breathing data, or current measurement data, such as heart rate, maximal oxygen uptake (VO2Max), temperature.

[0058] Incorporating such data as inputs to the model may allow for improved accuracy in determining second breathing target data and corresponding updated breathing instructions. For example, where heart rate exceeds a certain threshold, this may be an indicator that following the breathing pattern requires excessive effort and / or causes stress, which may be taken into account by temporarily reducing difficulty level and / or increasing difficulty level in smaller steps. Similarly, certain preconditions and / or age may require increasing difficulty level in smaller steps or even temporarily reducing difficulty level. However, depending on the model, such inputs may not be necessary, but rather taken into account indirectly via the observed user breathing data.

[0059] For example, some models, particularly non-ML models, may apply rule-based changes to the breathing target data by applying predetermined rules to the input data and making changes to the breathing target data based on the outcome of applying said rules. In case of ML-models that work based on pattern recognition, such input may not be required. Similarly, where a control loop, potentially with short iteration times, is used, such inputs also may also not be required and the adjustment may rely on relatively fast iteration towards suitable breathing target data merely based on a deviation, particularly how the deviation develops over time and / or iterations.

[0060] According to the present disclosure, first breathing target data may comprise a target breathing curve a-i(t). The observed user breathing data may comprise an observed breathing curve b(t) derived from the measurement data. The second breathing target data comprise an updated target breathing curve ai+j(t).

[0061] When using a breathing curve in the method of the present disclosure, a higher granularity for determining deviations and determining new breathing target data and corresponding breathing instructions can be provided. This also allows for more accurate updates. This in turn increases efficiency, as higher accuracy will mean better tailoring to the respective userand a user being able to more quickly arrive at a breathing pattern required for the application, e.g. medical procedure, at hand.

[0062] A breathing curve may represent a development of a signal of a breathing value over time, optionally after pre-processing of the signal, such as smoothing, normalizing, or the like. The breathing curve can be measured by observing the periodical deformations of a breathing body by different means, (e.g. Surface Camera, Belt, or the like). The correlation of the surface deformation to the movements of inner organs can, for example, be used to ensure a safer treatment with fewer side effects.

[0063] According to the present disclosure, the breathing instructions may comprise a visualization representative of the target breathing curve a-i(t) and the updated breathing instructions may comprise a visualization representative of the updated target breathing curve ai+j(t).

[0064] The visualization may be a depiction of the curve itself. Alternatively, the respective curves may be represented by visual elements, such as in a gaming environment. Such visual elements may be configured to accommodate for human physiology in terms of visibility of said elements, particular from a given distance and / or viewing angle, particularly irrespective of visual defects like near- or far-sightedness, perceptibility of movement, perceptibility of distance between elements, or the like. Compared to such elements, a depiction of the breathing curve may be physiologically difficult to perceive by a user, particularly when the user is in a typical treatment position, e.g., lying down. Particularly since a continued breathing guidance is provided by the visualized instructions, the user’s attaining of a required breathing pattern will be more efficient.

[0065] The method may comprise outputting, alongside a / the visualization representative of the target breathing curve and / orthe updated target breathing curve, a visualization of the observed breathing curve. This allows for a user attaining of a required breathing pattern more efficiently, as they receive feedback on where in the current breathing pattern an adjustment is required. This reduces trial and error over multiple breathing cycles. Alternatively or in addition, the method may comprise outputting a performance indicator representative of a performance calculated from the deviation of the observed user breathing data from the first breathing target data and / or second breathing target data based on a performance metric.

[0066] According to the present disclosure, determining the second breathing target data and outputting updated breathing instructions may be carried out in real-time during a medical procedure, such as radiation treatment procedure and / or medical imaging procedure like a CT scan.

[0067] As already explained above, a user’s breathing pattern is important for medical procedures.

[0068] In medical imaging, improper breathing may lead to occlusions and / or motion artifacts, particularly when the imaging is a prolonged scan. During radiation treatment, the breathing pattern, particularly breath hold, ensures that irradiation can be precisely applied to the correct position, with anatomical features being in appropriate positions to ensure proper irradiation to the target anatomical features, e.g. organs, while avoiding irradiation of other anatomical features.

[0069] The method of the present disclosure may comprise providing a control signal for one or more components of a medical system, such as a radiotherapy treatment component and / or image acquisition component, so as to operate the one or more components in accordance with a breathing phase of the user, the breathing phase derived from the observed user breathing data. In particular, this may involve refraining from certain steps carried out by said components in case the observed user breathing data does not meet certain requirements and / or certain steps being carried out only during certain times in the breathing cycle, e.g. during breath hold.

[0070] Alternatively or in addition, the method of the present disclosure may comprise adapting the breathing instructions and / or updated breathing instructions to be output to the user based on feedback from the one or more components of the medical system, such as the radiotherapy treatment component. In particular, the one or more components may provide feedback, for example remaining treatment time, such as residual beam time, or the like. The breathing target data and corresponding breathing instructions may be modified in accordance with said feedback and optionally based on data indicating requirements for the user’s breathing associated with said feedback.

[0071] The present disclosure also provides a system comprising a processing system, the system configured to carry out the method according to the present disclosure, particularly of the method claims.

[0072] In particular, the processing system may be configured to determine the breathing instructions based on the first breathing target data, use the model to determine the second breathing target data based on the deviation of the observed user breathing data from the first breathing target data, and determine the updated breathing instructions based on the second breathing target data.

[0073] The system may comprise an output device, particularly a display device, the output device operatively connected to the processing system and configured to output the breathing instructions and the updated breathing instructions, and optionally outputting a performance indicator. For example, the output may comprise visualizations of the breathing instructions, such as visualizations representative of a breathing curve. Reference is made to the features described above in the context of the method regarding the visualization. Alternatively, an output device external to the system may perform the output of the breathing instructions. To that end, it may be communicatively coupled with the processing system and receive instructions from the processing system to output the breathing instructions.

[0074] The system may comprise one or more measurement devices operatively coupled to the processing system, the one or more measurement devices configured to obtain measurement data representative of a user’s breathing and to provide the measurement data, particularly to the processing system, for obtaining the observed user breathing data. For example, a user’s breathing may be determined from image data, optical scanning data, acceleration data, or the like. The skilled person will be able select appropriate measurement device for the application at hand. Alternatively, one or more measurement device external to the system may obtain measurement data representative of a user’s breathing. To that end, the one or more measurement devices may be communicatively coupled with the processing system and provide measurement data to the processing system.

[0075] The system may comprise one or more components of a medical system, in particular a radiotherapy treatment component and / or image acquisition component, operatively coupled to the processing system, the one or more components, in particular, configured to receive a control signal so as to operate in accordance with a breathing phase of the user, the breathing phase derived from the observed user breathing data. Alternatively, one or more components of a medical system external to the system may be configured to receive a control signal so as to operate in accordance with a breathing phase of the user. To that end, the one or more measurement devices may be communicatively coupled with the processing system and receive the control signal from the processing system.

[0076] The present disclosure also provides a computer program product comprising instructions which, when the program is executed by a computing system, cause the computing system to carry out and / or control the method according to the present disclosure, particularly of the method claims.

[0077] The present disclosure may, in particular, provide a computer program product comprising instructions which, when the program is executed by the processing system of the system according to the present disclosure, cause the system to carry out the method according to the present disclosure, particularly of the method claims.

[0078] The present disclosure also provides a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out and / or control the method according to the present disclosure, particularly of the method claims.

[0079] The present disclosure may, in particular, provide a computer-readable medium comprising instructions which, when executed by the processing system of the system according to the present disclosure, cause the system to carry out the method according to the present disclosure, particularly of the method claims.

[0080] The features and advantages outlined above in the context of the method similarly apply to the system, computer program product, and computer-readable medium of the present disclosure (or other claim categories).

[0081] The present disclosure provides a computer program which, when running on at least one processor (for example, a processor) of at least one computer (for example, a computer) or when loaded into at least one memory (for example, a memory) of at least one computer (for example, a computer), causes the at least one computer to perform the above-described method according to the first aspect. The invention may alternatively or additionally relate to a (physical, for example electrical, for example technically generated) signal wave, for example a digital signal wave, carrying information which represents the program, for example the aforementioned program, which for example comprises code means which are adapted to perform any or all of the steps of the method according to the first aspect. A computer program stored on a disc is a data file, and when the file is read out and transmitted it becomes a data stream for example in the form of a (physical, for example electrical, for example technically generated) signal. The signal can be implemented as the signal wave which is described herein. For example, the signal, for example the signal wave is constituted to be transmitted via a computer network, for example LAN, WLAN, WAN, for example the internet. The present disclosure therefore may alternatively or additionally relate to a data stream representative of the aforementioned program. to the present disclosure may provide a non-transitory computer-readable program storage medium on which the program according to the fourth aspect is stored.

[0082] For example, the invention does not involve or in particular comprise or encompass an invasive step which would represent a substantial physical interference with the body requiring professional medical expertise to be carried out and entailing a substantial health risk even when carried out with the required professional care and expertise. For example, the invention does not comprise a step of positioning a medical implant in order to fasten it to an anatomical structure or a step of fastening the medical implant to the anatomical structure or a step of preparing the anatomical structure for having the medical implant fastened to it. More particularly, the invention does not involve or in particular comprise or encompass any surgical or therapeutic activity. The invention is instead directed as applicable to guidance of a user’s breathing to adapt the user’s breathing to be suitable for a medical procedure, such as radiation treatment and / or medical imaging. This does not require any treatment step, as the breathing monitoring and adaptation is independent of the treatment. For this reason alone, no surgical or therapeutic activity and in particular no surgical or therapeutic step is necessitated or implied by carrying out the invention.

[0083] Use of the method or system

[0084] The present disclosure also relates to the use of the method and / or system of the present disclosure for a medical procedure, such as radiation treatment and / or medical imaging, such as CT scanning.

[0085] DEFINITIONS

[0086] In this section, definitions for specific terminology used in this disclosure are offered which also form part of the present disclosure.

[0087] Computer implemented method

[0088] The method in accordance with the present disclosure is for example a computer implemented method. For example, all the steps or merely some of the steps (i.e. less than the total number of steps) of the method in accordance with the present disclosure can be executed by a computer (for example, at least one computer). An embodiment of the computer implemented method is a use of the computer for performing a data processing method. An embodiment of the computer implemented method is a method concerning the operation of the computer such that the computer is operated to perform one, more or all steps of the method.

[0089] The computer for example comprises at least one processor and for example at least one memory in order to (technically) process the data, for example electronically and / or optically. The processor being for example made of a substance or composition which is a semiconductor, for example at least partly n- and / or p-doped semiconductor, for example at least one of II-, III-, IV-, V-, VI- semiconductor material, for example (doped) silicon and / or gallium arsenide. The calculating or determining steps described are for example performed by a computer. Determining steps or calculating steps are for example steps of determining data within the framework of the technical method, for example within the framework of a program. A computer is for example any kind of data processing device, for example electronic data processing device. A computer can be a device which is generally thought of as such, for example desktop PCs, notebooks, netbooks, etc., but can also be any programmable apparatus, such as for example a mobile phone or an embedded processor. A computer can for example comprise a system (network) of "sub-computers", wherein each sub-computer represents a computer in its own right. The term "computer" includes a cloud computer, for example a cloud server. The term "cloud computer" includes a cloud computer system which for example comprises a system of at least one cloud computer and for example a plurality of operatively interconnected cloud computers such as a server farm. Such a cloud computer is preferably connected to a wide area network such as the world wide web (WWW) and located in a so-called cloud of computers which are all connected to the world wide web. Such an infrastructure is used for "cloud computing", which describes computation, software, data access and storage services which do not require the end user to know the physical location and / or configuration of the computer delivering a specific service. For example, the term "cloud" is used in this respect as a metaphor for the Internet (world wide web). For example, the cloud provides computing infrastructure as a service (laaS). The cloud computer can function as a virtual host for an operating system and / or data processing application which is used to execute the method of the present disclosure. The cloud computer is for example an elastic compute cloud (EC2) as provided by Amazon Web Services™. A computer for example comprises interfaces in order to receive or output data and / or perform an analogue-to-digital conversion. The data are for example data which represent physical properties and / or which are generated from technical signals. The technical signals are for example generated by means of (technical) detection devices (such as for example devices for detecting marker devices) and / or (technical) analytical devices (such as for example devices for performing (medical) imaging methods), wherein the technical signals are for example electrical or optical signals. The technical signals for example represent the data received or outputted by the computer. The computer is preferably operatively coupled to a display device which allows information outputted by the computer to be displayed, for example to a user. One example of a display device is a virtual reality device or an augmented reality device (also referred to as virtual reality glasses or augmented reality glasses) which can be used as "goggles" for navigating. A specific example of such augmented reality glasses is Google Glass (a trademark of Google, Inc.). An augmented reality device or a virtual reality device can be used both to input information into the computer by user interaction and to display information outputted by the computer. Another example of a display device would be a standard computer monitor comprising for example a liquid crystal display operatively coupled to the computer for receiving display control data from the computer for generating signals used to display image information content on the display device. A specific embodiment of such a computer monitor is a digital lightbox. An example of such a digital lightbox is Buzz®, a product of Brainlab AG. The monitor may also be the monitor of a portable, for example handheld, device such as a smart phone or personal digital assistant or digital media player.

[0090] The disclosure relates to a program which, when running on a computer, causes the computer to perform one or more or all of the method steps described herein and / or to a program storage medium on which the program is stored (in particular in a non-transitory form) and / or to a computer comprising said program storage medium and / or to a (physical, for example electrical, for example technically generated) signal wave, for example a digital signal wave, carrying information which represents the program, for example the aforementioned program, which for example comprises code means which are adapted to perform any or all of the method steps described herein.

[0091] Within the framework of the present disclosure, computer program elements can be embodied by hardware and / or software (this includes firmware, resident software, micro-code, etc.). Within the framework of the present disclosure, computer program elements can take the form of a computer program product which can be embodied by a computer-usable, for example computer-readable data storage medium comprising computer-usable, for example computer-readable program instructions, "code" or a "computer program" embodied in said data storage medium for use on or in connection with the instruction-executing system. Such a system can be a computer; a computer can be a data processing device comprising means for executing the computer program elements and / or the program in accordance with the present disclosure, for example a data processing device comprising a digital processor (central processing unit or CPU) which executes the computer program elements, and optionally a volatile memory (for example a random access memory or RAM) for storing data used for and / or produced by executing the computer program elements. Within the framework of the present disclosure, a computer-usable, for example computer-readable data storage medium can be any data storage medium which can include, store, communicate, propagate or transport the program for use on or in connection with the instruction-executing system, apparatus or device. The computer-usable, for example computer- readable data storage medium can for example be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, apparatus or device or a medium of propagation such as for example the Internet. The computer-usable or computer-readable data storage medium could even for example be paper or another suitable medium onto which the program is printed, since the program could be electronically captured, for example by optically scanning the paper or other suitable medium, and then compiled, interpreted or otherwise processed in a suitable manner. The data storage medium is preferably a non-volatile data storage medium. The computer program product and any software and / or hardware described here form the various means for performing the functions of the present disclosure in the example embodiments. The computer and / or data processing device can for example include a guidance information device which includes means for outputting guidance information. The guidance information can be outputted, for example to a user, visually by a visual indicating means (for example, a monitor and / or a lamp) and / or acoustically by an acoustic indicating means (for example, a loudspeaker and / or a digital speech output device) and / or tactilely by a tactile indicating means (for example, a vibrating element or a vibration element incorporated into an instrument). Forthe purpose of this document, a computer is a technical computerwhich for example comprises technical, for example tangible components, for example mechanical and / or electronic components. Any device mentioned as such in this document is a technical and for example tangible device.

[0092] Artificial intelligence module

[0093] An artificial intelligence module is an entity that processes one or more inputs into one or more outputs by means of an internal processing chain that typically has a set of free parameters. The internal processing chain may be organized in interconnected layers that are traversed consecutively when proceeding from the input to the output.

[0094] Many artificial intelligence modules are organized to process an input having a high dimensionality into an output of a much lower dimensionality. For example, an image in HD resolution of 1920 x 1080 pixels lives in a space having a 1920 x 1080 = 2,073,600 dimensions. A common job for an artificial intelligence module is to classify images into one or more categories based on, for example, whether they contain certain objects. The output may then, for example, give, for each of the to-be-detected objects, a probability that the object is present in the input image. This output lives in a space having as many dimensions as there are to-be-detected objects. Typically, there are on the order of a few hundred or a few thousand to-be-detected objects.

[0095] Such a module is termed “intelligent” because it is capable of being “trained.” The module may be trained using records of training data. A record of training data comprises training input data and corresponding training output data. The training output data of a record of training data is the result that is expected to be produced by the module when being given the training input data of the same record of training data as input. The deviation between this expected result and the actual result produced by the module is observed and rated by means of a “loss function”. This loss function is used as a feedback for adjusting the parameters of the internal processing chain of the module. For example, the parameters may be adjusted with the optimization goal of minimizing the values of the loss function that result when all training input data is fed into the module and the outcome is compared with the corresponding training output data.

[0096] The result of this training is that given a relatively small number of records of training data as “ground truth”, the module is enabled to perform its job, e.g., the classification of images as to which objects they contain, well for a number of records of input data that is higher by many orders of magnitude. For example, a set of about 100,000 training images that has been “labelled” with the ground truth of which objects are present in each image may be sufficient to train the module so that it can then recognize these objects in all possible input images, which may, e.g., be over 530 million images at a resolution of 1920 x 1080 pixels and a color depth of 8 bits.

[0097] Neural network

[0098] A neural network is a prime example of an internal processing chain of an artificial intelligence module. It consists of a plurality of layers, wherein each layer comprises one or more neurons. Neurons between adjacent layers are linked in that the outputs of neurons of a first layer are the inputs of one or more neurons in an adjacent second layer. Each such link is given a “weight” with which the corresponding input goes into an “activation function” that gives the output of the neuron as a function of its inputs. The activation function is typically a nonlinear function of its inputs. For example, the activation function may comprise a “pre-activation function” that is a weighted sum or other linear function of its inputs, and a thresholding function or other nonlinear function that produces the final output of the neuron from the value of the pre-activation function.

[0099] Convolutional neural network

[0100] A convolutional neural network is a neural network that comprises “convolutional layers”. In a “convolutional layer”, the output of neurons is obtained by applying a convolution kernel to the inputs of these neurons. This greatly reduces the dimensionality of the data. Convolutional neural networks are frequently used in image processing.

[0101] Generative adversarial network A generative adversarial network is a combination of two neural networks termed “generator” and “discriminator”. Such a network is used to artificially produce records of data that are indistinguishable from records taken from a given set of training records of data. The generator network is trained with the goal of creating, from an input record with random data, an output record that is indistinguishable from the records in the set of training records. I.e., given that output record alone, it cannot be distinguished whether it has been produced by the generator or whether it is contained in the set of training records. The discriminator, in turn, is specifically trained to classify given records of data as to whether they are likely “real” training records or “fake” records produced by the generator. The generator and the discriminator thus compete against each other.

[0102] For example, a generative adversarial network may be used to create photorealistic images that are indistinguishable from a set of training images. From a limited number of training images obtained, e.g., by medical imaging, a near-infinite number of fake images that can pass for such medical images can be generated. A prime application of this is the production of training data for other artificial intelligence modules, e.g., modules that are to be trained to classify whether certain features or objects are present in medical images.

[0103] Acquiring data

[0104] The expression "acquiring data" for example encompasses (within the framework of a computer implemented method) the scenario in which the data are determined by the computer implemented method or program. Determining data for example encompasses measuring physical quantities and transforming the measured values into data, for example digital data, and / or computing (and e.g. outputting) the data by means of a computer and for example within the framework of the method in accordance with the present disclosure. The meaning of "acquiring data" also for example encompasses the scenario in which the data are received or retrieved by (e.g. input to) the computer implemented method or program, for example from another program, a previous method step or a data storage medium, for example for further processing by the computer implemented method or program. Generation of the data to be acquired may but need not be part of the method in accordance with the present disclosure. The expression "acquiring data" can therefore also for example mean waiting to receive data and / or receiving the data. The received data can for example be inputted via an interface. The expression "acquiring data" can also mean that the computer implemented method or program performs steps in order to (actively) receive or retrieve the data from a data source, for instance a data storage medium (such as for example a ROM, RAM, database, hard drive, etc.), or via the interface (for instance, from another computer or a network). The data acquired by the disclosed method or device, respectively, may be acquired from a database located in a data storage device which is operably to a computer for data transfer between the database and the computer, for example from the database to the computer. The computer acquires the data for use as an input for steps of determining data. The determined data can be output again to the same or another database to be stored for later use. The database or database used for implementing the disclosed method can be located on network data storage device or a network server (for example, a cloud data storage device or a cloud server) or a local data storage device (such as a mass storage device operably connected to at least one computer executing the disclosed method). The data can be made "ready for use" by performing an additional step before the acquiring step. In accordance with this additional step, the data are generated in order to be acquired. The data are for example detected or captured (for example by an analytical device). Alternatively or additionally, the data are inputted in accordance with the additional step, for instance via interfaces. The data generated can for example be inputted (for instance into the computer). In accordance with the additional step (which precedes the acquiring step), the data can also be provided by performing the additional step of storing the data in a data storage medium (such as for example a ROM, RAM, CD and / or hard drive), such that they are ready for use within the framework of the method or program in accordance with the present disclosure. The step of "acquiring data" can therefore also involve commanding a device to obtain and / or provide the data to be acquired. In particular, the acquiring step does not involve an invasive step which would represent a substantial physical interference with the body, requiring professional medical expertise to be carried out and entailing a substantial health risk even when carried out with the required professional care and expertise. In particular, the step of acquiring data, for example determining data, does not involve a surgical step and in particular does not involve a step of treating a human or animal body using surgery or therapy. In order to distinguish the different data used by the present method, the data are denoted (i.e. referred to) as "XY data" and the like and are defined in terms of the information which they describe, which is then preferably referred to as "XY information" and the like.

[0105] Analytical devices

[0106] The movements of the treatment body parts are for example due to movements which are referred to in the following as "vital movements". Reference is also made in this respect to EP 2 189 943 A1 and EP 2 189 940 A1 , also published as US 2010 / 0125195 A1 and US 2010 / 0160836 A1 , respectively, which discuss these vital movements in detail. In order to determine the position of the treatment body parts, analytical devices such as x-ray devices, CT devices or MRT devices are used to generate analytical images (such as x-ray images or MRT images) of the body. For example, analytical devices are constituted to perform medical imaging methods. Analytical devices for example use medical imaging methods and are for example devices for analysing a patient's body, for instance by using waves and / or radiation and / or energy beams, for example electromagnetic waves and / or radiation, ultrasound waves and / or particles beams. Analytical devices are for example devices which generate images (for example, two-dimensional or three- dimensional images) of the patient's body (and for example of internal structures and / or anatomical parts of the patient's body) by analysing the body. Analytical devices are for example used in medical diagnosis, for example in radiology. However, it can be difficult to identify the treatment body part within the analytical image. It can for example be easier to identify an indicator body part which correlates with changes in the position of the treatment body part and for example the movement of the treatment body part. Tracking an indicator body part thus allows a movement of the treatment body part to be tracked on the basis of a known correlation between the changes in the position (for example the movements) of the indicator body part and the changes in the position (for example the movements) of the treatment body part. As an alternative to or in addition to tracking indicator body parts, marker devices (which can be used as an indicator and thus referred to as "marker indicators") can be tracked using marker detection devices. The position of the marker indicators has a known (predetermined) correlation with (for example, a fixed relative position relative to) the position of indicator structures (such as the thoracic wall, for example true ribs or false ribs, or the diaphragm or intestinal walls, etc.) which for example change their position due to vital movements.

[0107] Treatment beam

[0108] The present disclosure relates to and / or is applied in the context of the field of controlling a treatment beam. The treatment beam treats body parts which are to be treated and which are referred to in the following as "treatment body parts". These body parts are for example parts of a patient's body, i.e. anatomical body parts.

[0109] The present disclosure relates to and / or is applied in the field of medicine and for example to the use of beams, such as radiation beams, to treat parts of a patient's body, which are therefore also referred to as treatment beams. A treatment beam treats body parts which are to be treated and which are referred to in the following as "treatment body parts". These body parts are for example parts of a patient's body, i.e. anatomical body parts. Ionising radiation is for example used for the purpose of treatment. For example, the treatment beam comprises or consists of ionising radiation. The ionising radiation comprises or consists of particles (for example, sub-atomic particles or ions) or electromagnetic waves which are energetic enough to detach electrons from atoms or molecules and so ionise them. Examples of such ionising radiation include x-rays, high-energy particles (high- energy particle beams) and / or ionising radiation emitted from a radioactive element. The treatment radiation, for example the treatment beam, is for example used in radiation therapy or radiotherapy, such as in the field of oncology. For treating cancer in particular, parts of the body comprising a pathological structure or tissue such as a tumour are treated using ionising radiation. The tumour is then an example of a treatment body part. The treatment beam is preferably controlled such that it passes through the treatment body part. However, the treatment beam can have a negative effect on body parts outside the treatment body part. These body parts are referred to here as "outside body parts". Generally, a treatment beam has to pass through outside body parts in order to reach and so pass through the treatment body part.

[0110] Reference is also made in this respect to the following web pages: http: / / www.elekta.com / healthcare_us_elekta_vmat.php and http: / / www.varian.com / us / oncology / treatments / treatment_techniques / rapidarc.

[0111] Arrangement of treatment beams

[0112] A treatment body part can be treated by one or more treatment beams issued from one or more directions at one or more times. The treatment by means of the at least one treatment beam thus follows a particular spatial and temporal pattern. The term "beam arrangement" is then used to cover the spatial and temporal features of the treatment by means of the at least one treatment beam. The beam arrangement is an arrangement of at least one treatment beam.

[0113] The "beam positions" describe the positions of the treatment beams of the beam arrangement. The arrangement of beam positions is referred to as the positional arrangement. A beam position is preferably defined by the beam direction and additional information which allows a specific location, for example in three-dimensional space, to be assigned to the treatment beam, for example information about its co-ordinates in a defined co-ordinate system. The specific location is a point, preferably a point on a straight line. This line is then referred to as a "beam line" and extends in the beam direction, for example along the central axis of the treatment beam. The defined co-ordinate system is preferably defined relative to the treatment device or relative to at least a part of the patient's body. The positional arrangement comprises and for example consists of at least one beam position, for example a discrete set of beam positions (for example, two or more different beam positions), or a continuous multiplicity (manifold) of beam positions.

[0114] For example, one or more treatment beams adopt(s) the treatment beam position(s) defined by the positional arrangement simultaneously or sequentially during treatment (for example sequentially if there is only one beam source to emit a treatment beam). If there are several beam sources, it is also possible for at least a subset of the beam positions to be adopted simultaneously by treatment beams during the treatment. For example, one or more subsets of the treatment beams can adopt the beam positions of the positional arrangement in accordance with a predefined sequence. A subset of treatment beams comprises one or more treatment beams. The complete set of treatment beams which comprises one or more treatment beams which adopt(s) all the beam positions defined by the positional arrangement is then the beam arrangement.

[0115] Imaging methods

[0116] In the field of medicine, imaging methods (also called imaging modalities and / or medical imaging modalities) are used to generate image data (for example, two-dimensional or three-dimensional image data) of anatomical structures (such as soft tissues, bones, organs, etc.) of the human body. The term "medical imaging methods" is understood to mean (advantageously apparatus-based) imaging methods (for example so-called medical imaging modalities and / or radiological imaging methods) such as for instance computed tomography (CT) and cone beam computed tomography (CBCT, such as volumetric CBCT), x-ray tomography, magnetic resonance tomography (MRT or MRI), conventional x-ray, sonography and / or ultrasound examinations, and positron emission tomography. For example, the medical imaging methods are performed by the analytical devices. Examples for medical imaging modalities applied by medical imaging methods are: X- ray radiography, magnetic resonance imaging, medical ultrasonography or ultrasound, endoscopy, elastography, tactile imaging, thermography, medical photography and nuclear medicine functional imaging techniques as positron emission tomography (PET) and Single-photon emission computed tomography (SPECT), as mentioned by Wikipedia.

[0117] The image data thus generated is also termed “medical imaging data”. Analytical devices for example are used to generate the image data in apparatus-based imaging methods. The imaging methods are for example used for medical diagnostics, to analyse the anatomical body in order to generate images which are described by the image data. The imaging methods are also for example used to detect pathological changes in the human body. However, some of the changes in the anatomical structure, such as the pathological changes in the structures (tissue), may not be detectable and for example may not be visible in the images generated by the imaging methods. A tumour represents an example of a change in an anatomical structure. If the tumour grows, it may then be said to represent an expanded anatomical structure. This expanded anatomical structure may not be detectable; for example, only a part of the expanded anatomical structure may be detectable. Primary / high-grade brain tumours are for example usually visible on MRI scans when contrast agents are used to infiltrate the tumour. MRI scans represent an example of an imaging method. In the case of MRI scans of such brain tumours, the signal enhancement in the MRI images (due to the contrast agents infiltrating the tumour) is considered to represent the solid tumour mass. Thus, the tumour is detectable and for example discernible in the image generated by the imaging method. In addition to these tumours, referred to as "enhancing" tumours, it is thought that approximately 10% of brain tumours are not discernible on a scan and are for example not visible to a user looking at the images generated by the imaging method. Medical Workflow

[0118] A medical workflow (also referred to as medical procedure) may comprise a plurality of workflow steps performed during a medical treatment and / or a medical diagnosis. The workflow steps are typically, but not necessarily performed in a predetermined order. Each workflow step for example means a particular task, which might be a single action or a set of actions. Examples of workflow steps are capturing a medical image, positioning a patient, attaching a marker, performing a resection, moving a joint, placing an implant and the like.

[0119] BRIEF DESCRIPTION OF THE DRAWINGS

[0120] In the following, the invention is described with reference to the appended figures which give background explanations and represent specific embodiments of the invention. The scope of the invention is however not limited to the specific features disclosed in the context of the figures, wherein

[0121] Fig. 1 illustrates a method for breathing according to the present disclosure;

[0122] Fig. 2 schematically illustrates a system according to the present disclosure;

[0123] Fig. 3 a schematic illustration of a breathing curve with multiple breathing cycles including a respective breath hold phase;

[0124] Fig. 4 a schematic illustration of gaming type visual elements and feedback;

[0125] Fig. 5 a first workflow schematically illustrating a method according to the present disclosure; and

[0126] Fig. 5 a second workflow schematically illustrating a method according to the present disclosure.

[0127] DESCRIPTION OF EMBODIMENTS

[0128] Fig. 1 illustrates the basic steps of a computer-implemented method for breathing guidance according to the present disclosure.

[0129] The method comprises providing, in step S11 , first breathing target data. For example, a first target breathing curve may be provided. The first breathing target data may be selected based on the application at hand, such as a medical procedure at hand, and may optionally be patient-specific or may be generic first breathing target data for the medical procedure, such as a generic breathing curve. The method comprises outputting, in step S12, breathing instructions to a user. The breathing instructions are based on the first breathing target data. The outputting may comprise visual and / or audio output. As an example, the first breathing target data may be visualized on a display device.

[0130] The method comprises providing, in step S13, observed user breathing data. The observed user breathing data is based on measurement data representative of a user’s breathing. For example, the observed user breathing data may be based on measurement data representative of a user’s breathing acquired while the user attempts breathing in accordance with the breathing instructions. The observed user breathing data may optionally be output, e.g. visualized, alongside the breathing instructions. This may allow for a patient to better control the breathing in accordance with the instructions and live supervision of the adherence to the breathing curve by a third party.

[0131] The method comprises using, in step S14, a model to determine second breathing target data based on a deviation of the observed user breathing data from the first breathing target data. For example, where a deviation was relatively large, the user may not yet be sufficiently able to control their breathing to follow along the current breathing instructions, so the first breathing target data may be adjusted to an easier level. On the other hand, if the deviation is relatively small, this may be an indication that the user is sufficiently able to control their breathing to following along the current breathing instructions, and more ambitious breathing instructions. This can allow users to progress from their current ability towards the breathing requirements at hand, e.g. the breathing requirements of a medical procedure.

[0132] The method comprises outputting, in step S15, updated breathing instructions to a user, the updated breathing instructions based on the second breathing target data. The user may then attempt following the updated breathing instructions.

[0133] The steps S13 to S15 may optionally be repeated until the second breathing target data and / or corresponding updated breathing instructions are in an acceptable range for a given application based on breathing requirements associated with said application. A determination step may be carried out as part of step S13 to determine whether the second breathing target data and / or corresponding updated breathing instructions are within the acceptable range.

[0134] The method may comprise, in optional step S16, providing a control signal for one or more components of a medical system, such as a radiotherapy treatment component, so as to operate the one or more components in accordance with a breathing phase of the user, the breathing phase derived from the observed user breathing data. The method may comprise, in optional step S17, adapting the breathing instructions and / or updated breathing instructions to be output to the user based on feedback from the one or more components of the medical system, such as the radiotherapy treatment component.

[0135] Any suitable system, particularly a system according to the present disclosure, may be used for carrying out the above-described method. For example, a system as illustrated in Fig. 2 may be used.

[0136] Fig. 2 schematically illustrates a system 1 according to the present disclosure. The system comprises a processing system 2 with one or more processing devices and is configured to carry out the method of the present disclosure, such as outlined in the context of Fig. 1 or Figs. 3 to 6, the general description, or the claims.

[0137] The processing system 2 may be configured to determine the breathing instructions based on the first breathing target data, use the model to determine the second breathing target data based on the deviation of the observed user breathing data from the first breathing target data, and determine the updated breathing instructions based on the second breathing target data.

[0138] Fig. 2 shows an output device 4, particularly a display device, the output device operatively connected to the processing system 2 and configured to output the breathing instructions and the updated breathing instructions, and optionally outputting a performance indicator. The output device may be part of system 1 or separate from system 1 and configured to exchange data with system 1 .

[0139] Fig. 2 also shows an input device 3 for receiving user input, which may be provided separately or formed integrally with the output device 4, such as in a touch display. The input device may be part of system 1 or separate from system 1 and configured to exchange data with system 1 . A user interface may, for example, be provided by device 4 or devices 3 and 4 together.

[0140] Fig. 2 shows a measurement device 5, operatively coupled to the processing system 2, the measurement device being configured to obtain measurement data representative of a user’s breathing and to provide the measurement data, particularly to the processing system 2, for obtaining the observed user breathing data. More than one measurement device may optionally be provided. The measurement device may be part of system 1 or may be separate from system 1 and configured to exchange data with system 1 .

[0141] Fig. 2 also schematically show a component 6a of a medical system 6, which may, for example, be a radiotherapy treatment component. The medical system may be part of system 1 , as shown here, or it may be a separate system and configured to exchange data with system 1 . The component 6a is operatively coupled to the processing system 2. The component 6a is configured to receive a control signal so as to operate in accordance with a breathing phase of the user, the breathing phase derived from the observed user breathing data.

[0142] Further features and advantages of the system and method will be understood from the discussion below.

[0143] The method for for breathing guidance according to the present disclosure comprises providing first breathing target data, which may, for example, comprise a first target breathing curve. The following examples and features will be described making reference to the example of target breathing curves, but the features are similarly applicable to other types of breathing target data.

[0144] In an example, a first target breathing curve a1 (t) may be provided. This is also referred to as first model breathing curve. It may be determined based, for example, on a general breathing curve model and optionally based on patient-specific data, such as indication, age, or the like, and / or procedure-specific data, such as for a specific medical procedure. For example, a general model may be modified based on patient-specific data and / or procedure-specific data and / or a modelling that take patient-specific data and / or procedure-specific data may be used as input for modelling a first target breathing curve. The first target breathing curve may comprise one or more deep inhalation breath hold periods (DIBHs) and may be limited to a specific time interval.

[0145] Fig. 3 illustrates an exemplary target breathing curve with multiple DIBHs (indicated by the vertical lines and connecting horizontal lines.

[0146] The method of the present disclosure comprises outputting breathing instructions to a user, the breathing instructions based on the first breathing target data. For example, the instructions may be provided on a display device.

[0147] In an example, the breathing instructions could be a representation of the target breathing curve on a display, for example as illustrated in Fig. 3.

[0148] However, as already described above, instruction may also be provided in a different manner, e.g., a gaming-type visualization. An example for such a visualization is provided in Fig. 4. The direction for the breath is represented by a path through which an icon navigates in accordance with the user’s breathing. An optional score, which may be a representation of success, may also be provided. The method also comprises providing observed user breathing data, the observed user breathing data based on measurement data representative of a user’s breathing. Such data may, for example, be obtained by a breathing measurement device, including a camera, distance measuring device, an acceleration device, or the like. More specific examples may be tracking cameras in a medical environment, mobile device cameras of phones or tablets. Further examples may include a depth-sensor device, for example, Kinect (which employs RGB data) or a head mounted display, HMD, device comprising multiple cameras and / or a depth sensor. Particularly, such HMD devices may comprise augmented or mixed reality glasses.

[0149] The measurements may be made while the user tries to follow the breathing instructions. The measurement data may comprise or be processed to obtain the observed user breathing data. In the present example, the observed user breathing data comprise an observed breathing curve b(t). The better the observed breathing curve matches the first target breathing curve, the closer the user’s breathing pattern is to a target breathing pattern.

[0150] The method of the present disclosure comprises determining second breathing target data based on a deviation of the observed user breathing data from the first breathing target data, e.g., in this example, a deviation of the observed breathing curve from the first target breathing curve.

[0151] The deviation may, for example, be determined for a specific time span, e.g. a breathing window. As an example, the deviation may be determined for a breathing window or time span that comprises one or multiple DIBHs and / or a preparatory time immediately before the start of DIBH.

[0152] A model may be used to determine the second breathing target data based on the deviation. The second breathing target data may comprise a second or adapted breathing target curve a1+i(t). The model may optionally involve a machine learning model, e.g. based on reinforcement learning. However, non-ML models can alternatively be used. The skilled person would select an appropriate model based on the application at hand. For example, when relatively few and / or simply characteristics of breathing are observed and adapted in the breathing curve, such as length of DIBH, a ML-model can be omitted. The adapted breathing curve may simple include the same, shorter or longer DIBH based on how close the user’s observed DIBH length is to the target DIBH length of the current breathing target data. The amount of change and direction of change may use a relatively simple non-ML based model and still yield very accurate results in such an example.

[0153] After determining the second breathing target data, the method comprises outputting updated breathing instructions the second breathing target data, e.g. outputting the updated breathing curve. The outputting may be carried out in the same way or similarly to the outputting of breathing target data, e.g. target breathing curve, described above. The above steps may also be carried out repeatedly in iterations, wherein after second breathing target data is determined for the first time, for each iteration the second breathing target data of the preceding iteration is output to the user and used for determining the deviation and may then be updated based thereon. There may be a stop criterion for the repetitions that can be chosen according to the case at hand. For example, the iterations may lead towards procedure-specific breathing target data, e.g. specified in a definition of a medical workflow. The stop criterion may be that characteristics of the observed user breathing data are within an acceptable range of the procedure-specific breathing target data. For example, where a certain breath hold length and stability in the breath hold period is specified in the procedure specific breathing target data, a stop criterion may be that the observed user breathing data exhibit sufficient breath hold length and stability that are in an acceptable range compared to breath hold length and stability in the breath hold period as specified in the procedure specific breathing target data.

[0154] After the above steps, for example after the stop criterion is reached, breathing instructions based on the latest second breathing target data, e.g. target breathing curve, may be output as described above and a medical procedure may be carried out concurrently. At the same time, the user may be observed while following the breathing instructions. This may yield observed user breathing data, e.g. breathing curve, T(t). The observed user breathing data, e.g. curve T(t), may be used for providing a control signal. For example, based on a deviation between characteristics of the observed user breathing data, e.g. curve T (t), and the latest second breathing target data, a control signal may be issued to operate a piece of equipment only if the deviation does not exceed a threshold. For example, a radiotherapy beam may be turned on when it is determined that the threshold is not exceeded.

[0155] Optionally, there may be feedback from the medical procedure that may be used to further adapt the second breathing target data.

[0156] In the above examples, a quality measure may be determined based on the respective determined deviations and optionally, the measure may be output to the user (e.g. live) while the user tries following the breathing instructions, e.g., alongside the breathing instructions.

[0157] In Figure 5, an exemplary workflow for a method according to the present disclosure is shown. The workflow illustrates the use of the model (in the circular process indicated by the “online adaptation”). Merely for the sake of illustration the workflow also comprises the preceding step of training the model using data from a training database. This step is optional in the method of the present disclosure, as not all models require training or a pre-trained model may be employed, for example. The workflow illustrates that data from a training data base flows into the model. This will occur during training. T raining can be offline and optionally online. That is, the model may be trained before use (offline) and optionally also during use (online).

[0158] The use of the trained model may be that observed user breathing data and first breathing target data are input into the model or a deviation between the observed user breathing data and first breathing target data is input into the model (depending on how the model is configured and / or trained). The model outputs second breathing target data, also referred to as adapted breathing target data, such as the breathing curve shown in the workflow. In an iterative process, the first breathing target data may be an output of the model, in particular the adapted / second breathing target data from the preceding iteration. Thus, a loop-type adaptation of the breathing target data can be carried out. During carrying out the loop-type adaptation, breathing instructions representative of the respective iteration’s first breathing target data may be output to the observed user and the observed user breathing data may be obtained by monitoring the observed user. Accordingly, the loop-type adaptation may be referred to as online adaptation. It is to be understood that the user is supported to at the same time follow the breathing instructions. However, if user does not do so properly, the method will still work. The adaptation could then simply result in slower progress and / or more iterations, for example.

[0159] Figure 6 shows a similar workflow to Figure 5, wherein the right part of Figure 5 (“online adaptation”) is shown as the third element from the left. Accordingly, reference is made to the description of the workflow in the context of Figure 5 for elements shown in both figures. The workflow in Figure 6 additionally comprises the optional step, after the online adaptation, of obtaining a final curve. This may be the curve which will be output to the user during a medical procedure, optionally without any further adaptation thereof. Moreover, optionally, the final curve together with data providing context to said final curve, may be fed back to the training database. It can optionally be used for online training of the model and / or may be used for training other models, for example.

[0160] The method of the present disclosure, particularly the last step shown in Figure 6, allows for relative independence from external data sources. As each time the method is carried out, the database can be fed, in principle much of the training can be carried out with a basic pretrained model refined with data collected over the course of carrying out the method repeatedly, e.g. for different users, sessions, medical procedures, and the like.

[0161] As will be understood from the above discussion, overall, the method of the present disclosure allows for improving breathing patterns of a user, for example a patient of a medical procedure. A particular example where the method can be advantageous is in the context of DIBH workflows. DIBH breathing can be challenging for a patient, but at the same time it is often important for the success of the medical procedure. In the present disclosure, the second breathing target data may be obtained using an Al-backed system that analyzes the patient’s performance (based on observed breathing data) and the method may dynamically adjust the “difficulty” (e.g. breath hold duration, stability, amplitude) of the level. This can reduce overall time of a medical procedure, e.g., where the procedure takes place during DIBH, by increasing the breath hold time of the patient. Additionally, by adjusting to the patient’s specific capabilities rather than focusing only on a treatment plan, anxiety can be reduced and pain relieved.

[0162] A special example already addressed briefly above of how breathing instructions can be provided advantageously, are rendering of the instructions (such as amplitude or duration of breath hold) based on gaming, e.g. use of gamification elements. Games are made for particularly good processing by the human physiology. In an example, adapted breathing target data can also be seen as generated levels, as explained below.

[0163] In some examples of the present disclosure, different breathing target data may be associated with what may be referred to as levels, such as levels in a computing game, where different levels have different goals. The model could be seen as provided a level generator. Over the course of several levels, the goals may be such that the amount of time of breath hold is increased. It is noted that when breath hold time is high in individual breathing cycles, often this means that overall duration of a medical procedure can be shortened. Accordingly, there is not necessarily a specific global / overall goal to reach in terms of an absolute value of the breath hold duration, but, instead, the increase in duration may be considered to be the overall goal. The present disclosure may comprise adjusting upcoming level parts by analyzing the performance of preview parts. This allows for taking into account the patient’s capabilities and avoid overwhelming the patient, for example.

[0164] It is noted that, for DIBH workflows a patient is required to perform breath hold patterns. By gamification of this process the treatment experience of the patient and medical personal can be also be improved in addition to the above benefits. A reward system may be provided that playfully entices the patient to follow specific breathing patterns. This reduces pressure and anxiety compared to other types of breathing instructions and will, as explained above, improve overall breathing results because stress will make the controlled breathing more difficult.

[0165] An exemplary way of employing gaming-type visual elements. For example, a height of a gaming figure moving through its surroundings may be coupled directly to the breathing signal of the patient. Following a proper level design the aimed breathing signal can therefore be achieved. The output might be provided to the user using VR or a screen visible to a patient. A point-based reward system can be used to directly give feedback on the performance to the patient without increasing stress and pressure. Implementing a balanced and gratifying reward system might influence the experience and performance for the user.

[0166] The present disclosure is particularly advantageous when used during a medical procedure, e.g. radiation treatment or imaging. It can also be used for preparing a user for such a procedure. The method can be implemented independent of the hardware. For example, it may be run on any generic computing system, including hand-held devices. The method may allow for avoiding anxiety and provide stress relief, thereby improving the breathing capabilities of the user. A user can still be pushed towards higher difficulty and performance of breathing. The adjustment can be done on- the-fly, adapted to the current situation, circumstances, and capabilities, which even for the same person may be different at different times. Better breathing performance can reduce overall duration of a medical procedure as outlined above.

[0167] Some safety measures can be put into place that ensure that the modelled second breathing target data do not lead to undesired results. For example, the determination thereof may involve boundary conditions and / or constraints (with respective penalizations) such as maximum and / or minimum breath hold duration and / or amplitude. Threshold learning may be employed that is based on using a base curve and learning deviations. Constraints may be employed during the use phase and / or during the training phase of the model.

[0168] As the above discussions have shown, an adaptive method can be provided by the present disclosure that may include creating an optimal target breathing curve for the user, e.g. patient, for medical procedures that involve breath hold, particularly DIBH.

[0169] In that context, various models can be used. An exemplary model type is a reinforcement learning model. Such a model can be trained on data from past medical procedures involving breath hold. Data inputs into the model may include breathing target data, e.g. target breathing curves, observed user breathing data, e.g. an observed breathing curve, goals such as amplitude / level during breath hold and / or breath hold time and / or breath hold stability. Such goals may be extracted, for example, from a treatment plan, optionally in combination with breathing data for the user and / or a plurality of users. Data inputs that can be used to refine the model may include age, conditions, historical breathing data, data from previous breathing cycles, or the like.

[0170] Thus, it will be understood that the present method allows for live (real time) curve generation and adaptation based on the current user, e.g. patient, specifically their current breathing performance.

[0171] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered exemplary and not restrictive. The invention is not limited to the disclosed embodiments. In view of the foregoing description and drawings it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention, as defined by the claims.

Claims

Brainlab AGAttorney’s File: B19286WOCLAIMS1 . A computer-implemented method for breathing guidance, the method comprising: providing (S11) first breathing target data; outputting (S12) breathing instructions to a user, the breathing instructions based on the first breathing target data; providing (S13) observed user breathing data, the observed user breathing data based on measurement data representative of a user’s breathing; using (S14) a model to determine second breathing target data based on a deviation of the observed user breathing data from the first breathing target data; and outputting (S15) updated breathing instructions to a user, the updated breathing instructions based on the second breathing target data.

2. The method of claim 1 , wherein the model is configured to output second breathing target data such that the second breathing target data has a different difficulty level compared to the first breathing target data, according to on one or more predefined difficulty metrics, in particular, wherein the difficulty metrics pertain to a total breath hold time, how many breath holds are required to accomplish a target total breath hold time, duration of one or more individual breath hold periods, and / or breath hold consistency.

3. The method of claim 1 or 2, wherein the model is configured such that a change in difficulty level between the first breathing target data and the second breathing target data depends on the observed user breathing data, particularly on the deviation of the observed user breathing data from the first breathing target data.

4. The method of any of the preceding claims, wherein the model is a machine learning, ML, model trained to output, based on inputs comprising breathing target data and observed user breathing data, second breathing target data.

5. The method of any of claim 4, wherein the model is a model trained on past breathing data, in particular based on a previous session and optionally a current session, in particular by reinforcement learning.

6. The method of any of the preceding claims, wherein the deviation is determined based on only a subset of the breathing data, the subset being associated with only a part of the breathing cycle, in particular only with the breath hold period of abreathing cycle or only for a preparatory period prior to the breath hold cycle and the subsequent breath hold period, in particular, wherein the updated breathing instructions are updated for the part of the breathing cycle associated with the subset of the breathing data from which the deviation is determined.

7. The method of any of the preceding claims, wherein the first and / or second breathing target data comprise a target level and / or a target time and / or a target breathing curve; and / or wherein the observed user breathing data comprise an observed level and / or an observed time and / or an observed breathing curve; and / or wherein additional inputs may be provided to the model, comprising at least one of: age, gender, conditions, historical observed user breathing data, or current measurement data, such as heart rate, maximal oxygen uptake, and / or temperature.

8. The method of any of the preceding claims, wherein first breathing target data comprise a target breathing curve a1 (t), wherein the observed user breathing data comprises an observed breathing curve b(t) derived from the measurement data, and wherein the second breathing target data comprise an updated target breathing curve a1+i(t).

9. The method of any of the preceding claims, wherein the breathing instructions comprise a visualization representative of the target breathing curve a-i(t) and the updated breathing instructions comprise a visualization representative of the updated target breathing curve ai+j(t), and / or wherein the method comprises outputting, alongside a / the visualization representative of the target breathing curve and / orthe updated target breathing curve, a visualization of the observed breathing curve and / or outputting a performance indicator representative of a performance calculated from the deviation of the observed user breathing data from the first breathing target data and / or second breathing target data based on a performance metric.

10. The method of any of the preceding claims, wherein determining the second breathing target data and outputting updated breathing instructions is carried out in real-time during a medical procedure, such as radiation treatment procedure.11 . The method of any of the preceding claims, comprising providing (S15) a control signal for one or more components of a medical system, such as a radiotherapy treatment component, so as to operate the one or more components inaccordance with a breathing phase of the user, the breathing phase derived from the observed user breathing data, and / or comprising adapting (S16) the breathing instructions and / or updated breathing instructions to be output to the user based on feedback from the one or more components of the medical system, such as the radiotherapy treatment component.

12. A system (1) comprising a processing system (2), the system configured to carry out the method of any of the preceding claims, in particular, the processing system configured to determine the breathing instructions based on the first breathing target data, use the model to determine the second breathing target data based on the deviation of the observed user breathing data from the first breathing target data, and determine the updated breathing instructions based on the second breathing target data.

13. The system of claim 12, further comprising an output device (4), particularly a display device, the output device operatively connected to the processing system and configured to output the breathing instructions and the updated breathing instructions, and optionally outputting a performance indicator; and / or one or more measurement devices (5) operatively coupled to the processing system, the one or more measurement devices configured to obtain measurement data representative of a user’s breathing and to provide the measurement data, particularly to the processing system, for obtaining the observed user breathing data; and / or one or more components (6a) of a medical system (6), in particular a radiotherapy treatment component, operatively coupled to the processing system, the one or more components, in particular, configured to receive a control signal so as to operate in accordance with a breathing phase of the user, the breathing phase derived from the observed user breathing data.

14. A computer program product comprising instructions which, when the program is executed by a computing system, cause the computing system to carry out and / or control the method of any of claims 1 to 11 .

15. A computer readable medium having stored thereon instructions which, when the program is executed by a computing system, cause the computing system to carry out and / or control the method of any of claims 1 to 11 .

Images