Method for operating a dialysis device training system, dialysis device training system, medical system and augmented reality device

Abstract

The invention relates to training of a user of a medical device, in particular a dialysis device. To this end, instructions for performing a task to be trained are calculated and an aid for performing the task is provided to the user by means of an AR device. In particular, the AR device can also monitor the activity of the user, so that a response of the medical device to the activity of the user can be calculated and relevant virtual feedback can be provided to the user.

Description

[0001] This invention relates to a method for operating a dialysis device training system, particularly a dialysis device training system with an augmented reality (AR) device. The invention also relates to a dialysis device training system. Furthermore, the invention relates to a medical system and an AR device. Specifically, the invention relates to calculating instructions for training using an AR device.

[0002] Although the invention is described below by way of example with reference to a dialysis device, the general concept can also be applied to other devices, particularly other medical devices, especially blood therapy devices.

[0003] More and more medical devices are becoming increasingly complex. An example of such a complex medical device is a dialysis unit. Such devices require complex operations and configurations that the user must perform. Furthermore, many accessories and consumables must be considered when operating a medical device.

[0004] Because improper operation can be dangerous for patients, it is crucial that users, such as nurses or doctors, are familiar with the device. In uncertain situations, it is desirable to be able to guide users quickly, safely, and reliably to complete tasks.

[0005] Currently, users must read the user manual for each device to learn how to operate it. Furthermore, even if a user can be trained by one or more human trainers, the user and trainer must be in the same location for the training purpose. This can be very laborious and costly. In addition, human trainers are not always readily available.

[0006] Therefore, there is a need for a training system for medical devices such as dialysis machines that enables users to be trained even in the absence of a human trainer. Summary of the Invention

[0007] This invention provides a method for operating a dialysis device training system having the features of the independent claims, a dialysis device training system, a medical system, and an augmented reality device. Further advantageous embodiments are the subject of the dependent claims.

[0008] According to a first aspect, a method for training a dialysis device operation system is provided. Preferably, the method is computer-implemented. The method employs an augmented reality (AR) device. The AR device is used by a user while operating the dialysis device. The method includes the step of selecting a task to be trained. The task to be trained is performed on the dialysis device. The method also includes the step of calculating user instructions to guide the user in completing the task to be trained, particularly while the user is operating the dialysis device. Furthermore, the method includes the step of monitoring the user's activities, particularly the user's activities while operating the dialysis device. The user's activities can be monitored by at least one sensor component, particularly the sensor component of the AR device. The method may also include determining or algorithmically calculating or processing assistance having an instruction sequence. The method also includes the step of providing (particularly the determined) assistance. The provided assistance can be calculated assistance for performing the task to be trained. The assistance is generated or calculated based on the calculated user instructions and the monitored activities. In particular, the provided assistance may include outputting instructions on an output component, particularly the output component of the augmented reality device. Generally, the term "assisting the user" refers to providing assistance through technical means, particularly by controlling the AR device.

[0009] The first aspect relates to the claimed method. Features, advantages, or alternative implementations described or claimed with respect to the method can also be transferred or assigned to other claimed objects (e.g., training systems, medical systems, augmented reality devices), and vice versa. In other words, the apparatus can be improved using features described or claimed in the context of the method, and vice versa. In this case, the functional features of the method are embodied by the structural units of the device or apparatus or system, and vice versa. Generally, in computer science, software implementations and corresponding hardware implementations (e.g., as embedded systems) are equivalent. Therefore, for example, method steps for “storing” data can be performed using a storage unit and corresponding instructions for writing data into the storage device. To avoid redundancy, although the apparatus can also be used in alternative implementations described with reference to the method, these implementations are no longer explicitly described with respect to the apparatus. In principle, the claimed apparatus claims or device claims are suitable for performing the claimed method.

[0010] According to a second aspect, a dialysis device training system is provided. The dialysis device training system includes an augmented reality (AR) device and a control device. The control device can be configured to execute the method according to the first aspect. Specifically, the control device is configured to select a task to be trained on the dialysis device. The control device is also configured to calculate user instructions to guide the user in completing the task to be trained, particularly when the user operates the dialysis device. Furthermore, the control device is configured to monitor the user's activities. Activity can be monitored, for example, through at least one sensor component, particularly a sensor component of the AR device. Specifically, the user's activities are monitored when the user operates the dialysis device by executing the calculated user instructions. The control device is also configured to provide assistance for performing the task to be trained. Assistance is calculated or generated based on the calculated user instructions and the monitored activities. Specifically, assistance may include outputting instructions on an output component, particularly an output component of the AR device.

[0011] According to another aspect, a medical system is provided. The medical system includes at least one dialysis device and a dialysis device training system according to the second aspect.

[0012] According to another aspect, an AR device is provided. The AR device can be used in a dialysis device training system according to the second aspect. The AR device may include an output device and / or sensor components. The output device may be configured to provide instructions based on determined assistance when training a user on a dialysis device. The sensor components may be configured to capture data related to the dialysis device. Alternatively or additionally, the sensor components may be configured to capture data related to the user's activities operating the dialysis device.

[0013] According to another aspect, a computer program is provided. The computer program includes computer instructions for performing the methods described above.

[0014] This invention is based on the finding that operating complex medical devices such as dialysis machines is a significant user need. Users must study user manuals and undergo extensive training to use such devices. Furthermore, training on actual medical devices may be necessary to familiarize users with them. However, human trainers are not always available. Moreover, human trainers typically require additional time and financial investment.

[0015] Therefore, the present invention takes into account this finding and aims to provide a method for a technical solution for training users on medical devices such as dialysis machines. In particular, the present invention aims to provide a solution that enables improved training to be provided to users without the need for additional human trainers.

[0016] Therefore, the present invention provides a system for training users on complex medical devices using techniques such as AR devices and related control systems. In this way, not only can human trainers be eliminated, but such a system can also be customized to the needs and characteristics of the users to be trained.

[0017] The approach based on one aspect mentioned above can include the use of a Large Language Model (LLM). Specifically, an LLM can be used to provide assistance to a user, for example, in the form of computational instructions. An LLM can be trained to acquire the aforementioned capabilities by learning statistical relationships from text documents or other information sources or files during computationally intensive self-supervised and semi-supervised training processes. An LLM can be constructed using a decoder-only transformer-based architecture. Alternatively or additionally, other architectures such as recurrent neural network variants and Mamba (state-space model) can be used. In particular, an LLM can be used to obtain answers to questions about a dialysis device. The answers can be provided on the output of an AR device or on the output of a dialysis device or a user's (e.g., mobile) device. A user can use an LLM to query the system, for example, which part of the machine is currently in front of them, where disposable items are placed, where a certain part of the machine (e.g., a blood pump) is located, or request assistance for the next step. The system utilizes AR sensors and / or at least sensor components to identify the situation in the room, uses an LLM to accordingly evaluate the user's verbal interactions, and provides corresponding information, such as AR highlighting, text, arrows, and / or prepared videos and / or graphics, to answer verbal user queries. This can be done in any language if the LLM has appropriate language capabilities. The system can respond automatically in the language of the query.

[0018] In the context of this invention, an AR device can be understood as any type of device that provides AR features. For example, such an AR device could be an AR headset or a mixed reality headset, an XR headset, such as, for example, the Microsoft HoloLense. However, any other type of AR device or XR device is also possible. An AR device can also be any other type of device that provides AR features. For example, a smartphone, tablet computer, or any other type of device with a camera and a display can be used as an AR device. Generally, any type of device with the ability to provide an overlay of a real scene with additional (artificial) elements can be used as an AR device. In this context, "augmentation" refers to the fact that a real or physical scene can be enhanced or enriched using additional information, which can be provided as virtual instructions.

[0019] In the context of this invention, the output component can be any type of component for outputting visual, tactile, and / or auditory instructions. The output component can be implemented at least partially by the AR device mentioned above. In particular, the output component can be part of the AR device. For example, the output component can include a display for providing visual information or instructions. Specifically, the output component can include a display device for overlaying a real scene with additional virtual elements / instructions. For this purpose, a semi-transparent display can be used to provide additional instructions on the real scene pointed to by the display. Alternatively, the scene can be captured by a camera device and / or sensor components, and image data from the camera device and / or sensor components can be overlaid on the display with additional virtual instructions. Alternatively or additionally, the output component can be implemented on another device, particularly a mobile device and / or a medical device, although this output component is referred to as "the output component of the AR device." In the latter case, the output component can be outsourced or moved from the AR device to the outside of the AR device.

[0020] Typically, indications can be any type of data provided or represented on the output component. Indications can be provided in various forms, particularly visual, auditory, tactile, and / or other forms. Output components are selected accordingly (optical output, speaker output, vibration output, etc.). Different output components can be combined.

[0021] Visual cues can be any type of visual and / or graphic representation that can be used in the context of operating a medical device or its related parts (consumables, filters, etc.). For example, visual cues can be graphic elements, icons, images (e.g., photographs or diagrams of parts of a medical device), lines, arrows, boxes, circles or any other geometric elements, text boxes with text and / or other information, video sequences, and avatars. Cues, especially visual cues, can be static and / or dynamic, such as video sequences, moving or rotating graphic elements, etc.

[0022] Auditory cues can be, for example, sounds such as beeps or warning tones, sound sequences, melodies, or spoken text. Spoken text can be artificially generated text messages or text messages recorded by humans. However, any other type of auditory cue is also possible.

[0023] Tactile cues can be, for example, vibration, shaking, and / or any other tactile output.

[0024] In the context of this invention, the sensor component can be any type of input component, particularly any type of component for sensing environmental properties and / or for receiving user input, such as a user interface (UI), graphical user interface (GUI), etc. For example, the sensor component may include optical sensor elements, such as at least one camera device, at least one laser scanner, etc. The sensor component may alternatively or additionally include auditory sensor elements such as a microphone, which can be used, for example, to record user voice commands. The sensor component may alternatively or additionally include sensor elements such as a gyroscope, an accelerometer (for one or more dimensions), and / or a distance sensor, or any other type of sensor for sensing the user's position, posture, or movement. The sensor component may alternatively or additionally include one or more communication modules. For example, such a communication module may be a Wi-Fi interface, a Bluetooth interface, an RFID reader, or any other wireless communication module or wired communication module. However, it should be understood that the sensor component may also include any other type of sensor element.

[0025] The sensor component or a portion thereof may be part of the augmented reality device. Alternatively or additionally, particularly, the sensor component may be provided as an external device outside the augmented reality device. The sensor component may be at least partially included in a medical device and / or in another device. Data exchange is provided. The sensor component may be part of a mobile device (e.g., a smartphone, tablet, etc.).

[0026] One or more optical sensor elements of the sensor component can be used to capture images of the user's environment. Specifically, the optical sensor can capture image data of the environment the user is currently viewing. The image data captured by the optical sensor can, for example, be used to identify a medical device currently being operated by the user. In particular, the image data can be used to identify accessories and / or fittings mounted on or available in the environment of the medical device. For example, medical devices, accessories, or fittings can be identified based on pattern recognition methods using characteristic elements for identifying individual components. Scanning codes, such as QR codes, can also be used to identify components. However, any other method for identifying devices or other components is also possible.

[0027] Data from sensor components, particularly from one or more (especially optical) sensors, can also be used, for example, to identify the operational settings of a medical device. Operational settings can refer to the current technical configuration and / or state of the medical device and / or attributes of the medical device (such as, for example, the location of devices, device parts, and / or connected disposable items and / or accessories to be used for device operation). For example, image data can be analyzed to identify the current settings of control elements and / or to analyze the output of the display on the medical device.

[0028] Furthermore, data from sensor components, such as data from one or more (e.g., optical) sensors, can be used to monitor and / or evaluate user actions. For example, an optical sensor can capture image data of the user's hand. Such image data can be used to analyze current actions, such as movement toward a control element, actions for attaching or removing accessories, patient-related movements, or any other manual actions by the user.

[0029] Image data from one or more optical sensors can also be used, for example, to assess the available space around the user and medical device. For instance, the dimensions of the room where the user and medical device are located can be determined. Therefore, space constraints can be considered in situations where space is limited due to small rooms or other devices in the environment.

[0030] Furthermore, the spatial location of the user and / or medical device can be determined as an operational setting based on image data and / or other available data. For example, it can be determined whether the user and medical device are located at a hospital, a medical facility, a patient's home, etc. Additionally, other infrastructure can be identified as an operational setting, which will be available and / or necessary for operating the medical device.

[0031] The interface or communication module can be used for any type of communication purpose. Communication can be performed wirelessly or via wired communication. For example, communication can be established with a medical device. Such communication with a medical device can be used, for example, to determine the type of the medical device, determine the current configuration of the medical device, determine the status of the medical device, and / or obtain error messages from the medical device. Furthermore, communication can be established with one or more databases. For example, a database can be referenced to obtain additional information about the medical device, user information such as user preferences, training status, etc., patient information, operational information related to the medical device, receive updates such as those from the manufacturer, receive supplementary information to assist the user, etc. However, it should be understood that any other type of information can also be obtained from one or more databases.

[0032] The interface or communication module can also be used to establish any type of communication with either a human or artificial expert. In the case of a human expert, information for assisting the user can be obtained from a local or remote computing device. In the case of a human expert, a communication channel can be established with such a human expert to obtain additional assistance from the expert and / or to provide the expert with information about the medical device and / or the current user. In such a case, the human expert can be located near either the medical device or the user. For example, the expert and the user can be located in the same room. Alternatively, the expert can be spatially separated and located in another room, an external building, another city, country, or continent. When the human expert is spatially separated, any type of appropriate information can be provided to the remote expert, such as the configuration of the medical device, the status / fault messages of the medical device, attached or available accessories, user preferences, etc. Additional real-time information can also be provided to the human expert, such as images captured by a camera device, voice communication with the user, etc. Such information can be provided to the human expert, for example, through any type of output device such as a speaker, headset, smartphone or tablet computer, monitor, VR device, AR device, or XR device.

[0033] Alternatively or otherwise, a human expert may use a second AR device that provides the same or similar indications as the AR device being used by the user, indicating that the user has requested or will be provided with assistance. In this way, the outputs on the AR device being used by the user and the second AR device being used by the human expert are comparable, so as to simulate the user's situation at the medical device as realistically as possible.

[0034] In the context of this invention, the task to be trained can be any type of operation or action related to the relevant medical device or its related parts (such as consumables). The task may require at least one activity, and in particular a set or series of activities to be performed manually by the user or automatically via instructions. Such a task could be, for example, preparing a medical device for a desired task / work (e.g., setting up a medical device, particularly a dialysis machine, especially before commissioning the device). For this purpose, the user may have to install one or more accessories or attachments on the medical device, insert or connect one or more consumables, perform specific setups on the medical device, or perform any other required operations. Patient-specific data must be considered if necessary. The task could be, for example, monitoring the medical device during operation. For this purpose, the user may have to monitor the status messages of the medical device, monitor the fill level of consumables, monitor flow rates, etc. If any parameter exceeds tolerance, the user must react to ensure safe operation. In another example, the task could be, for example, reacting to a malfunction or error message. For example, if the medical device detects a malfunction or problem, the user must take the necessary steps (activities) to eliminate the cause of such a problem. However, it should be understood that the task may include any other type of operation related to the relevant medical device, such as maintenance, normal operation, testing, or training.

[0035] The task to be trained can be specified or selected by the user, for example, through user input. User input can be received on a user interface. The user interface can be deployed on the dialysis device and / or AR device and / or control device. Alternatively or concurrently, the task can be determined fully or at least partially automatically. Furthermore, it is also possible for others, such as trainers or instructors, to select the task that should be performed by the user. Such a task to be trained, at least partially in an automated manner, can also be determined or selected. For example, historical data, such as data stored in a relevant user database, can be referenced to assess the user's current training level and to identify appropriate tasks to be trained. However, any other methods for automatically and / or computer-aidedly determining the task to be trained are also possible.

[0036] The assistance can be any type of assistance that can be used to help a user operate a medical device. For example, such assistance may include information to guide the user through a desired or selected task, such as configuring the medical device, attaching accessories, adjusting the medical device for a specific patient, operating the medical device, performing steps in response to warnings or error messages, refilling consumables, and / or performing maintenance operations.

[0037] The method according to the invention may include an algorithmic calculation or determination of assistance. The algorithmic determination of assistance may define the content of the assistance in the form of, for example, a guidance message or instruction. The determined assistance is provided according to the determined assistance scheme. Assistance is provided on an output component, which may be implemented at least partially on an AR device and optionally on a related device, such as a mobile device assigned to a user and / or a medical device and / or a portion thereof. Assistance may include indicative outputs; for example, assistance may include a description of a single step or a series of steps that should be performed by the user to perform a desired task. Assistance may include, for example, information about the proper setup of the medical device, information about additional components or consumables required, etc. Assistance may also include, for example, instructions on one or more specific locations on the medical device or related components, instructions on how to operate control elements of the medical device, instructions on how to handle or grasp specific components, or any other information related to the desired task to be performed. Assistance may include or refer to visual, auditory, or tactile elements. For example, assistance may refer to or include tactile events, such as shaking or vibration, to attract the user's attention. Assistance can refer to or include auditory elements, such as beeps, warning sounds, melodies, any other type of sound, or even voice-text messages. Assistance can also refer to or include visual elements, such as arrows, lines, circles, boxes, icons, images, text messages, video sequences, etc., to guide users in completing related tasks. However, it should be understood that any other type of assistance can be used to help users perform the desired task.

[0038] Assistance can be provided to the user in any suitable manner. In particular, assistance can be provided to the user through appropriate output components. Output components may include, for example, visual devices, such as one or more displays for providing visual instructions. Output components may also include auditory elements, such as speakers, headphones, etc., to provide auditory information. In addition, output components may include any other suitable output elements of any type for providing tactile output, for example.

[0039] The output components may be at least partially included by the AR device. For example, visual indications may be provided by means of the AR device's display. Furthermore, auditory indications may be provided by means of the AR device's auditory output element. However, the AR device may also be connected to one or more additional output elements, such that at least a portion of the desired indication may also be provided by such external output elements not included in the AR device itself, for example on medical devices and / or mobile devices, which may be registered in the system. The output components may include multiple different output elements, each of which may be deployed on different devices such as dialysis devices and / or AR devices.

[0040] Operations for selecting training tasks, calculating user instructions, monitoring user activity, and providing assistance can be performed, for example, by a suitable control device. Such a control device can be implemented, for example, via a processor and a memory communicatively coupled to the processor. The memory can be configured to store program code that can be read by the processor. When executed by the processor, the program code enables the control device to perform appropriate operations, particularly operations for performing at least a portion of the methods for operating the dialysis device training system. The control device may include additional components, such as interfaces for receiving and / or outputting data. In particular, the control device may include interfaces for any type of wireless or wired communication. Furthermore, the control device may include elements such as any type of suitable input and / or output devices, for example, a user interface for user interaction. However, it should be understood that the control device may also include any other type of suitable components if desired.

[0041] User activity can refer to any action a user is performing with respect to a medical device, such as, for example, manipulating, operating, or moving the device or its parts or accessories, or taking actions that cause an effect on the medical device, such as clicking a button on the device or the user interface.

[0042] The control device can be arranged separately from the medical device, particularly from the dialysis device. However, the control device can also be at least partially integrated into the medical device. In particular, the control device can be implemented, for example, through suitable components of the dialysis device. In this way, the control device can be wholly or at least partially implemented through components of the medical device.

[0043] In a possible implementation, the selected task can be a simulation task. A simulation task can include simulated operations, i.e., operations that are not performed or not fully performed in reality. A simulation task includes one or more operations for setting at least one operating parameter and / or inputting at least one input on the dialysis device. In other words, to perform such a simulation task, the user does not actually operate the medical device, but simulates activities corresponding to the desired operation. In this case, the method can include a step of detecting the user's operation. The operation can be detected based on the user's monitored activity. Furthermore, the method can include a step of calculating a virtual result state of the dialysis device in response to the user's detected activity. In other words, a virtual result state of the dialysis device in response to the user's activity is simulated. The calculated result state or virtual result state can be provided to the user by virtually superimposing the calculated result state of the dialysis device. This can be performed by the output component of the AR device. Therefore, a virtual result of the user's operation is provided to the user without actually performing an operation through the dialysis device. Therefore, errors caused by the user do not lead to actual malfunctions of the dialysis device. Furthermore, no consumables are depleted or consumed, because the use of consumables is only virtual or simulated.

[0044] In possible implementations, the selected task may include one or more operations for installing or removing accessories. Furthermore, the selected task may also include one or more operations for filling media into the dialysis device. In this case, the method may include steps of calculating a virtual representation of the filled media and / or a virtual representation of the installed or removed accessories. The calculated virtual representation may be provided to the user by the output component of the AR device. Therefore, operations using disposable media or products can be trained using a real medical device without wasting such disposable products. Furthermore, operations using sensitive accessories can be trained without the risk of damaging such sensitive accessories.

[0045] In a possible implementation, the method may include the step of outputting a notification if the user's monitored activity deviates from a pre-stored target activity used for the task to be trained. The notification may be output by at least one output component, particularly an output component of the AR device. In this way, the user is immediately notified in the event of erroneous operation. Therefore, the user can promptly identify their malfunction and correct their operation. This can prevent the user from developing a habit of making erroneous operations.

[0046] In possible implementations, the method may include the step of calculating a training progress score. Specifically, the training progress score may be calculated based on a calculated error rate. Such an error rate can be calculated by comparing the user's monitored activity with pre-stored target activity. Alternatively or additionally, the training progress score may be calculated based on measured time for performing the task to be trained and / or the difficulty level assigned to the task to be trained. In particular, the training progress score may be related to the progress of the training level. In other words, the training progress score can be used to characterize or represent the effectiveness of the current training on the user. Therefore, such a training progress score can be used to (dynamically) adjust the training provided to the user.

[0047] In a possible implementation, user instructions are calculated based on the calculated training progress score. Specifically, if the training progress score deviates from a predefined threshold or range, the user instructions can be recalculated. For example, if it can be concluded from the calculated training progress score that the current method of training the user will not further increase the training progress score, the scheme or algorithm used to calculate the user instructions can be modified. This allows the user's training to be adjusted in such a way that the training effect can be enhanced. Therefore, better training results can be achieved in a shorter time period, thereby improving training efficiency.

[0048] In another embodiment, the method may include the step of determining the user's viewing direction. The user's viewing direction can be determined based on sensor data, particularly sensor data detected by sensor components, especially those of an AR device. For example, a camera device can be used to monitor the user's pupils. In this case, the provided assistance can be adjusted to guide the user's viewing direction towards a target viewing direction and / or a target location. For example, visual indicators such as arrows, lines, or items can be provided to guide the user to the target viewing location. However, tactile or auditory indicators, such as voice text, can also be provided to draw the user's attention to a specific location on the medical device. In this way, it can be ensured that the user focuses on the relevant part of the medical device.

[0049] In possible implementations, the method may include the step of configuring the speed at which the task to be trained is performed. For example, a user may manually specify the speed of task execution. However, the speed of task execution may also be configured at least partially automatically. For example, the availability of task execution may be adjusted based on a (automatically) determined skill level or training progress score. Thus, assistance is provided to the user based on the configured speed. In this way, the speed of the task to be trained can be adapted to the user's needs, allowing the user to follow the provided assistance without being overly challenged or under-challenged.

[0050] In possible implementations, the method may include the step of detecting predefined user input. Specifically, predefined user gestures may be detected. Such predefined user gestures may be, for example, specific gestures of the hand or one or more fingers. Gestures may be static or dynamic, such as specific movements of the hand or fingers. In this case, the assistance provided to the user for performing the task to be trained can be adjusted based on the detected user input. For example, the provided assistance may jump to a specific position in the computed instructions based on the detected user input. Such a specific position may be the starting point of the instructions, allowing the instructions to be repeated from the beginning. However, it may also jump to any other suitable position in the instructions, such as to repeat a specific part of the instructions. Furthermore, one or more instructions may be skipped in the provided assistance based on the detected user input. In this way, assistance for known parts of the process can be skipped. This allows focus to be placed only on the relevant parts of the training process. Furthermore, the level of detail used to provide assistance can be adjusted based on the detected user input. In this way, the user can easily adapt the level of detail to their current needs. For example, the level of detail may be related to the amount of information provided to the user for performing the desired task. Therefore, such information can be increased or decreased based on user input, especially based on user gestures. The level of detail can also be associated with, for example, activating or deactivating visual, auditory, or tactile aids. For instance, a user can individually activate or deactivate assistive visual, auditory, or tactile elements based on user input, particularly gestures. However, any other type of level of detail can also be adjusted based on user input. Furthermore, a new task to be performed can be selected based on detected user input. In this way, a user can easily select a new task to train on based on user input, particularly simple user gestures.

[0051] In possible implementations, the method may include the step of calculating a virtual character or virtual human. Such a virtual character or virtual human can be used to provide at least a portion of assistance for performing a task to be trained. The virtual character or virtual human can be provided by an output component of an AR device. In this way, such a virtual character or virtual human can simulate a human trainer. The virtual character or virtual human can be associated with a complete human simulation, which can be artificially generated. However, it is also possible to generate only a portion of such a human, such as an arm or hand. Furthermore, any type of artificial object can be generated as a virtual character for assisting or training a user.

[0052] In possible implementations, the method may further include receiving input messages representing input from a human trainer. The trainer may be spatially close to the user, for example, positioned adjacent to the user, particularly in the same room. However, the trainer may also be spatially distant, for example, in another room, or even in another building or another country. In this case, the trainer can connect to the user via an appropriate communication link. For example, the trainer may be provided with: information from the camera device (especially the AR device); auditory signals, for example, from the microphone of the AR device; and / or the output of the dialysis device. In this way, the trainer can grasp the current situation to assist the user. In particular, the calculation of user instructions and / or assistance can be adjusted based on the input messages. To this end, the trainer can provide additional information, which, by means of message transmission and reception, can be used to generate assistance for the user or to provide information about the trainer directly to the user.

[0053] In possible implementations, user activity is determined based on pattern recognition of image data captured by sensor components, particularly AR devices and / or camera devices of dialysis devices, and / or based on machine learning applied to data obtained from at least one sensor component, particularly AR devices and / or dialysis devices. By analyzing user activity, the user's current actions can be evaluated to identify correct or incorrect actions for performing the desired task. Therefore, this information can be taken into account to dynamically adjust user assistance, modify calculations of user instructions to perform the desired task, and perform any other type of operation related to the (immediate) instructions used to assist the user.

[0054] In the following detailed description of the accompanying drawings, non-limiting examples of embodiments having their features and additional advantages are discussed with reference to the drawings. Attached Figure Description

[0055] Figure 1 A schematic diagram of a medical system according to an embodiment is shown;

[0056] Figure 2 This is a schematic diagram illustrating a user utilizing an AR device according to an implementation method.

[0057] Figure 3 : A schematic diagram depicting a user utilizing another AR device according to an embodiment;

[0058] Figure 4 : A schematic diagram of the flowchart on which the method according to the embodiment is based is shown; and

[0059] Figure 5 : This is an illustration of a scenario for training a user on a medical device according to an implementation method; and

[0060] Figure 6 This is a schematic diagram of a scenario for training a user on a medical device according to another embodiment. Detailed Implementation

[0061] The invention will now be described in more detail below with reference to the accompanying drawings and embodiments.

[0062] Figure 1 A schematic diagram of the configuration of a medical system associated with a training system for a medical device, according to an embodiment, is shown. The medical system includes a medical device 1, particularly a dialysis device or other blood therapy device. The medical device 1 can be operated by a user 2.

[0063] Blood therapy devices, particularly dialysis devices, are medical devices that typically process a patient's blood outside the body. For example, medications can be added to or removed from the blood, and the blood can be heated or cooled. These processes are usually monitored by sensors. Blood therapy devices have various components for these purposes, such as pumps, valves, sensors, clamps, etc. In addition, disposable medical supplies are commonly used in blood therapy. For hygiene reasons, such disposable supplies are usually discarded after treatment. Disposable medical supplies, or so-called disposable items, include, for example, tubing assemblies for administering medical fluids such as blood or medications, filters such as adsorber filters or dialysis filters, and pouches for storing medications. These disposable supplies are typically inserted into or attached to the blood therapy device by the user and removed or detached from the device after treatment.

[0064] exist Figure 1 In the example, a medical device 1 is shown, wherein the medical device 1 includes: an input / output component 11; and at least one accessory component 12 representing any type of attachment, accessory, etc.; and a box component 13 for receiving (fluid) consumables.

[0065] The high complexity of such a medical device 1 poses a considerable challenge to its users. Therefore, user 2 must be properly trained before use. Furthermore, monitoring user 2 during device operation is desirable to identify potential malfunctions in the early stages. Additionally, providing guidance and support to user 2 during operation is highly beneficial.

[0066] In such Figure 1In the illustrated medical system, the augmented reality (AR) device 3 can be used to assist user 2, monitor user 2, and / or particularly to train user 2 in conjunction with the operation of medical device 1. Any suitable device suitable for AR purposes can be used. For example, the AR device can be any type of AR headset or AR glasses. Furthermore, any other type of device that can provide AR capabilities, such as a tablet computer, smartphone, etc., is also possible.

[0067] AR device 3 may include one or more output components or may be specifically data-connected to one or more output components. For example, the output components may provide visual / graphical output. Such visual / graphical output may be provided, for example, on a display. The display may be an opaque display on which an image of the environment, particularly an image of medical device 1 or at least a portion thereof, is overlaid with additional graphic elements. The display may also be a semi-transparent display, allowing user 2 to view through the display in order to recognize the real environment. Furthermore, additional graphic elements may be provided by the semi-transparent display, such that graphic indications on the semi-transparent display may assist user 2 during operation of medical device 1.

[0068] AR device 3 may also include one or more additional output components, such as speakers, headphones, or other devices for outputting signals, particularly auditory signals. In this way, auditory signals, such as sound or melody or voice-text messages, can be provided to user 2. In addition, any other suitable type of output component may be included in or connected to AR device 3. For example, AR device 3 may also include or be connected to an output component for providing tactile signals such as vibration, shaking, etc.

[0069] AR device 3 may also include or be connected to one or more sensor components for sensing the environment and / or operation of user 3. For example, the AR device may include one or more image sensing elements, such as a camera for capturing image data. AR device 3 may also include any other type of sensor component for sensing the movement of user 2, such as a gyroscope, accelerometer, velocity sensor, etc. However, any other suitable type of sensing component is also possible. For example, the sensor component may also be: a scanner, particularly a scanner for reading QR codes or barcodes; an RFID reader, etc. Alternatively or additionally, one or more sensor components may be provided separately from AR device 3, for example, as a room sensor, or a sensor for a mobile device, or other sensor components in a data connection.

[0070] Preferably, all or at least some of the output components and sensor components can be implemented in the AR device 3. However, as mentioned above, external output components and / or input components not implemented in the AR device 3 can also be used.

[0071] Figure 1 The other components depicted are below (in Figure 3 (Then) a more detailed description.

[0072] Figure 2 A schematic diagram of an AR device 3a used when a user handles or operates medical device 1 is shown, wherein AR device 3a includes a headset or AR glasses. In such an embodiment, AR device 3a can be positioned on the head of user 2. User 2 can view it through a semi-transparent display. Therefore, user 2 can still perceive the environment. In addition, graphical elements can be provided by the semi-transparent display, so that additional graphical elements can be displayed at locations related to elements in the real world. In addition, AR device 3a may also include additional output elements, such as speakers, to provide auditory output. Furthermore, AR device 3a may include a camera device 31 to capture image data, particularly image data in the direction that user 2 is currently viewing. In addition, AR device 3a may include any other type of sensor component for monitoring the movement of AR device 3a and the corresponding movement of user 2, or for acquiring any other type of appropriate sensor data. AR device 3a may also include any other type of sensor, such as a microphone for receiving voice commands from the user. Alternatively or additionally, sensor components, such as those in the form of camera device 31, may be provided externally to AR device 3a.

[0073] Figure 3A schematic diagram of another type of AR device 3b is shown. In this configuration, the AR device 3b can be implemented by a handheld device such as a tablet computer or smartphone. The handheld AR device 3b may include a camera device 31 for capturing image data, particularly during operation of the medical device 1. Furthermore, the handheld AR device 3b may include a display for combining the captured image data with additional graphic elements. In this way, image data of the real environment can be overlaid with additional graphic elements. Alternatively, the handheld AR device 3b may include a semi-transparent display through which the user 2 can view the image. In such a configuration, additional graphic elements can be provided on the semi-transparent display to provide additional information, particularly information related to one or more elements that the user 2 is viewing through the semi-transparent display. In any case, the handheld AR device 3b may also include output components for providing auditory and / or tactile outputs. Furthermore, the handheld AR device 3b may include or be connected to any type of sensor component, such as a microphone for sensing auditory signals, particularly user voice commands, or a sensor for monitoring the movement of the handheld AR device 3b.

[0074] AR devices 3, 3a, and 3b can be controlled, for example, by means of a control device 4. In particular, the control device 4 can be operated to train a user on the medical device 1, as will be described in more detail below.

[0075] exist Figure 1 In this diagram, the control device 4 is shown as a separate component disposed outside the medical device 1. However, the control device 4 can also be implemented within the medical device 1. In particular, it is even possible for the function of the control device 4 to be implemented through a component of the medical device 1.

[0076] Control device 4 can receive sensor data from one or more sensor components (particularly sensor components of AR device 3 or external sensor components) or other devices associated with AR device 3. Furthermore, control device 4 can also receive additional information or data from medical device 1. For example, control device 4 can receive: data related to the configuration of medical device 1; the current settings of medical device 1; data specifying the fill level of consumables in medical device 1; operational data such as flow rate, pressure value, etc.; status or error messages; or any other information related to medical device 1.

[0077] The control device 4 can also establish communication links with one or more external storage devices, particularly one or more databases 5. For example, such databases 5 can provide any type of information related to the medical device 1. Such information may include, for example, any type of data used to identify the medical device 1. For instance, the control device 4 can receive the serial number of the medical device 1 and refer to the database 5 to obtain additional information associated with that serial number. Image data, particularly image data captured by the sensor components of the AR device 3, can also be used to identify the medical device 1 or its accessories / accessories. For example, a QR code or barcode can be scanned, and the control device 4 can refer to the database 5 to identify the medical device 1 based on that code. Alternatively, any suitable features in an image representing the medical device 1 can be identified, and the database 5 can be referenced to identify the medical device 1 based on those features. However, any other method for identifying the medical device 1, its accessories or accessories, or for obtaining any other information related to the medical device 1 is also possible.

[0078] Database 5 may also provide additional relevant information, such as recommended settings, consumable fill levels, predefined operations for the user to operate medical device 1, and / or any other appropriate information related to medical device 1.

[0079] Database 5 can also provide any type of information for specifying operations to perform specific tasks related to medical device 1. Such specific tasks could be, for example, tasks for configuring medical device 1, tasks for operating or monitoring medical device 1 during operation, tasks for resolving warnings or malfunctions of medical device 1, tasks for maintaining medical device 1, or even tasks for repairing medical device 1. However, any other type of task can also be specified.

[0080] Database 5 may also include additional information related to one or more users 2. For example, database 5 may store and provide any type of preference of user 2. Such preferences may include, for example: specific configurations, particularly those for which information is provided by AR device 3, such as color schemes; how detailed the instructions for support / assistance to the corresponding user 2 should be; the knowledge level or skill level of the corresponding user 2; language skills; physical limitations of user 2; or any other data related to user 2.

[0081] In addition, database 5 can provide any type of information related to the patient to be treated by medical device 1. Such information may include, for example, the patient's physical characteristics, desired therapy / treatment, the patient's medications, or any other relevant data related to the patient.

[0082] In addition, database 5 can also provide any other type of relevant information that may be useful in relation to the operation of medical device 1.

[0083] Despite Figure 1 Database 5 is shown as a single database 5, but it should be understood that database 5 can also be implemented by multiple separate databases. For example, separate databases can provide attributes for medical device 1, data related to the operation of medical device 1, patient-related data, user preferences, etc. The various databases 5 can also be located in different spatial locations. In addition, at least some of the data can be stored in a local database within control device 4.

[0084] As in Figure 1 Furthermore, connections can be established with other experts, particularly another human expert 6. This other human expert 6 can be located close to user 2 and / or medical device 1. For example, expert 6 and user 1 can be located in the same room. However, expert 6 can also be located remotely. For example, expert 6 can be located in a separate room within the same building or in any other location. For example, expert 6 can be located in another city, country, or even continent.

[0085] Specifically, if expert 6 is located outside the room containing user 2 and medical device 1, any type of appropriate information relating to medical device 1 can be provided to expert 6. For example, expert 6 can receive information about the current status, operating characteristics, warnings, and malfunctions of medical device 1. Furthermore, expert 6 can receive information related to the desired task to be performed using medical device 1. Expert 6 can also receive information related to user 2 and / or the patient to be treated. However, any other type of information can also be provided to expert 6. For example, expert 6 can receive image data captured by one or more camera devices, particularly one or more cameras of AR device 3. Information can be provided to expert 6 in any suitable manner, such as by means of a monitor or display of a virtual reality (VR) device or AR device.

[0086] Figure 4 A flowchart is shown of a method for operating a dialysis device training system, particularly a training system employing an AR device 3 associated with a medical device 1, according to an embodiment.

[0087] The term "on or at" when used with or during the use of medical device 1 refers to the fact that training methods can be applied. All tasks or functions and / or actions to be performed in this respect can be supported. It is not necessarily required that user 2 operate directly on medical device 1, but rather that operation can be performed, for example, in the context of (intended) treatment using the medical device, at an accessory device or related device such as a disposable item. Furthermore, assistance may also be available during the setup or configuration phase of the medical device prior to its actual operation.

[0088] Typically, the method can be implemented, for example, in the medical system described above. Therefore, embodiments relating to systems, apparatus, or devices can include any type of means, components, or units used to implement the features of embodiments relating to methods. Thus, embodiments relating to methods can include any suitable method steps for implementing the features as described in conjunction with embodiments relating to systems, apparatus, or devices.

[0089] The method for operating such a dialysis device training system can be used, for example, to train user 2 on medical device 1. Training can involve the overall operation of dialysis device 1, specific tasks such as preparing, configuring, or maintaining dialysis device 1, or even single steps such as attaching components or filling media. Therefore, in the following text, the expression "task to be trained" can encompass all these types of operations. Thus, the task to be trained can be any type of operation related to medical device 1 that user 2 will learn. In particular, such a task to be trained can be an operation that is generally unrelated to the patient's actual treatment.

[0090] In the first step S1, user 2 selects the task to be trained on dialysis device 1. This task can be manually selected by user 2. For example, the user can input the desired task on the user interface, or select it from several available tasks. However, any other method for selecting the task to be trained is also possible.

[0091] Subsequently, in step S2, instructions are calculated to guide user 2 in completing the training task while operating dialysis device 1. For example, a suitable database storing instructions related to the relevant task can be referenced. Such a database can be a local database or a remote database, such as storage in the cloud. However, any other method for determining the instructions is also possible. For example, the individual instructions can be calculated based on a specific algorithm or by using a neural network.

[0092] The calculated instructions can be instructions for operating the control elements of medical device 1. For example, instructions can specify activating or deactivating a switch, turning a rotary knob, entering a value command on the control panel, etc. Instructions can also relate to filling media, particularly filling consumables into the container of medical device 1. Furthermore, instructions can relate to one or more operations for attaching or removing accessories from medical device 1. Instructions can also relate to operations such as monitoring the output of dialysis device 1, monitoring or controlling the filling level of consumables, responding to specific warning messages on dialysis device 1, or any other type of activity related to the operation of dialysis device 1.

[0093] In step S3, the activities of user 2 can be monitored. These activities can be monitored, for example, by at least one sensor component, particularly a sensor component of the AR device 3. For instance, the camera device of the AR device 3 can be used to capture images of user 2 or at least one or more body parts of user 2. For example, the camera device can monitor the movement of user 2's hands. In this way, operations performed by user 2 can be identified, particularly operations on or related to the medical device 1. For example, operations on the control panel, switches, or knobs of the medical device 1 can be detected. Furthermore, operations related to the connection for filling the dialysis device 1 with media, or operations related to the attachment or removal of accessories, can also be detected.

[0094] It can also monitor user activity to recognize any type of user input. For example, it can monitor a user's hand or one or more fingers to recognize predetermined patterns, such as gestures. Gestures can be static hand postures or specific configurations of one or more fingers. Gestures can also be dynamic movements of the hand or fingers. In this way, a specific command can be associated with such a gesture by user 2.

[0095] Furthermore, the viewing direction of user 2 can be monitored. For example, the pupils of user 2 can be monitored by a suitable sensor device, such as a camera. In particular, the sensor device of AR device 3 can be used for this purpose. Therefore, the direction or position that the user is currently looking at can be identified. In this way, the elements or features of the medical device 1 currently being used by user 2 can be obtained. In particular, this viewing position or direction can be compared with a position or direction considered to be related to the task to be trained, especially according to the calculated instructions used to perform the task to be trained.

[0096] Furthermore, in step S4, assistance can be provided to user 2. This assistance can be for performing the task to be trained. In particular, assistance can be generated based on calculated user instructions and monitored activities. Specifically, providing assistance can include outputting instructions on the output component of AR device 3.

[0097] As mentioned above, the aid can be any suitable type of assistance that can be used to guide user 2 to perform the desired task. For example, the aid can include visual cues such as lines, arrows, icons, images, text, etc. Furthermore, the aid can include auditory cues such as sound signals, melodies, spoken text, etc. The aid can also include tactile cues such as shaking, vibration, etc.

[0098] In possible implementations, assistance can be provided at least in part by means of a virtual assistant, such as a virtual character or virtual human. For example, an artificial character or humanoid for providing appropriate information to user 2 can be calculated. In possible implementations, the humanoid can be calculated and / or computer-animated. Such a humanoid can simulate a human trainer. Alternatively, any other type of artificial character, such as a static or animated object, is also possible. In particular, a dialogue can be established between user 2 and the virtual assistant.

[0099] When training user 2 on medical device 1, user 2 may have to perform multiple operations related to medical device 1. User 2 can make the input on medical device 1 realistic. In this case, the user can also identify the real output of medical device 1 in response to such operations. However, it is also possible that the user input and / or the response made by medical device 1 can be simulated by the training system, especially in conjunction with AR device 3.

[0100] For example, user actions on medical device 1 can be identified based on monitored user activity. In such a scenario, the user does not actually operate the control elements of the medical device, but only simulates such actions. The relevant actions are identified, and the response of medical device 1 to these actions can be calculated. Therefore, a representation of the simulated response of medical device 1 to user actions can be calculated. Such a simulated representation can be provided to the user, for example, by means of an AR device 3. For example, AR device 3 can provide user 2 with an output superimposed on a corresponding location on medical device 1, allowing the user to have the impression of a realistic response of medical device 1 to their actions. Such output can involve, for example, messages on a control panel, control indicators illuminated in a specific color, switches indicated in a specific location, sound signals such as control tones or warning messages, or any other type of action of medical device 1 in response to user actions. In this way, the operation of medical device 1 can be simulated even without active operation. Therefore, even critical operations, such as those that could damage medical device 1 if operated incorrectly, can be trained.

[0101] In possible implementations, operations related to consumables and / or accessories can also be trained virtually. For example, a virtual representation of a container containing a medium to be filled into a medical device can be generated. In such an example, the operation of filling a medium into medical device 1 can be simulated, for example, filling the container into the box of medical device 1 through an opening in medical device 1. On one hand, a virtual representation of the container with the medium can be provided to user 2, and this virtual representation can move according to the monitored activities of user 2 (in particular, the movement of the hand performing the operation of grasping the container and filling the medium into medical device 1). Alternatively or additionally, a virtual representation of the reaction of medical device 1 in response to such a simulated operation of filling a medium into medical device 1 can also be provided. For example, a virtual representation of an indicator of the fill level on medical device 1 can be calculated and provided to the user. Alternatively or additionally, any other reaction of medical device 1 can be simulated, for example, a warning message can be output if the simulated operation will result in overfilling of medical device 1. In addition, the consumption of such medium during the operation of medical device 1 can be calculated, and the time point when the quantity of consumables is used up can be calculated. In this regard, the quantity of consumables filled into medical device 1 can be calculated based on the activities monitored during the operation of filling the consumables into medical device 1.

[0102] In addition, the attachment or removal of accessories can be trained based on virtual representations. For example, a virtual representation of the accessory can be provided to the user using an output component, particularly the output component of AR device 3. Therefore, the user can perform activities aimed at attaching or removing such accessories. In this regard, the behavior of the accessory in response to the monitored activities of user 2 can be calculated, and the results of user 2's activities can be simulated. Therefore, an output can be provided to the user, representing the result of his / her operation in the form of a virtual result state of the medical device. For example, a virtual accessory can be provided to the user at the appropriate location on medical device 1 after successful installation of the corresponding accessory. Alternatively, in the event of a malfunction, the associated damage to the accessory can be calculated and provided to the user. In this way, the user can be trained on the operation of installing or removing such accessories without the risk of damage even in the event of a malfunction.

[0103] When using medical device 1, the user can also be supported by a large language model (LLM), such as that known in ChatGPT. The user can verbally interact with such an LLM. For example, spoken requests such as phrases or questions can be received and provided to the LLM. For this purpose, the request can be received by an input device, particularly the input device of AR device 3, such as a microphone, converted into text by a speech-to-text module, and provided to the LLM. However, any other type of input, such as text input via an input terminal, such as by means of a keyboard, is also possible. The LLM can provide an appropriate response to the user's input. This response can be provided to the user through auditory output, such as synthesized speech output, by providing text messages, by providing any other type of instruction such as lines, arrows, symbols, icons, images, and / or video clips. For this purpose, appropriate output components of AR device 3 can be used. The output can involve output dynamically generated by the LLM in response to the user's request. However, the output can also include, for example, pre-generated and pre-stored elements. For example, the output can include pre-stored images, such as images of medical device 1 in a specific configuration, images of specific parts of medical device 1, or video clips illustrating specific operations of any other type of data that has been prepared in advance.

[0104] When generating output, the LLM can take the user's language into account. Therefore, the LLM can provide a response in the same language the user used to formulate the request. For example, if the user asks a question in English, the LLM can also provide a response in English, and if the user asks a question in German, the LLM can provide a response in German. However, the processing and / or generation of responses can also be restricted to one or more specific languages. Furthermore, the LLM's responses can be provided in a specific language regardless of the language the input is formulated in. For example, the LLM can always provide a response in the language corresponding to setting up medical device 1.

[0105] The dialogue with the LLM can be used for any type of interaction, especially for any type of assistance that helps the user operate the medical device 1. For example, the user can ask which part or component of the medical device 1 is currently in front of them. Supplementary information can also be considered for this purpose. For example, input components, particularly those of the AR device 3, can detect the user's viewing direction and / or the user's environment, specifically detecting / identifying elements or objects in the user's environment. Therefore, this information can also be considered when the LLM generates a response to the user's request.

[0106] Users can also inquire about how to perform a desired operation, such as how to attach accessories or disposable items, or how to perform the next step of the desired operation. Alternatively, users can inquire about the location of specific components of the medical device. To generate a response to such requests, any additional information of any kind can be considered, such as information acquired by sensor components, particularly those of the AR device 3. For example, image data of the environment can be captured by optical sensor components such as those of a camera device. Image data can be processed to identify the medical device 1, its configuration, accessories, the spatial relationship between the user and the medical device 1, the user's current operation, or to detect the spatial location of a part of the user's body, such as the current position of the user's hand. Furthermore, supplementary information can be received from any other type of data source. For example, information regarding the current configuration of the medical device 1, its current state, operating parameters, or any other characteristics of the medical device 1 can be received via a wired or wireless communication link.

[0107] Therefore, the user can be assisted through verbal interaction with the LLM to perform desired tasks and / or obtain more information about the medical device 1. The information provided by the LLM does not necessarily have to refer only to the task the user is currently performing. The user can also interact with the LLM to obtain any other type of information, such as general information related to the medical device 1.

[0108] During training, user 2's activities can be monitored and analyzed. Therefore, it is possible to assess whether user 2 follows the provided assisted following instructions and whether user 2 correctly performs the expected operation. For example, user 2's monitored activities can be compared with target operations, particularly pre-stored target operations. If user 2's operation deviates from the target operation, appropriate signaling can be provided to user 2. Such signaling can be, for example, visual, tactile, or auditory signaling. Therefore, the user can immediately identify their malfunction and correct their operation.

[0109] Furthermore, analysis of user actions and evaluations based on comparisons with target actions can be used to assess user 2's training progress. For example, an error rate can be calculated. Such an error rate can be related to the number of deviations between user 2's actions and the relevant target actions. The error rate can be the absolute number of deviations between user 2's activities and the target actions. Additionally, relative values ​​can be considered, taking into account other characteristics such as complexity level, time period, number of repetitions performed for the same task to be trained, or any other suitable features.

[0110] The analysis of the matching between User 2's activities and the relevant target operations can be used not only to calculate simple error rates but also for more complex training progress scores. Such training progress scores can take into account the progress of the training level, e.g., how well or quickly User 2 learns during the training process. This training progress level can consider error rates, the time required for the user to perform the task to be trained, the (pre-defined / pre-stored) complexity level of the task to be trained, and / or any other appropriate parameters. In particular, the process of the training task can be adjusted based on training progress, especially based on the calculated training progress score. For example, the speed at which assistance is provided to User 2 can be adjusted based on the calculated training progress score. In a possible example, assistance can be provided at a lower speed to an inexperienced user with less training progress, and the speed increases as the training progress score increases. Furthermore, the level of detail used to provide assistance can be adjusted based on the calculated training progress score. For example, more detailed and / or a large amount of information can be provided to an inexperienced user, and the amount of information provided to the user for performing the desired task can be reduced as the training progress score increases. Additionally, if the calculated training progress score exceeds a predetermined threshold, at least some sub-steps in the task can be skipped, for example. However, any other method for adjusting the aid based on the training progress score is also possible.

[0111] For example, if the currently calculated training progress score deviates from a specific range or exceeds / falls below a threshold, auxiliary adjustments or calculations of instructions to perform the training task can be triggered.

[0112] In addition, user 2's training can be manually adjusted based on user 2's needs, particularly those specified by user 2. For example, the user can specify the speed of the training process, specifically the speed at which assistance will be provided for performing the task to be trained. The user can also specify other features, such as the language used to provide assistance, visual and / or auditory characteristics used to provide assistance, or any other type of preference. Preferences can be stored, for example, on or obtained from local or remote storage devices.

[0113] Preferences and any other type of user input can be provided through a suitable user interface. Furthermore, user gestures can be used to interact with the training system. Specifically, interaction with the training system can be based on hand or finger gestures as described above. For example, gestures can also be used to control the provision of assistance. For example, a specific gesture can be used to select the task to be trained. For example, one or more additional gestures can be used to configure the provision of assistance, such as adjusting the speed or level of detail when providing assistance. For example, additional gestures can be used to "navigate" within the assistance used to perform the task to be trained. For example, gestures can be used to switch back to the beginning of the task to be trained or to a specific location within the task to be trained. For example, additional gestures can be used to skip specific steps or subtasks or to terminate the task to be trained. However, any other actions can also be controlled based on user 2's gestures.

[0114] Furthermore, automated training can also be monitored, controlled, or supervised by a human trainer. The trainer can be a local trainer located near user 2, for example, in the same room. However, it is also possible for the trainer to be located away from user 2. In this case, relevant information can be provided to the trainer via a communication link. For example, voice communication can be established between user 2 and the trainer, and in particular, image data from one or more cameras of the AR device 3, or any other type of relevant information, can be provided to the trainer. The trainer can also assist the user in performing the training task; the trainer can specify the specific task to be trained, or can activate specific features, such as simulations of specific operational states of errors or medical devices. However, any other type of interaction between the trainer and user 2 and / or medical device 1 is also possible.

[0115] Figure 5 A schematic diagram illustrating an exemplary training scenario according to an implementation method is shown. Figure 5 In the example, user 2 is operating control panel 101. Specifically, the sensor components of AR device 3 can monitor the user's activities to determine the various actions the user takes on control panel 101. In response, the reaction of medical device 1 is calculated, and the corresponding output is provided to the user via virtual overlay 102. In other words, medical device 1 does not perform any operation in reality, but the corresponding response is calculated and simulated to the user via virtual overlay 102.

[0116] Figure 6 A schematic diagram illustrating an exemplary training scenario according to another implementation is shown. Figure 6In the example, user 2 simulates a filling operation using a virtual representation 110 of a container including the medium. The virtual representation 110 can be provided by the output component of the AR device 3. Furthermore, sensor components, particularly those of the AR device 3, can monitor user 2's actions. Therefore, the filling operation can be simulated, and the corresponding filling level can be calculated. This calculated filling level can also be provided to the user via the virtual representation 111. Thus, even without the need for a real consumable, the user can perform realistic operations related to the medical device 1.

[0117] In summary, the present invention relates to user training for medical devices, particularly dialysis devices. To this end, instructions for performing the task to be trained are calculated, and assistance is provided to the user for performing the task using an AR device. Specifically, the AR device can also monitor the user's activities, allowing the medical device's responses to the user's activities to be calculated, and related virtual feedback to be provided to the user.

Claims

1. A method for operating a training system for a dialysis device, the method employing an augmented reality (AR) device (3) associated with a dialysis device (1), the method comprising the following steps: - Select (S1) the task to be trained on the dialysis device (1); - Calculate (S2) user instructions, which are used to guide the user (2) to complete the task to be trained while operating the dialysis device (1); - Monitor (S3) the user's activities when operating the dialysis device (1), especially when executing the calculated user instructions, by means of at least one sensor component; - Based on the calculated user instructions and the monitored activities, provide (S4) assistance for performing the task to be trained, wherein providing (S4) assistance includes outputting an instruction on the output component of the AR device (3).

2. The method according to claim 1, wherein, The selected task is a simulation task that includes simulated operations, and the selected simulation task includes operations for setting at least one operating parameter and / or inputting at least one input on the dialysis apparatus (1), wherein the method includes: - Detect the user's (2) actions based on the monitored activities of the user (2); - Calculate the virtual result state of the dialysis device (1) in response to the detected activity of the user (2); - The virtual result state calculated by the dialysis device (1) is virtually superimposed, without actually performing the operation through the dialysis device (1).

3. The method according to claim 1 or 2, wherein, The selected task is one that includes operations for installing or removing accessories and / or filling media, and wherein the method includes: - Calculate a virtual representation of the filled medium and / or a virtual representation of the installed or removed fittings; and The calculated virtual representation is provided by the output component of the AR device (3).

4. The method according to any one of the preceding claims, comprising: If the activity detected by the user (2) deviates from the pre-stored target activity for the task to be trained, a notification is output by at least one output component, in particular the output component of the AR device (3).

5. The method according to any one of the preceding claims, comprising calculating a training progress score, wherein, The training process score is calculated based on the following: the calculated error rate, which is calculated by comparing the activity monitored by the user (2) with the pre-stored target activity; The measurement time for performing the task to be trained; And / or the difficulty level assigned to the task to be trained, in particular, wherein the training progress score represents the progress of the training level.

6. The method according to claim 5, wherein, The user instruction is calculated based on the calculated training process score. In particular, if the training process score deviates from a predefined threshold or range, the user instruction is recalculated.

7. The method according to any one of the preceding claims, comprising: - The viewing direction of the user (2) is determined based on sensor data detected by the sensor components. The provided assistance is adjusted to specifically guide the user's (2) viewing direction toward the target viewing direction, particularly the target location, based on the selected task.

8. The method according to any one of the preceding claims, comprising the step of: configuring a speed for performing the task to be trained, wherein, The assistance is provided according to the configured speed.

9. The method according to any one of the preceding claims, comprising detecting predefined user input, particularly predefined user gestures. in, The assistance provided for performing the task to be trained is adjusted based on the detected user input. Specifically, the provided assistance jumps to a specific position in the calculated user instructions based on the detected user input, skips one or more user instructions based on the detected user input, adjusts the level of detail for providing the assistance based on the detected user input, and / or selects a new task to be performed based on the detected user input.

10. The method according to any one of the preceding claims, comprising the step of: calculating a virtual character or virtual human, wherein, The virtual character or virtual human is used to provide at least a portion of the assistance for performing the task to be trained, and wherein the virtual character or virtual human is provided by the output component of the AR device (3).

11. The method according to any one of the preceding claims, comprising receiving an input message, wherein, The input message represents input from the trainer, wherein the user instruction is calculated (S2) and / or the assistance is provided (S4) to adjust based on the received input message.

12. The method according to any one of the preceding claims, wherein, Actions are determined based on the user's monitored activities, based on pattern recognition, and / or based on machine learning applied to data obtained from the at least one sensor component.

13. A dialysis device training system, comprising: Augmented Reality (AR) devices (3); Control device (4), the control device (4) is configured to perform the method according to any one of claims 1 to 12, and is configured to: - Select the task to be trained on the dialysis device (3); - Calculate user instructions, which are used to guide the user (2) to complete the task to be trained while operating the dialysis device (1); - Monitor the activities of the user (2) when operating the dialysis device (1), especially when executing calculated user instructions, by means of at least one sensor component; - Based on the calculated user instructions and the monitored activities, assistance is provided for performing the task to be trained, wherein providing assistance includes outputting instructions on the output component of the AR device (3).

14. A medical system comprising: At least one dialysis device (1); as well as The dialysis training system according to claim 13.

15. An augmented reality (AR) device (3) for use in a dialysis device training system according to claim 13, comprising: An output device configured to provide instructions based on determined assistance when training a user (2) on a dialysis device (1), and / or The sensor interface of the sensor component is configured to capture data related to the activities of the dialysis device (1) and / or the user (2) operating the dialysis device (1).

Images