Controlling a remote assistance session

Abstract

Disclosed is a computer-implemented method of controlling a remote assistance session that is established via a computer network between a consultation console and an assistant console. The remote assistance session is configured to provide the assistant console a function access to a data processing function of the consultation console. The method comprises: receiving participation information that describes a participation in the remote assistance session by an assistant operator at the assistant console or by a medical operator at the consultation console; monitoring the participation information for fulfillment of a predefined absence criterion that describes a lack of the participation; and if the absence criterion is fulfilled, revoking the function access.

Description

[0001] 2024PF00683

[0002] 02.03.2026

[0003] CONTROLLING A REMOTE ASSISTANCE SESSION

[0004] FIELD OF THE INVENTION

[0005] The invention relates to the field of collaborational sessions conveyed via a computer network.

[0006] BACKGROUND OF THE INVENTION

[0007] The health care industry is facing unprecedented challenges including increasing patient volumes, increasing rates of burnout and turnover among healthcare professionals as well as increasingly strained resources. To help medical facilities and healthcare providers adapt to such changes, new technologies and solutions are being made available to enable increased workflow efficiency and productivity with fewer staff members. Since the advent of the internet, telecommunication and remote control solutions providing new possibilities of remote collaboration and assistance have become widespread not only in the medical field.

[0008] In the field of radiology, solutions are known that enable remote support for operators of medical imaging devices and remote image acquisition from a central assistance facility, allowing expert imaging technologists to support on-site health professionals and technologists at medical imaging sites anywhere in the world.

[0009] US 2024 / 0 185 970 Al describes a method for patient-authorized secure and timelimited access to patient medical records utilizing key encryption.

[0010] SUMMARY OF THE INVENTION

[0011] It is an objective of the present invention to provide for a system and method for an improved computer-implemented method of controlling a remote assistance session established via a computer network, and a correspondingly configured computer program product and computer system as well.

[0012] In one aspect, there is disclosed a computer-implemented method of controlling a remote assistance session. The remote assistance session is established via a computer network between a consultation console and an assistant console, and is configured to provide the assistant console a function access to a data processing function of the consultation console. The method comprises receiving participation information that describes a participation in the remote assistance session by an assistant operator at the assistant console or by a medical operator at the consultation console. The method further comprises monitoring the participation information for fulfillment of a predefined absence criterion that describes a lack of the participation. The method further comprises revoking the function access upon2024PF00683

[0013] 2 02.03.2026

[0014] fulfillment of the absence criterion. The method further comprises fulfilling the absence criterion upon persistence of the lack of the participation for a predefined revoke period and revoking the function access, and wherein in case the monitoring results in a fulfillment of the absence criterion with respect to an unattended console selected from the consultation console and the assistant console, the other one of the consultation console and the assistant console being an attended console, and in case the participation information received from the attended console during the revoke period comprises a predefined call request indicator, instructing the unattended console to output a call signal asking to resume the participation in the remote assistance session.

[0015] In another aspect, there is disclosed a computer program product. The computer program product comprises a computer-readable medium, in particular a non-transitory computer-readable medium. The medium carries computer-executable instructions for execution by a processor controlling an apparatus. Execution of the instructions causes the processor to perform a method of controlling a remote assistance session. The remote assistance session is established via a computer network between a consultation console and an assistant console, and is configured to provide the assistant console a function access to a data processing function of the consultation console. The method comprises receiving participation information that describes a participation in the remote assistance session by an assistant operator at the assistant console or by a medical operator at the consultation console. The method further comprises monitoring the participation information for fulfillment of a predefined absence criterion that describes a lack of the participation. The method further comprises revoking the function access upon fulfillment of the absence criterion.

[0016] In a further aspect, there is disclosed a computer system. The computer system comprises a processor and a memory that is functionally coupled to the processor. The memory stores computerexecutable instructions. Execution of the computer-executable instructions by the processor causes the processor to perform a method of controlling a remote assistance session. The remote assistance session is established via a computer network between a consultation console and an assistant console, and is configured to provide the assistant console a function access to a data processing function of the consultation console. The method comprises receiving participation information that describes a participation in the remote assistance session by an assistant operator at the assistant console or by a medical operator at the consultation console. The method further comprises monitoring the participation information for fulfillment of a predefined absence criterion that describes a lack of the participation. The method further comprises revoking the function access upon fulfillment of the absence criterion.

[0017] These and other aspects of the invention will be apparent from and elucidated with reference to the examples described hereinafter. It is understood that one or more of the aforementioned examples may be combined as long as the combined examples are not mutually exclusive.2024PF00683

[0018] 3 02.03.2026

[0019] BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In the following, examples are described in greater detail making reference to the drawings in which:

[0021] Fig. 1 illustrates an environment where a method of controlling a remote assistance session may be executed;

[0022] Fig. 2 is a flow chart illustrating steps of a method of controlling a remote assistance session; and

[0023] Fig. 3 is a flow chart illustrating an extended application scenario for a method of controlling a remote assistance session.

[0024] DETAILED DESCRIPTION OF EMBODIMENTS

[0025] As described below in greater detail, examples may be beneficial because they may increase protection of sensitive information from getting disclosed to unauthorized persons via a remote assistance session, and may increase protection of devices getting used by unauthorized persons via a remote assistance session.

[0026] In a medical consultation (e.g., an examination or treatment) locally held at a medical facility (e.g., a hospital, a doctor’s practice, or other general healthcare provider), a remote assistance session may provide a telecommunication link (audio, video) and / or capabilities for remotely controlling medical devices (e.g., an imaging device, an imaging contrast injector, an interventional device such as a catheter, etc.) to a remote assistant (e.g., a technical or medical expert such as a CT / MR / NM technologist, radiologist, etc.) via a computer network (e.g., the internet, a local area network (LAN), a wireless (Wi-Fi) network, etc., and combinations thereof).

[0027] From a patient point of view, there is an interest that trustworthiness is preserved during the entire consultation. In particular, it is in a patient’s interest that access to medical information related to the patient, as well as capabilities of controlling medical equipment potentially interacting with the patient, stay limited to persons having a trusted accreditation to the consultation. These persons may include medical staff authorized to practice at the medical facility where the consultation takes place, and assistants (preferably one per consultation session) authorized by the medical facility to participate in medical consultations from a different location (e.g., a dedicated remote assistance center, or a technical service internally assisting the medical facility) that is connected to the facility via the computer network.

[0028] Although situations of medical consultation are used herein for demonstrating characteristics of the invention, it is understood that methods, steps, ideas, approaches, principles and other elements described herein may be applied to any other collaborational sessions that are established via a computer network outside of the medical, hospital or other clinical context.

[0029] The method described herein relates to a remote assistance session that is established via a computer network between a consultation console and an assistant console. The remote assistance session may comprise a communication connection between the consultation console and the assistant2024PF00683

[0030] 4 02.03.2026

[0031] console through the computer network. The data exchanged between the assistant console and the consultation console by performing the remote assistance session is referred to herein as the session payload. The data stream or data streams transporting the session payload between the consultation console and the assistant console maybe monomodal or multimodal. Exemplary data modes include audio, video, graphical, and / or text information; machine-executable commands that may be exchanged between the consultation console and the assistant console; information generated by sensors or other devices, including device status information; etc. In an exemplary multimodal setup, the remote assistance session may convey audiovisual information between the two consoles in a bidirectional data stream and device control commands in a unidirectional data stream from the assistant console to the consultation console.

[0032] As a result, the assistant operator and the medical operator may be able to see each other, hear each other, or share other kinds of information such as visual or textual information that, for instance, may be displayed on a user interface of the respective console. Additionally or alternatively, the remote assistance session may provide a remote control functionality between the consultation console and the assistant console. Such remote control connection may allow a user of the assistant console to use machine command payload as described above to remotely control a function of the consultation console or of a device that is attached to the consultation console.

[0033] The consultation console and the assistant console may each be regarded as a personal computer or a terminal computer having integrated and / or attached typical user interface functions such as input devices and output devices, and having a network interface allowing an exchange of data between the respective console and the computer network. Without limitation, each of the consoles may be a static computer system such as desktop workstation, or a mobile computing device such as a notebook or a tablet computer. The consultation console may additionally have an interface that allows for connecting the consultation console to an external device such as a mobile device of the medical operator or a peripheral device, for example medical imaging device. Both consoles may be equipped with additional input devices such as sensors, in particular a camera, a microphone, a dedicated presence detection sensor or proximity detector, or any other device producing data that may be used to determine whether a person is present at the assistant console or the consultation console or is actively participating in the remote assistance session.

[0034] Some or all of the steps of the method may relate to controlling the remote assistance session. Possible characteristics of this control are described herein in further detail. In particular, some or all steps of the method may be carried out by the consultation console; by the assistant console; or by a control device that may participate in the remote assistance session and may be a node of the computer network connecting the consultation console and the assistant console. In the following, it will be assumed for the sole purpose of simplified demonstration that the method is carried out by a single control device interfacing the assistant console and the consultation console. However, it shall be understood that any other way of performing the method, including a distributed execution of steps of the2024PF00683

[0035] 5 02.03.2026

[0036] method, may be suitable for implementing the method as well. The control device may be a computing device that forwards data between the assistant console and the consultation console, or may be a device that is attached to such forwarding computing device.

[0037] It is assumed, too, that the remote assistance session is initially configured to provide the assistant console a function access to a data processing function of the consultation console. In particular, the function access to the data processing function may be thought of as a configuration of the remote assistance session that allows a user of the assistant console to receive data that are processed by or accessible to the consultation console, or to exert control over functions of the consultation console or any other device that is connected to the consultation console. Without limitation, the data processing function may be thought of as an interface function of the consultation console. Non-limiting examples of the function access in a radiology scenario may include a view console, an edit console, and / or a function allowing to change examination parameters by selecting menus / buttons of a user interface provided by the assistant console).

[0038] It is understood further that, unless otherwise noted, any notion expressed herein as relating to the assistant console “or” the consultation console, or respectively, to the assistant operator “or” the medical operator includes the possibility that said notion may apply to the assistant console alone, or to the consultation console alone, or to the assistant console and the consultation console at the same time; or respectively, to the assistant operator and the medical operator at the same time.

[0039] In an exemplary implementation of the method, the control device receives participation information. The participation information may be thought of as any information describing a presence or an activity of the assistant operator or the medical operator with respect to the remote assistance session. The participation information may be part of the session payload or may, at least in portions, be transferred between respective data sources and the control device on one or more dedicated communication channels that do not participate in transferring the session payload.

[0040] In an example, the participation information is descriptive of: actions performed by the assistant operator or the medical operator with respect to the remote assistance session; a physical presence of the assistant operator or the medical operator within a vicinity of a device used for the remote assistance session; a focus of the assistant operator or the medical operator with respect to the remote assistance session; an activity of the assistant operator or the medical operator with respect to the remote assistance session; or combinations thereof. In an example, the participation information originates from: the consultation console; the assistant console; or both. In an example, the participation information comprises: information describing the assistant operator or the medical operator; information suitable for identifying the medical operator or the assistant operator; signals adapted for controlling the remote assistance session; signals adapted for controlling the function access; or combinations thereof. In an example, a signal adapted for controlling the remote assistance session comprises a call request indicator. In an example, a signal adapted for controlling the function access comprises a clearance information. In an example, the participation information is provided in one or more of the following data formats: image2024PF00683

[0041] 6 02.03.2026

[0042] data; audio data; text data; machine-executable commands; information generated by a data source device; or combinations thereof. In an example, the data source device comprises: a user input device; a sensor; a computer hosting a database; or combinations thereof. In an example, the image data comprises: video data; static image data; a graphical user interface output; or combinations thereof. In an example, the sensor comprises a camera; a microphone; a proximity sensor; an antenna; a pressure sensor; a temperature sensor; a chemical sensor; a motion detector; an accelerometer; or combinations thereof. In an example, the user input device comprises a computer mouse; a computer keyboard; a microphone; a fingerprint reader; an image scanner; or combinations thereof. It is noted that the terms “vicinity of a device”, “clearance information”, “call request indicator” and other terms can be defined at other passages herein. It is noted further that the term “participation information” may not be limited to the examples given above, but may rather be characterized by having a function of: enabling an automatic assessment as to whether the predefined absence criterion is fulfilled; enabling an automatic computer-based control of the remote assistance session; enabling an automatic computer-based control of the function access; or combinations thereof.

[0043] The control device may monitor the participation information for showing signs of lacking participation of the assistant operator or the medical operator. It may be assumed that the control device repeats the monitoring, e.g., at regular intervals, thus providing a quasi-continuous monitoring of the participation information. For the purpose of monitoring, the control device may search the participation information from the session payload and may apply a predefined absence criterion to the found participation information.

[0044] The predefined absence criterion may define numerical and / or logical terms such as one or more thresholds allowing the control device to discern between situations where the medical operator or the assistant operator is having a high probability of actively participating in the remote assistance session and situations where the medical operator or the assistant operator has a high likelihood of not actively participating in the remote assistance session. In order to do so, the control device may analyze at least one of the data modes described above, thus obtaining an analysis result to which the logic defined by the absence criterion can be applied. More specifically, the control device may perform a multimodal analysis of the session payload (taking into account, e.g., audio and video data) to determine the fulfillment of the absence criterion with a higher significance. If there are more than one data sources potentially providing participation information, the control device may apply rule-based logic to ensure that at least one of these data sources is detecting a participation of the respective operator.

[0045] The following practical examples shall demonstrate the monitoring of the participation information without limitation. Here, the evaluation of the absence criterion is described without deployment of identification technologies; presence detection and participation evaluation including personal identification will be discussed further below.

[0046] In one example, the assistant console or the consultation console is equipped with a digital camera such as a visual camera or an infrared camera. Examples of a digital camera may include a2024PF00683

[0047] 7 02.03.2026

[0048] built-in camera or another webcam usable for personal communication, or other cameras having a data link to the respective console, such as a fixedly installed camera having a wide field of view allowing to detect a person anywhere in the room where the respective console is set up. Performing the monitoring step, the control device may then analyze data produced by the digital camera to perform, e.g., a face detection without identification; or an eye-tracking algorithm to assess presence and / or activity of the respective operator with respect to the remote assistance session, which may include determining which screen among a plurality of screens the respective operator is currently focusing on.

[0049] In a radiology application, the consultation console may be an MR / CT console in a radiology control room, and a suitable camera may be an electronically controlled patient treatment zone camera. In particular, an already present contrast injector camera may be used, e.g., to look around the control room programmatically at intervals when it is not in contrast injector view mode. There may be more than one sensors delivering portions of the participation information that may be deployed in more than one location in or around the local radiology tech desk area - e.g., near the local MR / CT console or an observation window of the MR / CT control room, near a local stationary or mobile computer in the control room, inside the scanner room, etc.

[0050] Other possible examples of data sources providing data modes that may be suitable for monitoring the participation information include: a proximity sensor deploying infrared or ultrasound technology, for instance; a radar array sensor or other wireless electromagnetic device (e.g., a communication or sensing device) that may allow for determining a distance to a receiver device carried by a person; a microphone producing an audio signal that may be analyzed to determine whether a person is close to the microphone and speaking to the direction of the microphone or not; a computer mouse, a computer keyboard, or other manual input devices producing signals that may be indicative of a device that is currently used by a person, or of a level of activity of a person using the respective console or another device attached thereto; etc.

[0051] Taking into account the participation information, the control device may assess that the assistant operator or the medical operator is not participating in the remote assistance session in various ways. For instance, the control device may determine that one of the operators is not present anymore at or near the respective console participating in the remote assistance session; or that the operator has ceased actively participating in the remote assistance session or has directed their focus away from the remote assistance session. The absence criterion may define one or more of such scenarios indicating the lack of the participation of the respective operator in terms of one or more of the data modes that are included in the participation information.

[0052] It is noted that the present text contains multiple statements regarding possible manners of fulfillment of the absence criterion. These statements are freely combinable as long as they do not exclude each other. For example, the method may stipulate (a) that the absence criterion is fulfilled upon persistence of the lack of the participation for a predefined revoke period; (b) that it is fulfilled if the participation information indicates an intention of the assistant operator or the medical operator to revoke2024PF00683

[0053] 8 02.03.2026

[0054] the function access; and (c) that it is fulfilled upon inability to continue an identification of the assistant operator or the medical operator based on the participation information. In this exemplary scenario, one possible implementation of the method includes that the absence criterion is considered fulfilled as soon as one of (a), (b), and (c) is true (logical “OR” combination), i.e., the absence criterion would be fulfilled if (a) is true; the absence criterion would be fulfilled if (b) is true; and the absence criterion would be fulfilled if (c) is true. In the same exemplary scenario, an alternative possible implementation of the method includes that the absence criterion is considered fulfilled if (a), (b), and (c) are true at the same time (logical “AND” combination), i.e., the absence criterion would not be fulfilled if (a) is false; the absence criterion would not be fulfilled if (b) is false; and the absence criterion would not be fulfilled if (c) is false.

[0055] If the control device detects a lack of the participation, it may revoke the function access in response. This may yield the effect that the assistant console may lose its access to one or more data modes provided by the consultation console, i.e., the assistant console may become unable to receive one or more modes of data from the consultation console anymore and / or to exert control over one or more functions of the consultation console or a device connected to the consultation console. This may be equivalently described as the assistant operator losing a permission related to the respectively revoked data mode. The function access may be revoked immediately upon determining for the first time that the absence criterion is fulfilled, or at a later time as discussed further below.

[0056] In this manner, the control device may provide for an effective permission control for the assistant operator that may be implemented in various ways. In a particular example, the receipt of video data, audio data, screen content, and / or data generated by one or more sensors or devices integrated with or connected to the consultation console (such as a medical imaging device) may become blocked for the assistant console when the function access is revoked.

[0057] In a specific radiology example, the remote assistance session provides the assistant console with the possibility to view and edit virtual user interface that is displayed on a screen of the consultation console and shared on a screen of the assistant console. As the control device detects that the absence criterion is fulfilled, the control device may revoke one or more particular edit permissions for the assistant operator at the consultation console. For instance, the function access may be revoked as a graceful drop-off from edit mode into view mode for the entire virtual user interface, or by deactivating particular control elements (such as a “start scan” button) or regions (e.g., an screen region displaying currently scanned patient tissue slices) on the virtual user interface.

[0058] When performing remote-assisted medical examinations, current setups typically require a user at site of the consultation console to be logged in to an on-premises stationary or mobile computer. The remote user also has to be logged in at the assistance center. Information regarding who is logged in to devices or assistance centers may be known to users; however, if the local or remote user is actually present at or near the device or assistance center may be not communicated via the remote assistance2024PF00683

[0059] 9 02.03.2026

[0060] session. This is a problem in current setups, as there are no checks for operator presence or presencedependent security features available.

[0061] The method described herein may overcome this problem by providing participation detection technology on both the consultation site and the assistance site, in combination with participation-dependent permission control and access security features. In this way it may be ensured that safety-critical operations can only be performed when the correct local and remote staff are present. Again, the method may likewise be applicable to any other scenarios that do not have a medical background, but may involve disclosure of sensitive information, or remote control of security or safety restricted devices, from one site through a collaborational session via a computer network to another site for other reasons.

[0062] The method may also yield the advantage that sensitive information that may get disclosed from the place where the consultation console is operated via the remote assistance session to the assistant console may have a higher probability of disclosure at times when the assistant operator is actively participating in the remote assistance session then at times when the assistant operator is not actively participating. In other words, such sensitive information may become less likely to get accidentally disclosed to another person near the assistant console while the assistant operator is not actively participating in the remote assistance session.

[0063] Moreover, the method may provide additional protection for such sensitive information at times when the medical operator is not actively participating in the remote assistance session. In these times, the medical operator may be unable to verify that the assistant operator exhibits an expected behavior with respect to the sensitive information. For instance, the assistant operator may be able to obtain information via the remote assistance session that the medical operator would not have voluntarily shared with the assistant operator if they were actively participating in the remote assistance session; or the assistant operator may be able to perform actions, including controlling devices, in a manner that the medical operator would not have allowed if they were actively participating in the remote assistance session.

[0064] In the specific radiology application scenario, remote operation of imaging equipment and other safety-critical medical equipment comes with risks, in particular regarding the presence and focus of local and remote personnel. When a remote expert is operating a medical system, local staff should be available to ensure safety of the patient. When a local operator is making a medical system available for remote operation, it is desirable to ensure that only the expected person is able to access the medical system remotely and that the remote expert is really present when the local operator assumes this to be the case.

[0065] It is pointed out that the consultation console may have multiple data processing functions and the remote assistance session may provide multiple independent function accesses to these multiple data processing functions. In this scenario, the method may provide for multiple separate absence criteria against which the received participation information may be monitored. Accordingly, execution of the2024PF00683

[0066] 10 02.03.2026

[0067] method may lead to independent revocations of the separate function accesses upon fulfillment of each independent absence criterion.

[0068] In an example, the absence criterion is fulfilled upon persistence of the lack of the participation for a predefined revoke period.

[0069] This may have the advantage of giving the potentially absent operator time to resume their participation in the remote assistance session. For instance, the absence criterion may specify that the lack of the participation is given if the participation of the respective operator cannot be determined for more than a specified number of cycles in a row, or for a predefined highest tolerable number of seconds. In this manner, the number of times in which the function access is revoked for insignificant reasons may be reduced. In a non-limiting example, the revoke period is set to 30 seconds such that the assistant operator’s permission will get revoked if they do not return to the remote assistance session within 30 seconds.

[0070] In an example, the method further comprises instructing the consultation console or the assistant console to output an alert signal via a respective user interface immediately if the lack of the participation persists for a predefined alert period shorter than the revoke period.

[0071] Providing an alert signal to a potentially absent operator may cause the operator to return to the remote assistance session before the function access is revoked. This may prevent the potentially absent operator from focusing their attention away from the remote assistance session for insignificant reasons too long. In this manner, unnecessary waiting times at the other end of the remote assistance session may be reduced or avoided and the efficiency of the remote assistance session may be increased. The alert signal may support this in a suitable manner, e.g., by indicating the lack of the participation or by asking to resume the participation. A console receiving from the control device an instruction for outputting the alert signal may forward the alert signal to a predefined other device such as a mobile device carried by the operator of the respective console.

[0072] Alerts can be generated using many different kinds of user interfaces. For instance, there may be an audible alert such as a bell, a voice message, or any other sound; a visual indicator, such as a flashing light or colored light emitted by dedicated hardware such as an LED indicator, or as a visual indicator on a screen of or connected to the respective console, e.g., showing a symbol or message. If personal identification is enabled, the potentially absent operator may also be reminded, e.g., by a pop-up or similar on-screen message, to look into the camera or resume speaking or to present any other identification element to renew their identification.

[0073] The monitoring may result in a detection of the lack of the participation at either the consultation console or the assistant console. In the following, the console where the lack of the participation is detected will be referred to as the unattended console and the other one of the consultation console and the assistant console is referred to as the attended console. Analogously, the operator lacking participation with respect to the remote assistance session will be referred to as the potentially absent operator, while the respective other operator is referred to as the continuing operator. It is noted that the2024PF00683

[0074] 11 02.03.2026

[0075] alert signal may be output at the unattended console to ask the potentially absent operator to return to the remote assistance session, but likewise, the alert signal may be output at the attended console, informing the continuing operator that the operator at the unattended console appears to be missing.

[0076] Preferably, the alert period is not more than half as long as the revoke period. In the example above, a preferred alert period matching a revoke period of 30 seconds may be 10 seconds, 12 seconds, or 15 seconds, for instance. In an example, the alert period for one or both of the consoles may be predefined or may be set as a user preference setting by an operator of the consultation console and / or the assistant console. In an example, the alert period may be reviewed by the respective operator of each console. In an example, the alert period may be a default setting or may be reconfigured on an individual basis, e.g., as a configuration parameter for a given remote assistance session.

[0077] Generally, a user interface may be an input device, an output device, or an interface of the respective console enabling a wired or wireless communicative and / or controllable attachment of an external device (e.g., a Bluetooth or Wi-Fi interface, or a USB, serial, or parallel port) such that units of the attached external device may be used as input and / or output devices. It is noted that an input device may also serve as an output device if it has output capabilities, such as a computer keyboard having backlit keys that can be used to generate a visual signal in order to output the alert signal.

[0078] In an example, the method further comprises, in case the monitoring results in a fulfillment of the absence criterion with respect to an unattended console selected from the consultation console and the assistant console, the other one of the consultation console and the assistant console being an attended console, and in case the participation information received from the attended console during the revoke period comprises a predefined call request indicator, instructing the unattended console to output a call signal asking to resume the participation in the remote assistance session.

[0079] This may further increase the efficiency of the remote assistance session and may be a further means of reducing waiting times at the attended console. For instance, the continuing operator may be shown a user interface element enabling the continuing operator to call the potentially absent operator back to the remote assistance session. This may also be useful if intervention of the assistant operator is necessary (e.g., if a device control action by the assistant operator is necessary while a radiological scan is running at the site of the consultation console. Likewise, the assistant operator may ask the medical operator to return to the remote assistance session if the assistant operator notices an event such as a medical emergency or a device failure that may require intervention by the potentially absent medical operator. In an example, the user interface element enabling to issue the call request may get visible or activated for triggering only after the fulfillment of the absence criterion is detected for the first time. Preferably, the control device instructs the unattended console to output the call signal as an immediate response to the receipt of the call request indicator. The call signal may be output by a user interface integrated with or connected to the unattended console.

[0080] In an example, the revoke period is a predefined first revoke period applied for determining the persistence of the lack of the participation by the medical operator and a predefined2024PF00683

[0081] 12 02.03.2026

[0082] second revoke period applied for determining the persistence of the lack of the participation by the assistant operator, wherein the second revoke period differs from a duration of the first revoke period by at least 10 percent of the duration of the first revoke period.

[0083] The ability to set different revoke periods for the medical operator and the assistant operator may make it possible to prioritize revocations of the function access. In particular, it may be possible to set a shorter revoke period for one operator whose participation in the remote assistance session is considered more important, or whose absence from the remote assistance session is considered to have a greater impact on the security of sensitive information that may be revealed at the site where the assistant console is operated. For example, a shorter revoke period may be set for the assistant operator if a large amount of sensitive information is expected to occur at the site where the consultation console is operated, or if it is expected that the assistant operator has to switch their focus away from the remote assistance session frequently. In another example, a shorter revoke period may be set for the medical operator if it can be expected that personal intervention by the medical operator may be necessary at a high frequency or if it is assumed that the subject of the remote assistance session has a high priority compared to other events taking place concurrently at the site where the consultation console is operated. It is envisaged that the second revoke period should differ in duration from the first revoke period by at least 10% of the duration of the first revoke period to obtain a significant difference between the time scales at which the function access is respectively revoked.

[0084] In an example, the monitoring comprises continuously identifying the assistant operator or the medical operator based on the participation information, wherein the absence criterion is fulfilled if the assistant operator cannot be identified.

[0085] Continuously identifying the assistant operator or the medical operator may help ensure that the person at the respective other side of the remote assistance session is actually the person who is expected to attend the remote assistance session at the other side. This may increase the security provided by the remote assistance session. A continuous identification may require receiving accreditation information when the remote assistance session is set up, providing evidence that the medical operator and the assistant operator are authorized for participating in the remote assistance session.

[0086] It may be possible to implement a participation detection scheme with personal identification using suitable sensor technologies at either end of the remote assistance session. These may include, without limitation: a camera with artificial intelligence (Al) enabled face detection and identification (e.g., using a built-in tablet camera or other webcam also used for communication, or a camera having a wide field of view to detect a person anywhere in the respective room); a camera with Al-enabled detection of identity badges or uniforms; a wireless (e.g., RFID) sensor reading an correspondingly configured tag carried by the respective operator; a wireless (e.g., Bluetooth) receiver identifying and measuring the distance from a Bluetooth sender carried by the operator; voice recognition or other audio cue detection using a microphone in the room to detect the presence of an operator in the room even when not facing the console screen; etc.2024PF00683

[0087] 13 02.03.2026

[0088] Based on the portion of the participation information enabling a personal identification of the medical operator or the assistant operator by the control device (also referred to herein as the identification data), the control device may, for instance, determine a degree of confidence describing a probability that the identification data actually describes the identified person. This may be connected to a threshold of positive identification stipulating that an identification is successful if, e.g., the degree of confidence is equal to or greater than a predefined first identification threshold value. Moreover, it may be possible to ask the continuing operator if a person appearing at the unattended console is actually the expected person. This variant may, for instance, be triggered if the control device is able to determine an identity of the person appearing at the unattended console, but the degree of confidence of identifying the person at the unattended console is low, e.g., below a predefined second identification threshold value that is greater than the first identification threshold value, but smaller than a limit degree of confidence for the theoretical case of an identification without any uncertainty.

[0089] In an example, the absence criterion is fulfilled if the participation information indicates an intention of the assistant operator or the medical operator to revoke the function access.

[0090] This may increase the security of the remote assistance session further as it may provide an additional or alternative possibility to revoke the function access manually. This may be helpful to shorten the time until the function access would be revoked automatically, e.g., based on sensor-provided information within the participation information, and may thus lower the risk of disclosing sensitive information to the assistant console. In addition, this may enable the medical operator to revoke the function access immediately if they become aware that an unauthorized person at the site of the assistant console may obtain sensitive information or remotely control a security or safety enabled device using the remote assistance session. Furthermore, the assistant operator may proactively cause revocation of the function access if they expect unauthorized persons may accidentally or intentionally obtain control over the remote assistance session or receive sensitive information by witnessing the unattended remote assistance session.

[0091] In an example, the monitoring comprises continuously identifying the assistant operator or the medical operator based on the participation information, wherein the identification comprises determining a degree of confidence of the identification, wherein the method further comprises updating the consultation console or the assistant console to output a confidence information indicating the degree of confidence currently determined.

[0092] Outputting a confidence information indicating a degree of confidence of identification (as described above) of the operator at the other end of the remote assistance session may give the operator receiving the confidence information an additional indication about the operator at the other end. This may help the receiving operator decide whether to revoke the function access manually or not. This may be especially helpful in setups where the receiving operator does not receive video and / or audio information from the other side of the session. The definition of a user interface given above may be applicable also for the present example of outputting confidence information. The confidence information2024PF00683

[0093] 14 02.03.2026

[0094] may be output by a user interface integrated with or connected to the consultation console or the assistant console, respectively. The console outputting the confidence information may be updated (kept up-to-date) by repeatedly transferring the degree of confidence that was determined most recently to the respective console and instructing the respective console to output a corresponding confidence information reflecting said most recently determined degree of confidence.

[0095] In an application example, the degree of confidence of the identification may be determined using the functionality of individual sensors. For instance, if a camera is recognizing an identifying object, such as the user’s face, then there may have available an associated confidence level (e.g., a probability value) from the object recognition algorithm. Or, if the identification is performed by merging two or more information sources, such as a camera for face detection, a microphone for speech detection, an infrared or radar array or a Bluetooth connection for proximity detection, a use of input devices such as a computer mouse, a computer keyboard, etc., then some numerical estimate may be determined to represent a combination, such as a maximum or a mean value, of confidence levels for the individual detection channels. It could also be possible to binarize each detection, e.g., by defining fixed confidence thresholds or ranges for each source, that, when triggered, give a positive detection for that source. This may be used for confirming an operator’s presence by one or more positive independent detections.

[0096] In scenarios deploying identification based on a degree of confidence, a negative identification may further be subject to a time criterion, e.g., if the degree of confidence is below a predefined confidence threshold for a sequence of identification intervals or for a predefined time interval during which the degree of confidence stays low.

[0097] In an example, the monitoring comprises determining the participation in the remote assistance session by at least one of:

[0098] obtaining a degree of focus of the assistant operator or the medical operator on the remote assistance session, wherein the absence criterion is fulfilled if the degree of focus lies outside a predefined focus range;

[0099] obtaining a probability of physical presence of the assistant operator at the assistant console or of the medical operator at the consultation console, wherein the absence criterion is fulfilled if the probability of physical presence lies outside a predefined presence range;

[0100] obtaining an activity measure of usage of the assistant console or the consultation console, wherein the absence criterion is fulfilled if the activity measure lies outside a predefined range of least activity; or

[0101] a combination thereof.

[0102] Determining the participation by obtaining a degree of focus may increase security of the remote assistance session by enabling the revocation of the function access in situations when the potentially absent operator is physically present at the absence console, but not actively participating in the remote assistance session. For instance, the degree of focus may be determined by monitoring a view2024PF00683

[0103] 15 02.03.2026

[0104] direction of the respective operator (e.g., by determining their eye movement and / or head orientation), a body posture relative to the respective console, variations in loudness of their vocal expression, etc.

[0105] Determining the participation by obtaining a probability of physical presence may increase security of the remote assistance session by enabling the revocation of the function access in situations when the potentially absent operator is not present in a vicinity of the absence console. For this purpose, the vicinity of a console may be defined by the respective operator’s ability to interact with the respective console (e.g., position within arms’ reach of an input device to the console) and / or to notice outputs (e.g., visual and / or audible) provided by an output device of the console. The operator’s position relative to the respective console may be determined e.g., using camera images, wireless distance sensing, etc.

[0106] Determining the participation by obtaining an activity measure of console usage may increase security of the remote assistance session by enabling the revocation of the function access in situations when the potentially absent operator is physically present at the absence console and apparently (e.g., as determinable from their body posture) paying attention to the remote assistance session, but does not actively use the remote assistance session (e.g., if the operator has fallen asleep). An activity measure may be a mono- or multimodal assessment of variation in a device state of one or more input devices to the respective console (e.g., key strokes of a computer keyboard or computer mouse, movements of a computer mouse or trackball, etc.).

[0107] The degree of focus, the probability of physical presence, and / or the activity measure may be obtained by an analysis of the participation information by the control device, and / or may be received as a portion of the participation information that was generated by preprocessing data from any data source of the participation information described herein by a computing device other than the control device. In an example, the degree of focus, the probability of physical presence, and / or the activity measure are obtained by an analysis of video stream data, audio stream data, and sensor data contained in the participation information.

[0108] The terms „focus range“, „presence range“, and „range of least activity“ are labels denoting numerical ranges for the degree of focus, the probability of physical presence, and the activity measure, respectively. In a non-limiting example, a focus range may be defined using a reference axis that represents an operator directly looking at a display connected to the respective console, and stipulating that an operator is participating in the remote assistance session if, e.g., a viewing direction or a head orientation of the operator is within a predefined angle (e.g., ± 30 degrees) from the reference axis. In a non-limiting example, a presence range may be defined as a range of values of an operator’s probability of physical presence that can be regarded as the operator being sufficiently near the respective console such that a participation in the remote assistance session is possible, which may, for instance, be predefined to be true if the probability of physical presence is at least 50 percent, preferably at least 75 percent, and more preferably at least 90 percent. In a non-limiting example, a range of least activity may be defined with respect to a number of mouse clicks or key strokes within a certain time unit (e.g., per2024PF00683

[0109] 16 02.03.2026

[0110] minute), and stipulating, e.g., that the respective operator is participating in the remote assistance session if the number of mouse clicks or key strokes per time unit is greater than a predefined threshold (e.g., at least 2 clicks or strokes per minute), or if the time since the last key stroke or mouse click is less than a predefined threshold time (e.g., 1 minute, preferably 30 seconds, and more preferably 20 seconds).

[0111] It is noted that the above explanations on combinability of statements regarding possible manners of fulfillment of the absence criterion are also valid for the statements regarding possible manners of fulfillment of the absence criterion described under the present example, taken alone or in combination with any other statements regarding possible manners of fulfillment of the absence criterion described herein.

[0112] In an example, the method further comprises restoring the function access upon termination of the fulfillment of the absence criterion.

[0113] This may enable resuming the remote assistance session after a pause with a lower barrier than, e.g., having to set up the remote assistance session again with new operator accreditations. In particular, instead of actively requesting permission for restoring the function access, a revoked function access may be restored automatically once the potentially absent participant is detected again, so that no more user interaction may be required. Automatic function access restoral may be especially useful in combination with operator identification to ensure that the participation is actually resumed by the expected person.

[0114] In an example, the restoring of the function access further is performed in case the participation information comprises a clearance information indicating an input via a user interface of the attended console descriptive of an agreement to the restoring of the function access.

[0115] Restoring the function access in response to receiving a user-provided clearance information may increase security of the remote assistance session by leaving the re-authentication of the potentially absent operator to the continuing operator. This may be especially useful if the remote assistance session provides at least one communication channel (e.g., audio, video) between the consultation console and the assistant console. However, this does not necessarily have to be the case if, e.g., the continuing operator has a communication channel to the potentially absent operatorthat is independent of the remote assistance session (e.g., using a phone or computer), or if the continuing operator has independent information indicating, e.g., a time when the potentially absent operator is scheduled to return to the remote assistance session. A user interface that is suitable for the present example may be defined as elsewhere herein. The control device may receive the clearance information as part of the participation information. In a variant, the restoring of the function access upon termination of the fulfillment of the absence criterion is performed only in case of receiving the clearance information.

[0116] In an example, the data processing function enables a transfer of data to the assistant console; or the data processing function enables a remote control of medical equipment via the assistant console; or a combination thereof.2024PF00683

[0117] 17 02.03.2026

[0118] The data processing function enabling a transfer of data to the assistant console may ensure that the assistant operator cannot receive further information, potentially including sensitive information, when the function access is revoked. The data processing function enabling a remote control of medical equipment via the assistant console may ensure that the assistant operator cannot exert remote control of functions of the consultation console or devices connected thereto when the function access is revoked. In both cases, the security and / or safety provided by the remote assistance session may be increased. A user interface that is suitable for the present example may be defined as elsewhere herein.

[0119] In an example, the method further comprises, upon fulfilment of the absence criterion, causing the assistant console to notify the assistant operator about the revocation via a user interface of the assistant console.

[0120] Notifying the assistant operator about the revocation of the function access may reduce the probability that the assistant operator gets confused if they return to the remote assistance session, but the remote assistance session does not show an expected behavior because the function access has been revoked in the meantime. Moreover, a revocation notification may reduce stress for the assistant operator by showing that there is no need anymore to keep the function access alive by showing more user interaction. For instance, the assistant operator may be notified about the revocation by outputting a visual or audible signal, such as an LED light, a bell sound, or an informative message displayed on a screen or played back using a loudspeaker; and / or by providing a negative notification such as dimming a screen, shading a screen region, or muting a loudspeaker that, respectively, would be active if the function access was still enabled. A user interface that is suitable for the present example may be defined as elsewhere herein.

[0121] In the following, reference is made to the drawings, where similar elements are denoted by the same reference numerals.

[0122] Fig. 1 shows an exemplary environment 100 that may be suitable for executing implementations of the method 200. According to the example of Fig. 1, the environment 100 may comprise a control device 150 that is communicatively coupled to a computer network. The computer network may allow for data transfer between devices located at a consultation site and devices located at an assistance center. At the consultation site, at least a consultation console 110 may be communicatively coupled to the computer network. At the assistance site, at least an assistant console 130 may be communicatively coupled to the computer network. The consultation console 110 and the assistant console 130 may be configured to set up a remote assistance session via the computer network. The remote assistance session may allow for transferring information between the consultation console 110 and the assistant console 130, including, but not limited to, a video stream, an audio stream, device control signals, graphical user interface output, and other data or signals. Information exchanged between the consultation console 110 and the assistant console 130 via the computer network may form part of the participation information that is monitored by the control device 150 for fulfillment of a predefined absence criterion.2024PF00683

[0123] 18 02.03.2026

[0124] Atached to the consultation console 110 may be one or more output devices 112 such as a screen, a loudspeaker, etc., and one or more input devices 114 such as a computer keyboard, a computer mouse, etc. Optionally, a peripheral device 120 such as a medical imaging system may be connected to the consultation console 110. The consultation console 110 may also comprise a periphery interface 122 enabling an exchange of information between the consultation console 110 and the peripheral device 120; an input stream interface 124 enabling the consultation console 110 to receive information from the computer network, and an output stream interface 126 enabling the consultation console 110 to transfer information to the computer network. The peripheral device 120 or other devices such as sensors (e.g., microphones, cameras, ... ) reacting on events at the consultation site may be possibly, but not exclusively, sources of sensitive information that should be disclosed to the assistant console 130 only if trustworthiness of the remote assistance session can be assumed.

[0125] One or more participation sensors 116 and an alerting system 118 are also shown as elements that may be present at the consultation site. The control device 150 may receive information from the participation sensors 116 and may transfer signals to the alerting system 118. For this purpose, the participation sensors 116 and the alerting system 118 may be communicatively coupled to the control device 150 via the computer network or one or more communication connections provided by other infrastructure or by dedicated connections.

[0126] Analogously, the assistant console 130 may be equipped with one or more output devices 132 such as a screen, a loudspeaker, etc., and one or more input devices 134 such as a computer keyboard, a computer mouse etc. One or more participation sensors 136 and an alerting system 138 are also shown as elements that may be present at the assistance center. The control device 150 may receive information from the participation sensors 136 and may transfer signals to the alerting system 138. For this purpose, the participation sensors 136 and the alerting system 138 may be communicatively coupled to the control device 150 via the computer network or one or more communication connections provided by other infrastructure or by dedicated connections.

[0127] To implement steps of the method 200, the control device 150 may be configured with functions that are shown in the drawing as an event control module 152 and a permission control module 154. Each of the event control module 152 and the permission control module 154 may be implemented in hardware, in software, or as a combination thereof.

[0128] The permission control module 154 may be adapted for influencing a configuration of the remote assistance session at least with respect to its capability of allowing the assistant console 130 to access to the data processing function of the consultation console 110. The data processing function of the consultation console 110 may be any function that allows for processing or transferring data that potentially contains sensitive information and may be disclosed to the assistant console 130 during the remote assistance session. The interfaces 122, 124, and 126 may be examples of the data processing function of the consultation console 110. The permission control module 154 may be adapted for performing at least the step of revoking 206 the function access.2024PF00683

[0129] 19 02.03.2026

[0130] The event control module 152 may be adapted for receiving, processing, and transmitting information via the computer network, including but not limited to the remote assistance session, and / or communication channels other than the computer network or the remote assistance session. In particular, the event control module 152 may receive data generated by the participation sensors 116 at the consultation site and / or data generated by the participation sensors 136 at the assistance center; and may transmit signals to the alerting system 118 at the consultation site and / or to the alerting system 138 at the assistance center. The event control module 152 may exchange information with the consultation console 110, the assistant console 130, and / or any devices respectively connected thereto. The event control module 152 may receive 202 any portions of participation information as described herein.

[0131] Furthermore, event control module 152 may implement logic allowing the event control module 152 to perform the monitoring 204 of the participation information as described herein. Without limitation, the participation information may include any information that potentially contains a measure of participation for a medical operator being registered with the remote assistance session and interacting with the consultation console 110 or any other device at the consultation site participating in the remote assistance session, and / or for an assistant operator being registered with the remote assistance session and interacting with the assistant console 130 or any other device at the assistance center participating in the remote assistance session. In particular, portions of the participation information describing the participation may be descriptive of the medical operator or the assistant operator; descriptive of a physical presence or location of the medical operator with respect to the consultation console 110 or any other device at the consultation site participating in the remote assistance session, and / or a physical presence or location of the assistant operator with respect to the assistant console 130 or any other device at the assistance center participating in the remote assistance session; descriptive of a focus or attention of the medical operator with respect to the consultation console 110, any other device at the consultation site participating in the remote assistance session, or one or more of the participation sensor(s) 116 at the consultation site, and / or a focus or attention of the assistant operator with respect to the assistant console 130, any other device at the assistance center participating in the remote assistance session, or one or more of the participation sensor(s) 136 at the assistance center; a signal indicating a user’s intention to call the medical operator or the assistant operator back to the remote assistance session; a signal indicating a user’s intention to revoke the function access as soon as possible; or combinations thereof.

[0132] The event control module 152 may be configured with a predefined absence criterion against which the received participation information may be analyzed. The absence criterion may specify one or more conditions that the participation information must fulfil to allow for concluding that the medical operator or the assistant operator is absent from the remote assistance session or lacking focus, attention, or interaction with respect to the remote assistance session. In order to determine whether the absence criterion is fulfilled, the logic of the event control module 152 may implement various kinds of logic, including, but not limited to, timing logic allowing the event control module 152 to determine whether a predefined revoke period or a predefined alert period has expired; identification logic allowing2024PF00683

[0133] 20 02.03.2026

[0134] the event control module 152 to determine an identity of the medical operator or the assistant operator; logic allowing for determining a degree of focus of the assistant operator or the medical operator on the remote assistance session; logic allowing for determining a probability of physical presence of the assistant operator at the assistant console or of the medical operator at the consultation console; logic allowing for determining an activity measure of usage of the assistant console or the consultation console; or combinations thereof.

[0135] If the absence criterion is fulfilled, the event control module 152 may cause the permission control module 154 to configure the remote assistance session to revoke 206 the function access associated with the absence criterion, such that the assistant console 130, and correspondingly, any user of the assistant console 130, may become unable to use the data processing function of the consultation console 110 via the assistant console 130. For this purpose, the permission control module 154 may instruct the consultation console 110 to set a corresponding parameter controlling the access to the data processing function at least by the assistant console 130. In the example of Fig. 1, revoking the function access may include blocking access of the assistant console 130 to a data interface function of the consultation console 110 to prevent disclosure of sensitive information that is processed by the data interface function via the remote assistance session through the computer network to the assistant console 130 while the potentially absent operator continues to be not participating in the remote assistance session; and / or blocking access of the assistant console 130 to a control interface function of the consultation console 110 to prevent usage of the consultation console 110 and / or devices connected to the consultation console 110 such as the peripheral device 120 using the assistant console 130 via the remote assistance session through the computer network. Possible choices for the data interface function may be the periphery interface 122 and / or the output stream interface 126; a possible choice for the control interface function may be the input stream interface 124.

[0136] The environment 100 shown in the drawing has a particular configuration that has been chosen for the sole purpose of illustrating characteristics and notions of the invention. In this configuration, multiple elements of the environment 100 are shown with arrows symbolizing connections enabling data exchange and control flow. It is understood that numerous other configurations of the environment 100 may exist which may be suitable as well for executing implementations of the method 200. In particular, elements or connections shown in the drawing as components of the environment 100 may be missing in such alternative configurations; elements and connections not shown in the drawing may be components of the environment 100 in such alternative configurations; and connections shown in the drawing may have a different topology in such alternative configurations. More particularly, elements that are shown as having a dedicated connection to the control module 150 may be connected to one of the consoles 110, 130 instead in alternative configurations; and / or there may be alternative configurations where the control module 150 may have different functions, including scenarios where some or all of the functions shown in the drawing as functions of the control module 150 may be taken over by other elements of the environment 100 such as the consultation console 110, the assistant console2024PF00683

[0137] 21 02.03.2026

[0138] 130, or further devices not shown in the drawing that may exist along the connection through the computer network between the consultation console 110 and the assistant console 130. This may include the possibility that steps or sub-steps of the method 200 can be performed by the control device 150 and further devices within the environment 100 in a distributed execution. In other implementations, the alerting system 118 at the consultation site and / or the alerting system 138 at the assistance center may be missing or integrated with other devices such as the consultation console 110 or the assistant console 130, respectively. Likewise, there may be implementations where the participation sensor(s) 116 at the consultation site and / or the participation sensor(s) 136 at the assistance center may be missing because the participation information allowing for determining whether the absence criterion is fulfilled is generated by one or more different device(s).

[0139] Fig. 2 is a flow chart illustrating steps of the method 200 of controlling a remote assistance session that grants an assistant console 130, used by an assistant operator who is a participant of the remote assistance session, a function access to a data processing function, including potentially processing sensitive information, of a consultation console 110 used by a medical operator who is also a participant of the remote assistance session. The method 200 may include receiving 202 participation information that includes portions of information that may serve as a measure of participation of the assistant operator or the medical operator in the remote assistant session. It is to be understood that the word “receiving” may be used with synonyms, such as “obtaining”. The method 200 may further include monitoring 204 the participation information. The receiving 202 and the monitoring 204 may be quasi-continuous processes running or repeated throughout the remote assistance session. The monitoring 204 may include determining whether a predefined absence criterion describing a lack of the participation of the assistant operator or the medical operator with respect to the remote assistance session is fulfilled. If the absence criterion is not fulfilled, the method 200 may continue with receiving 202 the participation information and monitoring 204 the participation information. If the absence criterion is fulfilled, the method 200 may continue with revoking 206 the function access by configuring the remote assistance session or the consultation console 110 to block the access of the assistant console 130 to the data processing function of the consultation console 110. The receipt 202 of the participation information and the monitoring 204 of the participation information may continue concurrently as the function access is revoked.

[0140] Fig. 3 is a flow chart 300 illustrating a more complex application scenario of the environment 100 that may be subsumed under the method 200. According to the application scenario, the participation sensors 116 and 118 may provide participation information to the event control module 152. Based on predefined rules about maximum periods of lacking participation on each side, the event control module 152 may:

[0141] generate alerts using alerting system 118 and / or alerting system 138 to notify the continuing operator when the potentially absent operator on the other end of the remote assistance session appears to be absent or idle;2024PF00683

[0142] 22 02.03.2026

[0143] generate alerts using alerting system 118 and / or alerting system 138 to ask the potentially absent operator to return to the remote assistance session before the function access is revoked or before the continuing operator is notified; and

[0144] instruct the permission control module 154 to revoke the function access (e.g., by selectively blocking viewing or editing permissions on a user interface of the consultation console 110 that is shared with the assistant console 130 via the remote assistance session, or blocking other interaction permissions of the assistant console 130 with respect to the consultation console 110 or a device connected thereto).

[0145] The flow chart 300 may start with the control device 150 receiving 302 participation information describing a participation of the assistant operator or the medical operator (in the following shortly “the operator”) with respect to the remote assistance session. The control device 150 monitors 304 the participation information, determining whether a predefined absence criterion is fulfilled that defines an indication within the participation information that the operator is showing a lack of participation with respect to the remote assistance session.

[0146] If the monitoring 304 yields the result that the operator is participating with the remote assistance session, the flow chart 300 may continue with the steps of detecting 302 and monitoring 304. This may include the case that the operator has just returned to the remote assistance session after a period of lacking participation. In this case, the function access to the data processing function of the consultation console 110 may be restored 306 automatically or in response to receiving a positive indication from the continuing operator to a request issued by the control device 150 informing the continuing operator that the potentially absent operator seems to have just returned to the remote assistance session and asking the continuing operator whether the function access should be restored or not.

[0147] If the monitoring 304 shows 308 that the operator has not been participating in the remote assistance session for a predefined first alert period, the control device 150 may issue a first alert signal to request the operator to return to the remote assistance session. The first alert signal may be output using the respective alert system 118, 138 or a respective output device 112, 132 that may be connected to or integrated with the respective console 110, 130. If the first alert period has not yet elapsed, or if the operator returns to the remote assistance session before the first alert period expires, the flow chart 300 may return to continuing with receiving 302 further portions of participation information and monitoring 304 the participation information.

[0148] If, subsequently, the monitoring 304 shows 312 that the operator continues to be not participating in the remote assistance session for a predefined second alert period , the control device 150 may issue a second alert signal to notify 314 the continuing operator that the operator is potentially absent from the remote assistance session. The second alert signal may be output using the respective alert system 118, 138 or a respective output device 112, 132 that may be connected to or integrated with the respective console 110, 130. If the second alert period has not yet elapsed, or if the operator returns to the2024PF00683

[0149] 23 02.03.2026

[0150] remote assistance session before the second alert period expires, the flow chart 300 may return to continuing with receiving 302 further portions of participation information and monitoring 304 the participation information.

[0151] After waiting 316 for a predefined revoke period (or receiving an explicit acknowledgment of the continuing operator), a safety-critical permission of the assistant operator (the function access implemented, e.g., as an edit permission on a graphical user interface of the consultation console 110) may be revoked 318. In an example, the revoke period may last two minutes to allow the potentially absent operator to finish an important task. Once the operator has returned to the remote assistance session, any revoked permissions may be resumed 306 automatically or on request.

[0152] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed examples. It is understood that one or more of the aforementioned examples or embodiments of the invention may be combined as long as the combined embodiments are not mutually exclusive.

[0153] As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as an apparatus, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer executable code embodied thereon.

[0154] Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A ‘computer-readable storage medium’ as used herein encompasses any tangible storage medium which may store instructions which are executable by a processor or computational system of a computing device. The computer-readable storage medium may be referred to as a computer-readable non-transitory storage medium. The computer-readable storage medium may also be referred to as a tangible computer readable medium. In some embodiments, a computer-readable storage medium may also be able to store data which is able to be accessed by the computational system of the computing device. Examples of computer-readable storage media include, but are not limited to: a floppy disk, a magnetic hard disk drive, a solid state hard disk, flash memory, a USB thumb drive, Random Access Memory (RAM), Read Only Memory (ROM), an optical disk, a magneto-optical disk, and the register file of the computational system. Examples of optical disks include Compact Disks (CD) and Digital Versatile Disks (DVD), for example CD-ROM, CD-RW, CD-R, DVD-ROM, DVD-RW, or DVD-R disks. The term computer readable-storage medium also refers to various types of recording media capable of being accessed by the computer device via a network or communication link. For example,2024PF00683

[0155] 24 02.03.2026

[0156] data may be retrieved over a modem, over the internet, or over a local area network. Computer executable code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0157] A computer readable signal medium may include a propagated data signal with computer executable code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0158] ‘Computer memory’ or ‘memory’ is an example of a computer-readable storage medium. Computer memory is any memory which is directly accessible to a computational system. ‘Computer storage’ or ‘storage’ is a further example of a computer-readable storage medium. Computer storage is any non-volatile computer-readable storage medium. In some embodiments computer storage may also be computer memory or vice versa.

[0159] A ‘computational system’ as used herein encompasses an electronic component which is able to execute a program or machine executable instruction or computer executable code. References to the computational system comprising the example of “a computational system” should be interpreted as possibly containing more than one computational system or processing core. The computational system may for instance be a multi -core processor. A computational system may also refer to a collection of computational systems within a single computer system or distributed amongst multiple computer systems. The term computational system should also be interpreted to possibly refer to a collection or network of computing devices each comprising a processor or computational systems. The machine executable code or instructions may be executed by multiple computational systems or processors that may be within the same computing device or which may even be distributed across multiple computing devices.

[0160] Machine executable instructions or computer executable code may comprise instructions or a program which causes a processor or other computational system to perform an aspect of the present invention. Computer executable code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages and compiled into machine executable instructions. In some instances, the computer executable code may be in the form of a high-level language or in a pre-compiled form and be used in conjunction with an interpreter which generates the machine executable instructions on the fly. In other instances, the machine2024PF00683

[0161] 25 02.03.2026

[0162] executable instructions or computer executable code may be in the form of programming for programmable logic gate arrays.

[0163] The computer executable code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0164] Aspects of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It is understood that each block or a portion of the blocks of the flowchart, illustrations, and / or block diagrams, can be implemented by computer program instructions in form of computer executable code when applicable. It is further understood that, when not mutually exclusive, combinations of blocks in different flowcharts, illustrations, and / or block diagrams may be combined. These computer program instructions may be provided to a computational system of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the computational system of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0165] These machine executable instructions or computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function / act specified in the flowchart and / or block diagram block or blocks.

[0166] The machine executable instructions or computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0167] A ‘user interface’ as used herein is an interface which allows a user or operator to interact with a computer or computer system. A ‘user interface’ may also be referred to as a ‘human interface device.’ A user interface may provide information or data to the operator and / or receive information or data from the operator. A user interface may enable input from an operator to be received by the computer and may provide output to the user from the computer. In other words, the user interface may allow an operator to control or manipulate a computer and the interface may allow the computer to indicate the effects of the operator's control or manipulation. The display of data or information on a display or a2024PF00683

[0168] 26 02.03.2026

[0169] graphical user interface is an example of providing information to an operator. The receiving of data through a keyboard, mouse, trackball, touchpad, pointing stick, graphics tablet, joystick, gamepad, webcam, headset, pedals, wired glove, remote control, and accelerometer are all examples of user interface components which enable the receiving of information or data from an operator.

[0170] A ‘hardware interface’ as used herein encompasses an interface which enables the computational system of a computer system to interact with and / or control an external computing device and / or apparatus. A hardware interface may allow a computational system to send control signals or instructions to an external computing device and / or apparatus. A hardware interface may also enable a computational system to exchange data with an external computing device and / or apparatus. Examples of a hardware interface include but are not limited to: a universal serial bus, IEEE 1394 port, parallel port, IEEE 1284 port, serial port, RS-232 port, IEEE-488 port, Bluetooth connection, Wireless local area network connection, TCP / IP connection, Ethernet connection, control voltage interface, MIDI interface, analog input interface, and digital input interface.

[0171] A ‘display’ or ‘display device’ as used herein encompasses an output device or a user interface adapted for displaying images or data. A display may output visual, audio, and or tactile data. Examples of a display include, but are not limited to: a computer monitor, a television screen, a touch screen, tactile electronic display, Braille screen, Cathode ray tube (CRT), Storage tube, Bi-stable display, Electronic paper, Vector display, Flat panel display, Vacuum fluorescent display (VF), Light-emitting diode (LED) displays, Electroluminescent display (ELD), Plasma display panels (PDP), Liquid crystal display (LCD), Organic light-emitting diode displays (OLED), a projector, and Head-mounted display.

[0172] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.

[0173] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope.2024PF00683

[0174] 27 02.03.2026

[0175] LIST OF REFERENCE NUMERALS

[0176] 100 Environment

[0177] 110 Consultation console

[0178] 112 Output device(s)

[0179] 114 Input device(s)

[0180] 116 Participation sensor(s)

[0181] 118 Alerting system

[0182] 120 Peripheral device

[0183] 122 Periphery interface

[0184] 124 Input stream interface

[0185] 126 Output stream interface

[0186] 130 Assistant console

[0187] 132 Output device(s)

[0188] 134 Input device(s)

[0189] 136 Participation sensor(s)

[0190] 138 Alerting system

[0191] 150 Control device

[0192] 152 Event control module

[0193] 154 Permission control module

[0194] 200 Method

[0195] 202 Receive / obtain participation information

[0196] 204 Monitor participation information

[0197] 206 Revoke function access

[0198] 302 Receive participation information

[0199] 304 Monitor participation information

[0200] 306 Restore function access / request approval

[0201] 308 First alert period elapsed?

[0202] 310 Instruct unattended console to output alert signal

[0203] 312 Second alert period elapsed?

[0204] 314 Instruct attended console to output alert signal

[0205] 316 Wait until revoke period elapses or receipt of revoke intention signal 318 Revoke function access

Claims

2024PF0068302.03.2026CLAIMS:Claim 1. A computer-implemented method (200) of controlling a remote assistance session established via a computer network between a consultation console (110) and an assistant console (130), the remote assistance session being configured to provide the assistant console a function access to a data processing function of the consultation console, the method comprising:obtaining (202, 302) participation information describing a participation in the remote assistance session by an assistant operator at the assistant console or by a medical operator at the consultation console;monitoring (204, 304) the participation information for fulfillment of a predefined absence criterion describing a lack of the participation, wherein the absence criterion is fulfilled upon persistence of the lack of the participation for a predefined revoke period;upon fulfillment of the absence criterion, revoking (206, 318) the function access, and wherein in case the monitoring results in a fulfillment of the absence criterion with respect to an unattended console selected from the consultation console and the assistant console, the other one of the consultation console and the assistant console being an attended console, and in case the participation information received from the attended console during the revoke period comprises a predefined call request indicator, instructing the unattended console to output a call signal asking to resume the participation in the remote assistance session.Claim 2. The method of claim 1, further comprising instructing the consultation console or the assistant console to output an alert signal via a respective user interface immediately if the lack of the participation persists for a predefined alert period shorter than the revoke period.Claim 3. The method of any one of claims 1 or 2, wherein the revoke period is a predefined first revoke period applied for determining the persistence of the lack of the participation by the medical operator and a predefined second revoke period applied for determining the persistence of the lack of the participation by the assistant operator, wherein a duration of the second revoke period differs from a duration of the first revoke period by at least 10 percent of the duration of the first revoke period.Claim 4. The method of any one of the preceding claims, wherein the monitoring (204, 304) further comprises continuously identifying the assistant operator and / or the medical operator based on the2024PF0068329 02.03.2026participation information, wherein the absence criterion is fulfilled if the assistant operator cannot be identified.Claim 5. The method of any one of the preceding claims, wherein the absence criterion is fulfilled if the participation information indicates an intention of the assistant operator or the medical operator to revoke the function access.Claim 6. The method of claim 5, wherein the monitoring (204, 304) comprises continuously identifying the assistant operator or the medical operator based on the participation information, wherein the identification comprises determining a degree of confidence of the identification, wherein the method further comprises updating the consultation console or the assistant console to output a confidence information indicating the degree of confidence currently determined.Claim 7. The method of any of the preceding claims, wherein the monitoring comprises determining the participation in the remote assistance session by at least one of:obtaining a degree of focus of the assistant operator or the medical operator on the remote assistance session, wherein the absence criterion is fulfilled if the degree of focus lies outside a predefined focus range;obtaining a probability of physical presence of the assistant operator at the assistant console or of the medical operator at the consultation console, wherein the absence criterion is fulfilled if the probability of physical presence lies outside a predefined presence range;obtaining an activity measure of usage of the assistant console or the consultation console, wherein the absence criterion is fulfilled if the activity measure lies outside a predefined range of least activity; ora combination thereof.Claim 8. The method of any one of the preceding claims, further comprising restoring the function access upon termination of the fulfillment of the absence criterion.Claim 9. The method of claim 8, the restoring of the function access further being performed in case the participation information comprises a clearance information indicating an input via a user interface of the attended console descriptive of an agreement to the restoring of the function access.Claim 10. The method of any one of the preceding claims, the data processing function enabling a transfer of data to the assistant console; or the data processing function enabling a remote control of medical equipment via the assistant console; or a combination thereof.2024PF0068330 02.03.2026Claim 11. The method of any one of the preceding claims, further comprising, upon fulfilment of the absence criterion, causing the assistant console to notify the assistant operator about the revocation via a user interface of the assistant console.Claim 12. The method of any one of the preceding claims, wherein a control device configured to perform a multimodal analysis of session payload, the multimodal analysis comprising analyzing at least audio and video data, to determine the fulfillment of the absence criterion, the absence criterion having a higher significance.Claim 13. The method of claim 12, wherein the control device is further configured to detect lack of participation and configured to revoke the function access if such lack of participation is determined.Claim 14. The method of claim 13, wherein revoking the function access comprises limiting access to one or more data modes, such limitation of access of data modes comprising reducing data use, altering and / or blocking access, and / orexerting control over one or more functions of the consultation console or a device connected to the consultation console.Claim 15. The method of any one of claims 4-14, wherein the step of continuous identification of the assistant operator and / or the medical operator comprises identifying participation of both said operators, and wherein revocation of the function access is prioritized based on degree of importance of said operators, wherein optionally the step of continuous identification further comprises receiving accreditation information when the remote session is set up and / or requiring authorization of said operators.Claim 16. The method of any one of the preceding claims, wherein a revoked function access is restored automatically once a potentially absent participant is detected again.Claim 17. The method of any one of the preceding claims, wherein revoking the function access comprises blocking access of the assistant console to a data interface function of the consultation console and / or blocking access of the assistant console to a control interface function of the consultation console.Claim 18. The method of any one of the preceding claims, wherein upon fulfillment of the absence criterion, the consultation console is configured to block access of the processing function of the consultation console.2024PF0068331 02.03.2026Claim 19. A system comprising a processor adapted to perform the steps of the method of any one of claims 1-18.Claim 20. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method of any one of claims 1-18.

Images