Method, system and components for coupling a companion system to a primary system and enabling at least partial control of the primary system

Abstract

Disclosed is a method for coupling a companion system (12) to a primary system (10) and enabling at least partial control of the primary system (10) by the companion system (12), the companion system (12) comprising at least one input and output device including a display for interacting with a user and a connecting port for connecting the companion system (12) to the primary system (10), the primary system (10) being configured for directly executing a primary software, the method comprising the steps of connecting the companion system (12) to the primary system (10), generating a user interface (34), displaying the user interface (34) via the companion system (12), and giving the companion system (12) access to at least partially control the primary software via the displayed user interface (34).

Description

METHOD, SYSTEM AND COMPONENTS FOR COUPLING A COMPANION SYSTEM TO APRIMARY SYSTEM AND ENABLING AT LEAST PARTIAL CONTROL OF THE PRIMARY SYSTEMTECHNICAL FIELD

[0001] The invention concerns a method, a system and components for coupling a companion system to a primary system and enabling at least partial control of the primary system by the companion system. The invention is particularly useful when the primary system is a conferencing system and the companion system comprises a portable user device that shall be coupled to the conferencing system and shall be used to at least partially control the conferencing system.TECHNICAL BACKGROUND

[0002] In recent years, video and so-called hybrid conferences have become a standard way of human interaction, in particular for working. In hybrid conferences, some people are physically present in the same room while other participants are connected via corresponding conferencing hard- and software and a data exchange net, in particular the internet, to input and output devices in the room, such as displays, microphones and speakers.

[0003] While many companies have set up conferencing equipment such as displays, cameras, microphones, speakers, electronic whiteboards etc. in meeting rooms, there are two different general approaches towards using this equipment that can be categorized according where the respective conferencing software is run: on a device, such as a desktop, laptop, notebook or tablet computer, or even a smart phone, which is typically easy portable and which a user brings to the room, or on a dedicated conferencing system that typically stays in the room (note that ameeting "room" is not necessarily a fixed room in the classical sense, but comprises any type of meeting area). For sake of clarity and brevity, and to distinguish the portable devices from the equipment permanently or temporarily installed in the meeting area, the portable devices will be called user devices, even though they may not necessarily be brought by a user actually intending to use the conferencing system, or, in general, a primary system, but may instead haven been brought to the meeting room by someone else. The important thing is that the user is familiar with the "user device" because he uses such device regularly. Also, there may be situations where such device has been somehow fixed to the meeting room, for example to prevent theft on a booth during a trade show, so that the user device is actually no longer portable.

[0004] Some very successful tools and methods for putting a very easy to use BYOD approach into practice in video and hybrid meetings are described in WO 2013 / 037979 A1 and WO 2019 / 129696 A1. The innovations described in those documents enable users to host video meetings and to share content from their own user device by using their preferred conferencing software while benefitting from high quality audio and video equipment installed in a meeting room, leading to higher satisfaction of working, greater productivity and better so-called "user experience" as defined in DIN EN ISO 9241, 210. Sharing data such as presentations stored on a user device, which data otherwise would have to be somehow transferred to respective presentation equipment e.g. via an USB stick, and presenting from the user device is very efficient, enables easy collaboration and has shown to lead higher employee engagement.

[0005] In the second approach, where the conferencing software is not run on a device brought by the user, interaction between the user and a conferencing system is enabled by a controller, typically in form of a so-called table top controller placed on a meeting table, which allows the user to start a meeting, configure settings such as audio volume, mute / unmute the microphone, control the camera mode, configure auto-framing and / or composition, control pan-tilt-zoom parameters, monitor the system state, add or remove participants and so on. Such controllers are generally the only way of interacting with the conferencing system according to the second approach.

[0006] Due to different advantages, the BYOD approach has become common practice in many companies. Employees prefer working with their own devices, with which they are well familiar due to regular use, rather than working with equipment they only use now and then, which in turn is also beneficial for the company as working time is used more efficiently. However, thesecond approach has its advantages too. If the preferred conferencing software is directly executed on the conferencing system, only one licence might be necessary, the software might be managed and updated centrally, and certain security issued may be addressed more easily such as for example allowing physically present guests of the company access to the conferencing system via a secure channel which access would not be allowed when the meeting is hosted by a user device brought by an employee. However, the controller device necessary for interacting with the conferencing system in the second approach can introduce several challenges. For example, to interact with the conferencing system, the user must first understand the controller device and its user interface, with which he may not be familiar. Installing the controller in a professional manner and connecting it with the other other components of the conferencing system in a meeting room is costly, and the controller typically also requires regular maintenance, like battery replacement, firmware update and the like.SUMMARY

[0007] In view of the aforementioned problems, it is an object of the invention to facilitate coupling a user device, or more general, a companion system (which may comprise the user device), to a primary system (such as e.g. a conferencing system) and enabling the companion system to at least partially control the primary system, i.e. to control certain functions of the primary system.

[0008] The object is achieved by a method according to claim 1. Independent claims 17, 18 and 19 relate to a dongle, a conferencing system and a user device configured for being used in the method according to the invention. The dependent claims relate to advantageous embodiments of the invention.

[0009] According to the invention, a method for coupling a companion system to a primary system and enabling at least partial control of the primary system by the companion system, wherein the companion system comprises at least one input and output device including a display for interacting with a user and a connecting port for connecting the companion system to the primary system, the primary system being configured for directly executing a primary software, comprises the steps of connecting the companion system to the primary system, generating a user interface, displaying the user interface via the companion system, and giving the companion system access to at least partially control the primary software via the displayed user interface. Note that the term "connecting port" is used herein in the broadest sense, i.e. theport does not need to be a port for wire-bound connections such as e.g. a USB port into which a corresponding USB plug is physically plugged in, it can also be a port for wireless connections such as for example Bluetooth® or WiFi, and accordingly the connection between the user device and the conferencing system can be wireless or wire-bound. Also, it does not need to be a direct connection, it can also be a connection via other intermediate means that can be virtual, like e.g. a software module acting as a connection broker, or physical like e.g. a dongle as further explained below.

[0010] One advantage of the invention is that a user may at least partially control the primary system via a companion system with which the user is familiar, allowing easier, faster, safer and more efficient control of the primary system. If the primary system is a conferencing system comprising the typical conferencing equipment, the companion system may be enabled to start a meeting, either planned or unplanned, configure and control a camera, a microphone, a speaker, adjust audio volume, mute / unmute the microphone, configure auto-framing and / or composition, control pan-tilt-zoom parameters, and so on.

[0011] Another advantage of the invention is that it allows to upgrade existing primary systems such as e.g. a conferencing system installed in a meeting room, with the new functionality of being able to be controlled by a companion system brought by a user. Hence, no entire new primary system has to be bought and installed, and resources can be saved.

[0012] Yet another advantage of the invention is that additional functions and new uses can be made possible by coupling a companion system with a primary system. While primary systems such as conferencing systems are generally equipped with all devices necessary to work without additional devices, additional functions can be useful or even necessary for certain applications. For example, a conferencing system will be equipped with several input devices, also called sensors, such as one or more video cameras and one or more microphones, and with several output devices, also called actuators, such as one or more displays and one or more audio speakers. A companion system may be equipped or connected to a special sensor such as e.g. an electronic microscope, and pictures from that microscope may then easily be shared via the conferencing system.

[0013] The method according to the invention facilitates for example that when the primary system is a conferencing system and is set up according to the above defined "second approach", it can easily be controlled by users bringing their own devices, each device acting ascompanion system, so that the users do not have to know how to operate a special controller like a table top control, and such controller may be entirely superfluous, saving costs for acquisition, installation and maintenance.

[0014] Often the companion system is just a single user device such as a laptop or tablet computer and the primary system is a conferencing system. It should be appreciated that this is purely exemplary and that, even though components and modules (such as the display) are attributed to a single device in this example, said systems and / or devices may consist out of multiple devices and said functional modules may be differently distributed across the available devices. For example, said systems and / or devices may have additional camera devices, microphone devices, speaker devices, display devices and companion devices to enable said functionality while said user device may also have additional devices coupled to it. Furthermore, there may be more than one of said user device that is used in collaboration with said system and there may be more than one of said primary systems that is used in collaboration with said user device.

[0015] In a preferred embodiment the method comprises a step of checking an identification of the companion system prior to giving a user of such system access to at least partially control the primary software, which will in most cases be a conferencing software, and giving said access depending on the identification. Such identification check can be done in multiple ways well known in the art, like for example reading an IMEI number and checking, if this number appears in a look-up table of known companion systems. An identification check provides advantageously that only certain authorized users, for example employees of a company, can gain access to the primary system. This may also be used to grant users different types of access from their user device acting as companion systems to the primary system similar to admin and "standard" user rights in computer systems.

[0016] In a further preferred embodiment of the method, the user interface is generated by the primary system and send to the companion system for being displayed. Alternatively, the user interface may be generated by the companion system, or, if a separate component, such as in particular a dongle connected to the companion system, is used, by the separate component. This advantageously allows the person skilled in the art to choose the best suited embodiment for a specific application. Note that the term "dongle" is used herein to denote a comparatively small piece of computer hardware that connects to a port of the companion system to provide it with additional functionality, or enable a pass-through to a device that adds functionality.

[0017] In a further preferred embodiment of the method, generating the user interface is triggered by one of the following: automatically detecting the presence of the companion system in the proximity of the primary system, active signalling the presence of the companion system in the proximity of the primary system, coupling the user companion system with a dongle, activating a signal from a dongle after coupling said dongle to the companion system, signing into a conference hosted by the primary system, signing into a conference hosted by the companion system, signing into a room in which the primary system is installed, active requesting remote access to the primary system via the companion system, detecting a scheduled meeting in the primary system and / or the companion system.

[0018] Note that similarly removal or hiding of the user interface could be triggered by the opposite of the above listed events, i.e. detecting that a companion device going out of proximity to a primary system, removing a dongle, signing out of a conference or room, explicitly logging out, timing out may lead to automatic removal or hiding of the user interface.

[0019] Depending on the actual set-up of the primary system, the environment, in which it is used (e.g. company employees only or with guests) etc., the person skilled in the art can advantageously choose the best suited solution. In certain environments, such as a small meeting room, it may be very easy to automatically detect the presence of a user device acting as companion system in the proximity of the primary system e.g. using certain signals that are regularly sent from or to the user device anyway. Also, a special beacon sending out e.g. ultrasound signals that are picked up by the user device, which in turn starts communicating with the primary system, may be provided. The user may activate signalling the presence of the user device in the proximity of the primary system e.g. via a certain app on the user device. If a dongle is used, the mere act of coupling the user device to it may trigger the next steps and in particular generating a user interface. Alternatively, the dongle may be provided with a button that activates, when pressed, a signal to start generating the user interface. Coupling the user device with the dongle may be done in a wire-bound or wireless manner.

[0020] Typically, the primary system is a conferencing system. If a user uses a user device, which is as companion system in the sense of the invention, for signing into a (running)conference hosted by the conferencing system, this may also trigger the interface generation. If a meeting room has a reader for reading codes like a so-called matrix or 2D barcode such as a QR Code®, the user may generate such code on his device and it may be foreseen that this code is read by the reader in the meeting room when the user enters or wants to enter the room, which then triggers generating the interface on the user device. Likewise, if the room is locked and is openable via some app on the user device, opening the room may trigger the interface generation. In certain cases, it may also be useful to generate the user interface when a user actively requests remote access to the conferencing system via the user device, for example when the user has guests that need to use the conferencing system, while the user for some reason cannot accompany them.

[0021] In a further preferred embodiment of the method, the companion system is automatically signed into a primary software such as a running conference hosted by the primary system, which provides a particularly advantageous and time efficient way when a user wants to join the running conference. Likewise, it may be foreseen that a (running) conference meeting hosted by the companion system is automatically transferred to the primary system. Sometimes a user starts a conference on an own user device and notices that using a conferencing system might be necessary or advantageous for some reason. The user may then take the user device and go to the meeting room without interrupting the conference, where the conference hosting is transferred automatically to the conferencing system (which is a primary system in the sense of the invention). While it is obvious for a person skilled in the art, it is nevertheless noted that the term "conference" is used herein to denote all sizes of meetings independently of the number of persons participating, and a video call between two persons is a conference in this sense. Likewise, the terms "meeting room" and "meeting area" are interchangeable and not limited in size; a large conference hall is a meeting room in this sense.

[0022] In a further preferred embodiment of the method, the user interface is initially displayed on the companion system only for a predefined time period and afterwards only upon certain actions. This is particularly useful when the companion system is a user device such as a smart phone having only a relatively small display. The entire display or parts of it may be used to initially display the user interface and once the primary system, in particular a conferencing system is set up, the user interface may disappear until its displaying is requested by a certain action. If a dongle is used, said action may be pressing a button on the dongle. In general, the display size and type may be automatically detected and taken into account when generating the user interface. Also, it may be foreseen that when the primary software is a runningconference, the participants, shared content etc., is only displayed on displays of the conferencing system while the user interface is only displayed on the user device.

[0023] In a further preferred embodiment of the method, at least one further companion system may be connected to the primary system and the user interface may be displayed on only one of the companion systems connected to the primary system in accordance with predefined rules. Typically, more than one person attends a video conference in a meeting room, and each person may have brought their own user devices. If all persons are in principle allowed to gain control via their user devices, all user devices may be connected to the conferencing system while predefined rules may provide that the user interface for controlling the conferencing system is only displayed on the user device that was the first to connect with the conferencing system. It can also be foreseen that certain user devices are given priority over others, or it may be foreseen that whoever takes a certain action like e.g. pressing a button on a dongle connected to the respective user device, is given access to the control of the conferencing system.

[0024] In a further preferred embodiment of the method, it may be foreseen that when different companion systems try to connect to a primary system hosting a conference, and access to control the primary system is denied to a companion system based on its identification, this companion system is signed into the conference hosted by the primary system via a secure channel. This is particularly advantageous when company interns and externs attend in person a video conference in a meeting room of the company. The employees of the company may gain access to and control of the conferencing system and may use an internal network for communication with the system, while externs may participate in the conference using their own devices via a separate network, e.g. the internet, without being connected to the internal network.

[0025] In a further preferred embodiment of the method in which the companion system is connected to a dongle, the dongle may comprise executable software code connecting the companion system device with the primary system. Thus, the connection to the primary system can be established in a very simple manner ("plug & play"). It may even be foreseen that the software code is executed by the dongle when the companion system is connected with the dongle.

[0026] In a further preferred embodiment of the method, the user interface is generated according to preferences of the user of the companion system. This can be achieved in multiple ways known to a person skilled in the art, for example by identifying the companion system and obtaining information on certain preferences via a database. Likewise, such preferences may be stored in the companion system itself. Preferences may include the general layout of the interface like e.g. the size and the arrangement of certain virtual control buttons like buttons for increasing / decreasing the volume of a speaker, muting / unmuting a microphone. Automatically adapting the generated user interface to the user's preferences does not only lead to a better user experience, it also make the work more efficient as the user will be interacting with an interface known to her or him.

[0027] Further details and advantages will become apparent from the following detailed description of non-limiting examples of the invention and its parts in conjunction with the drawings. While the examples will primarily use a conferencing system as an example of a primary system and a user device as an example for a companion system, a person skilled in the art understands that the invention can be used with other primary and companion systems such as e.g. controlling certain equipment and machines via a companion system.BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Fig. 1 shows schematically a primary system and a companion system.

[0029] Fig. 2a and 2b are functional diagrams of a companion system providing a user interface for a companion system.

[0030] Fig. 3 is a schematic diagram showing an example of presence detection.

[0031] Figs. 4a, 4b and 4c are schematic diagrams showing an example where a dongle is used in conjunction with a companion system.

[0032] Fig. 5 is a schematic diagram showing a combination of a companion system and a dongle.

[0033] Figs. 6a, 6b and 6c are schematic diagrams showing examples of a companion system for sharing content.

[0034] Fig. 7 is a schematic diagram showing an example where a companion system is coupled to a dongle.

[0035] Fig. 8 is a schematic diagram showing an example of a companion system for sharing content without a dongle.

[0036] Figs. 9a and 9b are a schematic diagrams illustrating a synchronization process.

[0037] Fig. 10 is a schematic diagram illustrating how a peripheral function of a primary system is extended through a companion system.

[0038] Figs. 11a and 11b are a schematic diagrams showing two variants of a companion system for companion displays.

[0039] Figs. 12a and 12b are schematic diagrams showing a conventional conferencing system and an example of providing a controller device functionality through a companion system.

[0040] Fig. 13 is a schematic diagram for explaining functionalities added by a peripheral device connected to a user device.DETAILED DESCRIPTION

[0041] In the following drawings, primary systems 10 and companion systems 12 are very schematically shown, typically by boxes with solid or dashed lines. However, each of the systems may comprises different spaced apart devices, such as a camera, microphone, speakers. In the simplest embodiment, the companion system 12 may just be a single user device such as a tablet computer having different functionalities. The companion system may also comprise multiple devices. Thus, a companion system denotes 12 any device inside or outside a meeting area that may be used in relation to said primary system, which in a typical case will be a conferencing system. It should also be noted that the schematic diagrams are intended to show the functionalities rather than specific hardware to achieve them, so that for example the at least one input and output device including a display for interacting with a user and a connecting port for connecting the companion system to the primary system are not explicitly shown

[0042] Fig. 1 schematically shows a primary system 10 coupled to a companion system 12. The companion system 12 enables users to extend the functionalities of the primary system 10 and to interact with and control the primary system 10 in several ways.

[0043] The primary system 10 comprises a number of functionalities 14i, 142, 14N. These functionalities can be used ‘as-is’, meaning they are provided by said primary system 10 as a standalone functionality or they are provided by said primary system 10 as interface points with other systems. A combination of both is possible, where a functionality is a standalone functionality but can be impacted by an external system. The functionalities can have 14i , 142, ... , 14Ninterfaces 16i, 162, ... , 16N to monitor or impact said functionalities 14i, 142, ..., 14N. The interfaces 161 , 162, ... , 16N may be direct or indirect interfaces. Direct interfaces are designed to facilitate interoperability with other systems, for example an API (Application Programming Interface) that allows access to said functionality. Indirect interfaces are not necessarily designed to interface with, or require additional operations to facilitate said interfacing. In those cases, a means for interfacing may be applied to facilitate monitoring and / or impact of said feature in another way. Said means for interfacing is schematically indicated in Fig. 1 as a functionality interface proxy module 18, having functional "connectors", of which only one, 20N, is provided with a reference number, to one or more of the interfaces I61, 162, ... , 16N. Whether a functionality of said primary system 10 can be interfaced directly or indirectly via a functionality interface proxy module 18 depends on the primary system's 10 capabilities, the companion system capabilities 12, implementation, context, conditions, learned knowledge or preference. Note that throughout this description and the claims the term "module" is also used in a functional way, and a module may be hardware or software or both.

[0044] The companion system 12 in this example contains an onboarding module 22 for onboarding devices and systems which will take part in the companion system's 12 functionality. Onboarding may be static or it may be dynamic. Multiple devices and / or system may be onboarded and / or be active at the same time. An onboarded device and / or system may be used to facilitate the functions provided of the companion system. Some devices may, by default, be onboarded while others may be onboarded as the context, conditions, preferences or learned knowledge determines.

[0045] The companion system in this example is a user device 12 further contains a deployment module 24 to ensure that the functional components that are required for saidcompanion system 12, such as a functionality transformations provided by a functionality transformations module 26 and the functionality interface proxies provided by the functionality interface proxy module 18, are deployed to each onboarded device and the proper communication channels are setup. Said deployment may consist out of parts that are statically deployed and parts that are dynamically deployed, for example based on the actions done by the onboarding module 22. For example, one may have software statically deployed on a user device 12 that is capable of being onboarded, however said software may only be coupled to other components when said onboarding process has successfully occurred and maintained. A functionality transformation in said companion system is a monitoring and / or adaptation of a functionality in said primary system, which is schematically indicated by the "connectors", of which only one, 28i, is provided with a reference number, of the functionality transformations module 26 to one or more of the interfaces 16i , 162, ... , 16N. The functionality transformation is typically the goal of said companion system 12: by adding companion real- or virtual devices and / or systems, one may monitor, control and / or adapt the behaviour of the primary system 10.

[0046] To access controller functionalities on the companion system 12, a user interface that reflects user access to said controller functionalities is made available on companion system 12. The user interface may resemble the user interface that is typically shown on a controller for a primary system 10, but may have modifications or even a different look, and may use different modalities (audio / video / haptic / ...) or have modifications that enable a more natural interface to a user of the companion system 12. For example, the user interface may be adapted to match the style of interfaces in the companion device 10, and may have adaptations to the user preferences or user restrictions, improving both usability and accessibility. The said user interface may use speech detection and / or gestures to access said functionalities.

[0047] Access of the user interface may be triggered use a detection means and a trigger module. Fig. 2a shows schematically a companion system 12 comprising a detection means 30, a trigger module 32 and a user interface 34 to control the primary system 10. Being a functional diagram, said example does not convey information regarding the mapping of said components and modules to physical or virtual devices. In use, the detection means 30 detects whether said user interface 34 would be useful for a user. For example, the user interface 34 may only be useful for a user who is in the meeting area or may only be useful when provided on a device that is accessible by the user. In addition, the intent of said user may be taken into account in order to estimate whether the user interface 34 needs to be presented to said user or to said device. Once said detection is made, said at least one real- or virtual device is triggered toprovide said interface. As mentioned, said interface 34 may be adapted depending on the user, the device, the context, the conditions, the user role, learned knowledge and so on. Said adaptation may be done by said companion system 12. Said interface 34 may be directly presented, or there may be additional steps in between before said interface may be accessed. For example, there may be a notification or message that signals the user that said interface 34 is available and said interface may only be presented when said user indicates that this is what said user wants. The real- or virtual device which presents said user interface 34 is not limited to user devices. Said devices may be devices that are part of the meeting space, these may be virtual devices that can be accessed via a multitude of digital means or they may be generic devices. Nevertheless, as the user interface 34 belongs functionally to the companion system's side, it is shown as part of the companion system 12 in Fig. 2a.

[0048] As schematically shown in Fig. 2b, the communication between the user interface 34 and the primary system 12 may have a controller proxy module 36 in between that provides data translation and transformation capabilities between the user interface 34 and the primary system 10. Such controller proxy module 36 is typically needed when the primary system 10 does not provide sufficient interface capabilities for the user interface 34 to interact with it directly. A user may, for example, trigger said user interface 34 by doing one or more actions. As such, one example of a means for detection relates to detecting said one or more actions. For example, the user may press a physical or a virtual button on the companion device 12 or another coupled device or said user may use a voice command to access said user interface 34.

[0049] As shown in Fig. 3, a controller proxy module 36 may also be foreseen on the primary system's 10 side. In the example of Fig. 3, the primary system 10 further comprises a presence beacon 38. When a user enters a certain area, typically a meeting room, where the primary system 10 is installed, the detection means 30 may interact with the presence beacon 38 and trigger via the trigger module 32 the start of the user interface 34. One example of such presence beacon 38 is an ultrasound beacon that transmits coded information. The detection means 30 may be adapted to detect said ultrasound signal. The detection means 30 may also use indoor localization technologies, possibly relative to the position of the primary system 10. Different detection techniques may be used to trigger start of the user interface 34, for example radio signal fingerprints. Also, sensors such as microphones or cameras, may be used to understand who is in a certain area, e.g. a meeting room, and then contact the companion system 10. To avoid that companion systems are triggered accidentally, triggering maycomprise checking data intrinsic or extrinsic to a companion system. For example, it may be foreseen to ignore companion systems for which data, such as gyro data, indicate that they have been static for a while and were just brought to the meeting room by users without the intend to use them.

[0050] When a user enters a meeting room with a laptop acting as companion system 12, the companion system 12 may detect said entering action via the detection means 30 and directly provide an interface 34 on said laptop screen or provide data towards a process that decides when to provide said interface 34 to said user. Multiple devices may get a trigger to provide an interface. Each of said devices may interpret and implement said providing of an interface in a different way. For example, a laptop may have its operating system trigger a popup that can be clicked to open said interface or open said interface by default while a mobile device may in addition add a vibration to indicate to the user that said detection has happened. It is appreciated that there are many ways of alerting a user and / or presenting an interface to said user.

[0051] The detection means 30 may use techniques for estimating the intent of a user. For example, when a user intends to use a conferencing system, typical behavior may involve entering a meeting room where said conferencing system is deployed and sitting down. The agenda of said user may have a meeting planned at the time (or close to that time) which also provides a cue towards said intent estimation. Many other cues are possible, and all can be combined into generating an estimation of the user intent. Said estimation may then be used as a detection means 30 to determine whether said user interface 34 is made available to said user. It is appreciated that, for providing such an estimation, many types of detections, data, context and learned knowledge may be used and that the aforementioned is merely a nonlimiting example of such an estimation process.

[0052] The presence beacon 38 in Fig. 3 is used together with the detection means 30 to determine proximity, room presence or other spatial relations between the primary system 10, for example a conferencing system, and a companion system, which in this case is a user device 12. As described before, there are many possibilities for enabling said presence detection such as ultrasound, wireless signal fingerprinting, sensing using cameras and / or microphones, identity related data, and so on. The presence beacon 38 and the detection means 30 are merely illustrative of said presence detection functionality and may be replaced by other means for presence detection without changing the presented principles and aspects.Said presence detection is used as a means for detection which will activate the trigger module 32 when such detection has been made. Said trigger module 32 will then, in this example, trigger the user interface 34 to be shown on said user device 12. Note that triggering may include an indirection via, for example, a notification that may require an extra action to be activated before said user interface 34 is shown. Once said user interface 34 is shown, interacted with it is enabled, which may include sending control commands to the primary system 10 or observing its state. The user interface 34 may have other functionalities to observe or control other systems as well. The controller proxy module 36, which is optional, functions as described before and facilitates interactions between the user interface 34 and the primary system 10.

[0053] When a user interacts with a primary system 10, a coupling device such as dongle 40 may be useful to facilitate or improve the interaction. Figures 4a, 4b and 4c illustrate examples where a dongle 40 is used as, among other functions, a means for detection.

[0054] In Fig. 4a, coupling a dongle 40 to the companion system may be detected by the detection means 30, upon which the trigger module 32 triggers the start of the user interface 34. Alternatively, dongle 40 detection may depend on one or more conditions. For example, the detection means 30 may only send a trigger when said dongle 40 is coupled and said dongle 40 was able to communicate with the primary system 10, in particular the controller proxy module 36. When a trigger is generated, the companion system 12 may directly show the user interface 34. Alternatively, but there may also be an additional process in between that will only show said user interface 34 upon a certain condition.

[0055] As illustrated in Fig. 4b, it may also be foreseen that the dongle 40 handles the communication between the companion system 12, or more precisely the user interface 34, and the primary system 10, in particular the controller proxy module 36. By doing so, one does not require network connectivity between the companion system 12 and the primary system 10 and / or the controller proxy module 36, but instead the dongle's 40 connectivity can take care of this.

[0056] As illustrated in Fig. 4c, the functionalities of dongle detection 42, triggering 44 and user interface 34 also be executed within the dongle 40. In this case, no logic is needed on the companion system 12, and the dongle 40 detects, for example, that it is coupled to a companion system such as a user's laptop, where upon the dongle 40 may trigger itself to enable oractivate a user interface 34 for controlling the primary system 10 at least in part. The user interface 34 may be an interface where physical buttons or sensors are linked to actions and the dongle 40 may have actuators, such as lights, to indicate the status information within user interface 34. For example, one may have a dongle 40 with an actuator that, when coupled to a companion system 12, will have an action that is sent to said primary system 10 (directly or indirectly) when pressed or actuated. Said action may be context dependent. For example, when said primary system 10 is a conferencing system, a first press may start a conferencing session. During said conferencing session, pressing or actuating said actuator may, for example, mute or unmute said call. Another button or actuator may, for example, be used to close said conferencing session. It is appreciated that this is only an example and that a myriad of possibilities for providing an interface via a dongle 40. Multiple buttons may be available, each with different functionalities, a touch sensor may be available for more elaborate user actions, means for actuation may be present to indicate state within said user interface 34, a display may be available on said dongle 40 to accompany said interface with detailed visual feedback, and so on. The dongle 40 may also receive these control actions via the companion system 12. This can, for example, happen through USB HID or other standard USB device functionalities. For some user devices acting as companion systems 12, this may require support for USB dual-role capabilities where said USB port can switch between the host and the device role. This feature is sometimes also denoted DRD - Dual Role Device - or OTG - On The Go.

[0057] The companion system 12 can then expose some of its functionalities to the dongle 12 without requiring specific software / drivers / etc. on said companion system. For example, said dongle 40 may capture volume control commands of said user device 12 and use these to control said primary system 10 via the standard volume control commands of USB HID. Users can then simply press their volume up / down buttons on the keyboard or use other means available on the companion system 12 to control said primary system 10. The same principle can be used to expose other controls, expose a visual user interface, expose touch commands, provide audio feedback, etc.

[0058] In Fig. 4a, the user interface 34 communicates with the controller proxy module 36 and / or the primary system 10 via the companion system 12. Conversely, in Figures 4b and 4c, said communication happens via the dongle 40. These options are merely illustrative and said communication paths may be interchanged and / or be a combination of both. In addition, saidcommunication path does not necessarily imply direct communication; there may be modules and devices in between.

[0059] It should be appreciated that the device layout in all figures is purely exemplary and that, even though components and modules may be attributed to a single system in this example, a primary system such as a conferencing system and / or a companion system such as a user device and / or dongle 40 may consist out of multiple devices and said functional modules may be differently distributed across the available devices. For example, a conferencing system may have additional camera devices, microphone devices, speaker devices, display devices and devices to enable the functionalities while a user device may also have additional devices coupled to it. Furthermore, there may be more than one user device that is used in collaboration with a conferencing system and there may be more than one of said conferencing systems that are used in collaboration with a user device. There may be more than one of said dongles 40 that are used in collaboration with a conferencing system and / or a user device.

[0060] A dongle 40 may be coupled to a companion system such as a user device 12, which may be used to, directly or indirectly, to enable or improve the interaction between the user and / or the user device with a conferencing system. The dongle 40 may provide a wireless connection, for example Wi-Fi, 5G, Bluetooth, ultrasound, infrared, RFID, or any other wireless connection means to the conferencing system, to a device that is coupled to the conferencing system or to a module that is integrated in the conferencing system, and the dongle 40 may also be coupled wirelessly to the companion system. Said dongle 40 may provide a wired connection, for example USB, USB-A, USB-C, USB-C DP alt mode, HDMI, DisplayPort, DisplayLink, Thunderbolt, DVI, VGA, Ethernet, or any other means for coupling or any other wired connection means to said conferencing system, to a device that is coupled to said conferencing system or to a module that is integrated in said conferencing system, and may also be connected to a companion device via such wired connection.

[0061] The dongle 40 may enable a user to share content from a user device 12 acting as companion system, to which said dongle 40 is connected. The dongle 40 may provide access to functionalities of a conferencing system such as access to a (virtual) camera, (virtual) microphone, a (virtual) speaker, a (virtual) display, a conference call or a conference call identifier that is hosted by said conferencing system. Additionally or alternatively, the dongle 40 may provide access to functionalities of the user device to the conferencing system or other devices coupled to said user device, such as access to a (virtual) camera, (virtual) microphone,a (virtual) speaker, a (virtual) display, a conference call or a conference call identifier that is hosted by said user device.

[0062] The user interface 34 may include elements that are present on a typical controller device. For example, the user interface 34 may have an element to start a planned or unplanned conferencing session or control the audio and video signals that flow into or out of said conferencing system and so on. Next to said elements that are present on said typical controller devices, said user interface 34 may also include elements that are not present on said typical controller devices. Said user interface 34 may add elements that relate to said conferencing system, that relate to said companion system or that relate to any other system for which said user interface 34 may be useful. This furthermore exemplifies that said user interface 34 is not necessary a duplication or clone of said controller device interface but instead, as mentioned before, may have adaptations to said controller device interface such as adding or removing elements compared to said controller device interface or changing how the interface is to be presented or consumed by the user of said user interface 34, including modifying or adding to the modalities that are used by a classical controller device.

[0063] As used herein, a means for detection may consist out of multiple cues and inputs, said multiple cues being aggregated using, for example, predefined rules, adaptive rules or artificial intelligence. As such, a means for detection may leverage multiple cues in order to create a better performing means for detection.

[0064] When said means for detection has detected that a user interface 34 may be presented to at least one device comprised by the companion system, said companion system will trigger said at least one device to present said user interface 34. As mentioned, the companion system may itself be the device such as a user's laptop. A trigger mechanism may happen in different ways depending on the implementation, preferences, devices or conditions at hand. For example, on a mobile phone that is not actively being used, the trigger mechanism may include a haptic feedback that notifies said user that said user interface 34 is available. For example, a notification message may first be used on a device to indicate to the user that a user interface 34 is available. To access said user interface 34, said user may need to still interact with said notification message, for example by clicking or tapping on said message. Alternatively, said user interface 34 may be automatically shown or discarded based on timing, context, detections of behavior, intent or state, learned knowledge or other means for concluding that said user is presented with said user interface 34.

[0065] Determination of the triggers is part of a trigger mechanism. A trigger is a method to convey that a user interface 34 is available to be presented. There may be multiple triggers that are used for a single means for detection. Determination of the triggers will assess which trigger to use in what condition and may be influenced by preferences (user or other), context, conditions, learned knowledge, state and so on. For example, a certain trigger or multiple triggers may be used by the system when a user is first detect and / or identified, however said set of triggers may change as said companion system adapts to these various factors. If, for example, it is learned that a user never uses a trigger that is sent to its mobile but always uses a dongle 40 to interface with said companion system, one may refrain from using said mobile in future cases. If, for example, a user is already interacting with said system, either via said user interface 34 or via other means, one may prefer not to send said trigger to a newly entered user, or alter the set of triggers that is used. For example, when it is estimated that the need for accessing said user interface 34 is low, one may only use triggers that use more direct user actions or only create triggers for means for detection that are based on more direct user actions. If a user is already in a meeting area and is already coupled a primary system (for example said user was already presented with said user interface 34 or said user already interacted with said user interface 34), one may wish to increase the threshold with which one triggers another user in said meeting area. For example, one may not send a trigger to the mobile phone of another user but only trigger or present the user interface 34 when the other user makes a more explicit action. For example, when said user connects a dongle 40 to his computer, this can be seen as an explicit action to interact with the primary system (typically a conferencing system) and said companion system. The means for detection may be an explicit real or virtual button or sensor that needs actuation before triggering presentation of the user interface 34 may happen.

[0066] Said trigger mechanism may use more than one device to deliver said trigger. For example, a user may get a trigger on a mobile phone, on a laptop, on a meeting space display, on an audio speech interface, on a dongle 40, etc. As such, one trigger may be delivered via multiple means. The selection of said multiple means may be static or it may be dynamic based on preferences (user or other), conditions, actions, state, context, learned knowledge and so on.

[0067] Said trigger mechanism may use one or multiple devices to deliver said trigger while said user interface 34 may be provided by a different one or multiple devices. Said means for detection may be provided by yet another one or multiple device. The functionality of saidmeans for detection, said trigger mechanism and said presenting of said user interface 34 should thus be understood as functional components. Said functional components may be distributed freely across one or multiple devices.

[0068] While said trigger may impact the user experience as was discussed, said trigger may also be hidden from the user experience. For example, when a user plugs in a dongle 40 to the user device via, for example, LISB-C, software on said user device may detect said action and trigger a control interface to appear to said user, for example via said user device display. In that case, said trigger mechanism is not visible to the user and is simply responsible for linking said means for detection to the component that is responsible for presenting of said user interface 34 to said user. Similarly, said user interface 34 may be presented via said dongle 40 and in this case, said trigger is also hidden from the user experience. Nonetheless, said trigger mechanism is needed to determine which trigger to use and in which way to deliver said triggers.

[0069] Said trigger mechanisms may leverage capabilities and components that are available to the various devices to integrate in said device typical flow. For example, the technology to deliver notification may be leveraged as (part of) a means for triggering said device. So where said trigger mechanism is part of said companion system, this does not mean that all components that enable said trigger mechanism require distinct implementations or solutions and existing solutions may be leveraged to implement said trigger mechanism. External systems may be used and integrated in said companion system.

[0070] Interactions with said user interface 34 may be communicated to the respective systems to which said interactions have an impact. For example, when a user interface 34 interaction represents a ‘join meeting’ operation, said operation is communicated to said conferencing system, either directly or indirectly. For example, when a user interface 34 interaction represents a ‘zoom into the person speaking’ operation, said operation is communicated to said conferencing system and / or said camera device that needs to execute said operation. While some operations of said user interface 34 may be directly translated into an operation of a respective system, this may not always be the case. Some operations may require translations and transformation in order to send them to said respective system and an operation may be sent to multiple respective systems.

[0071] Said user interface 34 also includes interactions from said respective systems to said user. For example, status information may be communicated to said user via said user interface 34 or additional user guidance may be provided via said user interface 34. Said user interface 34 may even be integrated in the functionality of said respective systems such as said conferencing system. For example, one may use said user interface 34 to show information that is part of a conferencing session, such as chat messages, shared content or voice data.

[0072] Said user interface 34 may be a visual interface. Said visual user interface 34 may be rendered on top of other windows to ensure the user sees and has instant access to said visual user interface 34. Said visual user interface 34 may be covering part of the display device on which it is rendered or said visual user interface 34 may be covering the complete display area. Said visual user interface 34 may be hidden until a further action is done or a certain condition is met. For example, there may first be a message that is shown to the user that said interface is available, after which, either automatically or manually, said interface is made available to said user. Said visual user interface 34 may be adapted to adapt to specific conditions at hand. For example, when there is not a lot of space to render said visual user interface 34, one may adapt said interface to only hold part of said user interface 34 at a given time. Said visual user interface 34 may have a hierarchical structure. Said visual user interface 34 may hide certain aspects based on said hierarchical structure and may require additional actions to navigate through said visual user interface 34. Said user interface 34 may dynamically adapt based on preferences, conditions, learned knowledge, context and so on. For example, said interface may hide elements that are not relevant for a given person or a given context. For example, said interface may adapt based on learned knowledge and optimize accordingly.

[0073] Said user interface 34 may be used by a specific user. Said user interface 34 may also be used by different users. Said user interface 34 may be adapted depending on said target audience. For example, if a user interface 34 is targeted to be used by a specific user, said user preferences and learned knowledge related to said user may be used to optimize said interface. If a user interface 34, on the other hand, is targeted to be used by multiple users, said user interface 34 may need to be more generic and cater to multiple preferences at once. A combination of interfaces may be presented where some are targeted to individuals while other may be targeted at groups of individuals or be generic.

[0074] Said user interface 34 is not limited to a visual interface. Said user interface 34 may, for example, use real- or virtual buttons or sensors to present said user interface 34. Said real- orvirtual buttons may have appearance changes to reflect the functionality that these represent at a given time. For example, when using a dongle 40 that is coupled to a user laptop, one may map one or multiple buttons on said dongle 40 to certain functionalities that are exposed by said user interface 34. Said buttons may provide different functionalities at different points in time due to the state of one of the systems, for example the conferencing system, the preferences of the user, the conditions, the learned knowledge and so on. For example, a button on said dongle 40 may be used to connect to start said meeting when no meeting is active but may be used later on, when said meeting is active, to share content from said user laptop to said conferencing system. Said content may be explicitly selected by said user, said content may be implicitly selected by said user, for example by sharing a complete screen or said content may be selected implicitly by sharing the currently active content, window or application. Said user interface 34 may use various modalities to present said one or multiple user interfaces 34. Speech may for example be used to provide a user interface 34 where a user may use speech to interact with said user interface 34 and speech or other audio cues may be used by said user interface 34 to provide status updates, messages, feedback and so on. In general, a user interface 34 may use any combination of modalities to present its interface. Multiple user interfaces 34 may be active at the same time. It should be noted that in general a controller proxy module 36 may be optional depending on the capabilities, requirements, and preferences of said primary system. When no controller proxy module 36 is foreseen, the user interfaces 34 may directly communicate with a primary system.

[0075] As illustrated in Fig. 5, a combination of a user device 12 acting as companion system and a dongle 40 may be used to convey a user interface 34. Some of the logic, i.e. software realizing the functionalities represented by the modules, may run on the user device 12, some of the logic may run on the dongle 40 and yet more logic may run on yet another device. For example, when coupling the dongle 40 to the user device, for example via USB, a dongle 40- centric experience may be offered for which said user device does not require logic to be run. If, however, said user device is running some logic, for example due to preinstalled software or software that is activated and / or run after coupling said device, said user interface 34 may consist out of a combination of said user device and said dongle 40. In Fig. 5, the data paths are merely illustrative and these data paths may, for example, flow between user device and dongle 40 on the one hand and dongle 40 and controller proxy module 36 on the other hand or flow between the controller proxy module 30 and the user device on the one hand (not shown) and the user device and the dongle 40 on the other hand. Yet alternatively, hybrid communication paths are possible. In a further example, independent of the options shown in Figs. 2 to 5, saiddongle 40 may have a button that may be pressed or sensor that may be actuated in order to trigger said user interface 34 directly. A user interface 34 may also be triggered automatically or based on a number of conditions.

[0076] The communication between said the interface 34 and the controller proxy module 36 or, more generally, the primary system 10 as shown in Figs. 2 to 5 may undergo additional translations and / or transformations. Said additional translations and / or transformations are not shown on the figures. Said additional translations and / or transformations may, for example, happen in the dongle 40, the user device 12, the controller proxy module 36, the primary system 10 or any other device or system or device. Multiple such translations and / or transformations may happen. For example, the user device 12 may transform the user interface 34 inputs and / or outputs into a format that may be sent to the dongle 40, for example to prepare said data to be sent via a USB endpoint of said dongle 40. The dongle 40 may translate and / or transform the data to pass it over, for example, a wireless network, to the primary system's 10 side.

[0077] Figs. 6a to 6c illustrate some examples of a companion system for sharing content from a user device 12. In Fig. 6a, the user device 12 has a (software) module 30 to detect coupling of a dongle 40 to it. When coupled, a trigger generated by a trigger module 32 may be sent to a sharing module 46 to start sharing. Said trigger may have conditions that need to be fulfilled before said sharing can happen. For example, a condition may be the requirement of a user action, for example an action executed on said dongle 40. For example, said user may be required to press a button or actuate a sensor on said user device 12. For example, said user may be required to use a user interface 34 shown on said user device to indicate the user wants to share. Said trigger may involve a choice for the user to select what content should be shared. Said content may, for example, be files, streams, current displayed content, current open application windows, audio from said user device, purposefully rendered content towards said sharing function, etc.

[0078] When said content is selected, or a default selection is done, the sharing module 46 will capture and share said content towards a sharing proxy module 48 (which as all modules may be a software module) the primary system 10. Depending on the functionalities of the primary system 10, such sharing proxy module 48 may not be necessary. Fig. 6b shows a variant of said system where shared data is communicated via a dongle 40 connected to a user device 12 to a sharing proxy module 48. By routing sharing data via said dongle 40, said user device 12 does not require network connectivity with said sharing proxy module 48 and / or said primarysystem 10. Fig. 6c shows yet another variant of said system where a companion system formed by a user device 12 does not require specific software. Instead, the sharing module 46 is realized in a dongle 40. Coupling the user device 12 to a dongle 40 includes a content channel, for example by means of LISB-C DP alt mode, HDMI, DisplayPort, DisplayLink, Thunderbolt. Said content channel is used by said dongle 40 as the content that is shared. Said content may be transformed or translated by said dongle 40. For example, certain portions of said content may be extracted, certain portions may be changed, and so on. When said dongle 40 is detected to be coupled by a dongle detection module 42, a trigger may be sent. Said trigger may be automatic; i.e. no actions or conditions are required. Said trigger may require further user action, for example a press on a button on said dongle 40 or an actuation of a sensor on said dongle 40. Said trigger may require further conditions that need to be met before the sharing module is activated. Once said trigger is sent to a trigger module 44, said sharing module communicates said content, which may be potentially transformed, to a sharing proxy 48 and / or the primary system 10. Yet another variant is possible where a hybrid approach is used. Fig. 7 illustrates one such example.

[0079] As shown in Fig. 7, a user device 12 is coupled to a dongle 40 via a means for coupling that, similar to Fig. 6c, has a content channel (for example using LISB-C DP alt mode, HDMI, DisplayPort, DisplayLink, Thunderbolt). In this example however, said user device 12 is also executing software, schematically depicted by a sharing module 46, that enables selection of content. Selected content is then positioned on said content channel and / or said selection is signalled to said dongle 40 via a metadata channel between said user device and said dongle 40. For example, extra data may be signalled to indicate that not all data on said content channel should be shared. For example, said sharing module 46 on said user device may position shared content on said content channel. Said dongle 40 may then transform said content depending on said metadata (for example cut out the data that needs to be shared) and / or may transform said content to enable transmission via its own sharing module 46 towards a sharing proxy module 48 and / or the primary system 10. Said metadata may also be sent towards said sharing proxy module 48 and / or primary system 10. In that case, said sharing proxy module 48 and / or primary system 10 is responsible for transforming said content according to said metadata. Note that in case a sharing module 46 is provided, this may include all functionalities of the user interface. This is true for all embodiments described herein.

[0080] Fig. 8 illustrates an example of a companion system comprised by a user device 12 for sharing content without a dongle. In this example, a presence detection system comprised by adetection module 30 and a presence beacon 38 is used as a means for detection that will activate a trigger module 32 when the user device 12 and / or a user satisfies spatial conditions determined by said means for detection, for example when the user enters a meeting area. When the trigger module 32 is activated, one or more triggers may be sent to a sharing module 46. Said triggers may directly activate said sharing module 46, or said triggers may have conditions associated with activating said trigger module. For example, the user may be required to activate said sharing and / or the user may be required to indicate what content needs to be shared. Aspects related to Fig. 6 and 7 and their descriptions are referred to and are applicable in this example as well. As noted before in context with Fig. 3, said presence beacon 38 and detection module 30 as shown in Fig. 8 are merely illustrative of said presence detection functionality and may be replaced by other means for presence detection without changing the presented principles and aspects.

[0081] There may be multiple devices within said meeting area that log into a conferencing session. For example, a user in said meeting area may log into said conference session from its user device in addition to the login of said conferencing system in said meeting space. When said user is using said real or virtual dongle to share content into said conference session, said companion system may share said content to said conference session login of said user device instead of the conference session login of said conferencing system. Conversely, in some situations, said companion system may still share said content to said login of said conferencing system. The decision on where to share which data may be static, for example decided at implementation time, or it may be dynamic based on the conditions at hand, the user preferences, the context, the learned knowledge and so on. For example, some conferencing systems may allow multiple logins to, for example, share content at the same time while other systems may only allow a single shared content for all users in its conferencing session. Such factors may also be taken into account to determine how said shared content should be routed via said companion system.

[0082] A real or virtual dongle may be used for other purposes than sharing content as well. For example, by connecting a dongle one may synchronize conferencing sessions across said device and said conferencing system, or across multiple devices. When a conference session is active for example on said conferencing system in said meeting space, and a user couples said dongle to its user device, said action may check whether said user device may join the same conferencing session as the one joined by said conferencing system. This way, said user may be directly onboarded to said conferencing session via said user device, enabling said user todirectly interact with remote participants, get an own representation within said conferencing session with possibly its own video feed and make use of said user device in an effective manner. Conversely, when said user device is already in a conferencing session before said coupling to said dongle is done, one may also use said dongle to, when coupling said dongle to said user device, automatically connect and log in said conferencing system to the same session as the session that is active on said user device. By doing so, said dongle enables synchronization between multiple devices that are capable of logging into / joining a conferencing session. As noted, said synchronization of said conferencing session may happen in both directions; from said user device to said conferencing system or vice versa. Said synchronization may also happen between different devices where there is no explicit conferencing system device as such. As noted before, said functionality may also be provided based on a virtual dongle, which, for example, triggers said functionality when said user device is detected to be in a certain proximity of said meeting space, said conferencing system and / or said other devices. Figures 9(a) and 9(b) illustrate two examples of such synchronization.

[0083] In the examples of Figures 9(a) and 9(b), a dongle 40 is connected to a user deice 12, which triggers said synchronization capabilities. In Fig. 9(a), a primary system 10 is logged into a conferencing session 50. A device, for example the user device 12, is capable of joining a conferencing session as well. When coupling said dongle 40 to said user device 12, said coupling is detected and a trigger is sent. Said trigger may automatically involve said user device 12 to also join said conferencing session 50 or said trigger may be conditioned, for example first triggering the user to ask for permission to join to said conferencing session. When granted, said user device 12 joins the same conferencing session 50 as said primary system 10, ensuring a consistent and reliable conferencing experience.

[0084] In Fig. 9(b) a variant is shown where a user device 12 is logged in to a conferencing session 50, and by coupling a dongle 40 to said user device 12, said primary system 10 is automatically or via additional conditions synchronized and joins said conferencing session 50 as well.

[0085] Note that joining a session may have a different impact and / or experience depending on where said session is joined. For example, said joining of a session in said primary system may involve a complete audio- and video communication session whereas joining said session from said device, when may involve a different way of joining said session. For example, said sessions may be optimized to create a complementary experience where one system, forexample said primary system, takes care of the audio- and video experience whereas another system takes care of a parallel channel such as a chat channel. Said synchronization ensures that all modalities and signals relate to a consistent communication session.

[0086] For joining a session, meeting and / or user IDs may be exchanged between the primary system and the companion systems. This way, a companion system that is linked to a certain user ID - typically the user of a user device 12 - can exchange the ID info with the primary system 10 and said primary system 10 can then use this ID for personalize the experience of the primary system 10. For example, it can allow for the interface means of the primary system 10, for example a tablet on the table or a touch screen in the room, to be personalized and allow for the user to easily access their files for sharing without requiring interactions on the user device 12. Instead, the user device 12 is used to customize primary system 10 with additional interactivity means. When a user enters the meeting room (and / or when a dongle 40 is plugged in), this allows said user to directly access their files on the primary system 10 interface.

[0087] Said companion system may enhance said conferencing system by adding more devices and / or capabilities to said primary system. For example, said conferencing may be a so-called all-in-one device that has one or more cameras, one or more speakers, one or more microphones and / or contains compute capabilities to create its own login and representation within a conferencing session. Said all-in-one bar may be coupled to one or multiple display devices or be integrated with said one or more display devices. While such an all-in-one device is an effective way of equipping a meeting space with conferencing capabilities, such a device may also be limiting. For example, it may be difficult to add extra cameras, speakers, microphones or screens to such devices which makes it difficult to scale these up to more challenging spaces. Using said companion system however, one can couple multiple devices together and integrate them into a single consistent user experience. For example, when multiple devices are linked together via said companion system, the different components of each of said multiple devices may be virtualized to benefit from all of them. For example, one may create a virtual camera device that uses the camera information of the respective multiple devices to create a camera that provides the best video data depending on the conditions, preferences, context and learned knowledge at hand. For example, one may create a virtual microphone device that leverages said individual microphone and creates a virtual device that leverages said individual microphone signals for providing better audio pickup and audio quality. For example, one may create a virtual speaker device that enables the users to have better audio quality and one may create a virtual display device that enables content, either from saidconferencing system or from other sources to be adequately displayed. Using said companion system, one may onboard extra devices and said companion system will mediate said devices and said device signals towards said primary system.

[0088] Said multiple devices may be coupled in some manner, for example via network connectivity and may be linked via software for aid companion system to mediate said devices. Said multiple devices may be linked via one or multiple linking devices, said linking devices ensuring that said multiple devices may be mediated by said companion system into a consistent experience. Said linking device may, for example, be a dongle 40. Devices that are linked may be required to be coupled to one or multiple linking devices. Devices may be linked via a virtual linking device, for example implemented in software.

[0089] Said conferencing system is typically equipped with a camera or a camera system. Said system is typically designed to give a good overview of the users in said meeting space, but rarely gives a good view of all the individuals in said meeting space. By using said companion system, one may onboard additional devices and device capabilities to expands the functionality of said conferencing system. For example, by onboarding a user laptop to said companion system, one can integrate the camera of said user laptop into the camera feed that is provided by said conferencing system, effectively expanding said camera capabilities with the capabilities of said onboarded devices. For example, one may have camera devices that are onboarded via an onboarding dongle into said companion system, said onboarding dongle ensuring that said camera is integrated with said conferencing system and may be included in the video stream that is sent from said conferencing system to said active conferencing session.

[0090] Some conferencing systems provide local- or cloud based computations to split a single video image into multiple images that, for example, show individuals who are present in said meeting space. This way, said problem of said conferencing system camera only showing a good overview is being reduced. However, said split video images are still often not showing said individuals in an optimal manner. For example, said individual may not be looking at said conferencing system camera(s) and thus said split video image is not always a meaningful image. Using said companion system, one may onboard additional cameras in a modular manner and, for example, replace such split video image with a more suitable video image that is, for example, combine from the laptop camera of said individual that is represented in said split video image. Said system can dynamically select which source to use for said split video image based on preferences, context, conditions and learned knowledge.

[0091] The combination of said onboarded devices may result in multiple data streams. For example, one may have a video data stream that represents all users in a meeting space. One may also have a video data stream that represents a certain user in said meeting space or a group of users in said meeting space. One or multiple of these data streams may be sent to the conferencing system or another primary system, but they may also be sent other onboarded systems. For example, for the case where said conferencing system has logged in to a conferencing session together with one or more user devices that are also logged in to the same session, one may send the video data stream related to that user to the conferencing login of the user device rather than to the conferencing login of the conferencing system; conversely, one may send the video data stream related to the other users to the conferencing login of the conferencing system. This way, the user device represents its user while the conferencing system represents the rest of the users who did not login to said conferencing system, or who did not onboard their device to said companion system. One may send multiple streams to the same login if said conferencing system allows.

[0092] One may also use said companion system to reroute or adapt data streams between said conferencing system (or another primary system 10) and said onboarded devices. For example, for the case where said conferencing system has logged in to a conferencing session together with one or more user devices that are also logged in to the same conferencing session, one may send the microphone data from said conferencing system (and / or onboarded devices) to the conferencing system when someone is speaking who is not onboarded and / or logged in to their own conferencing login and, when a participant is talking who has an onboarded device and is logged in to the conference session, send said microphone data to said onboarded device. This way, when someone is talking in the meeting space, the representation of that person is either taken up by said conferencing system when there is no individual device available and the representation of said person is taken up by the individual device when that is the case. This way, said companion system enables users to add functionality and increase their representation in a very flexible manner while preserving the representation of those who did not couple to said companion system. Note that, instead of rerouting audio information, one may also transform said audio information to get the same effect, for example by making a zero-signal when said audio signal is supported to be routed away from a device.

[0093] While these examples have used cameras and microphones to illustrate said dynamic onboarding of devices towards said primary system, it is appreciated that the same can be done for other types of devices such as, for example, speakers, displays, haptic devices, wearables and so on.

[0094] Fig. 10 illustrates an example where a peripheral function of a primary system 10 is extended through a companion system comprised by two user devices 12, which thus act as "extension systems". Said peripheral function may be any function of said primary system 10 that may be extended. Examples of said peripheral functions may be a camera, a microphone, a speaker, a display, a haptic function, a sensing function, an actuation function, a state detection and / or adaptation device, a signal detection and / or adaptation function and so on. An extension system 12 may have a detection means 30 and a trigger module 32 that will onboard said extension system 12 to said companion system and said primary system 10 via a peripheral module 52 that acts as user interface and via a peripheral proxy 54, which may not be necessary if the primary system 10 is adapted to communicate with the peripheral modules 52 directly. It is appreciated that, while one primary system 10 and two extension systems have been depicted in Fig. 10, the system is not limited to these numbers and there may be one or multiple primary systems 10 and only one or more than the depicted two extension systems. Said components may also be mapped to devices without restriction. For example, a primary system 10 may be integrated in a device that also acts as an extension system, for example to other primary systems 10. Devices serving one ore multiple of said functionalities may also serve other functionalities. For example, a user device may serve as an extension device and / or primary system 10.

[0095] Said means for detection may be dynamic, for example by detecting when a user device enters a meeting area or when a user device is available for said extension capabilities. Said means for detection may also be static, for example by associating said extension systems with said companion system and / or primary system in a static manner, for example by registering said devices in a registry store or by linking devices together via an initial linking phase where said connectivity details are stored in the state of said devices and / or facilitating devices. Said means for detection may be facilitated by coupling to a device that provides said detection and / or linking. For example, a dongle may be used that has the ability to associate and / or link to other devices that need to be linked. Such dongle may be interoperable with other dongles, with other devices and with other software solutions that provide said means for detection and linking.

[0096] The trigger module 32 may be fully automated and, for example, conditioned only on said means for detection or said trigger module 32 may be conditioned on other aspects that need to be satisfied before said trigger will enable said peripheral module 52. For example, a user may be required to give consent for using a certain peripheral, for example when said peripheral is part of a user device 12 of said user. The peripheral proxy 54 and the one or multiple peripheral modules 52 communicate in order to provide a virtual peripheral that satisfies the functional requirements. Said communication may use a star architecture where data of said peripheral proxy 54 and / or said one or multiple peripheral modules 52 are gathered in a central location to provide said virtual peripheral signal. For example, said peripheral proxy 54 may function as said central role and may include processing to generate and consume said virtual peripheral signal. Said communication may use other architectures where said signal may be generated in a more distributed manner. For example, any one of said peripheral proxy 54 and / or one or multiple peripheral modules 52 may provide processing that facilitate said generation and / or consumption of said virtual peripheral signal.

[0097] Said peripheral function may include network connectivity. A companion system may expose said network connectivity to devices that are onboarded in order to facilitate a more streamlined behavior. For example, one may expose network connectivity from said primary system 10 and / or said peripheral proxy 54 to a peripheral module 52. Said network connectivity could, for example, be based on cellular technology (4G, 5G, 6G, ...), Wi-Fi technology, Ethernet, VLAN or other means for providing network connectivity. Said network connectivity may be configurable depending on the onboarded device. For example, in an enterprise context, one may have different users in said enterprise that have different rights. For example, a guest in said enterprise may not have access to enterprise-specific resources. In one example, a dongle is provided that enables users to take part in communication and / or conferencing sessions without requiring network access from said user device. By exposing said network connectivity through said companion system, one may, for example, reduce friction to collaborate in environments with hybrid rights and privileges. Said network connectivity may also be provided by said companion system towards said primary system as a fallback or installation means of connectivity, enabling easier deployment and maintenance of said primary system and / or improving reliability of said primary system.

[0098] The same type of onboarding may happen for displays provided by onboarded devices via said companion system. For example, when onboarding a laptop device to saidconferencing system via said companion system, one may, for example, automatically project (part of) the content that is being shown by said conferencing system on said onboarded device. For example, one may automatically show the content that is shared within the conference session on said onboarded device in addition to said content being shown in said meeting space via one or more displays coupled to said conferencing system or show the remote participants on said onboarded device. By doing so, one can provide facilitate a very accessible system where people who have trouble seeing said one or more displays that are coupled to said conferencing system and show said content on their onboarded device. Said content may be transformed and adapted before being rendered on said onboarded device to reflect preferences, context, conditions and learned knowledge. For example, one may adapt said content to facilitate for color blindness. Said onboarding may happen via software. Said onboarding may happen via an onboarding dongle, whereby coupling said dongle to said device would enable the described functionality, either automatically or after additional actions or conditions. Such onboarding dongle may be the same dongle as the dongle that was described before.

[0099] Fig. 11 illustrates two variants a companion system comprised by a user device 12 for companion displays. In Fig. 11a, a detection means 30 in conjunction with a presence beacon 38 is used to trigger onboarding a display of user device 12 via a display module 56 that communicates with a display proxy 58. In Fig. 11b, a dongle 40 is used to facilitate said onboarding. In both cases, the display module 56 acts as user interface. It is appreciated that many other means of onboarding are possible and that there are many other mappings to devices possible. Furthermore, there may be one or multiple primary systems and there may be one or multiple extra devices, such as one or more user devices.

[0100] When a companion display is onboarded, said display may be used to extend the functionality of the primary system 10 or be further integrated into said functionality of said primary system 10 by providing an adaptation to said primary system's functionality as well. For example, a user device 12 may be onboarded that has a display, and said display may show, by means of a display module 56 content that is shown by said primary system 10. Said content may also be transformed to further adapt to said user and / or said system behaviour. For example, said content may be transformed to only show shared content within a conferencing session, for example a presentation, and hide the videos of the remote participants on said user device. For example, said content on said primary system 10 may also be transformed due to said onboarding of said devices, whereby, for example, only remote participants are now shownon said primary system's display(s) and said content is, for example, only shown on said onboarded device display(s). These should be strictly seen as examples and not limit other variants that may be enabled by said system. Said different modules such as display proxy and display modules may have processing to modify the respective signals. Said different modules may communicate among each other for optimizing said functionality.

[0101] Fig, 12(a) shows a conventional conferencing system 60 connected to a controller device 62. The controller device 62 typically comprises a touchscreen. The touchscreen may be coupled to said conferencing system 60 via, for example, a LISB-C cable that provides a data channel for both video data and interaction data where video may pass via LISB-C DP alt mode and interaction data may pass via a HID USB endpoint. Alternatively, said touchscreen device may use a network connection, for example Ethernet or Wi-Fi, in a direct or indirect manner. Said touchscreen device may be directly connected to said conferencing system via said network connection or it may be indirectly connected, for example via a router, a computation device, an edge infrastructure, a cloud infrastructure and so on. Other examples for providing said connectivity between said controller device and said conferencing system may include HDMI, DisplayPort, DisplayLink, Thunderbolt, DVI, VGA, USB. Multiple means for communication may be combined, for example connecting visual data via HDMI and interaction data via USB.

[0102] Fig. 12 (b) illustrates an example of providing a controller device functionality to a primary system 10 through a companion system, in this example a user device 12 comprising a detection means 30, a trigger module 32, a user interface 34 and a channel transformation module 64, which communicates with a channel transformation module 68 on the primary system's side. The primary system 10 in this case is a conferencing system comprising a conferencing device 70. A virtual controller device 72 is created that presents itself to said conferencing system as a controller device. Said virtual controller device 72 may reside in a separate device or be integrated in said conferencing system device. Said virtual controller device 72 may then be adapted to send and receive signals related to said virtual controller device towards, for example, the user interface 34. This adaptation is done via the channel transformation module 64 and a corresponding transformation module 74 on the primary system's side. The modules 64 and 74 are connected via a communication channel, and thus the user interface 34 is coupled to said virtual controller device 72. In the shown example, a controller interface transformation module 76 is provided between the channel transformation module 74 and the virtual controller device 72.

[0103] Said adaptation may include compression / decompression, encryption / decryption and further optimizations towards bandwidth and / or latency. Said adaptation may be modality specific, for example one may provide a certain type of processing for adapting the interaction signals while one may provide a different type of optimization for adapting the video signals and yet another type for adapting the audio signals. Synchronization of said modalities may be facilitated by including metadata in said adapted data stream or said adapted data may have integrated synchronization. Synchronization may be limited to a subset of the available modalities. For example, one may choose to synchronize audio and video data, but prefer not to synchronize the interaction data in order to increase responsiveness of said user interface.

[0104] Signal transformations from a virtual touchscreen device acting as a virtual controller device 72 may be further adapted with the optional controller interface transformation module 76. As this module is optional, it is shown in Fig. 12 (b) with a dotted rectangle. Module 76 can transform user interface data that is provided by said conferencing system and transform this user interface data to a representation that is suitable or optimized towards usage with said user interface 34 provided by said user device 12. Module 76 may add or remove interface components from said user interface 34. This way, one may optimize the user interface 34 for a certain user or user group or optimize it for the conditions at hand. Furthermore, adding elements enables said interface to be integrated in the broader system and allows said interface to not only expose configuration and state changes of said primary system but also expose configuration and state changes of other systems, for example said companion system. This way, a single integrated interface can be presented to the user. Note that in Fig. 12 (b) the controller interface transformation functional module is located between said virtual controller device 72 and said channel transformation module 74, however this may also be located somewhere else in the system and multiple of said controller interface transformations modules 76 may be present in said system. For example, one may provide user interface transformations on a group level at the location as shown in Fig. 12 (b), but further adapt that on the device that presents said user interface 34 to said user. This way, transformations to said interface that are applicable to a group of users may be done centrally (only once) while transformations that are personal may be done on a per-user basis, for example by said user device.

[0105] Said virtual controller device may interact with said conferencing device via a similar interface as said real controller device interacts with it. For example, one may have data stream with image data which may be raw image data or encoded image data, one may have a datastream with audio data which may also be raw or encoded audio data, one may have a data stream with interactivity data, for example the raw or encoded data that represents the touch interacts that are actuated on said device. Said controller interface transformation module may include recognition techniques, for example image recognition techniques, in order to understand said data stream and apply said required transformations to them. Alternatively, data streams may be accompanied by metadata that, for example, describe the different parts of said interface signals and enables said transformations to be applied in a more effective manner. Said metadata may be retrieved from said primary system, said conferencing device in this case, via alternative means. For example, an API may be available to retrieve state information from said primary system which may help determine said interface state and interpret said interface signals.

[0106] Yet another alternative may consist out of interface data that does not (only) rely on said consumable modality data such as image, audio, etc. but represents said interface in a more abstract manner. For example, an API may be available on said conferencing device that allows for executing the state retrieval and interaction functionalities directly on said conferencing device instead of via said controller device. So the functions that are usually exposed via said controller device are now also accessible via a programmatic interface. In that case, said virtual controller device may interface with said conferencing device through said API and said transformations may remain at a more abstract level. For example, said data stream may provide the notion of a button, the message that should be shown on said button and what should happen when said button is pressed (for example calling a specific API function on said conferencing device). Such a more abstract notion of an interface makes it easier to transform said interface data streams. For example, it is very easy to remove one or more elements or add other elements on this abstract interface data stream. Said abstract data needs to however still be rendered to a version that can be consumed by, for example, a user. Said rendering may be done near said conferencing device, for example co-located with said virtual controller device or said rendering may be done near said user interface presentation, for example on said user device. Hybrid rendering approaches are also possible where a part of said rendering is done in one device and another part is done in another device. For example, said rendering may use said abstract representation of said button as an input and generate an image that depicts said button. A callback would also be generated that enables a signal to be generated when said user presses said button; said signal is then communicated to said virtual controller device via said described communication path (via the channel transformation modules and optionally said controller user interface transformation module) which can then call said function on saidconferencing device via said API. Yet another advantage of using said more abstract notion of an interface is that said rendering may be freely adapted to said usage conditions. For example, colors may be easily adapted to blend in with the rest of the system or facilitate users with visual disabilities such as color blindness. Modalities of said interface may be changed depending on the conditions and requirements. For example, one may have an interface that uses, for user A, a visual button to, for example, start a conferencing session. However, for user B, said functionality may be presented to said user B via a voice command.

[0107] In the embodiment shown in Fig. 12(b), the trigger module 32 is coupled to a detection means 30, which may, for example, be the presence detection as described above or said detection of a dongle being coupled as described before. Said trigger module may communicate with said virtual controller device 72 to enable / disable said user interface 34. This way, no interface needs to be provided when there are no active user interfaces 34. As noted before, there may be multiple user interfaces 34 active, within the same device or across devices and the system needs to route said transformed user interface data accordingly. Said routing is not depicted in Fig. 12(b).

[0108] Said trigger module may only have a local effect where, instead of communicating said activity state to said virtual controller device, said user interface is communicated instead to present or hide said interface to said user. Such implementation may be more easily deployed.

[0109] In one non-limiting example of said system, a primary system is integrated in a device that also contains components of the companion system. Said device, called "hybrid device" in the following, may, for example, contain at least one camera, at least one microphone, and / or at least one speaker. In one variant, said hybrid device may also contain at least one display. Said hybrid device may host at least a part of said primary system and typically includes computational capabilities to provide said primary system functionalities. Alternatively, said computational capabilities may be provided by a coupled system, for example a cloud computing unit.

[0110] Such hybrid device could provide, next to said at least one primary system functionalities, of which at least some functionalities are related to said companion system. Said functionalities may include, for example, an ultrasound transmitter module which may make use of said speaker to transmit an ultrasound beacon signal within the meeting space. Said beacon signal may include coded information that enables receivers of said signal to, for example,connect to said hybrid device in order to make use of its capabilities. For example, said beacon signal may include connection information to connect to the wireless network that is provided by said hybrid device. Said functionalities may furthermore include, for example, a virtual controller device module to communicate to the subsystem of said primary system that facilitates the use of, for example, a table-top alike controller device to monitor and control said primary system. Said virtual controller device module will interface with said primary system and expose and / or adapt said functionality to the other components and devices of said companion system. Said virtual controller device module may, for example, get a video stream related to said user interface and transform said video stream to compress said video stream data and encrypt said video stream as a preparation for secure transmission. Said virtual controller device module may, for example, send a data stream towards said primary system that represents the interaction data related to said user interface. Said interaction data may, for example, include the clicks, taps or swipes that are done by a user when interacting with said user interface.

[0111] In one variant, said virtual controller device interfaces with said primary system via an API - an "Application Programming Interface" - that enables a more abstract representation of said user interface. For example, said virtual controller device interface may construct a user interface 34 that includes elements that are accessible via said API. For example, said virtual user interface may expose a button that enables a user to connect or disconnect from a conference call by said primary system, said virtual user interface facilitating said button by calling an API function to either connect or disconnect to the schedule conference call. Said virtual user interface may additionally or alternatively also expose a colored control that indicates whether the primary system has muted the microphone; said virtual user interface would in that case translate the API function of monitoring said mute state into a control that, for example, visually shows said mute state.

[0112] Said user device may connect to said hybrid device and said connection may be facilitated by detecting said beacon via the microphone of said user device. Said detection may be done by a software component that is running on said user device. For example, said software component may analyze the audio data coming from a microphone that is coupled to said user device and isolate said coded information that is included in said beacon signal. Said analysis may be a continuous process, but said analysis may also be triggered by a condition being satisfied. For example, one may start analyzing when the user is on the move, when a certain program is activated / started on said user device, when another process indicates that the likelihood of entering a meeting space is increasing or when another process indicates thatthe user may intent to connect to said hybrid device. When said beacon signal is detected, said user device may automatically connect to said hybrid device or said connection may be conditioned on other factors, for example first giving a notification to the user that said hybrid device can be connected to.

[0113] Upon connection, said software component in said user device may connect to said virtual controller device module in said hybrid device to present said user interface to said user. Said presentation may be done directly upon connection, or it may be conditioned on other factors. For example, a user may be given a notification that said interface is ready to be presented or one may only show said interface when said user issued a command (e.g. via a user interface on said user device). When connecting to said virtual controller device module and said virtual controller device module exposes a video stream to said user interface module, said video stream coming from said virtual controller device module may be decrypted and / or decompressed and adapted to be presented on said user device. For example, said data may be scaled and / or cropped. For example, analysis may be performed on said data in order to facilitate further adaptations such as making said functionality accessible via other - possibly non-visual - means. After adapting said user interface data, said user interface may be presented. Said presentation may be based on different modalities, multiple of said modalities may be used at the same time. For example, a video-based interface may be augmented with audio commands or additional shortcuts in order to enhance the presentation and / or interactions with said user interface. For example, an API-facilitated interface may first require rendering to one or more modalities before said interface may be used. For example, if a toggle button is exposed to connect or disconnect from the conference call in said primary system, said toggle needs to be rendered and presented to said user, for example via an overlay Ul.

[0114] Furthermore, a component will be executed on said user device that will detect said interactions with said user interface, possibly transform said interactions and send these interactions back to said virtual controller device module in said hybrid device. Said interaction data may for example be transformed in order to align coordinate systems between said presented user interface and said exposed virtual controller device in said hybrid device. If, for example, a video interface was resized and / or cropped, said interaction data needs to invert these operations. Said interaction data may further be transformed in order to align on interaction modalities that have been made available to said user. For example, if voice commands are provided by said user interface but such commands were not exposed by said virtual controller device, said commands need to be aligned with the data interface that isexposed by said virtual controller device. The same needs to be done for other transformations that are applied to said user interface by said user device as these transformations will typically impact said interaction data channel and said component will need to re-align said interaction data to become compatible with said virtual user interface.

[0115] The aforementioned user interface transformations and the related interactions are done by said user device in this example, however these may also happen in said hybrid device or in another device. For example, said virtual controller device interface may directly expose an adapted user interface and said exposure may expose different modalities and interaction capabilities compared to those that are provided by said primary system. Said principles of adapting and transforming said data streams related to said user interface remain valid regardless of the mapping to particular devices. In a further variant on said non-limiting example of said system, an additional peripheral device is considered that, when connected to a user device, may provide an alternative to connecting said user device to said hybrid device and facilitate further or alternative interactions to said companion system. Said peripheral device may have a wireless transceiver and may contain the connectivity details for connecting said peripheral device to said hybrid device and, when coupling said peripheral device to said user device, said connection may be automatically setup. For example, said peripheral device may be a USB (or another port) device that contains a wireless transceiver that can automatically connect to the wireless connectivity capabilities of said hybrid device when connecting said peripheral device to said user device via said USB (or another) port. Said peripheral device may have sensors and / or actuators to present a user interface to said user; said user interface may not require said user device to execute a software module to facilitate said function. Said user interface that is presented by said peripheral device may be adapted to facilitate said peripheral capabilities.

[0116] When no video interface is available - for example when said peripheral device doesn’t have a screen - one may adapt said interface to use other modalities that are possible with said peripheral device. One may for example map a user interface action to one or more of the buttons and / or sensors that are available on said peripheral device. In addition, said user interface may also contain information such as state information of said primary system (for example, whether a conference call is active, the mute state of a microphone, the state of the camera and so on) that can be presented by said peripheral device.

[0117] If said peripheral device has a display, controllable led lights or other means of presenting said information may be foreseen. Said adaptations may be done by said peripheral device, by said hybrid device or by another coupled device. Said user interface may be triggered when said peripheral device is coupled to said user device. Alternatively, or additionally, said user interface may be triggered when a certain condition is met, for example when said user presses a button on said peripheral device.

[0118] Fig. 13 shows that a user interface 34 on a peripheral device 40 connected to a user device 12 may be accompanied by an additional user interface 78 that is shown on one or more displays that are connected to or coupled to a hybrid device 80 comprising the primary system 10. In this example, the peripheral device 40 is shown as having similar components and functionalities as the dongle described above, so the same reference numbers are used for the respective functionalities / modules.

[0119] When the peripheral device 40 is detected to be coupled to a user device 12, said additional user interface 78 may be presented on said one or more displays that are connected or coupled to said hybrid device 80 and said additional user interface 78 may, for example, provide said user with more information on how to use said peripheral device 40. This can compensate for the lack of a visual interface on said peripheral device 40. For example, said additional user interface 78 may show to said user what action would be done when interacting with buttons and / or sensors on said peripheral device. If a button is available on said peripheral device 40, said additional user interface 78 may give information about the action that would be executed when pressing this button. Furthermore, said additional user interface 78 may give additional information about the state of the system, for example whether said primary system 10 is in a conference call or whether the microphone is muted.

[0120] The hybrid system 80 is coupled to one or more displays 82 (only one is shown). Such display(s) 82 may be part of the hybrid device 80. In the shown embodiment, the display 82 is coupled to a rendering module 84, which may be part of the operating system of said hybrid device 80 or it may be a separate module that is provided to said hybrid device 80. Said primary system 10 may provide input to said rendering module 84 in addition to the user interface 78, and possibly other components of said hybrid device 80 that require rendering to said one or more displays. For example, user interface data may be presented at a corner of said one or more displays and being overlaid by output from said primary system 10. Said peripheral device may have a button, an actuator or another means of signaling said user interface 78 to bevisible or not. A controller proxy module 86 may facilitate implicit or explicit signaling between said the two (or more) user interface 34 and 78. Alternatively, the user interfaces 34, 78 may exchange data via other means of communication. For example, said user interface 34 in said peripheral device may signal, via said controller proxy module 86, to said user interface 78 when said user interface 34 is active and said user interface 78 may then be shown on said one or more displays.

[0121] In another variant, said additional user interface 78 may be presented on said user device 12 in addition to or as an alternative to said additional user interface 78 that is presented on said one or more displays that are coupled to said hybrid device 80. A software component on said user device may in that case detect said coupling of said peripheral device 40 and may provide said additional user interface 78. While this variant may require additional software on said user device 12, it does not require said user interface 78 to be shown on said one or more displays that are coupled to said hybrid device 80. Both approaches may co-exist, where, for example, said user interface 78 is used on said one or more displays that are coupled to said hybrid device 80 when no such software module is available on said user device 12 but one uses said user interface 34 on said user device instead when said software module is available. Any combination of said user interface 34, 78 are possible and are implementation dependent.

[0122] It should be understood that in general, a companion system may include a dongle that may be coupled to or connected to another device, for example a device that a user brought into the meeting space. If there is a conferencing system in said meeting area, said dongle may provide additional functionalities to said conferencing system. For example, if said conferencing system is connected to a conferencing call, for example based on audio and video, and said conferencing system has its own conferencing login, it can be difficult for a user to add content to said conference call. For example, if a user needs to share information that is accessible by a device, getting that content into said conference call may be difficult. One may be required to manually log into that same conference call with a device capable of doing so, access said to- be-shared content from said device capable of login into said conference call and then share said content in said conference call. This is often a difficult and ineffective process. By using said dongle, one may simply couple said dongle to a device, for example a user device, that has access to said to-be-shared content and said dongle will enable said content to be shared.

[0123] Using said companion system, one may directly share content from said user device that has access to said to-be-shared content towards said conferencing system and saidconferencing session. Said dongle may directly or indirectly connect to said conferencing system and present a way to said user device and said user to select and share said to-be- shared content. For example, software may be running on said user device that enables a user to indicate what needs to be shared and that will communicate said content with said dongle, which will then further communicate said content with said conferencing system for sharing locally to the meeting space and / or sharing within said conferencing session. For example, said dongle may expose a display to said user device and content that is placed on said additional display may be automatically communicated with said conferencing system for sharing locally to the meeting space and / or sharing within said conferencing session. In another example, said dongle may provide connectivity information to said user device so said user device can make a connection to said conferencing system for communicating said content to said conferencing system for sharing locally to the meeting space and / or sharing within said conferencing session.

[0124] Using said dongle of said companion system, one may facilitate onboarding of extra devices and its data in an effective manner, for example those that are brought by participants and improve the interaction possibilities of said conferencing system. Note that said dongle and said dongle functionalities may be provided in a virtualized manner as well, for example by means of software running on said user device or elsewhere.

Claims

CLAIMS1. A method for coupling a companion system to a primary system and enabling at least partial control of the primary system by the companion system, the companion system comprising at least one input and output device including a display for interacting with a user and a connecting port for connecting the companion system to the primary system, the primary system being configured for directly executing a primary software, the method comprising the steps of connecting the companion system to the primary system, generating a user interface, displaying the user interface via the companion system, and giving the companion system access to at least partially control the primary software via the displayed user interface.

2. The method according to claim 1, further comprising a step of checking an identification of the companion system prior to giving the companion system access to at least partially control the primary software, and giving said access depending on a result of checking the identification.

3. The method according to claim 1 or claim 2, wherein the user interface is generated by the primary system and send to the companion system for being displayed.

4. The method according to claim 1 or claim 2, wherein the user interface is generated by the companion system.

5. The method according to claim 1 or claim 2, wherein the user interface is generated by a separate component, in particular a dongle, connected to the companion system.

6. The method according to one of claims 1 to 5, wherein generating the user interface is triggered by one of- automatically detecting the presence of the companion system in the proximity of the primary system,- active signalling the presence of the companion system to the primary system,- coupling a separate component, in particular a dongle, with the companion system,- activating a signal generation on a separate component, in particular a dongle, after coupling the separate component with the companion system,- signing into a conference hosted by the primary system,- accessing an area in which the primary system is installed,- active requesting remote access to the primary system via the companion system.

7. The method according to one of claims 1 to 6, wherein the companion system is automatically signed into a running conference hosted by the primary system.

8. The method according to one of claims 1 to 7, wherein a conference hosted by the companion system is automatically transferred to the primary system.

9. The method according to one of claims 1 to 8, wherein the user interface is initially displayed on the companion system only for a predefined time period and afterwards only upon certain actions.

10. The method according to one of claims 1 to 9, wherein the user interface is displayed in a separate portion of display of the companion system.

11. The method according to one of claims 1 to 10, wherein the companion system comprises at least two user devices connected to the primary system and wherein the user interface is displayed on only one of the user devices connected to the primary system in accordance with predefined rules.

12. The method according to claims 2 and 11, wherein in case access to control the primary system is denied to a user device based on its identification, the user device is signed into a conference hosted by the primary system via a secure channel.

13. The method according to one of claims 1 to 12, wherein the companion system is connected to a dongle, and wherein the dongle comprises executable software code connecting the companion to the primary system.

14. The method according to claim 13, wherein the software code is executed by the dongle when the companion system is connected to the dongle.

15. The method according to one of claims 1 to 14, wherein the user interface is generated according to preferences of a user of the companion system.

16. The method according to one of claims 1 to 15, wherein the companion system is comprised by one or more user devices.

17. A dongle configured for being used in a method according to one of claims 1 to 16.

18. A conferencing system configured for being used as primary system in a method according to one of claims 1 to 16.

19. A user device configured for being used as companion system in a method according to one of claims 1 to 16.

Images