Remote virtual device
The system addresses the challenge of synchronized media playback across multiple remote devices by assigning roles and managing content distribution, achieving seamless synchronized media rendering.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-01-03
- Publication Date
- 2026-03-12
AI Technical Summary
Existing systems struggle to control multiple remote playback devices to play media content in a synchronized manner, with each device fulfilling a specific role, which is challenging due to varying capabilities and conditions.
A system is provided that includes an interface to multiple remote player devices, a media distribution network, and a user application, enabling the distribution of roles among devices based on user selection and device capabilities, with a cloud backend managing playback and content distribution to form a virtual device for synchronized media playback.
The system effectively manages multiple remote playback devices to perform synchronized media playback by assigning roles and distributing content, ensuring seamless and synchronized media rendering across devices.
Smart Images

Figure EP2025050131_12032026_PF_FP_ABST
Abstract
Description
[0001] CIN240904PEP-2025002689.DOCX 1
[0002] Remote virtual device
[0003] Description
[0004] It is often difficult to control multiple remote playback devices receiving streams from remote in such a way that each device plays according to a specific role. Techniques are here provided which address such a task.
[0005] In examples there is provided, inter alia, a system for assisting a streaming and a rendering of remote media content, the system including: an interface to a plurality of remote player devices , each controlling at least one media Tenderer unit and having playback capabilities and / or playback conditions; an interface to at least one remote media distribution network, CDN; an interface to a remote user application , to send, to the user application, a list of the remote player devices, and to receive, from the user application, a selection on a plurality of remote player devices to be part of a virtual device for a synchronized media playback, the system being configured to distribute roles among the remote player devices of the virtual device, based on the selection performed by the user application , the system being configured to send, to at least one of the remote player devices of the virtual device, role information which includes instructions on a respective role of each remote player device of the virtual device for participating to the synchronized media playback, the system being configured to send, to at least one of the remote player devices, information on media content to be retrieved and rendered in the synchronized media playback.
[0006] In examples there is provided, inter alia, a player device for participating to a synchronized media playback, the player device including: an hardware unit to render synchronized media playback of media content; an interface to a remote assisting system ; an interface to at least one other player device through a local connection; the player device being configured to receive, from the assisting system , information of being part of the virtual device and role information indicating a playback role that the player device shall have in the synchronized media playback, to thereby constitute a virtual device with the at least one other player device to perform the synchronized media CIN240904PEP-2025002689.DOCX 2 playback by receiving the media content and playingback the media content according to the playback role.
[0007] In examples there is provided, inter alia, a user application for commanding a streaming and a rendering of remote media content, the user application executing in a user equipment, the user application comprising: an interface to an assisting system to receive information on a plurality of remote player devices, each controlling at least one media Tenderer unit and having playback capabilities and / or playback conditions; the user application being configured for receiving, through the interface to the assisting system, information on media content on at least one remote media distribution network, CDN; the user application being configured to define a remote a virtual device from a plurality of remote player devices, and the user application being configured to receive playback control commands and / or content control commands from a user for the virtual device, and to retransmit the control playback commands to the assisting system.
[0008] In examples there is provided, inter alia, a method for assisting a streaming and a rendering of remote media content, the method including: interfacing with a plurality of remote player devices, each controlling at least one media Tenderer unit and having playback capabilities and / or playback conditions; interfacing with at least one remote media distribution network, CDN; interfacing a remote user application, to send, to the user application, a list of the remote player devices, and receiving, from the user application, a selection on a plurality of remote player devices to be part of a virtual device for a synchronized media playback, distributing roles among remote player devices of a virtual device, based on the selection performed by the user application, sending, to at least one of the remote player devices of the virtual device, role information which includes instructions on a respective role of each remote player device of the virtual device for participating to the synchronized media playback, sending, to at least one of the remote player devices (10), information on media content to be retrieved and rendered in the synchronized media playback. CIN240904PEP-2025002689.DOCX 3
[0009] In examples there is provided, inter alia, a method for participating to a synchronized media playback, including: rendering synchronized media playback of media content; interfacing with a remote assisting system; interfacing with at least one other player device through a local connection; the method including receiving, from the assisting system, information of being part of the virtual device and role information indicating a playback role that the rendering shall have in the synchronized media playback, to thereby constitute a virtual device with the at least one external player device to perform the synchronized media playback by receiving the media content and playingback the media content according to the playback role.
[0010] In examples there is provided, inter alia, a method for commanding a streaming and a rendering of remote media content, comprising: interfacing with an assisting system to receive information on a plurality of remote player devices, each controlling at least one media Tenderer unit and having playback capabilities and / or playback conditions; receiving, through the interface to the assisting system, information on media content on at least one remote media distribution network, CDN; defining a remote a virtual device from a plurality of remote player devices, and receiving playback control commands and / or content control commands from a user for the virtual device, and to retransmit the control playback commands to the assisting system.
[0011] Figs. 1A and IB show examples of an ecosystem involving the present technique.
[0012] Fig. 2 shows an example of a virtual device 110 according to the present technique.
[0013] Figs. 3A-3H show examples of an operational sequence according to the present technique. Figs. 4A and 4B show examples of an operational sequence according to the present technique.
[0014] Fig. 5 shows an example of operation.
[0015] Fig. 6 shows an example of group.
[0016] Figs. 7A-7H shows an example of how to generate a group.
[0017] Figs. 8A-8D show examples of selectable setups.
[0018] Figs. 9A-9B show examples of operation. CIN240904PEP-2025002689.DOCX 4
[0019] Fig. 10A illustrates an embodiment of a player device;
[0020] Fig. 10B illustrates an example of cloud backend;
[0021] Fig. 10C illustrates an implementation of user application apparatus;
[0022] Fig. 10D illustrates an implementation of ecosystem.
[0023] Examples
[0024] An example of virtual device 110 is provided in Fig. 2 for a distributed playback of media content. The virtual device 110 is shown as being formed by devices 10, which in this case are instantiated by: a single virtual device primary player device 10a two virtual device secondary player devices 10b and lOd (a player device 10c being excluded from the virtual device 110).
[0025] It will be shown that the virtual device primary device 10a receives media content streams 51 from remote, and forwards them to the virtual device secondary devices 10b and lOd, but not to the device 10c. It will be shown that the choice of the devices 10a, 10b, and lOd as constituting the virtual device 110 may be made by a user application 3 (e.g. commanded by a user 99) and may be provided to the devices 10 through an assisting system 4 (backend, which may be a cloud backend, e.g. geographically distributed). Each device 10a, 10b, lOd in the virtual device may be a specific role in the resulting distributed playback. The choice of taking part to the virtual device 110 may be based on the capabilities and / or conditions of each player device 10. In the example of Fig. 2, hence, the player device 10c is excluded e.g. by user's choice.
[0026] It will also be shown that the choice of which device, among the devices 10a, 10b, and lOd, will be the virtual device primary device may be either made (e.g. dynamically) by the assisting system 4 (e.g., based on the conditions and / or capabilities of each device) or may be decided by election, without the assisting system 4 even caring of which player device is elected as virtual device primary device. In any case, the user 99 and the user application 3 may not care of the election.
[0027] It will be shown that each device 10a, 10b, lOd of the virtual device 110 will have a particular role distributed by the assisting system 4, e.g. based on a request from the user application 3 (e.g., due to user's choice).
[0028] It is noted that, in examples, the player devices lOa-lOd do not maintain memory, of having been precedently part of the particular virtual device 110. In addition or alternatively, the CIN240904PEP-2025002689.DOCX 5 assisting system 4 and / or the user application 3 may maintain memory of the particular virtual device 110 and of which player devices 10a, 10b, lOd have taken part to the virtual device and in which role for each virtual device, e.g. without pre-establishing which is the virtual device primary player device and which is or are the virtual device secondary player device(s).
[0029] Here, reference is mostly made to an ecosystem in which there are a plurality of player devices 10 (lOa-lOd), an assisting system 4 (backend), at least one content distribution network (CDN) 5, a user application 3 for permitting control of a user 99. Notably:
[0030] 1) Each player device 10 may have an interface connected, in remote, with the assisting system 4 and with each other player device 10 in a same local network
[0031] 2) The assisting system 4 may have: a) an interface connected, in remote, with each player device 10 (e.g. forming at least one virtual device 110) b) an interface connected, in remote, with the user application 3 c) an interface connected, in termite, with at least one CDN 5
[0032] 3) Each CDN 5 may have: a) an interface connectable, in remote, with at least one player device (see below, a group primary player device and / o a virtual device primary player device) b) an interface connected, in remote, with the assisting system 4
[0033] (but no interface with the user application 3, in some examples)
[0034] 4) The user application 3 may have: a) an interface connected, in remote, with the assisting system 4
[0035] (but in principle no interface connected to player devices 10 or to the at least one CDN 4).
[0036] Fig. 1 shows an example of an ecosystem which involves the technique discussed here. Fig. 1 shows a player device 10 (which may be instantiated in any of player devices 10a, 10b, 10c, lOd), each player device 10 may control a device hardware 1 (which may be a display device and / or loudspeakers and / or a hardware connection to the loudspeakers, and may be understood as a hardware unit having substantially replay capabilities). The device hardware 1 may be contained in the player device 10, and may have substantially replay capabilities. The device hardware 1 may be controlled through a hardware abstract layer, HAL, block 11. The player device 10 (10a, 10b, 10c, lOd) may receive a media content 51 (51a as a stream), from a content service or media content distribution network (CDN) 5 (CDN 5 is also called content service; there may be multiple content services in competition with each other, and the user can shoes his / her preferred one). A cloud backend system 4 (assisting system or system for assisting streaming and rendering or remote media content) CIN240904PEP-2025002689.DOCX 6 may control the player device 10 (10a, 10b, 10c, lOd), e.g., through content control 38 (e.g., 37, 37a, 137, 137a) and / or playback control 34. The player device 10 may include a device SDK (software development kit) / player 2, which is a software to control the streaming 51 and / or the playback at the hardware device 1.
[0037] The player device 10 may be updated through updating data 61 (61b) from an updating network 6 (6b). The updating network 6b may be operated by a company which is not the company operating the CDN 5 (5a). The updating network 6b may be operated by a company which is the same (or which collaborates with) the company operating the backend system 4.
[0038] The media CDN 5 (5a) which provides the content is not the same as the cloud backend system 4. The content 51 is therefore not transported through the same connection as the control commands 34 and 38 (it may be that they both share the same physical connection, but anyway the content 51 and the control commands 34 and 38 are not in the same session).
[0039] The device SDK / player 2 (media runtime device software, installed in the player device 10) may provide the media content (e.g., audio and / or video streams) to a content decryption service 28, which can therefore decrypt the media content 27 (combing from 51). The decrypted media content 29a may therefore be provided to a hardware media Tenderer unit 1. The hardware media Tenderer unit 1 may include a hardware abstraction layer (HAL) 11. The HAL 11 may be understood as being provided by an operating system. The HAL 11 may be understood as a virtual input / output unit through which the SDK / player 2 controls the hardware device 11. The HAL 11 may control the rendering (e.g. final rendering) of the audio and / or video. The HAL 11 may be controlled by hardware control and / or hardware events 26, e.g. sent by the device SDK / player 2. The device SDK / player 2 may provide the hardware control and hardware events 26 based, in particular, on the control 34 and 38 accepted by the cloud backend 4. The player device 10 may be connected to a local network (e.g., LAN) 220.
[0040] The device SDK / player 2 may include a web browser 21. The device SDK / player 2 may include a media player 22. The device SDK / player may contain a cache 23 (e.g. in the media player 22). The device SDK / player 2 may contain a group playback entity 24. Inn some cases, different virtual devices 10 may forma group of devices rendering the same content.
[0041] The device SDK / player 2 may contain an updater 25, which may receive updated media content from the update CDN 5a. As can be seen from Fig. 1, the player device 10 may CIN240904PEP-2025002689.DOCX 7 receive content control 38 (e.g. content control commands), which may include, for example, information on queues 37 to be playedback. Fig. 1 shows many queues 37, because it may be possible that information on multiple queues 37 is sent for multiple, different devices 10 (see also below). The content control 38 may provide information on the media content to be rendered (e.g. through the information on the queues 37), e.g. together with address (e.g., URL address) of where the media content 51 is to be retrieved (e.g. address of locations associated to the CDN 5, 5a). The cloud backend 4 may transmit playback control commands 34. The playback control 34 may include, for example, well-known playback commands, such as "PLAY" "PAUSE", "STOP", "FAST FORWARD", "REWIND". The cloud backend 4 may therefore include a playback manager 33 and a device manager 35 which send the playback control 34, and the content control commands 38 respectively. It is to be noted that the device manager 35 may substantially have knowledge on the properties of the player device 10. The knowledge of the properties of the player device 10 may be achieved, for example, from setup information, which can be obtained, for example, at a time of a user's subscription. The device manager 35 may have a replica of the SDK / player 2 and may therefore have the substantial control of the operations of the player device 10. Both the playback manager 33 and the device manager 35 may be controlled through control commands 31 and 32 respectively. The playback control 31 may be received from a user application 3, which may be executed in a user equipment (e.g., a smart phone, mobile phone, tablet, assigned computer) of property of the user and / or may be integrated in the player device 10 (despite Fig. 1 suggesting that the user equipment and the player device 10 are not integrated in the same device, but they could be). The control device setup commands 32 may command the device management 35 on the virtual device 110 to be formed between the device 10 and other devices (e.g., devices 10a, 10b, 10c, lOd). Therefore, the particular virtual device 110 may be defined by the user through the application 3.
[0042] The application 3 may be provided e.g. by the cloud backend system 4 (or by an app store, such as Google Play Store, e.g. in a certified manner) to the user equipment or other device for controlling the player device 10 (and more in general all the player devices lOa-lOd). The application 3 may present a user interface for obtaining settings (and / or comments and for commanding the devices 10a, 10b, 10c, lOd) to render content.
[0043] Even if between the application 3, the cloud backend system 4, the player device(s) 10 and the media CDN 5 (5a) it is shown that the arrows are mostly directed towards one direction only, it is to be understood that these arrows mostly refer to the directions of the commands. It is notwithstanding possible that feedback information is transmitted in any different direction. For example, the application 3 may receive feedback from the cloud CIN240904PEP-2025002689.DOCX 8 backend 4 (e.g. to reproduced in a user interface), which in turn may receive the feedback from the player device 10. Also, the direction of the lines 51 could be understood bidirectional, since one device (e.g. the primary device 10a, see below) may be the device which requests the transmission to the media CDN 5 (5a) and keep some contacts with the media CDN 5 (5a) (e.g. by sending requests relating to which representation of adaptation set of the media content to receive, e.g. in response to the reception of a manifest and based on the monitoring of the reception speed, and so on). It will be shown that the lines 51 can actually be deactivated for all the devices 10 (10b, 10c, 10c) which are not the primary devicelOa (see below). It is also to be noted that the connections 51 may actually pass through the connection 220 (e.g. through a router).
[0044] The connections 51 and / or the connection 34 and 38 and / or the connection 31 and 32 and / or the local connection 220 may be either wireless connections, or wired connections, or partially wired and partially wireless connections. It is here mostly imagined that the connections 31 and 32 between the application (and the user equipment) and the cloud backend system 4 are mostly remote (e.g., through a mobile phone network, a telephone network, an internet connection, a TCP IP connection, and so on). The cloud backend system (assisting system) 4 may be a cloud entity and may include a plurality of servers dislocated geographically. In any case, the cloud backend system 4 may include at least one server (e.g., a proxy server or connected to a proxy server) which may send at least one of the commands 34 and 38 to the player devices 10. (e.g., through a remote connection, including a mobile phone network, an internet, a telephone network, and so on). The devices 10 (10a, 10b, 10c, lOd) may be, instead, part of a local network (e.g. including a wireless local connection such as Wi-Fi, WLAN and / or a wired connection (e.g., a wired LAN, an internet connection, a TCP / IP connection, an intranet connection, and so on). The local network 220 may be partially wired and partially wireless (e.g., some devices 10 may be a wired connection to the local network 220, while other ones may be wirelessly connected to the wireless network 220).
[0045] The cloud backend 4 (which may be a system for assisting a streaming and rendering of remote media content) may comprise the interface 405 for interfacing to the CDN 5 (5a). The cloud backend 4 may include the interface 410 for interfacing with at least a plurality of remote player devices 10 (e.g., 10a, 10b, 10c, lOd). The devices 10 (10a, 10b, 10c, lOd) may have playback capabilities and / or playback conditions. The playback capabilities may be understood as capabilities of rendering audio and / or video content. For example, rendering capability is the capability of rendering video content (i.e., to control a video display). Another capability may be the capability of controlling the video content with a particular CIN240904PEP-2025002689.DOCX 9 quality (e.g., resolution). Another capability may be the capability of rendering audio content (e.g., of controlling one or more loudspeakers). Another capability of rendering the audio content at a determined quality (e.g., at a particular sound resolution). A condition may be the condition of receiving or transmitting the media content, through the local connection, at a data rate and / or error rate.
[0046] The cloud backend 4 may be connected with the mobile application 3, e.g. through the interface 403 and in remote. The mobile application 3 (which may, as explained above) be installed in a user equipment, which may be the same as one of the player devices 10 and / or may a different external element for obtaining requests from any playback commands. The mobile application 3 may in addition or alternatively, provide content commands (i.e., which content queues 37 are to be rendered).
[0047] The cloud backend system 4 may include a controlling unit 420. The cloud backend system 4 may implement the following functions:
[0048] 1) device management 35 (e.g. for controlling which streams are to be downloaded by the player device 2)
[0049] 2) playback management 33 (e.g. for controlling the playback of each stream)
[0050] 3) capability and / or conditions manager and role distributor, e.g. implemented in the device management 35.
[0051] The controlling unit 420 may receive, through a mobile application interface 403, commands and / or requests 32 from the mobile application 3. The commands and / or requests 32 may insert the request of a particular stream 51 to be retrieved and / or a particular virtual device to be derived from the present player devices 10 (lOa-lOd). The controlling unit may construe, e.g. from commands from the user application 3, the virtual device 110 as resulting from the player devices 10 (lOa-lOd) at disposal. The controlling unit may distribute roles to each of the player devices 10 (lOa-lOd) constituting the virtual device 110. The controlling unit may instruct the player devices 10 constituting the virtual device 110 to perform a rendering of the media content based on the requests from the mobile application 3 and according to the respective roles.
[0052] For example, the backend system 4 (assisting system) may have (e.g. receive) the knowledge, for each device 10 (lOa-lOd), whether that device 10 has video capabilities (i.e. CIN240904PEP-2025002689.DOCX 10 capabilities of rendering a video signal, and its device hardware 1 has therefore at least one display unit) and / or audio capabilities (i.e. capabilities of rendering an audio signal, and its device hardware 1 has therefore e.g. loudspeaker(s)). The capabilities may also be deeper than simply rendering audio and / or rendering video, but they can comprise, as discussed above, a quality of the rendering (e.g., maximum audio resolution). Each device 10 (10a- lOd) may send, preliminary (e.g. in a setup or installation session) indication on its capabilities, and the backend system 4 (assisting system) may therefore maintain knowledge of the capabilities. Subsequently, where the backend system 4 (assisting system), and in particular the controlling unit 420, may decide which devices 10 shall be part of the virtual device 110 and which role that device 10 shall have, it will make use of the knowledge on the capabilities of each device 10. Or, the backend system 4 (assisting system), may simply forward a decision from the user application about which device 10 shall be part of the virtual device 110 and / or which role that device 10 shall have, it will make use of the knowledge on the capabilities of each device 10.
[0053] Therefore, there are many possibilities, each of which may be an embodiment of the present technique (in some cases, it is possible for the user to decide which of the options is to be followed):
[0054] The user application 3 decides which device 10 is in the virtual device and decides which role each device shall have (e.g. after having informed the user 99 of the conditions and / or capabilities of each device 10) in full autonomy.
[0055] The user application 3 decides which device 10 is in the virtual device, but it is the assisting system 4 which decides which role each device 10 shall have (e.g. after having received from the user application 3 information on the conditions and / or capabilities required for each device 10, and after having matched the conditions and / or capabilities required for each device 10 and the conditions and / or capabilities provided by each device 10).
[0056] The user application 3 decides which device 10 is in the virtual device and decides which role each device shall have (e.g. after having informed the user 99 of the conditions and / or capabilities of each device 10) by selecting a particular setup option from a plurality of setup options provided by a template information sent by the assisting system 4 to the user application 3 (the template information may be used for assisting the user application to define a matching between the player devices 10 and the roles; or the template information may simply request a setup selection among a plurality of setup options, like the options 110a, 110b, 110c in Figs. 8A-8C, and the user application will reply to that setup request). CIN240904PEP-2025002689.DOCX 1 1
[0057] The conditions may also include the capability of transmitting in the local connection 220, e.g. by guaranteeing a particular minimum error rate or data rate. Each device 10 (10a- lOd) may send, preliminary (e.g. in a setup or installation session) indication on its condition of transmitting in the local connection 220, and the backend system 4 (assisting system) may therefore maintain knowledge of the condition. Subsequently, when the backend system 4 (assisting system), and in particular the controlling unit 420, shall decide which devices 10 shall be part of the virtual device (or which device 10 shall be the primary device, see below), it will make use of the knowledge on the conditions of each device 10.
[0058] The operations of the virtual device primary player device 10a and of the virtual device secondary player device(s) 10b, lOd are here discussed. How the virtual device primary player device and the virtual device secondary player device(s) are selected or elected, this will be explained later.
[0059] The virtual device primary device 10a is the device which receives, from remote, the media stream 51 and retransmits it to the virtual device secondary devices lOa-lOd. In some cases, the retransmissions are simply relays of the same media content, without any modification of the media content. In some cases, the stream 51 may be processed, e.g. so as to suppress some portions which do not match with the roles of the virtual device secondary devices 10b, lOd. For example, the virtual device primary device 10a may suppress one or more channels for a retransmission of a media content towards a virtual device secondary device (10b, lOd) which does not need that suppressed channel (e.g. in a stereo rendering of audio, the left channel may be suppressed in the retransmitted media content in the case that the virtual device secondary device 10b only needs to render the right channel, while the virtual device primary device 10a renders the right audio content). For example, in the case of stereo audio signal generated by two player devices 10a and 10b (each having the role of rendering a particular channel), the primary device 10a may receive two audio channels and render only one first audio channel (e.g. left channel), but may forward to a secondary device 10b only one other audio channel (e.g. right channel, in this case), so that the virtual device 110 reproduces a stereo audio system. In this case, the primary device CIN240904PEP-2025002689.DOCX 12
[0060] 10a shall have the capability of decoding the received stream 51, processing it by suppressing the left channel, re-encoding the a right-channel stream 51 with only the right channel, and forwarding the re-encoded right-channel stream 51 to the secondary device 10b. In this case, the primary device 10a shall have encoding, processing, and decoding capabilities, while the secondary device 10b shall only have decoding capabilities and the capability of rendering one channel. Of course, the example of the channel suppression may be generalized: any media content which is not meant to be rendered by a secondary device can in principle be suppressed by the primary device 10a. In some cases, the virtual device primary device 10a may decode, process and re-encode the media content 51 to be retransmitted, so as to provide, to each virtual device secondary device lOb-lOd only a processed version of the stream 51.
[0061] The virtual device primary device 10a may receive an encrypted stream (encrypted through a particular first algorithm and / or a particular first decryption key) and forward the stream 51 to a virtual device secondary device 10b either encrypted or non-encrypted. The virtual device primary device 10a may have the capability of decrypting the stream according to a first algorithm and first decryption key through which the stream 51 has been encrypted at the CDN 5 side, and (optionally) of re-encrypting the stream 51 e.g. through a second algorithm and / or second decryption key through which the secondary device 10b is capable of decrypting. And, the virtual device secondary device shall only have the capability of decrypting according to the second algorithm and / or second decryption key, but not necessarily through the first algorithm and / or first decryption key.
[0062] The system 4 and / or the user application 3 may define the virtual device 110 as formed by the multiple player devices 10 (lOa-lOd), being connected with each other through a local connection 220.
[0063] The virtual device 110 is meant to provide a semantically unitary media scene (e.g. video and / or audio scene) to a user. The player devices 10 (lOa-lOd) may therefore participate to the synchronized media playback, where each player device 10 (lOa-lOd) has its particular role in the media rendering. Therefore, the cloud backend system 4 may send, to at least one (e.g. all) of the remote player devices 10 (lOa-lOd), instructions on its role of each in the virtual device 110 to participate to the synchronized media playback. In some CIN240904PEP-2025002689.DOCX 13 examples, the role of each player device 10 (lOa-lOd) is transmitted, through role information 137 and / or 137a, to the related player device 10 (lOa-lOd). Or, the role of each player device 10 (lOa-lOd) could be transmitted, through the role information 137 and / or 137a, only to one primary player device 10a among the plurality of player devices 10 (10a- lOd), and the primary player device 10a will forward role information to each of the other player devices 10b, lOd (which are called virtual device secondary player devices) of the same virtual device.
[0064] The role of each player device 10a, 10b, lOd may be chosen (e.g. by the user application 3 and / or by the assigning system 4) based on the capabilities and / or conditions of each player device. For example, a video rendering role will be assigned to a player device having video rendering capabilities, an audio rendering role will be assigned to a player device having audio rendering capabilities and so on. For the definition of the roles, the assisting system will make use of the capabilities and / or conditions of each player device.
[0065] The cloud backend system 4 may be interfaced, through the interface 405, with the CDN 5 (5a) (content service interface) to authorize the virtual device 110 to receive the media stream 51. The authorization may be routed through the remote connection 425 between the cloud backend system 4 and the CDN 5 (5a).
[0066] The cloud backend system 4 may perform an authorization procedure to authorize the remote player devices 10 forming the virtual device 110 to receive the media content. Importantly, the virtual device 110 will be considered as one single device receiving the media stream (and this could be important for the subscription). The cloud backend system 4 may provide, to the CDN 5, authentication data for authenticating at least one of the remote player devices which receives the media content from the CDN. In addition or alternatively, the cloud backend system 4 may receive from the CDN 5 an authentication information and provide the authentication information to at least one of the remote player devices 10, so that at least the virtual device primary remote player device 10a, by presenting the authentication information to the CDN 5, can receive the media content 51 from the CDN.
[0067] As will be explained, the virtual device 110 may include the virtual device primary device (10a) and one or a plurality of secondary devices (10b, lOd). It is preferred that only the virtual device primary player device 10a receives the media content 51 from the CDN 5 (5a) CIN240904PEP-2025002689.DOCX 14 and after that retransmits the media content through the local connection 220 by sending media frames 218. In some cases, the virtual device primary player device 10a may be chosen by virtual device primary election between the devices 10a, 10b, lOd constituting the virtual device 110. In other cases, it is possible to choose the primary player 10a by the cloud backend system 4 (and in particular the control unit 420) and / or by the user application 3, e.g. based on capabilities and / or conditions of the player devices 10. For example, the cloud backend system 4 and / or by the user application 3 can choose the primary player device 10a based on a capability and / or condition of receiving the media content from the CDN (5, 5a) at a data rate over a data rate threshold and / or an error rate below an error rate threshold, so that the primary player device 10a can receive the media content from the CDN (5, 5a) and re-transmit the media content to the other player devices lOb-lOd.
[0068] During the distributed synchronized playback of the virtual device 110, the player devices 10a, 10b, lOd perform a synchronized rendering. The virtual device 110 primary device (10a) may provide feedback to the cloud backend system 4 (which, in some examples, may in turn send it to the user application 3) indicating the state of the rendering (e.g., indicating timing information, such as which portion of the media content is being rendered).
[0069] Elements on how to choose the virtual device primary player device 10a are provided now. Fig. 2 shows an example of a plurality of player devices 10 (each indicated with a particular letter after 10, e.g., 10a, 10b, 10c, lOd). Each of the devices 10 (lOa-lOd) may be like the device 10 of Fig. 1 (e.g. a replica of the same) and the local network 220 may be the same. Each of the player devices 10 (10a, 10b, 10c, lOd) may be connected with the cloud backend system 4 (assisting system), so as to receive commands 34 and 38 from the cloud backend 4. In principle, each of the devices 10 (10a, 10b, 10c, lOd) may be connectable to the media CDN 5 (5a).
[0070] As explained above, the player devices 10 (10a, 10b, 10c, lOd) may be part to the virtual device 110, participating to the synchronized media playback (e.g., video playback and / or audio playback) together with the other player devices 10. Each player device 10, as explained above, may control the hardware media render unit 1 with the media content 51 (51a). As can be seen in Fig. 2, there is the possibility that a virtual device 110 is formed by a plurality of player devices, in this case devices 10a, 10b and lOd (player device 10c not being active, i.e. not being in the virtual device 110). This choice may be made, for CIN240904PEP-2025002689.DOCX 15 example, by the user (e.g., through the user application 3, routed through the command 32, the device management 35, the control device queues 36, the content control 38, the player device 10 and in particular the device SDK 2, the hardware control and / or other events 26, the HAL 11, and the hardware media Tenderer unit 1).
[0071] Fig. 3A shows an example of provision of the formation of the virtual devices (e.g. virtual device 110). In this case, information 137 and / or 137a of being part of the virtual device 110 is provided to the selected player devices 10a, 10b, lOd (the selection may be made by the user 99 through the user application 3 and / or by the assisting system 3). As shown in Fig. 3A, role information 137a is provided to player devices 10a, 10b and lOd (it is imagined that the information arrives in through the local network 220, i.e., that the content control 38 is transmitted through the local network 220 (e.g., through a router) but this is unnecessary). The information 137 and / or 137a may be sent, for example, as addressed data packets (e.g. data packets which are uniquely sent to player devices 10a, 10b, lOd, while the data packets are not sent to the player device 10c). Therefore, each of the devices 10a, 10b and lOd determines that it shall be part of the virtual device 110. Further, the information 137 and / or 137a may include role information indicating the role of each player device. The role information 137a may be decided by the user application 3 (e.g. by the user 99) and / or by the assisting system 4 so as to match the capabilities and / or conditions of each device 10a, 10b, lOd. Fig. 3B does not show the device 10c which has been excluded from the virtual device 110. The device 10c can be in principle part of another virtual device (not shown) together with other player devices (not shown) (e.g. by following a different content command 38). As shown in Fig. 3C, when receiving the information 137 of being part of the virtual device 110, each player device (10a, 10b, lOd) in the virtual device 110 sends a broadcast message 212. The broadcast message 212 may provide an indication of which virtual device the sending player device 10a, 10b, lOd is part of (e.g., each broadcast message 212 may have a field in which the information 137 is written, indicating which virtual device the device is part of). As shown in Fig. 3D, each player device 10a, 10b, lOd of the virtual device 110 also receives the broadcast messages 212 from the other player devices in the same virtual device 110. For example, the player device 10a reads the broadcast messages 212 sent from the player devices 10b and lOd; the player device 10b receives the broadcast messages 212 from the player devices 10a and lOd and so on. It will be shown that the broadcast message 212 sent by each player device 10a, 10b, lOd may also include (apart from the identification of the player device from which it CIN240904PEP-2025002689.DOCX 16 originates) discriminating information (which may be part of the conditions and / or capabilities) attributed to the sending player device and / or the status of the sending player device. The discriminating information may include at least one performance indicator of the sending player device which sends the broadcast information 212. The stability indicator may reflect the stability of the player device over time. The more the unexpected reboots, crashes or corrupted receive frames, the lower the stability will be. The discriminating information may include a performance indicator. The performance indicator may include an amount of RAM, CPU, available flash storage space, availability of hardware decoders and so on. The discriminating information may include the random value (e.g., a random integer value). The discriminating information may include at least a unique identifier of the sending player device.
[0072] The player devices 10a, 10b, lOd therefore elect the virtual device primary player device 10a by examining the discriminating information of all the player devices in the virtual device 110. For example, a rating may be defined for each player device 10a, 10b, lOd of the virtual device 110 based on the performance indicator, stability value, random value and unique identifier and defining the one which has the highest rating. Accordingly, the virtual device primary player device may be elected implicitly without any device deciding which shall be the virtual device primary player device 10a. In particular, the cloud backhand system 4 could have no interest in deciding which will be the virtual device primary player device 10a, whose decision will only be defined by the devices in the virtual device 110.
[0073] The virtual device primary player device 10a, once elected, will be the one connected with the CDN 5 (5a) for receiving the media content 51 (also here the media content 51 is shown as being obtained through the local network 220, e.g., throughout, but this is not strictly necessary) and the media content (51, 51a) could be received, for example, through remote network connection. Subsequently (Fig. 3H), the media content 51 (51a), 216 will be retransmitted (either identically relayed, or in a processed form, e.g. with some portions suppressed) as content 218 to the secondary player devices 10b, lOd, according to their roles.
[0074] In the example of the Figs. 3C, 3D, 3E, 3F it is shown that a broadcast message 212 is sent only indicating the information 137 of being part of the virtual device 110 or not while the discriminating information is sent only subsequently in a response message 214. Notwithstanding, it may also be possible that the response message 214 is not sent at all and only CIN240904PEP-2025002689.DOCX 17 the broadcast message 212 is sent having, together with information 137, also the discriminating information directed and coded there.
[0075] Accordingly, while the cloud backend system 4 (in the case of election) does not need to care of which device is the virtual device primary device 10a, the transmission of the media content 51 (51a) may be uniquely performed by only the virtual device primary player device 10a, and this technical choice therefore greatly reduces the amount of payload to be received. The virtual device primary player device 10a will forward the media content to the virtual device secondary devices 10b, lOd according to their role.
[0076] In some cases, it is possible that the virtual device primary device 10a disappears or is anyhow not responsible of the transmission anymore or is too slow (e.g. the retransmissions of media content arrive with delay over a predetermined delay-threshold and / or arrive with error rate over a predetermined error-rate threshold). In that case, a new election may be performed by the player devices of the virtual device 110. The further virtual device primary election may be the same of the previous virtual device primary election. The further virtual device primary election may take into consideration at least discriminating information attributed to the sending player devices and / or the status of the sending player device, exactly like in the previous virtual device primary election. In some examples, the previous virtual device primary player device 10a may also participate to the further virtual device primary election, but in other cases it may be excluded. In some examples, the discriminating information provided by each virtual device primary player device 10 (10a, 10b, lOd) may be in updated version (i.e., it may contain information which has been obtained during the distributed playback, and may include, for example, an updated performance indicator, at least one stability value attributed to the sending player device, at least one new random value, and / or unique identifier of the sending player device). In alternative examples, the choice of the new virtual device primary player device may be inherent in the previous virtual device primary election: for example, there may be performed a rating of all the player devices 10 in the virtual device 110, in such a way that, when it is understood that the previous virtual device primary device 10a is not available anymore, then the second virtual device player device in the rating automatically takes over and start operating as a new virtual device primary device. CIN240904PEP-2025002689.DOCX 18
[0077] The virtual device 110 may be addressable independently, once the virtual device 110 is formed by the virtual device primary device 10a is elected. For example, the user application 3 may output, to the user 99, a user interface for inputting commands by the user 99 relating to the virtual device 110 as a unique entity. Therefore, it may be that the commands 31 and 32 are provided by the user application 3 to the assisting system 4 in respect of the playback of the virtual device 110, and not in respect of the playback of each device 10a, 10b, lOd which is part of the virtual device 110. It will be the assisting system 4 which will provide, for each device 10a, 10b, lOd part of the virtual device 110, the commands 34 and / or 38. In some cases, the assisting system 4 may address each device 10a, 10b, lOd independently, while in other cases the assisting system 4 may simply send the commands 34 and / or 38 to the virtual device primary device 10a, and the virtual device primary device 10a will forward the commands 34 and / or 38 to each of the virtual device secondary devices 10b, lOd.
[0078] Apart from the commands 34 and / or 38 generated from the user 99 (e.g. from commands 31 and 32), the player device 10 (lOa-lOd) may have a physical interface (e.g., buttons and / or a graphic user interface, GUI, etc.) to receive commands for user's command for playback event (e.g. "PLAY" "PAUSE", "STOP", "FAST FORWARD", "REWIND"), also fur user's command. In case of the player device 10 being the virtual device primary player device (10a), it may adopt the user's command for playback event, and transmit, to the virtual device secondary player devices (10b, lOd), the user's command for playback event. In the case of the player device 10 being a virtual device secondary player (10b, lOd), the device 10 may send a request to the virtual device primary player device 10a for accepting the user's command for playback event controlling. In case the virtual device primary player device 10a accepts the request, the virtual device primary player device 10a will send a command for playback event to the virtual device secondary player devices 10b, lOd. Hence, all the player devices 10a, 10b, lOd of the virtual device 110 will adopt the user's command for playback event. E.g. in case of command "STOP", all the virtual device player device 10a, 10b, lOd will stop the reproduction of the media content.
[0079] In some cases, however, the request based on commands from the device 10 (whether virtual device primary 10a or virtual device secondary 10b, lOd) shall be provided to the assisting system before being retransmitted to other devices 10. It may be the assisting system 4 that actually sends the commands even in this case, as commands 34 and / or 38. CIN240904PEP-2025002689.DOCX 19
[0080] In some cases, however, it may be possible to command urgent events to be adopted immediately (e.g., volume control). In this case, even if the player device is a virtual device secondary player device, it may notwithstanding control its hardware Tenderer unit 1 to adopt the command. Then the virtual device secondary player device may (in some examples) send a request to the virtual device primary player device 10a to adopt the urgent command. The primary player device 10a will adopt the urgent command, and will send a request with the urgent command to the remaining virtual device secondary player devices.
[0081] Notably, the virtual device primary player device 10a may synchronize the other player devices 10a, lOd of the virtual device 110. For example: the virtual device primary player device (10a), sends, to each secondary player device, synchronization information; each virtual device secondary player device (10b, lOd) receives the synchronization information and synchronizes its rendering to the received synchronization information.
[0082] The device hardware 1 of any of the devices 10a, 10b, 10c, lOd may be controlled through a HAL 11, which in turn may be controlled by control and / or hardware control and / or hardware events 26. The cloud backend system 4 (having knowledge of the particular version of the HAL 11) may therefore provide, to the virtual device primary player device 10a an address of the media content according to a format compatible with the particular version of the HAL 11. Therefore, there is the possibility that neither the virtual device primary player device 10a nor the virtual device secondary player device 10b, lOd receives application program interfaces, APIs, during the synchronized media playback. The advantage is considerable, because otherwise the virtual device primary player device 10a should also retransmit the APIs to the virtual device secondary player devices 10b and lOd of the virtual device 110, uselessly clogging the local connection 220. Notably, the cloud backend system 4 may therefore have knowledge of the HAL version of each player device 10, but does not need to care of which player device will be elected virtual device primary.
[0083] It is possible to apply the present technique to a video-wall application. The synchronized media playback may include a video wall playback formed by a virtual device 110 shown in Fig. 8V. Here, multiple, adjacent player devices 10 (one audio device 10a or "device 1", one CIN240904PEP-2025002689.DOCX 20 audio device 10b or "device 2", and nine audio devices lOv, indicated as "devices 3...11") of the same virtual device 110 cooperate to generate a video wall displaying a global video content, each player device displaying a portion of the global video content. Here we may have: a virtual device primary player device (10a), which in this case is an audio device, and receives the media content (51, 51a) of the global content from the content distribution network (5, 5a), and playsback its own channel of the audio content; and the virtual device primary player device (10a) suppresses the video content and a channel of the audio content playedback by itself and transmits the reduced -portion audio content to the to the other audio device 10b (which is a virtual device secondary device); suppresses the audio content and separates the global video content among a plurality of video content portions, and retransmit, to each player device (10) of the virtual device 110 a video content portion to be displayed synchronously.
[0084] In other examples, it may be one video device lOv which is the virtual device primary device, and retransmits the media stream 51 after having suppressed unused portions.
[0085] Another example in Fig. 8A only for audio showing a stereo pair playback, where the audio device 10a is the virtual device primary player device for left channel, and the audio device 10b is the virtual device secondary player device for right channel.
[0086] Another example in Fig. 8B of a surround playback, where devices 10-10g are used (e.g. 10a for front left, 10b for front center, 10c for rear left, lOd for front right, lOf for LFE, and 10g for rear right). The virtual device primary player device is device 10a.
[0087] In general terms, the drawings of Figs. 8A-8C can be imagined not only as schematizations of the virtual device 110, but also as screenshots llOu (e.g. provided by the user application 3 through the user equipment to the user 99) showing the devices 10 constituting the virtual device 110. For example, the user application 3, may have received template information from the assisting system 4, the template information providing the three there different setup options 110a, 110b, and 110c. The template information may assist the user 99 in defining a matching between the devices 10 and the roles which are needed for the setup option chosen for the virtual device 110. For example, in order to provide the stereo- CIN240904PEP-2025002689.DOCX 21 mixed video wall of Fig. 8C, the template information may assist the user in defining matchings between the plyer devices 10 (e.g. after having taken into account their conditions and / or capabilities) and the roles that are needed for the virtual device 110. For example, the template information may indicate the role that shall be had by each device of the particular setup, and assist the user application 3 to obtain (from the user 99) matchings between the devices 10 and the roles that they will have in the synchronized playback. However, the assisting system 4 and / or the user application 3 may prevent the user 99 from requesting matchings between roles requiring particular capabilities and / or conditions and player devices 10 which have not the required particular capabilities and / or conditions. In Fig. 8C it may be requested that the user 99 indicates which device has which role (e.g., each position in the video wall and in the audio rendering), thereby matching the devices (and their capabilities) with the roles required by the setup. However, the user 99 may be guided, by the template information as visualized in the displayed by the user application 3, to not select, for an audio rendering role, a video device lOv having no hardware audio rendering capabilities, and for a video rendering role an audio device 10a or 10b having no hardware video rendering capabilities. For example, the assisting system and / or the user application 3 may prevent the user 99 from providing selecting a device which has not the conditions and / or the capabilities required for a role.
[0088] In some examples, the matching between the devices 10 and the roles are performed by the assisting system 4 based on the setup chosen by the user 99 among a choice between the different possible setups (e.g. those in Figs. 8A-8C) as indicated in the template information sent by the assisting system.
[0089] Once the virtual device 110 is generated, it may operate as a normal player device. The virtual device 110 may be addressed as a single device by the assisting system (which may simply send commands 34 and / or 38 to the virtual device primary player device 10a).
[0090] An option is to form groups. A group may be obtained as a set of devices and / or virtual devices which perform a synchronized playback. If there is a virtual device (e.g. 110) in the group, then the primary device 10a of the virtual device 110 will represent the virtual device 110 as an element of the group. Fig. 6 shows a group 210 formed by a device lOf, a device CIN240904PEP-2025002689.DOCX 22
[0091] 10g, and the virtual device 110 (e.g. the same of Figs. 2 and 3A-3G), while the player device 10c is excluded. The group 210 operates in such a way of having a group primary player device (which in this case is the device lOf) a group secondary player device 10g the virtual device 110, which in this case is a group secondary virtual device.
[0092] It is to be noted that in a group there is a group primary player device and a at least one group secondary player device. The group primary player device 10g is elected in the same way as the virtual device primary player device, through a group primary election which is performed analogously to the virtual device primary election. When the group 210 is formed, and the group primary election is performed, then the group primary player device lOf is elected and the group primary player device lOf will receive the media content 51 alone, retransmitting the media content to the group secondary player devices and virtual devices. In case of group primary election, the virtual device primary player device 10a will participate to the group primary election, but the virtual device secondary player devices 10b, lOd will not participate to the group primary election. In some examples, the retransmitted media content 51 does not change for different roles, or more in general the role, in the group 210, may not be taken into consideration.
[0093] The group primary device lOf is the device which receives, from remote, the media stream 51 and retransmits it to the group secondary devices 10 and virtual devices 110. In some cases, the retransmissions may be simply relays of the same media content 51, without any modification of the media content.
[0094] The group 210 may provide a semantically unitary media scene (e.g. video and / or audio scene) to a user. The player devices 10 (lOf-lOg) and virtual devices 110 may therefore participate to the synchronized media playback.
[0095] As will be explained, the group 210 may include the group primary device (lOf) and one or a plurality of secondary devices or virtual devices. It is preferred that only the group primary player device 10a receives the media content 51 from the CDN 5 (5a) and after that retransmits the media content through the local connection 220 by sending media frames 218. In some cases, the group primary player device lOf may be chosen by group primary election between the devices lOf, 10g, and the virtual device 110. CIN240904PEP-2025002689.DOCX 23
[0096] During the distributed synchronized playback of the group 210, the player devices lOf, 10g, and the virtual device 110 may perform a synchronized rendering. The group primary device (lOf) may provide feedback to the cloud backend system 4 (which, in some examples, may in turn send it to the user application 3) indicating the state of the rendering (e.g., indicating timing information, such as which portion of the media content is being rendered).
[0097] In the case of group 210 being active for receiving a media content 51, if the virtual device 110 is a group secondary device, then the virtual device primary player device 10a may refrain from receiving the media content from remote, and may receive the media content 51 from the group primary device lOf, and then redistribute the media content 51 to the media content secondary player devices 10b, lOd according to their roles.
[0098] Elements on how to choose the virtual group player device 10a are provided now. Each of the devices lOf, 10g, and the virtual device 110 may be connected with the cloud backend system 4 (assisting system), so as to receive commands 34 and 38 from the cloud backend 4 for the entire group.
[0099] Fig. 7A shows an example of provision of the formation of the group 210. In this case, information 37 of being part of the group 210 is provided to the selected player devices lOf, 10g and the virtual device 110 (the selection may be made by the user 99 through the user application 3 and / or by the assisting system 3). As shown in Fig. 7A, group identifier information 37a is provided to player devices lOf, 10g and the virtual device 110 (it is imagined that the information arrives in through the local network 220, i.e., that the content control 38 is transmitted through the local network 220 (e.g., through a router) but this is in principle unnecessary). The information 37 of being part of a group may be sent, for example, as addressed data packets (e.g. data packets which are uniquely sent to player devices lOf, 10g and the player device 10a on behalf of the virtual device 110 while the data packets are not sent to the excluded player device 10c and to the player devices 10b and lOd, which are already part of the virtual device 110, are excluded). Therefore, each of the player devices lOf, 10g and the virtual device 110 determines that it shall be part of the group 210. Further, the information 37 of being part of the group 210 may include group identifier information 37a. The group identifier information 137a may be decided by the user application 3 (e.g. by the user 99) and / or by the assisting system 4 so as to match CIN240904PEP-2025002689.DOCX 24 the capabilities and / or conditions of each device 10a, 10b, lOd. Fig. 7B does not show the device 10c which has been excluded from the group 210. The device 10c can be in principle part of another group (not shown) together with other player devices (not shown) (e.g. by following a different content command 38). As shown in Fig. 7C, when receiving the information 37 of being part of the group 210, each player device lOf, 10g and each virtual device HOin the group 210 sends a broadcast message 212. The broadcast message 212 may provide an indication of which group the sending player device lOf, 10g and the virtual device 110 is part of (e.g., each broadcast message 212 may have a field in which the information 37 is written, indicating which group the device or virtual device is part of). As shown in Fig. 7D, each player device lOf, 10g and the virtual device 110 also receives the broadcast messages 212 from the other player devices and virtual player devices in the same group 210. For example, the player device lOf reads the broadcast messages 212 sent from the virtual device 110 and the player device 10g; the virtual player device 110 receives the broadcast messages 212 from the player devices lOf and 10g and so on. It will be shown that the broadcast message 212 sent by each player device and virtual player device may also include (apart from the identification of the player device from which it originates) discriminating information attributed to the sending player device and / or the status of the sending player device or sending virtual player device. The discriminating information may include at least one performance indicator of the sending player device or virtual player device which sends the broadcast information 212. The stability indicator may reflect the stability of the player device or virtual player device over time. The more the unexpected reboots, crashes or corrupted receive frames, the lower the stability will be. The discriminating information may include a performance indicator. The performance indicator may include an amount of RAM, CPU, available flash storage space, availability of hardware decoders and so on. The discriminating information may include the random value (e.g., a random integer value). The discriminating information may include at least a unique identifier of the sending player device. In practice, in some examples the discriminating information for the virtual device primary election may be the same as the discriminating information for the group primary election.
[0100] The player devices and virtual player devices therefore elect the virtual device primary player device lOf by examining the discriminating information of all the player devices in the virtual device 110. For example, a rating may be defined for each player device and each virtual device 110 based on the performance indicator, stability value, random value CIN240904PEP-2025002689.DOCX 25 and unique identifier and defining the one which has the highest rating. Accordingly, the group primary player device may be elected implicitly without any device deciding which shall be the group primary player device lOf. In particular, the cloud backhand system 4 could have no interest in deciding which will be the group primary player device 10a, whose decision will only be defined by the devices in the virtual device 110.
[0101] The group primary player device lOf, once elected, will be the one connected with the CDN 5 (5a) for receiving the media content 51 (also here the media content 51 is shown as being obtained through the local network 220, e.g., throughout, but this is not strictly necessary) and the media content (51, 51a) could be received, for example, through remote network connection. Subsequently (Fig. 3H), the media content 51 (51a) will be retransmitted (either identically relayed, or in a processed form, e.g. with some portions suppressed) as content 218 to the group secondary player devices and virtual devices. Notably, the virtual device primary device 10a of the virtual device 110, having received the media content from the group primary player device lOf, will retransmit the media content to the virtual device secondary player devices 10b, lOd according to their roles.
[0102] Accordingly, while the cloud backend system 4 (in the case of election) does not need to care of which device is the group primary device lOf, the transmission of the media content 51 (51a) may be uniquely performed by only the group primary player device lOf, and this technical choice also reduces the amount of payload to be received. The group primary player device lOf will forward the media content to the group secondary devices and secondary virtual devices.
[0103] It is also possible to perform a new group primary device election, which would be the same of the virtual device primary election, and which is therefore not repeated here.
[0104] It is to be noted that the information 37 of being part of a group may be the same of a particular content information (e.g. a queue information 37), and the identifier information 37a may be the same of a queue information.
[0105] Fig. 8D shows an example of an application of the groups for the vide-wall of Fig. 8C. In Fig. 8D, however, there may be defined the group 210 as having a first virtual device 110' (for video rendering) and a second virtual device 110" (for audio rendering). Here, there is CIN240904PEP-2025002689.DOCX 26 a first virtual device primary player device for the first virtual device 110' (elected by virtual device primary election among the devices lOv) and a second virtual device primary player device for the second virtual device 110" (elected by virtual device primary election among the devices 10a and 10b), as well as a group primary virtual device elected between the two virtual devices 110' and 110" through group primary election. Of course, the stream 51 is actually received by the virtual device primary player device of the group primary virtual device and retransmitted to the virtual device primary player device of the group secondary virtual device. Both the two virtual device primary player devices will retransmit, to the virtual device secondary player devices of the respective virtual device, the media stream, e.g. processed according to the role of each virtual device secondary player device.
[0106] It is not necessary to have the groups. In the case in which the groups are not defined, the information 37 may be understood as simply content retrieving information (e.g. address in the CDN 5, such as URL address of the content to be downloaded).
[0107] In some examples, the stream for a group is not processed, but only replayed, by the group primary player device, while the virtual device primary player device may process the received stream according to the roles assigned to each virtual device secondary device.
[0108] Examples of Figs. 4A and 4B
[0109] Fig. 4A and 4B illustrate a procedure showing a sequence of steps / message between the user 99, the user application 3, the cloud backend 4, the media runtime device software 2 (e.g. in any of the player devices 10a, 10b, 10c, lOd) and the content service (CDN) 5 (5a) for playing content with the cloud ecosystem. The playing of content of the uploading of content within the ecosystems involves of the corresponding participants of the ecosystem such as device makers, content services, users. Furthermore, it is to be noted that the heavy-weight data or content is retrieved directly by the media runtime device, while the cloud backend only handles the playback commands. The data of authorized devices 10 (e.g. 10a, 10b, 10c, lOd) and authorized content services for a user are stored and retrieved from databases that are part of the cloud backend. These databases contain all data that is necessary for authorized access to a content service, where tokens are needed for access. When, in the Fig. 4A, Fig. 4B diagram such data is looked-up in the cloud backend, it is retrieved from a corresponding database 208. CIN240904PEP-2025002689.DOCX 27
[0110] Furthermore, it is to be noted that the illustration in Fig. 4A makes clear that each message, such as message 700, between the user application 3 and the cloud backend 4 refers to two actions of the entities involved. The first action is that the user application sends out the message 700 via the cloud backend interface 304, and the second action performed by the cloud backend 4 is that the cloud backend 4 receives the message 700 via the application interface 206.
[0111] Subsequently, Fig. 4A and Fig. 4B are discussed in more detail.
[0112] In the action 602, the user 99 opens the user application 3 running on the user application apparatus. In step 603, the user logs into the user application and, in reply to this logging into the app, the user authenticates itself via the message 700 from the user application to the cloud backend. Then, in order to further use the cloud backend, the user receives a token (which is an authenticating information) including the user's permissions and the corresponding user ID in message 605. The user token is used in all calls to the cloud backend to identify the user and to check if access is allowed for the requested function for the user.
[0113] Then, in order to finally come to a playing of a media content, the user application sends a request for a list of content services illustrated at 702. All content services that are authorized in the cloud backend are returned. Each content service is either authorized or not authorized by the user. Hence, due to the fact that the cloud backend replies to this message 702, with a full selection of content services authorized in the ecosystem, the selection what is valid for the specific user is performed in step 704. Here, the cloud backend checks, which content services are actually authorized for the user.
[0114] Then, for each identified content service, it is checked, if the user is allowed to use the content service, if the user is not allowed, the content service is removed from the result. In a step 708, it is checked, whether the content service allows the user. Probably, the user is denied for some reason, for example by not paying a bill, etc. This is done in step 708. Then, in step 710, the cloud backend returns a list of allowed content services to the user application. CIN240904PEP-2025002689.DOCX 28
[0115] This list is presented to the user via the user interface in the user application and the user then selects a content service from the list provided 710 in the step 712. Then, based on the selected content service, the user application 3 requests a list of content in message 714.
[0116] In response to that, the cloud backend 4 looks up, whether credentials (which are authenticating information) for this content services are stored in the cloud backend. This is done in step 716, and when such user credentials for the requested content service are found, an authentication message 718 is sent from the cloud backend to the content service via the communication interface 204 of the cloud backend. The result message 720 is illustrated in Fig. 4A and the result message can, for example, comprise a certain token (which is an authenticating information), from the content service, or any other positive authentication result.
[0117] In response to the positive result 720, the cloud backend requests content metadata in a message 722 from the content service 5 (5a), and the content service 5 (5a) sends back the requested content metadata in message 724. The content metadata in message 724 may also be authenticating information, which is provided to the content service 5 to receive, under authorization, the content 51 (51a).
[0118] This list of content is then forwarded from the cloud backend to the user application in step 725 for display to the user 99 via the user interface 302.
[0119] The user 99 selects a content for replay as shown in action 726 and, then, the user application requests allowed devices 10 (e.g. 10a, 10b, 10c, lOd) for the selected content from the cloud backend.
[0120] As illustrated in action 730, a list of devices 10 (e.g. 10a, 10b, 10c, lOd) that are authorized by the user is looked-up and, then, processed by means of steps 732 to 742.
[0121] This procedure represents the check, if a certain device 10 (e.g. 10a, 10b, 10c, lOd) is allowed from the list of devices 10 (e.g. 10a, 10b, 10c, lOd) that have been originally authorized by the user. Any negative answer in the list consisting of item 732 to 742 will exclude a corresponding device under consideration from the result. CIN240904PEP-2025002689.DOCX 29
[0122] Step 732 starts the processing sequence. However, the sequence of steps between 732 to 742 is arbitrary and any other sequence also provides a useful result.
[0123] Instead 734, it is determined, whether the content service under consideration allows the device brand of the user device. The information for answering this question is located in certain database 208 in the backend.
[0124] In step 736, it is determined, whether the device maker allows the content service 5 under consideration.
[0125] This is also determined from a certain database.
[0126] Furthermore, in step 738, it is determined whether the device maker allows the playback of the content that has been selected to play in step 726.
[0127] Furthermore, in step 740, it is determined, whether the user actually allows to playback the content on the device. This procedure is useful, when, for example, a certain profile of allowed content has been installed in the cloud backend, for example for the purpose of children control.
[0128] Now, it can be that the child acts as a certain user 99, and therefore, it can easily be that the child requests a content to play that, however, is not allowed to be played back on the device as defined earlier when establishing the certain profile in a library of the cloud backend.
[0129] Finally, it is determined whether the actual user device 10 (e.g. 10a, 10b, 10c, lOd) under consideration is online and ready.
[0130] Only when all the answers in the list are positively answered, a device 10 (e.g. 10a, 10b, 10c, lOd) survives in the list of allowed devices that is referred back from the cloud backend 4 to the user application via message 744. CIN240904PEP-2025002689.DOCX 30
[0131] Then, the user 99 (e.g. through the user application 3) may reply with a message 745 (which may include, or be an instantiation of, the control device setup commands 32 in Fig. 1) for selecting the devices (e.g. 10a, 10b, lOd which could form the virtual device 110 as requested by command 38) from the list of allowed devices (e.g. the list including 10a, 10b, 10c, lOd). Then, the user 99 (e.g. through the user application 3) may start the playback of the selected content on the virtual device 110 of devices via message 746 (which may include, or be an instantiation of, playback command request 31 in Fig. 1). This user action results in the message(s) 31, 32 from the user application 3 to the cloud backend 4 that requests the playback for the selected content on the selected devices (e.g. 10a, 10b, lOd, intended to form the virtual device 110), where this message is illustrated at 748 in Fig 4B.
[0132] At this point, the procedure of Figs. 3A-3F may start. The backend 4 (e.g. the device manager 35) may send (action 749a) content retrieval information 37, 37a as 36 and 38 to the selected devices 10a, 10b, lOd, as well as information 137 of being part of the virtual device 110 and the role 137a of each player device as defined by backend 4. The optional election 749b may occur (or in alternative, the selection may be made by the backend 4, which may send the selection result in action 749a. Optionally a message 749c (election result) from one of the player devices 10 (e.g. from the elected primary device 10a, but it could also be sent by any of devices 10b, 10c, lOd, according to the invention) indicates that the device 10a has been elected as primary.
[0133] The content metadata (including authorization information) may be sent (750) from the cloud backend 4 to the media runtime device software 2 on the primary device 10a (while the device 10c excluded from the virtual device 110 and the secondary devices 10a and lOd may be excluded from receiving the content metadata with the authorization information). This content metadata 750 can, for example, be or include a playable URL or any other data, by which the media runtime device software 2 running on the user devices 10 (10a, 10b, lOd) can request the content via a message 752 from the user device 10a via the connection to the typically remote data instances illustrated at 51 in Fig. 1. The request 752 may include or be accompanied by authentication information (e.g., obtained in message 748 from the backend 4) which allows the CDN 5 to determine that the device 10a is authorized to receive the content. Then, as soon as the playable URL, for example, is received by the remote data instance 5, the remote data instance 5 streams back the content as shown in message 756 and this takes place via media interface of the user device 10a CIN240904PEP-2025002689.DOCX 31 and the content is replayed as illustrated at 754 (of course, the primary 10a forwards the content 51 as 218 to the secondaries, like in Figs. 3G and 3H, according to their roles). The authenticating information from the primary player device 10a to the CDN 5 may be verified by the CDN 5. Or, the unique address of the primary player device 10a may be verified by the CDN 5, the unique address of the primary player device 10a serving as authenticating information.
[0134] Fig. 4B additionally illustrates the procedures that are done, when the user 99 wishes to control the playback via message 760. This control can, for example, be a pause action, a stop action, etc. As soon as the user application 3 has received a message from the user such as message 760, the user application 3 replies with a request control playback message 762 (which may include or embody the playback control command 31 of Fig. 1) to the cloud backend 4. Only when the cloud backend 4 confirms that such a procedure is allowed, where this check is not illustrated in Fig. 4B, the cloud backend 4 requests control playback via message 764 (which may include or embody the playback control 38) from the media runtime device software 2 of the primary device 10a. As soon as the control message 764 (e.g. 38) is received by the user device via the user application 3, the user application 3may react with an execution of the playback control illustrated at 766 and report this executed control via step 768 back to the cloud backend 4 and a corresponding confirmation is sent from the cloud backend 4 back to the user application via message 770. As explained above and below, however, commands may also be provided directly on the primary playback device 10a (which are then forwarded to the secondary devices 10b and lOd) and / or on the secondary devices (10b, lOd), which are notwithstanding forwarded to the primary device 10a).
[0135] It is to be emphasized that the order of the selection of the content service 712, the selection of the content to play 726, and the selection of the devices from the list of allowed device 745 can also be done in a different order. Furthermore, in case of only a single content service and only a single user device, several checks and messages can be omitted in order to nevertheless implement the full control of the cloud backend 4 over the user device, which the media runtime device software 2 is installed.
[0136] Examples of Figs. 9A and 9B CIN240904PEP-2025002689.DOCX 32
[0137] Figs. 9A and 9B show an example of operation with the virtual device 110. The user 99 may open the application at 901. Then, the user may log (902) into the user application. The application 3 may authenticate the user at 904. The assisting system 904 may provide authenticating information at 904 to the user application 3, after the identity of user has been checked and / or his / her entitlement to the service has been checked. At 905 the user is therefore logged in.
[0138] In order to create the virtual device 110, the user 99 may request the creation at 910. At 911, the user application 3 may request to the assisting system 4 the list of virtual devices and / or templates. At 912, the backend system 4 checks whether the user 99 is entitled to the creation of the virtual device 110. At 913 the assisting system 4 provides template information (e.g. indicating the different setups options 110a, 110b, 110c in Figs. 8A-8C). The options may be displayed, by the user equipment on which the user application 3 executes. At 914, the user 99 user selects the preferred setup option. At 915, the user application 3 may display the selected setup option(i.e. the selected virtual template).
[0139] The cycle 920 is iterated for each role (which may appear as a slot in the user interface) to be assigned to a player device having the capabilities and / or conditions for having that role. At 921, the role (slot) to be assigned is selected. At 921, a list of allowed and supported player devices 10 for the selected role (slot) is requested by the user application 3 to the assisting system 4. At 923, the assisting system 3 search for player devices 10 which the user 99 is allowed to use.
[0140] The cycle 930 is repeated by the assisting system 4 for each allowed player device 10. At 932, it is optionally checked whether the player device is allowed to be part of a virtual device by its maker. At 933, it is optionally checked whether the player device is allowed to be part of a virtual device by its user 99. At 934, it is checked whether the capabilities and / or conditions required by the role are complaint with the capabilities and / or conditions of the plyer device. After the iterations of the cycle 930, the assisting system 4 may send at 937 to the user application 3 a list of allowed and supported playback devices. At 938 the user 99 may select a matching between the role and the allowed player device 10. When all the roles of the selected setup option are taken, the assisting system may start the procedure of generating the virtual device 110 like in figs. 3A-3H (e.g. triggered by sending the information 137 of being part of the virtual device 110 to the plyer devices 10a, 10b, and lOd). CIN240904PEP-2025002689.DOCX 33
[0141] It is now explained a technique according to which the primary player device 10a can synchronize the playback between the whole virtual device 110. Notably, the backend 4 can be completely agnostic on the synchronization.
[0142] At first, the primary player device 10a may send, periodically, in the local connection 220, a beacon packet having a clock signal data field in which there is encoded information on the current time according to the primary player device 10a (the current time may include, for example, the current time as known by the primary player device 10a from its internal clock, or the playback time which is an incremental time of the media stream as authored currently played back by the device hardware 1 of the primary player device 10a). Accordingly, each secondary player device (10b, lOd) has knowledge of the current time at the primary player device 10a. There may be provided solutions for also taking into account the delay in the transmission of the beacon signal (for example, the secondary player device 10b, lOd may measure the time instant at which the beacon packet is received and add it to the time indicated in the clock signal data field in the beacon packet). In any case, each secondary player device (10b, lOd) may resynchronize to the clock of the primary player device 10a.
[0143] Moreover, each secondary player device (10b, lOd) may transmit an acknowledgment packet in which it replies to the beacon packet. Each secondary player device (10b, lOd) may also indicate, in a time stamp data field of the acknowledgement packet, the playback time (i.e. incremental time of the media stream as authored and which is currently played back by the device hardware 1 of itself, secondary player device 10b or lOd) which is currently rendered (e.g., an indication of which second of film is currently displayed). Accordingly, the primary player device 10a has the knowledge of the latency impairing the playback at each secondary player device (10b, lOd). The primary player device 10a may calculate the delay of each secondary player device (10b, lOd), by keeping into account the time necessary for receiving the acknowledgement packet after having transmitted the beacon packet: for example, the time delay from the primary player device's transmission of the beacon packet and the DP primary player device's reception of the acknowledgment packet from each secondary player device (10b, lOd) may permit to estimate the actual time instant at which a particular media content (e.g. a particular photogram and / or a CIN240904PEP-2025002689.DOCX 34 particular audio frame) is rendered by each secondary player device (10b, lOd). The latency may be estimated, for example, as the time delay for receiving the acknowledgment packet divided by two, thus inferring the latency of each secondary player device (10b, lOd) in the virtual device 110. By inferring the time instant in which each secondary player device (10b, lOd) has actually rendered a particular media content, the primary player device 10a may also synchronize all the secondary player devices (10b, lOd) and its device hardware 1. For example, the primary player device 10a may indicate, in one field in the beacon packet, also the latency of each of the secondary player devices (10b, lOd). Therefore, each secondary player device (10b, lOd) will know its own latency and will resynchronize the rendering of its media content to the timing of the primary player device 10a (e.g. causing its playback time to coincide with the playback time of the primary device 10a by subtracting the latency from the currently rendered playback time). The primary player device 10a may also transmit a presentation time stamp encoded in a presentation time stamp field of the beacon packet, indicating its playback time, i.e. the timing instant which is currently played back by its device hardware 1 (or anyway, the scheduled time which the primary player device 10a wants a particular media content to be rendered). The primary player device 10a may control the rendering of the media content based on the latency of the secondary player devices (10b, lOd): for example, the primary player device 10a may reschedule the rendering of the media content base on the maximum latency among the secondary player device (10b, lOd), so as to permit all the device hardware 1 to be synchronous.
[0144] Alterative techniques may be taken into consideration.
[0145] More in general, it may be required that:
[0146] 1) The primary player device 10a provides, to the secondary player devices 10b, lOd, information on its own timing.
[0147] 2) Each secondary player device 10b, lOd synchronizes to the timing of the primary player device 10a.
[0148] For example, it may be required that:
[0149] 1) The primary player device 10a provides (e.g. in the beacon message), to the secondary player devices 10b, lOd, information on its current playback time (e.g. which time instant of the media stream as authored is currently played back). CIN240904PEP-2025002689.DOCX 35
[0150] 2) Each secondary player device 10b, lOd provides, to the primary player device 10a, information on its current playback time.
[0151] 3) The primary player device 10a calculates the latency of each secondary device 10b, lOd, and indicates to each of them, in a subsequent message, the latency information of each secondary device 10b, lOd, so that each secondary device 10b, lOd changes the playback time to be rendered to be its previous playback time in accordance with the latency.
[0152] For example, it may be that:
[0153] 1) The primary player device 10a provides (e.g. in the beacon message), to the secondary player devices 10b, lOd, information on its own timing (e.g. information on the playback time currently rendered and / or information on its own internal time from its own clock).
[0154] 2) Each secondary player device 10b, lOd provides information on its own timing (e.g. information on the playback time currently rendered).
[0155] 3) From the information on each own timing of the secondary player devices, the primary player device 10a obtains the knowledge on the latency of each secondary player device, and provides resynchronization information (e.g., to each of the secondary player devices.
[0156] For example, it may be that:
[0157] 1) The primary player device 10a provides (e.g. in the beacon message), to the secondary player devices 10b, lOd, information on its own timing (e.g. information on the playback time currently rendered and / or information on its own internal time from its own clock).
[0158] 2) Each secondary player device 10b, lOd provides information on its own timing (e.g. information on the playback time currently rendered).
[0159] 3) From the information on each own timing of the secondary player devices, the primary player device 10a obtains the knowledge on the latency of the each secondary player device 10b, lOd in the virtual device 110, and in particular of the latest of the secondary player device 10b, lOd in the virtual device 110.
[0160] 4) Subsequently, the primary player device 10a provides resynchronization information to each of the secondary player devices indicating the latency that has to be observed to each of the secondary devices 10b, lOd in the virtual device 110, to be synchronized with each other
[0161] For example, it may be: CIN240904PEP-2025002689.DOCX 36 the primary player device 10a sends information on its own timing to each secondary player device (10b, lOd) in the virtual device 110 (the information on its own timing including either information on the playback time which is currently rendered or the internal timing as obtained from its internal clock), each secondary player device (10b, lOd) in the virtual device 110 sends information on the playback time which is currently rendered by itself (the secondary player device), thereby providing knowledge on the latency impairing itself (the secondary player device) between the playback time at the primary player device and the playback time at each secondary player device; and the primary player device 10a sends resynchronization information to each secondary player device in the virtual device 110, the resynchronization information indicating the latency impairing the secondary player device; and so that each resynchronization information resynchronizes its internal clock or its playback time based on the latency.
[0162] More in the specific, it may be: the primary player device 10a sends information on its own timing (e.g. including either information on the playback time which is currently rendered or the internal timing as obtained from the primary device's internal clock) to each secondary player device (10b, lOd) in the virtual device 110, each secondary player device in the virtual device 110 provides information on the playback time which is currently rendered by itself (the secondary player device), thereby obtaining knowledge on the latency impairing each secondary player device in the virtual device 110 between the playback time at the primary player device and the playback time at each secondary player device, so that the primary player device 10a determines the latest secondary player devices among the latest secondary devices in the virtual device 110; and the primary player device 10a sends resynchronization information to each secondary player device in the virtual device 110 indicating additional latency to be observed by the addressed secondary device, so that the addressed secondary player device resynchronizes to the secondary player device in the virtual device 110 which is impaired by highest latency; and both the primary player device 10a and the secondary player devices (10b, lOd) resynchronize to the latest secondary device by synchronizing the playback time by inserting the additional latency. CIN240904PEP-2025002689.DOCX 37
[0163] Example of Fiq. 5
[0164] As illustrated in Fig. 3, a device maker 500 is performing several procedures to generate a cloud service-ready user device.
[0165] Particularly, the device maker 500 may produce a hardware device with the hardware device (e.g. 1 ) and, then, as illustrated in item 501 , put the media runtime device software 2 on the device 1 . Furthermore, as illustrated in item 502 a set of unique device IDs is acquired and, then, as illustrated in step 503, the device IDs are securely put on the device, one device ID per device.
[0166] Furthermore, the device manufacturers create and configure a device descriptor as illustrated in item 504 and, additionally, set rules for allowing or disallowing certain media apps as illustrated in 505.
[0167] In an embodiment, these rules are stored in the cloud backend 4 and, particularly, in the certain libraries or databases of the cloud backend.
[0168] Examples of Figs. 10A-10D
[0169] Fig. 10A illustrates a user device 100 (indicated above as 10, 10a) comprising a device hardware 102 (indicated above also as 1) and a media runtime device software . This media runtime device software is running on the user device and is configured for controlling the device hardware as illustrated by the connection between block 2 and 102. The media runtime device software comprises a control interface 112 to a cloud backend in order to exchange messages between the media runtime device software in the device and the cloud backend. The user device comprises a media interface to a typically remote data instance for connecting to a remote data instance typically via a playable URL (unique resource locator) in order to then obtain a stream of media data from the remote data instance. Alternatively, the media interface can also send a request for a URL to the remote data instance and, then, receives this data from the data instance via the media interface and, then, an upload from the user device 100 to the remote data instance can be performed when the data instance is a data sink such as the YouTube platform or the TikTok platform. The data that are uploaded via the media interface 114 are typically data acquired by the user device hardware, which is, for example, a camera or a microphone. CIN240904PEP-2025002689.DOCX 38
[0170] For replaying a data stream received via the media interface 114, the user device typically has a display or speakers or both. Particularly, the media runtime device software 2 is configured to receive data for controlling the device hardware via the control interface 112 and to receive or transmit content via the media interface 114. Thus, it is made sure that the high volume data traffic only takes place via the media interface 114, and this high-volume traffic is kept out from the cloud backend, since the media interface 114 is not interfacing with the cloud backend.
[0171] In an embodiment, the media runtime device software is configured to receive, via the control interface, metadata comprising information identifying the data instance under consideration and an access information such as a playable URL combining the metadata and the identification of the data instance to be accessed. Based on this information, the media runtime device software is configured to access the data instance using the received access information via the media interface 114.
[0172] In one embodiment, the data instance is a content provider. In such an implementation, the media runtime device software 2 is configured to receive playback metadata identifying an intended content data for playback and the access information via the control interface 112, and to send the playback metadata to the content provider via the media interface 114 and to receive the content for playback from the content provider via the media interface 114.
[0173] In the embodiment, the user device comprises a hardware abstraction (HAL) layer 11 interfacing the device hardware on the one hand and the media runtime device software 2 on the other hand. The hardware abstraction layer 11 is configured for detecting a playback event. The media runtime device software is configured to receive the playback event from the hardware abstraction layer 11 via control line 26. However, this playback event is not immediately executed, but is forwarded to the cloud backend via the control interface 112. When a confirmation on the playback event from the cloud backend is received via the control interface 112, then the hardware abstraction layer 11 is instructed by the media runtime device software 2 to perform the playback event. However, when no such confirmation for the playback event is received from the cloud backend or when an explicit “action forbidden” message is received from the cloud backend, then the playback event is not executed.
[0174] Thus, it is made sure that the content provider has full control not only on the fact that an unintended decryption is forbidden, but also how the media data is handled in the user device. CIN240904PEP-2025002689.DOCX 39
[0175] Nevertheless, in order to give the user of the user device as much control as possible, a pressing of a certain switch on the user device which is typically, a start button, a stop button, a fast forward or fast rewind or pausing button, is nevertheless detected, but not immediately executed as is done, in contrast, in prior art devices. Instead, this hardware event is forwarded to the cloud backend and only when the cloud backend confirms this event, then this event is actually executed.
[0176] The user device comprises a secure storage for storing a unique device ID which is required for identifying the user device with respect to the cloud backend. The device hardware is also configured to only replay content identified via a command received via the control interface and received via the media interface. In a further embodiment, the media runtime device software or “SDK kit” is configured to also have a web browser functionality 21 , a media player functionality 22, and a content decryption functionality 28. In a further embodiment, the media runtime device software 2 is configured to receive a playable unique resource locator (URL) from the control interface, and the user device then interfaces with the specific remote data instance, from which the playable URL is coming from via the media interface to stream media data located at the playable URL at the content provider via the media interface 114. Other ways to access a data source or data sink platform different from via a playable URL can be performed as well.
[0177] Fig. 10B illustrates an implementation of the cloud backend 200 that has a user device interface 202 for interfacing with the user device such as the user device 100 of Fig. 10A. The cloud backend comprises a communication interface for interfacing with a typically remote data instance such as any one of the data instances indicated at 400 in Fig. 10D, where individual data instances 405 are illustrated. These remote data instances also correspond to the media CDN 5 of Fig. 1 .
[0178] The cloud backend 4 (200) is configured to provide messages to the user device interface for controlling the user device to handle media data and to receiving messages from the user device via the user device interface 202. The cloud backend is configured to communicate to the data instance via the communication interface 204 regarding an authorization of a user of the user device, where this communication comprises receiving data from the data instance or sending data to the data instance. However, this data does not comprise media data to be downloaded or to be uploaded from the user device to the data instances. The cloud backend comprises an application interface 206 for interfacing with a management application such as the user application 300 illustrated in Fig. 10C or 10D. CIN240904PEP-2025002689.DOCX 40
[0179] The cloud backend 200 (4) additionally comprises, for performing the corresponding tasks, libraries / data bases 208 that can be distributed in the cloud or that can also be located on a single server. The cloud backend also comprises a processing engine 210’ that can also be a distributed processing engine or a concentrated processing engine. The playback / up- load control of the user device is performed by the cloud backend typically without high- volume traffic in the cloud backend. Only control data is communicated within the cloud backend and control data is received via the cloud backend interfaces 202, 204, 206 and corresponding control data is output via these interfaces as well.
[0180] Fig. 10C illustrates an apparatus 300 (e.g. using user application 3) for performing a user application which comprises a user interface 302 which is exemplarily a human machine interface such as a touch screen interface or a speech interface in order to provide messages to a user or to receive messages from a user. The apparatus comprises a cloud backend interface 304 for communicating with a cloud backend such as the cloud backend of Fig. 10B or the corresponding elements located in the cloud backend in Fig. 1 comprising a playback management element, device queues, or a device management element 35 for example.
[0181] The apparatus is configured to communicate, via the user interface 302, to receive user commands regarding a selection of one or more data instances and a selection of content to be sent to a data instance or to be retrieved from a data instance and to display messages for the user, to communicate via the cloud backend interface 304 regarding the selection of the one or more viewed data instances to be accessed by the user devices, and to communicate via the cloud backend interface regarding the selected content for sending from the user device to a corresponding data instance or for sending from the data instance to the user device. Hence, the cloud backend interface 304 cares for the selection of data instances for the selection and control of user devices, and for the selection of media content for the upload and download.
[0182] Fig. 10D illustrate an overview of the cloud ecosystem and how the individual interfaces are connected to each other in the four greater “entities”:
[0183] The user device 100 (above indicated as 10, 10a, 10b...) with the media runtime device software 2. The user application apparatus 300 (carrying the user application 3) with the user interface 302 on the one hand and the cloud backend interface on the other hand. CIN240904PEP-2025002689.DOCX 41
[0184] The remote data instances 405, with the high volume data connection 51 between the media interface 114 of the user device 100 and the media interface of the corresponding data instances.
[0185] The cloud backend for communicating with the remote data instances, the user device 100 and the user application apparatus 300.
[0186] The user application apparatus 300 for communicating with the cloud backend and the user herself of himself.
[0187] In an embodiment, the user device is responsible for playback of content from the remote data instances, and this device, in a minimum implementation, only consists of logic to ensure this capability and, additionally, has the ability to display the playback state in case of a screen connected to the user device or integrated in the user device. The user device only reacts and performs actions based on commands received from the cloud backend via the control interface 112 and there is no local ability to change what is being played. The user device only operates in conjunction with the cloud backend except for already cached or pre-fetched content which will continue to play even without the connection to the cloud backend. Any buttons included in the hardware of the user device can remain active, but not active in the sense that a command implemented by a button is immediately executed, but only active in the sense that the command is detected, forwarded to the cloud backend, confirmed by the cloud backend and only when the confirmation is there, executed in the hardware of the user device. The user device supports all playback capabilities provided by the media runtime device software including playback of DRM protected content and synchronized playback.
[0188] Devices on the same local network can be used in a group playback leveraging synchronized playback, where the synchronization and distribution of content is performed locally. This is realized by electing one device as the primary device, this primary device will receive a content stream and will distribute it over the local network to all other participating devices. This primary device is elected algorithmically among all participating devices and can change in the course of active playback, specifically in case of devices becoming online or offline in the course of playback.
[0189] The user device also provides an interface for onboarding purposes that allows an app on a mobile device to provide network information to the user device in order to connect it to a local network. The actual hardware used in the user device is, in an abstract representation, a hardware abstraction layer being provided by a device maker and used together with the CIN240904PEP-2025002689.DOCX 42 media runtime device software. The HAL is used to control network interfaces, receive signal input signals and receive status information for further use from the media runtime device software 2, among other needed abstractions to interact with provided hardware such as audio or video output. The device always identifies itself towards all other participants in the ecosystem based on a unique device ID securely stored on the user device 100.
[0190] The cloud backend comprises multiple services with detailed responsibilities. The cloud backend 200 is the main control and management interface within the ecosystem, where all changes in the ecosystem are managed and controlled via the cloud backend 200. Exemplarily, the cloud backend comprises a device management element 35 illustrated in Fig. 1. In this device management, it is made sure that every user device is registered as owned by a user. Users have the ability to see their own devices, to name these devices, to tag these devices, and to deregister them. Users can create and delete device groups which allows using a device group like a single device.
[0191] The cloud backend comprises a playback management that enables the users of the ecosystem to control various playback aspects of the owned devices of the user. Exemplarily, playback of audio, images, and videos is supported as well as an upload of audio data, video data or image data or media data.
[0192] Users can control single devices, view content from content providers and control the playback thereof independently from the local network. This means that the controlling device such as a mobile phone, browser or tablet or generally the user application apparatus 300 can be connected to any network. The playback control includes start, stop, seek, skip, and other functionalities offered by the content provider the content is coming from. Restrictions set by the content provider are respected, for example, skip limits, etc. Users can create playlists that can be composed of items from any content provider of any supporting media type. The user-created playlists can be queued on a device owned by the user. The device can offer any playback control operations directly via buttons and the user can control playback directly from there. However, such control via the buttons is not immediately executed but is only executed when confirmed by a message originating from the user device interface 202 of the cloud backend 200. Users can control playback targeting a device group containing arbitrary devices the user owns. Each separate network segment of the device group will play the content synchronously and technically act as one device. Users can control playback targeting a virtual device and the content will be played according to the virtual device definition. CIN240904PEP-2025002689.DOCX 43
[0193] Additionally, the cloud backend comprises a content service management that allows the end user to connect to content services or, as named before, data instances that are integrated via the functionalities performed by the user application apparatus 300. End users need to have a subscription / account for the corresponding service at the remote data instance they want to connect and the corresponding user application must allow the user to have access to the service.
[0194] The cloud backend additionally comprises a media app management functionality that defines all meta data for content services, decryptions, translations, icons as well as a specification on how to access content via an API or an URL or URI for web applications. The media app management also allows service providers to specify the requirements their service has towards a device (codecs, screen resolution, DRM, etc.). It also allows a content service to configure restrictions for the service allowing to restrict a service availability by device meta data (manufacturer, type, model, etc.), regions or accounts. These restrictions can be formed by using deny lists and allow lists.
[0195] Additionally, an update management is provided in the cloud backend. In general, devices receive updates of their running device media player software over the air. For that, the devices check frequently if there is a new software version available for a specific device model and install it accordingly. The update will be installed when the device is not performing playback, however, devices performing continuous playback will forcefully install the updates after a defined time. Update packages are stored on the distributed CDN (Content Delivery Network) illustrated at 5 in Fig. 1 to ensure high availability and low latency. An update package contains a specific version of the media runtime device software that runs on the device and informative meta data for the update mechanism to be able to select the correct update package. Update packages can be private, so that they can be restricted for development purposes for specific manufacturers, device models or users. The updates are managed through the cloud backend 200 and the device manufacturers. Manufacturers can define if their device models receive a specific version of the update or not.
[0196] The user application apparatus 300 relies on rendering the web frontend exemplarily within a native application. On top of rendering the web frontend, the user application is also integrated in a playback device also allowing remote control via the web frontend or other applications. On top of these functionalities, a further purpose of the user application apparatus 300 is to provide support for onboarding of devices.
[0197] The onboarding of devices is happening by using a camera of a mobile device to scan a QR code. The QR code contains information about the device ID (uniquely identifying the CIN240904PEP-2025002689.DOCX 44 device) as well as optionally information about a local wireless access-point of the device. After scanning the QR code, the user application 300 will connect the mobile device to the wireless access-point of the device. In the next step, the device itself is providing the mobile app with a list of available wireless networks that the device can connect to. The user can select the wireless network to connect to and will be asked to provide the necessary wireless network password. This information is transferred from the user application to the device on which the user application is running and, then, a connection to the wireless network and a connection to the cloud backend 200 are established.
[0198] Discussion
[0199] Virtual Devices are an innovative way to leverage multiple devices in combination. Virtual Devices build on Group Playback and Synchronized Playback capabilities.
[0200] Virtual Device configuration is happening via the Apps and is stored in the Cloud Backend, individual devices and Media Runtime Player Software don't store information about being part of a Virtual Device, Virtual Devices can only act in their intended role controlled via the Cloud Backend.
[0201] Virtual Device is a functionality, that allows Devices which have one or more speakers, and / or one or more video displays, each device connected to a SoC which can control the speaker(s) and / or display(s) and that are provided with the Media Runtime Device Software and could e.g., be located in the same local area network (LAN), each Device connected to the Internet.
[0202] Such devices as referenced above are referred to as Player(s) hereafter. Players can be grouped into a virtual device by the Cloud Backend arbitrarily and the layout of groups is not restricted, i.e., any combination of Players can compose a group.
[0203] A Player can be member of none, one, or multiple virtual devices. At playback time, it is preferred that only one group containing the same Player can be active, i.e., the Player consumes content from one group at a time. The group that is currently selected and active, that a Player is part of, is called ACTIVE GROUP hereafter.
[0204] The Active Group and its content are controlled and changed by the Cloud Backend. The Player only plays a passive role and reacts to playback changes sent by the Cloud Backend. CIN240904PEP-2025002689.DOCX 45
[0205] In an embodiment, the references to the content that will be played back is contained in an ordered queue. The content of the queue is fully controlled by the Cloud Backend. Each queue contains an identifier that uniquely identifies the queue and an indicator, if it is the content of an Active Group.
[0206] On activation of a Group Playback, each Player of the Active Group receives a queue that is flagged as group queue from the Cloud Backend. All Players that belong to the Active Group share the same queue with the same unique identifier.
[0207] When a Group Playback is activated, all Players that belong in the same LAN network in the group organize themselves by determining a primary Cinemo Player and declaring all others as secondaries. This is called Primary Election hereafter.
[0208] Configuration of the Virtual Device can include assigning a device a specific role, for instance left and right in the case of a stereo pair, front left, center, etc. in case of a surround sound virtual device, or top left etc. in case of a video wall Virtual Device.
[0209] Virtual Devices combine multiple existing physical devices to form a new device, with its own unique capabilities, allowing for new playback options. The capabilities of each physical device are analyzed and the Virtual Device will have the capabilities of each individual device, or depending on the combination the intersection of the capabilities. In some possible combinations it can also result in new capabilities for the Virtual Device.
[0210] The Virtual Device is handled and addressed as a new device and not as only a grouping of individual devices.
[0211] Within the Virtual Device there will be a primary device and one or more secondary devices. Depending on the type of Virtual Device this primary device is either elected or is selected based on the needed capabilities.
[0212] The primary election may be performed as follows. Every Player of the Active Group sends a broadcast message containing the unique queue identifier to the local network. If a Player receives a broadcast message, it responds to it in case its own queue identifier matches the broadcasted one. The response message contains a performance indicator of the HW, a stability value and a random integer value for disambiguation. If a primary Player is already existing, the response messages contain the address of the primary. This allows for joining an existing Group Playback. CIN240904PEP-2025002689.DOCX 46
[0213] The performance indicator is based on one or more of the following capabilities: the amount of RAM, CPU cores, current CPU load, available flash storage space, availability of hardware decoders and other identifiers relevant to the performance of a hardware device.
[0214] The stability indicator reflects the stability of a Player over time. That means, that it degrades on unexpected reboots, crashes and other not anticipated, unexpected behavior.
[0215] After a short time, everyone in the group that shares the same queue identifier must have received an answer from all other Players in the group.
[0216] In case the response messages do not contain an address of a primary, the Players will elect the one that replied with the highest performance indicator and the highest stability indicator and highest random number (compared by that order) as the new primary.
[0217] The primary Player will initiate playback and the secondaries will connect to it, and receive the content from the primary Player.
[0218] The primary device is controlling the Virtual Device and is responsible of receiving the main media stream from a Content Service Provider, it is responsible to re-distribute this stream to all other devices in the Virtual Device, in cases where the Virtual Device is part of a Group and Group Playback is being used the primary device is the only device that receives the Group Playback stream. Depending on the specific Virtual Device the primary device might only re-distribute parts of the received stream, for instance in case a display device is part of a Virtual Device regular speakers might not receive the video stream. The Cloud Backend communicates to the primary device which parts of the stream to distribute to which device.
[0219] Some Virtual Devices will require additional configuration based on physical information, like position of a speaker in a room, to provide a good experience. This additional information could either be determined based on device capabilities (like microphones) or will be determined from within the App, this data is stored in the Cloud Backend.
[0220] Devices that form a Virtual Device can still be used as individual devices, as long as the Virtual Device is not in use at the same time.
[0221] Cloud Ecosystem
[0222] This preferred embodiment used the group playback invention within a new cloud ecosystem, which consists of multiple parts and roles: CIN240904PEP-2025002689.DOCX 47
[0223] Cloud Ecosystem - a uniform cloud media platform connecting Users, Devices, and Content Services via the Cloud Backend and providing the Management Services. Device Maker - any party designing and / or manufacturing Devices. It includes also private technical savvy users acting as such Device Maker by installing the Media Runtime Device Software on a device of their own (e.g., a private camera connected to an SoC of a User).
[0224] Device Brand - a commercial brand under which a Device Maker offers Devices Content Services - any party having cloud services for sending (downstream from cloud service) or receiving (upstream to cloud service) content on or from a Device (examples for sending are e.g., services like Spotify, Netflix, or advertising content services, business radio services, private streaming services, content management systems (including systems handling scheduling, delivery and curation) and for receiving can be e.g., a service to store digital camera data in the cloud).
[0225] Media App - a descriptor defining and enabling the connection between the Content Services, the Cloud Backend and the Device. And enabling a Content Service additional functionality by communicating with the Device via the Media Runtime Device Software, like e.g., like saving content on a Device. A Media App is only available in the Cloud Backend, the Device and the Media Runtime Device Software do not have information about the Media App, providing additional security and separation of concerns.
[0226] Device - a hardware device having at least one CPU and a connection to the internet, having Device Capabilities and having integrated the Media Runtime Device Software. A Device is uniquely identified by a Device ID and can have a User as its owner. Media Runtime Device Software - embedded software running on a device and communicating both with the device hardware and with the Cloud Backend, and supporting Formats and other capabilities depending on the Device Capabilities. The Media Runtime Device Software is independent of specific Media Apps, and is compiled to many OS and SoCs systems.
[0227] Cloud Backend - cloud services enabling and maintaining the connection and control of the Media Runtime Device Software of all Devices. Through this connection enabling the Cloud Ecosystem. The Cloud Backend has its own Web APIs (e.g., REST APIs). Media Apps are used in the Cloud Backend to communicate content for playback from and to Devices.
[0228] Device Capabilities - the list of all hardware capabilities of a device supported by the Cloud Ecosystem. This includes all supported Formats, and including capabilities like supported audio and video codecs for encoding and for decoding, performance capabilities, security capabilities such as secure boot, DRM capabilities, payment capabilities, etc. Device ID - a unique identifier for each Device CIN240904PEP-2025002689.DOCX 48
[0229] App - an application by using the Management Services to manage the Devices and the Account. Optionally, an App can have its own Media Runtime Device Software making the App a Device itself. An App can be a mobile app running on a smart phone or tablet or can be running in a browser or an application running on a PC.
[0230] User - a user with an Account. Users can own their own Devices or use Devices of any other user (e.g., colleagues, or family members).
[0231] Role - each User has one or more roles to define the rights of the User
[0232] Account - a unique user with a free or paid subscription to the Cloud Ecosystem Formats - at least one of audio, video, pictures, text and interactive (web) content play- ing / displaying, and / or generating at least one of audio, video, pictures, text and interactive (web) content. Examples for playing are (i.) speakers for audio, (ii.) screens, TVs for video and pictures, (iii.) screens, keyboard for interactive content. Examples for generating are (i.) microphones for audio, (ii.) cameras for video and pictures.
[0233] Play, Playback, playback or played or play - is broadly defined as playing Formats on a Device and / or generating Formats on a Device
[0234] Store - a service of the Cloud Backend enabling Users to search, find, select and unselect Media Apps and authenticate them with the new Cloud Ecosystem using the Content Services credentials, and vice-versa. This allows the Cloud Ecosystem to e.g., access content from Content Services.
[0235] Management Services - Services in the Cloud Ecosystem available via the Cloud Backend for participants to control and manage some or all aspects for one or many Devices (e.g., managing Devices including onboarding and offboarding, play queue management, playback management, etc.).
[0236] The ecosystem is a system consisting of
[0237] An unlimited number of Devices
[0238] An unlimited number of Users
[0239] An unlimited number of Media Apps
[0240] Enabling access to each User's Devices and Content Services over Apps and / or Management Services, with the possibility to delegate rights to other Users.
[0241] Giving Content Services full control over (i.) minimum Device Capabilities a Device must have to be allowed and able to play their content (for example codec support, secure boot, DRM, etc.) for the purpose of e.g., ensuring best possible playback quality, and / or the minimum security requirements (ii.) Devices from which Device Brand(s) will be allowed to play their content, or to not play their content, (iii.) which Devices get access to which Media App enabling a Content Service for example to exclusively one Device Brand, or any combination / permutation, (iv.) allow Content Services to CIN240904PEP-2025002689.DOCX 49 extend their own applications the capability to manage Devices with their own Media Apps (e.g., remote control).
[0242] Giving Device Makers full control over (i.) which Media Apps they want to allow on their Devices, (ii.) which Device Capabilities of the Cloud Ecosystem the Device should support and make available to the Cloud Ecosystem, (iii.) allow Device Makers to extend their own applications the capability to manage Devices via Management Services (e.g., Device onboarding to the Cloud Ecosystem).
[0243] Giving Users full control over (i.) selecting Media Apps via the Store and authenticate the Cloud Ecosystem with them, (ii.) using Apps and the Management Services to manage and control the Devices they own, (iii.) permissions of the Media Apps which Device Capabilities they can access via the Management Services (e.g., for a Device with a speaker and a microphone, a User should explicitly allow the Media App to control playback on the speaker, and to play from the microphone, i.e. to access the microphone input and send it to the Content Service), (iv.) delegating User's rights partially or fully to other Users using e.g., Roles.
[0244] Advantages of our ecosystem are, amongst others:
[0245] Separation of Devices and Media Runtime Device Software versus the Content Services enabling any Content Service to run on any Device just based on the Device Capabilities, the Media App service mapping, the Device Maker and Content Services restrictions and the User choices only, fully managed in real-time in the cloud.
[0246] The ecosystem can run on any supported Device running the Media Runtime Device Software, regardless of OS and SoC.
[0247] Any participant keeps the control it wants, at any time.
[0248] Description of Further Embodiments of the virtual devices invention
[0249] A virtual device can be managed and this virtual device can, for example, consist of a monitor and four loud speakers. In embodiments, there are templates for virtual devices, and there are capabilities that map to the whole virtual device rather than to the individual elements of the virtual device. Additionally, there can even be some signal processing such as providing the correct low frequency range to a subwoofer when one of the speakers in the virtual device is, for example, a subwoofer.
[0250] In an embodiment, such a virtual device has a primary device and the remaining individual devices of the virtual device are secondary devices. The primary device locally distributes CIN240904PEP-2025002689.DOCX 50 the content to the secondary devices. In line with an initiating procedure, the virtual device is initiated, and the virtual device is then managed as if it consists of a single (real) device
[0251] Discussion regarding Figs. 9A and 9B
[0252] Virtual devices 110 may be derived from so called templates. Virtual device teplates define a pre-configured setup that allow multiple independent devices 10a, 10b, lOd to act as one device 110. Virtual device templates may consist of (or more in general comprise) so called slots that represent the role each device 10a, 10b, lOd that is part of the virtual device 110 takes. Roles can be e.g. audio channels, device positions in a video wall or a mixture of that. Roles reguire capabilities and / or conditions which shall be compliant with the capabilities and / or conditions of the devices 10a, 10b, lOd which take the roles.
[0253] Before using the virtual device 110, the user 99 may create the virtual device 110 by choosing a virtual device template option (e.g. one of the options shown in Figs. 8A, 8B, and 8C) and assigning one device 10 to each slot within the template as displayed in the user interface of the user eguipment on which the user application 3 executes. Only devices 10 that support the specific role reguired for a slot can be selected to play that role (in addition of the permission aspect of the ecosystem).
[0254] Virtual device playback could be considered, to some extent, to have some common aspects with the playback of a group 210 of independent devices (like in Figs. 6 and 7A-7H). Instead of group members, the devices assigned to template slots will group, elect a leader (virtual device primary player device a and care about the playback. Additionally, the virtual device definition (as part of the information 137 of being part of the virtual device 110) is sent to the leader 10a which distributes it to the follower devices (virtual device primary player devices 10b and lOd), so that any device 10b, lOd that is part of the virtual device 110 knows about its role and can limit its playback to the role's definition.
[0255] Side note: Even if a device is part of the virtual device 110, it still can be used as individual device or as a member of other device groups. Of course it always can only act in one mode at a time.
[0256] Select device and content
[0257] First, the user 99 selects content and the device 10a, 10b, 10c... where the content 51 is intended to be played back. In this case the virtual device 110 is selected.
[0258] Playback on selected device CIN240904PEP-2025002689.DOCX 51
[0259] After content 51 4and device (here virtual device 110) have been selected, there is prepared for and started playback. The user 99 can execute full control of the virtual device 110 during playback as if it was a physical device.
[0260] Description of Further Embodiments of the Ecosystem
[0261] A preferred embodiment is illustrated in the enclosed figure illustrating several elements of the invention. The hatched line indicates what is included in the typically mobile device owned by the user. However, the Media CDN (content delivery network) does not necessarily need to be located in the device as a library or any other collection of media, but can also be located anywhere in the cloud. In this embodiment, the mobile device has an interface for interfacing with the Media CDN in the cloud.
[0262] A "final" or endpoint device is the device HW that typically consists of a replay functionality such as a loudspeaker or a video screen and some interface buttons such as stop buttons, pause buttons, etc.
[0263] This "device HW" interfaces with a "device SDK / player". This functional unit "device SDK / player" interfaces, on the one hand, with a media provider or, generally, with a distributed content delivery network (media App CDN).
[0264] Furthermore, the "device SDK / player" interfaces with the cloud backend.
[0265] Furthermore, there exists a Cinemo app on the user's mobile (or stationary) device such as the user's mobile phone, tablet or even a user's stationary computer. This app on the mobile device mainly interfaces with the cloud backend and interfaces, via the onboarding functionality, with the device SDK / player unit. However, the app to be stored, for example, on the user's mobile device or on the user's stationary computer does not interface with the final playback device or "device HW".
[0266] Furthermore, this app does not interface with the media app content delivery network that comprises, for example, a streaming platform such as Netflix, Spotify, Amazon Prime, etc.
[0267] Seen from the perspective of the cloud backend, the cloud backend interfaces on the one hand, with the app on the user's mobile or stationary device, with the media app content delivery network, and with the device SDK / player. However, the cloud backend does not interface directly to the actual replay device or "device HW". CIN240904PEP-2025002689.DOCX 52
[0268] Due to the fact that the cloud backend is distributed in a cloud, this functional unit maintains, among others, several databases that include, among others, "Live Services" functionalities that represents the actual interface between the media app content delivery network (media app CDN) and the cloud backend.
[0269] Generally, at least four instances will cooperate with each other or with only selected ones of all instances. These are the final replay hardware deceive (1), the device SDK / player (2), the app (3) on the user's mobile / stationary device, the cloud backend (4) and the distributed content delivery network (CDN) (5).
[0270] The present invention allows a distributed Communication between the Content Data and the Control Data.
[0271] It is preferred that the cloud backend only communicates with the media app content distribution network with respect to control data that only require a low volume of data transmission. On the other hand, the actual content such as video or audio content is directly transmitted from the distributed content delivery network (CDN) to the actual hardware device via the device SDK / player functionality which is a software program stored on a storage of the actual player device. This distributed transmission of content data on the one hand and control data on the other hand has the advantage of only having low volume traffic in the cloud backend. The high volume traffic only occurs between the content delivery network such as the streaming provider and the device SDK / player unit stored in a storage of the end point device including the "device HW" instance.
[0272] Furthermore, the controlled situation in that the user's access to the content delivery network or streaming provider is only done via the cloud backend is advantageous for the content provider, since the content provider can trust the cloud backend and can also trust the hardware end point device, since all the replay functionalities are controlled by the device SDK / player instance that tightly cooperates with the cloud backend. In case of a security breach the cloud backend can easily bar the compromised device or a class or group of compromised devices (e.g., from the same producer or distributor) by building and maintaining a corresponding "no go" database.
[0273] A specific functionality provided to the user of the hardware player is to also press buttons on the loudspeaker or the video screen device giving the user the impression that the user has the full control over the replay functionalities, although these Hardware Abstraction Layer (HAL) events are not forwarded directly to the streaming website, but only via the cloud backend. CIN240904PEP-2025002689.DOCX 53
[0274] Furthermore, embodiments of the cloud backend make sure that correct reports back to the artist with respect to Copyrights, etc. are assured by the cloud backend.
[0275] Further functionalities regarding the good control for the content provider such as databases stored with user data etc. are preferred for this invention.
[0276] An inventively encoded signal can be stored on a digital storage medium or a non-transi- tory storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.
[0277] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
[0278] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed.
[0279] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
[0280] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.
[0281] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier or a non-transitory storage medium. CIN240904PEP-2025002689.DOCX 54
[0282] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
[0283] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.
[0284] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.
[0285] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
[0286] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
[0287] In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.
[0288] The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.
[0289] Some aspects
[0290] 1. Apparatus, method or computer program for operating a player being included in a virtual device as described above
[0291] 2. Apparatus, method or computer program of aspect 1, wherein the player is a primary player or a secondary player of the virtual device CIN240904PEP-2025002689.DOCX 55
[0292] 3. Apparatus, method or computer program for controlling a virtual device comprising at least two players as described above, wherein the apparatus, method or computer program is separate from the virtual device comprising the at least two players such as the primary player and the secondary player.
[0293] 4. Apparatus, method or computer program for communicating between the player and the controller as described above.
[0294] 5. Apparatus, method or computer program of one of the preceding aspects, wherein the controller is an element of the ecosystem and comprises a cloud backend.
[0295] 6. System, method or computer program for implementing a virtual device, the virtual device comprising a primary player, a plurality of secondary players, wherein the system furthermore comprises a controller being separate from the primary player and the plurality of secondary players as described above.
[0296] In examples, there are provided, inter alia, a system (e.g. 4) for assisting a streaming and a rendering of remote media content, the system including: an interface (e.g. 410) to a plurality of remote player devices (e.g. 10), each controlling at least one media Tenderer unit (e.g. 1) and having playback capabilities and / or playback conditions; an interface (e.g. 405) to at least one remote media distribution network, CDN (e.g. 5, 5a); an interface (e.g. 403) to a remote user application (e.g. 3), to send, to the user application, a list of the remote player devices (e.g. 10a, 10b, lOd, lOd), and to receive, from the user application, a selection on a plurality of remote player devices (e.g. 10a, 10b, lOd) to be part of a virtual device (e.g. 110) for a synchronized media playback, the system (e.g. 4) being configured to distribute roles among the remote player devices (e.g. 10a, 10b, lOd) of the virtual device (e.g. 110), based on the selection performed by the user application (e.g. 3), the system (e.g. 4) being configured to send, to at least one (e.g. 10a) of the remote player devices (e.g. 10) of the virtual device (e.g. 110), role information (e.g. 137a) which includes instructions on a respective role of each remote player device (e.g. 10) of the virtual device (e.g. 110) for participating to the synchronized media playback, the system (e.g. 4) being configured to send, to at least one of the remote player devices (e.g. 10), information (e.g. on media content to be retrieved and rendered in the synchronized media playback. CIN240904PEP-2025002689.DOCX 56
[0297] The system may provide, to the remote user application (e.g. 3), indication on the playback capabilities and / or playback conditions of at least one of the plurality of player devices (e.g. 10), and configured to receive, from the remote user application, a request for performing the synchronized media playback, so that the system (e.g. 4) defines, based on the request, the virtual device (e.g. 10).
[0298] The system may receive, from the remote user application, commands (e.g. 31) for playback control of the virtual device (e.g. 110) as a unique entity, and to send the commands (e.g. 34) for playback control at least one of the player devices of the virtual device.
[0299] The system may perform an authorization procedure to authorize the at least one of the remote player devices (e.g. 10) to receive the media content from the CDN (e.g. 5, 5a).
[0300] The system may receive from the CDN (e.g. 5, 5a) authentication data to be provided to at least one of the remote player devices (e.g. 10) to subsequently receive the media content from the CDN (e.g. 5, 5a).
[0301] The system may receive from the CDN (e.g. 5, 5a) an authentication information and to provide the authentication information to the at least one of the remote player devices (e.g. 10), to thereby permit the reception the media content from the CDN (e.g. 5, 5a).
[0302] The system may receive, from at least one of the remote player devices, information on the playback capabilities and / or playback conditions, so as to inform the remote user application (e.g. 3) on the playback capabilities and / or playback conditions of at least one of the remote player devices, in such a way that the selection of the remote player devices is compliant with the playback capabilities and / or playback conditions.
[0303] The system may distribute the roles among the remote player devices (e.g. 10a, 10b, lOd) of the virtual device (e.g. 110) as requested by the user application (e.g. 3).
[0304] The system may, in distributing the roles of the of the virtual device (e.g. 110), verify whether the roles as requested by the user application (e.g. 3) for the player devices (e.g. 10) are compliant with the capabilities and / or conditions of the respective player devices, CIN240904PEP-2025002689.DOCX 57 and, in case the roles as requested by the user application (e.g. 3) for the player devices (e.g. 10) are not compliant with the capabilities and / or conditions of the respective player devices, to send an error message to the user application (e.g. 3) without assigning the roles to the player devices.
[0305] The system may include a capability of rendering video content, the system (e.g. 4) being configured to define the virtual device (e.g. 10) as having at least one player device (e.g. 10) with the capability of rendering the video content.
[0306] The system may include a capability of rendering video content with a particular quality, the system (e.g. 4) being configured to define the virtual device (e.g. 10) as having at least one player device (e.g. 10) with the capability of rendering video content at a preferred quality.
[0307] The system may include a capability of rendering audio content, the system (e.g. 4) being configured to define the virtual device (e.g. 10) as having at least one player device (e.g. 10) with the capability of rendering the audio content.
[0308] The system may include a capability of rendering audio content with a particular quality, the system (e.g. 4) being configured to define the virtual device (e.g. 10) as having a paler device (e.g. 10) having the capability of rendering the audio content at a preferred quality.
[0309] The system may select a virtual device primary player device (e.g. 10a) among the player devices (e.g. 10a, 10b, lOd) constituting the virtual device (e.g. 110), so that the virtual device primary player device (e.g. 10a) is the player device that receives the media content from the CDN (e.g. 5, 5a).
[0310] The system may provide the user application (e.g. 3) with template information (e.g. 913), the template information providing different setup options (e.g. 110a, 110b, 110c), to therefore receive, from the user application (e.g. 3), information on a particular selected setup option and matching information between the player devices (e.g. 10) and the roles which are needed for the particular selected setup option. CIN240904PEP-2025002689.DOCX 58
[0311] The system may provide, for each role of the template information, information on the player devices which have the playback conditions and / or playback capabilities to take the role, so as to not permit a selection of a player device not having the playback conditions and / or playback capabilities to take the role.
[0312] The system may provide, for each role of the template information, a list of only the player devices which have the playback conditions and / or playback capabilities to take the role.
[0313] Th system may maintain memory of the virtual device (e.g. 110) and of which player devices (e.g. 10a, 10b, lOd) have taken part to the virtual device (e.g. 110), and which role has had each virtual device.
[0314] The system may define at least one group (e.g. 210) of player devices, the group of player devices including the virtual device (e.g. 210)
[0315] The system may define multiple virtual devices according to request from the user application.
[0316] According to examples there is provided, inter alia, a player device (e.g. 10, 10a, 10b, 10c, lOd, lOe, lOf, 10g) for participating to a synchronized media playback, the player device including: an hardware unit (e.g. 1) to render synchronized media playback of media content; an interface to a remote assisting system (e.g. 4); an interface to at least one other player device (e.g. lOb-lOd) through a local connection; the player device (e.g. 10) being configured to receive, from the assisting system (e.g. 4), information (e.g. 137) of being part of the virtual device (e.g. 110) and role information (e.g. 137a) indicating a playback role that the player device (e.g. 10) shall have in the synchronized media playback, to thereby constitute a virtual device (e.g. 110) with the at least one other player device (e.g. lOb-lOd) to perform the synchronized media playback by receiving the media content and playingback the media content according to the playback role. CIN240904PEP-2025002689.DOCX 59
[0317] The player device may send, to the assisting system (e.g. 4), information on its own player device's playback capabilities and / or playback conditions.
[0318] The system may include a capability of rendering video content, so as to subsequently receiving the playback role of rendering the video content.
[0319] The system may include a capability of rendering video content with a particular quality, so as to subsequently receiving the playback role of rendering the video content at the particular quality.
[0320] The system may include a capability of rendering audio content, so as to subsequently receiving the playback role of rendering the audio content.
[0321] The system may include a capability of rendering audio content with a particular quality, so as to subsequently receiving the playback role of rendering the audio content at the particular quality.
[0322] The player device (e.g. 10) may participate to a player device primary election, with the at least one other player device (e.g. lOb-lOd) constituting the virtual device (e.g. 110), the virtual device primary election electing a virtual device primary player device (e.g. 10a), the remaining player device or player devices (e.g. 10b, lOd) in the virtual device (e.g. 110) being defined virtual device secondary player device or virtual device secondary player devices (e.g. 10b, lOd); in case of the player device (e.g. 10) being elected virtual device primary player device (e.g. 10a), receive media content (e.g. 51) from a content distribution network, CDN, (e.g.
[0323] 5, 5a) and retransmit the media content (e.g. 51) to the virtual device secondary player devices (e.g. 10b, lOd), and in case of the player device not being elected virtual device primary player device (e.g. 10a) and therefore resulting a virtual device secondary player (e.g. 10b, lOd), receive media content (e.g. 51) from the virtual device primary player device (e.g. 10a), both in case of being elected virtual device primary player device and in case of being a virtual device secondary player device, control the at least one hardware media Tenderer unit (e.g. 1) to render the media content (e.g. 51). CIN240904PEP-2025002689.DOCX 60
[0324] The player device may receive a content retrieval information (e.g. 37) from the assisting system (e.g. 4) informing of media content to be retrieved from the CDN (e.g. 5) and to be subsequently rendered, so that: in case of the player device (e.g. 10) being elected virtual device primary player device (e.g. 10a), receiving, from the content distribution network (e.g. 5, 5a), the media content indicated in the content retrieval information and retransmitting the media content to the virtual device secondary player device or virtual device secondary player devices (e.g. 10b, lOd); and in case of the player device (e.g. 10) being a virtual device secondary player device (e.g. 10b, lOd), receiving, from the virtual device primary playback device (e.g. 10a), the media content indicated in the content retrieval information.
[0325] The player device may include at least one address of the media content to be retrieved and / or information to be provided to the CDN (e.g. 5) to request the media content.
[0326] The player device may include authenticating information for authenticating at least the virtual device primary player device (e.g. 10a) to receive the media content from the CDN (e.g. 5).
[0327] The player device may receive, from the assisting system (e.g. 4), an identifier (e.g. 37a) identifying the content retrieval information (e.g. 37).
[0328] The player device may receive a queue information (e.g. 37) from the assisting system (e.g. 4) informing of a queue of media content to be rendered, so that: in case of the player device (e.g. 10) being elected the virtual device primary player device (e.g. 10a), receiving, from the content distribution network (e.g. 5, 5a), the media content according to an order defined by the queue (e.g. 37) and retransmitting the media content to the virtual device secondary player device or virtual device secondary player devices (e.g. 10b, lOd); and in case of the player device (e.g. 10) being a virtual device secondary player device (e.g. 10b, lOd), receiving, from the virtual device primary playback device (e.g. 10a), the media content according to the order defined by the queue (e.g. 37).
[0329] The player device may include a queue identifier which identifies the virtual device. CIN240904PEP-2025002689.DOCX 61
[0330] The player device may, once having received the information (e.g. 137a) of being part of the virtual device (e.g. 110): transmit, through the local network (e.g. 220), to the other player devices (e.g. 10a, 10b, lOd) of the virtual device (e.g. 110), information (e.g. 214) including at least discriminating information attributed to the player device and / or the status of the player device, receive, through the local network (e.g. 220), information (e.g. 214) from the other player devices (e.g. 10a, 10b, lOd) of the virtual device (e.g. 110), each including at least discriminating information attributed to the sending player devices and / or the status of the sending player device, thereby defining the virtual device primary player device (e.g. 10a) based on the discriminating information of all the player devices of the virtual device (e.g. 110).
[0331] The player device may include at least one performance indicator of the sending player device.
[0332] The player device may include at least one stability value attributed to the sending player device.
[0333] The player device may include at least one random value.
[0334] The player device may include at least one unique identifier of the sending player device.
[0335] The player device may apply a predetermined rule shared with the other player devices of the virtual device (e.g. 110) to elect the primary player device (e.g. 10a) by examining the discriminating information of all the player devices in the virtual device (e.g. 110).
[0336] The player device may: in the case of the player device (e.g. 10b, lOd) being a virtual device secondary player (e.g. 10b, lOd), and in the case of not receiving the media content (e.g. 51) and / or a heartbeat message from the virtual device primary player device (e.g. 10a) for a time delay over a predetermined delay threshold, participate to a further virtual device primary election to elect a new virtual device primary player device, and, in the case of the player device CIN240904PEP-2025002689.DOCX 62 being elected as new virtual device primary device, receive media content (e.g. 51) from the content distribution network (e.g. 5, 5a) and retransmit the media content (e.g. 51) to the virtual device secondary player devices (e.g. 10b, lOd), and in case of the player device not being elected new virtual device primary player device (e.g. 10a) and therefore resulting a virtual device secondary player (e.g. 10b, lOd), receive media content (e.g. 51) from the new virtual device primary player device (e.g. 10a), and both in case of being elected new virtual device primary player device and in case of being a virtual device secondary player device, control the at least one hardware media Tenderer unit (e.g. 1) to render the media content (e.g. 51) according to the assigned role.
[0337] The player device may: in the case of the player device (e.g. 10b, lOd) being a virtual device secondary player (e.g. 10b, lOd), and in the case of receiving the media content (e.g. 51) and / or heartbeat messages from the primary player device (e.g. 10a) with an error rate over a predetermined error-rate threshold, participate to a further virtual device primary election to elect a new virtual device primary player device, and, in the case of the player device being elected as new virtual device primary device, receive media content (e.g. 51) from a content distribution network (e.g. 5, 5a) and retransmit the media content (e.g. 51) to the virtual device secondary player devices (e.g. 10b, lOd) of the virtual device (e.g. 110), and in case of the player device not being elected new virtual device primary player device (e.g. 10a) and therefore resulting a virtual device secondary player (e.g. 10b, lOd), receive media content (e.g. 51) from the new virtual device primary player device (e.g. 10a), and both in case of being elected new virtual device primary player device and in case of being a virtual device secondary player device, control the at least one hardware media Tenderer unit (e.g. 1) to render the media content (e.g. 51)Aaccording to the assigned role.
[0338] The player device may be performed as the virtual device primary election.
[0339] The player device may be excluded from the virtual device primary election.
[0340] The player device may have, from the first primary player device election, a knowledge of the first secondary player device to be automatically elected in case of the virtual device CIN240904PEP-2025002689.DOCX 63 primary player device is not available without further virtual device primary device election, and: in case of the player device being the new virtual device primary player device, receive the media content (e.g. 51) from the content distribution network (e.g. 5, 5a) and retransmit the media content (e.g. 51) to the virtual device secondary player devices (e.g. 10b, lOd) in the virtual device, and in the case of the player device not being elected the new virtual device primary player device and therefore resulting a virtual device secondary player (e.g. 10b, lOd), receive media content (e.g. 51) from the new virtual device primary player device (e.g. 10a), both in case of being elected the virtual device primary player device and in case of being a virtual device secondary player device, control the at least one hardware media Tenderer unit (e.g. 1) to render the media content (e.g. 51) according to the assigned role.
[0341] The player device may: in case of the player device (e.g. 10) being the virtual device primary player device (e.g. 10a), adopt the user's command for playback event, and transmit, to the virtual device secondary player devices (e.g. 10b, lOd), the user's command for playback event, and in case of the player device being a virtual device secondary player (e.g. 10b, lOd), send a request to the virtual device primary player device (e.g. 10a) for accepting the user's command for playback event.
[0342] The player device may receive a command from the assisting system (e.g. 4) for playback event (e.g. 34) , the player device being configured to adopt the command irrespective of the player device being the virtual device primary player device or a virtual device secondary player device.
[0343] The player device may: adopt the user's command for immediate playback event by controlling the at least one hardware media Tenderer unit (e.g. 1) to adopt the command for immediate playback event both in the case of the player device being the primary player device and in the case of the player device being a secondary player device.
[0344] The player device may, in the case of being a secondary player device, to also send a request to the primary player device (e.g. 10a) for accepting the user's command for immediate playback event. CIN240904PEP-2025002689.DOCX 64
[0345] The player device may regard at least the playback sound volume.
[0346] The player device may receive the media content without installing new application program interfaces, APIs.
[0347] The player device may be subjected to a primary selection, together with the at least one other player device (e.g. lOb-lOd) constituting the virtual device (e.g. 110), performed by the assisting system (e.g. 4), the virtual device primary selection selecting a virtual device primary player device (e.g. 10a), the remaining player device or player devices (e.g. 10b, lOd) in the virtual device (e.g. 110) being defined virtual device secondary player device or devices (e.g. 10b, lOd); in case of the player device (e.g. 10) being selected virtual device primary player device (e.g. 10a), receive media content (e.g. 51) from a content distribution network (e.g. 5, 5a) and retransmit the media content (e.g. 51) to the virtual device secondary player device(e.g. s) (e.g. 10b, lOd), and in case of the player device not being selected virtual device primary player device (e.g. 10a) and therefore resulting a virtual device secondary player (e.g. 10b, lOd), receive media content (e.g. 51) from the virtual device primary player device (e.g. 10a), both in case of being selected virtual device primary player device and in case of being a virtual device secondary player device, control the at least one hardware media Tenderer unit (e.g. 1) to render the media content (e.g. 51) according to the assigned role.
[0348] The system may include a capability and / or condition of receiving and / or transmitting the media content, through the local connection, at a particular data rate and / or error rate, so as to be subsequently selected as virtual device primary device or virtual device secondary device based at least in part of the capability and / or condition of receiving and / or transmitting the media content, through the local connection, at the particular data rate and / or error rate.
[0349] The player device may be part of a group (e.g. 210) for a synchronized playback with at least one additional player device (e.g. 10g) or virtual player device. CIN240904PEP-2025002689.DOCX 65
[0350] The player device may take part to a group primary election, wherein, in the case of the player device being elected as the virtual device primary player device, the player device participating to the group primary election, and being configured to: in case of being elected as the group primary virtual device, receive from the CDN (e.g. 5) the media content (e.g. 51) and retransmit the media content to the both the virtual device secondary player devices and the other player devices and virtual player devices of the group (e.g. 210), and, in case of not being elected the primary virtual device and consequently being a group secondary virtual device, receive the media content from the elected group primary player device or virtual device and retransmit the received media content to the virtual device secondary player devices of the virtual device.
[0351] The player device may be performed in the same way as the virtual device primary election.
[0352] The player device may, once having received information (e.g. 37) of being part of the group (e.g. 210): transmit, through the local network (e.g. 220), to the other player devices and virtual devices of the group, information (e.g. 214) including at least discriminating information attributed to the player device and / or the status of the player device, receive, through the local network (e.g. 220), information (e.g. 214) from the other player devices and virtual devices of the group, each including at least discriminating information attributed to the sending player devices and / or the status of the sending player device, thereby defining the group primary player device (e.g. lOf) based on the discriminating information of all the player devices player devices and virtual devices of the group.
[0353] The player device may receive information (e.g. 37) of being part of the group (e.g. 210) as information on media content to be playedback, wherein the group is formed by the player devices and virtual devices having the same media content to be playedback.
[0354] According to examples there is provided, inter alia, a user application (e.g. 3) for commanding a streaming and a rendering of remote media content (e.g. 51), the user application executing in a user equipment, the user application comprising: CIN240904PEP-2025002689.DOCX 66 an interface to an assisting system (e.g. 4) to receive information on a plurality of remote player devices (e.g. 10), each controlling at least one media Tenderer unit (e.g. 1) and having playback capabilities and / or playback conditions; the user application being configured for receiving, through the interface to the assisting system, information on media content on at least one remote media distribution network, CDN (e.g. 5, 5a); he user application being configured to define a remote a virtual device (e.g. 110) from a plurality of remote player devices (e.g. 10), and the user application being configured to receive playback control commands (e.g. 34) and / or content control commands (e.g. 38) from a user (e.g. 99) for the virtual device (e.g. 110), and to retransmit the control playback commands to the assisting system (e.g. 4).
[0355] The user application may receive from the assisting system (e.g. 4) template information, the template information providing different setup options (e.g. 110a, 110b, 110c), the user application being configured to select, in a user interface, a setup option among the other setup options, to receive from the user a selection of the setup option and to retransmit the selected setup option to the assisting system (e.g. 4).
[0356] The user application may receive from the assisting system (e.g. 4), for each role of the template information, information on which player devices have the playback conditions and / or playback capabilities to take the role, so as to not permit the selection of a player device not having the playback conditions and / or playback capabilities to take the role.
[0357] The user application may receive from the assisting system (e.g. 4), for each role of the template information, a list with only the player devices have the playback conditions and / or playback capabilities to take the role.
[0358] The user application may maintain memory of the virtual device (e.g. 110) and of which player devices (e.g. 10a, 10b, lOd) have taken part to the virtual device (e.g. 110), and which role has had each virtual device, so as to redefine the virtual device in the future in case of user's request.
[0359] The user application may define at least one group (e.g. 210) of player devices, the group of player devices including the virtual device (e.g. 210). CIN240904PEP-2025002689.DOCX 67
[0360] The user application may define multiple virtual devices according to request from the user application.
[0361] According to examples there is provided, inter alia, a method (e.g. 4) for assisting a streaming and a rendering of remote media content, the system including: interfacing (e.g. 410) with a plurality of remote player devices (e.g. 10), each controlling at least one media Tenderer unit (e.g. 1) and having playback capabilities and / or playback conditions; an interfacing (e.g. 405) with at least one remote media distribution network, CDN (e.g. 5, 5a); an interfacing (e.g. 403) interfacing a remote user application (e.g. 3), to send, to the user application, a list of the remote player devices (e.g. 10a, 10b, lOd, lOd), and receiving, from the user application, a selection on a plurality of remote player devices (e.g. 10a, 10b, lOd) to be part of a virtual device (e.g. 110) for a synchronized media playback,
[0362] (e.g. distributing roles among remote player devices (e.g. 10a, 10b, lOd) of a virtual device (e.g. 110), based on the selection performed by the user application (e.g. 3),
[0363] (e.g. sending, to at least one (e.g. 10a) of the remote player devices (e.g. 10) of the virtual device (e.g. 110), role information (e.g. 137a) which includes instructions on a respective role of each remote player device (e.g. 10) of the virtual device (e.g. 110) for participating to the synchronized media playback,
[0364] (e.g. sending, to at least one of the remote player devices (e.g. 10), information (e.g. on media content to be retrieved and rendered in the synchronized media playback.
[0365] According to examples there is provided, inter alia, a method for participating to a synchronized media playback, including:
[0366] (e.g. rendering synchronized media playback of media content; interfacing with a remote assisting system (e.g. 4); interfacing with at least one other player device (e.g. lOb-lOd) through a local connection;
[0367] (e.g. the method including receiving, from the assisting system (e.g. 4), information (e.g. 137) of being part of the virtual device (e.g. 110) and role information (e.g. 137a) indicating a playback role that the (e.g. rendering shall have in the synchronized media playback, to thereby constitute a virtual device (e.g. 110) with the at least one CIN240904PEP-2025002689.DOCX 68 external player device (e.g. lOb-lOd) to perform the synchronized media playback by receiving the media content and playingback the media content according to the playback role.
[0368] According to examples there is provided, inter alia, a method for commanding a streaming and a rendering of remote media content (e.g. 51), may comprise: interfacing with an assisting system (e.g. 4) to receive information on a plurality of remote player devices (e.g. 10), each controlling at least one media Tenderer unit (e.g. 1) and having playback capabilities and / or playback conditions; receiving, through the interface to the assisting system, information on media content on at least one remote media distribution network, CDN (e.g. 5, 5a); defining a remote a virtual device (e.g. 110) from a plurality of remote player devices (e.g. 10), and receiving playback control commands (e.g. 34) and / or content control commands (e.g. 38) from a user (e.g. 99) for the virtual device (e.g. 110), and to retransmit the control playback commands to the assisting system (e.g. 4).
[0369] According to examples there is provided, inter alia, a non-transitory storage unit storing instructions which, when executed by a processor, may cause the processor to perform a method according to previous examples of methods.
Claims
CIN240904PEP-2025002689.DOCX 69Claims1. A system (4) for assisting a streaming and a rendering of remote media content, the system including: an interface (410) to a plurality of remote player devices (10), each controlling at least one media Tenderer unit (1) and having playback capabilities and / or playback conditions; an interface (405) to at least one remote media distribution network, CDN (5, 5a); an interface (403) to a remote user application (3), to send, to the user application, a list of the remote player devices (10a, 10b, lOd, lOd), and to receive, from the user application, a selection on a plurality of remote player devices (10a, 10b, lOd) to be part of a virtual device (110) for a synchronized media playback, the system (4) being configured to distribute roles among the remote player devices (10a, 10b, lOd) of the virtual device (110), based on the selection performed by the user application (3), the system (4) being configured to send, to at least one (10a) of the remote player devices (10) of the virtual device (110), role information (137a) which includes instructions on a respective role of each remote player device (10) of the virtual device (110) for participating to the synchronized media playback, the system (4) being configured to send, to at least one of the remote player devices (10), information on media content to be retrieved and rendered in the synchronized media playback.
2. The system of any of the preceding claims, configured to provide, to the remote user application (3), indication on the playback capabilities and / or playback conditions of at least one of the plurality of player devices (10), and configured to receive, from the remote user application, a request for performing the synchronized media playback, so that the system (4) defines, based on the request, the virtual device (10).
3. The system of claim 2, configured to receive, from the remote user application, commands (31) for playback control of the virtual device (110) as a unique entity, and to send the commands (34) for playback control at least one of the player devices of the virtual device.CIN240904PEP-2025002689.DOCX 704. The system of any of the preceding claims, configured to perform an authorization procedure to authorize the at least one of the remote player devices (10) to receive the media content from the CDN (5, 5a).
5. The system of claim 4, configured to receive from the CDN (5, 5a) authentication data to be provided to at least one of the remote player devices (10) to subsequently receive the media content from the CDN (5, 5a).
6. The system of claim -5, configured to receive from the CDN (5, 5a) an authentication information and to provide the authentication information to the at least one of the remote player devices (10), to thereby permit the reception the media content from the CDN (5, 5a).
7. The system of any of the preceding claims, configured to receive, from at least one of the remote player devices, information on the playback capabilities and / or playback conditions, so as to inform the remote user application (3) on the playback capabilities and / or playback conditions of at least one of the remote player devices, in such a way that the selection of the remote player devices is compliant with the playback capabilities and / or playback conditions.
8. The system of any of the preceding claims, configured to distribute the roles among the remote player devices (10a, 10b, lOd) of the virtual device (110) as requested by the user application (3).
9. The system of any of the preceding claims, configured, in distributing the roles of the of the virtual device (110), to verify whether the roles as requested by the user application (3) for the player devices (10) are compliant with the capabilities and / or conditions of the respective player devices, and, in case the roles as requested by the user application (3) for the player devices (10) are not compliant with the capabilities and / or conditions of the respective player devices, to send an error message to the user application (3) without assigning the roles to the player devices.
10. The system of any of the precedng claims, wherein the playback capabilities include a capability of rendering video content, the system (4) being configured to define the virtualCIN240904PEP-2025002689.DOCX 71 device (10) as having at least one player device (10) with the capability of rendering the video content.
11. The system of any of the preceding claims, wherein the playback capabilities include a capability of rendering video content with a particular quality, the system (4) being configured to define the virtual device (10) as having at least one player device (10) with the capability of rendering video content at a preferred quality.
12. The system of any of the preceding claims, wherein the playback capabilities include a capability of rendering audio content, the system (4) being configured to define the virtual device (10) as having at least one player device (10) with the capability of rendering the audio content.
13. The system of any of the preceding claims, wherein the playback capabilities include a capability of rendering audio content with a particular quality, the system (4) being configured to define the virtual device (10) as having a paler device (10) having the capability of rendering the audio content at a preferred quality.
14. The system of any of the preceding claims, configured to select a virtual device primary player device (10a) among the player devices (10a, 10b, lOd) constituting the virtual device (110), so that the virtual device primary player device (10a) is the player device that receives the media content from the CDN (5, 5a).
15. The system of any of the preceding claims, configured to provide the user application (3) with template information (913), the template information providing different setup options (110a, 110b, 110c), to therefore receive, from the user application (3), information on a particular selected setup option and matching information between the player devices (10) and the roles which are needed for the particular selected setup option.
16. The system of claim 15, configured to provide, for each role of the template information, information on the player devices which have the playback conditions and / or playback capabilities to take the role, so as to not permit a selection of a player device not having the playback conditions and / or playback capabilities to take the role.CIN240904PEP-2025002689.DOCX 7217. The system of claim 15, configured to provide, for each role of the template information, a list of only the player devices which have the playback conditions and / or playback capabilities to take the role.
18. The system of any of the preceding claims, configured to maintain memory of the virtual device (110) and of which player devices (10a, 10b, lOd) have taken part to the virtual device (110), and which role has had each virtual device.
19. The system of any of the preceding claims, configured to define at least one group (210) of player devices, the group of player devices including the virtual device (210)20. The system of any of the preceding claims, configured to define multiple virtual devices according to request from the user application.
21. A player device (10, 10a, 10b, 10c, lOd, lOe, lOf, 10g) for participating to a synchronized media playback, the player device including: an hardware unit (1) to render synchronized media playback of media content; an interface to a remote assisting system (4); an interface to at least one other player device (10b-10) through a local connection; the player device (10) being configured to receive, from the assisting system (4), information (137) of being part of the virtual device (110) and role information (137a) indicating a playback role that the player device (10) shall have in the synchronized media playback, to thereby constitute a virtual device (110) with the at least one other player device (lOb-lOd) to perform the synchronized media playback by receiving the media content and playingback the media content according to the playback role.
22. The player device of claim 21, configured to send, to the assisting system (4), information on its own player device's playback capabilities and / or playback conditions.
23. The player device of claim 22, wherein the playback capabilities include a capability of rendering video content, so as to subsequently receiving the playback role of rendering the video content.CIN240904PEP-2025002689.DOCX 7324. The player device of any of claims 22-23, wherein the playback capabilities include a capability of rendering video content with a particular quality, so as to subsequently receiving the playback role of rendering the video content at the particular quality.
25. The player device of any of claims 22-24, wherein the playback capabilities include a capability of rendering audio content, so as to subsequently receiving the playback role of rendering the audio content.
26. The player device of any of claims 22-25, wherein the playback capabilities include a capability of rendering audio content with a particular quality, so as to subsequently receiving the playback role of rendering the audio content at the particular quality.
27. The player device (10) of any of claims 21-26, configured to participate to a player device primary election, with the at least one other player device (lOb-lOd) constituting the virtual device (110), the virtual device primary election electing a virtual device primary player device (10a), the remaining player device or player devices (10b, lOd) in the virtual device (110) being defined virtual device secondary player device or virtual device secondary player devices (10b, lOd); in case of the player device (10) being elected virtual device primary player device (10a), receive media content (51) from a content distribution network, CDN, (5, 5a) and retransmit the media content (51) to the virtual device secondary player devices (10b, lOd), and in case of the player device not being elected virtual device primary player device (10a) and therefore resulting a virtual device secondary player (10b, lOd), receive media content (51) from the virtual device primary player device (10a), both in case of being elected virtual device primary player device and in case of being a virtual device secondary player device, control the at least one hardware media renderer unit (1) to render the media content (51).
28. The player device of claim 27, configured to receive a content retrieval information (37) from the assisting system (4) informing of media content to be retrieved from the CDN (5) and to be subsequently rendered, so that: in case of the player device (10) being elected virtual device primary player device (10a), receiving, from the content distribution network (5, 5a), the media content indicatedCIN240904PEP-2025002689.DOCX 74 in the content retrieval information and retransmitting the media content to the virtual device secondary player device or virtual device secondary player devices (10b, lOd); and in case of the player device (10) being a virtual device secondary player device (10b, lOd), receiving, from the virtual device primary playback device (10a), the media content indicated in the content retrieval information.
29. The player device of claim 28, wherein the content retrieval information (37) includes at least one address of the media content to be retrieved and / or information to be provided to the CDN (5) to request the media content.
30. The player device of claim 28-29, wherein the content retrieval information (37) includes authenticating information for authenticating at least the virtual device primary player device (10a) to receive the media content from the CDN (5).
31. The player device of any of claims 28-30, configured to receive, from the assisting system (4), an identifier (37a) identifying the content retrieval information (37).
32. The player device of any of claims 27-31, configured to receive a queue information (37) from the assisting system (4) informing of a queue of media content to be rendered, so that: in case of the player device (10) being elected the virtual device primary player device (10a), receiving, from the content distribution network (5, 5a), the media content according to an order defined by the queue (37) and retransmitting the media content to the virtual device secondary player device or virtual device secondary player devices (10b, lOd); and in case of the player device (10) being a virtual device secondary player device (10b, lOd), receiving, from the virtual device primary playback device (10a), the media content according to the order defined by the queue (37).
33. The player device of claim 32, wherein the queue information includes a queue identifier which identifies the virtual device.
34. The player device of any of claims 27-33, configured, once having received the information (137a) of being part of the virtual device (110):CIN240904PEP-2025002689.DOCX 75 transmit, through the local network (220), to the other player devices (10a, 10b, lOd) of the virtual device (110), information (214) including at least discriminating information attributed to the player device and / or the status of the player device, receive, through the local network (220), information (214) from the other player devices (10a, 10b, lOd) of the virtual device (110), each including at least discriminating information attributed to the sending player devices and / or the status of the sending player device, thereby defining the virtual device primary player device (10a) based on the discriminating information of all the player devices of the virtual device (110).
35. The player device of claim 34, wherein the discriminating information includes at least one performance indicator of the sending player device.
36. The player device of claim 34 or 35, wherein the discriminating information includes at least one stability value attributed to the sending player device.
37. The player device of any of claims 34-36, wherein the discriminating information includes at least one random value.
38. The player device of any of claims 34-37, wherein the discriminating information includes at least one unique identifier of the sending player device.
39. The player device of any of claims 34-38, configured to apply a predetermined rule shared with the other player devices of the virtual device (110) to elect the primary player device (10a) by examining the discriminating information of all the player devices in the virtual device (110).
40. The player device of any of claims 27-39, configured: in the case of the player device (10b, lOd) being a virtual device secondary player (10b, lOd), and in the case of not receiving the media content (51) and / or a heartbeat message from the virtual device primary player device (10a) for a time delay over a predetermined delay threshold, participate to a further virtual device primary election to elect a new virtual device primary player device, and, in the case of the player device being electedCIN240904PEP-2025002689.DOCX 76 as new virtual device primary device, receive media content (51) from the content distribution network (5, 5a) and retransmit the media content (51) to the virtual device secondary player devices (10b, lOd), and in case of the player device not being elected new virtual device primary player device (10a) and therefore resulting a virtual device secondary player (10b, lOd), receive media content (51) from the new virtual device primary player device (10a), and both in case of being elected new virtual device primary player device and in case of being a virtual device secondary player device, control the at least one hardware media renderer unit (1) to render the media content (51) according to the assigned role.
41. The player device of any of claims 27-40, configured: in the case of the player device (10b, lOd) being a virtual device secondary player (10b, lOd), and in the case of receiving the media content (51) and / or heartbeat messages from the primary player device (10a) with an error rate over a predetermined error-rate threshold, participate to a further virtual device primary election to elect a new virtual device primary player device, and, in the case of the player device being elected as new virtual device primary device, receive media content (51) from a content distribution network (5, 5a) and retransmit the media content (51) to the virtual device secondary player devices (10b, lOd) of the virtual device (110), and in case of the player device not being elected new virtual device primary player device (10a) and therefore resulting a virtual device secondary player (10b, lOd), receive media content (51) from the new virtual device primary player device (10a), and both in case of being elected new virtual device primary player device and in case of being a virtual device secondary player device, control the at least one hardware media renderer unit (1) to render the media content (51) according to the assigned role.
42. The player device of any of the claims 40 and 41, wherein the further virtual device primary election is performed as the virtual device primary election.
43. The player device of any of the claims 40 and 41 and 42, wherein the previous virtual device primary player device is excluded from the virtual device primary election.
44. The player device of any of claims 27-43, configured to have, from the first primary player device election, a knowledge of the first secondary player device to be automaticallyCIN240904PEP-2025002689.DOCX 77 elected in case of the virtual device primary player device is not available without further virtual device primary device election, and: in case of the player device being the new virtual device primary player device, receive the media content (51) from the content distribution network (5, 5a) and retransmit the media content (51) to the virtual device secondary player devices (10b, lOd) in the virtual device, and in the case of the player device not being elected the new virtual device primary player device and therefore resulting a virtual device secondary player (10b, lOd), receive media content (51) from the new virtual device primary player device (10a), both in case of being elected the virtual device primary player device and in case of being a virtual device secondary player device, control the at least one hardware media renderer unit (1) to render the media content (51) according to the assigned role.
45. The player device of any of claims 27-44, further comprising at least one physical interface for receiving a physical user's command for playback event, configured so that: in case of the player device (10) being the virtual device primary player device (10a), adopt the user's command for playback event, and transmit, to the virtual device secondary player devices (10b, lOd), the user's command for playback event, and in case of the player device being a virtual device secondary player (10b, lOd), send a request to the virtual device primary player device (10a) for accepting the user's command for playback event.
46. The player device of any of claims 27-45, configured to receive a command from the assisting system (4) for playback event (34) , the player device being configured to adopt the command irrespective of the player device being the virtual device primary player device or a virtual device secondary player device.
47. The player device of any of claims 27-46, further comprising at least one physical interface for receiving a user's command immediate playback event, configured so as to: adopt the user's command for immediate playback event by controlling the at least one hardware media renderer unit (1) to adopt the command for immediate playback event both in the case of the player device being the primary player device and in the case of the player device being a secondary player device.CIN240904PEP-2025002689.DOCX 7848. The player device of claim 47, configured, in the case of being a secondary player device, to also send a request to the primary player device (10a) for accepting the user's command for immediate playback event.
49. The player device of claim 47 or 48, wherein the command for immediate playback event regards at least the playback sound volume.
50. The player device of any of claims 27-49, configured to receive the media content without installing new application program interfaces, APIs.
51. The player device of any of claims 27-50, configured to be subjected to a primary selection, together with the at least one other player device (lOb-lOd) constituting the virtual device (110), performed by the assisting system (4), the virtual device primary selection selecting a virtual device primary player device (10a), the remaining player device or player devices (10b, lOd) in the virtual device (110) being defined virtual device secondary player device or devices (10b, lOd); in case of the player device (10) being selected virtual device primary player device (10a), receive media content (51) from a content distribution network (5, 5a) and retransmit the media content (51) to the virtual device secondary player device(s) (10b, lOd), and in case of the player device not being selected virtual device primary player device (10a) and therefore resulting a virtual device secondary player (10b, lOd), receive media content (51) from the virtual device primary player device (10a), both in case of being selected virtual device primary player device and in case of being a virtual device secondary player device, control the at least one hardware media renderer unit (1) to render the media content (51) according to the assigned role.
52. The player device of claim 51, wherein the playback capabilities and / or playback conditions include a capability and / or condition of receiving and / or transmitting the media content, through the local connection, at a particular data rate and / or error rate, so as to be subsequently selected as virtual device primary device or virtual device secondary device based at least in part of the capability and / or condition of receiving and / or transmitting the media content, through the local connection, at the particular data rate and / or error rate.CIN240904PEP-2025002689.DOCX 7953. The player device of any of claims 27-52, wherein the virtual device (110) is configured to be part of a group (210) for a synchronized playback with at least one additional player device (10g) or virtual player device.
54. The player device of claim 53, wherein the virtual device (110) is configured to take part to a group primary election, wherein, in the case of the player device being elected as the virtual device primary player device, the player device participating to the group primary election, and being configured to: in case of being elected as the group primary virtual device, receive from the CDN (5) the media content (51) and retransmit the media content to the both the virtual device secondary player devices and the other player devices and virtual player devices of the group (210), and, in case of not being elected the primary virtual device and consequently being a group secondary virtual device, receive the media content from the elected group primary player device or virtual device and retransmit the received media content to the virtual device secondary player devices of the virtual device.
55. The player device of any of claims 53-54, wherein the group primary election is performed in the same way as the virtual device primary election.
56. The player device of any of claims 53-55, configured, once having received information (37) of being part of the group (210): transmit, through the local network (220), to the other player devices and virtual devices of the group, information (214) including at least discriminating information attributed to the player device and / or the status of the player device, receive, through the local network (220), information (214) from the other player devices and virtual devices of the group, each including at least discriminating information attributed to the sending player devices and / or the status of the sending player device, thereby defining the group primary player device (lOf) based on the discriminating information of all the player devices player devices and virtual devices of the group.
57. The player device of any of claims 53-56, configured, to receive information (37) of being part of the group (210) as information on media content to be playedback, whereinCIN240904PEP-2025002689.DOCX 80 the group is formed by the player devices and virtual devices having the same media content to be playedback.
58. A user application (3) for commanding a streaming and a rendering of remote media content (51), the user application executing in a user equipment, the user application comprising: an interface to an assisting system (4) to receive information on a plurality of remote player devices (10), each controlling at least one media Tenderer unit (1) and having playback capabilities and / or playback conditions; the user application being configured for receiving, through the interface to the assisting system, information on media content on at least one remote media distribution network, CDN (5, 5a); the user application being configured to define a remote a virtual device (110) from a plurality of remote player devices (10), and the user application being configured to receive playback control commands (34) and / or content control commands (38) from a user (99) for the virtual device (110), and to retransmit the control playback commands to the assisting system (4).
59. The user application of claim 58, configured to receive from the assisting system (4) template information, the template information providing different setup options (110a, 110b, 110c), the user application being configured to select, in a user interface, a setup option among the other setup options, to receive from the user a selection of the setup option and to retransmit the selected setup option to the assisting system (4).
60. The user application of claim 58, configured to receive from the assisting system (4), for each role of the template information, information on which player devices have the playback conditions and / or playback capabilities to take the role, so as to not permit the selection of a player device not having the playback conditions and / or playback capabilities to take the role.
61. The user application of claim 58, configured to receive from the assisting system (4), for each role of the template information, a list with only the player devices have the playback conditions and / or playback capabilities to take the role.CIN240904PEP-2025002689.DOCX 8162. The user application of claims 58-61, configured to maintain memory of the virtual device (110) and of which player devices (10a, 10b, lOd) have taken part to the virtual device (110), and which role has had each virtual device, so as to redefine the virtual device in the future in case of user's request.
63. The user application of claims 58-62, configured to define at least one group (210) of player devices, the group of player devices including the virtual device (210).
64. The user application of claims 58-63, configured to define multiple virtual devices according to request from the user application.
65. A method (4) for assisting a streaming and a rendering of remote media content, the method including: interfacing (410) with a plurality of remote player devices (10), each controlling at least one media Tenderer unit (1) and having playback capabilities and / or playback conditions; interfacing (405) with at least one remote media distribution network, CDN (5, 5a); interfacing (403) a remote user application (3), to send, to the user application, a list of the remote player devices (10a, 10b, lOd, lOd), and receiving, from the user application, a selection on a plurality of remote player devices (10a, 10b, lOd) to be part of a virtual device (110) for a synchronized media playback, distributing roles among remote player devices (10a, 10b, lOd) of a virtual device (110), based on the selection performed by the user application (3), sending, to at least one (10a) of the remote player devices (10) of the virtual device (110), role information (137a) which includes instructions on a respective role of each remote player device (10) of the virtual device (110) for participating to the synchronized media playback, sending, to at least one of the remote player devices (10), information on media content to be retrieved and rendered in the synchronized media playback.
66. A method for participating to a synchronized media playback, including: rendering synchronized media playback of media content; interfacing with a remote assisting system (4); interfacing with at least one other player device (lOb-lOd) through a local connec- tion;CIN240904PEP-2025002689.DOCX 82 the method including receiving, from the assisting system (4), information (137) of being part of the virtual device (110) and role information (137a) indicating a playback role that the rendering shall have in the synchronized media playback, to thereby constitute a virtual device (110) with the at least one external player device (lOb-lOd) to perform the synchronized media playback by receiving the media content and playingback the media content according to the playback role.
67. A method for commanding a streaming and a rendering of remote media content (51), comprising: interfacing with an assisting system (4) to receive information on a plurality of remote player devices (10), each controlling at least one media Tenderer unit (1) and having playback capabilities and / or playback conditions; receiving, through the interface to the assisting system, information on media content on at least one remote media distribution network, CDN (5, 5a); defining a remote a virtual device (110) from a plurality of remote player devices (10), and receiving playback control commands (34) and / or content control commands (38) from a user (99) for the virtual device (110), and to retransmit the control playback commands to the assisting system (4).
68. A non-transitory storage unit storing instructions which, when executed by a processor, cause the processor to perform a method according to any of claims 65-67.
Citation Information
Patent Citations
Techniques for synchronizing content playback across devices
US11089352B1
Remotely controlling playback devices
US11297369B2
Time-synchronized, multizone media streaming
US20180262792A1
Methods, systems, and media for providing dynamic media sessions
US20220210502A1
Authorization management in a media playback system
US20220360585A1