Scheduled playback control

Abstract

There is disclosed a user device (100) comprising: a device hardware (102), and a media run time device software (110) configured for controlling the device hardware (102), wherein the media run time device software (102) comprises: a first control interface (112) to a cloud backend (200); and a media interface (114) to a content service (400), wherein the media run time device software (110) is configured to have a queue functionality to manage scheduled content to be played back, wherein the media run time device software (110) is configured to request, receive and playback both selected content to be played by the user device (100), and additional, unselected content to be scheduled into the first queue to derive a second, expanded queue. There is also disclosed a cloud backend (200) comprising: a user device interface (202) for interfacing with a user device (100); and a communication interface (204) for interfacing with a content service (400), wherein the cloud backend is configured to provide messages to the user device interface (202) for controlling the user device to handle media data, and to 20 receive messages from the user device via the user device interface (202), and the cloud backend further comprising an application interface (206) for interfacing with a user application transmitting information on a user input, wherein the cloud backend is configured to receive, via the application interface (206), a request for a selected content to play (726), wherein the cloud backend is configured to (749) schedule the selected content according to a first content queue with the selected content to be played in sequence by the user device (100), insert additional, unselected content to be scheduled into the first queue to derive a second, expanded queue, and 30 (751) send queue information providing information on the second, expanded queue to the user device (100) through the user device interface (202).

Description

[0001] CIN240908PEP-2025002701.DOCX 1

[0002] SCHEDULED PLAYBACK CONTROL

[0003] Description

[0004] There are disclosed techniques for scheduled playback control.

[0005] Content Services (e.g. content providers) are integrated into the Ecosystem via the Content Provider API, so users can consume content. In some scenarios a Content Service might need to insert different (unselected) content into an ongoing playback, for instance repeating announcements or advertisement to finance content. Traditionally this is done by inserting this content into the playing stream.

[0006] However, this process requires reencoding of streams at the content provider, which consumes efforts and resources.

[0007] According to an example there is provided, inter alias, a user device comprising: a device hardware , and a media run time device software configured for controlling the device hardware, wherein the media run time device software comprises: a first control interface to a cloud backend; and a media interface to a content service, wherein the media run time device software is configured to have a queue functionality to manage scheduled content to be played back, wherein the media run time device software is configured to request, receive and playback both selected content to be played by the user device, and additional, unselected content to be scheduled into the first queue to derive a second, expanded queue.

[0008] According to an example there is provided, inter alias, a cloud backend comprising: a user device interface for interfacing with a user device; and a communication interface for interfacing with a content service, wherein the cloud backend is configured to provide messages to the user device interface for controlling the user device to handle media data, and to receive messages from the user device via the user device interface, and the cloud backend further comprising an application interface for interfacing with a user application transmitting information on a user input, wherein the cloud backend is configured to receive, via the application interface, a request for a selected content to play, wherein the cloud backend is configured to CIN240908PEP-2025002701.DOCX 2 schedule the selected content according to a first content queue with the selected content to be played in sequence by the user device, insert additional, unselected content to be scheduled into the first queue to derive a second, expanded queue, and send queue information providing information on the second, expanded queue to the user device through the user device interface.

[0009] According to an example there is provided a method for a user device comprising a device hardware, and a media run time device software controlling the device hardware, wherein the media run time device software comprises a first control interface to a cloud backend and a media interface to a content service, wherein the method includes managing a queue functionality to manage scheduled content to be played back, wherein the media run time device software requests, receives and plays back both selected content to be played by the user device, and additional, unselected content to be scheduled into the first queue to derive a second, expanded queue.

[0010] According to an example there is provided a method for a cloud backend comprising a user device interface for interfacing with a user device and a communication interface for interfacing with a content service, the method including providing messages to the user device interface for controlling the user device to handle media data, and receiving messages from the user device via the user device interface, and the method including interfacing with a user application transmitting information on a user input, wherein the cloud backend is configured to receive a request for a selected content to play, the method including scheduling the selected content according to a first content queue with the selected content to be played in sequence by the user device, inserting additional, unselected content to be scheduled into the first queue to derive a second, expanded queue, and sending queue information providing information on the second, expanded queue to the user device through the user device interface.

[0011] Preferred embodiments of the present invention are subsequently disclosed with respect to the accompanying drawings, in which: CIN240908PEP-2025002701.DOCX 3

[0012] Fig. 1 illustrates an implementation of a cloud ecosystem in accordance with an embodiment;

[0013] Fig. 2a illustrates a preferred embodiment of the user device;

[0014] Fig. 2b illustrates a preferred embodiment of the cloud backend;

[0015] Fig. 2c illustrates a preferred implementation of a user application apparatus;

[0016] Fig. 2d illustrates a preferred implementation of the ecosystem consisting of the user device, the user application, the cloud backend and the data instances;

[0017] Fig. 3 illustrates procedures to be done by a device maker for producing the user device;

[0018] Fig. 4 illustrates a communication sequence for the purpose of onboarding a device that acts as the apparatus for performing a user application;

[0019] Fig. 5 illustrates a sequence of messages for the purpose of user information handover;

[0020] Fig. 6a illustrates a first portion of a sequence of messages between different entities for the purpose of playing content; and

[0021] Fig. 6b illustrates a further part of the sequence of messages for the purpose of playing a content and or the purpose of performing control playback during playing

[0022] Fig. 6c illustrates a further part of the sequence of messages for the purpose of playing a content and or the purpose of performing control playback during playing.

[0023] Fig. 1 shows an example of a cloud in which the invention lives. The cloud includes a cloud backend 4 (200), a user application 3 (300), a user device 100 (10, also called player device) with a media runtime device software 110 and a hardware device 1 (102), and a content service (data instance) 5, 400. Further features are discussed here below.

[0024] Fig. 2a illustrates a user device 100 (player device 10) comprising a device hardware 102 (e.g. display, loudspeaker, etc.) and a media runtime device software 110 (sdk player 2). This media runtime device software is running on the user device and is configured for controlling the device hardware as illustrated by the connection between block 1 10 and 102. The media runtime device software comprises a control interface 1 12 to the cloud backend 4 (200) in order to exchange messages between the media runtime device software in the device and the cloud backend. Furthermore, the user device 100 comprises a media interface to a typically remote data instance for connecting to a remote data instance typically via a playable URL (unique resource locator) in order to then obtain a stream of CIN240908PEP-2025002701.DOCX 4 media data from the remote data instance. Alternatively, the media interface can also send a request for a URL to the remote data instance and, then, receives this data from the data instance via the media interface and, then, an upload from the user device 100 to the remote data instance can be performed when the data instance is a data sync such as the YouTube platform or the TikTok platform. The data that are uploaded via the media interface 114 are typically data acquired by the user device hardware, which is, for example, a camera or a microphone.

[0025] For replaying a data stream received via the media interface 1 14, the user device typically has a display or speakers or both.

[0026] Particularly, the media runtime device software 110 is configured to receive, as for controlling the device hardware via the control interface 1 12 and to receive or transmit content via the media interface 114.

[0027] Thus, it is made sure that the high volume data traffic only takes place via the media interface 114, and this high-volume traffic is capped out from the cloud backend, since the media interface 114 is not interfacing with the cloud backend.

[0028] In a preferred embodiment, the media runtime device software is configured to receive, via the control interface, metadata comprising information identifying the data instance under consideration and an access information such as a playable URL. Based on this information, the media runtime device software is configured to access the data instance using the received access information via the media interface 114.

[0029] In one embodiment, the data instance is a content provider. In such an implementation, the media runtime device software 1 10 is configured to receive playback metadata identifying an intended content data for playback and the access information via the control interface 112, and to send the playback metadata to the content provider via the media interface 114 and to receive the content for playback from the content provider via the media interface 114.

[0030] In the preferred embodiment, the user device comprises a hardware abstraction layer 11 interfacing the device hardware on the one hand and the media runtime device software 110 on the other hand. The hardware abstraction layer 1 1 is configured for detecting a playback event. CIN240908PEP-2025002701.DOCX 5

[0031] The media runtime device software is configured to receive the playback event from the hardware abstraction layer 11 e.g. via control line 26. However, this playback event is not executed, but is forwarded to the cloud backend 200 (4) via the control interface 1 12. When a confirmation on the playback event from the cloud backend is received via the control interface 112, then the hardware abstraction layer 1 1 is instructed by the media runtime device software 1 10 to perform the playback event.

[0032] However, when no such confirmation for the playback event is received from the cloud backend, then the playback event is not executed.

[0033] Thus, it is made sure that the content provider 400 (5) has full control not only on the fact that an unintended description is forbidden or so, but also how the media data is handled in the user device.

[0034] Nevertheless, in order to give the user of the user device as much control as possible, a pressing of a certain switch on the user device which is typically, a start button, a stop button, a fast forward or fast rewind or pausing button, is nevertheless detected, but immediately executed as in prior art devices. Instead, this hardware event is forwarded to the cloud backend and only when the cloud backend confirms this event, then this event is actually executed.

[0035] Furthermore, the user device comprises a secure storage for storing a unique device ID which is required for identifying the user device with respect to the cloud backend.

[0036] Furthermore, the device hardware is configured to only replay content identified via a command received via the control interface and received via the media interface.

[0037] In a further embodiment, the media runtime device software or “SDK kit” is configured to also have a web browser functionality 21 , a media player functionality 22, a content description functionality 28, and a queue functionality 116.

[0038] In a further embodiment, the media runtime device software 1 10 is configured to receive a playable unique resource locator from the control interface, and the user device then interfaces with the specific remote data instance, from which the playable URL is coming CIN240908PEP-2025002701.DOCX 6 from via the media interface to stream media data located at the playable URL at the content provider via the media interface 1 14.

[0039] The media runtime device software 110 may access (or include) a cache 23, in which already-playedback media content (in particular the unselected media content) may be stored, so as to be subsequently re-playedback.

[0040] Importantly, the media run time device software 1 10 has the queue functionality 116 to manage scheduled content to be played back. The media run time device software 110 requests, receives and playsback both the content to be played indicated in the queue. The cloud backend 200 may be inputted, from the user application 300, with content information on a user input, and in particular may receive a request for a selected content to play. The cloud backend 200 may: schedule the selected content according to a first content queue with the selected content to be played in sequence by the user device, insert additional, unselected content to be scheduled into the first queue to derive a second, expanded queue, and send queue information providing information on the second, expanded queue to the user device through the user device interface.

[0041] While the first queue has content selected by the user, the second queue is expanded and includes unselected content. This unselected content could be, for example, commercial information or non-semantic information.

[0042] In practice, the cloud backend 200 may send information on how to retrieve the content of the second, expanded queue. For example, the cloud backend 200 may send content metadata including the address of both the selected content and the unselected content, for example. In this way, the user device 100 may easily retrieve the unselected content. The content metadata may include playable URL or other data, by which the media runtime device software 1 10 running on the user device can request the content via a message (752, see below) to the content service 400.

[0043] The information on how to retrieve the additional, unselected content may be managed by application programming interfaces, APIs. The APIs may run in the run time device software 110, but be received from the cloud backend 200 e.g. in a run time device software initialization session. The cloud backend 200 may receive the content service APIs from the CIN240908PEP-2025002701.DOCX 7 content service, e.g. in another initialization session. Once the content service APIs are downloaded by the run time device software 1 10, they can stay resident and continue operating, e.g. until a, update session (e.g. through updated network 6 which sends updating data 61 ), for example. It is the content service APIs that, during playback, manage the queues (e.g. they receive the information 749, 751 ).

[0044] It is possible that the cloud backend 200 to send to the user device 100 (e.g. in particular to the media run time device software 110) instructions directed to temporarily store the additional, unselected content, to thereby refrain from re-receiving the additional, unselected content after a first reception. This aspect may be particular relevant in the case of the additional, unselected content including commercial information.

[0045] It is possible that the cloud backend 200 to send to the user device 100 (e.g. in particular to the media run time device software 1 10) instructions directed to monitor a connection between the user device 100 and the content service 400 (3). In cases, it is possible for the user device 100 to playback the previously-stored additional, unselected content (stored in the cache 23) in case of breakdown of the connection between the user device and the content service.

[0046] It is possible that the cloud backend 200 send to the user device 100 (e.g. in particular to the media run time device software 1 10) instructions directed to mix at least one selected content with at least one unselected content, to thereby playback a mixed scene with both the at least one selected content and the at least one unselected content. For example, the unselected content could be a commercial banner, which appears overlapped onto a selected video, e.g. occupying a portion of a display. Or, the unselected content could be an audio message, which appears mixed with a selected audio signal. Or, the unselected content could be an audio message, played back synchronously with a selected video. Or, the unselected content could be a video message, displayed synchronously with a selected audio signal.

[0047] In some cases, the user device receives the unselected content based on its position (e.g. geographical position). The cloud backend 200 shall retrieve the geographical position of the user device (e.g., from network-based information and / or from sensor information, such as information from global positioning system, GPS). In other cases, instead of the geographical position, it could be possible to network position (e.g., according to a particular local network to which the device is connected to). In examples, the position may be CIN240908PEP-2025002701.DOCX 8 obtained from an address (e.g.,. TCP / IP address). For example, different user devices may receive different unselected content by virtue of their different positions.

[0048] The cloud backend 200 may receive a request for a playback of a selected content on a selected device from the user application, and to send, to the user device (100), the metadata for the selected content to the selected device.

[0049] Fig. 2b illustrates an implementation of the cloud backend 200 that has a user device interface 202 for interfacing with the user device such as the user device 100 of Fig. 2a. Furthermore, the cloud backend comprises a communication interface for interfacing with a typically remote data instance such as any one of the data instances indicated at 400 in Fig. 2d, where individual data instances 401 , 402, 403, 404, 405 are illustrated. These remote data instances also correspond to the media CDN 5 of Fig. 1 .

[0050] The cloud backend is configured to provide messages to the user device interface for controlling the user device to handle media data and to receiving messages from the user device via the user device interface 202. Furthermore, the cloud backend is configured to communicate to the data instance via the communication interface 204 regarding an authorization of a user of the user device, where this communication comprises receiving data from the data instance or sending data to the data instance. However, this data does not comprise media data to be downloaded or to be uploaded from the user device to the data instances.

[0051] Furthermore, the cloud backend comprises an application interface 206 for interfacing with a management application such as the user application 300 illustrated in Fig. 2c or 2d.

[0052] The cloud backend 200 additionally comprises, for performing the corresponding tasks, libraries / data bases 208 that can be distributed in the cloud or that can also be located on a single server. Furthermore, the cloud backend also comprises a processing engine 210 that can also be a distributed processing engine or a concentrated processing engine.

[0053] The playback / upload control of the user device is performed by the cloud backend typically without high-volume traffic in the cloud backend. Only control data is communicated within the cloud backend and control data is received via the cloud backend interfaces 202, 204, 206 and corresponding control data is output via these interfaces as well. CIN240908PEP-2025002701.DOCX 9

[0054] Fig. 2b shows a queue generator 207 which defines the first queue with the selected content) and the second, expanded queue with the unselected content. Queue information will provide information on the media content indicated in the first and / or second queue.

[0055] Fig. 2b also shows that the libraries / data bases 208 include a database 209 with content service APIs. In alternative or additionally, the content service APIs may be received from the content service 200.

[0056] Fig. 2c illustrates an apparatus 300 for performing a user application which comprises a user interface 302 which is preferably a human machine interface such as a touch screen interface or speech interface in order to provide messages to a user or from a user.

[0057] Furthermore, the apparatus comprises a cloud backend interface 304 for communicating with a cloud backend such as the cloud backend of Fig. 2b or the corresponding elements located in the cloud backend in Fig. 1 comprising a playback management element, device cues, a device management element 35, for example.

[0058] The apparatus is configured to communicate, via the user interface 302, to receive user commands regarding a selection of one or more data instances and a selection of content to be sent to a data instance or to be retrieved from a data instance and to display messages for the user, to communicate via the cloud backend interface 304 regarding the selection of the one or more viewed data instances to be accessed by the user devices, and to communicate via the cloud backend interface regarding the selected content for sending from the user device to a corresponding data instance or for sending from the data instance to the user device.

[0059] Hence, the cloud backend interface 304 cares for the selection of data instances for the selection and control of user devices, and for the selection of media content for the upload and download.

[0060] Fig. 2d illustrate an overview of the cloud ecosystem and how the individual interfaces are connected to each other in the four greater “entities” i.e., the user device 100 with the media runtime device software 1 10, the user application apparatus 300 with the user interface 302 on the one hand and the cloud backend interface on the other hand, the remote data instances 401 , 402, 403, 404, 405, with the high volume data connection 51 between the media interface 1 14 of the user device and the media interface of the corresponding data CIN240908PEP-2025002701.DOCX 10 instances and, on the other hand, the communication interface 204 of the cloud backend for communicating with the remote data instances, the user device interface 202 of the cloud backend for communicating with the user device and the application interface 206 for communicating with the user application apparatus 300.

[0061] In an embodiment, the user device is responsible for playback of content from the remote data instances, and this device, in a minimum implementation, only consists of logic to ensure this capability and, additionally, has the ability to display the playback state in case of a screen connected to the user device or integrated in the user device. The user device only reacts and performs actions based on commands received from the cloud backend via the control interface 1 12 and there is no local ability to change what is being played. The user device only operates in conjunction with the cloud backend except for already cached or pre-fetched content which will continue to play even without the connection to the cloud backend. Any buttons included in the hardware of the user device can remain active, but not active in the sense that a command implemented by a button is immediately executed, but only active in the sense that the command is detected, forwarded to the cloud backend, confirmed by the cloud backend and only when the confirmation is there, executed in the hardware of the user device.

[0062] The user device supports all playback capabilities provided by the media runtime device software including playback of DRM protected content and synchronized playback.

[0063] Devices on the same local network can be used in a group playback leveraging synchronized playback, where the synchronization and distribution of content is performed locally. This is realized by electing one device as the primary device, this primary device will receive reactive content stream and will distribute it over the local network to all other participating devices. This primary device is being elected algorithmically among all participating devices and can change in the course of active playback, specifically in case of devices becoming online or offline in the course of playback.

[0064] The user device also provides an interface for onboarding purposes that allows an app on a mobile device to provide network information to the user device in order to connect it to a local network.

[0065] The actual hardware used in the user device is, in an abstract representation, a hardware abstraction layer being provided a device maker and used together with the media runtime CIN240908PEP-2025002701.DOCX 1 1 device software. The HAL is used to control network interfaces, receive signal input signals and receive status information for further use from the media runtime device software 110, among other needed abstractions to interact with provided hardware such as audio or video output. The device always identifies itself towards all other participants in the ecosystem based on a unique device ID securely stored on the user device 100.

[0066] The cloud backend comprises multiple services with detailed responsibilities. The cloud backend 200 is the main control and management interface within the ecosystem, where all changes in the ecosystem are managed and controlled via the cloud backend 200. Preferably, the cloud backend comprises a device management element 35 illustrated in Fig. 1. In this device management, it is made sure that every user device is registered as owned by a user. Users have the ability to see their own devices, name, tech, and deregister them. Users can create and delete device groups that allow using a device group like a single device.

[0067] Furthermore, the cloud backend comprises a playback management that enables the users of the ecosystem to control various playback aspects of the owned devices of the user. Preferably, playback of audio, images, and videos is supported as well as an upload of audio data, video data or image data or media data.

[0068] Users can control single devices, view content from content providers and control the playback thereof independently from the local network. This means that the controlling device such as a mobile phone, browser or tablet or generally the user application apparatus 300 can be connected to any network. The playback control includes start, stop, seek, skip, and other functionalities offered by the content provider the content is coming from.

[0069] Restrictions set by the content provider are respected, for example, skip limits, etc. Users can create playlists that can be composed of items from any content provider of any supporting media type. The user-created playlists can be queued on a device owned by the user. The device can offer any playback control operations directly via buttons and the user can control playback directly from there. However, such control via the buttons is not immediately executed but is only executed when confirmed by a message originating from the user device interface 202 of the cloud backend 200. CIN240908PEP-2025002701.DOCX 12

[0070] Users can control playback targeting a device group containing arbitrary devices the user owns. Each separate network segment of the device group will play the content synchronously and technically act as one device.

[0071] Users can control playback targeting virtual device and the content will be played according to the virtual device definition.

[0072] Additionally, the cloud backend comprises a content service management that allows the end user to connect to content services or, as named before, data instances that are integrated via the functionalities performed by the user application apparatus 300. End users need to have a subscription / account for the corresponding service at the remote data instance they want to connect and the corresponding user application must allow the user to have access to the service.

[0073] The cloud backend additionally comprises a media app management functionality that defines all meta data for content services, descriptions, translations, icons as well as specification on how to access content via an API for web applications. It also allows a content service to configure restrictions for the service allowing to restrict a service by availability by device meta data (manufacturer, type, model, etc.), regions or accounts. These restrictions can be formed by using deny lists and allow lists.

[0074] Additionally, an update management is provided in the cloud backend. In general, devices receive updates of their running device media player software over the air. For that, the device may check frequently if there is a new software version available for a specific device model and install it accordingly. The update will be installed when the device is not performing playback, however, devices performing continuous playback will forcefully install the updates after a defined time. Update packages are stored on the distributed CDN (Content Delivery Network) illustrated at 5 in Fig. 1 to ensure high availability and low latency. An update package contains a specific version of the media runtime device software that runs on the device and informative meta data for the update mechanism to be able to select the correct update package. Update packages can be private, so that they can be restricted for development purposes for specific manufacturers, device models or users. The updates are managed through the cloud backend 200 and the device manufacturers. Manufacturers can define if their device models receive a specific version of the update or not. CIN240908PEP-2025002701.DOCX 13

[0075] The user application apparatus 300 relies on rendering the web frontend preferably within a native application. On top of rendering the web frontend, the user application is also integrated in a playback device also allowing remote control via the web frontend or other applications. On top of these functionalities, a further purpose of the user application apparatus 300 is to provide support for onboarding of devices.

[0076] The onboarding of devices is happening by using a camera of a mobile device to scan a QR code. The QR code contains information about the device ID (uniquely identifying the device) as well as optionally information about a local wireless access-point of the device. After scanning the QR code, the user application 300 will connect the mobile device to the wireless access-point of the device. In the next step, the device itself is providing the mobile app with a list of available wireless networks that the server device can connect to. The user can select the wireless network to connect to and will be asked to provide the necessary wireless network password. This information is transferred from the user application to the device on which the user application is running and, then, a connection to the wireless network is established and, then, connection to the cloud backend 200.

[0077] Subsequently, further aspects of the present invention are discussed in more detail.

[0078] A further element that is mentioned in Fig. 3 is a device descriptor. Particularly, as illustrated in Fig. 3, a device maker 500 is performing several procedures to generate a cloud serviceready user device.

[0079] Particularly, the device maker produces a hardware device and, then, as illustrated in item 501 , put the media runtime device software on the device. Furthermore, as illustrated in item 502 a set of unique device IDs is acquired and, then, as illustrated in step 503, the device IDs are securely put on the device, one device ID per device.

[0080] Furthermore, the device manufacturers create and configure a device descriptor as illustrated in item 504 and, additionally, set rules for allowing or disallowing certain media apps as illustrated in 505.

[0081] In an embodiment, these rules are stored in the cloud backend and, particularly, in the certain libraries or databases of the cloud backend illustrated at 208 at Fig. 2b. CIN240908PEP-2025002701.DOCX 14

[0082] Fig. 4 illustrates a preferred procedure of onboarding of the user application apparatus 300 to become a user device. This is done implicitly after the user 99 downloads 601 the app from a store and logs into the app for the first time. Particularly, for this purpose, the user 99 starts the app in step 602, and the user application receives, via the user interface 302, a starting message of the app 602. Then, the app indication apparatus 300 displays a login screen and receives a login message 603 from the user 99.

[0083] When the user is logged in, the application 300 receives a device ID that is internally assigned to the user so that the user becomes the owner of the user device, on which the user application is running, i.e., the owner of the user application apparatus.

[0084] In an embodiment illustrated in Fig. 4, the user application sends out credentials to the cloud backend 200. This occurs by sending out the message 604 via the cloud backend interface 304 of the user application to the application interface 206 of the cloud backend. Then, in reply to this message 604, the cloud backend 200 replies, via the application interface 206, with a message 605 comprising a token, which is received from the user application 300.

[0085] Then, using this token, and when the starting is performed for the first time as illustrated in item 606, the application retrieves, from the apparatus, on which the user application is running, a MAC information (media access control information), and sense this data via a message 607 to the cloud backend 200 via the application interface 206 of the cloud backend. In response to this data, the cloud backend creates a device ID as illustrated in block 608 and, additionally, stores device data derived using the EMEI or the MAC data and outputs the device ID using a method 610 via the application interface to the cloud backend interface 304 of the user application. The user application then stores this device ID in the apparatus, on which the user application is running.

[0086] Now, the device, in which the user application 300 is running has a unique device ID known and managed by the cloud backend 200. Therefore, in order to be able to finally become a device, it is only required that, for example, using the device ID, the media runtime device software already existing on the apparatus, in which the user application is running, is activated, or this data is downloaded so that the device, in which the user application is running, is implemented to be a compliant device in backhaul system. CIN240908PEP-2025002701.DOCX 15

[0087] Fig. 5 illustrates a further sequence of messages between the user 99, the user application 300, the cloud backend 200 and the data instance or content service 400 for the purpose of performing a user information handover.

[0088] A user has access to all content services or data instances that are allowed to be used by the user. To be able to use a specific content service within and under the control of the cloud backend, the user has to authorize the specific content service or data instance.

[0089] For this purpose, the user 99 sends an authorized content service message 620 to the user application via the user interface 302 and the user application receives this message 620 via its user interface.

[0090] In response to this message 620, the user application requests an authorization of the content service for this user via the cloud backend interface 304 and a message 621 .

[0091] In response to this message 621 received by the cloud backend 200 via the application interface 206, the cloud backend looks-up a certain authorization method which is offered by the content service under consideration using a certain database where this information is stored in the cloud backend. Then, when this specific authorization method is found by the cloud backend, the cloud backend sends an authorization request message 622 to the content service or remote data instance via the communication interface 204 illustrated in Fig. 2 in order to access the remote data instance under consideration, but not for media download or upload but only for control purposes.

[0092] In response to this request 602, the content service replies with a request 623 to the cloud backend 200 which is forwarded from the cloud backend 200 to the user application 300. In the message 223, the user application is required to provide credentials i.e., login data for the specific content service 400 and preferably also to provide the consent to the terms required by the specific content service.

[0093] This data is input from the user 99 via the user interface (not illustrated in Fig. 5) and sent back via message 624 to the cloud backend and from there to the content service 400. In response to that, the content service sends an access token via message 625 and additionally, also a refreshment of the access token as required under the specific circumstances is performed. CIN240908PEP-2025002701.DOCX 16

[0094] Furthermore, the cloud backend stores this token and typically also the expiration of the token is illustrated in 626 for further use as a playing content functionality illustrated subsequently with respect to Fig. 6a or 6b.

[0095] Thus, after successful authorization, the ecosystem stores the tokens together with the user ID so that they can be fetched as soon as to content service API is accessed. The tokens are kept only in the cloud backend and do not have to be managed by the application 300 in any sense. Furthermore, due to the fact that the user has authorized the content service a single time, a next access to the content service via the cloud backend by the user application will not require in any further input of the credentials or passwords by the user.

[0096] When a list of authorized content services is needed, a database included in block 208 of the cloud backend 200 (Fig.2b) is queried for all content services where tokens for the user under consideration exist.

[0097] Fig. 6a and 6b and 6c illustrate a procedure showing a sequence of steps / message between the user 99, the user application 300, the cloud backend 200, the media runtime device software 1 10 and the content service 400 for playing content with the cloud ecosystem. The playing of content of the uploading of content within the ecosystems involves of the corresponding participants of the ecosystem such as device makers, content services, users. Furthermore, it is to be noted that the heavy-weight data or content is retrieved directly by the device media runtime, while the cloud backend only handles the playback commands. The data of authorized devices and authorized content services for a user are stored and retrieved from databases that are part of the cloud backend. These databases contain all data that is necessary for authorized access to a content service, where tokens are needed for access. When, in the Fig. 6a, Fig. 6b, Fig. 6c diagram such data is looked- up in the cloud backend, it is retrieved from a corresponding database 208.

[0098] Furthermore, it is to be noted that the illustration in Fig. 6a makes clear that each message, such as message 700, between the user application 300 and the cloud backend 200 refers to two actions of the entities involved. The first action is that the user application sends out the message 700 via the cloud backend interface 304, and the second action performed by the cloud backend 200 is that the cloud backend 200 receives the message 700 via the application interface 206. For each of the corresponding messages that are exchanged between two entities in the Fig. 6a, Fig. 6b, Fig. 6c illustration or in the Fig. 5 and Fig. 4 illustration, there exist the two actions, i.e., on behalf of the entity sending the message and CIN240908PEP-2025002701.DOCX 17 on behalf of the entity receiving the message, where the corresponding interfaces involved are clear from the illustration. Hence, although not explicitly outlined all the time, these two actions are always performed.

[0099] Subsequently, Fig. 6a and Fig. 6b and 6c are discussed in more detail.

[0100] In the action 602, the user 99 opens the user application running on the user application apparatus. In step 603, the user logs into the app and, in reply to this logging into the app, the user authenticates itself via the message 700 from the user application to the cloud backend. Then, in order to further use the cloud backend, the user receives a token including the user’s permissions and the corresponding user ID in message 605. The user token is used in all calls to the cloud backend to identify the user and to check if access is allowed for the requested function for the user.

[0101] Then, in order to finally come to a playing of a media content, the user application sends a request for a list of content services illustrated at 702. All content services that are authorized in the cloud backend are returned. Each content service is either authorized or not authorized by the user. Hence, due to the fact that the cloud backend replies to this message 702, with a full selection of content services authorized in the ecosystem, the selection what is valid for the specific user is performed in step 704. Here, the cloud backend checks, which content services are actually authorized for the user.

[0102] Then, for each identified content service, it is checked, if the user is allowed to use the content service, if the user is not allowed, the content service is removed from the result. In a step 708, it is checked, whether the content service allows the user. Probably, the user is denied for some reason, for example by not paying a bill, etc. This is done in step 708. Then, in step 710, the cloud backend returns a list of allowed content services to the user application.

[0103] This list is presented to the user via the user interface in the user application and the user then selects a content service from the list provided 710 in the step 712. Then, based on the selected content service, the user application 300 requests a list of content in message 714.

[0104] In response to that, the cloud backend looks up, whether credentials for this content services are stored in the cloud backend. This is done in step 716, and when such user CIN240908PEP-2025002701.DOCX 18 credentials for the requested content service are found, an authentication message 718 is sent from the cloud backend to the content service via the communication interface 204 of the cloud backend. The result message 720 is illustrated in Fig. 6a and the result message can, for example, comprise a certain token, from the content service, or any other positive authentication result.

[0105] In response to the positive result 720, the cloud backend requests content metadata in a message 722 from the content service 400, and the content service 400 sends back the requested content metadata in message 724.

[0106] This list of content is then forwarded from the cloud backend to the user application in step 725 for display to the user 99 via the user interface 302.

[0107] The user 99 selects a content for replay as shown in action 726 and, then, the user application requests (728) allowed devices for the selected content from the cloud backend.

[0108] As illustrated in action 730, a list of devices that are authorized by the user is looked-up and, then, processed by means of steps 732 to 742.

[0109] This procedure represents the check, if a certain device is allowed from the list of devices that have been originally authorized by the user. Any negative answer in the list consisting of item 732 to 742 will exclude a corresponding device under consideration from the result.

[0110] Step 732 starts the processing sequence. However, the sequence of steps between 732 to 742 is arbitrary and any other sequence also provides a useful result (743 is shown to indicate that other exclusions may be performed).

[0111] Instead 734, it is determined, whether the content service under consideration allows the device brand of the user device. The information for answering this question is located in certain database 208 in the backend.

[0112] In step 736, it is determined, whether the device maker allows the content service under consideration.

[0113] This is also determined from a certain database. CIN240908PEP-2025002701.DOCX 19

[0114] Furthermore, in step 738, it is determined whether the device maker allows the playback of the content that has been selected to play in step 726.

[0115] Furthermore, in step 740, it is determined, whether the user actually allows to playback the content on the device. This procedure is useful, when, for example, a certain profile of allowed content has been installed in the cloud backend, for example for the purpose of children control.

[0116] Now, it can be that the child acts as a certain user 99, and therefore, it can easily be that the child requests a content to play that, however, is not allowed to be played back on the device as defined earlier when establishing the certain profile in a library of the cloud backend.

[0117] Finally, it is determined whether the actual user device under consideration is online and ready.

[0118] Only when all the answers in the list are positively answered, a device has in the list of allowed devices that is referred back from the cloud backend 200 to the user application via message 744.

[0119] Then, the user 99 replies with a message 745 for selecting a device from the list of allowed devices and, then, the user starts the playback of the selected content on the selected device via message 746. This user action results in a message from the user application to the cloud backend that request the playback for the selected content on the selected device, where this message is illustrated at 748 in Fig 6b.

[0120] With action 750, the cloud backend 200 provides content metadata 750 to the runtime software 1 10 (device 10, 10a). This content metadata 750 can, for example, be a playable URL or any other data, by which the media runtime device software 1 10 running on the user device can request the content via a message 752 from the user device media interface 114 via the connection to the typically remote data instances illustrated at 51 in Fig. 2d.

[0121] Then, as soon as the playable URL, for example, is received by the remote data instance, the remote data instance streams back the content as shown in message 756 and this takes CIN240908PEP-2025002701.DOCX 20 place via the media interface 114 of the user device 100 and the content is replayed as illustrated at 754.

[0122] Notably, in parallel to the event 750, unselected content is injected as indicated with 800. The unselected content is chosen in action 749, e.g. by the queue generator 207. Hence, unselected content metadata 751 are provided to the runtime software 110 (device 10, 10a). Also in this case, the unselected content metadata 750 can, for example, be a playable URL or any other data, by which the media runtime device software 1 10 can request the content via a message 752 from the user device media interface 1 14 via the connection to the typically remote data instances illustrated at 51 in Fig. 2d. The unselected content is therefore requested at 751 a (according to the metadata 751 ), and played back (751 b) and received (751 c).

[0123] In practice, the content metadata 750 refer to the first queue (the one specifically requested by the user 99), while the content metadata 751 refer to the additional, injected, unselected content to be rendered in any case. Therefore, the content metadata 750 and 751 , together, may be an example of the information providing information on the second, expanded queue.

[0124] Fig. 6c additionally illustrates the procedures that are done, when the user 99 wishes to control the playback via message 760. This control can, for example, be a pause action, a stop action, etc. As soon as the user application 300 has received a message from the user such as message 760, the user application replies with a request control playback message 762 to the cloud backend 200. Only when the cloud backend confirms that such a procedure is allowed, where this check is not illustrated in Fig. 6b, the cloud backend request control playback via message 764 from the media runtime device software 110 running on the user device 100. As soon as the control message 764 is received by the user device via the control interface 112, the user device reacts with an execution of the playback control illustrated at 766 and reports this executed control via step 768 back to the cloud backend 200 and a corresponding confirmation is sent from the cloud backend 200 back to the user application via message 770. In some examples, however, between actions 762 and 764 a check may be performed by the backend 200, to check whether the particular playback control is inhibited. In some cases, indeed, a playback control directed to avoid the playback of the unselected content is not allowed (i.e., is inhibited), thereby avoiding that the user refrains from watching or listening to the unselected content (particularly advantageous for commercial purposes). CIN240908PEP-2025002701.DOCX 21

[0125] It is to be emphasized that the order of the selection of the content service 712, the selection of the content to play 726, and the selection of the device from the list of allowed device 745 can also be done in a different order. Furthermore, in case of only a single content service and only a single user device, several checks and messages can be omitted in order to nevertheless implement the full control of the cloud backend 200 over the user device, which the media runtime device software 1 10 is installed.

[0126] The operations regarding the queues are in general transparent to the user. For example, the user application 300 is not notified of the second, expanded queue.

[0127] Discussion

[0128] Within the Ecosystem it is possible for the Content Service to define a schedule e.g. via the Content Provider API, to change playback to specified different streams e.g. at a specified time or interval. Greatly reducing the efforts to changing an existing stream, lowering the entry barrier for a Content Service to integrate this functionality, providing additional flexibility by being able to specify which stream to play at the scheduled time.

[0129] The schedule (second, expanded queue) is not visible to the user itself but is fully controlled from the Content Service. Scheduling content for playback via the Content Provider API is only possible while content from the Content Service is played on a device, it is not possible to inject content into the playback of other Content Services. Starting some scheduled playback via the Content Provider API is also not possible, preventing devices from randomly starting playback.

[0130] Cloud Ecosystem

[0131] This preferred embodiment used the group playback invention within a new cloud ecosystem, which consists of multiple parts and roles:

[0132] 1 . Cloud Ecosystem - a uniform cloud media platform connecting Users, Devices, and Content Services via the Cloud Backend and providing the Management Services.

[0133] 2. Device Maker - any party designing and / or manufacturing Devices. It includes also private technical savvy users acting as such Device Maker by installing the Media Runtime Device Software on a device of their own (e.g., a private camera connected to an SoC of a User).

[0134] 3. Device Brand - a commercial brand under which a Device Maker offers Devices CIN240908PEP-2025002701.DOCX 22

[0135] 4. Content Services 3, 400 - any party having cloud services for sending (downstream from cloud service) or receiving (upstream to cloud service) content on or from a Device (examples for sending are e.g., services like Spotify, Netflix, or advertising content services, business radio services, private streaming services, content management systems (including systems handling scheduling, delivery and curation) and for receiving can be e.g., a service to store digital camera data in the cloud).

[0136] 5. Media App - a descriptor defining and enabling the connection between the Content Services, the Cloud Backend and the Device. And enabling a Content Service additional functionality by communicating with the Device via the Media Runtime Device Software, like e.g., like saving content on a Device. A Media App is only available in the Cloud Backend, the Device and the Media Runtime Device Software do not have information about the Media App, providing additional security and separation of concerns.

[0137] 6. Device - a hardware device having at least one CPU and a connection to the internet, having Device Capabilities and having integrated the Media Runtime Device Software. A Device is uniquely identified by a Device ID and can have a User as its owner.

[0138] 7. Media Runtime Device Software - embedded software running on a device and communicating both with the device hardware and with the Cloud Backend, and supporting Formats and other capabilities depending on the Device Capabilities. The Media Runtime Device Software is independent of specific Media Apps, and is compiled to many OS and SoCs systems.

[0139] 8. Cloud Backend 4, 200 - cloud services enabling and maintaining the connection and control of the Media Runtime Device Software of all Devices. Through this connection enabling the Cloud Ecosystem. The Cloud Backend has its own Web APIs (e.g., REST APIs). Media Apps are used in the Cloud Backend to communicate content for playback from and to Devices.

[0140] 9. Device Capabilities - the list of all hardware capabilities of a device supported by the Cloud Ecosystem. This includes all supported Formats, and including capabilities like supported audio and video codecs for encoding and for decoding, performance capabilities, security capabilities such as secure boot, DRM capabilities, payment capabilities, etc.

[0141] 10. Device ID - a unique identifier for each Device

[0142] 11 . App - an application using the Management Services to manage the Devices and the Account. Optionally, an App can have its own Media Runtime Device Software making the App a Device itself. An App can be a mobile app running on CIN240908PEP-2025002701.DOCX 23 a smart phone or tablet or can be running in a browser or an application running on a PC.

[0143] 12. User - a user with an Account. Users can own their own Devices or use Devices of any other user (e.g., colleagues, or family members).

[0144] 13. Role - each User has one or more roles to define the rights of the User

[0145] 14. Account - a unique user with a free or paid subscription to the Cloud Ecosystem

[0146] 15. Formats - at least one of audio, video, pictures, text and interactive (web) content playing / displaying, and / or generating at least one of audio, video, pictures, text and interactive (web) content. Examples for playing are (i.) speakers for audio, (ii.) screens, TVs for video and pictures, (iii.) screens, keyboard for interactive content. Examples for generating are (i.) microphones for audio, (ii.) cameras for video and pictures.

[0147] 16. Play, Playback, playback or played or play - is broadly defined as playing Formats on a Device and / or generating Formats on a Device

[0148] 17. Store - a service of the Cloud Backend enabling Users to search, find, select and unselect Media Apps and authenticate them with the new Cloud Ecosystem using the Content Services credentials, and vice-versa. This allows the Cloud Ecosystem to e.g., access content from Content Services.

[0149] 18. Management Services - Services in the Cloud Ecosystem available via the Cloud Backend for participants to control and manage some or all aspects for one or many Devices (e.g., managing Devices including onboarding and offboarding, play queue management, playback management, etc.).

[0150] The ecosystem is a system consisting of

[0151] An unlimited number of Devices 100

[0152] An unlimited number of Users

[0153] An unlimited number of Media Apps 300

[0154] Enabling access to each User’s Devices and Content Services over Apps and / or Management Services, with the possibility to delegate rights to other Users.

[0155] Giving Content Services full control over (i.) minimum Device Capabilities a Device must have to be allowed and able to play their content (for example codec support, secure boot, DRM, etc.) for the purpose of e.g., ensuring best possible playback quality, and / or the minimum security requirements (ii.) Devices from which Device Brand(s) will be allowed to play their content, or to not play their content, (iii.) which Devices get access to which Media App enabling a Content Service for example to exclusively one Device Brand, or any combination / permutation, (iv.) allow Content Services to extend their own CIN240908PEP-2025002701.DOCX 24 applications the capability to manage Devices with their own Media Apps (e.g., remote control).

[0156] Giving Device Makers full control over (i.) which Media Apps they want to allow on their Devices, (ii.) which Device Capabilities of the Cloud Ecosystem the Device should support and make available to the Cloud Ecosystem, (iii.) allow Device Makers to extend their own applications the capability to manage Devices via Management Services (e.g., Device onboarding to the Cloud Ecosystem).

[0157] - Giving Users full control over (i.) selecting Media Apps via the Store and authenticate the Cloud Ecosystem with them, (ii.) using Apps and the Management Services to manage and control the Devices they own, (iii.) permissions of the Media Apps which Device Capabilities they can access via the Management Services (e.g., for a Device with a speaker and a microphone, a User should explicitly allow the Media App to control playback on the speaker, and to play from the microphone, i.e. to access the microphone input and send it to the Content Service), (iv.) delegating User’s rights partially or fully to other Users using e.g., Roles.

[0158] Advantages of our ecosystem are, amongst others:

[0159] Separation of Devices and Media Runtime Device Software versus the Content

[0160] Services enabling any Content Service to run on any Device just based on the Device Capabilities, the Media App service mapping, the Device Maker and Content Services restrictions and the User choices only, fully managed in realtime in the cloud.

[0161] The ecosystem can run on any supported Device running the Media Runtime Device Software, regardless of OS and SoC.

[0162] - Any participant keeps the control it wants, at any time.

[0163] Description of Further Embodiments of the scheduled playback invention

[0164] In a further embodiment, a content provider can perform a time-controlled playback and, among this time-controlled playback, the content provider can also deactivate, for example for a certain period, a stop or pause button on the device hardware e.g., via the “HAL control” interface from the device SDK / player to the hardware device or in response to the HAL events coming from the hardware to the device SDK / player.

[0165] Specifically, the queue processor in the device SDK / player core box is operating to activate or deactivate a queue with the content to be inserted into the playback of the “main” playback queue. A specific feature of this functionality is that the content provider is not CIN240908PEP-2025002701.DOCX 25 necessarily a commercial service such as Spotify or Netflix but the content provider is, for example, the instance that controls the music in, for example, all supermarket stores in a certain region or fast food restaurants in a certain region. Then, all speakers in the supermarket stores or the fast food restaurants are provided with a device SDK / player software and a communication takes place so that a certain content e.g., with advertisements in between are provided to the players via separate streams and, importantly, several instructions to be applied onto the players are enabled or disabled. A specific application of this topic would be to insert into the supermarket store speaker music some information on special offers occurring e.g., short before closure of the store. This is done by not inserting different content into a single stream played out via a certain queue but by managing the interlaced playing out of different queues with non-modified content.

[0166] A specific feature of an embodiment is the flexibility of the user or the cloud backend controller and the independence from the local network for the user to perform some content rendering. This is use when a commercial application is considered where the (local wireless) network, by which the user device is connected to the content provider website, has low resources or is subject to interruptions. For making sure that the hardware device performs the content rendering, the device SDK / player functionality provides a queue functionality for the hardware device where this queue is explicitly managed by the cloud backend.

[0167] The queue manager preferably makes sure that a replay functionality takes place even though a temporary network breakdown between the user and the content website takes place. Hence a specific set of queues exists used by the local device and managed by the cloud backend The queue uses memory in the device SDK / player regarding the queue, and the management of the queue(s) is implemented by the device SDK / player and can be locally manifested in a random access memory of the hardware device.

[0168] Description of Further Embodiments of the Ecosystem

[0169] A preferred embodiment is illustrated in the enclosed figure illustrating several elements of the invention. The hatched line indicates what is included in the typically mobile device owned by the user. However, the Media CDN (content delivery network) does not necessarily need to be located in the device as a library or any other collection of media, but can also be located anywhere in the cloud. In this embodiment, the mobile device has an interface for interfacing with the Media CDN in the cloud. CIN240908PEP-2025002701.DOCX 26

[0170] A “final” or endpoint device is the device HW that typically consists of a replay functionality such as a loudspeaker or a video screen and some interface buttons such as stop buttons, pause buttons, etc.

[0171] This “device HW” interfaces with a “device SDK / player”. This functional unit “device SDK / player” interfaces, on the one hand, with a media provider or, generally, with a distributed content delivery network (media App CDN).

[0172] Furthermore, the “device SDK / player” interfaces with the cloud backend.

[0173] Furthermore, there exists a app on the user’s mobile (or stationary) device such as the user’s mobile phone, tablet or even a user’s stationary computer. This app on the mobile device mainly interfaces with the cloud backend and interfaces, via the onboarding functionality, with the device SDK / player unit. However, the app to be stored, for example, on the user’s mobile device or on the user’s stationary computer does not interface with the final playback device or “device HW”.

[0174] Furthermore, this app does not interface with the media app content delivery network that comprises, for example, a streaming platform such as Netflix, Spotify, Amazon Prime, etc.

[0175] Seen from the perspective of the cloud backend, the cloud backend interfaces on the one hand, with the app on the user’s mobile or stationary device, with the media app content delivery network, and with the device SDK / player. However, the cloud backend does not interface directly to the actual replay device or “device HW”.

[0176] Due to the fact that the cloud backend is distributed in a cloud, this functional unit maintains, among others, several databases that include, among others, “Live Services” functionalities that represents the actual interface between the media app content delivery network (media app CDN) and the cloud backend.

[0177] Generally, at least four instances will cooperate with each other or with only selected ones of all instances. These are the final replay hardware deceive (1), the device SDK / player (2), the app (3) on the user’s mobile / stationary device, the cloud backend (4) and the distributed content delivery network (CDN) (5).

[0178] The present invention allows a distributed Communication between the Content Data and the Control Data. CIN240908PEP-2025002701.DOCX 27

[0179] It is preferred that the cloud backend only communicates with the media app content distribution network with respect to control data that only require a low volume of data transmission. On the other hand, the actual content such as video or audio content is directly transmitted from the distributed content delivery network (CDN) to the actual hardware device via the device SDK / player functionality which is a software program stored on a storage of the actual player device. This distributed transmission of content data on the one hand and control data on the other hand has the advantage of only having low volume traffic in the cloud backend. The high volume traffic only occurs between the content delivery network such as the streaming provider and the device SDK / player unit stored in a storage of the end point device including the “device HW” instance.

[0180] Furthermore, the controlled situation in that the user’s access to the content delivery network or streaming provider is only done via the cloud backend is advantageous for the content provider, since the content provider can trust the cloud backend and can also trust the hardware end point device, since all the replay functionalities are controlled by the device SDK / player instance that tightly cooperates with the cloud backend. In case of a security breach the cloud backend can easily bar the compromised device or a class or group of compromised devices (e.g., from the same producer or distributor or ...) by building and maintaining a corresponding “no go” database.

[0181] A specific functionality provided to the user of the hardware player is to also press buttons on the loudspeaker or the video screen device giving the user the impression that the user has the full control over the replay functionalities, although these Hardware Abstraction Layer (HAL) events are not forwarded directly to the streaming website, but only via the cloud backend.

[0182] Furthermore, embodiments of the cloud backend make sure that correct reports back to the artist with respect to Copyrights, etc. are assured by the cloud backend.

[0183] Further functionalities regarding the good control for the content provider such as databases stored with user data etc. are preferred for this invention.

[0184] An inventively encoded signal can be stored on a digital storage medium or a non- transitory storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet. CIN240908PEP-2025002701.DOCX 28

[0185] Some aspects

[0186] 1 . Apparatus, method or computer program for operating a user device as described above

[0187] 2. Apparatus, method or computer program for controlling a user device, wherein the apparatus, method or computer program is separate from the user device.

[0188] 3. Apparatus, method or computer program for communicating between the user device and the controller as described above.

[0189] 4. Apparatus, method or computer program of one of the preceding aspects, wherein the user device and / or the controller is / are an element of an ecosystem as described above, the ecosystem comprising a cloud backend.

[0190] 5. System, method or computer program for implementing a guest device procedure between a user device and a controller being separate from the user device as described above.

[0191] According to an example there is provided, inter alias, a user device (e.g., 100) comprising: a device hardware (e.g., 102), and a media run time device software (e.g., 1 10) configured for controlling the device hardware (e.g., 102), wherein the media run time device software (e.g., 102) comprises: a first control interface (e.g., 112) to a cloud backend (e.g., 200); and a media interface (e.g., 114) to a content service (e.g., 400), wherein the media run time device software (e.g., 1 10) is configured to have a queue functionality to manage scheduled content to be played back, wherein the media run time device software (e.g., 110) is configured to request, receive and playback both selected content to be played by the user device (e.g., 100), and additional, unselected content to be scheduled into the first queue to derive a second, expanded queue.

[0192] The user device may send information on how to retrieve the content of the second, expanded queue.

[0193] The user device may receive content service application programming interfaces, APIs, from the user device (e.g., 100), to manage the queue functionality.

[0194] The user device may temporarily store the additional, unselected content, to thereby refrain from re-receiving the additional, unselected content after a first reception. CIN240908PEP-2025002701.DOCX 29

[0195] The user device may monitor a connection status of a connection between the user device and the content service, to thereby reproduce the stored additional, unselected content in case of breakdown of the connection between the user device and the content service.

[0196] The user device may mix at least one selected content with at least one unselected content, to thereby playback a mixed scene with both the at least one selected content and the at least one unselected content.

[0197] The media run time device software (e.g., 1 10) may be configured to receive commands for controlling the device hardware (e.g., 102) via the control interface (e.g., 1 12), and to receive or transmit content via the media interface (e.g., 114) in response to a corresponding command received via the control interface (e.g., 112),

[0198] The media run time device software (e.g., 1 10) may be configured to receive, via the control interface (e.g., 1 12), content metadata (e.g., 750) comprising information identifying the content service and an access information, and wherein the media run time device software (e.g., 110) is configured to access (e.g., 752) the content service (e.g., 400) using the received access information via the media interface (e.g., 1 14).

[0199] The content service (e.g., 400) is a content provider, wherein the media run time device software (e.g., 1 10) is configured to receive playback metadata identifying intended content data for playback via the control interface (e.g., 1 12), to send (e.g., 750) to the playback metadata to the content provider via the media interface (e.g., 1 14), and to receive (e.g., 756) the content for playback from the content provider via the media interface (e.g., 1 14).

[0200] The user device, may further comprise: a hardware abstraction layer (e.g., 1 1 ) for interfacing the device hardware (e.g., 102) and the media run time device software (e.g., 110), wherein the hardware abstraction layer (e.g., 1 1 ) is configured for detecting a playback event, wherein the media run time device software (e.g., 110) is configured to receive the playback event from the hardware abstraction layer, to forward the playback event to the cloud backend via the control interface (e.g., 1 12), to receive a confirmation on the playback CIN240908PEP-2025002701.DOCX 30 event from the cloud backend (e.g., 200) via the control interface (e.g., 112), and to only instruct the hardware abstraction layer (e.g., 11 ) to perform the playback event, when the configuration on the playback event is received from the loud backend. user device, wherein the device hardware (e.g., 102) is configured to only reply content identified via a command received via the control interface (e.g., 112) and received via the media interface (e.g., 1 14).

[0201] The cloud backend (e.g., 200) may comprise: a user device interface (e.g., 202) for interfacing with a user device (e.g., 100); and a communication interface (e.g., 204) for interfacing with a content service (e.g., 400), wherein the cloud backend is configured to provide messages to the user device interface (e.g., 202) for controlling the user device to handle media data, and to receive messages from the user device via the user device interface (e.g., 202), and the cloud backend further comprising an application interface (e.g., 206) for interfacing with a user application transmitting information on a user input, wherein the cloud backend is configured to receive, via the application interface (e.g., 206), a request for a selected content to play (e.g., 726), wherein the cloud backend is configured to

[0202] (e.g., 749) schedule the selected content according to a first content queue with the selected content to be played in sequence by the user device (e.g., 100), insert additional, unselected content to be scheduled into the first queue to derive a second, expanded queue, and

[0203] (e.g., 751 ) send queue information providing information on the second, expanded queue to the user device (e.g., 100) through the user device interface (e.g., 202).

[0204] The cloud backend may send a content metadata request (e.g., 722) to the content service (e.g., 400) via the communication interface (e.g., 204), and to receive (e.g., 724), via the communication interface (e.g., 204), the content metadata and to forward, via the application interface (e.g., 206), a list of content derived from the content metadata, the content metadata (e.g., 750) comprising information identifying the content service and an access information.

[0205] The cloud backend may send information on how to retrieve the content of the second, expanded queue. CIN240908PEP-2025002701.DOCX 31

[0206] The cloud backend may send content service application programming interfaces, APIs, through the user device interface (e.g., 202) to the user device (e.g., 100).

[0207] The cloud backend may receive the content service APIs from the content service and to forward them to the user device (e.g., 100).

[0208] The cloud backend may send to the user device instructions directed to temporarily store the additional, unselected content, to thereby refrain from re-receiving the additional, unselected content after a first reception.

[0209] The cloud backend may send to the user device instructions directed to monitor a connection between the user device and the content service, to thereby reproduce the stored additional, unselected content in case of breakdown of the connection between the user device and the content service.

[0210] The cloud backend may send to the user device instructions directed to mix at least one selected content with at least one unselected content, to thereby playback a mixed scene with both the at least one selected content and the at least one unselected content.

[0211] The cloud backend may receive (e.g., 748) a request for a playback of a selected content via the application interface (e.g., 206), and to send, via the user device interface (e.g., 202), metadata for the selected content to the user device.

[0212] The cloud backend may receive (e.g., 762) a control playback information via the application interface (e.g., 206), and to send a control playback message (e.g., 764) to the user device via the user device interface (e.g., 202).

[0213] The cloud backend, may check whether the control playback information is directed to inhibit the playback of the unselected content, and, in case of the check having positive result, to refrain from sending the control playback message (e.g., 764) to the user device.

[0214] The cloud backend, may choose at least one additional, unselected content based on a position of the user device. CIN240908PEP-2025002701.DOCX 32

[0215] The cloud system may comprise: one or more user devices as above; a cloud backend as above.

[0216] Further aspects

[0217] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0218] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed.

[0219] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0220] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0221] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier or a non-transitory storage medium.

[0222] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer. CIN240908PEP-2025002701.DOCX 33

[0223] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.

[0224] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.

[0225] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0226] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0227] In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0228] The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.

Claims

CIN240908PEP-2025002701.DOCX 34Claims1 . User device (100) comprising: a device hardware (102), and a media run time device software (1 10) configured for controlling the device hardware (102), wherein the media run time device software (102) comprises: a first control interface (112) to a cloud backend (200); and a media interface (1 14) to a content service (400), wherein the media run time device software (1 10) is configured to have a queue functionality to manage scheduled content to be played back, wherein the media run time device software (110) is configured to request, receive and playback both selected content to be played by the user device (100), and additional, unselected content to be scheduled into the first queue to derive a second, expanded queue.

2. User device of any of the preceding claims, configured to send information on how to retrieve the content of the second, expanded queue.

3. User device of any of the preceding claims, configured to receive content service application programming interfaces, APIs, from the user device (100), to manage the queue functionality.

4. User device of any of the preceding claims, configured to temporarily store the additional, unselected content, to thereby refrain from re-receiving the additional, unselected content after a first reception.5 User device of claim 4, configured to monitor a connection status of a connection between the user device and the content service, to thereby reproduce the stored additional, unselected content in case of breakdown of the connection between the user device and the content service.

6. User device of any of the preceding claims, configured to mix at least one selected content with at least one unselected content, to thereby playback a mixed scene with both the at least one selected content and the at least one unselected content.CIN240908PEP-2025002701.DOCX 357. User device (100) of any of the preceding claims, wherein the media run time device software (1 10) is configured to receive commands for controlling the device hardware (102) via the control interface (112), and to receive or transmit content via the media interface (1 14) in response to a corresponding command received via the control interface (1 12),8. User device (100) of any of the preceding claims, wherein the media run time device software (1 10) is configured to receive, via the control interface (112), content metadata (750) comprising information identifying the content service and an access information, and wherein the media run time device software (1 10) is configured to access (752) the content service (400) using the received access information via the media interface (1 14).

9. User device (100) of any of the preceding claims, wherein the content service (400) is a content provider, wherein the media run time device software (1 10) is configured to receive playback metadata identifying intended content data for playback via the control interface (112), to send (750) to the playback metadata to the content provider via the media interface (1 14), and to receive (756) the content for playback from the content provider via the media interface (114).

10. User device of one of the preceding claims, further comprising: a hardware abstraction layer (1 1 ) for interfacing the device hardware (102) and the media run time device software (110), wherein the hardware abstraction layer (1 1 ) is configured for detecting a playback event, wherein the media run time device software (1 10) is configured to receive the playback event from the hardware abstraction layer, to forward the playback event to the cloud backend via the control interface (1 12), to receive a confirmation on the playback event from the cloud backend (200) via the control interface (1 12), and to only instruct the hardware abstraction layer (11 ) to perform the playback event, when the configuration on the playback event is received from the loud backend.CIN240908PEP-2025002701.DOCX 361 1 . User device of one of the preceding claims, wherein the device hardware (102) is configured to only reply content identified via a command received via the control interface (1 12) and received via the media interface (114).

12. Cloud backend (200) comprising: a user device interface (202) for interfacing with a user device (100); and a communication interface (204) for interfacing with a content service (400), wherein the cloud backend is configured to provide messages to the user device interface (202) for controlling the user device to handle media data, and to receive messages from the user device via the user device interface (202), and the cloud backend further comprising an application interface (206) for interfacing with a user application transmitting information on a user input, wherein the cloud backend is configured to receive, via the application interface (206), a request for a selected content to play (726), wherein the cloud backend is configured to(749) schedule the selected content according to a first content queue with the selected content to be played in sequence by the user device (100), insert additional, unselected content to be scheduled into the first queue to derive a second, expanded queue, and(751 ) send queue information providing information on the second, expanded queue to the user device (100) through the user device interface (202).

13. Cloud backend of claim 12, configured to send a content metadata request (722) to the content service (400) via the communication interface (204), and to receive (724), via the communication interface (204), the content metadata and to forward, via the application interface (206), a list of content derived from the content metadata, the content metadata (750) comprising information identifying the content service and an access information.

14. Cloud backend of any of claims 12-13, configured to send information on how to retrieve the content of the second, expanded queue.

15. Cloud backend of any of claims 12-14, configured to send content service application programming interfaces, APIs, through the user device interface (202) to the user device (100).CIN240908PEP-2025002701.DOCX 3716. Cloud backend of claim 15, configured to receive the content service APIs from the content service and to forward them to the user device (100).

17. Cloud backend of any claims 12-16, configured to send to the user device instructions directed to temporarily store the additional, unselected content, to thereby refrain from re-receiving the additional, unselected content after a first reception.

18. Cloud backend of claim 17, configured to send to the user device instructions directed to monitor a connection between the user device and the content service, to thereby reproduce the stored additional, unselected content in case of breakdown of the connection between the user device and the content service.

19. Cloud backend of any of claims 12-18, configured to send to the user device instructions directed to mix at least one selected content with at least one unselected content, to thereby playback a mixed scene with both the at least one selected content and the at least one unselected content.

20. Cloud backend of any of claims 12-19, wherein the cloud backend is configured to receive (748) a request for a playback of a selected content via the application interface (206), and to send, via the user device interface (202), metadata for the selected content to the user device.

21. Cloud backend of any of claims 12-20, wherein the cloud backend is configured to receive (762) a control playback information via the application interface (206), and to send a control playback message (764) to the user device via the user device interface (202).

22. Cloud backend of claim 21 , configured to check whether the control playback information is directed to inhibit the playback of the unselected content, and, in case of the check having positive result, to refrain from sending the control playback message (764) to the user device.CIN240908PEP-2025002701.DOCX 3823. Cloud backend of any of claims 12-22, configured to choose at least one additional, unselected content based on a position of the user device.

24. Cloud system comprising: one or more user devices as defined in one of claims 1 to 11 ; a cloud backend as defined in one of the claims 12 to 23.

25. A method for a user device (100) comprising a device hardware (102), and a media run time device software (1 10) configured for controlling the device hardware (102), wherein the media run time device software (102) comprises a first control interface (112) to a cloud backend (200) and a media interface (114) to a content service (400), wherein the method includes managing a queue functionality to manage scheduled content to be played back, wherein the media run time device software (1 10) requests, receives and plays back both selected content to be played by the user device (100), and additional, unselected content to be scheduled into the first queue to derive a second, expanded queue.

26. A method for a cloud backend (200) comprising a user device interface (202) for interfacing with a user device (100) and a communication interface (204) for interfacing with a content service (400), the method including providing messages to the user device interface (202) for controlling the user device to handle media data, and receiving messages from the user device via the user device interface (202), and the method including interfacing with a user application transmitting information on a user input, wherein the cloud backend is configured to receive a request for a selected content to play (726), the method including scheduling the selected content according to a first content queue with the selected content to be played in sequence by the user device (100), inserting additional, unselected content to be scheduled into the first queue to derive a second, expanded queue, and sending queue information providing information on the second, expanded queue to the user device (100) through the user device interface (202).

27. A non-transitory storage unit storing instructions which, when executed by a processor, cause the processor to perform the method of claim 25 or 26.

Images