User device, cloud backend, apparatus for performing a user application, related methods and computer programs and a cloud ecosystem

Abstract

A user device (100) comprising a device hardware (102), and a media runtime device software (110) configured for controlling the device hardware (102), wherein the media runtime device software (102) comprises a first control interface (112) to a cloud backend (200); and a media interface (114) to a data instance (400), wherein the media runtime device software (110) is configured to receive commands for controlling the device hardware (102) via the control interface (112), and to receive or transmit content via the media interface (114) in response to a corresponding command received via the control interface (112). A user application apparatus (300), a cloud backend (200) and a remote data instance (400) together with the user device (100) form a cloud system.

Description

[0001] User Device, Cloud Backend, Apparatus for Performing a User Application, Related Methods and Computer Programs and a Cloud Ecosystem

[0002] Specification

[0003] The present invention relates to data processing and, specifically, to the efficient and secure handling of data relying on a cloud backend support.

[0004] Typically, there exist user devices such as mobile phones, tablets, laptop computers or, generally, media rendering devices such as smart speakers or smart television sets. Such devices can access content providers such as Netflix, Amazon Prime, other streaming platforms or any Internet-based service for downloading data. Additionally, there exist internet platforms where users can upload media data. Such platforms are YouTube or TikTok to only name a few.

[0005] Typically, a user has to manage, in her or his device, a plurality of application programing interfaces (APIs), which are required in order to interface with a provider platform in order to download or stream, for example, a movie. From the perspective of the user, this managing of different procedures in order to access different platforms is tedious, time consuming and increases the required hardware and software resources on the user’s device.

[0006] From the perspective of the content provider, the current situation is problematic in that there do exist many different smart devices for rendering the typically very valuable content. Some devices may be better protected against attacks in order to gain non-authorized access to a media stream than others. It is complicated for the content providers to contract with numerous hardware manufacturers in order to set minimum standards for data protection. However, this is necessary, since the most valuable good provided by the content providers is a protected content and, therefore, securing this protected content against non-authorized access is of ultimate importance for the content provider.

[0007] It is, therefore, an object of the present invention to provide an improved concept for handling digital data. This object is achieved by a user device in accordance with claim 1 , a cloud backend in accordance with claim 9, an apparatus for performing a user application in accordance with claim 22, a method of operating a user device in accordance with claim 30, a method of operating a cloud backend in accordance with claim 31 , a method of performing a user application of claim 32, a computer program of claim 33, or a cloud system of claim 34.

[0008] A cloud system in accordance with the present invention comprises several individual elements that cooperate with each other via corresponding interfaces. These interfaces can be hardware interfaces, logical interfaces, or mixtures between hardware interfaces and logical interfaces. The hardware interface of a user device can, for example, implement the control interface to a cloud backend and, at the same time, the media interface to a remote data instance. Contrary thereto, a user interface implemented by the apparatus for performing a user application will typically be a touch-sensitive display of the device implementing the user application in order to receive a user input while the cloud backend interface may be implemented by means of a hardware interface that may be a wired or wireless interface between the device, on which the user application is running, and an entry point of the cloud backend such as a hardware reachable via a cloud backend access address.

[0009] At the same time, a user device interface of the cloud backend and a communication interface for interfacing with a data instance such as a content provider or upload platform is implemented in a cloud backend together with the application interface for interfacing with a management application providing a user input. The realization of such interfaces can be done via hardware elements and / or software routines known for internet platforms or internet services and the related interfaces.

[0010] In an embodiment, a user device comprises a device hardware, a media runtime device software configured for controlling the device hardware, and typical means such as a processor and a memory for operating the user device. The media runtime device software comprises a control interface to a cloud backend device or platform and a media interface to a remote data instance. Particularly, the media runtime device software is configured for receiving commands for controlling the device hardware via the control interface and to receive or transmit content via the media interface.

[0011] Thus, the user device and, particularly, the media runtime device software which is implemented on the user device typically already from a device manufacturer makes sure that the full control of the device hardware is established via the control interface to the cloud backend and, on the other hand, the high volume data traffic to a typically remote data instance is done only via the media interface without introducing and such data into the cloud backend.

[0012] Thus, it is made sure that the high volume data traffic is not rooted within the cloud backend, since the media interface of the media runtime device software is not open to the cloud backend in the normal media streaming operation either for the purpose of downloading media or uploading media data.

[0013] Thus, a relatively smart procedure is possible in that the full control is done via the cloud backend without the high volume data traffic, while the high volume data traffic only takes place between the user device itself and the typically remote data instance selected for the purpose of uploading or downloading digital data such as media data.

[0014] On the other hand, a full control of what happens with the available media streams provided by the content provider typically against payment of a fee is made sure by means of the media runtime device software running on the user device which is configured for fully controlling the device hardware of the user device.

[0015] A further element of the cloud ecosystem is the cloud backend that comprises a user device interface for interfacing with a user device, a communication interface for interfacing with a data instance and an application interface for interfacing with a management application providing a user input. The cloud backend can be a distributed or concentrated system and, basically, comprises one or several processors and one or several memories. Thus, the cloud backend apparatus is configured to provide messages to the user device for controlling the user device to handle media data, and the cloud backend is additionally configured to receive messages from the user device via the user device interface. Particularly, the cloud backend is configured to communicate to the data instance via the communication interface only regarding an authorization of a user of the user device to receive data necessary for the authorization from the data instance or to send corresponding authorization data to the data instance. However, any media data are not exchanged between the cloud backend on the one hand and the data instances under consideration on the other hand. Only control data are exchanged so that the high volume data traffic that occurs, when a data stream from a content provider is established to a user device is kept out of the cloud backend in order to have smart and efficient cloud backend operation.

[0016] The cloud backend comprises an application interface for interfacing with a management application providing a user input. The user input is, for example, a request for a certain media stream or a request for uploading data captured by the user device, and such a request is forwarded from the user control application or user management application or user app to the cloud backend, and the backend receives this data via the application interface of the cloud backend.

[0017] A third constituent of the ecosystem is an apparatus for performing a user application or user app which once again comprises, as typically computer devices or smart devices, a memory and a processor. The apparatus for performing the user application comprises a user interface for receiving an input from a user and a cloud backend interface for communicating with a cloud backend. However, the apparatus for performing the user application does not have an interface to the individual user devices, since these individual user devices are only controlled via the cloud backend rather than by the user application directly.

[0018] The user interface is typically a human-machine interface such as a touch-sensitive display or a speech interface or any similar implementation. The cloud backend interface manages the communication to and from the cloud backend for the purpose of the selection of data instances such as data sources or content providers or data sinks, i.e. , upload platforms. A selection and control of the user devices is done via the cloud backend and a selection of media content for up or download is also done and managed via the cloud backend interface. Again, any high-volume traffic occurring when an actual stream is downloaded or uploaded is kept out of the user application apparatus, since this traffic only takes place between the user device itself and the data instance(s) via the media interface implemented by the media runtime device software running on the user device.

[0019] Hence, when the ecosystem is considered as comprising the user device, the user application apparatus, the cloud backend and the (remote) data instances, each constituent has a very dedicated and limited interface situation.

[0020] The user application interfaces with the user via the user interface and with the cloud backend via the cloud backend interface. However, the user application does not interface with the remote data instances or with the user device via the media runtime device software configured for controlling the device hardware, when the user application is implemented on a device that does not also have such a media runtime device software. However, when the user application is also implemented in the same hardware device that also involves the constituents of the user device, then the media runtime device software nevertheless makes sure that the user application does not access the media interface or does not access the control interface to the cloud backend.

[0021] When looking to the cloud backend, the cloud backend only interfaces with the user application via the application interface, with the user device via the user device interface and with the remote data instances via a control interface for managing access and authorization issues. However, the cloud backend does not interface with the media interface of the user device or with the media port of the remote data instances by means of which the remote data instances output media streams or digital download data or receive media streams or digital input data.

[0022] On the other hand, the remote data instances do not interface with the user application but only interface, for control purposes, with the cloud backend and, for data receiving or data outputting purposes with the media interface of the user device. However, there is no possibility that the remote data instance communicates with the control interface operated in the user device.

[0023] From the point of view of the user device, it is to be noted that the user device is limited to only communicate with the cloud backend via the control interface implemented by the media runtime device software and to not interface with the user application and to only interface with the remote data instances via the media interface.

[0024] By means of these technical features, it is made sure that an efficient control is obtained that provides a high comfort for the user. This is due to the fact that the user only has to manage her or his account with the ecosystem via the user application, since the individual application programming interface (API) requirements for interfacing with different remote data instances are all handled in the cloud backend without further user interaction when the user has, typically at the first time of accessing the cloud service, installed the required login data at the remote data instances. From the perspective of the data instances, the media runtime device software controlling the user device on the one hand and the cloud backend processing on the other hand make sure that the required data protection from the beginning to the very end point in the device hardware is respected and enforced. The fact that a translation of the different application programming interfaces required by the different data instances into a specific and generic protocol between the media runtime device software in the user device and the cloud backend is established, allow the operators of the remote data instances to implement changes easily. The changes only have to be adapted and respected by the cloud backend, but any changes at the data content provider do not result in any changes at the user device, and even do not result in any changes for the media runtime device software running on the user device, since the communication between the cloud backend and this service on the user device is always the same for each data instance and for each update of each data instance.

[0025] Preferred embodiments of the present invention are subsequently disclosed with respect to the accompanying drawings, in which:

[0026] Fig. 1 illustrates an implementation of a cloud ecosystem in accordance with an embodiment;

[0027] Fig. 2a illustrates an embodiment of the user device;

[0028] Fig. 2b illustrates an embodiment of the cloud backend;

[0029] Fig. 2c illustrates an implementation of a user application apparatus;

[0030] Fig. 2d illustrates an implementation of the ecosystem consisting of the user device, the user application, the cloud backend and the data instances;

[0031] Fig. 3 illustrates procedures to be done by a device maker for producing the user device;

[0032] Fig. 4 illustrates a communication sequence for the purpose of onboarding a device that acts as the apparatus for performing a user application; Fig. 5 illustrates a sequence of messages for the purpose of user information handover;

[0033] Fig. 6a illustrates a first portion of a sequence of messages between different entities for the purpose of playing content; and

[0034] Fig. 6b illustrates a second portion of a sequence of messages for the purpose of playing a content and for the purpose of performing control playback during playing.

[0035] Fig. 2a illustrates a user device 100 comprising a device hardware 102 and a media runtime device software 110. This media runtime device software is running on the user device and is configured for controlling the device hardware as illustrated by the connection between block 110 and 102. The media runtime device software comprises a control interface 112 to a cloud backend in order to exchange messages between the media runtime device software in the device and the cloud backend. The user device comprises a media interface to a typically remote data instance for connecting to a remote data instance typically via a playable URL (unique resource locator) in order to then obtain a stream of media data from the remote data instance. Alternatively, the media interface can also send a request for a URL to the remote data instance and, then, receives this data from the data instance via the media interface and, then, an upload from the user device 100 to the remote data instance can be performed when the data instance is a data sink such as the YouTube platform or the TikTok platform. The data that are uploaded via the media interface 114 are typically data acquired by the user device hardware, which is, for example, a camera or a microphone.

[0036] For replaying a data stream received via the media interface 114, the user device typically has a display or speakers or both. Particularly, the media runtime device software 110 is configured to receive data for controlling the device hardware via the control interface 112 and to receive or transmit content via the media interface 114. Thus, it is made sure that the high volume data traffic only takes place via the media interface 114, and this high- volume traffic is kept out from the cloud backend, since the media interface 114 is not interfacing with the cloud backend.

[0037] In an embodiment, the media runtime device software is configured to receive, via the control interface, metadata comprising information identifying the data instance under consideration and an access information such as a playable URL combining the metadata and the identification of the data instance to be accessed. Based on this information, the media runtime device software is configured to access the data instance using the received access information via the media interface 114.

[0038] In one embodiment, the data instance is a content provider. In such an implementation, the media runtime device software 110 is configured to receive playback metadata identifying an intended content data for playback and the access information via the control interface 112, and to send the playback metadata to the content provider via the media interface 114 and to receive the content for playback from the content provider via the media interface 114.

[0039] In the embodiment, the user device comprises a hardware abstraction (HAL) layer 11 interfacing the device hardware on the one hand and the media runtime device software 110 on the other hand. The hardware abstraction layer 11 is configured for detecting a playback event. The media runtime device software is configured to receive the playback event from the hardware abstraction layer 11 via control line 26. However, this playback event is not immediately executed, but is forwarded to the cloud backend via the control interface 112. When a confirmation on the playback event from the cloud backend is received via the control interface 112, then the hardware abstraction layer 11 is instructed by the media runtime device software 110 to perform the playback event. However, when no such confirmation for the playback event is received from the cloud backend or when an explicit “action forbidden” message is received from the cloud backend, then the playback event is not executed.

[0040] Thus, it is made sure that the content provider has full control not only on the fact that an unintended decryption is forbidden, but also how the media data is handled in the user device.

[0041] Nevertheless, in order to give the user of the user device as much control as possible, a pressing of a certain switch on the user device which is typically, a start button, a stop button, a fast forward or fast rewind or pausing button, is nevertheless detected, but not immediately executed as is done, in contrast, in prior art devices. Instead, this hardware event is forwarded to the cloud backend and only when the cloud backend confirms this event, then this event is actually executed. The user device comprises a secure storage for storing a unique device ID which is required for identifying the user device with respect to the cloud backend. The device hardware is also configured to only replay content identified via a command received via the control interface and received via the media interface. In a further embodiment, the media runtime device software or “Cinemo SDK kit” is configured to also have a web browser functionality 21 , a media player functionality 22, and a content decryption functionality 28. In a further embodiment, the media runtime device software 110 is configured to receive a playable unique resource locator (URL) from the control interface, and the user device then interfaces with the specific remote data instance, from which the playable URL is coming from via the media interface to stream media data located at the playable URL at the content provider via the media interface 114. Other ways to access a data source or data sink platform different from via a playable URL can be performed as well.

[0042] Fig. 2b illustrates an implementation of the cloud backend 200 that has a user device interface 202 for interfacing with the user device such as the user device 100 of Fig. 2a. The cloud backend comprises a communication interface for interfacing with a typically remote data instance such as any one of the data instances indicated at 400 in Fig. 2d, where individual data instances 401 , 402, 403, 404, 405 are illustrated. These remote data instances also correspond to the media CDN 5 of Fig. 1.

[0043] The cloud backend is configured to provide messages to the user device interface for controlling the user device to handle media data and to receiving messages from the user device via the user device interface 202. The cloud backend is configured to communicate to the data instance via the communication interface 204 regarding an authorization of a user of the user device, where this communication comprises receiving data from the data instance or sending data to the data instance. However, this data does not comprise media data to be downloaded or to be uploaded from the user device to the data instances. The cloud backend comprises an application interface 206 for interfacing with a management application such as the user application 300 illustrated in Fig. 2c or 2d.

[0044] The cloud backend 200 additionally comprises, for performing the corresponding tasks, libraries / data bases 208 that can be distributed in the cloud or that can also be located on a single server. The cloud backend also comprises a processing engine 210 that can also be a distributed processing engine or a concentrated processing engine. The playback / upload control of the user device is performed by the cloud backend typically without high-volume traffic in the cloud backend. Only control data is communicated within the cloud backend and control data is received via the cloud backend interfaces 202, 204, 206 and corresponding control data is output via these interfaces as well.

[0045] Fig. 2c illustrates an apparatus 300 for performing a user application which comprises a user interface 302 which is exemplarily a human machine interface such as a touch screen interface or a speech interface in order to provide messages to a user or to receive messages from a user. The apparatus comprises a cloud backend interface 304 for communicating with a cloud backend such as the cloud backend of Fig. 2b or the corresponding elements located in the cloud backend in Fig. 1 comprising a playback management element, device queues, or a device management element 35 for example.

[0046] The apparatus is configured to communicate, via the user interface 302, to receive user commands regarding a selection of one or more data instances and a selection of content to be sent to a data instance or to be retrieved from a data instance and to display messages for the user, to communicate via the cloud backend interface 304 regarding the selection of the one or more viewed data instances to be accessed by the user devices, and to communicate via the cloud backend interface regarding the selected content for sending from the user device to a corresponding data instance or for sending from the data instance to the user device. Hence, the cloud backend interface 304 cares for the selection of data instances for the selection and control of user devices, and for the selection of media content for the upload and download.

[0047] Fig. 2d illustrate an overview of the cloud ecosystem and how the individual interfaces are connected to each other in the four greater “entities”:

[0048] The user device 100 with the media runtime device software 110. The user application apparatus 300 with the user interface 302 on the one hand and the cloud backend interface on the other hand.

[0049] The remote data instances 401 , 402, 403, 404, 405, with the high volume data connection 51 between the media interface 114 of the user device 100 and the media interface of the corresponding data instances.

[0050] The cloud backend for communicating with the remote data instances, the user device 100 and the user application apparatus 300. The user application apparatus 300 for communicating with the cloud backend and the user herself of himself.

[0051] In an embodiment, the user device is responsible for playback of content from the remote data instances, and this device, in a minimum implementation, only consists of logic to ensure this capability and, additionally, has the ability to display the playback state in case of a screen connected to the user device or integrated in the user device. The user device only reacts and performs actions based on commands received from the cloud backend via the control interface 112 and there is no local ability to change what is being played. The user device only operates in conjunction with the cloud backend except for already cached or pre-fetched content which will continue to play even without the connection to the cloud backend. Any buttons included in the hardware of the user device can remain active, but not active in the sense that a command implemented by a button is immediately executed, but only active in the sense that the command is detected, forwarded to the cloud backend, confirmed by the cloud backend and only when the confirmation is there, executed in the hardware of the user device. The user device supports all playback capabilities provided by the media runtime device software including playback of DRM protected content and synchronized playback.

[0052] Devices on the same local network can be used in a group playback leveraging synchronized playback, where the synchronization and distribution of content is performed locally. This is realized by electing one device as the primary device, this primary device will receive a content stream and will distribute it over the local network to all other participating devices. This primary device is elected algorithmically among all participating devices and can change in the course of active playback, specifically in case of devices becoming online or offline in the course of playback.

[0053] The user device also provides an interface for onboarding purposes that allows an app on a mobile device to provide network information to the user device in order to connect it to a local network. The actual hardware used in the user device is, in an abstract representation, a hardware abstraction layer being provided by a device maker and used together with the media runtime device software. The HAL is used to control network interfaces, receive signal input signals and receive status information for further use from the media runtime device software 110, among other needed abstractions to interact with provided hardware such as audio or video output. The device always identifies itself towards all other participants in the ecosystem based on a unique device ID securely stored on the user device 100.

[0054] The cloud backend comprises multiple services with detailed responsibilities. The cloud backend 200 is the main control and management interface within the ecosystem, where all changes in the ecosystem are managed and controlled via the cloud backend 200. Exemplarily, the cloud backend comprises a device management element 35 illustrated in Fig. 1. In this device management, it is made sure that every user device is registered as owned by a user. Users have the ability to see their own devices, to name these devices, to tag these devices, and to deregister them. Users can create and delete device groups which allows using a device group like a single device.

[0055] The cloud backend comprises a playback management that enables the users of the ecosystem to control various playback aspects of the owned devices of the user. Exemplarily, playback of audio, images, and videos is supported as well as an upload of audio data, video data or image data or media data.

[0056] Users can control single devices, view content from content providers and control the playback thereof independently from the local network. This means that the controlling device such as a mobile phone, browser or tablet or generally the user application apparatus 300 can be connected to any network. The playback control includes start, stop, seek, skip, and other functionalities offered by the content provider the content is coming from. Restrictions set by the content provider are respected, for example, skip limits, etc. Users can create playlists that can be composed of items from any content provider of any supporting media type. The user-created playlists can be queued on a device owned by the user. The device can offer any playback control operations directly via buttons and the user can control playback directly from there. However, such control via the buttons is not immediately executed but is only executed when confirmed by a message originating from the user device interface 202 of the cloud backend 200. Users can control playback targeting a device group containing arbitrary devices the user owns. Each separate network segment of the device group will play the content synchronously and technically act as one device. Users can control playback targeting a virtual device and the content will be played according to the virtual device definition.

[0057] Additionally, the cloud backend comprises a content service management that allows the end user to connect to content services or, as named before, data instances that are integrated via the functionalities performed by the user application apparatus 300. End users need to have a subscription / account for the corresponding service at the remote data instance they want to connect and the corresponding user application must allow the user to have access to the service.

[0058] The cloud backend additionally comprises a media app management functionality that defines all meta data for content services, decryptions, translations, icons as well as a specification on how to access content via an API or an URL or URI for web applications. The media app management also allows service providers to specify the requirements their service has towards a device (codecs, screen resolution, DRM, etc.). It also allows a content service to configure restrictions for the service allowing to restrict a service availability by device meta data (manufacturer, type, model, etc.), regions or accounts. These restrictions can be formed by using deny lists and allow lists.

[0059] Additionally, an update management is provided in the cloud backend. In general, devices receive updates of their running device media player software over the air. For that, the devices check frequently if there is a new software version available for a specific device model and install it accordingly. The update will be installed when the device is not performing playback, however, devices performing continuous playback will forcefully install the updates after a defined time. Update packages are stored on the distributed CDN (Content Delivery Network) illustrated at 5 in Fig. 1 to ensure high availability and low latency. An update package contains a specific version of the media runtime device software that runs on the device and informative meta data for the update mechanism to be able to select the correct update package. Update packages can be private, so that they can be restricted for development purposes for specific manufacturers, device models or users. The updates are managed through the cloud backend 200 and the device manufacturers. Manufacturers can define if their device models receive a specific version of the update or not.

[0060] The user application apparatus 300 relies on rendering the web frontend exemplarily within a native application. On top of rendering the web frontend, the user application is also integrated in a playback device also allowing remote control via the web frontend or other applications. On top of these functionalities, a further purpose of the user application apparatus 300 is to provide support for onboarding of devices. The onboarding of devices is happening by using a camera of a mobile device to scan a QR code. The QR code contains information about the device ID (uniquely identifying the device) as well as optionally information about a local wireless access-point of the device. After scanning the QR code, the user application 300 will connect the mobile device to the wireless access-point of the device. In the next step, the device itself is providing the mobile app with a list of available wireless networks that the device can connect to. The user can select the wireless network to connect to and will be asked to provide the necessary wireless network password. This information is transferred from the user application to the device on which the user application is running and, then, a connection to the wireless network and a connection to the cloud backend 200 are established.

[0061] Subsequently, further aspects of the present invention are discussed in more detail.

[0062] A further element that is mentioned in Fig. 3 is a device descriptor. Particularly, as illustrated in Fig. 3, a device maker 500 is performing several procedures to generate a cloud serviceready user device. Particularly, the device maker produces a hardware device and, then, as illustrated in item 501 , puts the media runtime device software on the device. As illustrated in item 502 a set of unique device IDs is acquired and, then, as illustrated in step 503, the device IDs are securely put on the devices, one device ID per device. The device manufacturers create and configure a device descriptor as illustrated in item 504 and, additionally, set rules for allowing or disallowing certain media apps as illustrated in 505. In an embodiment, these rules are stored in the cloud backend and, particularly, in the certain libraries or databases of the cloud backend illustrated at 208 at Fig. 2b.

[0063] Fig. 4 illustrates an example procedure of onboarding of the user application apparatus 300 to become a user device. This is done implicitly after the user 99 downloads 601 the app from a store and logs into the app for the first time. Particularly, for this purpose, the user 99 starts the app in step 602, and the user application receives, via the user interface 302, a starting message of the app 602. Then, the application apparatus 300 displays a login screen and receives a login message 603 from the user 99.

[0064] When the user is logged in, the application 300 receives a device ID that is internally assigned to the user so that the user becomes the owner of the user device, on which the user application is running, i.e., the owner of the user application apparatus. In an embodiment illustrated in Fig. 4, the user application sends out credentials to the cloud backend 200. This occurs by sending out the message 604 via the cloud backend interface 304 of the user application to the application interface 206 of the cloud backend. Then, in reply to this message 604, the cloud backend 200 replies, via the application interface 206, with a message 605 comprising a token, which is received from the user application 300.

[0065] Then, using this token, and when the starting is performed for the first time as illustrated in item 606, the application retrieves, from the apparatus, on which the user application is running, a MAC information (media access control information), or an IMEI (International mobile equipment identity) information and sends this data via a message 607 to the cloud backend 200 via the application interface 206 of the cloud backend. In response to this data, the cloud backend creates a device ID as illustrated in block 608 and, additionally, stores device data derived using the IMEI or the MAC data and outputs the device ID using a message 610 via the application interface to the cloud backend interface 304 of the user application. The user application then stores this device ID in the apparatus, on which the user application is running. Now, the device, in which the user application 300 is running has a unique device ID known and managed by the cloud backend 200. Therefore, in order to be able to finally become a device, it is only required that, for example, using the device ID, the media runtime device software already existing on the apparatus, in which the user application is running, is activated, or this data is downloaded so that the device, in which the user application is running, is implemented to be a compliant device.

[0066] Fig. 5 illustrates a further sequence of messages between the user 99, the user application 300, the cloud backend 200 and the data instance or content service 400 for the purpose of performing a user information handover. A user has access to all content services or data instances that are allowed to be used by the user. To be able to use a specific content service within and under the control of the cloud backend, the user has to authorize the specific content service or data instance. For this purpose, the user 99 sends an authorized content service message 620 to the user application via the user interface 302 and the user application receives this message 620 via its user interface. In response to this message 620, the user application requests an authorization of the content service for this user via the cloud backend interface 304 and a message 621. In response to this message 621 received by the cloud backend 200 via the application interface 206, the cloud backend looks-up a certain authorization method which is offered by the content service under consideration using a certain database where this information is stored in the cloud backend. When this specific authorization method is found by the cloud backend, the cloud backend sends an authorization request message 622 to the content service or remote data instance via the communication interface 204 illustrated in Fig. 2 in order to access the remote data instance under consideration, but not for media download or upload but only for control purposes. In response to this request 602, the content service replies with a request 623 to the cloud backend 200 which is forwarded from the cloud backend 200 to the user application 300. In the message 223, the user application is required to provide credentials i.e., login data for the specific content service 400 and exemplarily also to provide the consent to the terms required by the specific content service. This data is input from the user 99 via the user interface (not illustrated in Fig. 5) and sent back via message 624 to the cloud backend and from there to the content service 400. In response to that, the content service sends an access token via message 625 and additionally, also a refreshment of the access token as required under the specific circumstances is performed. The cloud backend stores this token and typically also the expiration of the token, as illustrated in 626, for further use in a playing content functionality illustrated subsequently with respect to Fig. 6a or 6b.

[0067] Thus, after successful authorization, the ecosystem stores the tokens together with the user ID so that they can be fetched as soon as the content service API is accessed. The tokens are kept only in the cloud backend and do not have to be managed by the application 300 in any sense. Due to the fact that the user has authorized the content service a single time, a next access to the content service via the cloud backend by the user application will not require any further input of the credentials or any passwords by the user. When a list of authorized content services is needed, a database included in block 208 of the cloud backend 200 (Fig.2b) is queried for all content services where tokens for the user under consideration exist.

[0068] Fig. 6a and 6b illustrate a procedure showing a sequence of steps / message between the user 99, the user application 300, the cloud backend 200, the media runtime device software 110 and the content service 400 for playing content with the cloud ecosystem. The playing of content or the uploading of content within the ecosystems involves actions of the corresponding participants of the ecosystem such as device makers, content services, users. It is to be noted that the heavy-weight data or content is retrieved directly by the device media runtime software of the user device, while the cloud backend only handles the playback commands. The data of authorized devices and authorized content services for a user are stored and retrieved from databases that are part of the cloud backend. These databases contain all data that is necessary for authorized access to a content service, where tokens are needed for access. When, in the Fig. 6a, Fig. 6b embodiment such data is looked-up in the cloud backend, it is retrieved from a corresponding database 208. It is to be noted that the illustration in Fig. 6a makes clear that each message, such as a message 700 between the user application 300 and the cloud backend 200 refers to two actions of the entities involved. The first action is that the user application sends out the message 700 via the cloud backend interface 304, and the second action performed by the cloud backend 200 is that the cloud backend 200 receives the message 700 via the application interface 206. For each of the corresponding messages that are exchanged between two entities in the Fig. 6a, Fig. 6b illustration or in the Fig. 5 and Fig. 4 illustrations, there exist the two actions, i.e. , on behalf of the entity sending the message and on behalf of the entity receiving the message, where the corresponding interfaces involved are clear from the illustration. Hence, although not explicitly outlined all the time, these two actions are always performed.

[0069] Subsequently, Fig. 6a and Fig. 6b are discussed in more detail. In the action 602, the user 99 opens the user application running on the user application apparatus. In step 603, the user logs into the app and, in reply to this logging into the app, the user authenticates herself or himself via the message 700 from the user application to the cloud backend. Then, in order to further use the cloud backend, the user receives a token including the user’s permissions and the corresponding user ID in message 605. The user token is used in all calls to the cloud backend to identify the user and to check if access is allowed for the requested function for the user.

[0070] Then, in order to finally come to a playing of a media content, the user application sends a request for a list of content services illustrated at 702. All content services that are authorized in the cloud backend are returned. Each content service is either authorized or not authorized by the user. Hence, due to the fact that the cloud backend replies to this message 702, with a full selection of content services authorized in the ecosystem, the selection what is valid for the specific user is performed in step 704. Here, the cloud backend checks, which content services are actually authorized for the user.

[0071] Then, for each identified content service, it is checked, if the user is allowed to use the content service. If the user is not allowed, the content service is removed from the result. In a step 708, it is checked, whether the content service allows the user. Probably, the user is denied for some reason, for example by not having paid a bill, etc. This is done in step 708. Then, in step 710, the cloud backend returns a list of allowed content services to the user application 300. This list is presented to the user via the user interface in the user application and the user then selects a content service from the list provided 710 in the step 712. Then, based on the selected content service, the user application 300 requests a list of content in message 714. In response to that, the cloud backend looks up, whether credentials for this content services are stored in the cloud backend. This is done in step 716, and when such user credentials for the requested content service are found, an authentication message 718 is sent from the cloud backend to the content service via the communication interface 204 of the cloud backend. The result message 720 is illustrated in Fig. 6a and the result message can, for example, comprise a certain token, from the content service, or any other positive authentication result.

[0072] In response to the positive result 720, the cloud backend requests content metadata in a message 722 from the content service 400, and the content service 400 sends back the requested content metadata in message 724. This list of content is then forwarded from the cloud backend to the user application in step 725 for display to the user 99 via the user interface 302. The user 99 selects a content for replay as shown in action 726 and, then, the user application requests allowed devices for the selected content from the cloud backend. As illustrated in action 730, a list of devices that are authorized by the user is looked-up and, then, processed by means of steps 732 to 742. This procedure represents the check, if a certain device is allowed from the list of devices that have been originally authorized by the user. Any negative answer in the list consisting of item 732 to 742 will exclude a corresponding device under consideration from the result.

[0073] Step 732 starts the processing sequence. However, the sequence of steps between 732 to 742 is arbitrary and any other sequence also provides a useful result. In step 734, it is determined, whether the content service under consideration allows the device brand of the user device. The information for answering this question is located in a certain database 208 in the backend. In step 736, it is determined, whether the device maker allows the content service under consideration. This is also determined from a certain database. In step 738, it is determined whether the device maker allows the playback of the content that has been selected to play in step 726. In step 740, it is determined, whether the user actually allows to playback the content on the device. This procedure is useful, when, for example, a certain profile of allowed content has been installed in the cloud backend, for example for the purpose of children control. Now, it can be that the child acts as a certain user 99, and therefore, it can easily be that the child requests a content to play that, however, is not allowed to be played back on the device as defined earlier when establishing the certain children control profile in a library of the cloud backend. Finally, it is determined whether the actual user device under consideration is online and ready. Only when all the answers in the list are positively answered, a device survives in the list of allowed devices that is referred back from the cloud backend 200 to the user application via message 744.

[0074] Then, the user 99 replies with a message 745 for selecting a device from the list of allowed devices and, then, the user starts the playback of the selected content on the selected device via message 746. This user action results in a message from the user application to the cloud backend that request the playback for the selected content on the selected device, where this message is illustrated at 748 in Fig 6b.

[0075] Then, the content metadata are sent from the cloud backend 200 to the media runtime device software 110 on the user device. This content metadata can, for example, be a playable URL or any other data, by which the media runtime device software 110 running on the user device can request the content via a message 752 from the user device media interface 114 via the connection to the typically remote data instances illustrated at 51 in Fig. 2d. As soon as the playable URL, for example, is received by the remote data instance, the remote data instance streams back the content as shown in message 756 and this takes place via the media interface 114 of the user device 100 and the content is replayed as illustrated at 754.

[0076] Fig. 6b additionally illustrates the procedures that are done, when the user 99 wishes to control the playback via message 760. This control can, for example, be a pause action, a stop action, etc. As soon as the user application 300 has received a message from the user such as message 760, the user application replies with a request control playback message 762 to the cloud backend 200. Only when the cloud backend confirms that such a procedure is allowed, where this check is not illustrated in Fig. 6b, the cloud backend request control playback via message 764 from the media runtime device software 110 running on the user device. As soon as the control message 764 is received by the user device via the control interface 112, the user device reacts with an execution of the playback control illustrated at 766 and reports this executed control via step 768 back to the cloud backend 200 and a corresponding confirmation is sent from the cloud backend 200 back to the user application via message 770.

[0077] It is to be emphasized that the order of the selection of the content service 712, the selection of the content to play 726, and the selection of the device from the list of allowed device 745 can also be done in a different order. In case of only a single content service and only a single user device, several checks and messages can be omitted in order to nevertheless implement the full control of the cloud backend 200 over the user device, which the media runtime device software 110 is installed.

[0078] Subsequently, features of embodiments are summarized that make up a preferred cloud ecosystem. Other embodiments of the cloud ecosystem can include more or less parts or roles than described below. An embodiment consists of multiple parts and roles as described subsequently:

[0079] 1 . Cinemo - the inventor and operator of the new Cloud Ecosystem

[0080] 2. Cloud Ecosystem - a uniform cloud media platform connecting Users, Devices, and Content Services via the Cloud Backend and providing the Management Services.

[0081] 3. Device Maker - any party designing and / or manufacturing Devices. It includes also private technical savvy users acting as such Device Maker by installing the Media Runtime Device Software on a device of their own (e.g., a private camera connected to an SoC of a User).

[0082] 4. Device Brand - a commercial brand under which a Device Maker offers Devices

[0083] 5. Content Services - any party having cloud services for sending (downstream from cloud service) or receiving (upstream to cloud service) content on or from a Device (examples for sending are e.g., services like Spotify, Netflix, or advertising content services, business radio services, private streaming services, content management systems (including systems handling scheduling, delivery and curation) and for receiving can be e.g., a service to store digital camera data in the cloud).

[0084] 6. Media App - a descriptor defining and enabling the connection between the Content Services, the Cloud Backend and the Device. And enabling a Content Service additional functionality by communicating with the Device via the Media Runtime Device Software, like e.g., like saving content on a Device. A Media App is only available in the Cloud Backend, the Device and the Media Runtime Device Software do not have information about the Media App, providing additional security and separation of concerns.

[0085] 7. Device - a hardware device having at least one CPU and a connection to the internet, having Device Capabilities and having integrated the Media Runtime Device Software. A Device is uniquely identified by a Device ID and can have a User as its owner. 8. Media Runtime Device Software - embedded software running on a device and communicating both with the device hardware and with the Cloud Backend, and supporting Formats and other capabilities depending on the Device Capabilities. The Media Runtime Device Software is independent of specific Media Apps, and is compiled to many OS and SoCs systems.

[0086] 9. Cloud Backend - cloud services enabling and maintaining the connection and control of the Media Runtime Device Software of all Devices. Through this connection enabling the Cloud Ecosystem. The Cloud Backend has its own Web APIs (e.g., REST APIs). Media Apps are used in the Cloud Backend to communicate content for playback from and to Devices.

[0087] 10. Device Capabilities - the list of all hardware capabilities of a device supported by the Cloud Ecosystem. This includes all supported Formats, and including capabilities like supported audio and video codecs for encoding and for decoding, performance capabilities, security capabilities such as secure boot, DRM capabilities, payment capabilities, etc.

[0088] 11. Device ID - a unique identifier for each Device

[0089] 12. App - an application by Cinemo using the Management Services to manage the Devices and the Account. Optionally, an App can have its own Media Runtime Device Software making the App a Device itself. An App can be a mobile app running on a smart phone or tablet or can be running in a browser or an application running on a PC.

[0090] 13. User - a user with an Account. Users can own their own Devices or use Devices of any other user (e.g., colleagues, or family members).

[0091] 14. Role - each User has one or more roles to define the rights of the User

[0092] 15. Account - a unique user with a free or paid subscription to the Cloud Ecosystem

[0093] 16. Formats - at least one of audio, video, pictures, text and interactive (web) content playing / displaying, and / or generating at least one of audio, video, pictures, text and interactive (web) content. Examples for playing are (i.) speakers for audio, (ii.) screens, TVs for video and pictures, (iii.) screens, keyboard for interactive content. Examples for generating are (i.) microphones for audio, (ii.) cameras for video and pictures.

[0094] 17. Play, Playback, playback or played or play - is broadly defined as playing Formats on a Device and / or generating Formats on a Device

[0095] 18. Store - a service of the Cloud Backend enabling Users to search, find, select and unselect Media Apps and authenticate them with the new Cloud Ecosystem using the Content Services credentials, and vice-versa. This allows the Cloud Ecosystem to e.g., access content from Content Services.

[0096] 19. Management Services - Services in the Cloud Ecosystem available via the Cloud Backend for participants to control and manage some or all aspects for one or many Devices (e.g., managing Devices including onboarding and offboarding, play queue management, playback management, etc.).

[0097] 20. Device Descriptor defining and enabling the connection between the device maker, the Cloud Backend and the Device. The Device Descriptor contains at least the Device Capabilities, a device name, a device picture and the device type.

[0098] An embodiment is a system consisting comprising:

[0099] - An unlimited number of Devices

[0100] - An unlimited number of Users

[0101] - An unlimited number of Media Apps

[0102] Enabling access to each User’s Devices and Content Services over Apps and / or Management Services, with the possibility to delegate rights to other Users.

[0103] Giving Content Services full control over (i.) minimum Device Capabilities a Device must have to be allowed and able to play their content (for example codec support, secure boot, DRM, etc.) for the purpose of e.g., ensuring best possible playback quality, and / or the minimum security requirements (ii.) Devices from which Device Brand(s) will be allowed to play their content, or to not play their content, (iii.) which Devices get access to which Media App enabling a Content Service for example to exclusively one Device Brand, or any combination / permutation, (iv.) allow Content Services to extend their own applications the capability to manage Devices with their own Media Apps (e.g., remote control).

[0104] Giving Device Makers full control over (i.) which Media Apps they want to allow on their Devices, (ii.) which Device Capabilities of the Cloud Ecosystem the Device should support and make available to the Cloud Ecosystem, (iii.) allow Device Makers to extend their own applications the capability to manage Devices via Management Services (e.g., Device onboarding to the Cloud Ecosystem).

[0105] Giving Users full control over (i.) selecting Media Apps via the Store and authenticate the Cloud Ecosystem with them, (ii.) using Apps and the Management Services to manage and control the Devices they own, (iii.) permissions of the Media Apps which Device Capabilities they can access via the Management Services (e.g., for a Device with a speaker and a microphone, a User should explicitly allow the Media App to control playback on the speaker, and to play from the microphone, i.e. to access the microphone input and send it to the Content Service), (iv.) delegating User’s rights partially or fully to other Users using e.g., Roles.

[0106] Advantages of the embodiment are, amongst others: Separation of Devices and Media Runtime Device Software versus the Content Services enabling any Content Service to run on any Device just based on the Device Capabilities, the Media App service mapping, the Device Maker and Content Services restrictions and the User choices only, fully managed in real-time in the cloud. The ecosystem can run on any supported Device running the Media Runtime Device Software, regardless of OS and SoC. Any participant keeps the control it wants, at any time.

[0107] An embodiment is illustrated in Fig. 1 illustrating several elements of the invention. The hatched line indicates what is included in the typically mobile device owned by the user. The Media CDN (content delivery network) is located anywhere in the cloud, i.e., remote from the user device 10 or remote from the apparatus in which the application program is executed, by which the user can control her or his user devices 10. In this embodiment, the mobile device has an interface for interfacing with the Media CDN in the cloud.

[0108] Fig. 1 shows an example of an ecosystem in particular in the case of providing media content (the same may be obtained, however, for generic services to be provided, and not only media streams to be transmitted). Fig. 1 shows a player device 10 (which may the first processing device 10 in Fig. 2A and 2B or the second processing device in Fig. 3). Each device 10 may control a device hardware 1 (which may be a display device and / or loudspeakers and / or a hardware connection to the loudspeakers. The device 10 may receive a service e.g. media content 51 (which may be divided between 51a and 51b in some examples), which is indicated as 202 in Figs. 2A, 2B, and 3, from a media content distribution network (CDN) 5 (which may be divided between media update CDN 5a, sending 51a, and CDN 5b, sending 51b, in some examples, but this is not strictly necessary; 51a , 51 b, 51 are examples of 202). The assisting system (which may be a cloud backend system) 4 may control the player device 10 (10a, 10b, 10c, 10d), e.g., through content control 38 and / or playback control 34. The player device 10 may include a device SDK / player 1. The assisting system 4 may control the playback and the reception of the media streams through control commands 34, 48, received through the first remote connection 401. The media CDN 5 (second processing system in Figs. 2A and 2B, first processing system in Fig. 3) which provides the content is not the same as the cloud backend system (assisting system) 4. The content 51 (51a, 51 b, 202) is transported through the third remote connection 403 and not the first remote connection 401 as the control commands 34 and 38 (it may be that they both share the same physical connection, but anyway the content 51 and the control commands 34 and 38 are not in the same session).

[0109] The device SDK / player 2 may provide the media content (e.g., audio and / or video streams) to a content decryption service 28, which can therefore decrypt the media content 27 (combing from 51 , 51a, 51 b, 202). The decrypted media content 29a may therefore be provided to a hardware media Tenderer unit 1. The hardware media Tenderer unit 1 may include the hardware abstraction layer (HAL) 11. The HAL 11 may be provided by an operating system. The HAL 11 may control the rendering of the audio and / or video. The HAL 11 may be controlled by hardware control and / or hardware events 26, e.g. sent by the device SDK / player 2. The device SDK / player 2 may provide the hardware control and hardware events 26 based, in particular, on the control 34 and 38 accepted by the cloud backend system 4 (assisting system). The player device 10 may be connected to a local network (e.g., LAN) 220. The device SDK / player 2 may include a web browser 21. The device SDK / player 2 may include a media player 22. The device SDK / player may contain a cache 23 (e.g. in the media player 22). The device SDK / player 2 may contain a group playback entity 24.

[0110] The device SDK / player 2 may contain an updater 25, which may receive updated media content from the update CDN 5a (5). As can be seen from Fig. 1 , the player device 10 (first processing system) may receive content control 38 (e.g. content control commands), which may include, for example, information on queues or more in general on content 37. Fig. 1 shows many queues 37, because it may be possible that information on multiple queues 37 is sent for multiple, different devices 10 (see also below). The content control 38 may provide information on the media content to be rendered (e.g. through the information on the queues or more in general of the content 37), e.g. together with address (e.g., URL address) of where the media content 51 (51a, 51 b, 202) is to be retrieved (e.g. address of locations associated to the CDN 5. The cloud backend (assisting system) 4 may transmit playback control commands 34. The playback control 34 may include, for example, well- known commands, such as “PLAY” “PAUSE”, “STOP”, “FAST FORWARD”, “REWIND”. The cloud backend (assisting system) 4 may therefore include a playback manager 33 and a device manager 35 which send the playback control 34, and the content control commands 38 respectively. It is to be noted that the device manager 35 may substantially have knowledge on the properties of the player device 10. The device manager 35 may have a virtual replica of the player device 10 and may therefore have the substantial control of the operations of the player device 10. Both the playback manager 33 and the device manager 35 may be controlled through control commands 31 and 32 respectively. The playback control 31 may be received from a user application 3, which may be executed in a user equipment (e.g., a smart phone, mobile phone, tablet, assigned computer) of property of the user and / or may be integrated in the player device 10 (despite Fig. 1 suggesting that the user equipment and the player device 10 are not integrated in the same device, but they could be). The control device setup commands 32 may command the device management 35 on the groups to be formed between the device 10 and other devices (e.g., devices 10a, 10b, 10c, 10d) (it will be shown that the groups to be formed can be, in some examples, identified by the queues 37 to be reproduced). Therefore, the particular group may be defined by the user through the UE 3.

[0111] The UE 3 may be provided e.g. by the cloud backend system (assisting system) 4 (or by an app store, such as Google Play Store, e.g. in a certified manner) to the user equipment or other device for controlling the player device 10 (and more in general all the player devices 10a-10d). The application 3 may present a user interface for obtaining settings to render content.

[0112] Even if between the UE 3, the cloud backend system (assisting system) 4, the player device 10 (e.g. first processing device 10) and the media CDN 5 (e.g. second processing device) it is shown that the arrows are mostly directed towards one direction only, it is to be understood that these arrows mostly refer to the directions of the most relevant commands. It is notwithstanding possible that feedback information may be transmitted in any different direction. For example, the UE 3 may receive feedback from the cloud backend (assisting system) 4 (e.g. to reproduced in a user interface), which in turn may receive the feedback from the player device 10. Also, the direction of the lines 51a and 51b could be understood bidirectional, since one device (e.g. the primary device 10a, see below) may be the device which requests the transmission to the media CDN 5 and keep some contacts with the media CDN 5 (e.g. by sending requests relating to which representation of adaptation set of the media content to receive, e.g. in response to the reception of a manifest and based on the monitoring of the reception speed, and so on). Subsequently, further embodiments of the ecosystem are described. A “final” or endpoint device is the device HW that typically consists of a replay functionality such as a loudspeaker or a video screen and some interface buttons such as stop buttons, pause buttons, etc. This “device HW’ interfaces with a “device SDK / player”. This functional unit “device SDK / player” interfaces, on the one hand, with a media provider or, generally, with a distributed content delivery network (media App CDN). The “device SDK / player” interfaces with the cloud backend 4. There exists an application on the user’s mobile (or stationary) device 3 such as the user’s mobile phone, tablet or even a user’s stationary computer. This app on the mobile device 3 mainly interfaces with the cloud backend system 4 and interfaces, via the onboarding functionality, with the device SDK / player unit. However, the app to be stored, for example, on the user’s mobile device 3 or on the user’s stationary computer 3 does not interface with the final playback device 10 or “device HW” 1. This app does not interface with the media app content delivery network 5 that comprises, for example, a streaming platform such as Netflix, Spotify, Amazon Prime, etc.

[0113] Seen from the perspective of the cloud backend system 5, the cloud backend system 5 interfaces on the one hand, with the app on the user’s mobile 3 or stationary device 3, with the media app content delivery network, and with the device SDK / player. However, the cloud backend system 3 does not interface directly to the actual replay device or “device HW’ 1. In the cases in which the cloud backend system 4 is distributed in a cloud, this functional unit maintains, among others, several databases that include, among others, “Live Services” functionalities that represents the actual interface between the media app content delivery network (media app CDN) 5 and the cloud backend system 4 (assisting system). The present invention allows a distributed Communication between the Content Data and the Control Data.

[0114] It is preferred that the Cinemo cloud backend only communicates with the media app content distribution network with respect to control data that only require a low volume of data transmission. On the other hand, the actual content such as video or audio content is directly transmitted from the distributed content delivery network (CDN) to the actual hardware device via the device SDK / player functionality which is a software program stored on a storage of the actual player device. This distributed transmission of content data on the one hand and control data on the other hand has the advantage of only having low volume traffic in the Cinemo cloud backend. The high volume traffic only occurs between the content delivery network such as the streaming provider and the device SDK / player unit stored in a storage of the end point device including the “device HW” instance.

[0115] The controlled situation in that the user’s access to the content delivery network or streaming provider is only done via the Cinemo cloud backend is advantageous for the content provider, since the content provider can trust the Cinemo cloud backend and can also trust the hardware end point device, since all the replay functionalities are controlled by the device SDK / player instance that tightly cooperates with the Cinemo cloud backend. In case of a security breach the Cinemo cloud backend can easily bar the compromised device or a class or group of compromised devices (e.g., from the same producer or distributor or ...) by building and maintaining a corresponding “no go” database.

[0116] A specific functionality provided to the user of the hardware player is to also press buttons on the loudspeaker or the video screen device giving the user the impression that the user has the full control over the replay functionalities, although these Hardware Abstraction Layer (HAL) events are not forwarded directly to the streaming website, but only via the Cinemo cloud backend. Embodiments of the Cinemo cloud backend make sure that correct reports back to the artist with respect to Copyrights, etc. are assured by the Cinemo cloud backend. Further functionalities regarding the full control for the content provider such as databases stored with user data etc. are preferred for this invention.

[0117] Further embodiments relate to an apparatus, method or computer program for operating any element of the ecosystem as described above, or to an apparatus, method or computer program for communicating between any two elements of the ecosystem as described above, or to an apparatus, method or computer program for operating the ecosystem comprising at least two elements of the ecosystem as described above, or to an apparatus, method or computer program of one of the preceding embodiments, wherein an element of the ecosystem is one element of a group of elements comprising: User Devices, Content Services, Apps, Management Services, Content Decryption Service, Device Hardware, Device SDK Player, Media CDN, Mobile / Desktop Applications, Device Management, Playback Management and Device Queues as described.

[0118] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0119] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0120] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier. Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier or a non-transitory storage medium. In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer. A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet. A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein. A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein. In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are exemplarily performed by any hardware apparatus.

[0121] The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.

Claims

Claims1 . User device (100) comprising: a device hardware (102), and a media runtime device software (110) configured for controlling the device hardware (102), wherein the media runtime device software (102) comprises: a control interface (112) to a cloud backend (200); and a media interface (114) to a data instance (400), wherein the media runtime device software (110) is configured to receive commands for controlling the device hardware (102) via the control interface (112), and to receive or transmit content via the media interface (114) in response to a corresponding command received via the control interface (112).

2. User device (100) of claim 1 , wherein the media runtime device software (110) is configured to receive, via the control interface (112), content metadata (750) comprising information identifying the data instance and an access information, and wherein the media runtime device software (110) is configured to access (752) the data instance (400) using the received access information via the media interface (114).

3. User device (100) of claim 1 or 2, wherein the data instance (400) is a content provider, wherein the media runtime device software (110) is configured to receive playback metadata identifying intended content data for playback via the control interface (112), to send (750) to the playback metadata to the content provider via the media interface (114), andto receive (756) the content for playback from the content provider via the media interface (114).

4. User device of one of the preceding claims, further comprising: a hardware abstraction layer (11) for interfacing the device hardware (102) and the media runtime device software (110), wherein the hardware abstraction layer (11) is configured for detecting a playback event, wherein the media runtime device software (110) is configured to receive the playback event from the hardware abstraction layer, to forward the playback event to the cloud backend via the control interface (112), to receive a confirmation on the playback event from the cloud backend (200) via the control interface (112), and to only instruct the hardware abstraction layer (11) to perform the playback event, when the configuration on the playback event is received from the loud backend.

5. User device of one of the preceding claims, comprising a storage for storing a unique device ID.

6. User device of one of the preceding claims, wherein the device hardware (102) is configured to only reply content identified via a command received via the control interface (112) and received via the media interface (114).

7. User device of one of the preceding claims, wherein the media runtime device software (110) is configured to have a web browser functionality, a media player functionality, or a content decryption functionality, wherein the functionalities are fully controlled by control information received from the cloud backend via the control interface (112).

8. User device of one of the preceding claims, wherein the media runtime device software (110) is configured to retrieve (750) a unique resource locater for a specific remote data instance (401 to 405) via the control interface (112), to interface (752, 756) with the specific remote data instance using the playable URL via the media interface (114) and to stream the media data at the unique resource locater via the media interface (114).

9. Cloud backend (200) comprising: a user device interface (202) for interfacing with a user device (100); and a communication interface (204) for interfacing with a data instance (400), wherein the cloud backend is configured to provide messages to the user device interface (202) for controlling the user device to handle media data, and to receive messages from the user device via the user device interface (202), and wherein the cloud backend is configured to communicate to the data instance (400) via the communication interface (204) regarding an authorization of a user of the user device to receive data from the data instance or to send data to the data instance.

10. Cloud backend of claim 9, further comprising an application interface (206) for interfacing with a user application transmitting information on a user input.

11. Cloud backend of claim 10, wherein the cloud backend is configured to receive, via the application interface (206), a user authentication request (700), to check whether the user is authorized to use an authorization data base, and to send (705) a user token via the application interface (206) in case the user is authorized.

12. Cloud backend of claim 10 or 11 , wherein the cloud backend is configured for receiving a request (702) for one or more user-authenticated data instances via the application interface (206), to check (704, 706, 708) if the user is authorized for the one or more data instances using a data instance database and to send (710), via the application interface (206) an information about the one or more user authenticated data instances.

13. Cloud backend of one of claims 10 to 12, wherein the cloud backend is configured to receive, via the application interface, a request (714) for a list of content of a selected user - authenticated data instance,to send, via the communication interface (204), user-specific credentials (718) for the selected data instance which are retrieved from a credential database in the cloud backend for a user authentication at the selected data instance; and to receive (720), via the communication interface (204), an authentication result.

14. Cloud backend of one of the preceding claims, wherein the cloud backend is configured to send, in case of a positive authentication result, a content metadata request (722) to the selected data instance via the communication interface (204), and to receive (724), via the communication interface (204), the content metadata and to forward, via the application interface (206), a list of content derived from the content metadata.

15. Cloud backend of one of the claims 10 to 14, wherein the cloud backend is configured to receive, via the application interface (206), a request for a selected content to play (726), to check, via an allowed device’s database in the cloud backend, whether a user has control over a user device allowed for the selected content (732 to 743), and in case of a positive result of the check to transmit, via the application interface (206), an information on the one or more allowed devices (744).

16. Cloud backend of claim 15, wherein the cloud backend is configured to perform at least one check of the following checks using a device-related data base in the cloud backend: does the content service allow a device brand of the user device (734), does a device maker allow the content service (736), does the device maker allow to play back the selected content (730), does the user (99) allow to play back the selected content on the user device; andis the user device online and ready.

17. Cloud backend of one of the preceding claims, wherein the cloud backend is configured to receive (748) a request for a playback of a selected content on a selected device via the application interface (206), and to send, via the user device interface (202), metadata for the selected content to the selected device.

18. Cloud backend of one of the preceding claims, wherein the cloud backend is configured to receive (762) a control playback information via the application interface (206), to send a control playback message (764) to the user device via the user device interface (202); and to receive a playback response message (768) via the user device interface (202) from the user device (100).

19. Cloud backend of one of claims 9 to 18, wherein the cloud backend is configured to receive a playback event message from the user device (100) via the user device interface (202), to check whether the playback event is allowed for a selected content, and to send a playback event confirmation via the user device interface only when the playback event is allowed for the selected content and the user device.

20. Cloud backend of one of claims 10 to 19, wherein the cloud backend is configured to perform an onward procedure comprising: receiving (604) credentials via the application interface (206);storing the credentials in a credential database associated with a user identification; sending (604) a token to the user device via the user device interface (202); receiving device data (607) via the user device interface (202), the device data comprising information on a hardware of the user device and an information of a unique hardware identification; creating (608) a device identification and storing (609) the device identification and the device data in a device data base in the cloud backend; and sending the device identification (610) via the application interface (206).

21. Cloud backend of one of the claims 10 to 20, wherein the cloud backend is configured to receive a request (620) for a data instance authorization from the user via the application interface (206); look-up an authorization method required by the data instance in a data instance database in the cloud backend; send an authorization request (622) using the authorization method via the communication interface (204); receiving an invitation (623) from the data instance via the communication interface (204) asking for credentials and / or consent and forwarding the information on the credentials and / or consent via the application interface (206); receiving the credentials (604) via the application interface (206) and storing the credentials in a credential database in the cloud backend; receiving (625) an access token for the user via the communication interface (204) and storing (626) the access token in the cloud backend, wherein the cloud backend is configured to check whether a stored token is expired and to not use an expired token for further usage.

22. Apparatus (300) for performing a user application, comprising: a user interface (302) for receiving an input from a user (99); a cloud backend interface (304) for communicating with a cloud backend (200), wherein the apparatus is configured to communicate, via the user interface (300) to receive user commands regarding a selection of one or more data instances and a selection of content to be sent to a data instance or to be retrieved from a data instance and to display messages for the user, to communicate, via the cloud backend interface (304), regarding the selection of the one or more data instances to be accessed by the user devices, and to communicate, via the cloud backend interface (304), regarding the selected content for sending from the user device to the data instance or from the data instance to the user device.

23. Apparatus of claim 22, wherein the apparatus is configured to receive a data instance authorization request (620) via the user interface (302), to send a data instance authorization message (621) via the cloud backend interface (304), to receive a credentials input request (623) for the data instance from the cloud backend interface (304), to receive the credentials via the user interface (302) and to send the credentials via the cloud backend interface (304); and to receive (624) a confirmation via the cloud backend interface (304) and to output the confirmation (627) via the user interface (302).

24. Apparatus of claim 22 or 23, wherein the apparatus is configuredto receive a login request (603) from the user interface (302), to send an authorization request (604) via the cloud backend interface (304); and to receive a user token (605) including a user ID via the cloud backend interface (304).

25. Apparatus of one of claims 22 to 24, wherein the apparatus is configured to request (702) information on one or more data instances for a user identification of the user via the cloud backend interface (304); and to receive (710) a list of one or more allowed data instances for the user via the cloud backend interface (304).

26. Apparatus of one of claims 22 to 25, wherein the apparatus is configured to receive a data instance selection (712) via the user interface (302); to request a list of contents (714) for the selected data instance via the cloud backend interface; and to receive (725) the list of contents via the cloud backend interface (304).

27. Apparatus of one of claims 22 to 26, wherein the apparatus is configured to receive (726) a user selection of a content from the data instance via the user interface (302); to request one or more allowed devices (728) for the content selected by the user via the cloud backend interface (304); to receive information on one or more allowed devices (744) via the cloud backend interface (304); to present a selection of allowed devices in the user interface (302).

28. Apparatus of one of claims 22 to 27, wherein the apparatus is configured to receive a start playback message (746) via the user interface (302); and to send a request playback message (748) via the cloud backend interface for the purpose of playing back a selected content on a selected device.

29. Apparatus of one of claims 22 to 28, wherein the apparatus is configured, during playback of a content on a user device, receiving a control playback message (760) via the user interface (302), to send a control playback request (762) via the cloud backend interface (304); and to receive a control playback confirmation (770) via the cloud backend interface (304).

30. Method of operating a user device comprising: a device hardware (102), and a media runtime device software (110) configured for controlling the device hardware (102), comprising: maintaining a control interface (112) to a cloud backend (200); maintaining a media interface (114) to a data instance (400); receiving commands for controlling the device hardware (102) via the control interface (112); and receiving or transmitting content via the media interface (114) in response to a corresponding command received via the control interface (112).

31. Method of operating a cloud backend (200), comprising: interfacing with a user device (100) via user device interface (202); interfacing with a data instance (400) via a communication interface (204);providing messages to the user device interface (202) for controlling the user device to handle media data, and receiving messages from the user device via the user device interface (202), and communicating, to the data instance (400), via the communication interface (204) information regarding an authorization of a user of the user device to receive data from the data instance or to send data to the data instance.

32. Method of operating a user application (300), comprising: receiving an input from a user (99) via a user interface (302); communicating with a cloud backend (200) via a cloud backend interface (304); communicating, via the user interface (300), to receive user commands regarding a selection of one or more data instances and a selection of content to be sent to a data instance or to be retrieved from a data instance and to display messages for the user; and communicating, via the cloud backend interface (304), regarding the selection of the one or more data instances to be accessed by the user devices; and communicating, via the cloud backend interface (304), regarding the selected content for sending from the user device to the data instance or from the data instance to the user device.

33. Computer program for performing, when running on a computer or a processor, the method of one of claims 30 to 32.

34. Cloud system comprising: one or more user devices as defined in one of claims 1 to 9; a cloud backend as defined in one of the claims 10 to 21 ; and one or more apparatuses for performing a user application of one of claims 22 to 29:wherein the cloud backend interface (304) of the one or more user applications (300) is connected to the application interface (206) of the cloud backend, wherein the control interface (112) of the one or more user devices (100) is connected to the user device interface (202) of the cloud backend, wherein the communication interface (204) of the cloud backend is connected to a remote data instance for control purposes only; and wherein the media interface (114) of the one or more user devices (100) is connected to the remote data instance (400) for the purpose of downloading or uploading media data.

Images