Apparatus and method of managing a communication between a user instance and a plurality of different data instances, computer program and ecosystem

Abstract

The method of managing a communication between a user instance (90, 100, 300) and a plurality of data instances (401, 402, 403) comprising a first data instance and a second data instance being different from the first data instance, comprising: receiving (50) a user request (501) directed to the first data instance at a service engine (224), the user request (501) being in accordance with a service engine application programming interface (API) (222) dedicated for a communication between the user instance and the service engine (224); transforming (52) the user request (501) into a first data instance request (502) in accordance with a first data instance application programing interface (101a) dedicated for a communication with the first data instance; sending (54) the first data instance request (502); receiving (56) a first data instance reply (503) in accordance with the first data instance application programing interface (101a); transforming (58) the first data instance reply (503) into a user reply (504) being in accordance with the service engine application programing interface (222); and transmitting (60) the user reply (504).

Description

[0001] Apparatus and Method of managing a Communication between a User Instance and a Plurality of Different Data Instances, Computer Program and Ecosystem

[0002] Specification

[0003] The present invention relates to the communication between a user instance such a user device or a user application and a plurality of different data instances such as content providers for the purpose of downstreaming data to the user instance or content sinks for the purpose of uploading data from the user instance.

[0004] Typically, there exist user devices such as mobile phones, tablets, laptop computers or, generally, media rendering devices such as smart speakers or smart television sets. Such devices can access content providers such as Netflix, Amazon Prime, other streaming platforms or any Internet-based service for downloading data. Additionally, there exist internet platforms where users can upload media data. Such platforms are YouTube or TikTok to only name a few.

[0005] Typically, a user has to manage, in her or his device, a plurality of application programing interfaces (APIs), which are required in order to interface with a provider platform in order to download or stream, for example, a movie. From the perspective of the user, this managing of different procedures in order to access different platforms is tedious, time consuming and increases the required hardware and software resources on the user’s device.

[0006] From the perspective of the content provider, the current situation is problematic in that there do exist many different smart devices for rendering the typically very valuable content. Some devices may be better protected against attacks in order to gain non-authorized access to a media stream than others. It is complicated for the content providers to contract with numerous hardware manufacturers in order to set minimum standards for data protection. However, this is necessary, since the most valuable good provided by the content providers is a protected content and, therefore, securing this protected content against non-authorized access is of ultimate importance for the content provider.

[0007] A versatile media player can render a multitude of media streams, including various audio and video formats. In online streaming applications, licensed premium content is usually requested from commercial content providers in the cloud. Each content provider defines his own means and constraints on how content may be accessed and handled.

[0008] This situation, i.e., that each content provider defines its own means and constraints on how content may be accessed and handled is problematic for the communication between a certain data instance and a media player, since such a media player must be aware of many different protocols, prescriptions, rules and definitions etc.

[0009] There typically exists a situation in which there are many user instances such as media players or user applications which are owned by individuals and which can be reached not so easily for the purpose of updating, when a certain content provider or content sink is changed or has updated its requirements. On the other hand, when a new content provider or content sink appears anywhere in the Internet, and when this new content provider or data sink has again new rules and requirements, then it will be problematic for the content provider to enforce these rules within many available and already existing data end points or user instances such as media players or user applications, for example, or media data acquisition devices such as devices with microphones or cameras.

[0010] It is, therefore, an object of the present invention to provide an improved concept for a communication between a user instance and a plurality of different data instances.

[0011] This object is achieved by a method of managing a communication between a user instance and a plurality of different data instances of claim 1 , an apparatus for managing a communication of claim 29, a computer program of claim 30, or a cloud system of claim 31.

[0012] A method of managing a communication between a user instance and a plurality of different data instances comprises a step of receiving a user request directed to a first data instance of the plurality of different data instances from the user instance at a service engine, the user request being in accordance with a service engine application programming interface (API). This service engine API is dedicated for a communication between the user instance and the service engine. The service engine transforms the user request into a first data instance request in accordance with a first data instance application programming interface (API) dedicated for a communication with the first data instance. Additionally, this first data instance request is sent out to the first data instance. A first data instance reply is received from the first data instance, and this reply is also in accordance with the first data instance application programming interface. The service engine transforms the first data instance reply into a user reply which is in accordance with the service engine application programming interface and transmits this user reply back to the user instance.

[0013] Thus, it is made sure that the user instance can also communicate with the service engine in a generic application programming interface-directed way, and the service engine performs a translation of such a generic request into many different application programming interface requirements for many different data instances, which all have different formats, different languages, different individual requirements and different procedures on how to reply to a certain request and on which data is sent out in response to the individual request.

[0014] Typically, the service engine is connected to a data instance library, where the data instance library comprises, for each data instance, a dedicated model, where the transformation from the user request into the corresponding data instance request and the corresponding translation from the data instance reply back to the user reply takes place in accordance with the data stored within the library for the purpose of how the individual requests and replies operate in accordance with the application specific interfaces under consideration.

[0015] Thus, each data instance typically has its own model stored in the model library and this model comprises, for each API request of the corresponding data instance, an information of how this API can be mapped to the generic API used for the communication between the user instance and the service engine.

[0016] The objective of the Live Services is to enable a media player for a wide range of content providers to

[0017] • access content according to the provider defined methods

[0018] • establish compliant playback within the constraints of the provider

[0019] • enable the player to interact with the content in the provider intended manner

[0020] In accordance with embodiments, each content provider offers its own API with accompanied contracts for accessing, handling and interacting with content. Instead of interfacing with the different content provider APIs, the media player is enabled to interface with the uniform Live Services API. For each supported content provider, the Live Services Provider Library contains a model that is used by the Live Services Engine to establish a mapping to the provider specific API and contracts. Embodiments of the present invention provide several advantages. These comprise a significant ease of performing updating services, or a significant ease of provide onboarding capabilities and, additionally, update form independence.

[0021] The system design abstracts the content provider APIs and contracts using a uniform Live Services Engine API. The Live Services Engine API is consumed by (usually) players embedded on devices that exhibit limited updating capabilities. Hence, the API needs to be kept as constant as possible. The content provider APIs on the other hand exhibit update cycles according to their roadmap and product requirements. The Live Services Engine allows to decouple the update cycles of the API used by the media player and the API of the content providers. This eases the process of updating to a new version of a content provider API without breaking the media player function.

[0022] The market of content providers is vast and ever changing. The Live Services Engine enables support of emerging content providers by providing the appropriate model. There is no need to update the embedded player. The same holds for removing content providers.

[0023] Using a small set of basic concepts to model content providers, the Live Services concept is platform independent and does not stick to any third-party virtual machine or language. An instance of the Live Services Engine needs to implement a small set of basic concepts only and can be provided for any language and on any platform.

[0024] The inventive method and apparatus of managing a communication between a user instance and a plurality of different data instances is particularly suited for the embedding into a cloud ecosystem, where there exists several constituents of the cloud ecosystem. These constituents comprise a user device with a media runtime device software that has the media player or media capture capabilities. A user application is preferably provided that comprises the human machine interface, where this user application can be implemented within the user device itself or can be implemented in a separate device such as a separate mobile device, stationary device, or this top device. The cloud ecosystem comprises a cloud backend and the individual typically remote data instances that all have different API rules. In an embodiment, the service engine is located within the cloud backend and is, therefore, a single point of contact for all the remote data instances so that the remote data instances can negotiate with the cloud backend and, particularly, with the service engine any update requirements, onboarding requirements, or any other changes in the way how the remote data instances wish to have their content handled by the user device. The communication between the user device or the user application with the cloud backend, in which the service engine is located, takes place in the generic application programming interface directed way, while the communication between the cloud backend and the remote data instances for the purpose of control takes place in the individual APIs required by the individual remote data instances.

[0025] Thus, the user device can always communicate in a universal way without having to implement all the specifics required by the individual remote instances and, on the other hand, the remote data instances have a single point of contact for implementing any changes, since these changes are “automatically” absorbed by the service engine, since the service engine performs the transformation from the generic application programming interface into the data instance specific application programming interface in both directions, i.e. , in the “upload” direction and the “download” direction.

[0026] A further advantage of this implementation is that the service engine provides control data to the user device until that point, at which this control data is sufficient for uploading or downloading high volume traffic media content. However, this upload or downloading between the user device and the remote data instances is then done without interaction with the cloud backend or the service engine so that this high volume data traffic is not routed via the cloud backend or via the service engine, but is processed without “congesting” the data processing roots within the cloud backend.

[0027] Preferred embodiments of the present invention are subsequently disclosed with respect to the accompanying drawings, in which:

[0028] Fig. 1 illustrates an implementation of a cloud ecosystem in accordance with an embodiment;

[0029] Fig. 2a illustrates an embodiment of the user device;

[0030] Fig. 2b illustrates an embodiment of the cloud backend;

[0031] Fig. 2c illustrates an implementation of a user application apparatus;

[0032] Fig. 2d illustrates an implementation of the ecosystem consisting of the user device, the user application, the cloud backend and the data instances; Fig. 3 illustrates a preferred implementation of the method of managing a communication;

[0033] Fig. 4a illustrates a more detailed implementation of the apparatus for managing a communication comprising the service engine and the data instance library;

[0034] Fig. 4a illustrates an implementation of an embodiment of the inventive method;

[0035] Fig. 5 illustrates an implementation of an embodiment where the transformation is performed using a sequence of well-defined operations;

[0036] Fig. 6 illustrates a schematic representation of modelling a certain application programming interface (API) of a certain content provider;

[0037] Fig. 7 illustrates an implementation of the service engine comprising an item cache;

[0038] Fig. 8 illustrates an implementation of the service engine using interactions;

[0039] Fig. 9 illustrates a representation of a state machine implemented in the service engine;

[0040] Fig. 10 illustrates a sequence of steps between different entities for the purpose of preparing a playback and creating compliant reports in the context of playing a track;

[0041] Fig. 11 illustrates a sequence of steps between different entities in the context of playing a radio list and corresponding reporting operations;

[0042] Fig. 12a illustrates a first portion of a service model for an exemplarily service provider;

[0043] Fig. 12b illustrates a second portion of this service model of Fig. 12a;

[0044] Fig. 13a illustrates a model of the exemplary service provider for a browse request; Fig. 13b illustrates a sequence of transformation steps and messages performed by the service engine;

[0045] Fig. 13c illustrates a user reply or user response in the generic API format;

[0046] Fig. 14a illustrates a model for the track request for the exemplary service provider;

[0047] Fig. 14b illustrates a sequence of operations performed for the purpose of back-and- forth transforming between the different APIs;

[0048] Fig. 14c illustrates a user reply;

[0049] Fig. 15a illustrates a model for the exemplarily service provider for the event message;

[0050] Fig. 15b illustrates operations performed for transforming;

[0051] Fig. 16a illustrates the model for an exemplary service provider for another event message;

[0052] Fig. 16b illustrates the sequence of operations performed for generating a corresponding API reply to the service provider;

[0053] Fig. 17a illustrates the model for another event message from the media player;

[0054] Fig. 17b illustrates the sequence of operations performed in the service engine;

[0055] Fig. 18 illustrates a summary over the API requests illustrated in Figs. 13a to 17b;

[0056] Fig. 19 illustrates a schematic sequence of API requests and API responses;

[0057] Fig. 20 illustrates an exemplary request in the generic API and an exemplary response to the generic API;

[0058] Figs. 21a-21b illustrate services provider specific requests and responses; Figs. 22a-22b illustrate a sequence of requests / responses required for obtaining the necessary data for another exemplary service provider;

[0059] Figs. 23a-23c illustrate another request / response sequence necessary for obtaining the user reply illustrated in Fig. 20.

[0060] Before discussing the present invention related to the service engine in more detail, the exemplary cloud ecosystem, into which this service engine is advantageously embedded, is illustrated. However, the inventive method and apparatus of managing a communication between a user instance and a data instance of a plurality of different data instances is not limited to this cloud system. Instead, the service engine can operate on a certain processor independent on any cloud backend operations, i.e., in a standalone server in a certain network.

[0061] Fig. 2a illustrates a user device 100 comprising a device hardware 102 and a media runtime device software 110. This media runtime device software is running on the user device and is configured for controlling the device hardware as illustrated by the connection between block 110 and 102. The media runtime device software comprises a control interface 112 to a cloud backend in order to exchange messages between the media runtime device software in the device and the cloud backend. The user device comprises a media interface to a typically remote data instance for connecting to a remote data instance typically via a playable URL (unique resource locator) in order to then obtain a stream of media data from the remote data instance. Alternatively, the media interface can also send a request for a URL to the remote data instance and, then, receives this data from the data instance via the media interface and, then, an upload from the user device 100 to the remote data instance can be performed when the data instance is a data sink such as the YouTube platform or the TikTok platform. The data that are uploaded via the media interface 114 are typically data acquired by the user device hardware, which is, for example, a camera or a microphone.

[0062] For replaying a data stream received via the media interface 114, the user device typically has a display or speakers or both. Particularly, the media runtime device software 110 is configured to receive data for controlling the device hardware via the control interface 112 and to receive or transmit content via the media interface 114. Thus, it is made sure that the high volume data traffic only takes place via the media interface 114, and this high- volume traffic is kept out from the cloud backend, since the media interface 114 is not interfacing with the cloud backend.

[0063] In an embodiment, the media runtime device software is configured to receive, via the control interface, metadata comprising information identifying the data instance under consideration and an access information such as a playable URL combining the metadata and the identification of the data instance to be accessed. Based on this information, the media runtime device software is configured to access the data instance using the received access information via the media interface 114.

[0064] In one embodiment, the data instance is a content provider. In such an implementation, the media runtime device software 110 is configured to receive playback metadata identifying an intended content data for playback and the access information via the control interface 112, and to send the playback metadata to the content provider via the media interface 114 and to receive the content for playback from the content provider via the media interface 114.

[0065] In the embodiment, the user device comprises a hardware abstraction (HAL) layer 11 interfacing the device hardware on the one hand and the media runtime device software 110 on the other hand. The hardware abstraction layer 11 is configured for detecting a playback event. The media runtime device software is configured to receive the playback event from the hardware abstraction layer 11 via control line 26. However, this playback event is not immediately executed, but is forwarded to the cloud backend via the control interface 112. When a confirmation on the playback event from the cloud backend is received via the control interface 112, then the hardware abstraction layer 11 is instructed by the media runtime device software 110 to perform the playback event. However, when no such confirmation for the playback event is received from the cloud backend or when an explicit “action forbidden” message is received from the cloud backend, then the playback event is not executed.

[0066] Thus, it is made sure that the content provider has full control not only on the fact that an unintended decryption is forbidden, but also how the media data is handled in the user device.

[0067] Nevertheless, in order to give the user of the user device as much control as possible, a pressing of a certain switch on the user device which is typically, a start button, a stop button, a fast forward or fast rewind or pausing button, is nevertheless detected, but not immediately executed as is done, in contrast, in prior art devices. Instead, this hardware event is forwarded to the cloud backend and only when the cloud backend confirms this event, then this event is actually executed.

[0068] The user device comprises a secure storage for storing a unique device ID which is required for identifying the user device with respect to the cloud backend. The device hardware is also configured to only replay content identified via a command received via the control interface and received via the media interface. In a further embodiment, the media runtime device software or “Cinemo SDK kit” is configured to also have a web browser functionality 21 , a media player functionality 22, and a content decryption functionality 28. In a further embodiment, the media runtime device software 110 is configured to receive a playable unique resource locator (URL) from the control interface, and the user device then interfaces with the specific remote data instance, from which the playable URL is coming from via the media interface to stream media data located at the playable URL at the content provider via the media interface 114. Other ways to access a data source or data sink platform different from via a playable URL can be performed as well.

[0069] Fig. 2b illustrates an implementation of the cloud backend 200 that has a user device interface 202 for interfacing with the user device such as the user device 100 of Fig. 2a. The cloud backend comprises a communication interface for interfacing with a typically remote data instance such as any one of the data instances indicated at 400 in Fig. 2d, where individual data instances 401 , 402, 403, 404, 405 are illustrated. These remote data instances also correspond to the media CDN 5 of Fig. 1.

[0070] The cloud backend is configured to provide messages to the user device interface for controlling the user device to handle media data and to receiving messages from the user device via the user device interface 202. The cloud backend is configured to communicate to the data instance via the communication interface 204 regarding an authorization of a user of the user device, where this communication comprises receiving data from the data instance or sending data to the data instance. However, this data does not comprise media data to be downloaded or to be uploaded from the user device to the data instances. The cloud backend comprises an application interface 206 for interfacing with a management application such as the user application 300 illustrated in Fig. 2c or 2d. The cloud backend 200 additionally comprises, for performing the corresponding tasks, libraries / data bases 208 that can be distributed in the cloud or that can also be located on a single server. The cloud backend also comprises a processing engine 210 that can also be a distributed processing engine or a concentrated processing engine. The playback / upload control of the user device is performed by the cloud backend typically without high-volume traffic in the cloud backend. Only control data is communicated within the cloud backend and control data is received via the cloud backend interfaces 202, 204, 206 and corresponding control data is output via these interfaces as well.

[0071] Fig. 2c illustrates an apparatus 300 for performing a user application which comprises a user interface 302 which is exemplarily a human machine interface such as a touch screen interface or a speech interface in order to provide messages to a user or to receive messages from a user. The apparatus comprises a cloud backend interface 304 for communicating with a cloud backend such as the cloud backend of Fig. 2b or the corresponding elements located in the cloud backend in Fig. 1 comprising a playback management element, device queues, or a device management element 35 for example.

[0072] The apparatus is configured to communicate, via the user interface 302, to receive user commands regarding a selection of one or more data instances and a selection of content to be sent to a data instance or to be retrieved from a data instance and to display messages for the user, to communicate via the cloud backend interface 304 regarding the selection of the one or more viewed data instances to be accessed by the user devices, and to communicate via the cloud backend interface regarding the selected content for sending from the user device to a corresponding data instance or for sending from the data instance to the user device. The cloud backend interface 304 cares for the selection of data instances for the selection and control of user devices, and for the selection of media content for the upload and download.

[0073] Fig. 2d illustrate an overview of the cloud ecosystem and how the individual interfaces are connected to each other in the four greater “entities”:

[0074] The user device 100 with the media runtime device software 110. The user application apparatus 300 with the user interface 302 on the one hand and the cloud backend interface on the other hand. The remote data instances 401 , 402, 403, 404, 405, with the high volume data connection 51 between the media interface 114 of the user device 100 and the media interface of the corresponding data instances.

[0075] The cloud backend for communicating with the remote data instances, the user device 100 and the user application apparatus 300.

[0076] The user application apparatus 300 for communicating with the cloud backend and the user herself of himself.

[0077] In an embodiment, the user device is responsible for playback of content from the remote data instances, and this device, in a minimum implementation, only consists of logic to ensure this capability and, additionally, has the ability to display the playback state in case of a screen connected to the user device or integrated in the user device. The user device only reacts and performs actions based on commands received from the cloud backend via the control interface 112 and there is no local ability to change what is being played. The user device only operates in conjunction with the cloud backend except for already cached or pre-fetched content which will continue to play even without the connection to the cloud backend. Any buttons included in the hardware of the user device can remain active, but not active in the sense that a command implemented by a button is immediately executed, but only active in the sense that the command is detected, forwarded to the cloud backend, confirmed by the cloud backend and only when the confirmation is there, executed in the hardware of the user device. The user device supports all playback capabilities provided by the media runtime device software including playback of DRM protected content and synchronized playback.

[0078] Devices on the same local network can be used in a group playback leveraging synchronized playback, where the synchronization and distribution of content is performed locally. This is realized by electing one device as the primary device, this primary device will receive a content stream and will distribute it over the local network to all other participating devices. This primary device is elected algorithmically among all participating devices and can change in the course of active playback, specifically in case of devices becoming online or offline in the course of playback.

[0079] The user device also provides an interface for onboarding purposes that allows an app on a mobile device to provide network information to the user device in order to connect it to a local network. The actual hardware used in the user device is, in an abstract representation, a hardware abstraction layer being provided by a device maker and used together with the media runtime device software. The HAL is used to control network interfaces, receive signal input signals and receive status information for further use from the media runtime device software 110, among other needed abstractions to interact with provided hardware such as audio or video output. The device always identifies itself towards all other participants in the ecosystem based on a unique device ID securely stored on the user device 100.

[0080] The cloud backend comprises multiple services with detailed responsibilities. The cloud backend 200 is the main control and management interface within the ecosystem, where all changes in the ecosystem are managed and controlled via the cloud backend 200. Exemplarily, the cloud backend comprises a device management element 35 illustrated in Fig. 1. In this device management, it is made sure that every user device is registered as owned by a user. Users have the ability to see their own devices, to name these devices, to tag these devices, and to deregister them. Users can create and delete device groups which allows using a device group like a single device.

[0081] The cloud backend comprises a playback management that enables the users of the ecosystem to control various playback aspects of the owned devices of the user. Exemplarily, playback of audio, images, and videos is supported as well as an upload of audio data, video data or image data or media data.

[0082] Users can control single devices, view content from content providers and control the playback thereof independently from the local network. This means that the controlling device such as a mobile phone, browser or tablet or generally the user application apparatus 300 can be connected to any network. The playback control includes start, stop, seek, skip, and other functionalities offered by the content provider the content is coming from. Restrictions set by the content provider are respected, for example, skip limits, etc. Users can create playlists that can be composed of items from any content provider of any supporting media type. The user-created playlists can be queued on a device owned by the user. The device can offer any playback control operations directly via buttons and the user can control playback directly from there. However, such control via the buttons is not immediately executed but is only executed when confirmed by a message originating from the user device interface 202 of the cloud backend 200. Users can control playback targeting a device group containing arbitrary devices the user owns. Each separate network segment of the device group will play the content synchronously and technically act as one device. Users can control playback targeting a virtual device and the content will be played according to the virtual device definition.

[0083] Additionally, the cloud backend comprises a content service management that allows the end user to connect to content services or, as named before, data instances that are integrated via the functionalities performed by the user application apparatus 300. End users need to have a subscription / account for the corresponding service at the remote data instance they want to connect and the corresponding user application must allow the user to have access to the service.

[0084] The cloud backend additionally comprises a media app management functionality that defines all meta data for content services, decryptions, translations, icons as well as a specification on how to access content via an API or an URL or URI for web applications. The media app management also allows service providers to specify the requirements their service has towards a device (codecs, screen resolution, DRM, etc.). It also allows a content service to configure restrictions for the service allowing to restrict a service availability by device meta data (manufacturer, type, model, etc.), regions or accounts. These restrictions can be formed by using deny lists and allow lists.

[0085] Additionally, an update management is provided in the cloud backend. In general, devices receive updates of their running device media player software over the air. For that, the devices check frequently if there is a new software version available for a specific device model and install it accordingly. The update will be installed when the device is not performing playback, however, devices performing continuous playback will forcefully install the updates after a defined time. Update packages are stored on the distributed CDN (Content Delivery Network) illustrated at 5 in Fig. 1 to ensure high availability and low latency. An update package contains a specific version of the media runtime device software that runs on the device and informative meta data for the update mechanism to be able to select the correct update package. Update packages can be private, so that they can be restricted for development purposes for specific manufacturers, device models or users. The updates are managed through the cloud backend 200 and the device manufacturers. Manufacturers can define if their device models receive a specific version of the update or not. The user application apparatus 300 relies on rendering the web frontend exemplarily within a native application. On top of rendering the web frontend, the user application is also integrated in a playback device also allowing remote control via the web frontend or other applications. On top of these functionalities, a further purpose of the user application apparatus 300 is to provide support for onboarding of devices.

[0086] The onboarding of devices is happening by using a camera of a mobile device to scan a QR code. The QR code contains information about the device ID (uniquely identifying the device) as well as optionally information about a local wireless access-point of the device. After scanning the QR code, the user application 300 will connect the mobile device to the wireless access-point of the device. In the next step, the device itself is providing the mobile app with a list of available wireless networks that the device can connect to. The user can select the wireless network to connect to and will be asked to provide the necessary wireless network password. This information is transferred from the user application to the device on which the user application is running and, then, a connection to the wireless network and a connection to the cloud backend 200 are established.

[0087] An embodiment is illustrated in Fig. 1 illustrating several elements of the invention. The hatched line indicates what is included in the typically mobile device owned by the user. The Media CDN (content delivery network) is located anywhere in the cloud, i.e., remote from the user device 10 or remote from the apparatus in which the application program is executed, by which the user can control her or his user devices 10. In this embodiment, the mobile device has an interface for interfacing with the Media CDN in the cloud.

[0088] Fig. 1 shows an example of an ecosystem in particular in the case of providing media content (the same may be obtained, however, for generic services to be provided, and not only media streams to be transmitted). Fig. 1 shows a player device 10 (which may the first processing device 10 in Fig. 2A and 2B or the second processing device in Fig. 3). Each device 10 may control a device hardware 1 (which may be a display device and / or loudspeakers and / or a hardware connection to the loudspeakers. The device 10 may receive a service e.g. media content 51 (which may be divided between 51a and 51b in some examples), which is indicated as 202 in Figs. 2A, 2B, and 3, from a media content distribution network (CDN) 5 (which may be divided between media update CDN 5a, sending 51a, and CDN 5b, sending 51b, in some examples, but this is not strictly necessary; 51a , 51 b, 51 are examples of 202). The assisting system (which may be a cloud backend system) 4 may control the player device 10 (10a, 10b, 10c, 10d), e.g., through content control 38 and / or playback control 34. The player device 10 may include a device SDK / player 1.

[0089] The assisting system 4 may control the playback and the reception of the media streams through control commands 34, 48, received through the first remote connection 401. The media CDN 5 (second processing system in Figs. 2A and 2B, first processing system in Fig. 3) which provides the content is not the same as the cloud backend system (assisting system) 4. The content 51 (51a, 51 b, 202) is transported through the third remote connection 403 and not the first remote connection 401 as the control commands 34 and 38 (it may be that they both share the same physical connection, but anyway the content 51 and the control commands 34 and 38 are not in the same session).

[0090] The device SDK / player 2 may provide the media content (e.g., audio and / or video streams) to a content decryption service 28, which can therefore decrypt the media content 27 (combing from 51 , 51a, 51 b, 202). The decrypted media content 29a may therefore be provided to a hardware media Tenderer unit 1. The hardware media Tenderer unit 1 may include the hardware abstraction layer (HAL) 11. The HAL 11 may be provided by an operating system. The HAL 11 may control the rendering of the audio and / or video. The HAL 11 may be controlled by hardware control and / or hardware events 26, e.g. sent by the device SDK / player 2. The device SDK / player 2 may provide the hardware control and hardware events 26 based, in particular, on the control 34 and 38 accepted by the cloud backend system 4 (assisting system). The player device 10 may be connected to a local network (e.g., LAN) 220. The device SDK / player 2 may include a web browser 21. The device SDK / player 2 may include a media player 22. The device SDK / player may contain a cache 23 (e.g. in the media player 22). The device SDK / player 2 may contain a group playback entity 24.

[0091] The device SDK / player 2 may contain an updater 25, which may receive updated media content from the update CDN 5a (5). As can be seen from Fig. 1 , the player device 10 (first processing system) may receive content control 38 (e.g. content control commands), which may include, for example, information on queues or more in general on content 37. Fig. 1 shows many queues 37, because it may be possible that information on multiple queues 37 is sent for multiple, different devices 10 (see also below). The content control 38 may provide information on the media content to be rendered (e.g. through the information on the queues or more in general of the content 37), e.g. together with address (e.g., URL address) of where the media content 51 (51a, 51 b, 202) is to be retrieved (e.g. address of locations associated to the CDN 5. The cloud backend (assisting system) 4 may transmit playback control commands 34. The playback control 34 may include, for example, well- known commands, such as “PLAY” “PAUSE”, “STOP”, “FAST FORWARD”, “REWIND”. The cloud backend (assisting system) 4 may therefore include a playback manager 33 and a device manager 35 which send the playback control 34, and the content control commands 38 respectively. The device manager 35 may substantially have knowledge on the properties of the player device 10. The device manager 35 may have a virtual replica of the player device 10 and may therefore have the substantial control of the operations of the player device 10. Both the playback manager 33 and the device manager 35 may be controlled through control commands 31 and 32 respectively. The playback control 31 may be received from a user application 3, which may be executed in a user equipment (e.g., a smart phone, mobile phone, tablet, assigned computer) of property of the user and / or may be integrated in the player device 10 (despite Fig. 1 suggesting that the user equipment and the player device 10 are not integrated in the same device, but they could be). The control device setup commands 32 may command the device management 35 on the groups to be formed between the device 10 and other devices (e.g., devices 10a, 10b, 10c, 10d) (it will be shown that the groups to be formed can be, in some examples, identified by the queues 37 to be reproduced). The particular group may be defined by the user through the UE 3.

[0092] The UE 3 may be provided e.g. by the cloud backend system (assisting system) 4 (or by an app store, such as Google Play Store, e.g. in a certified manner) to the user equipment or other device for controlling the player device 10 (and more in general all the player devices 10a-10d). The application 3 may present a user interface for obtaining settings to render content.

[0093] Even if between the UE 3, the cloud backend system (assisting system) 4, the player device 10 (e.g. first processing device 10) and the media CDN 5 (e.g. second processing device) it is shown that the arrows are mostly directed towards one direction only, it is to be understood that these arrows mostly refer to the directions of the most relevant commands. It is notwithstanding possible that feedback information may be transmitted in any different direction. For example, the UE 3 may receive feedback from the cloud backend (assisting system) 4 (e.g. to reproduced in a user interface), which in turn may receive the feedback from the player device 10. Also, the direction of the lines 51a and 51b could be understood bidirectional, since one device (e.g. the primary device 10a, see below) may be the device which requests the transmission to the media CDN 5 and keep some contacts with the media CDN 5 (e.g. by sending requests relating to which representation of adaptation set of the media content to receive, e.g. in response to the reception of a manifest and based on the monitoring of the reception speed, and so on).

[0094] Fig. 3 illustrates an implementation for illustrating a method of managing a communication between a user instance such as the media player 90 and the plurality of different data instances collectively illustrated at 400 and individually illustrated as content providers or content sinks, 401 , 402, 403, 404. Fig. 4b illustrates an embodiment of the inventive method.

[0095] Particularly, the communication between the media player 90 and the content providers 400 regarding control information takes place via a live services apparatus 220 that comprises the service engine illustrated in Fig. 4a. The media player 90 corresponds with the live services engine 220 via a user request from the media player 90 to the live services and a user reply from the live services to the media player. On the other hand, the communication between the live services 220 and a content provider takes place using a first data instance request or a second data instance request or any other data instance request and a corresponding first data instance reply, second data instance reply, and so on.

[0096] The media stream 51 is directly routed between the content provider or content sink and the media player or media capture device 90 as illustrated at 51. Although the stream is only directed from the content provider to the media player in Fig. 3, it is to be noted that this illustrates the streaming of download embodiment, although the uploading embodiment can also be performed in the same way using the live services apparatus 220. In this embodiment, the media stream goes from the media player 90 to the content sink 400, but the communication for the purpose of how to upload this data from the data instance 90 to the content provider 400 takes place via the live services transforming capabilities performed by the service engine.

[0097] Fig. 4a illustrates a preferred implementation of the live services apparatus 220. The live services apparatus 220 comprises a live services engine 224 that is connected, at the side directed to the media player, to an API contract device 222. For each content provider of the content providers 401 , 402, 403, an individual model 231 , 232, 233 is available, where these models are stored in a live services provider library 230.

[0098] The media player 90 sends a certain user request to the service engine and, particularly, a user request that is in accordance with the service engine application programming interface 222. The service engine 224 transforms this request into a first data instance request which is in accordance with a first data instance application programming interface illustrated at 401a for the first data instance 401. When the user intends to get media data from the content provider 2, then the transformation target is a second data instance in request with the second data instance application programming interface 402a dedicated for a communication with the second data instance.

[0099] Fig. 5 illustrates a preferred implementation of the method for managing a communication between the user instance and the plurality of different data instances comprising a first data instance and a second data instance being different from the first data instance. Generally, one of the different data instances is indicated as the Nthdata instance. In a step 50, the method starts with receiving a user request directed to the Nthdata instance from the user at a service engine, where the user request is in accordance with a service engine application programming interface illustrated at 222 in Fig. 4a.

[0100] In step 52, the transformation, by the service engine 224 of the user request 501 into a Nthdata instance request is performed. This Nthdata instance request is in accordance with a Nthdata instance API such as API 403a of Fig. 4a which is dedicated for a communication with the Nthdata instance. In a step 54, the Nthdata instance request is sent out.

[0101] In step 56, the service engine 224 received a Nthinstance reply which is in accordance with the Nthdata instance application programming interface 403a. In step 58, this Nthdata instance reply is transformed into a user reply. The user reply is again in accordance with service engine application programming interface 22. Finally, the user reply is transmitted as illustrated in step 60.

[0102] Fig. 5 illustrates a preferred implementation of the inventive procedure, where individual messages 501-504 are illustrated, and where the transformation of step 52 Fig. 4b in the direction from left to right in Fig. 5 and the transformation in the opposite direction illustrated at step 58 in Fig. 4b are both performed using several individual operations 61-64. These operations can take place in a sequence of operations or only as individual operations, and which operations are performed for which transformation is preferably defined in the model for the data instance 401-403 under consideration, wherein the individual models are illustrated at 231-233 in Fig. 4a. The mapping between the different APIs is founded on a set of basic concepts that are repeatedly applied to establish the Live Services API functionality. In essence, the following concepts are the main building blocks of the Live Services Engine.

[0103] Special operations are used by the playback to emit events to manipulate the playback state. The operations 61-64 operated by the service engine are configured to generate a context 520 at the beginning of a transformation operation, to operate and communication with this context 520 during a transformation and to typically clear the context at the end of a transformation or at the end of a complete forward / backward transformation cycle performed by the service engine.

[0104] An expression system is implemented that allows performing simple calculations on the context values as well as the definition of Boolean expressions required for control flow operations. Operations may use expressions anywhere instead of fixed values or vice versa.

[0105] The context for the transformation is held in a value store. The context is hierarchical and partitioned in subtrees with different access types and scope. Context is the only means of passing data inside the transformation.

[0106] The config subtree makes available configuration options of the engine such as language settings or resolutions. The service subtree contains static information on the content provider that is active during the execution. The auth subtree carries all information for provider request authorization. The input to the transformation of made available in the params subtree. Finally, the var subtree allows read and write access and acts as scoped store for the transformation. The player subtree is only available during playback and forms the playback context. The “auth” parameters are also named “session” parameters in the examples from page 12a to 24b. The “access” column in the above table indicates where only a read (R) or where a read access and a write (RW) access is possible. The “live time” illustrates the live time of a certain data where, particularly, the data in the “params” and “var” fields are valid over a certain transformation. The player data, for example, can be used for performing the corresponding state machine data illustrated data in Fig. 9.

[0107] Fig. 6 illustrates a certain content provider model which has specific layers or constituents which are a manifest layer 240a, an authorization layer 240b, a content access layer 240c and a playback layer 240d. The provider model configures the Live Services Engine for a specific content provider and contains all information to access, playback and interact with the content. The model contains multiple semantically different sections. Each section configures a different function of the engine.

[0108] The manifest section contains mandatory properties of the content provider such as unique ID, display name and references to icons and other resources. The manifest contains information on available capabilities of the provider such as supported search types. Optional properties may be defined in the manifest. All properties from the manifest - mandatory as well as optional - are made available in the service subtree of the context. Thus, they are available for transformations and expressions used to model the API mappings between engine and provider. The properties are also available to the player via the browse API as content (see below).

[0109] The authorization section allows to configure the engine to retrieve the necessary information for accessing the content provider API on behalf of the user. A provider model can contain multiple authorization methods identified by their ID. Each authorization method can define two phases: the login phase and the refresh phase.

[0110] The login phase is modeled using one transformation for each login type supported by the content provider. Typical login types are OAuth2 flows like the authorization code flow or the device flow. The result of the login is user credentials that can be stored.

[0111] The refresh phase is also modeled as a transformation that uses the stored credentials to retrieve valid access tokens or similar for the content provider API as well as up-to-date user information, such as language settings and username. Each time an access towards the content provider API is initiated using a request executor operation, it may reference an authorization method by ID and the refresh phase is executed if no access token is available. If the request to the content provider API fails with an authorization error, the refresh phase is executed again to establish a new access token and the request is retried.

[0112] The content access section makes use of transformations to retrieve content and to interact with content. Content access transformations have access to the context information with “global” and “transformation” lifetime and can use expressions that are evaluated by the engine for calculations and conditional operations. The section comprises four different ways of accessing content: browse, search, lookup, and interaction. The details on how the engine handles these transformations are provided in subsequent sections.

[0113] The playback section requires the definition of the playback context. Based on the playback context, a state machine based model makes use of transformations and expression to achieve a dynamic state of playback that is used to generate reports to the provider that are required for compliant playback. The model and the processing of the playback transformations are detailed in subsequent sections.

[0114] The content of providers is made available with the content retrieval functionality of the engine. All content retrieval functions return the same data type Item.

[0115] An Item holds the following mandatory information: Unique ID, Display name, Item Type

[0116] An item may hold arbitrary metadata as key-value pairs. Common metadata keys such as “artist” or “duration” are defined in the Live Services API to enable the media player to interpret and handle the metadata.

[0117] Fig. 7 illustrates the functionality of providing an item cache 235 within the life services apparatus 220, so that data can be noted into the item cache 235 or can be retrieved from the item cached 235 as illustrated at item 236 illustrating a transformation T with items fetched from the item cache 235.

[0118] The Live Services API allows fetching pages of items by index and page size, thus enabling the player to perform random access on the returned items. To map this type of request to the provider supported pagination, the content retrieval makes use of an item cache. Each request first arrives at the cache. The cache determines whether items are missing and provides the appropriate information for the transformation to execute and fetch the missing items. The cache features the following mechanism:

[0119] • Supports provider pagination via index, next page token as well as previous page

[0120] • Adheres to HTTP caching directives including ETags

[0121] • Configurable memory limit

[0122] The service engine API provides additional API cause for browsing, searching, content lookup.

[0123] The browse API allows to list the children of a container type. As input, the browse request requires the ID of the item that shall be browsed. On empty ID the root item is returned.

[0124] The search API allows querying for items using text matching. Depending on the capabilities of the content provider, the search can be performed in different categories (such as e.g. “artist”) and result types (such as e.g. “albums”). The supported category and result type combinations are announced in the provider description and available to the media player.

[0125] The lookup API allows querying items based on external information. It can be used to turn a content provider specific id that has been retrieved from a 3rd party source to a Live Services item. Lookup does not offer pagination nor caching.

[0126] For the purpose of content interaction, Fig. 8 illustrates a functionality, where the content interaction with items is illustrated at 237. Content interactions are executed on items and allow manipulation of the content of the provider. Typical content interactions are “like” and “dislike” operations or adding and removing items from favorites.

[0127] The possible interactions vary from provider to provider and the media player may choose to only implement a subset of content interactions. Consequently, the set of possible interactions can be discovered as part of the metadata of each item.

[0128] For each interaction supported by the engine, one of the following states is available in the item: unsupported: the interaction is unsupported for the item enabled: the interaction is supported and can be triggered disabled: the interaction is supported and cannot be triggered The interaction API takes a single optional parameter and may return items, depending on the interaction at hand.

[0129] Fig. 9 illustrates a state machine 538 implemented for correctly interpreting certain playback events depending on the preceding states NTC.

[0130] The Live Services engine supports compliant playback of content. The engine supports compliance with providing the following mechanisms in order of significance:

[0131] 1 . Playback reporting

[0132] The engine allows to report all playback events to the content provider to enable proper billing

[0133] 2. Content protection

[0134] The engine supports the setup of a DRM system such as Widevine [widevine] to enable the playback of protected content

[0135] 3. Playback interaction

[0136] The engine allows to interact with the tracks in the provider intended way by e.g. enabling like and dislike functions

[0137] To accomplish this, the engine needs to be playback aware. This is achieved by defining player events, that need to be sent to the engine by the player. Player events include typical user interactions such as start, seek, and stop as well as events on timers that are bound to the playback time.

[0138] For all player events, the engine provides mechanisms that enable robust transmission of the events. The player needs to persist unsent events in case of connection loss and needs to retry sending these events once connection is back. If too many events have been persisted, the player needs to cease playback to prevent non-compliant playback.

[0139] The playback events can manipulate the playback state that is held in the engine for each playback. The playback state realized using four components:

[0140] 1 . Statechart processor

[0141] Events are processed using hierarchical state machines (statecharts). The processor implements the scxml recommendation [scxml2015] with the extensions mentioned outlined in 2 to 4. 2. Event handler

[0142] The statechart on-enter and on-exiting handlers allow to use transformation as scripts.

[0143] 3. Event filtering expression

[0144] Event transitions may use the engine expressions to define boolean conditions as filters.

[0145] 4. Player context

[0146] The player state is held in the context as subtree with the identifier player. The statechart event handlers have read / write access to the player context.

[0147] Figs. 10 and 11 illustrate two different playback types of content. Fig. 10 illustrates the playback type “tracks” and Fig. 11 illustrates the playback type “radio”. The live services engine 224 supports playback of two different types of content: tracks and radios. Tracks are defined by a single binary resource that can be played from beginning to end. In contrast, radios are composed of multiple tracks, which are played on after another.

[0148] Fig. 10 illustrates the cooperation between the media player 19, 100, 300, the service engine 224 and a data instance such as a data provider 401 , 402, 403. Subsequent to a triggering event 601 instructing the media player 9 to play a track and init_playback message 602 is sent to the service engine 224. In response to this message, a player context 520 is created 602a in the live services apparatus 220. In step 604, a resolve message is sent from the media player 92 the service engine 224 and, particularly, using the player context 520. The live service engine generates a request message 605 to the provider and in response to this request message, a response message 606 is obtained by the provider that preferably comprises a playable URL i.e. , the necessary data for the media player to access the corresponding provider in order to immediately start the download of the content located at this URL. The service engine 224 transforms this message 606 into a message 607 to the media player that has the playable URL information.

[0149] The actual playback is not illustrated in Fig. 10, since this takes place directly between the media player 90 and the provider 401 separate from the service engine 224 as illustrated in Fig. 3 at item 51. Additionally, Fig. 10 illustrates an event report 608 generated from the media player and a corresponding translated report message from the engine to the provider illustrated at 609. In other words, the media player sends an init_playback API call to the engine once playback start is triggered. The engine creates the player context. The statechart is initialized and the respective statechart event handler transformations are processed. As response, the player receives the ID of the created player context which needs to be used for all successive calls related to this playback. Before the player can start streaming, it needs to retrieve a playable URL for the stream from the content provider. Playable URLs are fetched on demand using the engine resolve API since they are usually short-lived. The resolve API uses the transformation defined in the provider model to access the content provider API and map the request and responses accordingly. Finally, the resolve API returns the playable URL and the player can start streaming. If the URL expires during playback or if connection losses occur, the player can always make a new call to the resolve API and get a new playable streaming URL. The engine guarantees that the result is deterministic, and the streams are always byte compatible.

[0150] The message 604 represents a user request in accordance with the service engine API. The message 605 represents the first data instance request in accordance with the first data instance API. The first data instance reply is represented by message 606 in Fig. 10, and the message 607 corresponds to the user reply which is again in accordance with the service engine API. The event message 608 from the media player to the engine 224 represents the user request in accordance with the service engine API and the report 609 sent out to the provider for fulfilling the requirements for compliant playback is in accordance with a first data instance API. However, in the situation of a report message, a “reply path” from the provider back to the media player is not ultimately necessary, as is the case in Fig. 10, since there does not exist any reaction from the provider back to the media player in reply to the report message 609. However, in other embodiments, an acknowledgement operation can be done which is sent out from the provider in accordance with the provider API and which is then transformed, from the engine 224, into the engine API and is then sent back to the media player.

[0151] Additional transformations performed in response to the resolve message 604 or in advance of the resolve message 604 are the operations of sending the initialized playback message

[0152] 602 from the media player to the engine, the subsequent creation 602a of the player context 520 and the subsequent sending back of the player context ID 603. Both messages 602,

[0153] 603 are in accordance with the service engine API and represent operations that have to be performed one after the other in response to the resolve message or in advance of the resolve message. Subsequently, Fig. 11 is discussed which illustrates the situation when the radio playback is required. After playback has started, all player events may be processed as defined in the playback statechart in the content provider model. Using the transformations in the on- enter and on-exit handlers, the engine creates the reports required for compliant playback of the content. Radios play one track after another in a defined sequence. The engine realizes this by using two playback contexts: the first playback context represents the radio and holds all information concerning the sequence of tracks. The second playback context represents the currently playing track. Once radio playback is started, the engine creates the radio context and initializes the state machine similar to the track playback mechanism. The player then uses the radio_tracks API of the engine to fetch the sequence of tracks for this radio. For the first track in the list, the player creates a track playback as described above and starts rendering the stream. If all tracks from the radio_track API have been played, the player should make a new call to the API to retrieve a new set of tracks.

[0154] Particularly, a trigger 650 is received by the media player either from the user application 300 or the cloud backend 200 or from the media player 10 itself, for example, when there was an inactivation on the device by a user or by any other external event.

[0155] An initialization playback message 651 is sent to the service engine, and the service engine 224 creates 651a a player context radio 521 and a corresponding player context ID is returned back via message 652. The player again replies via a request for the radio tracks with message 653 which is transformed into a get radio tracks message 654, and this transformation is done from the service engine API into the radio tracks provider API and the provider replies via a “first data instance reply 655” comprising the track IDs in accordance with the providers API, and the engine transforms this message, with the help of the player context radio 521 into a service-engine API conformant message 656 so that the media player has the sequence of track IDs to be replayed one after the other.

[0156] A certain sequence of steps 602 to 607 as has been discussed before with respect to Fig. 10 is performed, and particularly, for each individual track of the track IDs received via the message 656.

[0157] In addition to the back-forth transformation of messages 604, 605, 606, 607 illustrated with respect to Fig. 10 and discussed in Fig. 11 , another back-forth transformation is performed. Particularly, the message 653 where the browsing for radio tracks is requested corresponds to a user request in accordance with the service engine API. The service engine transforms this message into the get radio tracks message 654 in accordance with the radio provider API. The provider replies with the first data instance reply which is in accordance with the providers API and the service engine transforms this data, with the help of the context 521 , into the user reply which is then sent back to the media player so that the media player has all the individual track IDs that the media player requested with the initial message 653.

[0158] Similar to the earlier discussed event message 608 and the corresponding event report 609, an event can also be signaled with respect to the radio procedure using the player context radio 521 illustrated at message 657 in the service engine API rules and with the report message generated by the transformation in the service engine 224 in the radio provider instance API.

[0159] Typically, content providers enable means of interacting with the currently playing track. Playback interactions such as liking a track or adding a track to the favorites are available for many providers and need to be supported to provide seamless user experience among platforms.

[0160] The Live Service supports configurable playback interactions. For each interaction type (such as like, add to favorites, ...) a transformation can be used to assemble and execute the respective call to the provider API. The transformation has access to the playback context and thus can make use of the current state to assemble the request.

[0161] Available interactions are responded to the player during the init_playback API. This allows the media player to show the respective controls. The state of whether interactions are disabled or enabled is initialized based on the model using expressions on the context. To allow dynamic updates of the enabled interaction, this state is reevaluated with every event sent to the context and changes are responded to the player.

[0162] For protected content, the engine provides the API to download the Digital Rights Management certificates and licenses for the widespread Widevine [widevine] system.

[0163] Subsequently, a full example of the invention with respect to several different user requests in accordance with the service engine API are illustrated with respect to Figs. 12a to 18. The following live services engine xml models a fictional example service provider at a full office playback of the user’s favorite audio tracks for the purpose of illustrating a full example of the present invention. This service model is illustrated in Figs. 12a and 12b and is typically stored for this specific example service provider in a model for the content provider such as model 231 , 232, or 233 of Fig. 4a.

[0164] For ease of explanation, the example service provider is simplified in that the browser key only has one layer, a search is omitted, a lookup is omitted, a content interaction is omitted, an authentication is omitted, the metadata is reduced to only name, artist, and album, a playback reporting only offers a reporting on a reporting event and a playback interaction only offers a “promote” action. The model comprises a variable section 1201 , sessions section which comprises a refresh tone section 1202a, a fetch user ID section 1202b.

[0165] Then follows a browser key section 1203, a resolve track section 1204, a playback reporting end points section 1205, state transitions section 1206 that starts with an authorization portion and an actual state transition portion. A user request 501 that results in a browsing of the content is illustrated at 501 in accordance with the service engine API. A corresponding model is illustrated in Fig. 13a that comprises a refresh tone stage 1 , a fetch user ID stage 2 and a list tracks stage.

[0166] Fig. 13b illustrates a sequence of operations such as operations 61 to 64 of Fig. 5 that are performed in response to a user request 501 , i.e., the browsing to request the first ten favorite tracks.

[0167] In step 1 , it is checked whether the result is already in cache and valid. In step 2, a session is established by the service engine. In step 3, it is determined that the session is not yet cached. In step 4, a fetch refresh token from the token store that is also part of the live services apparatus is performed.

[0168] A first message (HTTP POST) is generated and output to the content provider in accordance with the content provider API within step 5. A subsequent operation is performed in step 6, where it is determined that the result is in the JSON (JavaScript object notation). And, as illustrated in the step 6, the result is stored in the context 520 for this request, i.e., a file named cloud.result.doc. Particularly, the data received in reply to the HTTP POST request is stored in the context. In step 7, an access token is stored in the session, and in step 8, an access token is fetched from the store. In step 9, another request to the content provider API 401a, for example, is transmitted in order to get the user profile from the service provider using the access token as the authorization header. This operation is the HTTP GET operation illustrated in step 9. A data provider reply is received and parsed in step 10 and stored in the context for this transaction.

[0169] In step 11 , a user ID is stored in a session, i.e., in the context for this transition, and, in step 12, a request for the favorite tracks of the user is generated using the session data, i.e., the context data and sent out as an HTTP GET request. This HTTP GET request corresponds to the first data instance request 502a and there result to this request, i.e., what the service engine receives from the data instance 401a is the first data instance reply 503a. The content is then parsed and stored in step 13 and, as illustrated in step 14, the following steps are repeated for all received tracks.

[0170] Particularly, the action or operation for each track is in step 15 the creation of a new response item, in step 16, the setting of the corresponding metadata to a certain value received from the content provider and in step 17 the setting of reference for a playback transform.

[0171] The result of the procedures from step 1 to step 17, i.e., the result of all these operations is illustrated in Fig. 13c, which shows a user reply in accordance with the service engine API which is generated by the service engine and transmitted to the media player 90, for example, of Fig. 3. When the example in Fig. 13a to Fig. 13c is compared to Fig. 4b, it becomes clear that the reception of the user message 501 at the service engine in Fig. 13a corresponds to step 50.

[0172] The transforming into the data instance request comprises steps 1 to 11 , and the sending out of the first data instance request in accordance with the first data instance API corresponds to the sending out of the HTTP GET message in step 12, and the reception of the content in reply to this get message corresponds to step 56. The steps 13 to 17 correspond to the transforming 58 of this data into the user reply in the service engine API and the actual transmitting of the message 504 in Fig. 13c to the media player 90, 100, 300, corresponds to step 60 of Fig. 4b.

[0173] It is to be emphasized that the minimum operations for the transforming 52 comprise the generation of the user profile in the step 9 and the generation of the HTTP GET message as illustrated in step 12 provided that an authorization, for example, has taken place before. The transforming 58 requires the steps 15, 16, 17, for each track in order to fill up the required fields cloud: name, cloud: artist, cloud: album and cloud: path as exemplarily required for the user reply. Generally, a cloud-path indication and any cloud metadata, preferably, a title will be a minimum the requirement for the user reply in accordance with an embodiment. However, more data as illustrated in Fig. 13a and the corresponding steps in order to fill up these data fields is useful.

[0174] Fig. 14a to 14c illustrate the sequences of transformations for replying to a user request 501 a for streaming a track until the response to the user illustrated in item 504a. Particularly, the requirements for the response are a playable URL which is, in this embodiment, the URL indicated as “1. HLS”, where this represents an exemplarily HTTP live streaming Protocol URL. However, other formats than the HLS format are, of course, useful and implementable.

[0175] In reply to the stream track request 501a, the first step is to acquire a cached session with the ID that has been generated before, i.e. , in an already existing context for this player.

[0176] In step 2, the HTTP GET message is generated. Sending out the HTTP GET message in step 2 of Fig. 14b corresponds to block 54, and the building of this message using the information from message 501a corresponds to the transforming operation 52 that only requires a single data formatting operation.

[0177] The receipt of the reply to the HTTP GET message corresponds to message 503i and the steps 3, 4, 5, 6, 7, correspond to the operations that are performed within the transforming step 58 of Fig. 4b. Particularly, the steps 3, 4, 5, 6, 7, are done in order to fill up the service engine API-conform user reply 504a in Fig. 14c which is finally sent or transmitted as illustrated in step 60 of Fig. 4b. Particularly, the user reply needs the cloud: path information, i.e., the playable URL “1. HLS”, and the information that a promotion with a thumbs up event is possible and the context information that track.1 is considered as required by the user within the original user request 501a. Step 2 of Fig. 14b corresponds to step 605, 606 of Figs. 10 and 11 , and step 5 corresponds to the building of the context illustrated at 602a in Fig. 10 or Fig. 11.

[0178] Fig. 15a and 15b illustrates a further embodiment, where only an “upstream reporting” is done. This starts with a service engine API compliant event 501 b, where the media player wants to report that it has started to play. The corresponding operations are step 1 , step 2, step 3 and step 4 and, finally, the generating the HTTP POST message 502k. Steps 1 , 2, 3, 4 are operations 61 to 64 that have been done in order to generate the first data instance request to the service provider in the form of the HTTP POST message 502k of Fig. 15b.

[0179] Particularly, in step 1 , the event that a player has started is received and, therefore, the state machine 538 in Fig. 9 is transitioned to the state running from the state start. In step 2, the state running is actually entered into the state machine that is part of the context 520. In step 3, a conformant report is generated in accordance with the model illustrated in Fig. 15a. The report transform is executed and the corresponding message is generated in steps 4 and 5 and the message is sent out where the information to the content provider includes an ID and the playing time.

[0180] Fig. 16a and Fig. 16b illustrate another situation, i.e., that the user liked the track. The corresponding model for the content provider is illustrated in Fig. 16a, and this model is used for generating the reply message HTTP POST at 502I of step 5 in Fig. 16b in response to the user request “event thumbs up”.

[0181] Again, in step 1 , a transition to the same state is performed in the state machine, since the thumbs up promotion does not change the player state. In step 2, an action transform with respect to the user’s liking of the track is generated and processed as illustrated in step 3 and, then, the session is generated in step 4 and, then, the message is generated in step 5 as message 502I and sent out in accordance with the content provider’s application programming interface (API).

[0182] Fig. 17a and 17b illustrate a situation, when the player has finished playing, i.e., when the user request is event player exit as illustrated at 501 d in Fig. 17a. The corresponding model for the content provider is illustrated in Fig. 17a and the transformations done in response to this procedure is illustrated in Fig. 17b, where the state machine is transitioned to the finished state and of life of player to context is determined, i.e., the context 520 is cleared. In this example, any further report to the content provider is not specifically generated. However, there can also be content provider that also require the report on the player exit event. Fig. 18 illustrates the summary over several user requests browse (Fig. 13a to 13c), track (Fig. 14a to 14c), event (Fig. 15a to 15b), report (Fig. 15a 15b, 17a, 17b) and “action” (Fig. 16a and Fig. 16b).

[0183] Subsequently, further examples for illustrating different transforms for the purpose of communicating with different data instances are illustrated.

[0184] The general scheme is shown in Fig. 19, where a live service API conformant request 501 is transformed (52 of Fig. 4b) in a provider specific API request 502, and where a provider specific API response 503 is transformed (58 of Fig. 4b) into a service engine API response 504.

[0185] The initial browse request 501 and the response 504 is illustrated in Fig. 20. All the different alternatives in Fig. 21a to Fig. 24b result in a same Fig. 20 response 504 and originate from the same user request 501. However, for each example in Figs. 21a to Fig. 24b, the provider API was different, i.e., had different rules and requirements for communication. The scenario illustrated is that the user browses his favorite audio tracks on a human machine interface exemplarily implemented as the user interface 302 of Fig. 2c. The user replies by telling the user interface 302 to display tracks 3 and 4 from the favorites track. To display the track, the metadata title artist and album shall be returned, and in order to playback the track later, a playback reference shall be included as illustrated in Fig. 20 at item 504 illustrating the corresponding required reply and the playback reference included in the field “cloud: path”.

[0186] The request 501 represents a single browse request to the folder “favorite_tracks” and requests two items starting from index 2. In this notation, the following format is used to reference such a PI cause: “request-type: / / resource?params.”

[0187] For this example, the request to the API consequently takes the form illustrated in Fig. 20, item 501. The response to a service engine API browse request is, in this example, a list of items, where each item consists of key value pairs. For simplicity, the following JSON notation for the responses is used. For the above example, the response illustrated at 504 is expected. This corresponds to the title, artist, album metadata of two tracks and a path for each track that allows to reference the track for playback via a request of type track to the service engine API. All subsequent examples in Figs. 21a to 24b will result in this response 504 of Fig. 20. Fig. 21a illustrates an HTTP GET request 502a to the provider in accordance with the JSON format. The response to this format is illustrated in 503a. Alternatively, Fig. 21 b illustrates a similar request, but in the XML format, and the XML response is illustrated at 503a. The different appearances of the response 1 in Fig. 21a and response 1 in Fig. 21 b illustrate the different content providers APIs, to which the request 502a have been sent, and from which the different replies 503a have been received.

[0188] The request transform 52 of Fig. 19 was a transformation of request 501 into 502a of Fig. 21a and Fig. 21 b, and the response transform 58 of Fig. 19 corresponds to the transformation of the result in Fig. 21a or Fig. 21b into the format of Fig. 20.

[0189] While Fig. 21a and Fig. 21b illustrate the situation, when the different content providers’ application programing interfaces operate in different formats or “languages”, the examples in Fig. 22a to Fig. 24b illustrate the situation, when the application programing interfaces are different in what they expect to receive and what they send out in reply to a certain request. Fig. 22 illustrates the situation where a service provider does not allow a random access, but only provides page tokens for next and potentially previous pages. In such a situation, the request 502b is shown which has been generated by a request transform 52 from the request 501 of Fig. 20. The first response is illustrated at 503b and only results in an old song, another old song and a page token for the next page. This information is not yet enough for generating the full response 504 in Fig. 20, since only the two new songs are required rather than the old songs. As illustrated in Fig. 22b, the service engine generates a second request shown in 502c for obtaining, in response to the next page token, the songs from the new page, i.e., the new song and the another song with IDs 2 and 3.

[0190] Now, enough information is there in order to bid, from the information in Fig. 22b, the full response in Fig. 20, i.e., for filling the cloudmame, cloud:artist, cloud:album and cloud:pathfields of Fig. 20.

[0191] The request transform 52 not only comprises an operation to generate request 1 502b from the user request 501 , but the additional operation is to generate request 2 502c using the output of the earlier response 503b, i.e., the token information “ABCD” which is received from the response 1 and which is the input into the request 2 502c in Fig. 22b. Fig. 23a to Fig. 23c illustrates the situation, where a certain content provider does not provide enough metadata in response to a corresponding request. Particularly, the content provide has such an API protocol that, in response to a favorite track’s request, only the name of the requested songs are provided as shown in Fig. 23a, but, in order to fully fill the request in accordance with the service engine API, more metadata is required as illustrated in Fig. 20. Therefore, in order to fully obtain the necessary information for the reply 504 in Fig. 20, further requests on top of request 1 502d in Fig. 23a are necessary. These are the request 2 502e where metadata for the album field is requested, which is received in response 503e, and, additionally, a third request 502f is required, where further metadata for the further album, i.e., track 3 is requested. A response 2 operates for requesting the missing metadata for ID 2 and response 3 is therefore acquiring the necessary metadata for ID 3.

[0192] The request transform 52 comprises, in the example of Fig. 23a, the generation of request 1 502d, request 2 502e and request 3 502f, where the output of the first response 503d, i.e., the track ID is used as input into the request 2 502e and the request 3 502f. The response transform 58 consists of using the information in response 1 503d, response 2, 503e, and response 3, 503f and putting this information into the format of the service engine API as illustrated in Fig. 20.

[0193] Figs. 24a and 24b illustrate the situation, when a certain filtering operation is performed. The results that are obtained in response to a certain request may be filtered on specific metadata such as the explicitly of the content. This has an impact on the paging and number of requests that need to be performed.

[0194] The first provider specific API request 502g is illustrated in Fig. 24a and a reply, i.e., a provider specific API response is shown at 503g, where the explicity of the individual tracks is given. However, for the example, it is assumed that only the non-explicit content is to be provided to the user via the response 504 in Fig. 20.

[0195] Due to the fact that item 504 requires two tracks with two corresponding metadata, and the response in Fig. 24a only provided a single result, i.e., ID 2 as a non-explicit content, a further request, i.e., request 2 is made in step 502h and the result is illustrated in 503h bringing back two other tracks with IDs 3 and 4 that do not have explicit content. The information required for the reply in Fig. 20 can be provided, while it is typically selected to provide two songs with ID 2 and ID 3, and the information on ID 4 is superfluous in this example and will be discarded for the generation of the response message 504.

[0196] The transformation 52 underlying the procedure in Fig. 24a to Fig. 24b comprises the generation of the request 1 502g from the request 501 in Fig. 20, and the generation of request 2 dependent on the result of the first request, i.e., response 1. Particularly, the index is increased by two in the request 2 502h, i.e., to a starting index of 4 (request 1 502g had the starting index of 2). The request 2 depends on the situation that response 1 did not provide two non-explicit tracks. Only the track with ID 2 was non-explicit. Therefore, the tracks for ID 3 and ID 4 had to be obtained. In an exemplary situation, where the response 2 only results in explicit content, another request 3 is to be generated again with an increased starting index, i.e., a starting index of 6.

[0197] Subsequently, further advantages and features of the embedding of the service engine and, particularly, the live services apparatus 220 into a cloud backend such as item 200 in Fig. 2b are illustrated, which refer to the situation of the cloud backend, the user device, the data instances and user application within a cloud system.

[0198] A cloud system in accordance with the present invention comprises several individual elements that cooperate with each other via corresponding interfaces. These interfaces can be hardware interfaces, logical interfaces, or mixtures between hardware interfaces and logical interfaces. The hardware interface of a user device can, for example, implement the control interface to a cloud backend and, at the same time, the media interface to a remote data instance. Contrary thereto, a user interface implemented by the apparatus for performing a user application will typically be a touch-sensitive display of the device implementing the user application in order to receive a user input while the cloud backend interface may be implemented by means of a hardware interface that may be a wired or wireless interface between the device, on which the user application is running, and an entry point of the cloud backend such as a hardware reachable via a cloud backend access address.

[0199] At the same time, a user device interface of the cloud backend and a communication interface for interfacing with a data instance such as a content provider or upload platform is implemented in a cloud backend together with the application interface for interfacing with a management application providing a user input. The realization of such interfaces can be done via hardware elements and / or software routines known for internet platforms or internet services and the related interfaces.

[0200] In an embodiment, a user device comprises a device hardware, a media runtime device software configured for controlling the device hardware, and typical means such as a processor and a memory for operating the user device. The media runtime device software comprises a control interface to a cloud backend device or platform and a media interface to a remote data instance. Particularly, the media runtime device software is configured for receiving commands for controlling the device hardware via the control interface and to receive or transmit content via the media interface.

[0201] Thus, the user device and, particularly, the media runtime device software which is implemented on the user device typically already from a device manufacturer makes sure that the full control of the device hardware is established via the control interface to the cloud backend and, on the other hand, the high volume data traffic to a typically remote data instance is done only via the media interface without introducing and such data into the cloud backend.

[0202] Thus, it is made sure that the high volume data traffic is not rooted within the cloud backend, since the media interface of the media runtime device software is not open to the cloud backend in the normal media streaming operation either for the purpose of downloading media or uploading media data.

[0203] Thus, a relatively smart procedure is possible in that the full control is done via the cloud backend without the high volume data traffic, while the high volume data traffic only takes place between the user device itself and the typically remote data instance selected for the purpose of uploading or downloading digital data such as media data.

[0204] On the other hand, a full control of what happens with the available media streams provided by the content provider typically against payment of a fee is made sure by means of the media runtime device software running on the user device which is configured for fully controlling the device hardware of the user device.

[0205] A further element of the cloud ecosystem is the cloud backend that comprises a user device interface for interfacing with a user device, a communication interface for interfacing with a data instance and an application interface for interfacing with a management application providing a user input. The cloud backend can be a distributed or concentrated system and, basically, comprises one or several processors and one or several memories. Thus, the cloud backend apparatus is configured to provide messages to the user device for controlling the user device to handle media data, and the cloud backend is additionally configured to receive messages from the user device via the user device interface. Particularly, the cloud backend is configured to communicate to the data instance via the communication interface only regarding an authorization of a user of the user device to receive data necessary for the authorization from the data instance or to send corresponding authorization data to the data instance. However, any media data are not exchanged between the cloud backend on the one hand and the data instances under consideration on the other hand. Only control data are exchanged so that the high volume data traffic that occurs, when a data stream from a content provider is established to a user device is kept out of the cloud backend in order to have smart and efficient cloud backend operation.

[0206] The cloud backend comprises an application interface for interfacing with a management application providing a user input. The user input is, for example, a request for a certain media stream or a request for uploading data captured by the user device, and such a request is forwarded from the user control application or user management application or user app to the cloud backend, and the backend receives this data via the application interface of the cloud backend.

[0207] A third constituent of the ecosystem is an apparatus for performing a user application or user app which once again comprises, as typically computer devices or smart devices, a memory and a processor. The apparatus for performing the user application comprises a user interface for receiving an input from a user and a cloud backend interface for communicating with a cloud backend. However, the apparatus for performing the user application does not have an interface to the individual user devices, since these individual user devices are only controlled via the cloud backend rather than by the user application directly.

[0208] The user interface is typically a human-machine interface such as a touch-sensitive display or a speech interface or any similar implementation. The cloud backend interface manages the communication to and from the cloud backend for the purpose of the selection of data instances such as data sources or content providers or data sinks, i.e. , upload platforms. A selection and control of the user devices is done via the cloud backend and a selection of media content for up or download is also done and managed via the cloud backend interface. Again, any high-volume traffic occurring when an actual stream is downloaded or uploaded is kept out of the user application apparatus, since this traffic only takes place between the user device itself and the data instance(s) via the media interface implemented by the media runtime device software running on the user device.

[0209] When the ecosystem is considered as comprising the user device, the user application apparatus, the cloud backend and the (remote) data instances, each constituent has a very dedicated and limited interface situation.

[0210] The user application interfaces with the user via the user interface and with the cloud backend via the cloud backend interface. However, the user application does not interface with the remote data instances or with the user device via the media runtime device software configured for controlling the device hardware, when the user application is implemented on a device that does not also have such a media runtime device software. However, when the user application is also implemented in the same hardware device that also involves the constituents of the user device, then the media runtime device software nevertheless makes sure that the user application does not access the media interface or does not access the control interface to the cloud backend.

[0211] When looking to the cloud backend, the cloud backend only interfaces with the user application via the application interface, with the user device via the user device interface and with the remote data instances via a control interface for managing access and authorization issues. However, the cloud backend does not interface with the media interface of the user device or with the media port of the remote data instances by means of which the remote data instances output media streams or digital download data or receive media streams or digital input data.

[0212] On the other hand, the remote data instances do not interface with the user application but only interface, for control purposes, with the cloud backend and, for data receiving or data outputting purposes with the media interface of the user device. However, there is no possibility that the remote data instance communicates with the control interface operated in the user device.

[0213] From the point of view of the user device, it is to be noted that the user device is limited to only communicate with the cloud backend via the control interface implemented by the media runtime device software and to not interface with the user application and to only interface with the remote data instances via the media interface.

[0214] By means of these technical features, it is made sure that an efficient control is obtained that provides a high comfort for the user. This is due to the fact that the user only has to manage her or his account with the ecosystem via the user application, since the individual application programming interface (API) requirements for interfacing with different remote data instances are all handled in the cloud backend without further user interaction when the user has, typically at the first time of accessing the cloud service, installed the required login data at the remote data instances.

[0215] From the perspective of the data instances, the media runtime device software controlling the user device on the one hand and the cloud backend processing on the other hand make sure that the required data protection from the beginning to the very end point in the device hardware is respected and enforced. The fact that a translation of the different application programming interfaces required by the different data instances into a specific and generic protocol between the media runtime device software in the user device and the cloud backend is established, allow the operators of the remote data instances to implement changes easily. The changes only have to be adapted and respected by the cloud backend, but any changes at the data content provider do not result in any changes at the user device, and even do not result in any changes for the media runtime device software running on the user device, since the communication between the cloud backend and this service on the user device is always the same for each data instance and for each update of each data instance.

[0216] It is to be mentioned here that all alternatives or aspects as discussed before and all aspects as defined by independent claims in the following claims can be used individually, i.e., without any other alternative or object than the contemplated alternative, object or independent claim. However, in other embodiments, two or more of the alternatives or the aspects or the independent claims can be combined with each other and, in other embodiments, all aspects, or alternatives and all independent claims can be combined to each other.

[0217] Due to the fact that the Cinemo cloud backend is distributed in a cloud, this functional unit maintains, among others, several databases that include, among others, the “Cinemo Live Services” functionalities that represents the actual interface between the media app content delivery network (media app CDN) and the Cinemo cloud backend.

[0218] The Cinemo cloud backend makes sure that correct reports back to the artist with respect to Copyrights, etc. are assured by the Cinemo cloud backend. This service can be performed by the Cinemo Live Services. A feature of the Cinemo cloud backend is the Cinemo Live Services. This block provides a kind of “translation” of the language requirements of different content providers such as Netflix, Spotify, Disney Plus, Amazon Prime, etc., into a single “language” that is processed by a Cloud Server box of the device SDK player so that the core online player can process the data received from the content provider.

[0219] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0220] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed.

[0221] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0222] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0223] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier or a non-transitory storage medium. In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0224] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.

[0225] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.

[0226] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0227] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0228] In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0229] The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein. References

[0230] [scxml2015]: State Chart XML (SCXML): State Machine Notation for Control Abstraction,

[0231] W3C Recommendation 1 September 2015, htps: / / www.w3.org / TR / scxml /

[0232] [widevine]: https: / / www.widevine.com /

Claims

Claims1. Method of managing a communication between a user instance (90, 100, 300) and a plurality of data instances (401 , 402, 403) comprising a first data instance and a second data instance being different from the first data instance, comprising: receiving (50) a user request (501) directed to the first data instance at a service engine (224), the user request (501) being in accordance with a service engine application programming interface (API) (222) dedicated for a communication between the user instance and the service engine (224); transforming (52) the user request (501) into a first data instance request (502) in accordance with a first data instance application programing interface (101a) dedicated for a communication with the first data instance; sending (54) the first data instance request (502); receiving (56) a first data instance reply (503) in accordance with the first data instance application programing interface (101a); transforming (58) the first data instance reply (503) into a user reply (504) being in accordance with the service engine application programing interface (222); and transmitting (60) the user reply (504).

2. Method of claim 1 , further comprising: receiving a second user request directed to the second data instance from the user device, the second user request being in accordance with the service engine API (222); transforming (52) the second user request into a second data instance request in accordance with a second data instance API (402a) dedicated for a communication with the second data instance and sending (54) the second data instance request, the second data instance API (402a) being different from the first data instance API;receiving (56) the second data instance reply being in accordance with the second data instance API (402a); transforming (58) the second data instance reply into a second user reply being in accordance with the service engine API (222); and transmitting (60) the second user reply.

3. Method of claim 1 or 2, wherein the service engine (224) is connected to a data instance library (230), the data instance library comprising, for each data instance (401 , 402, 403) a dedicated model (231 , 232, 233), and wherein the steps of transforming (52, 58) are configured to use, in the transforming, the corresponding model (231 , 232, 233) associated with the corresponding data instance (401 , 402, 403).

4. Method of one of the preceding claims, wherein the different data instances are content providers, and wherein the user request (501) is a stream track request (501a).

5. Method of one of the preceding claims, wherein the transforming (52) into the first or a second data instance request comprises: creating (602a, 651a), by the service engine (224), a player context in response to a stream track request, the player context (520) comprising a unique player context ID; and sending (503), by the service engine (222), the unique player ID to the user instance (90, 100, 300).

6. Method of claim 5, wherein the user request comprises a resolve message request (604), and wherein the transforming (50) the user request into a first or a second data instance request comprises transforming the resolve message (604) into a request message (605) as the first data instance request to ask for track location information.

7. Method of claim 6, wherein the receiving the first data instance reply comprises receiving (56) the track location information in accordance with the first data instance application programing interface, and wherein the transforming (58) the first data instance reply (606) comprises transforming the track location information into the user reply (607) based on the player context.

8. Method of one of the preceding claims, wherein the transforming (52, 58) comprises a sequence of at least two operations (61 , 62, 63, 64), wherein a later operation is configured to receive, as an input, an output of an earlier operation.

9. Apparatus of one of the claims 5 to 8, wherein the service engine (224) is configured to generate, as the context information, at least one of the following information: configuration information, comprising language settings or resolutions; service information comprising static information on the corresponding data instance active during an execution of a task related to the data instance; authorization information carrying information required for authorizing at a certain data instance; parameter information comprising information related to a specific user request, based on which the context (520) has been created; and variable data comprising storing translation results or operation results; and playback data comprising data related to the playback or playback context data.

10. Method of claim 3, wherein the data instance library (230) comprises, for each data instance, at least one of manifest data (240a) comprising mandatory data for the data instance;authorization data (240b) related to an authorization method for the data instance; content excess data (240c) relating to how a content can be accessed at the certain data instance; and playback data (240d) relating to a playback context.

11. Method of one of claims 5 to 10, wherein the plurality of data instances are different content providers, the method comprising: receiving (608, 657, 501b, 501c) playback event message in accordance with the service engine API (222); transforming (52) the playback event into the first or a second data instance API report message using the player context; and sending the playback event report obtained by the transforming (52).

12. Method of one of the preceding claims, wherein the first data instance API and the second data instance API are different in a syntax format, or in a semantic format indicating a certain way of data access in response to a request or indicating a different amount of metadata provided in response to a request.

13. Method of claim 12, wherein the syntax format comprises a JSON format or a XML format, or wherein the semantic format comprises a format, in which random access is not allowed, but only an access to a next page, a current page or a previous page is allowed, or wherein the semantic format comprises a format, in which a predefined amount of metadata is not provided in response to a request, but more metadata than required by the service engine API or less metadata than required by the service engine API is provided.

14. Method of one of the preceding claims, wherein the user request (501) comprises a browse request (501) for browsing favorite content items; wherein the transforming (52) into the first data instance request (502) comprises setting up a context (520) for the request, generating a get information request to the first data instance, the get information request comprising an information on the first data instance, and an information on the user, wherein the receiving (56) comprises receiving a reply to the get information request as the first data instance reply (503), wherein the transforming in (58) into the user reply (504) comprises parsing the first data instance reply and storing parsed data in the context for the user request and generating reply in accordance with the service engine API (222) comprising the parsed data and a reference for each favorite content item.

15. Method of claim 14, wherein the transforming (52) into the first data instance request comprises: fetching a token for the user; requesting an authentication information from the first data instance using the token; receiving an access information from the first data instance and storing the access information in the context (520); requesting a user profile information from the first data instance using the access information; and storing the user profile information in the context (520), when the user profile information is included in the first data instance reply.

16. Method of claim 14 or 15, wherein the transforming (58) into the user reply (504) comprises: receiving user favorite tracks information;generating a new response item for each favorite track, and formatting the user reply so that, for each item, a content name and a track identification is included in the user reply (504).

17. Method of one of claims 1 to 13, wherein the user request comprises a stream track request (501a) for accessing a track with a track ID; wherein the transforming (52) into the first data instance comprises generating a message requesting a playback metadata for the track ID; and wherein the transforming (58) into the user reply (504) comprises generating an item with the track ID and the playback metadata.

18. Method of claim 17, wherein the transforming (58) into the user reply comprises adding to the user reply, an information on one or more allowed player operations.

19. Method of claim 17 or 18, wherein the user instance (90, 100, 300) is a user device (100) comprising a media runtime device software (110) having a control interface (112) to a cloud backend (200) and wherein the service engine (224) is placed within the cloud backend (200).

20. Method of one of claims 1 to 19, wherein the user instance is a user application apparatus (300) comprising a user interface (302) to a user (99).

21. Method of one of the preceding claims, wherein the user instance comprises a user device having a device hardware (102), and a media runtime device software (110) configured for controlling the device hardware (102), wherein the media runtime device software (102) comprises a control interface (112) to a cloud backend (200) and the media interface (114) to a data instance (400), wherein the user instance comprises an apparatus (300) for performing a user application, the apparatus (300) comprising a user interface (302) for receiving aninput from an user (99), and a cloud backend interface (304) for communicating with a cloud backend (200), and wherein the service engine (224) is placed in a cloud backend (200) comprising a user device interface (202) for interfacing with a user device (100); and a communication interface (204) for interfacing with the first data instance (401) and the second data instance (402).

22. Method of one of the preceding claims, further comprising: receiving an event message from the user in accordance with the service engine API (222), transforming the event message into a report message in accordance with the first data instance API (401a) or the second data instance API (402a) and generating an information posting request; and sending the information posting request to the first data instance or the second data instance.

23. Method of claim 22, wherein the event message comprises a player start message, a promotion message, or a player exit message.

24. Method of claim 22 or 23, wherein the transforming comprises advancing a state machine (538) maintained by the service engine (224) in accordance with an earlier state of a context (520) and the event message from the user instance and creating a report information depending on the event message and introducing the report information into the report message.

25. Method of one of the preceding claims, wherein the transforming (52, 58) comprises one or more operations (61 , 62, 63, 64) comprising: a control flow operation, a context read operation, a context write operation, a logging operation, a request execution operation, a response parsing operation, and an event emitting operation.

26. Method of claim 25, wherein the operations of the group of operations are limited to the operations in claim 25 and do not include more operations.

27. Method of one of the claims 1 to 4, wherein the transforming (52) into the first data instance request (502) comprises creating (651a) a radio player context (521) and sending a player context identification message (652), and receiving a request (653) for radio tracks for the radio player context (521) and introducing the request for the radio tracks into the first data instance request (502).

28. Method of claim 27, wherein the transforming (58) comprises introducing track IDs into the user reply (656), wherein the track IDs are obtained from the first data instance reply (655).

29. Apparatus for managing a communication between a user instance (90, 100, 300) and a plurality of data instances (401 , 402, 403) comprising a first data instance and a second data instance being different from the first data instance, the apparatus comprising: a receiver for receiving (50) a user request (501) directed to the first data instance at a service engine (224), the user request (501) being in accordance with a service engine application programming interface (API) (222) dedicated for a communication between the user instance and the service engine (224); a transformer for transforming (52) the user request (501) into a first data instance request (502) in accordance with a first data instance application programing interface (101a) dedicated for a communication with the first data instance; a transmitter for sending (54) the first data instance request (502); wherein the receiver is configured for receiving (56) a first data instance reply (503) in accordance with the first data instance application programing interface (101a); wherein the transformer is configured for transforming (58) the first data instance reply (503) into a user reply (504) being in accordance with the service engine application programing interface (222); andwherein the transmitter is configured for transmitting (60) the user reply (504).

30. Computer program for performing when running on the computer or a processor, the method of one of the claims 1 to 28.

31. Cloud system comprising: one or more user devices, a user device (100) comprising: a device hardware (102), and a media runtime device software (110) configured for controlling the device hardware (102), wherein the media runtime device software (102) comprises: a control interface (112) to a cloud backend (200); and a media interface (114) to a data instance (400), wherein the media runtime device software (110) is configured to receive commands for controlling the device hardware (102) via the control interface (112), and to receive or transmit content via the media interface (114) in response to a corresponding command received via the control interface (112); a cloud backend (200) comprising: a user device interface (202) for interfacing with the user device (100); an application interface (206) for interfacing with a user application transmitting information on a user input, and a communication interface (204) for interfacing with the data instance (400), wherein the cloud backend is configured to provide messages to the user device interface (202) for controlling the user device to handle media data, and to receive messages from the user device via the user device interface (202), and wherein the cloud backend is configured to communicate to the data instance (400) via the communication interface (204) regarding an authorization of a user of the user device to receive data from the data instance or to send data to the data instance; and one or more apparatuses for performing the user application, an apparatus (300) for performing the user application comprising: a user interface (302) for receiving an input from the user (99); a cloud backend interface (304) for communicating with the cloud backend (200), wherein the apparatus is configured to communicate, via the user interface (300) to receive user commands regarding a selection of one or more data instances and a selection of content to be sent to the data instance or to be retrieved from the data instance and to display messages for the user, to communicate, via the cloud backend interface (304), regarding the selection of the one or more data instances to be accessed by the user devices, and tocommunicate, via the cloud backend interface (304), regarding the selected content for sending from the user device to the data instance or from the data instance to the user device, wherein the cloud backend interface (304) of the one or more user applications (300) is connected to the application interface (206) of the cloud backend (200), wherein the control interface (112) of the one or more user devices (100) is connected to the user device interface (202) of the cloud backend (200), wherein the communication interface (204) of the cloud backend (200) is connected to the data instance (400, 401 , 402) for control purposes only; wherein the media interface (114) of the one or more user devices (100) is connected to the remote data instance (400) for the purpose of downloading or uploading media data; and wherein the cloud backend (200) is configured to perform the method as defined in one of claims 1 to 28.

Images