Method and apparatus for media streaming

Abstract

A server arrangement for delivering media comprises two elements, a processor and a job processor. The processor is arranged to receive requests for media from multiple client devices across a network

Description

BACKGROUND OF THE INVENTION This invention relates to delivery of media to user devices and in particularly to delivery at large scale, namely to large numbers of devices. The delivery of media to large numbers of devices at scale is a known problem. Whenever multiple client devices connect to a network and request realtime streaming of audio, video or more generally media, multiple simultaneous requests need to be serviced consuming resources of both computational power and bandwidth. Such media therefore includes audio, video or combinations thereof as well as interactive media such as games and object-based media. Various network arrangements are known for addressing this type of problem. Content delivery networks (CDN) arrange intelligent storage of media and nodes within a network such that devices may retrieve media that is requested more frequently from nodes more local to the device. In this way, frequently requested media is delivered once from a central node to local nodes and may then be delivered multiple times from those local nodes to locally connected user devices. Various processing arrangements are also known for scheduling of processing. In the context of deployment of GPUs, it is cost prohibitive to give every user an instance of a GPU in a cloud-based service, and for that reason deployment is usually by GPUs being “sliced” arbitrarily and deployed as separate virtual machines. However, as the deployment is arbitrary in the sense that it does not manage resources between users, this is an inefficient use of GPU resources. In general, known processes for processing and / or storage and delivery of media provide partitioning of processing and storage of media to improve delivery, but fail to properly address the delivery at large scale to large numbers of devices. SUMMARY OF THE INVENTION We have appreciated that existing arrangements do not adequately allow scaling of delivery of media, in particular object based media to heterogenous devices for large numbers of users. In broad terms, the invention provides systems and processes by which media is deployed in a granular way comprising jobs which may be executed by a processor according to a scheduling algorithm. In this way, demands from multiple client devices, in particular at very large scale of many thousands of client devices, can be efficiently executed by a processor whilst avoiding redundancy of processing and degradation of performance between such client devices submitting requests for media. An embodiment of the invention may be considered in two parts. First, a job scheduling algorithm is deployed which receives requests from client devices and cooperatively controls a processor, in particular a GPU, by providing scheduling of the requests to the processor. The scheduling is cooperative in the sense that multiple requests for processing the same job from multiple client devices will be handled by only processing such requests once, where possible, and providing the output multiple times. Second, the arrangement of the jobs which represent requested media with dependencies as defined by client devices allows the scheduling algorithm to be deployed in a range of different use cases, with the adaptation being by dependencies asserted by the jobs as specified by client devices. The invention is defined in the claim to which reference is directed with preferred features set out in the dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS The invention will be described in more detail by way of example with reference to the accompanying drawings, in which: Fig. 1 is a diagram showing the main components of a system implementing the invention; Fig. 2 is a diagram showing the arrangement of figure 1 in the context of multiple clients and a specific embodiment relating to processing of image frames; Fig. 3 is a functional diagram showing the relationship between experiences requested by clients and the delivery of those experiences by a scheduling algorithm according to an embodiment; Fig. 4 shows an initial state of a schedule of a scheduling algorithm according to an embodiment; Fig. 5 shows a subsequent state of a schedule after addition of jobs for one of the experiences (A) processed; Fig. 6 shows a subsequent state of a schedule after addition of jobs for one of the experiences (C) processed; Fig. 7 shows a further subsequent state of a schedule after addition of jobs for one of the experiences (A) processed; Fig. 8 is a schematic diagram showing the structure of client experiences that may be processed according to an embodiment; Fig. 9 shows an example of top down dependencies of structured client experiences; Fig. 10 shows an example of bottom up dependencies of structured client experiences; DESCRIPTION OF A PREFERRED EMBODIMENT OF THE INVENTION The invention may be embodied in a method of delivering media to a client, devices for delivering client media, methods and devices for receiving media and systems involving such processes and devices. Overview Embodiments of the invention provide the ability to deliver media such as object-based media to end devices in a more flexible and scalable manner. In this regard, media includes audio, video, object-based media, games and any combination of these that may be presented to users on client devices. Client or user devices include televisions, handheld devices, tablets, smartphones and other devices that receive and present audio and video to users. A goal in delivery of media across networks is so-called sub-linear scaling, namely increasing the resources needed in terms of bandwidth and computation of processing at a lower rate than increasing the number of users or size of network. Such sub-linear scaling can provide significant technical advantage in terms of reduced computational or bandwidth requirement. In broad terms, the invention resides in providing GPU computational services to multiple different user devices under control of a CPU operating a single CPU process. In this way, a single GPU may be used in its most efficient manner and intermediate results stored for reuse by multiple client devices. Such an approach may be repeated for multiple GPUs delivering at large-scale to even greater numbers of client devices. A simultaneous gain in efficiency of processing and saving of bandwidth is provided. A particular example of the use of a single GPU by multiple client devices under control of a single CPU process is in the context of rendering. Rendering may be considered a computational process that takes a descriptive form for delivery of audio, video or more generally media and converting to a form that can be output by a device. In the context of video whether 2D or 3D the descriptive form includes data such as coordinates, textures and shaders and the output of the rendering process would be pixels coded in any format suitable for an end device. Rendering is also applicable to audio as audio may be described in terms of a sound field and the rendering process will convert that to output which may be delivered as MP3 or other audio format. Whilst the invention may be implemented for audio or video the invention may also be applicable to other data that can be processed by a GPU. Various embodiments of the invention will be described, but common to those embodiments is at least one GPU operable to execute jobs under the control of the CPU that provides scheduling of requests for jobs from multiple users in accordance with a load sharing algorithm and a VRAM arranged to store sub- products of the jobs that may be shared amongst multiple user requests. The control of a single GPU in accordance with the scheduling algorithm by one CPU on behalf the request of multiple users ensures the most efficient use of GPU resources and can also allow for synergy between the requests to allow multiple requests to be presented to the GPU as a single job. The use of a VRAM arranged to store sub- products from the GPU allows the results of processing to be reused amongst multiple requests. The above is all undertaken by a single operating system process and so the GPU is genuinely a single GPU delivering results to multiple users, in contrast to prior arrangements that might provide several virtual GPU’s on a single hardware device in order to deliver services to multiple connected competing client devices. The algorithm operable on the single operating system process can be tailored to the particular use case. In the example of graphics rendering, factors such as latency, resolution, bandwidth and other measures of quality may be taken into account in prioritising jobs scheduled for the GPU. Discussion of Embodiments Embodiments of the invention optimise the utilisation of server-side computing resources when delivering computationally demanding media experiences to a large number of remote users to enable sub-linear scaling of costs of compute resource. While this application has significant benefits, the invention is also applicable beyond this domain and its broader applications are also described. Mature audio and video services, such as video-on-demand platforms over IP, provide users with a catalog of content and facilitate streaming. For example, a service provider maintains a website featuring a wide range of offerings, similar to the BBC iPlayer. From this platform, audience members can browse, select, and play back content using a web browser or dedicated app. Behind the scenes, various processes occur to enable seamless playback. Ultimately, this results in clients requesting specific content, which is then rendered and played back. While there may be variations in terms of the user interface (e.g., replacing browsers with apps), the core experience remains consistent. In both cases, the computational profiles are similar: servers push data, clients contain video decoders - often implemented as hardware due to complexity - and play back the content. While server-side workloads can be high, client-side computational complexity is relatively low. At the lower or niche end of the market, devices may be limited to decoding a single (or possibly two) videos at a time. Now consider Emerging online Media Services. Emerging online media services exhibit several key characteristics: 1. Advanced client-side processing, involving the composition of multiple videos, layering of graphics (including 3D elements), and in some cases, neural feature detection for tasks like background removal. 2. Online services where one or more of these features are rendered on the server side, allowing for a distributed approach to computational complexity. 3. A hybrid model, where a balance is struck between client-side and server-side processing, with each handling the computational demands that suit them best. At one extreme, we see online gaming experiences where everything is rendered client-side, with only event data or requests for resources transmitted over the internet. At the other extreme, we find services that render the entire experience on the server side, sending a video stream (often referred to as “pixel streaming”) to the client and receiving limited control signals to shape the gaming experience. More moderate examples of these emerging trends include compositing multiple videos and images together in object-based branching narratives. Examples of this include the BBC’s “Click 1000”, Netflix’s “Bandersnatch”, Amazon’s “Luma” service, and others that blur the lines between storytelling and interactivity. If you don’t prioritise universal access, you can focus on high-end client hardware and target those devices exclusively. However, for both Public Service Media Services (such as public broadcasters) and Private Media Services, universal or near-universal access is crucial. Public Service Media Services, in particular, rely on universality due to the universality of funding. For private media services, the benefit lies in market size. But, by necessity, public service broadcasters must care about universality. To achieve universal access, we have appreciated that you need to provide either (or both) of the two options described above: server-side rendering (“2”) or client-side rendering with server-side support / offload (“3”). For Private Media Services, the cost of running the service can be offset through linear scaling. You simply charge users for the new service. In contrast, Public Media Services face a fixed income challenge. A popular service that heavily utilises “2” or “3” would require enabling sub-linear scaling to efficiently use and reuse server resources. Embodiments of the invention specifically target enabling one form of sub-linear scaling: efficient use and reuse of server compute resources. To explain this, we will first discuss linear scaling techniques before discussing how embodiments of the invention enable sub-linear scaling. To illustrate the challenges involved in implementing “pixel streaming” and its related issues, we will consider an example: remotely rendered game experiences that stream as video to the user, who then controls the game remotely. To implement this simplest form of “pixel streaming”: • You have a discovery / catalog service similar to a VOD service • The user starts their client, browses the catalog, and selects a game • The client connects using its game remote access client to the server • The server either starts or has the game running already on dedicated hardware • The output from that dedicated hardware is streamed to the client • The client decodes this video stream, and commands (via a keypad / joystick / mouse) are sent over the network back to the server, which controls the game. We have appreciated that, there are many challenges in implementing such a system, even at its most basic level. One of the more fundamental issues is the cost of “one machine per user.” This can be addressed by using virtualisation techniques. One simple way to do this is to use a high-end machine with multiple CPU cores and a large graphics card, then utilise virtualisation techniques to share the devices. There are various approaches, but most common is timeslicing CPU usage via standard hypervisor mechanisms like KVM, VMWare, Virtualbox, and so on. These technologies all have one thing in common: they schedule items on all VMs on a single machine to ensure fair access to resources for competing processes. The underlying mechanisms vary, but ultimately, this means that low-level scheduling of resources has no understanding of the applications running on top of the system. By using this virtualisation approach, the “one machine per user” becomes “one machine for n-users.” This also makes better use of resources and reduces the cost per user. However, we’re still using “one virtual machine per user,” so the cost scaling is still linear relative to the size of the audience. We will now re-examine Emerging Media Experiences, considering not just the extreme case but also the medium-term reality of modern media experiences, including compositing multiple videos and rendering imagery onto one another in object-based branching narratives. This reconsideration is important because it clarifies opportunities we have appreciated for imagery reuse. In this scenario, we have a user enjoying a branching narrative that overlays 2-3 videos on top of each other. There may be multiple users watching the same content, which presents an opportunity to reuse generated imagery and audio. However, this requires a system that can evaluate a media experience for a given client and determine if it can reuse results created for another client enjoying the same media experience. If rendering systems are completely isolated, as in the VM-per-user case, this kind of reusability is not feasible except at a very coarse level. The solution proposed by embodiments of the invention is to enable application-level hinting of low-level rendering requests, allowing a scheduler to perform rendering requests in a way that enables the creation of reusable results while maintaining application-level coherence. The results are “reusable” as a result of application level coherence as a result of the manner in which a media “experience” is defined. A media experience such as a portion of video, a game or other media is defined as an experience with an associated experience ID. Requests for such a media experience from client devices include the experience ID. The media experience is also defined as a collection of jobs, each with an associated job ID. Accordingly, two or more client devices requesting a given media experience at a similar time can be efficiently handled by generating results of execution of jobs and delivering the result of each job to the two or more client devices, thereby reducing the execution overhead. This application level hinting therefore comprises two parts: experience id - which is an application level identifier for a given experience, so that jobs from experiences of the same id can be scheduled together so that two or more clients may enjoy the same experience with the same experience id thereby enabling application level coherence; and job id such as a job hash - this uniquely identifies a particularly piece of work to be done. If work relating to the job hash has already been performed, the results relating to that job hash can be reused. This enables the creation and use of reusable results. Embodiments of the invention also include an extension that allows for even tighter application-level hinting, ensuring that certain processing occurs in a specific sequence. Examples include layering draw commands on a canvas or detecting active speakers before performing dependent tasks. Some key aspects of embodiments of the invention will now be summarised. First, is how application-level hinting is authored within media experiences, what it consists of, and how it’s passed to the server for rendering. Second, we will explore how the scheduler uses this hinting information to ensure computational coherency during execution, as well as how caching of results is handled. We have appreciated a core challenge that lies in how the scheduler handles application-level hinting of work to be done, ensuring that rendering requests are executed efficiently and effectively while maintaining coherence with the overall media experience. To illustrate the system’s functionality, we will first describe a simple abstraction and build upon it to demonstrate progressively more complex examples. In practical terms, the media experience represents computation to be done. This means that render requests are sent to a rendering service, which returns a sequence of frames (video frames) to the user’s client device for display on their screen. We will then describe how embodiments handle more complex requests for more complex results. This could involve reusing the simplified media experiences as jobs within a larger system, allowing us to nest complexity through job hierarchies. We will further describe how embodiments enable specifying Jobs with the following constraint: between this piece of work and the next, do not perform any other work on the compute resource. This is important for efficient resource utilisation, particularly in scenarios where caching is sensitive to the existence of various hardware components (CPU, GPU, IO subsystems, etc.). The combination of the above discussed application-level hinting, dependency trees, and application-aware scheduling to avoid cache busting can be viewed as application-aware co-operative scheduling of compute experiences. This approach is novel and distinct from arrangements discussed above in the background section. In summary, embodiments of the invention will be discussed which involve one or more of the following features: 1. A discovery / catalogue service, similar to a VOD (Video-on-Demand) service, allows users to browse and select media experiences. 2. The user starts their client, navigates the catalogue, and chooses an experience to view. 3. The client connects to the server using its remote access client and sends jobs to be executed. 4. These jobs contain application-level hinting information, enabling low-level scheduling of jobs that takes into account the specific application requirements. 5. The server schedules the jobs on its compute resource (e.g., GPU) and executes them. 6. The results from the job processor or cache are obtained and streamed to the client. 7. The client decodes the video stream for playback. As an example, a simplest media-oriented context for the invention may consist of: 1. Users who want to enjoy Experiences 2. A Server that provides rendering services 3. A User client that interacts with the system and displays the results 4. A Server that offers a Job Processor (GPUMgr), which is a GPU, performing rendering computations 5. A mechanism for representing work as Jobs - rendering commands (GPU shaders) - along with application-level metadata 6. A mechanism for representing media Experiences as a dependency tree of jobs 7. A cache on the server that maintains job results, eliminating the need to re-perform jobs unless necessary For such media, an embodiment of the invention’s simplest operation may be summarised as follows: 1. The user starts their client, which selects an Experience. 2. The client sends the next shader Job(s) to the Server. 3. The Scheduler sends these Jobs to the GPUMgr (Job Processor), if necessary, or jumps to 5. 4. The GPUMgr presents commands to the GPU and generates frames (which get cached). 5. The resulting or appropriate frame is sent back to the client. 6. The client renders these frames for the user. 7. The client repeats from step 2 until the Experience is finished or stopped. In this simplest context: • Rendering commands in Jobs consist of vertex shaders, fragment shaders, and associated metadata. • Frames generated by the GPU are raw images frames. • These frames are converted to JPEG images to be streamed to users (ala MJPEG). • Clients receive these images, decode them, and display them. • Experiences are represented as a tree, where each node represents a Job, and the tree represents a dependency ordering. • Experiences are experienced through a tree walk - either top-down breadth-first or bottom-up breadth-first. (Non-simple forms could use other traversals.) • Experiences are created randomly. • Jobs are created randomly but in a constrained manner to ensure only 216 possible jobs exist. The simple example above demonstrates the key aspects of embodiments of the invention. Some key points are noted as follows: • Real-world Jobs would be more complex, but this is sufficient for the simplest representative example. • The dependency tree ensures correct Job ordering. • Job results are shared and shareable across multiple users enjoying the same experience or even across experiences. (For example, logos, video stings, and on-screen “dogs” are common in media across multiple stories - which maps to Experiences in our case.) • Caching of Job results and use of caches achieve sublinear scaling. We will now describe the above features of embodiments of the invention in terms of each of the key components. Client Application The client in an embodiment is an application that users can start on their device. This application connects to a catalogue of experiences, allowing users to browse and select what they want to view. In a full-scale system, clients may be able to process some or all of the jobs that make up an experience natively. However, for the purposes of this description, we assume that clients are compute-poor devices that are unable to execute the rendering commands Gobs) that comprise an experience. In the simplest representation of the embodiment of the invention, the client is modeled as an Actor, following the Actor Model. This concurrent object in the system is capable of sending and receiving messages. The Client Actor represents a user enjoying an experience. It has an associated experience, which can be configured to render frames top-down or bottom-up. In this context, the Client Actor sends requests to the Job Scheduler, asking it to render frames based on Open GL shaders, while also providing information about its current experience. In response, the Job Scheduler returns the rendered frames, encoded as JPEGs (suitable for network transfer and comparable to a simplified i-frame-only version of video, similar to MJPEG). Media Experiences And Dependency Trees In this embodiment, media experiences are composed of a tree-like structure of jobs, representing complex dependencies for rendering (or branched experiences). More complex media experiences would encompass stories, interactive stories, or other interactive experiences. To represent these experiences and their dependencies, we use Xnodes (experience nodes), which express dependencies between shader nodes. This approach allows us to generate arbitrary numbers of random experiences. The dependency tree structure enables us to capture the concept of dependencies between shader nodes and similar within the graph. Experience nodes come equipped with built-in traversals that match two forms of topological sorting: 1. Top-down topological sorting: This captures the idea of “in order to play out or render these things, you have to then also play out or render these other things in this order.” For example, consider a scene where you first render the background, then add hills, followed by houses and space for trees. On each house, you add multiple windows, and in the forest, you do trunks and then add leaves, matching a top-down breadth-first search. 2. Bottom-up breadth-first traversal: This is also supported for experiences or usecases where this makes sense. Randomised trees / experiences are also supported in this simplest example. The ordering for traversal is defined at the root node of the tree. More complex experiences can be categorised under one or both of these dependency trees or relative to interaction within the tree. Jobs And Job Representation In this embodiment, Jobs consist of two basic shaders (vertex and fragment) along with metadata. Each Job’s result is a single video frame or image, which forms part of a larger media experience sequence. A Job in this context is represented as follows: • Job Hash: A unique identifier for the job, generated based on factors like colour involved. This can be extended to more complex hashes (e.g., MD5) considering shaders and resources used. • Experience ID: Captured from the root node of the dependency tree representing the experience. • Client Reference: The client that this Job belongs to. • Complexity Metric: Derived from the colour of the fragment shader, but ultimately depends on the application context. • Budget: In this example, it’s a time budget based on a target job rate. This budget is translated as a cost inside the scheduler, allowing for application-level control over scheduling. A Job also includes: • Vertex Shader and Fragment Shader, which can be more complex in practice. For simplicity, they are represented as: - Vertex shader: void main(){ gl_Position = vec4(0.0, 0.0, 0.0, 1.0);} - Fragment shader: out vec4 fragColor; void main(){ fragColor = vec4(1.0, 0.0, 0.0, 1.0);} Note: This is, as stated, a simple example. More complex examples would have jobs containing many more shaders of higher complexity for rendering more complex frames. However, the principle would be exactly the same. Server Endpoint And Job Processing The server provides an endpoint for clients to connect and send jobs to. This endpoint accepts job requests from clients, which are then passed to the Job Processor. When a job is presented to the Job Processor, it performs the necessary computations on its compute resource (in this example, a GPU). The resulting imagery is stored in a cache, allowing later jobs with cached results to be served from the cache without needing to recompute them. Before scheduling a job, the cache is checked to see if the required imagery is already available. Additionally, some time after receiving a job request, the cache is again checked to see if another client has sent a compatible job that was processed between the receipt of this job and its execution. This check occurs just before we pass the job shaders for execution. The processed imagery is returned to the client in the form of a JPEG image (similar to MJPEG), which can then be displayed by the client. Job Processor (GPUMgr) In this embodiment, the Job Processor is represented by the GPUMgr. The GPUMgr accepts jobs, including vertex and fragment shaders, and processes them as follows: • Input: (somejd, vertex_shader_code, fragment_shader_code) • Execution: Sends the shaders to the GPU for execution • Output: A stream of JPEG image strings, in the form (somejd, image_data) The output result is sent back to the scheduler, along with a reference to the original job. This allows the scheduler to send the result back to the original client. As noted, the result generated for a job is fundamentally an image, which is defined by the job_hash. The system checks the cache for an image with the given job_hash: • Upon receiving the job, it checks the cache before scheduling the job (if necessary). • Before presenting the job’s shaders to the GPU for execution, it checks the cache again. Scheduler Algorithm And Differences From “Standard” Scheduling Before describing the scheduler algorithm used in the embodiment, we will consider what a traditional job server and scheduler would look like. In a typical scenario, a job scheduler would: • Accept jobs that require computation • Store incoming jobs as they are received • Assign a cost metric to each job (even if all jobs have the same cost) • Maintain a priority queue based on this cost metric • Use a heap data structure, or similar, to efficiently manage the order of “next job” to be performed, with new jobs added and the next job to be done at the top of the heap However, this approach has a significant limitation: it assumes that there are no direct relationships between the jobs being added to the scheduler. This leads to issues when experiences are dependent on each other, as the scheduler may prioritise jobs based solely on their individual cost metrics, causing experiences to be experienced out of order. The embodiment of the invention appreciates the above problem and the unique approach taken by the embodiment of the invention can be distinguished, which we discuss next. In particular, we stress that the scheduling algorithm is not a generic process scheduler of the type that typically simply switches CPU usage between long running activities for which there is no reuse opportunity. Instead, the experiences herein are not like processes - running the same process twice does not imply any opportunity of reuse of results. Experiences which reuse the same commands provide opportunity for reuse since performing the same job twice is the same as performing it once and reusing the result twice. Scheduler Algorithm And Experience Queue The Scheduler is another Actor that receives messages of work to be done from clients; sends individual jobs to the the GPUMgr (Job Processor), and sends results back to clients. When a job is received, the scheduler checks if the job result - identified by job_hash- is already cached and returns it if necessary. If not, it schedules the job until there is resource available in Job Processor (GPUMgr). To maintain ordering within an experience, the scheduler uses an Priority Queue (heap) of ExperienceQueues instead of storing jobs directly in a single priority queue (heap). The ExperienceQueue represents the jobs pending rendering for a specific experience. This ensures that jobs are processed in order for each experience. Each ExperienceQueue may / will have jobs from multiple clients for the same experience. The fact that each experience queue comprises jobs from different clients for the same experience is an important point that assists with sub- linear scaling due to the experience coherence provided. As a simple example, if two separate client devices request the same experience at approximately the same time, the maintenance of a queue of job requests for that experience, along with a cache of intermediate results from executing jobs from that queue, allows that queue to be managed to appropriately keep jobs together for execution and for results to be provided from the cache if they are already available. Specifically, results of execution of jobs for that experience for one client, may be provided to another client. A way of operating the experience queue is as follows. The ExperienceQueue (XQ) has three key attributes: 1. Experience ID: the identifier of the experience related to the jobs 2. Queue of Jobs: the collection of jobs awaiting processing 3. Current Cost: the total cost of the queue, which allows the scheduler to schedule these queues as a whole When a job is appended to an ExperienceQueue, the scheduler updates the current cost in 0(1) time. The scheduler then rebalances its Priority Queue of Experience queues. This maintains the priority order of experience queues. NOTE: The ordering is INVERTED from normal to allow this to be used in a heapq and to maintain heap ordering in the priority queue with the highest weight first, rather than lowest. This still allows an 0(1) choice to be made when choosing which job to be done next - we take the first item from the highest priority ExperienceQueue. By appending jobs to an ExperienceQueue and maintaining a priority queue of ExperienceQueues, the scheduler ensures that work is processed in order for each experience. This property is known as computational coherence. Importantly, this approach prevents out-of-order processing of jobs, which can occur on traditional job servers due to assuming job independence. Instead, jobs are scheduled according to an application-level controlled cost, while maintaining a priority queue of ExperienceQueues. This unique combination of caching and serving results alongside application-level hinting and control makes this scheduler distinct from others. The ability to maintain computational coherence in the work managed by the scheduler for composite work is particularly useful. The scheduling algorithm may be further extended to include “starvation” of one experience as a result of multiple requests for another experience by a variety of techniques known to the skilled person. Data Flow Through The Simplest Example The following section provides a detailed description of the data flow through each component in the system, using the Actor Model. In this model, Actors are concurrent objects that can send and receive messages. Client Side Viewpoint From the client’s perspective, the process begins with the creation of a catalogue of experiences by content providers. This catalogue presents the available experiences for the client to choose from. Upon selecting an experience, the client navigates through a dependency tree of Xnodes. Each Xnode represents a job containing render instructions and metadata. The client traverses this tree one Xnode at a time, sending job requests to the Job Server for each job. The client then waits for responses in the form of JPEG images, and then displays them. Note that if the shaders were more complex the resulting frames would tell a story. From a computational perspective however, there is no difference between - it’s still shaders creating frames. Server Side Viewpoint From the server’s perspective, jobs are accepted from clients. Each received job is passed to the scheduler for later execution. At some point, the scheduler returns an image, which is then cached and sent back to the client as a JPEG. Job Processor / GPU Perspective The Job Processor (GPUMgr) accepts jobs from the scheduler when the GPU becomes available. The GPUMgr then sends the job, or shader, to the GPU for processing. Once the work is complete, the GPUMgr sends an image representing the current state back to the scheduler, along with a reference to the original job. Scheduler Perspective Message Processing: The scheduler receives messages representing Jobs from clients. It checks if the job has been performed before by examining the job hash. If it has, the scheduler retrieves the corresponding frame and returns it to the client as a JPEG image. Scheduling Work: If the job hasn’t been performed previously, the scheduler schedules the work. This involves finding the experience queue associated with the experience ID stored in the job. The job is then appended to that experience queue. If no experience queue exists, one is created. Either way, the cost property of the experience queue is updated. The scheduler also recalculates and restores the priority queue’s heap property. Ongoing Operations: The scheduler continuously loops to process work in experience queues. It does this by repeatedly checking the priority queue for the next job to be processed. This involves popping off a job from the current experience queue at the top of the heap; removing the first job from the experience queue; and then pushing the experience queue back onto the heap. The scheduler then uses the job’s hash to check if it has been done before. If so, it simply returns to the client the cached result as a JPEG image. If not, the scheduler sends the job to the GPU for execution. When a response is received, the scheduler caches the JPEG / frame result and also sends the resulting image back to the client. Core Extensions Extending the System to have “hashed client state” We have appreciated the embodiment may take advantage of a “hashed client state” which means that in some cases, there may be multiple clients may be actually enjoying the same experience - whether they know it or not. Specifically enjoying the same experience at the same time. In a traditional service this might be akin to watching a live video stream, or engaging in a “watch” party where multiple people are choosing watch the same programme together from different locations. As a result, aside from client end points, the clients are no longer viewed as unique. In this example, it recognises that some forms of job processor may require some form of constant state for clients that connect. However, in order to determine if two clients can receive the same data, we don’t actually store data on a per client basis. We store data a “client group” basis. So if you have 3 clients who make a request for the same video resource - as determined by their sequence of job hashes being the same, then we can treat all 3 users the same and share the directly back to the client. Extending Results to Support Multiple Videos A straightforward extension is to replace shader-based jobs with multiplevideo compositing. This enables server-side composition for clients that cannot perform this operation themselves, such as rendering two videos side by side (e.g., an episode of ’’Professor What” with British Sign Language signing overlaid). To achieve this: • Modify the Job format to describe multiple video sources and a single composition / layout - rather than shaders • The Job Processor - which previously was the GPUMgr - is a “streaming compositor” in this context • The Client remains the same • The Scheduler logic remains the same • The results transport is modified with respect to the form the result take. The results can take one of three forms: • A complete video with the two videos composited side by side • A reference to a sequence of dash fragments for client-side playback • A sequence of frames, similar to the original approach In each case, the results are cached as before. In this case the cost function could change to be related to the length of the video Media Generation with Command Lists In the domain of media generation, Jobs can take the form of command lists - simple or complex sequences of instructions that dictate how to generate specific media assets. These command lists, which could be represented as S-expressions, are used by the Job Processor (Command List Processor) to create single images, video segments, or other media formats for clients to playback. Experiences, identified by Experience IDs, contain multiple Jobs or command lists that must be processed and rendered in a specific order. The Scheduler is responsible for scheduling these Jobs based on their dependencies and priorities, ensuring that the correct sequence of commands is executed to generate the desired media assets. For example, an Experience ID “ROBOT_ATTACK” might contain two Jobs: one command list that instructs the Job Processor to draw a Robot on screen, and another command list that loads a video segment of a Robot attack. The Scheduler would schedule these Jobs in the correct order, taking into account their dependencies and priorities. Obviously there would be more jobs for more scenes depending on what the content creator wants the experience to be. The Job Processor then processes each Job by executing the corresponding command list, creating the necessary media segments along the way. The Cache stores the results of processing each Job, using a Job hash to determine which Jobs are equivalent and can be reused. This enables efficient reuse and reduction of redundant work, allowing multiple clients requesting the same Experience ID to quickly retrieve the necessary media assets from the Cache instead of re-processing the entire Experience. This approach simplifies the process of creating complex media experiences by breaking them down into smaller, manageable Jobs or command lists. It also allows for efficient caching and reuse of processed Jobs, reducing the need for redundant processing and improving overall system performance. Extending Results to Support Branching Videos Building on the previous extension, this concept takes the dependency tree traversal to the next level by treating it as an interactive experience controlled by the audience member. This is reminiscent of choose-your-own-story-style narratives where the viewer’s choices determine the direction of the story. In this scenario, imagine a Professor What episode where you, the audience member, get to choose the perspective through which the story unfolds. You could select options like “follow the Professor” or “explore the alien world,” and the video would adapt accordingly, presenting different scenes, characters, or plot twists based on your decisions. To achieve this: • Modify the Job format to include branching logic, allowing for multiple possible paths through the dependency tree. • Update the Job Processor to accommodate these new narrative possibilities, possibly by incorporating Al-driven decision-making or user input. • The cost function may need to be revised to account for the increased complexity and variability of these branching videos. Extending the System to Use Different Compute Resources While our primary focus has been on using GPUs for compute-intensive tasks like shader rendering, there’s no inherent limitation that prevents us from applying this approach to other types of compute resources. In fact, this framework can be easily adapted to leverage various compute architectures and technologies. For instance: • Instead of shaders, the Job could represent Cuda Kernels processing Al workloads on NVIDIA CUDA-enabled GPUs, (this is a fairly obvious context in which to use our scheduler) • The Job Server could manage a pool of CPUs for tasks like machine learning or data processing, allowing clients to request and receive results from CPU-based computations. • Even cloud-based services like Google Cloud’s Tensor Processing Units (TPUs) or Amazon Web Services’ SageMaker could be integrated as compute resources, enabling the system to scale and adapt to changing demands. However, the key point here is that this framework operates based on application level experience coherance, ensuring coherent compute scheduling across different compute resources. This means that the Job Server can: • Schedule Jobs on multiple compute resources simultaneously, taking into account their unique characteristics and strengths • Dynamically adjust compute resource allocation based on changing workload demands and availability • Ensure that compute-intensive tasks are properly coordinated and optimised relative to a given experience defined at application level, rather than just focusing on individual jobs By abstracting the compute resource away from the Job itself and incorporating application-level scheduling, we could create a scalable and adaptable system using this approach. This enables us to: • Scale compute resources dynamically based on demand • Migrate workloads between different compute architectures as needed • Leverage the strengths of each compute resource type for specific tasks while ensuring coherent compute scheduling across multiple compute resources, optimising overall application performance Keeping Jobs Together A further particular benefit of the embodiment builds upon the previous simple example by enabling the concept of keeping specific Jobs together, without interruption. This may be necessary for certain experiences where clearing the compute’s cache (e.g., GPU Cache) would be a particularly expensive task. Examples of such tasks include: • Jobs that reuse expensive textures just loaded into the GPU by one task and then use them immediately in subsequent tasks • Al tasks that run on the GPU, resulting in a model being loaded or trained in one Job and then used in the next Job Note that these examples assume either a different style of Job (and different Job processor), different compute resource, or different type of GPU usage (Cuda kernels rather than vertex / fragment shaders). Despite these differences, the approach taken by the scheduler when dealing with these Jobs can be precisely the same: check the cache for results, put the Jobs onto experience queues, schedule experience queues, and so on. This “Keep Together” concept involves ensuring that two or more jobs are always executed together using this system. We have appreciated that there are two possible approaches to achieve this: 1. Modify Job Type and GPUMgr One approach is to modify the job type and extend the GPUMgr to bundle jobs together, allowing them to be executed as a single unit. This specifically modifies the jobs to be bundled, and modifying the GPUMgr to unbundle jobs to execute them together. This would require no changes to the scheduler itself, as shader objects are already computational objects from a scheduling perspective. 2. “Keep Together Aware” Jobs and Scheduler The other approach involves modifying both the jobs and the scheduler to be aware of the need to keep certain jobs together. This is the approach taken by the second main embodiment. A detailed description of the necessary modifications follows. Extending the Job Concept The concept of a Job is extended to include two flags: keep_with_prev and keep_with_next. Both are necessary for ensuring that jobs that need to be kept together are executed correctly. • keep_with_prev is used to ensure that jobs are executed in the correct order within the scheduler. • keep_with_next is used to inform the scheduler that the current job has a follow-on Job that must be executed next. These sound similar, but are subtly different. In the embodiment we may extend Experience nodes (Xnodes) and author them as before, but Jobs now include the concept of keep_with_prev. This means that “this Job expects the previous Job to have executed on the GPU last”. Note that keep with next can be calculated dynamically and added to the correct nodes to avoid inconsistencies. Updating Experience Nodes The interpretation of Experience Nodes (XNodes) may be extended to keep track of the “last” node when iterating through an experience. If the current node has the keep_with_prev flag set, the “last” node’s keep_with_next flag is updated accordingly. When a Client starts, it triggers this automated setting of the keep_with_next flag for the experience nodes. Updating the Experience Queue The ExperienceQueue is extended to insert any new Jobs into the correct location of the queue if the keep_with_prev flag is set. This can be either immediately following the Job that should follow, or at the end of the queue. The job it should follow is the one where the Client is the same in both Jobs. If the keep_with_prev flag is set, the cache is NOT checked before appending the Job to the experience queue - only when taken from the queue. This is to ensure that keep_with_prev jobs are executed in the correct order. The Job and queue are then scheduled as normal. Updating the Scheduler The scheduler is extended to check Jobs for a keep_with_next flag. If it’s set, the scheduler sets a flag “keepFlag” so that it will recheck the same experience queue next time round for a follow-on job. If found, the new Job is removed and the Priority Queue ordering is updated. In the case of the second major embodiment, this means that the heap representing the priority queue is “re-heapified” as per the standard heap algorithm. Detailed Embodiment One particular embodiment of the embodiments discussed above will now be described with reference to the figures. Figure 1 provides a system overview of an arrangement that may embody the invention and is applicable to a range of use cases. The arrangement includes one or more clients 10, a job scheduler 12 and a processor here shown as a GPU 14. Whilst a single client, a single job scheduler and a single GPU are shown, in reality there will be many thousands of clients, job schedulers and GPUs, but with a larger number of clients than job schedulers and GPUs due to the non-linear scaling provided by the invention. A client 10 is arranged to issue requests for media content to a server that provides an instance of job scheduler 12 and to receive responses therefrom in the form of media, shown in this example as a frame of an image. The job scheduler 12 implements a job scheduling algorithm which delivers the non-linear efficiency of the whole system. The job scheduler 12 arranges a schedule of jobs to be executed by the processor shown as the GPU 14. The GPU 14 then executes such jobs and delivers the results to the client. There are thus broadly speaking there are 3 main elements: Clients which want jobs performed which relate to experiences. A Job Scheduler which may accept jobs, which are then scheduled such that common tasks are scheduled together, based upon application level determined cost metrics. The job scheduler maintains a cache of results. A Job Processor - which can accept jobs to perform, performs them, and returns results. In the preferred embodiment, this is a GPUMgr, which accepts jobs consisting of shaders, and returns frames (images) to the client via the Job Scheduler. The scheduling itself is arranged to provide an efficient return to clients by two key approaches which work separately as well as in synergy with one another: (1) If the jobs are not new - for example as determined by an application level hash of the job - cached versions are returned to the client; and (2) If jobs are new, these are scheduled to be presented to a processor -a GPU Manager in the preferred embodiment - for processing. The results are then cached, and also passed back to the client. The use of caching helps achieve sub-linear scaling of compute work relative to audience size (number of client devices). Furthermore, by serving results back based on determining whether a job is new such as by a hash, any jobs common between experiences - such as rendering of a logo - are reused. Ordering of jobs (work / processing) required to fulfill an experience is maintained using application level metrics in the scheduler. The use of application level information also allows multiple clients to be scheduled together increasing the chances of cache hits, and cache coherency within the Job Processor - such as the GPU in the preferred embodiment. We have appreciated that the scheduling algorithm may be applicable to other fields including scheduling of processes for Al or indeed scheduling of many processes either internal to computers or external including control of external systems. However, the scheduling algorithm is particularly beneficial in the field of delivery of media at scale as will be described in relation to an embodiment. An embodiment of the invention will be described in relation to Figure 2. In this embodiment, multiple client devices 10 requests media to be executed and delivered from a GPU 14 via a job scheduler 12 as previously described. The embodiment described is the execution and provision of image frames for simplicity of discussion, but the invention is scalable to delivery of many different types of media. In addition to the above components, a frame store 16 implemented by VRAM is provided. The VRAM is at the heart of the embodiment in which shared assets are stored. The output of the GPU is provided both to requesting clients and to the VRAM such that subsequent requests for the same GPU output may be provided from the VRAM, rather than by execution again in the GPU. Take the case of a million people wanting to watch the same video but with a different overlay. If we can avoid separately rendering and downloading that video but instead render once and store the result we have made a significant saving in computation and bandwidth. This requires some coherence in the rendering while still allowing some personalisation. The saving of intermediate results in the VRAM allows such coherence. The shared assets may be shared whilst allowing separate output. The VRAM may be terabytes and have large caches able to provide meshes, shaders, and rendered images. The intermediate store in the VRAM of subproducts is what provides many of the advantages. Low latency in real time and reusing results may be provided by the fact that a single operating system process is managing request from multiple clients. The intermediate products may be any results along the way e.g. background versus foreground. Returning to Figure 2, in this example a number of clients are started (each client being a separate client device or potentially a different instance of a client application running on the same device). They each have an experience (sequence of jobs) that they wish to have running. These clients connect to the Job Scheduler, and start sending jobs to the Job Scheduler. The concept of an experience comprising a sequence of jobs will be explored more later, but in broad terms an experience may be the receipt of an individual image frame, a sequence of video, an entire television programme or game. The level of granularity depends upon the use case. In this particular embodiment, the experience will be described as a single frame. The Job Scheduler 12 checks to see if it already has the result in its cache (in this case a frame store 16). If it does, it returns the frame to the client. If it doesn’t it schedules the job to be performed. Just before it presents jobs to the Job Processor, it checks the cache again. This is because the results of multiple scheduled pieces of similar work may be able to share the same results. This is especially possibly given that we schedule work common to application experiences (sets of work) together. If the result isn’t already in the cache (frame store 16 in this instance), the job is presented to the Job Processor (GPU Manager 14 in this instance) for the work to be done. The results are cached and returned to the client. The jobs, consist of a number of key piece of information in the preferred embodiment: A hash of the job. Since this is determined by the creators of the experience, this is an application level hash. It is this application level hash that is checked by the Job Scheduler for compatibility, shareability of results between clients and experiences. In the case of shaders this could be either an id for the shader or a hash of the normalised shader source. In the preferred embodiment, this is derived from the shader source, based on the specific changing aspects within the shaders. The term “hash” is used to describe this identifier as a hash function is a computationally efficient way of uniquely identifying a job from some aspect of the job, but more generally the hash may be a separate identifier from which the job may be identified. The budget I required cost for a job. This, again, is determined by the creators of the experience. As a result, this cost is an application level cost, and could refer to real world costs, etc. In the context of the preferred embodiment this is taken as a time budget / cost - that is it is a cost representing the expected time that the job will taken to perform. In some implementations of this system, it is reasonable to conceive that a cost / budget based on metrics of the job could be created within the job processor, and then scheduled as per the rest of the system. For this, the calculate_cost function would be implemented. This makes sense where the jobs are automated work (such as the compiler example), rather than GPU jobs relating to object based media. The job - the work to be done - itself. In the preferred embodiment this is shaders to be executed by the GPU manager. However if the Job Processor is a command list executor, the jobs could be command lists, or commands in a command list. If the jobs were programs to be compiled the job processor would naturally be a compiler. (For example the back-end service of a compiler system / IDE for remote users where common code is used by many users) A reference to the specific client So that the results of the job can be returned to the client using the client reference. An experience ID The experience ID will be used in conjunction with the other metadata to schedule jobs based upon experience popularity / cost and to maintain ordering of jobs (work) within an experience. Figure 3 is a functional diagram of the key aspects implemented by a scheduling algorithm. As shown, experiences A, B and C may be requested by respective clients 1, 2 and 3 those experiences comprising multiple jobs with dependencies shown by the graph (and described in more detail later). As previous he noted, the experiences may be entire sections of video, games or other media, but in this example we will discuss the provision of a image frame. The clients are arranged to request the respective experiences from a job scheduler which initiates a set of scheduling queues shown as “experience queues”. The maintenance of those cues will be described below. The job processor is shown separately and is preferably a GPU as previously described which has access to a cache, preferably a VRAM as previously described. The GPU performs jobs, the VRAM caches job results based upon a job hash and the job scheduler takes requests from clients and fulfills them. A given client sends jobs with the experience id + cost to the scheduler. The scheduler uses the cost, experience id, hash and work to be done to: (1) Make optimal use of the VRAM cache based on "have I done this", (2) Ensure ordering of work within experience for a given client remains in order, (3) Choose which experiences to prioritise based on workload - cumulative cost of tasks to be done, (4) Choose next actual work to based on experience priorities. The scheduling algorithm therefore operates on jobs with the following characteristics: • (hash, budget (cost), work to be done, client, experience id) The Job Scheduler maintains an Experience Queue for each experience id. An Experience Queue: • Is a queue • Has an associated total cost. • Relates to a specific experience id. When a job is added to a given Experience Queue the following takes place: • The job is appended to the end of this queue • The cost of this Experience Queue is increased by the budget (cost) of this job When a job is removed from a given Experience Queue the following takes place: The job is removed from the head of the queue • the cost of this Experience Queue is decreased by the budget (cost) of this job Since Experience Queues themselves have a cost, this means the Experience Queues can be scheduled using a priority queues approach. Specifically, this means that the Experience Queues are stored within a priority queue using the cost of the queue to determine priority. In the preferred embodiment, this priority queue is a heap data structure, and orders based upon the experience queue cost, with highest cost items at the top of the heap. A heap is a partially ordered list of values, which maintains that partial ordering such that identification of the “next” value is an 0(1) operation. Adding / removing values from the heap is an O(log N) operation. In this case rather than N being relative to the number of clients or jobs, it is relative to the number of experiences. Therefore maintaining this heap is an efficient operation, close to 0(1) efficiency for choosing the next piece of work to do. So the work of the Job Scheduler is summarised as follows: • To accept jobs to perform • To determine if the work has already been done • To schedule to work to be done if it hasn’t • To repeatedly choose the next piece of work to be done. Given the description above, this means: • Accepting jobs from an inbound queue of work from clients • Schedule the work: - To add those jobs to the Experience Queue for experience id represented inside the job - To update the cost for the Experience Queue - To re-balance (re-heapify) the Priority Queue based on the updated cost of this Experience Queue • To repeatedly choose the next piece of work: - This looks for the Experience Queue with the highest current cost / budget. This is an application level cost, and represents both cost of the experience and the popularity of this experience. - Identification of this Experience Queue is an 0(1) operation. - Removal of the Job from the Queue is also an 0(1) operation - Rebalance of the Priority Queue is an O(Log N) operation based on the experiences running. The scheduler: • Does not care about what the experiences are, nor how they are implemented. The preferred embodiment is scheduling GPU shader jobs, but the scheduler does not delve into these details. It simply passes the jobs to the job processor. This means the experiences (traversals of trees of dependent sets of work) could be any kind of processing that could benefit from this kind of scheduling. • Does not need to do a cost calculation to schedule the work, since this is done upfront for the application level experiences. A further embodiment of this system could choose to implement a cost metric based upon the job or a variety of other metrics, instead of deferring this to application level. This deferral to the application level, again increases the ability to use this scheduler with a variety of co-operative tasks performing processing leading towards common results. • Does not perform job equivalence analysis for the basis of reusing results, deferring this to an application level hash. It is clear that specialised versions of this system could however do this inside the Job Scheduler where there is a clear reason to do so. • Is therefore not specific to any particular kind of workload, though it is designed in the context of GPU shader workloads. This is despite the scheduler being able to be made sufficiently application level aware for cooperative workloads. In summary of the overall process implemented by the scheduling algorithm, the following aspects are provided. • Experiences exist which consist of jobs that comprise dependencies with associated costs of each job, identified by id • Clients walk through these sending jobs to the job scheduler to get frames • The Scheduler stores jobs for later work based on: o Do I already have this result in the VRAM / Cache based on job hash (early check) o If not pop in experience queue related to that specific experience id o Cost of experience queue depends on total cost of all jobs in queue • Scheduler chooses next job to work on o Treats the experience queues as a priority queue - picking the "highest cost" as priority o Takes a job from the queue to perform o Checks the VRAM / Cache to see if the work has been done • If it has, just return that result now (late check) o If not, sends the job to the GPU o Store result in VRAM / cache by job hash o Returns result to user. So the scheduler schedules jobs co-operatively with regard to experiences, based on application level hashes and costs, while maintaining sequencing of job dependencies within experiences. The scheduler does not need to understand the hash creation, the dependencies nor costs - by maintaining job ordering within experiences. The embodiment of the invention thus provides the injection of application / experience level intelligence into the scheduler to enable cooperative scheduling rather than competitive. The following section walks through an example of how the experience queue costs might change given some example jobs from specific clients. Figure 4 shows an initial scheduling state of the series of queue of jobs in the scheduler. In this diagram, you can see there are 3 clients (1,2,3), which are running 2 different experiences (A, C), each of which has a collection of jobs associated with them. The system at this instant has 3 Experience Queues, one each for experience (A,B,C). There are no clients connected experiencing B, so B’s queue is empty with a budget / cost of 0. The budgets / costs associated with the other two queues is A: 20, C: 22. That means the next Experience Queue to have work taken off / performed is queue C. That would mean, if the work was performed, job d for client 3 would be executed next. Figure 5 shows a subsequent scheduling state in which extra work has been added by clients 1 and 2. These increases the cost / budget of Experience Queue A to 79 - which is well above that of Experience Queue C. This means that any work to be done will be based upon Experience Queue A, until it drops below the cost of Experience C. Note that the ordering jobs (work to be done) for each of the two clients relating to their experience ids is maintained, ie work within an experience is not performed out of order. Figure 6 shows a subsequent scheduling state in which client 3 sends through more jobs towards the completion of Experience C. This does increase the cost / budget of Experience Queue C to 31, but this isn’t higher than 79, so Experience C remains lower priority than Experience A at this point. Figure 7 shows the effects of clients 1 and 2 adding in extra work to the queues. From this it is even clearer that clients Experiencing A will progress towards completion, due to the much higher costs of Experience A. It should be clear from this that experiences that need more work done will get more job processor time. Furthermore, since ordering is maintained within experiences, there is a high likelihood of serving content from the cache since scheduling work from within experiences, means that common results will be cached. Under real world overload circumstances, there would need to be minor extensions to this core algorithm, in order to protect quality of experience. But the ability to modify the cost metric, simplifies this task. For example, in the even that unpopular and / or cheap experiences may get starved out of CPU time due to an experience queue never moving forward we can easily modify the core scheduling cost function to deal with this, due to the extensible nature of the scheduler. For example, we could use an additional metric of “time since last activity”, or frequency of throughput as a modifier for any given queue. Using this as a multiplier (for example) when it comes to rebalancing the priority queue would result in processing blocked experiences. Similarly, rather than a simple heap, the costs of experience queues could be used to control a stochastic mechanism that pulls from all the queues in a weighted fashion, thereby progressing all experiences, but preserving priorities. A trivial extension of this is for the Job Scheduler to be provided with a list of Job Processors (eg multiple GPUs) it can send work to, and for it to create an affinity between Experience Queues and Job Processors. This would further boost cache coherence within the Job Processors, and reduce risks of starvation noted above. Figures 8, 9 and 10 show respectively the logical arrangement by which an experience may comprise a number of jobs, the arrangements of those jobs are the top-down dependency or the arrangement of these jobs with a bottom-up dependency. Each client (which in the preferred embodiment relates to a user driven client) essentially “runs” an experience, causing jobs to be scheduled, and the result of the work presented to the user. In the preferred embodiment, this is a sequence of images. Experiences are pre-authored by the service provider, and may be unique to the client or shared among many clients. (Such as a menu or bank of experiences to be chosen from.) Client experiences are represented as a tree structure as per figure The nodes in the tree are referred to here as XNodes (short for “Experience Nodes”). Each XNode: • Has work to be done associated with the node. This work is referred to as a “job”, and in the preferred embodiment jobs are shaders - such as vertex shaders and fragment shaders. • It has 2 child nodes - left and right. The structure is authored to be specific for the experience. For testing purposes these trees of XNodes can be created automatically. The nodes in the tree form a simple binary tree where parent nodes have with left and right child nodes. These are ordered to represent dependencies in a given experience. There are two key possible interpretations of such nodes: a) that the parent node is dependent upon the completion of the work in the child nodes, or b) that the child nodes are dependent upon work from the parent node being completed first. The Xnodes do not impose either interpretation. In order to express whether the parents are dependent upon children, or whether the children are dependent upon parents is represented by a tree traversal as shown in figure 9. Top down, breadth first traversal of the experience tree represents / is used for experiences / work that requires the results of parent nodes to be performed before the child nodes. For some kinds of scene - such as some kinds of 2D and 3D scene rendering this ability for parent nodes to be rendered first really matters. By way of illustration consider a simple painter experience - you put a base colour then a base for the sky and one for the land. On the land you then perhaps do a base for trees and a house. Then individual trees and windows added to the house. And so on. Shaders can be used in a not too dissimilar fashion. Bottom up, breadth first traversal of the experience tree represents / is used for experiences / work that requires the results of child nodes to be performed before the parent nodes. This traversal is represented in figure 10. 5 While the preferred embodiment is GPU shader jobs, if the jobs to be provided to a “Compiler” Job Processor here were programs to compile and link, you would need to use this kind of bottom up traversal for the dependencies to be satisfied. 10

Claims

1. A server arrangement for delivering media, comprising:- a processor arranged to receive requests for media from multiple client devices across a network and to operate an instance of an operating system process that implements a scheduling algorithm;- a job processor arranged to execute jobs under control of the scheduling algorithm operated by the processor;- a cache arranged to store intermediate results of jobs executed by the job processor;- an encoder arranged to receive the intermediate results from the cache and to encode the intermediate results as encoded media and to deliver the encoded media to multiple client devices;- wherein the scheduling algorithm is arranged to schedule jobs such that at least one intermediate result required for encoded media by multiple client devices may be executed once and delivered to multiple client devices in encoded media.

2. The server arrangement of claim 1, wherein the scheduling algorithm is arranged to check the cache on receipt of a job and to cause the encoder to retrieve intermediate results from the cache, and the job processor to not execute that job, if intermediate results for that job are already available in the cache.

3. The server arrangement of claim 1 or 2, wherein the scheduling algorithm is arranged to maintain a queue of jobs to be executed and to add a job to the queue for a received request for media if intermediate results for that job are not available in the cache.

4. The server arrangement of claim 3, wherein the scheduling algorithm is arranged to retrieve the next job from the queue in turn, to check if intermediate results are available for that job in the cache, and to cause the encoder to retrieve intermediate results from the cache, and the job processor to not execute that job, if intermediate results for that job are already available in the cache.

5. The server arrangement according to claim 3 or 4, wherein each job includes an experience ID related to the requested media and the scheduling algorithm maintains a respective queue of jobs for each respective experience ID.

6. The server arrangement according to claim 5, wherein the scheduling algorithm maintains a priority order for selecting from each respective queue of jobs, each respective queue of jobs having an associated total cost of jobs in the queue, the scheduling algorithm arranged to prioritise selection of jobs from the queue with the highest cost.

7. The server arrangement of claim 6, wherein each job has an associated cost metric and the total cost of jobs in the queue is a sum of the cost metrics of jobs in the queue.

8. The server arrangement of any of claims 5, 6 or 7, wherein each experience ID corresponds to a respective media experience and wherein each respective queue of jobs for each respective experience ID comprises jobs requested by multiple client devices.

9. The server arrangement of any of claims 3 to 7, wherein each queue of jobs is arranged in order of receipt of each job, and the scheduling algorithm is arranged to select the job to process next in order of receipt.

10. The server arrangement of any of claims 8, wherein at least one job in the queue includes an indicator that a specified subsequent job in the queue is to be executed upon completion of that at least one job, and the scheduling algorithm is arranged to cause the job processor to execute the specified subsequent job upon completion of the at least one job.

11. The server arrangement of any of claims 8, wherein at least one job includes an indicator that the at least one should be executed upon completion of a specified previous job, and the scheduling algorithm is arranged to add the at least one job to the queue immediately after the specified previous job.

12. The server arrangement of any preceding claim, wherein each job includes an associated job identifier.

13. The server arrangement of claim 12, wherein the job identifier comprises a hash of the job.

14. The server arrangement of claim 12 or 13, wherein the scheduling algorithm is arranged to determine whether or not to execute a job using the job identifier to determine whether intermediate results for the job are already available in the cache.

15. The server arrangement of any preceding claim, wherein media comprises video and the intermediate result comprises an image frame.

16. The server arrangement of any preceding claim, wherein the media is represented as a graph of jobs comprising an experience, the graph is accessible by client devices, and requests for media for the experience from client devices are received based on the graph.

17. The server arrangement of claim 16, wherein the graph is a tree.

18. The server arrangement of claim 16 or 17, wherein the tree is provided fromthe server arrangement to the client devices.

19. The server arrangement of any of claims 16, 17 or 18, wherein the requests for media are received according to a client side scheduler operated by client devices, the client side scheduler arranged to issue requests for execution of jobs by traversing the graph of jobs for the experience.

20. The server arrangement of any preceding claim, wherein each job comprises instructions for producing media.

21. The server arrangement of claim 20, wherein the instructions for producing media comprise one of instructions for image rendering, shader definitions for aGPU, CUDA kernls for a GPU or rendering definitions suitable for a software renderer executing on a CPU.

22. The server arrangement ofany preceding claim, wherein the processor is a CPU.

23. The server arrangement of any preceding claim, wherein the job processor is a GPU.

24. The server arrangement of of any preceding claim, wherein the cache is a VRAM.

25. A client device arranged to request media from a server arrangement of any preceding claim, the wherein the media is represented as a graph of jobs comprising an experience, the graph is accessible by the client device, and requests for media for the experience are issued by the client device based on the graph.

26. The client device of claim 25, wherein the graph is a tree.

27. The client device of claim 25 or 26, wherein the graph is retrieved by theclient device from the server arrangement.

28. The client device of any of claims 25, 26 or 27, wherein client includes a client side scheduler, the client side scheduler arranged to issue requests for execution of jobs by traversing the graph of jobs for the experience.

29. The client device of any of claims 25 to 28, wherein each job comprises instructions for producing media.

30. The client device of claim 29, wherein the instructions for producing media comprise one of instructions for image rendering, shader definitions for a GPU, CUDA kernls for a GPU or rendering definitions suitable for a software renderer executing on a CPU.

31. A system comprising the server arrangement of any of claims 1 to 24 and the client device of any of claims 25 to 30.

32. A method for delivering media, comprising:- receiving, at a processor, requests for media from multiple client devices across a network and operating an instance of an operating system process that implements a scheduling algorithm;- executing, at a job processor, jobs under control of the scheduling algorithm operated by the processor;- storing, in a cache, intermediate results of jobs executed by the job processor;- encoding, in an encoder, intermediate results from the cache as encoded media and delivering the encoded media to multiple client devices;- wherein the scheduling algorithm is arranged to schedule jobs such that at least one intermediate result required for encoded media by multiple client devices may be executed once and delivered to multiple client devices in encoded media.

33. A method of retrieving media operable at a client device arranged to request media from a server arrangement of any of claims 1 to 24, the wherein the media is represented as a graph of jobs comprising an experience, the graph is accessible by the client device, and comprising issuing requests to the server arrangement for media for the experience from the client device based on the graph.

33. A computer program comprising code which when executed undertakes the method of claim 32 or 33.AMENDMENTS TO THE CLAIMS HAVE BEEN FILED AS FOLLOWS:20 01 25CLAIMS1. A server arrangement for delivering media, comprising:5 - a processor arranged to receive requests for media from multiple clientdevices across a network and to operate an instance of an operating system process that implements a scheduling algorithm;- a job processor arranged to execute jobs under control of the scheduling algorithm operated by the processor;10 - a cache arranged to store intermediate results of jobs executed by the jobprocessor;- an encoder arranged to receive the intermediate results from the cache and to encode the intermediate results as encoded media and to deliver the encoded media to multiple client devices;15 - wherein the scheduling algorithm is arranged to schedule jobs such thatat least one intermediate result required for encoded media by multiple client devices may be executed once and delivered to multiple client devices in encoded media.20 2. The server arrangement of claim 1, wherein the scheduling algorithm isarranged to check the cache on receipt of a job and to cause the encoder to retrieve intermediate results from the cache, and the job processor to not execute that job, if intermediate results for that job are already available in the cache.25 3. The server arrangement of claim 1 or 2, wherein the scheduling algorithmis arranged to maintain a queue of jobs to be executed and to add a job to the queue for a received request for media if intermediate results for that job are not available in the cache.30 4. The server arrangement of claim 3, wherein the scheduling algorithm isarranged to retrieve the next job from the queue in turn, to check if intermediate results are available for that job in the cache, and to cause the encoder to retrieve intermediate results from the cache, and the job processor to not execute that job, if intermediate results for that job are already available in the cache.20 01 255. The server arrangement according to claim 3 or 4, wherein each job includes an experience ID related to the requested media and the scheduling algorithm maintains a respective queue of jobs for each respective experience ID.

6. The server arrangement according to claim 5, wherein the scheduling algorithm maintains a priority order for selecting from each respective queue of jobs, each respective queue of jobs having an associated total cost of jobs in the queue, the scheduling algorithm arranged to prioritise selection of jobs from the queue with the highest cost.

7. The server arrangement of claim 6, wherein each job has an associated cost metric and the total cost of jobs in the queue is a sum of the cost metrics of jobs in the queue.

8. The server arrangement of any of claims 5,6 or 7, wherein each experience ID corresponds to a respective media experience and wherein each respective queue of jobs for each respective experience ID comprises jobs requested by multiple client devices.

9. The server arrangement of any of claims 3 to 7, wherein each queue of jobs is arranged in order of receipt of each job, and the scheduling algorithm is arranged to select the job to process next in order of receipt.

10. The server arrangement of any of claims 8, wherein at least one job in the queue includes an indicator that a specified subsequent job in the queue is to be executed upon completion of that at least one job, and the scheduling algorithm is arranged to cause the job processor to execute the specified subsequent job upon completion of the at least one job.

11. The server arrangement of any of claims 8, wherein at least one job includes an indicator that the at least one should be executed upon completion of a specified previous job, and the scheduling algorithm is arranged to add the at least one job to the queue immediately after the specified previous job.20 01 2512. The server arrangement of any preceding claim, wherein each job includes an associated job identifier.

13. The server arrangement of claim 12, wherein the job identifier comprises a 5 hash of the job.

14. The server arrangement of claim 12 or 13, wherein the scheduling algorithm is arranged to determine whether or not to execute a job using the job identifier to determine whether intermediate results for the job are already available in the 10 cache.

15. The server arrangement of any preceding claim, wherein media comprises video and the intermediate result comprises an image frame.15 16. The server arrangement of any preceding claim, wherein the media isrepresented as a graph of jobs comprising an experience, the graph is accessible by client devices, and requests for media for the experience from client devices are received based on the graph.20 17. The server arrangement of claim 16, wherein the graph is a tree.

18. The server arrangement of claim 16 or 17, wherein the tree is provided fromthe server arrangement to the client devices.25 19. The server arrangement of any of claims 16, 17 or 18, wherein the requestsfor media are received according to a client side scheduler operated by client devices, the client side scheduler arranged to issue requests for execution of jobs by traversing the graph of jobs for the experience.30 20. The server arrangement of any preceding claim, wherein each jobcomprises instructions for producing media.

21. The server arrangement of claim 20, wherein the instructions for producing media comprise one of instructions for image rendering, shader definitions for a20 01 25GPU, CUDA kernls for a GPU or rendering definitions suitable for a software renderer executing on a CPU.

22. The server arrangement ofany preceding claim, wherein the processor is a 5 CPU.

23. The server arrangement of any preceding claim, wherein the job processor is a GPU.10 24. The server arrangement of of any preceding claim, wherein the cache is aVRAM.

25. A client device arranged to request media from a server arrangement of any preceding claim, the wherein the media is represented as a graph of jobs 15 comprising an experience, the graph is accessible by the client device, and requests for media for the experience are issued by the client device based on the graph.

26. The client device of claim 25, wherein the graph is a tree.2027. The client device of claim 25 or 26, wherein the graph is retrieved by the client device from the server arrangement.

28. The client device of any of claims 25, 26 or 27, wherein client includes a 25 client side scheduler, the client side scheduler arranged to issue requests for execution of jobs by traversing the graph of jobs for the experience.

29. The client device of any of claims 25 to 28, wherein each job comprises instructions for producing media.3030. The client device of claim 29, wherein the instructions for producing media comprise one of instructions for image rendering, shader definitions for a GPU, CUDA kernls for a GPU or rendering definitions suitable for a software renderer executing on a CPU.20 01 2531. A system comprising the server arrangement of any of claims 1 to 24 and the client device of any of claims 25 to 30.

32. A method for delivering media, comprising:5 - receiving, at a processor, requests for media from multiple client devicesacross a network and operating an instance of an operating system process that implements a scheduling algorithm;- executing, at a job processor, jobs under control of the scheduling algorithm operated by the processor;10 - storing, in a cache, intermediate results of jobs executed by the jobprocessor;- encoding, in an encoder, intermediate results from the cache as encoded media and delivering the encoded media to multiple client devices;- wherein the scheduling algorithm is arranged to schedule jobs such that 15 at least one intermediate result required for encoded media by multipleclient devices may be executed once and delivered to multiple client devices in encoded media.

33. A method of retrieving media operable at a client device arranged to 20 request media from a server arrangement of any of claims 1 to 24, the wherein the media is represented as a graph of jobs comprising an experience, the graph is accessible by the client device, and comprising issuing requests to the server arrangement for media for the experience from the client device based on the graph.2534. A computer program comprising code which when executed undertakes the method of claim 32 or 33.

Images