Multiplayer gaming system and method

Abstract

A system for managing the execution of a plurality of video game instances at a server in response to inputs from a single client device, the system comprising an input receiving unit which receives a

Description

BACKGROUND OF THE INVENTION Field of the invention This disclosure relates to a multiplayer gaming system and method. Description of the Prior Art The "background" description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present invention. With the increasing level of complexity and visual quality that are able to be provided in games, the demands upon processing hardware have increased significantly over time. For instance, more powerful processing hardware may be required to execute games, or limits may be placed upon the functionality of such games in order to enable them to be executed by a typical device. One example of this is multiplayer experiences that can only be experienced in an online or networked setting - rather than in so-called 'couch co-op' or split-screen modes within a game. While there is a desire to offer a more local experience, technical limitations of executing devices can impose a restriction. This technical limitation may be imposed by the hardware of a games console (which is largely standardised by the manufacturer), or by any other computing arrangement; for instance, content may be developed on the basis of known console capabilities or information about average or expected computing power available to eventual users. While in some cases a device may have sufficient computing power to provide a local multiplayer experience for a game (particularly in the case of a modern computing device executing an older game), many games may not be designed with this in mind. It is in the context of the above discussion that the present disclosure arises. SUMMARY OF THE INVENTION This disclosure is defined by claim 1. Further respective aspects and features of the disclosure are defined in the appended claims. It is to be understood that both the foregoing general description of the invention and the following detailed description are exemplary, but are not restrictive, of the invention. BRIEF DESCRIPTION OF THE DRAWINGS A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein: Figure 1 schematically illustrates an entertainment system; Figure 2 schematically illustrates a networked system; Figure 3 schematically illustrates a method; Figure 4 schematically illustrates a system for executing a video game; Figure 5 schematically illustrates a first processing device; Figure 6 schematically illustrates a method for executing a video game by a first processing device; Figure 7 schematically illustrates a method for interacting with remote instances of an application; Figure 8 schematically illustrates a system for managing the execution of a plurality of application instances at a server in response to inputs from a single client device; Figure 9 schematically illustrates an alternative arrangement of the units of Figure 8; and Figure 10 schematically illustrates a method for managing the execution of a plurality of application instances at a server in response to inputs from a single client device. DESCRIPTION OF THE EMBODIMENTS Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views, embodiments of the present disclosure are described. Referring to Figure 1, an example of an entertainment system 10 is a computer or console. The entertainment system 10 comprises a central processor or CPU 20. The entertainment system also comprises a graphical processing unit or GPU 30, and RAM 40. Two or more of the CPU, GPU, and RAM may be integrated as a system on a chip (SoC). Further storage may be provided by a disk 50, either as an external or internal hard drive, or as an external solid state drive, or an internal solid state drive. The entertainment device may transmit or receive data via one or more data ports 60, such as a USB port, Ethernet® port, Wi-Fi® port, Bluetooth® port or similar, as appropriate. It may also optionally receive data via an optical drive 70. Audio / visual outputs from the entertainment device are typically provided through one or more A / V ports 90 or one or more of the data ports 60. Where components are not integrated, they may be connected as appropriate either by a dedicated data link or via a bus 100. An example of a device for displaying images output by the entertainment system is a head mounted display 'HMD' 120, worn by a user 1. Interaction with the system is typically provided using one or more handheld controllers 130, and / or one or more VR controllers (130A-L,R) in the case of the HMD. Figure 2 schematically illustrates a networked system in accordance with implementations of the present disclosure. In this Figure, a local device 200 is shown in communication, via a network represented by the line, with a remote device 210. The remote device 210 may be a processing device, such as a games console or personal computer, which is typically (but not required to be) located outside of the environment of the local device 200. This may include any location such as a different room or building; in some cases, the remote device 210 may be a processing device belonging to one of the users of the local device 200 and may be located accordingly at that user's house. Rather than being a device such as a games console, the remote device 210 may be a server which provides processing functionality so as to enable execution of a game instance at that server. The remote device 210 can therefore be any suitable device for executing a game instance and communicating with the local device 200 via a network connection. The local device 200 may comprise any computing arrangement that is communicable with a display (which may be integrated with or external to a processing device) for outputting video content to users. The local device 200 is also configured to receive inputs from control devices operated by those users, and is configured to receive data via a network connection such as a local area network or the internet. An example of this is the entertainment system 10 of Figure 1, which is able to output video to a display via the A / V port 90, can be utilised with controllers such as the handheld controller 130, and can receive data via the data port 60. Examples of suitable devices include games consoles, personal computers, televisions, mobile phones, and portable gaming devices. Implementations according to the present disclosure provide the ability for two or more users to interact with separate game instances via the same local device; this enables a local multiplayer 3 experience to be provided for content for which this would otherwise not be an option for that local device - typically due to technical constraints such as limited available processing power. At least one of these instances is executed remotely to that local device, with a network connection being used to transmit images, audio, and / or data to the local device. The present disclosure refers to games as an example of an implementation to aid the clarity of the reader's understanding; however it should be appreciated that the techniques described in this disclosure can be equally applied to any other suitable application in which multiple users may wish to participate simultaneously. This may include applications such as media applications, for example, such as applications which enable users to access free viewpoint video content - each of a plurality of users may wish to view the content from their own viewpoint, thereby causing a user desire for a multi-user arrangement. Figure 3 schematically illustrates a general method in accordance with this function. While the steps are shown in a particular order, this should not be regarded as limiting; it will be appreciated that these may be performed in any suitable order, with some steps being performed substantially simultaneously. For instance, the second instance may be initiated before the first instance, or the second player may be added whilst the execution of the second instance is initiated. A step 300 comprises executing the first instance of a game at a local processing device (that is, the device with which the users are directly interacting - such as a games console in the room with them). This instance is executed in a single player mode, in that only a single player is able to provide inputs to control the execution of that instance. A step 310 comprises adding a second user to the game; this can be in response to inputs from the user of the first instance of the game, a request from the second user, and / or a combination of the two (such as an invitation-based implementation). At this stage, this may comprise associating a user profile of the second user with the game or inserting their user avatar into a game - adding the second user is not taken to mean that the second user is able to interact with the first instance, and in the case that the second instance is not currently being executed no functionality may be available to the second user initially. Adding a second user to the game means that the first game instance and the second game instance will each provide interactivity with a shared game environment - for instance, meaning that the first user's avatar and the second user's avatar are present in the same game environment. A step 320 comprises executing a second instance of the game. This is performed by a remote device (that is, not the local processing device); while this device is typically expected to be remote in the sense that it is not present in the environment of the local device, this is not a requirement and remote may be taken to mean 'separate' in that the second instance is implemented by a device which is independent of the local device. As discussed above with reference to the remote device 210 of Figure 2, the second instance may be implemented by a second games console or a cloud gaming server, for example. A step 330 comprises transmitting an output from the second instance; this output may comprise any suitable data or content as appropriate for a given implementation. For instance, data regarding an avatar's location or interactions may be output, or the results of physics simulations associated with the second user's actions within the second instance may be output to the first instance to enable elements of the second instance to be incorporated into the first instance. Alternatively, or in addition, the output from the second instance comprises video and / or audio of the second instance - such as the rendered video showing the second user's interactions with the second instance. In the case that no video is output by the second instance, the execution of the second instance may be modified so as to not render any images - this can reduce a processing burden upon the remote device, enabling implementation by a device with reduced processing power and / or improving the energy efficiency of such an arrangement. A step 340 comprises interacting with each instance, with the interactions being controlled by users operating respective control devices which each provide inputs to the local device. In the case of the second user's inputs, these are transmitted to the remote device to allow the second instance to be controlled. The local device is configured to display the results of the interactions with the two separate instances of the games; this can be achieved in any suitable manner. For example, in some cases it may be considered appropriate to present the video output of each instance in a split-screen mode such that each instance is shown in a spatially distinct manner. This may be achieved by executing the first instance of the video game locally to generate a video output, while decoding video received from the remote device which comprises the output of the second instance of the video game. Alternatively, the first instance may be updated based upon the output of the second instance so as to represent both instances. This can comprise a shared screen for both players, so that both appear to be within the same game instance - with both player characters appearing within the same camera view, for instance. For example, an object in the first instance may move in dependence upon physics simulations performed by the second instance, with the results of those simulations (or movement information for an object, for example) being output by the second instance for use by the first instance. Implementations in accordance with the method described above can therefore provide a local multiplayer gaming experience while utilising multiple gaming instances executed by different devices. While the above has been described in the context of two players being provided with a gaming experience, it is considered that this could be extended. For example, in the case that each game 5 instance supports four players, two instances could be used to provide an (up to) eight player gaming experience in the same manner. Similarly, it is considered that a greater number of instances of a game could be utilised in combination so as to provide a gaming experience for a greater number of players. Of course, a combination of the two could also be utilised in which three or more game instances each able to support two or more players is used. In any case, the result is achieved in which more players than would otherwise be able to play a game locally are able to play via a single device despite technical limitations. Figure 4 schematically illustrates a system for executing a video game in accordance with the discussion provided above. The system comprises two or more control devices 400, a first processing device 410, a second processing device 420, and a display device 430. While shown here as a part of the system, the second processing device 420 is typically located apart from the other elements and as such can be considered to be distinct from the system as an external unit which provides inputs to the system of the elements 400, 410, and 430. The two or more control devices 400 may include any suitable input devices which enable users to provide inputs to control processing; these may be the control devices 130 or 130A of Figure 1, for example. In some cases the inputs from users may not be button presses or movements of a control device; for instance, gestures or audio inputs. In such a case, the control devices 400 may be any suitable hardware which enables the capture of these inputs such as cameras and / or microphones. The two or more control devices are associated with respective users, such that at least a first control device is associated with a first user and a second control device is associated with a second user. Here, associated may be taken to mean that they are operated by that user; however in some cases further functionality may be considered such as a control device 400 being logged in or registered with a particular user's profile or the like. The first processing device 410 is configured to receive inputs from each of the two or more control devices and to execute a first instance of a video game; the functionality of this device 410 is discussed in more detail below with reference to Figure 5. In a typical implementation, the first processing device 410 may be embodied by a games console or other local processing device; however in some cases it may be preferable that a cloud gaming service is utilised to provide this functionality. In such a case, a local device may be provided which is configured to communicate with the server so as to transmit inputs received from the controllers and to receive video of gameplay for display. The first processing device 410 is further configured to output images for display to the first and second users, the images being generated in dependence upon both the first and second instances of a video game. This may be images generated by the first instance in dependence upon data output by the 6 second instance, or may include images generated by both the first and second instances. The first processing device may also be configured to output audio for at least one of the instances of the video game; in some implementations this may include outputting audio for each instance of the video game via a different respective audio channel such that each user can be provided with audio for a corresponding instance of the video game. The second processing device 420 is configured to execute the second instance of the video game responsive to inputs from the second control device, with these inputs being provided to the second processing device 420 via the first processing device 410. The second processing device 420 is a separate device to the first processing device 410, with the two being in communicably connected via a network connection. The second processing device 420 may be a games console remote to the first processing device 410, for example, or a cloud-based processing arrangement (cloud gaming server) which is configured to execute an instance of a video game. The display device 430 configured to display images generated by the first processing device 410 to both the first and the second user, with the displayed images being dependent upon both the first and second instances of the video game. Turning to Figure 5, this Figure schematically illustrates a configuration of the first processing device 410 of Figure 4 in more detail. The processing device 410 comprises a processor 500, a communication unit 510, an input control unit 520, and an image generation unit 530. These functions may be implemented by the CPU 20, GPU 30, and data port 60 of the entertainment system 10 of Figure 1, for example; however any suitable processing hardware may be used to realise this functionality. The processor 500 is configured to execute a first instance of the video game responsive to inputs received from the first control device 400, such as those to control a user's avatar within the game environment. These inputs may include those which enable a second user to join the game via the second instance, such as issuing an invitation to that second user or configuring the first instance to enable other users to join. The processor 500 may be configured to adapt one or more settings of the first instance of the video game in dependence upon one or more parameters (such as video settings) associated with the second instance of the video game, the output video of the second instance of the video game (should this be provided by the second processing device 420), and / or properties of the network connection between the first processing device 410 and the second processing device 420. This can enable the presentation of the first instance of the video game to be in keeping with (that is, appearing similar or the same) that of the second instance (or the expected presentation of the second instance, in the case that it is not displayed); this can be particularly desirable in an implementation in which both instances are displayed 7 simultaneously in a split-screen fashion. In the case in which only the first instance is displayed, this may still provide advantages in providing video content that accounts for the display settings of the second instance such as brightness which may be important considerations for ensuring user comfort and content visibility. The processor 500 may also, or instead, be configured to modify a camera viewpoint associated with the first instance of the video game in dependence upon an output by the second instance of the video game. For instance, based upon data indicating the location of the second user's avatar in the second instance of the video game a camera viewpoint may be adjusted to ensure that both user's avatars are visible in the same image generated from the first instance of the video game. This can aid an implementation in which a single image is displayed to the users which is representative of the gameplay of both users. In some cases, it is considered that based upon such data the output video may be switched between split-screen and a single image in dependence upon a threshold distance between the first and second user's avatars such that when the threshold distance is exceeded the display is changed to a split-screen view. In some implementations the processor 500 may be configured to identify a latency associated with the receiving of data from the second processing device (such as a latency associated with the network connection and / or a processing time in transmitting inputs from the first processing device 410 to the second processing device 420), and to apply an input latency and / or display latency to the execution of the first instance in dependence upon this. In other words, the processing of the first instance can be adapted to provide an equal (or at least similar) latency to that of the second instance so that each user is able to interact with their respective game instances in a mutually consistent manner. An input latency refers to delaying the provision of the inputs to the game, whilst a display latency refers to delaying the display of images of the game to the user. The introduced latency may be a fixed value which is representative of an average or expected latency, or it may be responsive to live measurements of said latency. The processor 500 may be further configured to adapt settings associated with the first instance of the video game and / or instruct the adapting of settings of the second instance of the video game so as to manage a local processing load or the like. For instance, some hardware may find that executing a game while decoding received video content represents a significant processing burden - in such a case, the video quality of either instance (or indeed both instances) may be modified so as to reduce this burden and ensure that processing can be effectively managed. The communication unit 510 is configured to receive data from a second processing device 420 via a network, the data corresponding to a second instance of the video game being executed concurrently with the first instance. The communication unit 510 may be further configured to perform other communications, such as the transmission of inputs described with reference to the input control unit 520. In some implementations, the communication unit 510 is configured to transmit information to the second processing device 420 comprising information about the initiation of a game session - such as a location of the first user in the in-game environment, or other game state information such as a current stage, user loadout, and quest. The data corresponding to the second instance of the video game may be in any suitable format. In some implementations, the communication unit 510 is configured to receive data comprising output video of the second instance of the video game (optionally with the associated audio). Alternatively, or in addition, the communication unit 510 is configured to receive data comprising the results of one or more simulations (such as physics simulations for in-game interactions by the second user) performed by the second instance of the video game, wherein the results of the one or more simulations are provided to the first instance of the video game. The communication unit 510 can be considered optional, as a number of different implementations may not require any communication - such as when the two game instances are being executed locally by the processor 500 of the first processing device 410. In such a case, the two game instances may be configured to communicate directly without the need of a separate communication unit as is the case when the second instance is being executed by a remote server or the like. In such implementations, the functionality of the second processing device is realised by the first processing device as appropriate to provide two separate instances of the same game using a single device. The input control unit 520 is configured to provide inputs received from the first control device to the first instance of the video game, and to transmit inputs received from the second control device to the second processing device via the network connection. This may be performed by an in-game function associated with the first instance of the video game (or a separate game-specific tool which is executed alongside the video game), or it may be handled externally to the game such as by a system-level function provided by an operating system run by the first processing device 410. The image generation unit 530 is configured to generate images for display in dependence upon both the first and second instances of the video game, with these images being provided for output to both the first and second user by the display device 430 of Figure 4. In some implementations, the image generation unit 530 is configured to generate a split-screen image comprising output video of each of the first and second instances of the video game. However, this is not considered to be limiting; in some cases it may be preferable that the first instance of the video is used to generate images for display which are representative of the gameplay of both users. It is also envisaged that the format is a dynamic one which is responsive to in-game events or conditions - such as based upon user proximity in the game environment such that as the users move apart a split-screen is preferred, or switching to a single screen during cut-scenes or the like. As described above, in the exemplary implementations discussed each of the first instance and second instance of the video game are capable of supporting a single player only. However, it is considered that the same techniques may be extended to any case in which the number of users exceeds the number of users that a single instance of a video game is capable of supporting. Similarly, the number of processing devices is not limited to two, but could be increased to any suitable number. While discussed above with the first instance of the video game being executed locally, it is also considered that the first processing device could be implemented as a thin client or the like which decodes video received from two remote game instances. This may be particularly suitable for low-powered devices such as mobile phones or portable gaming consoles. This thin client may comprise the communication unit 510, input control unit 520, and image generation unit 530 whilst the functionality of the processor 500 is provided remotely (such as by a games console or cloud gaming server). As such, the thin client is configured to receive inputs from the two or more control devices, route these to the appropriate game instances, and receive video which is to be displayed to the users who are local to the thin client. The arrangement of Figure 4 is an example of a system for executing a video game which comprises a first processing device configured to receive inputs from each of two or more control devices associated with respective users, comprising a first control device associated with a first user and a second control device associated with a second user, the first processing device being communicably connected with a second processing device. Figure 5, which illustrates the first processing device, is an example of an arrangement which can be implemented using a processor (for example, a GPU and / or CPU located in a games console or any other computing device) that is operable to receive inputs from each of the two or more control devices to control respective instances of a video game, and in particular is operable to: execute a first instance of the video game at the first processing device; receive data from a second processing device via a network, the data corresponding to a second instance of the video game being executed concurrently with the first instance; provide inputs received from the first control device to the first instance of the video game; transmit inputs received from the second control device to the second processing device; generate images for display in dependence upon both the first and second instances of the video game; and display the generated images to both the first and the second user via the same display device, wherein the first instance of the video game is responsive to inputs received from the first control device, and the second instance of the video game is responsive to inputs received from the second control device. This functionality may be provided in accordance with any of the hardware configurations described elsewhere in this disclosure; for instance, the first processing device may be implemented as the entertainment system 10 of Figure 1. In this case, processing is performed by the CPU 20 and / or GPU 30. Figure 6 schematically illustrates a method for executing a video game by a first processing device configured to receive inputs from each of two or more control devices associated with respective users, comprising a first control device associated with a first user and a second control device associated with a second user. A first instance of the video game is responsive to inputs received from the first control device, and a second instance of the video game is responsive to inputs received from the second control device. A step 600 comprises executing a first instance of the video game at the first processing device. A step 610 comprises receiving data from a second processing device via a network, the data corresponding to a second instance of the video game being executed concurrently with the first instance. A step 620 comprises managing inputs received from the two or more control devices, with the management comprising providing inputs received from the first control device to the first instance of the video game and transmitting inputs received from the second control device to the second processing device. A step 630 comprises generating images for display in dependence upon both the first and second instances of the video game. A step 640 comprises displaying the generated images to both the first and the second user via the same display device. When providing content in the manner described above, such that two or more separate applications (such as video games) are executed to generate a split-screen multi-user experience, it is considered important that the separate applications are able to remain synchronised. Due to the split-screen nature of the display, a lack of synchronisation can be jarring to a user (for instance, if both view the same video content with a time offset from one another) for example, or can cause one of the users to have a competitive advantage when the application is a game. In some cases there can be a loss of immersion in an application if both users are experiencing events at different times on the same display - as such, preserving a synchronised execution of the two instances is considered to provide significant advantages. In implementations in which two players are playing in a shared game environment, via different game instances, using a single display it is therefore apparent that methods of reducing latency (or at least causing the respective latencies associated with each instance to be more similar) are desirable. This can enable the two instances to be run in a more consistent manner with respect to one another - thereby avoiding disadvantaging a player due to a difference in locations of their respective instances, or applications to otherwise be run with a reduced level of synchronicity (which can lead to issues with displaying the applications side-by-side, for instance). One source of latency which is relevant in this context is that of input latency; each of the controllers used with the local processing device generates a separate input stream, with these to be provided to a respective separate instance of the game. It is proposed that a single compute process is used to manage the input streams, despite being intended for the control of different application instances at different processing devices. This can reduce processing requirements at the local processing device (versus using a separate compute process to manage each stream, for instance), as well as ensuring that the input streams are handled in the same manner (thereby reducing the chance for latency to be introduced for a single input stream relative to another). Another source of relative latency (that is, the difference in latency associated with each of the application instances) and a processing burden is in a scenario in which both instances of the application are being executed remotely to the local processing device; in this case, compositing the received video streams locally can place a significant processing burden upon the local processing device. This can be amplified in the case that the video is high quality (such as 4K) and / or has a high frame rate (such as 60 or 120 frames per second), or in the case in which processing is performed on the video content locally -such as a resizing, upsampling, or applying other effects. It is proposed that the manner in which server-hosted application instances are managed is modified to address this issue. A common element for implementing arrangements in accordance with these proposals is that of a single process being configured to manage resources relating to both instances of the application. In the case that an instance of the application is being executed locally, this application may be designated as the 'primary application' and this application may accordingly be used to manage 12 resources relating to both instances of the application. For instance, this process may be configured to receive the video input of the second instance of the application, and to perform a video compositing process which can synchronise the video content based upon application data (such as event information which can be identified from the local processing and in the video content independently). Similarly, in the case that both instances are being executed remotely at different respective locations (such as a server-based instance and a remote console-based instance), a single process may be executed at the local processing device which is configured to manage the respective inputs and received videos. In the case that both instances of the application are being executed at the same server, such a process may be executed by the local processing device or by the server itself; alternatively, processes may be implemented at either end to enable such functionality. In the case that a process is executed at the server which is executing both instances, this may be either as a part of a designated primary instance of the application or as a standalone process which is able to communicate with each instance of the application. While the below discussion focuses on this latter arrangement, in which both instances of the application are being executed by the same server, it should be appreciated that similar advantages may be obtained in other arrangements with changes to the implementation as appropriate. Figure 7 schematically illustrates a method in accordance with the above discussion, in which a server executes both instances of an application in dependence upon inputs received from a local processing device interacted with by a respective user for each instance of the application. A step 700 comprises receiving two or more sets of inputs at the local processing device and generating a single input stream. Each of the sets of inputs corresponds to a particular user, and is received from a respective control device (such as a games controller, or a pair of devices such as a mouse and keyboard) for the control of a corresponding instance of an application. A step 710 comprises transmitting the single input stream to the server. This single input stream may be configured in any suitable manner, so long as the separate input streams corresponding to the respective control device can be derived from the single input stream. For instance, the single input stream may comprise inputs which share a memory window of DMA (Direct Memory Allocation). A step 720 comprises passing these inputs to respective instances of the application. This may be performed by either a standalone process at the server, or by a primary instance of the application being executed at the server. In the latter case, the single input stream may be provided to the primary instance of the application (this can be selected arbitrarily - the primary instance is simply the one 13 configured to perform this processing), which then separates the input streams and passes the respective input streams to their corresponding application instances as appropriate. A step 730 comprises executing the instances of the application responsive to the corresponding inputs. As discussed above, for instance with reference to Figure 3, each of these instances is executed largely independently with information being passed between the two instances to inform them of events or the like happening in the other of the instances. In some implementations, it is considered that the users need not be interacting in a multi-user manner - the same advantages may be obtained even if the users are each interacting with an application instance entirely independently of one another and simply sharing a display. A step 740 comprises generating a respective video output in dependence upon the execution of each application. A step 750 comprises compositing the generated video outputs into a single stream; this may be performed by the primary instance of the application, should this be defined, or by a separate process hosted by the server. The compositing may be performed in any suitable manner, as appropriate for a given implementation. For instance, in some cases the two video outputs may be transmitted separately within the single stream, such that each can be decoded individually. Alternatively, the two videos may be arranged side-by-side (or in any other configuration) so as to generate a single display video within the stream. This may be performed with knowledge of display conditions at the local processing device, for example, such as an aspect ratio of a display, an orientation of a display, or other physical display properties such as DPI or resolution. In the case that the local processing device is associated with more than one display device (such as multiple monitors, or a plurality of head-mountable display devices), such information may be obtained for each of the display devices. Either of steps 740 or 750 may be implemented with a view to the manner in which the videos will be displayed by the local processing device. This can include generating video (or cropping video) to account for the fact that the videos will share a display space - such as generating two videos with an 8:9 aspect ratio rather than a 16:9 aspect ratio so that the local processing device is not required to resize the video or otherwise make changes to enable their correct display. A step 760 comprises transmitting the stream to the local processing device. A step 770 comprises decoding the received video at the local processing device, and applying any desired post-processing effects prior to display. Effects may include modifications to the video display, such as colour or contrast changes, as well as changing the arrangement of component parts of the video (such as adding an offset or borders to separate the respective video displays). This may include modifications to the display of content on a single display, or over a plurality of displays where appropriate. In the latter case in particular, it may be useful to consider the entire display area associated with the local processing device as a single addressable display space with the processing being configured to assign different parts of the decoded video to respective parts of the display space. A step 780 comprises displaying the decoded video, thereby providing each of the users with a view of their respective instance of the application. The decoded video is displayed using a single display device, or split over a plurality of display devices each associated with the same local processing device (such as a pair of displays used with a single computer, with each display showing a different user's instance of the application, or a pair of HMDs associated with a single games console). By using an implementation in line with the above method, a local processing device can be used to provide two or more users with respective interactive application experiences in an efficient manner -the use of local resources for compute processing is reduced, and the efficiency of data transmission between the device and the server is improved. By linking the inputs and outputs of the respective application instances prior to transmission, the relative latency between the instances can be maintained - thereby also offering an improved user experience. Figure 8 schematically illustrates a system for managing the execution of a plurality of application instances at a server in response to inputs from a single client device; this system may be configured to provide functionality such as that described with reference to Figure 7. The system comprises an input receiving unit 800, an input processing unit 810, a plurality of application execution units 820, a video compositing unit 830, and a video output unit 840. The functionality of these units may be realised using one or more CPUs and or GPUs located at the server as desired; the exact configuration of these hardware units may be selected freely, with examples being discussed below. The input receiving unit 800 is configured to receive (from the client device) a single input stream comprising input information from two or more input devices associated with the client device, each of the input devices being operated by a different user to control a corresponding application instance. As discussed above, the single input stream may comprise a plurality of input streams which share a direct memory allocation memory window; however any suitable format may be used (such as interleaved frames of input data). The input receiving unit 800 may be executed as a standalone unit at the server which is configured to communicate with each of the application instances being executed; alternatively, a first application execution unit 820 may provide the functionality of the input receiving unit 800, such that a first instance of the application receives the single input stream. The input stream can then be provided to the input processing unit 810 by the application. The input processing unit 810 is configured to process the single input stream to obtain a plurality of input streams each corresponding to a different input device; in other words, the single input stream is decomposed into the respective input streams that were provided by the users at the client device. Once obtained, the separate input streams can be provided to their respective application instances for use in controlling the processing of the corresponding application instance. The input processing unit 810 may be configured to determine a latency between the execution of the respective instances of the application, and to delay the transmission of inputs to one or more of the respective instances in dependence upon this latency. For instance, if two instances of an application are running with a 10ms latency between them the input processing unit 810 may delay the transmission of inputs to the leading instance by 10ms to align the execution more closely. The input processing unit 810 may be executed as a standalone unit at the server which is configured to communicate with each of the application instances being executed; alternatively, the first application execution unit 820 may provide the functionality of the input processing unit 810, such that the first instance of the application processes the single input stream. The plurality of application execution units 820 are each configured to execute a respective instance of the application in dependence upon a corresponding one of the plurality of input streams, wherein each instance generates a respective video output providing a view of that instance of the application. The application execution units 820 may be implemented using separate hardware for each instance (such that the application execution units 820 are each implemented using different CPUs and / or GPUs as appropriate); alternatively, the functionality of more than one application execution unit 820 may be implemented using the same hardware. The different application execution units 820 may share any resources as appropriate; for instance, the application execution units 820 may utilise a shared memory for application data. For instance, in some cases the plurality of application execution units 820 are implemented using respective compute servers in the same server rack; alternatively, the plurality of application execution units 820 are instead implemented using the same compute server. Of course, in some cases a plurality of application execution units 820 may be implemented using a plurality of compute servers in an N to 1 ratio, where N is any number greater than 1. As a further option, the application execution units 820 may be located in different server racks or different servers altogether; it is not required that the application instances are executed at the same physical location. The video compositing unit 830 is configured to generate a single video stream comprising at least a portion of each of the respective video outputs of the executed instances of the application. This may include cropping or otherwise resizing one or more of the respective video outputs to generate a single 16 video stream suitable for display by the client; similarly, other processing may be performed such as adding borders between the video outputs or to insert content to fill gaps between those outputs (such as a scoreboard or map if there is a space in the single video stream), or applying visual effects such as depth of field or motion blur. The video compositing unit 830 may also be configured to control the visibility of post-process layers such as UI or HUD elements; these are typically overlaid after the rendering of an image is completed, and so can be implemented as a separate process to that of the rendering. This may be advantageous in that these can be overlaid with more complete knowledge about how the content will be displayed - for instance, a small display size may be identified and UI elements may be scaled up accordingly to preserve their visibility. The placement and other parameters (such as a resolution) may also be managed as a part of this process. The first application execution unit 820 may also provide the functionality of the video compositing unit 830, such that the first instance of the application generates the single video stream; however it may be considered preferable in some cases to utilise a separate unit which is independent of any of the application instances. The video output unit 840 is configured to output the single video stream to the client device, with the client device being configured to obtain, decode, and display the single video stream. The client device may be associated with two or more display devices which each display one of the respective video outputs obtained from the single video stream; in such a case the client device may be configured to process the obtained single video stream to enable such display. The arrangement of Figure 8 is an example of a processing device (for example, GPUs and / or CPUs located in a server) that is operable to manage the execution of a plurality of application instances at a server in response to inputs from a single client device, and in particular is operable to: receive a single input stream comprising input information from two or more input devices associated with the client device, each of the input devices being operated by a different user to control a corresponding application instance; process the single input stream to obtain a plurality of input streams each corresponding to a different input device; execute a plurality of respective instances of the application in dependence upon a corresponding one of the plurality of input streams, wherein each instance generates a respective video output; generate a single video stream comprising at least a portion of each of the respective video outputs of the executed instances of the application; and 17 output the single video stream to the client device. Figure 9 schematically illustrates an alternative arrangement of the units of Figure 8, in line with the above discussion. In particular, Figure 9 shows an implementation in which a first application execution unit 900 provides the functionality of the input receiving unit 800, input processing unit 810, and video compositing unit 830 in addition to executing a first instance of the application. The processing may be executed by the first application execution unit 900 separately to the first instance of the application, or the functionality may be provided as a part of that first instance of the application. Such an arrangement may be preferable as it may allow data generated by the first instance of the application to be used to generate an improved single video stream or otherwise improve the multi-user application experience. Of course, it is not required that the first application execution unit 900 provide all of this functionality -only portions may be provided, with the remaining parts being handled by other processing units. Figure 10 schematically illustrates a method for managing the execution of a plurality of application instances at a server in response to inputs from a single client device. The method is implemented by hardware discussed with reference to Figures 8 or 9, for example. A step 1000 comprises receiving a single input stream comprising input information from two or more input devices associated with the client device, each of the input devices being operated by a different user to control a corresponding application instance. A step 1010 comprises processing the single input stream to obtain a plurality of input streams each corresponding to a different input device. A step 1020 comprises executing a plurality of respective instances of the application in dependence upon a corresponding one of the plurality of input streams, wherein each instance generates a respective video output. A step 1030 comprises generating a single video stream comprising at least a portion of each of the respective video outputs of the executed instances of the application. A step 1040 comprises outputting the single video stream to the client device. While the above discussion has focused upon an implementation in which both the single input stream and the single video stream are utilised, it should be understood that this is not a requirement. This is because each of these may be implemented independently of one another. For instance, an implementation may be used in which a single input stream is generated by the local processing (client) device and transmitted to a server but the corresponding video processing is not performed. The server (as part of executing an application or otherwise) can then unpack the single 18 input stream into respective input streams which are provided to corresponding application instances. These application instances can then provide their respective video streams to the local processing device separately, with the local processing device then decoding each video stream and arranging them for display locally. In such an implementation there can still be a latency reduction and more efficient transmission of inputs, and as such technical benefits are able to be realised independent of the single video stream. Similarly, separate input streams may be transmitted by the local processing device to the server. The server can then execute each of the application instances in dependence upon these input streams, with the server then generating a single video stream on the basis of the outputs of these application instances. This single video stream can then be transmitted to the local processing device for display. In such an implementation there is still a reduction of the processing burden upon the local processing device as well as an increase in video transmission efficiency; as such, technical benefits are able to be realised independent of the single input stream. The techniques described above may be implemented in hardware, software or combinations of the two. In the case that a software-controlled data processing apparatus is employed to implement one or more features of the embodiments, it will be appreciated that such software, and a storage or transmission medium such as a non-transitory machine-readable storage medium by which such software is provided, are also considered as embodiments of the disclosure. Thus, the foregoing discussion discloses and describes merely exemplary embodiments of the present invention. As will be understood by those skilled in the art, the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting of the scope of the invention, as well as other claims. The disclosure, including any readily discernible variants of the teachings herein, defines, in part, the scope of the foregoing claim terminology such that no inventive subject matter is dedicated to the public.

Claims

1. A system for managing the execution of a plurality of application instances at a server in response to inputs from a single client device, the system comprising:an input receiving unit configured to receive a single input stream comprising input information from two or more input devices associated with the client device, each of the input devices being operated by a different user to control a corresponding application instance;an input processing unit configured to process the single input stream to obtain a plurality of input streams each corresponding to a different input device;a plurality of application execution units each configured to execute a respective instance of the application in dependence upon a corresponding one of the plurality of input streams, wherein each instance generates a respective video output;a video compositing unit configured to generate a single video stream comprising at least a portion of each of the respective video outputs of the executed instances of the application; anda video output unit configured to output the single video stream to the client device.

2. A system according to claim 1, wherein a first application execution unit provides the functionality of the input receiving unit, such that a first instance of the application receives the single input stream.

3. A system according to any preceding claim, wherein the first application execution unit provides the functionality of the input processing unit, such that the first instance of the application processes the single input stream.

4. A system according to any preceding claim, wherein the plurality of application execution units are implemented using respective compute servers in the same server rack.

5. A system according to any of claims 1-3, wherein the plurality of application execution units are implemented using the same compute server.

6. A system according to any preceding claim, wherein the application execution units utilise a shared memory for application data.

7. A system according to any preceding claim, wherein the video compositing unit is configured to resize one or more of the respective video outputs.

8. A system according to any preceding claim, wherein the single input stream comprises a plurality of input streams which share a direct memory allocation memory window.

9. A system according to any preceding claim, wherein the input processing unit is configured to determine a latency between the execution of the respective instances of the application, and to delay the transmission of inputs to one or more of the respective instances in dependence upon this latency.

10. A system according to any preceding claim, wherein the first application execution unit provides the functionality of the video compositing unit, such that the first instance of the application generates the single video stream.

11. A system according to any preceding claim, comprising the client device which is configured to obtain, decode, and display the single video stream.

12. A system according to claim 11, wherein the client device is associated with two or more display devices which each display one of the respective video outputs obtained from the single video stream.

13. A method for managing the execution of a plurality of application instances at a server in response to inputs from a single client device, the method comprising:receiving a single input stream comprising input information from two or more input devices associated with the client device, each of the input devices being operated by a different user to control a corresponding application instance;processing the single input stream to obtain a plurality of input streams each corresponding to a different input device;executing a plurality of respective instances of the application in dependence upon a corresponding one of the plurality of input streams, wherein each instance generates a respective video output;generating a single video stream comprising at least a portion of each of the respective video outputs of the executed instances of the application; andoutputting the single video stream to the client device.

14. Computer software comprising instructions which, when the software is executed by a computer, causes the computer to carry out the method of claim 13.

15. A non-transitory machine-readable storage medium which stores computer software according to claim 14.

Images