Multiplayer gaming system and method

Abstract

A system for providing a synchronised multi-user game application at a local processing device, comprising first and second application processing units executing respective first and second instances

Description

BACKGROUND OF THE INVENTION Field of the invention This disclosure relates to a multiplayer gaming system and method. Description of the Prior Art The "background" description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present invention. With the increasing level of complexity and visual quality that are able to be provided in games, the demands upon processing hardware have increased significantly over time. For instance, more powerful processing hardware may be required to execute games, or limits may be placed upon the functionality of such games in order to enable them to be executed by a typical device. One example of this is multiplayer experiences that can only be experienced in an online or networked setting - rather than in so-called 'couch co-op' or split-screen modes within a game. While there is a desire to offer a more local experience, technical limitations of executing devices can impose a restriction. This technical limitation may be imposed by the hardware of a games console (which is largely standardised by the manufacturer), or by any other computing arrangement; for instance, content may be developed on the basis of known console capabilities or information about average or expected computing power available to eventual users. While in some cases a device may have sufficient computing power to provide a local multiplayer experience for a game (particularly in the case of a modern computing device executing an older game), many games may not be designed with this in mind. It is in the context of the above discussion that the present disclosure arises. SUMMARY OF THE INVENTION This disclosure is defined by claim 1. Further respective aspects and features of the disclosure are defined in the appended claims. It is to be understood that both the foregoing general description of the invention and the following detailed description are exemplary, but are not restrictive, of the invention. BRIEF DESCRIPTION OF THE DRAWINGS A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein: Figure 1 schematically illustrates an entertainment system; Figure 2 schematically illustrates a networked system; Figure 3 schematically illustrates a method; Figure 4 schematically illustrates a system for executing a video game; Figure 5 schematically illustrates a first processing device; Figure 6 schematically illustrates a method for executing a video game by a first processing device; Figure 7 schematically illustrates a method for modifying the display of video content; Figure 8 schematically system for providing a synchronised multi-user application experience at a local processing device; and Figure 9 schematically method for providing a synchronised multi-user application experience at a local processing device. DESCRIPTION OF THE EMBODIMENTS Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views, embodiments of the present disclosure are described. Referring to Figure 1, an example of an entertainment system 10 is a computer or console. The entertainment system 10 comprises a central processor or CPU 20. The entertainment system also comprises a graphical processing unit or GPU 30, and RAM 40. Two or more of the CPU, GPU, and RAM may be integrated as a system on a chip (SoC). Further storage may be provided by a disk 50, either as an external or internal hard drive, or as an external solid state drive, or an internal solid state drive. The entertainment device may transmit or receive data via one or more data ports 60, such as a USB port, Ethernet® port, Wi-Fi® port, Bluetooth® port or similar, as appropriate. It may also optionally receive data via an optical drive 70. Audio / visual outputs from the entertainment device are typically provided through one or more A / V ports 90 or one or more of the data ports 60. Where components are not integrated, they may be connected as appropriate either by a dedicated data link or via a bus 100. An example of a device for displaying images output by the entertainment system is a head mounted display 'HMD' 120, worn by a user 1. Interaction with the system is typically provided using one or more handheld controllers 130, and / or one or more VR controllers (130A-L,R) in the case of the HMD. Figure 2 schematically illustrates a networked system in accordance with implementations of the present disclosure. In this Figure, a local device 200 is shown in communication, via a network represented by the line, with a remote device 210. The remote device 210 may be a processing device, such as a games console or personal computer, which is typically (but not required to be) located outside of the environment of the local device 200. This may include any location such as a different room or building; in some cases, the remote device 210 may be a processing device belonging to one of the users of the local device 200 and may be located accordingly at that user's house. Rather than being a device such as a games console, the remote device 210 may be a server which provides processing functionality so as to enable execution of a game instance at that server. The remote device 210 can therefore be any suitable device for executing a game instance and communicating with the local device 200 via a network connection. The local device 200 may comprise any computing arrangement that is communicable with a display (which may be integrated with or external to a processing device) for outputting video content to users. The local device 200 is also configured to receive inputs from control devices operated by those users, and is configured to receive data via a network connection such as a local area network or the internet. An example of this is the entertainment system 10 of Figure 1, which is able to output video to a display via the A / V port 90, can be utilised with controllers such as the handheld controller 130, and can receive data via the data port 60. Examples of suitable devices include games consoles, personal computers, televisions, mobile phones, and portable gaming devices. Implementations according to the present disclosure provide the ability for two or more users to interact with separate game instances via the same local device; this enables a local multiplayer experience to be provided for content for which this would otherwise not be an option for that local device - typically due to technical constraints such as limited available processing power. At least one of these instances is executed remotely to that local device, with a network connection being used to transmit images, audio, and / or data to the local device. The present disclosure refers to games as an example of an implementation to aid the clarity of the reader's understanding; however it should be appreciated that the techniques described in this disclosure can be equally applied to any other suitable application in which multiple users may wish to participate simultaneously. This may include applications such as media applications, for example, such as applications which enable users to access free viewpoint video content - each of a plurality of users may wish to view the content from their own viewpoint, thereby causing a user desire for a multi-user arrangement. Figure 3 schematically illustrates a general method in accordance with this function. While the steps are shown in a particular order, this should not be regarded as limiting; it will be appreciated that these may be performed in any suitable order, with some steps being performed substantially simultaneously. For instance, the second instance may be initiated before the first instance, or the second player may be added whilst the execution of the second instance is initiated. A step 300 comprises executing the first instance of a game at a local processing device (that is, the device with which the users are directly interacting - such as a games console in the room with them). This instance is executed in a single player mode, in that only a single player is able to provide inputs to control the execution of that instance. A step 310 comprises adding a second user to the game; this can be in response to inputs from the user of the first instance of the game, a request from the second user, and / or a combination of the two (such as an invitation-based implementation). At this stage, this may comprise associating a user profile of the second user with the game or inserting their user avatar into a game - adding the second user is not taken to mean that the second user is able to interact with the first instance, and in the case that the second instance is not currently being executed no functionality may be available to the second user initially. Adding a second user to the game means that the first game instance and the second game instance will each provide interactivity with a shared game environment - for instance, meaning that the first user's avatar and the second user's avatar are present in the same game environment. A step 320 comprises executing a second instance of the game. This is performed by a remote device (that is, not the local processing device); while this device is typically expected to be remote in the sense that it is not present in the environment of the local device, this is not a requirement and remote may be taken to mean 'separate' in that the second instance is implemented by a device which is independent of the local device. As discussed above with reference to the remote device 210 of Figure 2, the second instance may be implemented by a second games console or a cloud gaming server, for example. A step 330 comprises transmitting an output from the second instance; this output may comprise any suitable data or content as appropriate for a given implementation. For instance, data regarding an avatar's location or interactions may be output, or the results of physics simulations associated with the second user's actions within the second instance may be output to the first instance to enable elements of the second instance to be incorporated into the first instance. Alternatively, or in addition, the output from the second instance comprises video and / or audio of the second instance - such as the rendered video showing the second user's interactions with the second instance. In the case that no video is output by the second instance, the execution of the second instance may be modified so as to not render any images - this can reduce a processing burden upon the remote device, enabling implementation by a device with reduced processing power and / or improving the energy efficiency of such an arrangement. A step 340 comprises interacting with each instance, with the interactions being controlled by users operating respective control devices which each provide inputs to the local device. In the case of the second user's inputs, these are transmitted to the remote device to allow the second instance to be controlled. The local device is configured to display the results of the interactions with the two separate instances of the games; this can be achieved in any suitable manner. For example, in some cases it may be considered appropriate to present the video output of each instance in a split-screen mode such that each instance is shown in a spatially distinct manner. This may be achieved by executing the first instance of the video game locally to generate a video output, while decoding video received from the remote device which comprises the output of the second instance of the video game. Alternatively, the first instance may be updated based upon the output of the second instance so as to represent both instances. This can comprise a shared screen for both players, so that both appear to be within the same game instance - with both player characters appearing within the same camera view, for instance. For example, an object in the first instance may move in dependence upon physics simulations performed by the second instance, with the results of those simulations (or movement information for an object, for example) being output by the second instance for use by the first instance. Implementations in accordance with the method described above can therefore provide a local multiplayer gaming experience while utilising multiple gaming instances executed by different devices. While the above has been described in the context of two players being provided with a gaming experience, it is considered that this could be extended. For example, in the case that each game instance supports four players, two instances could be used to provide an (up to) eight player gaming experience in the same manner. Similarly, it is considered that a greater number of instances of a game could be utilised in combination so as to provide a gaming experience for a greater number of players. Of course, a combination of the two could also be utilised in which three or more game instances each able to support two or more players is used. In any case, the result is achieved in which more players than would otherwise be able to play a game locally are able to play via a single device despite technical limitations. Figure 4 schematically illustrates a system for executing a video game in accordance with the discussion provided above. The system comprises two or more control devices 400, a first processing device 410, a second processing device 420, and a display device 430. While shown here as a part of the system, the second processing device 420 is typically located apart from the other elements and as such can be considered to be distinct from the system as an external unit which provides inputs to the system of the elements 400, 410, and 430. The two or more control devices 400 may include any suitable input devices which enable users to provide inputs to control processing; these may be the control devices 130 or 130A of Figure 1, for example. In some cases the inputs from users may not be button presses or movements of a control device; for instance, gestures or audio inputs. In such a case, the control devices 400 may be any suitable hardware which enables the capture of these inputs such as cameras and / or microphones. The two or more control devices are associated with respective users, such that at least a first control device is associated with a first user and a second control device is associated with a second user. Here, associated may be taken to mean that they are operated by that user; however in some cases further functionality may be considered such as a control device 400 being logged in or registered with a particular user's profile or the like. The first processing device 410 is configured to receive inputs from each of the two or more control devices and to execute a first instance of a video game; the functionality of this device 410 is discussed in more detail below with reference to Figure 5. In a typical implementation, the first processing device 410 may be embodied by a games console or other local processing device; however in some cases it may be preferable that a cloud gaming service is utilised to provide this functionality. In such a case, a local device may be provided which is configured to communicate with the server so as to transmit inputs received from the controllers and to receive video of gameplay for display. The first processing device 410 is further configured to output images for display to the first and second users, the images being generated in dependence upon both the first and second instances of a video game. This may be images generated by the first instance in dependence upon data output by the second instance, or may include images generated by both the first and second instances. The first processing device may also be configured to output audio for at least one of the instances of the video 6 game; in some implementations this may include outputting audio for each instance of the video game via a different respective audio channel such that each user can be provided with audio for a corresponding instance of the video game. The second processing device 420 is configured to execute the second instance of the video game responsive to inputs from the second control device, with these inputs being provided to the second processing device 420 via the first processing device 410. The second processing device 420 is a separate device to the first processing device 410, with the two being in communicably connected via a network connection. The second processing device 420 may be a games console remote to the first processing device 410, for example, or a cloud-based processing arrangement (cloud gaming server) which is configured to execute an instance of a video game. The display device 430 configured to display images generated by the first processing device 410 to both the first and the second user, with the displayed images being dependent upon both the first and second instances of the video game. Turning to Figure 5, this Figure schematically illustrates a configuration of the first processing device 410 of Figure 4 in more detail. The processing device 410 comprises a processor 500, a communication unit 510, an input control unit 520, and an image generation unit 530. These functions may be implemented by the CPU 20, GPU 30, and data port 60 of the entertainment system 10 of Figure 1, for example; however any suitable processing hardware may be used to realise this functionality. The processor 500 is configured to execute a first instance of the video game responsive to inputs received from the first control device 400, such as those to control a user's avatar within the game environment. These inputs may include those which enable a second user to join the game via the second instance, such as issuing an invitation to that second user or configuring the first instance to enable other users to join. The processor 500 may be configured to adapt one or more settings of the first instance of the video game in dependence upon one or more parameters (such as video settings) associated with the second instance of the video game, the output video of the second instance of the video game (should this be provided by the second processing device 420), and / or properties of the network connection between the first processing device 410 and the second processing device 420. This can enable the presentation of the first instance of the video game to be in keeping with (that is, appearing similar or the same) that of the second instance (or the expected presentation of the second instance, in the case that it is not displayed); this can be particularly desirable in an implementation in which both instances are displayed simultaneously in a split-screen fashion. In the case in which only the first instance is displayed, this may still provide advantages in providing video content that accounts for the display settings of the second 7 instance such as brightness which may be important considerations for ensuring user comfort and content visibility. The processor 500 may also, or instead, be configured to modify a camera viewpoint associated with the first instance of the video game in dependence upon an output by the second instance of the video game. For instance, based upon data indicating the location of the second user's avatar in the second instance of the video game a camera viewpoint may be adjusted to ensure that both user's avatars are visible in the same image generated from the first instance of the video game. This can aid an implementation in which a single image is displayed to the users which is representative of the gameplay of both users. In some cases, it is considered that based upon such data the output video may be switched between split-screen and a single image in dependence upon a threshold distance between the first and second user's avatars such that when the threshold distance is exceeded the display is changed to a split-screen view. In some implementations the processor 500 may be configured to identify a latency associated with the receiving of data from the second processing device (such as a latency associated with the network connection and / or a processing time in transmitting inputs from the first processing device 410 to the second processing device 420), and to apply an input latency and / or display latency to the execution of the first instance in dependence upon this. In other words, the processing of the first instance can be adapted to provide an equal (or at least similar) latency to that of the second instance so that each user is able to interact with their respective game instances in a mutually consistent manner. An input latency refers to delaying the provision of the inputs to the game, whilst a display latency refers to delaying the display of images of the game to the user. The introduced latency may be a fixed value which is representative of an average or expected latency, or it may be responsive to live measurements of said latency. The processor 500 may be further configured to adapt settings associated with the first instance of the video game and / or instruct the adapting of settings of the second instance of the video game so as to manage a local processing load or the like. For instance, some hardware may find that executing a game while decoding received video content represents a significant processing burden - in such a case, the video quality of either instance (or indeed both instances) may be modified so as to reduce this burden and ensure that processing can be effectively managed. The communication unit 510 is configured to receive data from a second processing device 420 via a network, the data corresponding to a second instance of the video game being executed concurrently with the first instance. The communication unit 510 may be further configured to perform other communications, such as the transmission of inputs described with reference to the input control unit 520. In some implementations, the communication unit 510 is configured to transmit information to the second processing device 420 comprising information about the initiation of a game session - such as a location of the first user in the in-game environment, or other game state information such as a current stage, user loadout, and quest. The data corresponding to the second instance of the video game may be in any suitable format. In some implementations, the communication unit 510 is configured to receive data comprising output video of the second instance of the video game (optionally with the associated audio). Alternatively, or in addition, the communication unit 510 is configured to receive data comprising the results of one or more simulations (such as physics simulations for in-game interactions by the second user) performed by the second instance of the video game, wherein the results of the one or more simulations are provided to the first instance of the video game. The communication unit 510 can be considered optional, as a number of different implementations may not require any communication - such as when the two game instances are being executed locally by the processor 500 of the first processing device 410. In such a case, the two game instances may be configured to communicate directly without the need of a separate communication unit as is the case when the second instance is being executed by a remote server or the like. In such implementations, the functionality of the second processing device is realised by the first processing device as appropriate to provide two separate instances of the same game using a single device. The input control unit 520 is configured to provide inputs received from the first control device to the first instance of the video game, and to transmit inputs received from the second control device to the second processing device via the network connection. This may be performed by an in-game function associated with the first instance of the video game (or a separate game-specific tool which is executed alongside the video game), or it may be handled externally to the game such as by a system-level function provided by an operating system run by the first processing device 410. The image generation unit 530 is configured to generate images for display in dependence upon both the first and second instances of the video game, with these images being provided for output to both the first and second user by the display device 430 of Figure 4. In some implementations, the image generation unit 530 is configured to generate a split-screen image comprising output video of each of the first and second instances of the video game. However, this is not considered to be limiting; in some cases it may be preferable that the first instance of the video is used to generate images for display which are representative of the gameplay of both users. It is also envisaged that the format is a dynamic one which is responsive to in-game events or conditions - such as based upon user proximity in the game environment such that as the users move apart a split-screen is preferred, or switching to a single screen during cut-scenes or the like. As described above, in the exemplary implementations discussed each of the first instance and second instance of the video game are capable of supporting a single player only. However, it is considered that the same techniques may be extended to any case in which the number of users exceeds the number of users that a single instance of a video game is capable of supporting. Similarly, the number of processing devices is not limited to two, but could be increased to any suitable number. While discussed above with the first instance of the video game being executed locally, it is also considered that the first processing device could be implemented as a thin client or the like which decodes video received from two remote game instances. This may be particularly suitable for low-powered devices such as mobile phones or portable gaming consoles. This thin client may comprise the communication unit 510, input control unit 520, and image generation unit 530 whilst the functionality of the processor 500 is provided remotely (such as by a games console or cloud gaming server). As such, the thin client is configured to receive inputs from the two or more control devices, route these to the appropriate game instances, and receive video which is to be displayed to the users who are local to the thin client. The arrangement of Figure 4 is an example of a system for executing a video game which comprises a first processing device configured to receive inputs from each of two or more control devices associated with respective users, comprising a first control device associated with a first user and a second control device associated with a second user, the first processing device being communicably connected with a second processing device. Figure 5, which illustrates the first processing device, is an example of an arrangement which can be implemented using a processor (for example, a GPU and / or CPU located in a games console or any other computing device) that is operable to receive inputs from each of the two or more control devices to control respective instances of a video game, and in particular is operable to: execute a first instance of the video game at the first processing device; receive data from a second processing device via a network, the data corresponding to a second instance of the video game being executed concurrently with the first instance; provide inputs received from the first control device to the first instance of the video game; transmit inputs received from the second control device to the second processing device; generate images for display in dependence upon both the first and second instances of the video game; and display the generated images to both the first and the second user via the same display device, wherein the first instance of the video game is responsive to inputs received from the first control device, and the second instance of the video game is responsive to inputs received from the second control device. This functionality may be provided in accordance with any of the hardware configurations described elsewhere in this disclosure; for instance, the first processing device may be implemented as the entertainment system 10 of Figure 1. In this case, processing is performed by the CPU 20 and / or GPU 30. Figure 6 schematically illustrates a method for executing a video game by a first processing device configured to receive inputs from each of two or more control devices associated with respective users, comprising a first control device associated with a first user and a second control device associated with a second user. A first instance of the video game is responsive to inputs received from the first control device, and a second instance of the video game is responsive to inputs received from the second control device. A step 600 comprises executing a first instance of the video game at the first processing device. A step 610 comprises receiving data from a second processing device via a network, the data corresponding to a second instance of the video game being executed concurrently with the first instance. A step 620 comprises managing inputs received from the two or more control devices, with the management comprising providing inputs received from the first control device to the first instance of the video game and transmitting inputs received from the second control device to the second processing device. A step 630 comprises generating images for display in dependence upon both the first and second instances of the video game. A step 640 comprises displaying the generated images to both the first and the second user via the same display device. When users interact with applications in accordance with the above implementations, it is considered that the users will have an overlap in the respective videos output by their respective application instances. In other words, it is considered that at least some of the time the videos displayed for each user would be similar or at least share a number of common elements. An example of this is when playing a game - each of the users may be provided with a view of a cutscene at the same time, or the users may have similar viewports and so have similar views within the game. The users may also view 11 the same HUD elements, such as a map of the game environment or a health bar of a common enemy being faced. Similarly, when viewing free-viewpoint media content the same scenarios may arise - such as if two users sit next to each other in an immersive sports stadium experience. Displaying such video side-by-side may be distracting to a user in some cases, while the rendering and transmission of duplicated content can be considered to be inefficient. These negative effects are amplified as the number of users increases; therefore while the below discussion primarily focuses upon an implementation in which only two users are interacting, it is considered that it would be desirable to extend these teachings to arrangements directed toward a greater number of users. It is therefore desirable to exploit the similarities between the respective video outputs where possible to improve efficiency of the system. Figure 7 schematically illustrates a method which seeks to achieve this aim. A step 700 comprises executing a first and second instance of an application. These instances may be executed at any suitable devices in accordance with the above discussion; for instance, one may be executed at a local processing device with another being executed remotely (such as at a second processing device associated with a second user, or at a server), or both may be executed remotely. The implementation of this method is able to be adapted to account for these different options without undue burden upon the skilled person. A step 710 comprises obtaining video output data from each of the first and second instances of the application. The video output data may comprise the video output itself; alternatively, or in addition, other data which characterises the video output or an application state corresponding to the video output may be obtained. For instance, information about the location and orientation of an in-game camera may be obtained, or information indicating the start / end of a cutscene. Information representing common elements between the instance's respective video outputs may also be obtained, such as identifying common HUD elements in a game and optionally values associated with those (such as identifying whether a map being displayed in each instance covers the same area). Such data may be obtained separately to the video, or it may be encoded as metadata alongside the video, for example. A step 720 comprises determining a degree of similarity between the video outputs in dependence upon the data obtained in step 710. This may be performed in any suitable manner for the data that is obtained; for instance, if the video output itself is obtained, then an image matching process may be used to compare image frames to determine similarity. Similarly, if information about a camera position / orientation is obtained, then these parameters may be compared to determine a similarity in the resulting view of a virtual environment depicted in the video outputs. In the case that cutscene start / end information is obtained from both application instances then it can be assumed that the threshold is exceeded without further processing (unless it is possible for the cutscenes to be different for each user, of course). The threshold degree of similarity may be determined in any suitable manner, and defined in accordance with the parameters being considered. A lower threshold may be associated with implementations in which the desire for processing or transmission efficiency is increased - such as when using a low-powered local processing device, or when a user's internet connection is not entirely reliable. In such cases, the users may be more willing to compromise on a shared viewpoint than they otherwise would be due to the desire for the resulting benefits. Similarly, a higher threshold may be associated with arrangements in which the users are not concerned about efficiency. The threshold may also be responsive to user preferences; in particular given that a shared viewpoint may be displayed in a format that is larger than either of the separate video contents would have been. This can therefore incentivise some users to allow the viewpoint to be shared with a lower threshold to improve their viewing experience. The threshold may also be content-specific; some content may lend itself to a shared viewpoint more readily (such as a third person co-operative game, in which users are likely to be near each other), while others are less so (such as a first person game). Some content may also be prone to errors - such as in a driving game, whereby the scenery viewed by both players may be similar despite the viewpoints being far apart, or in a game in which small differences in viewpoint can lead to significantly different views. A step 730 comprises modifying the display of the respective video outputs in response to at least a threshold degree of similarity being identified between them. In some implementations, this step comprises simply not displaying one of the two video outputs; this can be performed by the local processing device, or this can be effected before the local processing device receives the video outputs. For instance, only one of the video outputs may be transmitted (thereby improving transmission efficiency); alternatively one of the video outputs may not be generated at all (thereby also improving processing efficiency). In the case that one of the video outputs is not displayed by the local processing device, the other of the video outputs may be modified to account for this. For instance, the display size may be increased or the aspect ratio may be modified to make use of the space that would otherwise have been occupied by the video output no longer being displayed. This may be achieved by modifying the video output directly, or by modifying one or more parameters within the application to cause output video to be generated with the desired parameters. In the case that multiple display devices are used to display the video content at the local processing device (such as a dual-monitor setup, or each user wearing a respective HMD), the other video output may be duplicated and optionally modified where appropriate to enable its display on the other of the display devices. In some cases, the execution of one or both of the instances of the application may be modified in response to the determination that one of the video outputs should not be displayed. For example, the camera parameters may be modified in the instance being displayed so as to capture a larger field of view - this can be useful to capture any differences in viewpoint between the instances, so that there is no loss of visual content by not displaying the video output of the second instance. This can be implemented by defining camera parameters so as to capture a field of view which encompasses both of the respective fields of view of the two instances, for example. Similarly, the first instance can adjust its video parameters to increase the resolution or level of detail; this may be considered advantageous as in a typical arrangement the video output would be displayed with a larger size (as it would no longer be being presented in a split-screen manner). In some implementations, the instance of the application which is no longer having its video output viewed (or is not generating a video output) may be configured to adapt its output accordingly. This may include providing information to the other instance of the application to modify the generation of the output video, or generating alternative visual content that can be overlaid upon the other output video. For example, in a video game a player's health statistic may be output to enable a corresponding UI element to be generated by the other instance; alternatively, the UI element can be output directly and subsequently overlaid upon the other output video. More specific information may also be used to generate other visual content, such as information about an enemy being targeted by a user of the second (non-displayed) instance to enable a corresponding targeting graphic to be generated in the first instance. A step 740 comprises displaying the video output of a single instance of the application, this video output being the one generated by modification in step 730 where applicable. Implementations in accordance with the above method therefore enable an efficient manner of providing video content to users who are interacting with separate instances of content via a single local processing device. The above process focuses on the management of the visual component of video content; it would be understood that the corresponding audio component may be managed in any suitable manner independent of the management of the visual component. For instance, the audio associated with each of the instances may still be reproduced locally (particularly if the users have separate audio reproduction means, to avoid audio clash). Alternatively, the audio associated with a non-displayed video output may be omitted - in many cases it can be assumed that if the viewpoints are sufficiently 14 similar, then the audio would also be similar and as such comprise similarly high levels of redundant content. It is also considered that the non-displayed instance may output instance-specific audio elements (such as a low-health warning, or character-specific audio) that can be played alongside, or incorporated into, the audio for the displayed instance. Implementations of such a method may be performed for a range of different arrangements of processing devices, with the processing being performed by any of the processing devices as appropriate. Examples of such implementations include: 1. A first instance being executed locally by a local processing device (such as a games console), with the second being executed by a remote processing device (such as a second games console). 2. A first instance being executed locally by a local processing device, with the second being executed by a server. 3. A first instance being executed by a remote processing device or server, with the second being executed by a different remote processing device or a second server. 4. Both the first and second instances being executed by a server. A determination of which device should perform the processing can be made in dependence upon any technical considerations. For instance, it may be preferable for the processing to be performed by the device having the most spare processing capacity (typically the more powerful processing device, but this may not be the case when the instances of the application have different display settings or the like). Alternatively, it may be preferred that the device which executes the 'primary' instance performs the processing - the primary instance here is the one which is associated with the video output that is preserved, with the secondary instance being the one which is associated with the video output that is not displayed when the similarity conditions are met. The selection may also be made based upon the amount of latency that would be introduced by each option - with a lowest absolute latency, or a lowest latency between the two instances of the application, being considered desirable. As discussed above, in some cases it is preferable that a locally executed instance is designated as the primary instance; this can improve transmission efficiency by reducing the amount of data being transmitted in respect of the second instance. However, in other cases it may be preferable that the locally executed instance is designated as the secondary instance - this can reduce a processing burden upon the local processing device, which can lead to improved performance overall (as the primary instance may have higher quality video output if the other device / server has a higher processing capability) as well as preserving the battery life in the case that the local processing device is a portable one. In the case that the first and second instances are both being executed remotely to the local processing device, at least the determination of the degree of similarity may be performed by the local processing device. Based upon this, the local processing device may output information to one or both of the remote processing devices or servers to control the modifications to their operation or output. This may be preferred in implementations in which the remote processing devices / servers are not in communication with one another. In other cases, the first and second remote instances may be in communication with one another either directly or via a third process executed at one of the remote devices (or a different remote device). In such a case, the processing burden upon the local processing device can be reduced further; this may also reduce the amount of latency in the system due to the more direct communication. Figure 8 schematically system for providing a synchronised multi-user application experience at a local processing device. As noted above, implementations of such a system may be realised with a range of different arrangements of physical processing means (such as CPUs and / or GPUs located in multiple devices). The system comprises a first application processing unit 800, a second application processing unit 810, a similarity determining unit 820, a modification unit 830, and a display unit 840. Not shown in this Figure are the local processing device and any remote processing devices or servers; this is because the functional units shown may be distributed between such devices as appropriate for a given implementation. The first application processing unit 800 is configured to execute a first instance of the application responsive to inputs received from the user of a first input device associated with the local processing device. The first application processing unit 800 may be implemented by any suitable arrangement of hardware, such as a CPU and GPU in communication with one another, as may the second application processing unit 810. The second application processing unit 810 is configured to execute a second instance of the application responsive to inputs received from the user of a second input device associated with the local processing device, wherein at least one of the first and second application processing units is remote to the local processing device. In some implementations, the first application processing unit 800 is located at the local processing device and the second application processing unit 810 is located at a remote processing device or a server. Alternatively, the first application processing unit 800 is located at a remote processing device or a server, and the second application processing unit 810 is also located at a remote processing device or a server. In the latter case, the functionality of the first and second application processing units may be realised by the same server. The key feature regarding the first and second application processing units is that they execute separate instances of the same application in a multi-user configuration; as such, the specific locations of the instances are able to be selected freely while the inputs to both are still received by the local processing device shared by the users. The similarity determining unit 820 is configured to determine a degree of similarity between video outputs associated with each of the instances of the application. This can be performed directly by comparing the video outputs themselves (or representations thereof), more indirectly via a comparison of information about the video outputs or their content, or a combination of each. The threshold degree of similarity may be determined in dependence upon the content of one or both of the video outputs, such as based upon events that are occurring. For instance, the threshold degree of similarity may be lower when there is less action in the video output and higher when there is more action. Otherwise, or additionally, the threshold may be set based upon user preferences, a content creator, and / or processing / network capabilities, for instance. For instance, the similarity determining unit 820 may be configured to compare image frames of the respective video outputs to determine a degree of similarity, the image frames comprising pairs of image frames associated with the same display time in each video output. In other words, the image frames which would be displayed simultaneously can be compared to identify any visual similarity - this may be performed using any suitable method. For instance, one image may be subtracted from another to determine a residual (wherein a smaller residual indicates a high degree of similarity), edge detection may be performed on each image and the results compared, or a trained machine learning model may be used to detect similarity between images. Alternatively, or in addition, the similarity determining unit 820 may be configured to utilise information output by the first and / or second instances of the application to determine a degree of similarity, the information being indicative of one or more in-application parameters. For instance, an in-application parameter may be information about a camera viewpoint associated with the video output, or information about the proximity of these respective viewpoints for each video output. Information about the proximity of two user-controlled avatars in a game is another example, as proximity can be indicative of similar viewpoints (particularly in a third-person game). Another possible alternative or additional approach is one in which the similarity determining unit 820 is configured to utilise metadata associated with one or both of the video outputs to determine a degree of similarity, the metadata being indicative of the content of the respective video output. For instance, metadata may indicate the start or end of a cutscene in a game, or may be used to indicate information about the viewpoint or what is visible in the video. For instance, it may be determined that the video outputs are similar if the same objects (or at least the same significant objects, significance being determined based upon the application) are visible. The modification unit 830 is configured to, in response to at least a threshold degree of similarity being determined, cause a video output associated with the first or second instance of the application to no longer be displayed. The selection of which of the instances to no longer display the corresponding video output can be made freely by the skilled person in dependence upon the specific arrangement of the first and second application processing units and which benefit is sought - for instance, seeking to improve content transmission efficiency or to improve battery life of a local processing device. Rather than being limited only to the prevention of display of a particular video output, the modification unit 830 may be configured to modify the display of the remaining video output and optionally the execution of one or both of the application instances. For example, the modification unit 830 may be configured to modify the operation of the instance of the application corresponding to the displayed video output. This can include changing a viewpoint, for example, or generating new user interface (UI) elements to replace UI elements which would have been displayed in the other of the video outputs. The change in viewpoint may be to broaden the field of view, for example, or to otherwise adjust it so that the viewpoints of the two instances are both well-represented in the displayed video output. Alternatively, or in addition, the modification unit 830 may be configured to modify the video output to be displayed, the modification comprising a rescaling or upsampling of the video output. This change may be motivated by the expected increase in display size of the video to be displayed relative to when two video outputs were each displayed together. Such a change can therefore improve the display quality and / or viewing experience of the users. It is considered that the modification unit 830 may be responsive to user inputs in some implementations, such that the user is able to adjust the display of content. This can include resizing, reshaping, or rearranging content, for example, or selecting / deselecting particular elements for display (such as hiding HUD elements). Similarly, the modification unit is configured to overlay one or more elements on the video output to be displayed, the elements comprising one or more outputs of the instance of the application to no longer be displayed. For instance, the instance of the application which is to no longer have its video output displayed may be configured to output graphical elements representative of aspects of that instance of the application - such as a corresponding user's health bar in a game. These can be overlaid upon the video to be displayed either as a part of the execution of the corresponding instance of the application or at the local processing device, for instance. One or more audio elements may also be output for reproduction alongside the video output to be displayed as a part of this modification. The modification unit 830 may be configured to operate in response to the (at least) threshold degree of similarity being observed for at least a predetermined period of time. This period of time may be defined as a fixed number of seconds (or fractions of a second), for instance, or as a number of successive frames. It may be required that each frame within this period of time exhibits the threshold degree of similarity, or that at least a particular proportion of frames (such as seventy or ninety percent) do so. Similarly, it may be the case that it is required that no more than n successive frames exhibit a below-threshold degree of similarity within that time (n being an integer number of frames). The modification unit 830 may be configured to suspend operation in response to the degree of similarity falling below the threshold for at least a predetermined period of time. In other words, should the degree of similarity no longer meet the threshold the system may return to standard operation in which both the video outputs are displayed without modification. The predetermined period of time may be defined as a fixed number of seconds (or fractions of a second), for instance, or as a number of successive frames. The display unit 840 is configured to display a video output associated with the other of the first or second instance of the application at a display device associated with the local processing device. In some cases there may be multiple display devices associated with a single local processing device; in this case, the video output may be duplicated for each display or split across those displays as appropriate for a given implementation. The arrangement of Figure 8 is an example of a processor (for example, a GPU and / or CPU located in a games console or any other computing device) that is operable to provide a synchronised multi-user application experience at a local processing device, and in particular is operable to: execute a first instance of the application responsive to inputs received from the user of a first input device associated with the local processing device; execute a second instance of the application responsive to inputs received from the user of a second input device associated with the local processing device, wherein at least one of the first and second application instances is executed remotely to the local processing device; determine a degree of similarity between video outputs associated with each of the instances of the application; cause, in response to at least a threshold degree of similarity being determined, a video output associated with the first or second instance of the application to no longer be displayed; and display a video output associated with the other of the first or second instance of the application at a display device associated with the local processing device. Figure 9 schematically illustrates a method for providing a synchronised multi-user application experience at a local processing device. This method may be implemented by the system of Figure 8, for example, and in accordance with any of the implementations described above. A step 900 comprises executing a first instance of the application responsive to inputs received from the user of a first input device associated with the local processing device. A step 910 comprises executing a second instance of the application responsive to inputs received from the user of a second input device associated with the local processing device, wherein at least one of the first and second application instances is executed remotely to the local processing device. A step 920 comprises determining a degree of similarity between video outputs associated with each of the instances of the application. A step 930 comprises modifying the display, which includes causing, in response to at least a threshold degree of similarity being determined, a video output associated with the first or second instance of the application to no longer be displayed. A step 940 comprises displaying a video output associated with the other of the first or second instance of the application at a display device associated with the local processing device. The techniques described above may be implemented in hardware, software or combinations of the two. In the case that a software-controlled data processing apparatus is employed to implement one or more features of the embodiments, it will be appreciated that such software, and a storage or transmission medium such as a non-transitory machine-readable storage medium by which such software is provided, are also considered as embodiments of the disclosure. Thus, the foregoing discussion discloses and describes merely exemplary embodiments of the present invention. As will be understood by those skilled in the art, the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting of the scope of the invention, as well as other claims. The disclosure, including any readily discernible variants of the teachings herein, defines, in part, the scope of the foregoing claim terminology such that no inventive subject matter is dedicated to the public.

Claims

1. A system for providing a synchronised multi-user application experience at a local processing device, the system comprising:a first application processing unit configured to execute a first instance of the application responsive to inputs received from the user of a first input device associated with the local processing device;a second application processing unit configured to execute a second instance of the application responsive to inputs received from the user of a second input device associated with the local processing device, wherein at least one of the first and second application processing units is remote to the local processing device;a similarity determining unit configured to determine a degree of similarity between video outputs associated with each of the instances of the application;a modification unit configured to, in response to at least a threshold degree of similarity being determined, cause a video output associated with the first or second instance of the application to no longer be displayed; anda display unit configured to display a video output associated with the other of the first or second instance of the application at a display device associated with the local processing device.

2. A system according to claim 1, wherein the first application processing unit is located at the local processing device and the second application processing unit is located at a remote processing device or a server.

3. A system according to claim 1, wherein the first application processing unit is located at a remote processing device or a server, and the second application processing unit is located at a remote processing device or a server.

4. A system according to any preceding claim, wherein the similarity determining unit is configured to compare image frames of the respective video outputs to determine a degree of similarity, the image frames comprising pairs of image frames associated with the same display time in each video output.

5. A system according to any preceding claim, wherein the similarity determining unit is configured to utilise information output by the first and / or second instances of the application to determine a degree of similarity, the information being indicative of one or more in-application parameters.

6. A system according to any preceding claim, wherein the similarity determining unit is configured to utilise metadata associated with one or both of the video outputs to determine a degree of similarity, the metadata being indicative of the content of the respective video output.

7. A system according to any preceding claim, wherein the modification unit is configured to operate in response to the threshold degree of similarity being observed for at least a predetermined period of time.

8. A system according to any preceding claim, wherein the modification unit is configured to modify the operation of the instance of the application corresponding to the displayed video output.

9. A system according to any preceding claim, wherein the modification unit is configured to modify the video output to be displayed, the modification comprising a rescaling or upsampling of the video output.

10. A system according to any preceding claim, wherein the modification unit is configured to overlay one or more elements on the video output to be displayed, the elements comprising one or more outputs of the instance of the application to no longer be displayed.

11. A system according to any preceding claim, wherein the modification unit is configured to suspend operation in response to the degree of similarity falling below the threshold for at least a predetermined period of time.

12. A system according to any preceding claim, wherein the threshold degree of similarity is determined in dependence upon the content of one or both of the video outputs.

13. A method for providing a synchronised multi-user application experience at a local processing device, the method comprising:executing a first instance of the application responsive to inputs received from the user of a first input device associated with the local processing device;executing a second instance of the application responsive to inputs received from the user of a second input device associated with the local processing device, wherein at least one of the first and second application instances is executed remotely to the local processing device;determining a degree of similarity between video outputs associated with each of the instances of the application;causing, in response to at least a threshold degree of similarity being determined, a video output associated with the first or second instance of the application to no longer be displayed; and displaying a video output associated with the other of the first or second instance of the application at a display device associated with the local processing device.

14. Computer software comprising instructions which, when the software is executed by a computer, causes the computer to carry out the method of claim 13.

15. A non-transitory machine-readable storage medium which stores computer software accordingto claim 14.

Images