Multiplayer gaming system and method

Abstract

A system for providing a synchronised multi-user application experience at a local processing device, having: a processor executing a first instance of the application associated with a first user of

Description

BACKGROUND OF THE INVENTION Field of the invention This disclosure relates to a multiplayer gaming system and method. Description of the Prior Art The "background" description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present invention. With the increasing level of complexity and visual quality that are able to be provided in games, the demands upon processing hardware have increased significantly over time. For instance, more powerful processing hardware may be required to execute games, or limits may be placed upon the functionality of such games in order to enable them to be executed by a typical device. One example of this is multiplayer experiences that can only be experienced in an online or networked setting - rather than in so-called 'couch co-op' or split-screen modes within a game. While there is a desire to offer a more local experience, technical limitations of executing devices can impose a restriction. This technical limitation may be imposed by the hardware of a games console (which is largely standardised by the manufacturer), or by any other computing arrangement; for instance, content may be developed on the basis of known console capabilities or information about average or expected computing power available to eventual users. While in some cases a device may have sufficient computing power to provide a local multiplayer experience for a game (particularly in the case of a modern computing device executing an older game), many games may not be designed with this in mind. It is in the context of the above discussion that the present disclosure arises. SUMMARY OF THE INVENTION This disclosure is defined by claim 1. Further respective aspects and features of the disclosure are defined in the appended claims. It is to be understood that both the foregoing general description of the invention and the following detailed description are exemplary, but are not restrictive, of the invention. BRIEF DESCRIPTION OF THE DRAWINGS A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein: Figure 1 schematically illustrates an entertainment system; Figure 2 schematically illustrates a networked system; Figure 3 schematically illustrates a method; Figure 4 schematically illustrates a system for executing a video game; Figure 5 schematically illustrates a first processing device; Figure 6 schematically illustrates a method for executing a video game by a first processing device; Figure 7 schematically illustrates a system for providing a synchronised multi-user application experience at a local processing device; Figure 8 schematically illustrates an exemplary method which utilises the system of Figure 7; and Figure 9 schematically illustrates a method for providing a synchronised multi-user application experience at a local processing device. DESCRIPTION OF THE EMBODIMENTS Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views, embodiments of the present disclosure are described. Referring to Figure 1, an example of an entertainment system 10 is a computer or console. The entertainment system 10 comprises a central processor or CPU 20. The entertainment system also comprises a graphical processing unit or GPU 30, and RAM 40. Two or more of the CPU, GPU, and RAM may be integrated as a system on a chip (SoC). Further storage may be provided by a disk 50, either as an external or internal hard drive, or as an external solid state drive, or an internal solid state drive. The entertainment device may transmit or receive data via one or more data ports 60, such as a USB port, Ethernet® port, Wi-Fi® port, Bluetooth® port or similar, as appropriate. It may also optionally receive data via an optical drive 70. Audio / visual outputs from the entertainment device are typically provided through one or more A / V ports 90 or one or more of the data ports 60. Where components are not integrated, they may be connected as appropriate either by a dedicated data link or via a bus 100. An example of a device for displaying images output by the entertainment system is a head mounted display 'HMD' 120, worn by a user 1. Interaction with the system is typically provided using one or more handheld controllers 130, and / or one or more VR controllers (130A-L,R) in the case of the HMD. Figure 2 schematically illustrates a networked system in accordance with implementations of the present disclosure. In this Figure, a local device 200 is shown in communication, via a network represented by the line, with a remote device 210. The remote device 210 may be a processing device, such as a games console or personal computer, which is typically (but not required to be) located outside of the environment of the local device 200. This may include any location such as a different room or building; in some cases, the remote device 210 may be a processing device belonging to one of the users of the local device 200 and may be located accordingly at that user's house. Rather than being a device such as a games console, the remote device 210 may be a server which provides processing functionality so as to enable execution of a game instance at that server. The remote device 210 can therefore be any suitable device for executing a game instance and communicating with the local device 200 via a network connection. The local device 200 may comprise any computing arrangement that is communicable with a display (which may be integrated with or external to a processing device) for outputting video content to users. The local device 200 is also configured to receive inputs from control devices operated by those users, and is configured to receive data via a network connection such as a local area network or the internet. An example of this is the entertainment system 10 of Figure 1, which is able to output video to a display via the A / V port 90, can be utilised with controllers such as the handheld controller 130, and can receive data via the data port 60. Examples of suitable devices include games consoles, personal computers, televisions, mobile phones, and portable gaming devices. Implementations according to the present disclosure provide the ability for two or more users to interact with separate game instances via the same local device; this enables a local multiplayer experience to be provided for content for which this would otherwise not be an option for that local device - typically due to technical constraints such as limited available processing power. At least one of these instances is executed remotely to that local device, with a network connection being used to transmit images, audio, and / or data to the local device. The present disclosure refers to games as an example of an implementation to aid the clarity of the reader's understanding; however it should be appreciated that the techniques described in this disclosure can be equally applied to any other suitable application in which multiple users may wish to participate simultaneously. This may include applications such as media applications, for example, such as applications which enable users to access free viewpoint video content - each of a plurality of users may wish to view the content from their own viewpoint, thereby causing a user desire for a multi-user arrangement. Figure 3 schematically illustrates a general method in accordance with this function. While the steps are shown in a particular order, this should not be regarded as limiting; it will be appreciated that these may be performed in any suitable order, with some steps being performed substantially simultaneously. For instance, the second instance may be initiated before the first instance, or the second player may be added whilst the execution of the second instance is initiated. A step 300 comprises executing the first instance of a game at a local processing device (that is, the device with which the users are directly interacting - such as a games console in the room with them). This instance is executed in a single player mode, in that only a single player is able to provide inputs to control the execution of that instance. A step 310 comprises adding a second user to the game; this can be in response to inputs from the user of the first instance of the game, a request from the second user, and / or a combination of the two (such as an invitation-based implementation). At this stage, this may comprise associating a user profile of the second user with the game or inserting their user avatar into a game - adding the second user is not taken to mean that the second user is able to interact with the first instance, and in the case that the second instance is not currently being executed no functionality may be available to the second user initially. Adding a second user to the game means that the first game instance and the second game instance will each provide interactivity with a shared game environment - for instance, meaning that the first user's avatar and the second user's avatar are present in the same game environment. A step 320 comprises executing a second instance of the game. This is performed by a remote device (that is, not the local processing device); while this device is typically expected to be remote in the sense that it is not present in the environment of the local device, this is not a requirement and remote may be taken to mean 'separate' in that the second instance is implemented by a device which is independent of the local device. As discussed above with reference to the remote device 210 of Figure 2, the second instance may be implemented by a second games console or a cloud gaming server, for example. A step 330 comprises transmitting an output from the second instance; this output may comprise any suitable data or content as appropriate for a given implementation. For instance, data regarding an avatar's location or interactions may be output, or the results of physics simulations associated with the second user's actions within the second instance may be output to the first instance to enable elements of the second instance to be incorporated into the first instance. Alternatively, or in addition, the output from the second instance comprises video and / or audio of the second instance - such as the rendered video showing the second user's interactions with the second instance. In the case that no video is output by the second instance, the execution of the second instance may be modified so as to not render any images - this can reduce a processing burden upon the remote device, enabling implementation by a device with reduced processing power and / or improving the energy efficiency of such an arrangement. A step 340 comprises interacting with each instance, with the interactions being controlled by users operating respective control devices which each provide inputs to the local device. In the case of the second user's inputs, these are transmitted to the remote device to allow the second instance to be controlled. The local device is configured to display the results of the interactions with the two separate instances of the games; this can be achieved in any suitable manner. For example, in some cases it may be considered appropriate to present the video output of each instance in a split-screen mode such that each instance is shown in a spatially distinct manner. This may be achieved by executing the first instance of the video game locally to generate a video output, while decoding video received from the remote device which comprises the output of the second instance of the video game. Alternatively, the first instance may be updated based upon the output of the second instance so as to represent both instances. This can comprise a shared screen for both players, so that both appear to be within the same game instance - with both player characters appearing within the same camera view, for instance. For example, an object in the first instance may move in dependence upon physics simulations performed by the second instance, with the results of those simulations (or movement information for an object, for example) being output by the second instance for use by the first instance. Implementations in accordance with the method described above can therefore provide a local multiplayer gaming experience while utilising multiple gaming instances executed by different devices. While the above has been described in the context of two players being provided with a gaming experience, it is considered that this could be extended. For example, in the case that each game instance supports four players, two instances could be used to provide an (up to) eight player gaming experience in the same manner. Similarly, it is considered that a greater number of instances of a game could be utilised in combination so as to provide a gaming experience for a greater number of players. Of course, a combination of the two could also be utilised in which three or more game instances each able to support two or more players is used. In any case, the result is achieved in which more players than would otherwise be able to play a game locally are able to play via a single device despite technical limitations. Figure 4 schematically illustrates a system for executing a video game in accordance with the discussion provided above. The system comprises two or more control devices 400, a first processing device 410, a second processing device 420, and a display device 430. While shown here as a part of the system, the second processing device 420 is typically located apart from the other elements and as such can be considered to be distinct from the system as an external unit which provides inputs to the system of the elements 400, 410, and 430. The two or more control devices 400 may include any suitable input devices which enable users to provide inputs to control processing; these may be the control devices 130 or 130A of Figure 1, for example. In some cases the inputs from users may not be button presses or movements of a control device; for instance, gestures or audio inputs. In such a case, the control devices 400 may be any suitable hardware which enables the capture of these inputs such as cameras and / or microphones. The two or more control devices are associated with respective users, such that at least a first control device is associated with a first user and a second control device is associated with a second user. Here, associated may be taken to mean that they are operated by that user; however in some cases further functionality may be considered such as a control device 400 being logged in or registered with a particular user's profile or the like. The first processing device 410 is configured to receive inputs from each of the two or more control devices and to execute a first instance of a video game; the functionality of this device 410 is discussed in more detail below with reference to Figure 5. In a typical implementation, the first processing device 410 may be embodied by a games console or other local processing device; however in some cases it may be preferable that a cloud gaming service is utilised to provide this functionality. In such a case, a local device may be provided which is configured to communicate with the server so as to transmit inputs received from the controllers and to receive video of gameplay for display. The first processing device 410 is further configured to output images for display to the first and second users, the images being generated in dependence upon both the first and second instances of a video game. This may be images generated by the first instance in dependence upon data output by the second instance, or may include images generated by both the first and second instances. The first processing device may also be configured to output audio for at least one of the instances of the video 6 game; in some implementations this may include outputting audio for each instance of the video game via a different respective audio channel such that each user can be provided with audio for a corresponding instance of the video game. The second processing device 420 is configured to execute the second instance of the video game responsive to inputs from the second control device, with these inputs being provided to the second processing device 420 via the first processing device 410. The second processing device 420 is a separate device to the first processing device 410, with the two being in communicably connected via a network connection. The second processing device 420 may be a games console remote to the first processing device 410, for example, or a cloud-based processing arrangement (cloud gaming server) which is configured to execute an instance of a video game. The display device 430 configured to display images generated by the first processing device 410 to both the first and the second user, with the displayed images being dependent upon both the first and second instances of the video game. Turning to Figure 5, this Figure schematically illustrates a configuration of the first processing device 410 of Figure 4 in more detail. The processing device 410 comprises a processor 500, a communication unit 510, an input control unit 520, and an image generation unit 530. These functions may be implemented by the CPU 20, GPU 30, and data port 60 of the entertainment system 10 of Figure 1, for example; however any suitable processing hardware may be used to realise this functionality. The processor 500 is configured to execute a first instance of the video game responsive to inputs received from the first control device 400, such as those to control a user's avatar within the game environment. These inputs may include those which enable a second user to join the game via the second instance, such as issuing an invitation to that second user or configuring the first instance to enable other users to join. The processor 500 may be configured to adapt one or more settings of the first instance of the video game in dependence upon one or more parameters (such as video settings) associated with the second instance of the video game, the output video of the second instance of the video game (should this be provided by the second processing device 420), and / or properties of the network connection between the first processing device 410 and the second processing device 420. This can enable the presentation of the first instance of the video game to be in keeping with (that is, appearing similar or the same) that of the second instance (or the expected presentation of the second instance, in the case that it is not displayed); this can be particularly desirable in an implementation in which both instances are displayed simultaneously in a split-screen fashion. In the case in which only the first instance is displayed, this may still provide advantages in providing video content that accounts for the display settings of the second 7 instance such as brightness which may be important considerations for ensuring user comfort and content visibility. The processor 500 may also, or instead, be configured to modify a camera viewpoint associated with the first instance of the video game in dependence upon an output by the second instance of the video game. For instance, based upon data indicating the location of the second user's avatar in the second instance of the video game a camera viewpoint may be adjusted to ensure that both user's avatars are visible in the same image generated from the first instance of the video game. This can aid an implementation in which a single image is displayed to the users which is representative of the gameplay of both users. In some cases, it is considered that based upon such data the output video may be switched between split-screen and a single image in dependence upon a threshold distance between the first and second user's avatars such that when the threshold distance is exceeded the display is changed to a split-screen view. In some implementations the processor 500 may be configured to identify a latency associated with the receiving of data from the second processing device (such as a latency associated with the network connection and / or a processing time in transmitting inputs from the first processing device 410 to the second processing device 420), and to apply an input latency and / or display latency to the execution of the first instance in dependence upon this. In other words, the processing of the first instance can be adapted to provide an equal (or at least similar) latency to that of the second instance so that each user is able to interact with their respective game instances in a mutually consistent manner. An input latency refers to delaying the provision of the inputs to the game, whilst a display latency refers to delaying the display of images of the game to the user. The introduced latency may be a fixed value which is representative of an average or expected latency, or it may be responsive to live measurements of said latency. The processor 500 may be further configured to adapt settings associated with the first instance of the video game and / or instruct the adapting of settings of the second instance of the video game so as to manage a local processing load or the like. For instance, some hardware may find that executing a game while decoding received video content represents a significant processing burden - in such a case, the video quality of either instance (or indeed both instances) may be modified so as to reduce this burden and ensure that processing can be effectively managed. The communication unit 510 is configured to receive data from a second processing device 420 via a network, the data corresponding to a second instance of the video game being executed concurrently with the first instance. The communication unit 510 may be further configured to perform other communications, such as the transmission of inputs described with reference to the input control unit 520. In some implementations, the communication unit 510 is configured to transmit information to the second processing device 420 comprising information about the initiation of a game session - such as a location of the first user in the in-game environment, or other game state information such as a current stage, user loadout, and quest. The data corresponding to the second instance of the video game may be in any suitable format. In some implementations, the communication unit 510 is configured to receive data comprising output video of the second instance of the video game (optionally with the associated audio). Alternatively, or in addition, the communication unit 510 is configured to receive data comprising the results of one or more simulations (such as physics simulations for in-game interactions by the second user) performed by the second instance of the video game, wherein the results of the one or more simulations are provided to the first instance of the video game. The communication unit 510 can be considered optional, as a number of different implementations may not require any communication - such as when the two game instances are being executed locally by the processor 500 of the first processing device 410. In such a case, the two game instances may be configured to communicate directly without the need of a separate communication unit as is the case when the second instance is being executed by a remote server or the like. In such implementations, the functionality of the second processing device is realised by the first processing device as appropriate to provide two separate instances of the same game using a single device. The input control unit 520 is configured to provide inputs received from the first control device to the first instance of the video game, and to transmit inputs received from the second control device to the second processing device via the network connection. This may be performed by an in-game function associated with the first instance of the video game (or a separate game-specific tool which is executed alongside the video game), or it may be handled externally to the game such as by a system-level function provided by an operating system run by the first processing device 410. The image generation unit 530 is configured to generate images for display in dependence upon both the first and second instances of the video game, with these images being provided for output to both the first and second user by the display device 430 of Figure 4. In some implementations, the image generation unit 530 is configured to generate a split-screen image comprising output video of each of the first and second instances of the video game. However, this is not considered to be limiting; in some cases it may be preferable that the first instance of the video is used to generate images for display which are representative of the gameplay of both users. It is also envisaged that the format is a dynamic one which is responsive to in-game events or conditions - such as based upon user proximity in the game environment such that as the users move apart a split-screen is preferred, or switching to a single screen during cut-scenes or the like. As described above, in the exemplary implementations discussed each of the first instance and second instance of the video game are capable of supporting a single player only. However, it is considered that the same techniques may be extended to any case in which the number of users exceeds the number of users that a single instance of a video game is capable of supporting. Similarly, the number of processing devices is not limited to two, but could be increased to any suitable number. While discussed above with the first instance of the video game being executed locally, it is also considered that the first processing device could be implemented as a thin client or the like which decodes video received from two remote game instances. This may be particularly suitable for low-powered devices such as mobile phones or portable gaming consoles. This thin client may comprise the communication unit 510, input control unit 520, and image generation unit 530 whilst the functionality of the processor 500 is provided remotely (such as by a games console or cloud gaming server). As such, the thin client is configured to receive inputs from the two or more control devices, route these to the appropriate game instances, and receive video which is to be displayed to the users who are local to the thin client. The arrangement of Figure 4 is an example of a system for executing a video game which comprises a first processing device configured to receive inputs from each of two or more control devices associated with respective users, comprising a first control device associated with a first user and a second control device associated with a second user, the first processing device being communicably connected with a second processing device. Figure 5, which illustrates the first processing device, is an example of an arrangement which can be implemented using a processor (for example, a GPU and / or CPU located in a games console or any other computing device) that is operable to receive inputs from each of the two or more control devices to control respective instances of a video game, and in particular is operable to: execute a first instance of the video game at the first processing device; receive data from a second processing device via a network, the data corresponding to a second instance of the video game being executed concurrently with the first instance; provide inputs received from the first control device to the first instance of the video game; transmit inputs received from the second control device to the second processing device; generate images for display in dependence upon both the first and second instances of the video game; and display the generated images to both the first and the second user via the same display device, wherein the first instance of the video game is responsive to inputs received from the first control device, and the second instance of the video game is responsive to inputs received from the second control device. This functionality may be provided in accordance with any of the hardware configurations described elsewhere in this disclosure; for instance, the first processing device may be implemented as the entertainment system 10 of Figure 1. In this case, processing is performed by the CPU 20 and / or GPU 30. Figure 6 schematically illustrates a method for executing a video game by a first processing device configured to receive inputs from each of two or more control devices associated with respective users, comprising a first control device associated with a first user and a second control device associated with a second user. A first instance of the video game is responsive to inputs received from the first control device, and a second instance of the video game is responsive to inputs received from the second control device. A step 600 comprises executing a first instance of the video game at the first processing device. A step 610 comprises receiving data from a second processing device via a network, the data corresponding to a second instance of the video game being executed concurrently with the first instance. A step 620 comprises managing inputs received from the two or more control devices, with the management comprising providing inputs received from the first control device to the first instance of the video game and transmitting inputs received from the second control device to the second processing device. A step 630 comprises generating images for display in dependence upon both the first and second instances of the video game. A step 640 comprises displaying the generated images to both the first and the second user via the same display device. When providing content in the manner described above, such that two or more separate applications (such as video games) are executed to generate a split-screen multi-user experience, it is considered important that the separate applications are able to remain synchronised. Due to the split-screen nature of the display, a lack of synchronisation can be jarring to a user (for instance, if both view the same video content with a time offset from one another) for example, or can cause one of the users to have a competitive advantage when the application is a game. In some cases there can be a loss of immersion 11 in an application if both users are experiencing events at different times on the same display - as such, preserving a synchronised execution of the two instances is considered to provide significant advantages. Such a problem is not encountered in typical multi-user applications, as these are provided using either a single instance of an application or a server is configured to host a game separately to the game instances. While in such cases it may be desirable to reduce latency, this is to increase responsiveness rather than to improve synchronisation (which requires the latency to be similar, rather than necessarily minimised). In implementations according to the present disclosure, such a synchronisation is realised via the detection of events associated with the applications being executed; this may be performed by the first instance of the game, for example, or may be implemented using a separate agent / background process or a system-level implementation for a given device. This may be at a device which is local to the user, or may be implemented remotely in the case that both application instances are hosted by a cloud processing arrangement. Figure 7 provides a schematic illustration of an arrangement configured to perform such a synchronisation process. Figure 7 schematically illustrates a system for providing a synchronised multi-user application experience (such as a multiplayer game, or a multi-user multimedia experience) at a local processing device, the system comprising a processor 700, a communication unit 710, an identification unit 720, an analysis unit 730, and a modification unit 740. These functional units may be implemented by a processor (such as a CPU and / or GPU) at the local processing device in some implementations; however it is also considered that processing resources at a remote server or a remote processing device may be used - particularly in the case in which the first and second instances of the application are both executed remotely to the local processing device. In some implementations, it is considered that the processing may be divided between such devices; for instance, the processor 700 may be realised by a remote CPU whilst the remaining units are realised by a CPU of the local processing device. The processor 700 is configured to execute a first instance of the application, the first instance of the application being associated with a first user of the local processing device; as discussed above, the application may be a video game, or another application such as a media application. The application may be associated with the first user by virtue of being executed by that user at a local device, for example, or the first user's account being used when executing the application remotely. In any case, the first instance of the application is controlled by the first user via a corresponding input device. In implementations in which the first instance of the application is executed remotely, but in which other functionality is realised locally, a corresponding communication unit may be configured to output 12 the results of the execution (such as a generated video of the application being interacted with, and any desired metadata or the like) to the local processing device for the subsequent processing to be performed (such as the identification of the synchronisation point). The communication unit 710 is configured to receive data output by a second instance of the application being executed by a remote processing device, such as a server or a remotely-located equivalent to the first processing device, the second instance of the application being associated with a second user of the local processing device. As above, the second instance of the application is associated with the second user in that it is the inputs provided by the second user (via a corresponding control device) that controls processing of the second instance of the application. This correspondence may be realised via a user account of the second user being logged into the local processing device, for instance, or the control device may be identifiable as belonging to that user. The received data may comprise information in any suitable format; this can include video and / or audio associated with the second instance of the application, for example, and / or additional data such as event information, interaction information, and / or the values of one or more in-application parameters identified from the second instance of the application. In the case in which the application is a video game, this may be game state information or parameters such as a character's health, for example, location information for a particular element, or information about actions taken by the character controlled by the second user. The identification unit 720 is configured to identify a synchronisation point within the first instance of the application in dependence upon application state data and / or user input data associated with the first user. The synchronisation point may be any identifiable time in the processing of the application that can be used to determine whether the two instances of the application are synchronised. For instance, if two users are interacting with a media application providing respective views of a football match an example of a synchronisation point is the scoring of a goal or the half time whistle being blown. These events should be identifiable in both instances, and are both associated with an objective time; as such, any time offset between the instances would be able to be identified based upon consideration of such events. In such a case, the synchronisation point may be identified as the occurrence of a predetermined event within the first instance of the application; this may be predetermined in the sense that it is scripted (such as the appearance of a boss in a game), or predetermined in that the nature of the event is defined in advance (such as the scoring of a goal in a live football match). In some cases, the synchronisation point may be identified in response to a cut-scene to be reproduced in both the first and second instances of the application. In the case in which multiple synchronisation points are identified within a single session of executing the application (which is typical during an extended session, so as to ensure that the synchronisation is maintained), each of these methods of identifying a synchronisation point may be utilised as desired - it is not necessary that the same approach is used for each identification of a synchronisation point. Synchronisation points may also be identified in response to a predefined period of time having elapsed within the first instance of the application; for instance, after an elapsed time the application state of the first instance may be recorded. This is an alternative to the event-driven approach, in that synchronisation points can be generated at regular (or at least known) intervals independent of any particular occurrences in the application. The application state may be recorded based upon data output by the application, or may be inferred from the video output (such as identifying visible elements in a given configuration, and later seeking this in a video output of the second instance). In addition to identifying a synchronisation point, it may be advantageous to characterise the synchronisation point so as to ensure that the same point is being identified in both instances of the application. For instance, if two goals were scored in a football match in quick succession it would be helpful to identify the scoreline at each point to ensure that the correct goal is being considered for synchronisation purposes. To this end, synchronisation points may be characterised by the occurrence of an event or interaction within the first instance of the application and / or the values of one or more in-application parameters identified from the first instance of the application. These are effectively parameters within the application which enable similar synchronisation points to be distinguished from another, and as such may comprise any information which is more specific to the user's interactions with the application instance than the definition of the synchronisation point. Synchronisation points may be identified on the basis of any suitable information associated with the first instance of the application. For instance, the application state data may comprise an event log, in which the occurrence of an event is considered to represent the state of an application at that time. User input data may be provided by the first instance of the application, and may be translated into inapplication actions such as 'user jumped'; alternatively, the user input data may be recorded at the time the user provides the inputs to the local processing device. Similarly, this information may be derived from video / audio output associated with the first instance of the application (such that the identification unit 720 may be configured to identify a synchronisation point in dependence upon a video and / or audio output associated with the first instance of the application), either using an image recognition process (such as recognising a particular element being displayed) or a machine learning based process in which game context can be inferred from video content. The analysis unit 730 is configured to analyse the received data to identify a corresponding synchronisation point associated with the second instance of the application, and to calculate a current temporal offset between the first and second instances of the application in dependence upon the respective identified synchronisation points. In other words, the timing of a synchronisation event is identified in both the first and second instances and the difference in these times is determined to be the current temporal offset. This temporal offset may be determined on the basis of synchronised clocks at each of the respective processing devices, or may be based upon a clock at the device implementing the analysis unit. In the latter case, this enables the device to determine the relative timing in a manner that accounts for latency due to transmission of the output of the second instance of the application (and in some cases, the first instance of the application should both be executed remotely to the analysis unit 730). The current temporal offset may be tracked over time, so as to generate a temporal offset history which can be used to determine how stable the temporal offset is, or how it changes over time - this can be used to inform a more appropriate modification selection. When identifying the corresponding synchronisation point in the second instance of the application, any information output by the second instance of the application may be utilised to identify the occurrence of the same event or the same set of parameters (for example). In other words, the information generated by identifying (and optionally characterising) the synchronisation point in the first instance of the application is used to identify a corresponding point in the second instance of the application from the information output by that instance. In the case that video and / or audio is output by the second instance of the application, the analysis unit 730 may be configured to perform a respective processing on the video and / or audio to identify the corresponding synchronisation point. This may be implemented as an image search for known events or features within that video content, for example, based upon the information identifying / characterising the synchronisation point. For instance, if the synchronisation point is based upon an event in which a boss appears then the corresponding synchronisation point may be identified based upon the identification of an image of that boss in the video output by the second instance of the application. The modification unit 740 is configured to apply a modification to the first instance of the application and / or transmit information regarding a modification of the second instance of the application to the remote processing device (that is, the device executing the second instance of the application), wherein the modification is determined in dependence upon the calculated temporal offset and, when applied, the modification causes the temporal offset between the first and second instances of the application to be reduced. In the case that the first instance of the application is executed remotely to the modification unit 740, the modification unit 740 may instead transmit information regarding a modification of the first instance of the application to the corresponding processing device. The modification may comprise increasing or decreasing a speed associated with a given instance of the application, such as causing a game to run at a higher speed. Alternatively, or in addition, the modification may comprise adding an artificial latency to the execution and / or display of a given instance of the application - this can be implemented by adding an artificial latency to controller inputs, for instance, or by delaying the display of image frames after they have been received / rendered. A further alternative, or additional, option is that of pausing the execution of a given instance of the application for a period equal to the calculated temporal offset; this can enable the other of the instances to 'catch up', at which point execution of that instance continues normally (or subject to other modifications so as to reduce the chance of losing synchronisation again during further execution). The magnitude and / or duration of the modification may be dependent upon the magnitude of the calculated temporal offset; alternatively, or in addition, the expected rate of change of temporal offset caused by the modification may be considered. For instance, it may be preferred that the degree of synchronisation is improved within a threshold amount of time (such as a predetermined number of seconds) and the magnitude of the offset is determined accordingly. This can therefore balance the desire for the instances to be synchronised with the impact of performing the synchronisation - a reduced level of increase in the game speed for a longer period may be preferable to a higher level of increase for a shorter period in respect of its impact upon gameplay, for example. Of course, in some instances a modification is applied only in the case that the calculated temporal offset exceeds a threshold value - a small temporal offset may not be noticeable by a user, and as such it may be more efficient to not apply any modifications in such a case. While not shown in Figure 7, it is considered that the system may also comprise the remote processing device configured to execute the second instance of the application and to apply a modification to the second instance of the application in response to receiving information regarding a modification of the second instance of the application from the local processing device. As discussed above, the hardware used to execute instances of the application in accordance with implementations of the present disclosure may include: • A local processing device (that is, a device which is in the same physical environment as the first and second users) which executes the first instance application, and a remote processing device which executes the second instance of the application; • A local processing device which obtains the results of execution of the first instance at a server, and a remote processing device which executes the second instance of the application; • A local processing device which obtains the results of execution of the first instance at a first remote processing device, and obtains the results of execution of the second instance at a second remote processing device (different to the first remote processing device); and • A local processing device which obtains the results of execution of the first instance at a server, and obtains the results of execution of the second instance at a server (which may or may not be the same server). In each case the distribution of processing (that is, the location of the functional units of Figure 7) may be determined freely as appropriate for that given implementation; there is no requirement for the local processing device to perform specific processing tasks in each arrangement, for instance. Figure 8 schematically illustrates an exemplary method which utilises the system of Figure 7; this is in the context of a video game, but it is intended that a wide range of applications could be used rather than being limited only to video games. Figure 7 is discussed in the context of the first of the options for hardware arrangements which may be used, as described above. A step 800 comprises executing a first instance of a game at a local processing device such as a games console; this is a processing device which is interacted with by two users via respective control devices (or other inputs, such as gesture-based inputs). The first instance of the game is controlled in dependence upon inputs from the first user, and not controlled in dependence upon inputs from the second user. A step 810 comprises transmitting the inputs provided by the second user (via the second control device) to a remote processing device (such as a second games console) to control processing of a second instance of the same game. The remote processing device is one which is not directly interacted with by a user; this may be located elsewhere in the same room (in that it is remote to the first processing device), or may be located further away (such as in a different room in the same building, or at the home of the second user). The two processing devices may communicate with one another in any suitable manner; typically this is via an internet connection, either directly or via a server which acts as an intermediary. A step 820 comprises receiving, at the first processing device and from the remote processing device, an output of the second instance of the game. This may include any suitable information as appropriate for enabling the first instance of the game to incorporate elements from the second instance, as described above with reference to Figure 3. Further data to assist with determining the degree of synchronisation may also be provided, such as an event log with time stamps, although this may be performed on the basis of the information (such as a video output) which is usually transmitted. A step 830 comprises identifying a degree of synchronisation between the two instances of the game, based upon information about the first instance and the information received about the second instance in step 820. This step can be implemented in a number of ways; for instance, comparing the relative timing of the start of a cutscene in the respective game instances, comparing the relative timing of an event, or comparing the relative timing between two identical game states. Other options for determining the degree of synchronisation may also be considered, such as identifying a first user's inputs to the first instance and determining a delay before their effect in the second instance of the game is realised. A step 840 comprises modifying the execution of one or both of the instances of the game so as to increase the degree of synchronisation between the two instances of the game (that is, to reduce any temporal offset between them). A game speed in one or both instances may be increased / decreased as appropriate; this may increase the game speed of the 'lagging' (that is, delayed) instance, reduce the game speed of the leading game instance, or both with a reduced magnitude. In some cases, the display of video corresponding to the first instance may be delayed (assuming this is the leading game instance) so as to cause it to be synchronised with the display of video of the second instance received from the remote processing device. By implementing such a method, the two instances of the game can be presented in a more synchronised manner (that is, with a reduced temporal offset from one another) in an effective and efficient manner despite neither game having a traditional multiplayer implementation. As discussed above, a machine learning based process in which game context can be inferred from video content may be utilised to aid in identifying synchronisation points within the second instance of an application without requiring additional data to be provided alongside the video. This may be implemented in any suitable manner, in dependence upon how the synchronisation points are to be defined. For instance, a machine vision process may be utilised which has been trained to recognise key elements within images - these elements may be a given subset from amongst the game assets which are considered to have some significance or at least a given rarity (to enable them to be distinguishing). For instance, the machine vision process may be configured to recognise rare enemies, loading screens, effects (such as a screen flash when damage is taken), or any other feature which can be indicative of a particular moment in time. The process may be configured to analyse images from both instances, with the respective times of recognition of an element in each being used to determine a temporal offset. Alternatively, based upon combinations of key elements being identified an event can be identified -this can be trained based upon predefined relationships, or from training data which comprises videos 18 of a given event that can be processed to determine common elements. Similarly, based upon an output of the first instance of the application the process can be configured to search for particular elements rather than identifying any key elements. Machine learning may also be useful to predict an expected view in the second instance based upon information output by the first instance - for instance, to account for a different viewpoint within the application, different display settings, or differences in how events are shown (such as if players are on different teams in a game, so events may be reported differently as they may be positive for one player but negative for the other). A model may be trained based upon labelled pairs (or larger sets) of videos of the same events occurring in an application being executed in multiple instances; based upon this, the model can learn how events in a first instance will impact the view in other instances - and therefore corresponding events can be identified and used as synchronisation points. Figure 9 schematically illustrates a method for providing a synchronised multi-user application experience at a local processing device. This may be implemented in accordance with the discussion of the hardware of Figure 7, for instance, as exemplified by the method of Figure 8. A step 900 comprises executing a first instance of the application, the first instance of the application being associated with a first user of the local processing device. A step 910 comprises receiving data output by a second instance of the application being executed by a remote processing device, the second instance of the application being associated with a second user of the local processing device. A step 920 comprises identifying a synchronisation point within the first instance of the application in dependence upon application state data and / or user input data associated with the first user. A step 930 comprises analysing the received data to identify a corresponding synchronisation point associated with the second instance of the application. A step 940 comprises calculating a current temporal offset between the first and second instances of the application in dependence upon the respective identified synchronisation points. A step 950 comprises applying a modification to the first instance of the application and / or transmitting information regarding a modification of the second instance of the application to the remote processing device, wherein the modification is determined in dependence upon the calculated temporal offset and, when applied, the modification causes the temporal offset between the first and second instances of the application to be reduced. The techniques described above may be implemented in hardware, software or combinations of the two. In the case that a software-controlled data processing apparatus is employed to implement one or more features of the embodiments, it will be appreciated that such software, and a storage or transmission medium such as a non-transitory machine-readable storage medium by which such 5 software is provided, are also considered as embodiments of the disclosure. Thus, the foregoing discussion discloses and describes merely exemplary embodiments of the present invention. As will be understood by those skilled in the art, the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting of the scope of the 10 invention, as well as other claims. The disclosure, including any readily discernible variants of the teachings herein, defines, in part, the scope of the foregoing claim terminology such that no inventive subject matter is dedicated to the public.

Claims

1. A system for providing a synchronised multi-user application experience at a local processing device, the system comprising:a processor configured to execute a first instance of the application, the first instance of the application being associated with a first user of the local processing device;a communication unit configured to receive data output by a second instance of the application being executed by a remote processing device, the second instance of the application being associated with a second user of the local processing device;an identification unit configured to identify a synchronisation point within the first instance of the application in dependence upon application state data and / or user input data associated with the first user;an analysis unit configured to analyse the received data to identify a corresponding synchronisation point associated with the second instance of the application, and to calculate a current temporal offset between the first and second instances of the application in dependence upon the respective identified synchronisation points; anda modification unit configured to apply a modification to the first instance of the application and / or transmit information regarding a modification of the second instance of the application to the remote processing device, wherein the modification is determined in dependence upon the calculated temporal offset and, when applied, the modification causes the temporal offset between the first and second instances of the application to be reduced.

2. A system according to claim 1, wherein the synchronisation point is identified as the occurrence of a predetermined event within the first instance of the application.

3. A system according to claim 1, wherein the synchronisation point is identified in response to a predefined period of time having elapsed within the first instance of the application.

4. A system according to any preceding claim, wherein the synchronisation point is identified in response to a cut-scene to be reproduced in both the first and second instances of the application.

5. A system according to any preceding claim, wherein the synchronisation point is characterised by the occurrence of an event or interaction within the first instance of the application and / or the values of one or more in-application parameters identified from the first instance of the application.

6. A system according to any preceding claim, wherein the identification unit is configured to identify a synchronisation point in dependence upon a video output associated with the first instance of the application.

7. A system according to any preceding claim, wherein the received data comprises video and / or audio associated with the second instance of the application and wherein the analysis unit is configured to perform a respective processing on the video and / or audio to identify the corresponding synchronisation point.

8. A system according to any preceding claim, wherein the received data comprises data output by the second instance of the application, including one or more of event information, interaction information, and / or the values of one or more in-application parameters identified from the second instance of the application.

9. A system according to any preceding claim, wherein a modification may comprise increasing or decreasing a speed associated with a given instance of the application, adding an artificial latency to the execution and / or display of a given instance of the application, and / or pausing the execution of a given instance of the application.

10. A system according to any preceding claim, wherein the magnitude and / or duration of a modification is dependent upon the expected rate of change of temporal offset caused by the modification.

11. A system according to any preceding claim, wherein a modification is applied only in the case that the calculated temporal offset exceeds a threshold value.

12. A system according to any preceding claim, comprising the remote processing device configured to execute the second instance of the application and to apply a modification to the second instance of the application in response to receiving information regarding a modification of the second instance of the application from the local processing device.

13. A method for providing a synchronised multi-user application experience at a local processing device, the method comprising:executing a first instance of the application, the first instance of the application being associated with a first user of the local processing device;receiving data output by a second instance of the application being executed by a remote processing device, the second instance of the application being associated with a second user of the local processing device;identifying a synchronisation point within the first instance of the application in dependence upon application state data and / or user input data associated with the first user;analysing the received data to identify a corresponding synchronisation point associated with the second instance of the application;calculating a current temporal offset between the first and second instances of the applicationin dependence upon the respective identified synchronisation points; andapplying a modification to the first instance of the application and / or transmitting information regarding a modification of the second instance of the application to the remote processing device,5 wherein the modification is determined in dependence upon the calculated temporal offset and, when applied, the modification causes the temporal offset between the first and second instances of the application to be reduced.

14. Computer software comprising instructions which, when the software is executed by a computer, causes the computer to carry out the method of claim 13.10 15. A non-transitory machine-readable storage medium which stores computer software accordingto claim 14.Application No: GB2417714.9 Examiner: Dr Niall DeakinClaims searched: 1-15Date of search: 2 June 2025Patents Act 1977: Search Report under Section 17Documents considered to be relevant:Category Relevant to claims Identity of document and passage or figure of particular relevance X 1-15 US 2023 / 0381672 Al (HENDERSON et al.) See paragraphs 23, 28, 57 X 1-15 WO 2022 / 076966 Al (MICROSOFT TECHNOLOGY LICENSING LLC) See paragraphs 21, 28, 30, 31 X 1-15 US 2014 / 0349748 Al (HABERMAN) See figure 5 A; paragraphs 39, 49 A - WO 2024 / 033913 Al (SITAGO LTD) See page 16, lines 1-6 A - WO 2023 / 158929 Al (SONY INTERACTIVE ENTERTAINMENT INC) See figure 7 and page 14, line 8 - page 15, line 5Categories:X Document indicating lack of novelty or inventive step A Document indicating technological background and / or state of the art. Y Document indicating lack of inventive step if P Document published on or after the declared priority date but combined with one or more other documents of before the filing date of this invention. same category. & Member of the same patent family E Patent document published on or after, but with priority date earlier than, the filing date of this application.Field of Search:Search of GB, EP, WO &US patent documents classified in the following areas of the UKCX :International Classification:Subclass Subgroup Valid From A63F 0013 / 355 01 / 01 / 2014 A63F 0013 / 843 01 / 01 / 2014 A63F 0013 / 847 01 / 01 / 2014

Images