Latency compensation using machine learning predictions of user input
A machine learning model predicts user inputs on game controllers to compensate for latency in networked games, improving gameplay consistency and realism by generating control data ahead of actual input receipt.
Patent Information
- Application Number
- JP2022529084
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-11-19
- Filing Date
- 2020-11-13
- Publication Date
- 2024-12-09
- Estimated Expiration
- 2040-11-13
AI Technical Summary
Existing networked video game platforms suffer from latency issues due to factors like network congestion, data transfer speed, computing resource bandwidth, and game complexity, leading to inaccurate player action predictions and degraded gameplay quality, particularly in sensitive genres like first-person shooters.
Implementing a machine learning model trained on historical sensor and game data to predict user inputs on game controllers, generating corresponding game control data ahead of actual input receipt to compensate for latency.
The solution provides precise latency compensation, enhancing gameplay experience by ensuring consistent and realistic player actions across multiplayer games, reducing false positives, and conserving resources while adapting to player behavior changes.
Smart Images

Figure 0007596379000001 
Figure 0007596379000002 
Figure 0007596379000003
Abstract
Description
[Technical field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a PCT application claiming priority to U.S. Patent Application No. 16 / 688,833, filed November 19, 2019, entitled “LATENCY COMPENSATION USING MACHINE-LEARNED PREDICTION OF USER INPUT,” which is hereby incorporated by reference in its entirety. [Background technology]
[0002] The constant or near-constant availability of wide area network communications, coupled with improvements in the performance of client machines and wireless communication protocols, has led to an increase in the popularity of networked video game platforms, including cloud-based video game streaming platforms, where players use "thin clients" to play video games running on remote servers, and in-home video game streaming platforms. Despite advances in networked video game platforms, various factors can still cause latency between player actions and player perception in a video game. For example, in a video game streaming platform, network congestion can cause a noticeable delay between the player's actions and the player's perception. Other factors that may contribute to latency between the player's actions and the player's perception are the data transfer rate of the network, the bandwidth or capacity of available computing resources (e.g., processing resources, memory resources, display resources, etc.), the utilization of those computing resources, and / or the complexity of the video game itself (e.g., more data-intensive video games with complex graphics tend to experience more latency than less data-intensive video games with relatively simple graphics). Some video game genres, such as first-person shooters, are highly sensitive to latency and experience severe degradation in quality of play whenever latency occurs.
[0003] Dead reckoning is one example of an existing approach that attempts to compensate for latency observed in networked video game platforms. Using dead reckoning, a server computer can determine the past position, velocity, and / or acceleration of a player-controlled character over a period of time (e.g., over the past 200 milliseconds) and use the past data to predict the future position of the player-controlled character in order to compensate for latency. However, dead reckoning remains a relatively unsophisticated approach to predicting future player actions, often resulting in inaccurate predictions, and is therefore rarely used in networked video game platforms today. Without a viable mechanism for accurately predicting future player actions, latency cannot be compensated for in a practical manner, and the quality of play of many video games continues to deteriorate with each additional latency that occurs. The disclosure made herein is presented with respect to these and other considerations. [Brief description of the drawings]
[0004] The detailed description will now be described with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number identifies the figure in which the reference number first appears. Use of the same reference number in different figures indicates similar or identical components or features.
[0005] [Figure 1] FIG. 1 illustrates an example environment including a computing system configured to train and use machine learning models to predict user inputs to a player's video game controller and proactively generate corresponding game control data to compensate for latency. [Diagram 2] FIG. 1 is a block diagram illustrating example components of a game controller configured to provide sensor data to a remote computing system, among other things, for the purpose of predicting user input to the game controller using a trained machine learning model. [Diagram 3]FIG. 1 is a block diagram illustrating example components of a computing system configured to predict user inputs to a game controller using a trained machine learning model. [Figure 4] FIG. 1 is a flow diagram of an example process for training a machine learning model to predict the probability that a type of user input will be provided to a player's game controller. [Diagram 5] FIG. 1 is a flow diagram of an example process for utilizing a trained machine learning model to predict user input to a player's game controller and pre-generating game control data using machine learning predictions of user input to compensate for latency. [Figure 6] FIG. 1 is a flow diagram of an example process for measuring latency of a computer network and selecting one of a plurality of trained machine learning models to be used to predict user input to a game controller based on the measured latency. [Figure 7] FIG. 13 is a flow diagram of an example process for comparing actively generated game control data based on machine learning predictions of user inputs with actual game control data to determine a prediction error and recording the prediction error for purposes of validating the performance of the model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0006] Described herein, among other things, are techniques, devices, and systems for predicting user inputs to a video game controller using trained machine learning models and proactively generating corresponding game control data to compensate for latency between player actions and player perception. The disclosed techniques may be implemented, at least in part, by a remote computing system that provides a video game service for a user community to play video games using client machines configured to access the video games from the computing system. These client machines may individually install a client application configured to run the video games and / or output content thereof received (e.g., downloaded, streamed, etc.) from the remote computing system. The video game platform enables registered users of the community to play the video games as "players." For example, a player may load the client application, log in with a registered user account, select a desired video game, and play the video game that may be executed on the client machine via the client application or executed in the remote computing system to stream video game data to the client machine for output on the client machine.
[0007] Whenever such a user accesses and uses the video game platform, data may be collected by the remote computing system, and this data may be maintained by the remote computing system. For example, game state data including game state of the video game that occurred during past sessions of the video game (e.g., player movements, location in the game map, character class, weapon, etc.) may be collected and maintained by the remote computing system. Additionally or alternatively, sensor data generated by the game controller and received from the client machine during past video game sessions may be collected and maintained by the remote computing system. As today's game controllers are equipped with an increasing number of sensors (e.g., gyroscopes, accelerometers, touch sensors, pressure sensors, etc.), the resulting sensor data generated by these sensors may be utilized for purposes of predicting user inputs to such game controllers for purposes of latency compensation, as described herein. By way of example, whenever a user plays a video game via a video game platform, the user may use a game controller to control aspects of the video game. A given game controller may include one or more sensors, such as, but not limited to, a gyroscope, an accelerometer, and / or a touch sensor, among other possible sensors. A touch sensor (e.g., a capacitive pad) may be disposed under or on the surface of the device and / or in or on a finger-operated control. The touch sensor may be configured to detect the proximity of a finger to the surface or finger-operated control, and in response, the touch sensor may generate sensor data indicative of the proximity of the finger to the touch sensor.As another example, a gyroscope and / or accelerometer attached to the housing of the game controller may detect movement of the game controller in different directions (e.g., movement by translation, rotation, and / or tilting, etc.), and in response, the gyroscope and / or accelerometer may generate sensor data indicative of characteristics of this movement. In general, the sensor data may be utilized for a variety of purposes, such as to control aspects of a video game (e.g., to control a player-controlled character, to rotate a virtual camera that directs what is seen on a display, etc.). When the sensor data is used as game control data in this manner, the sensor data may in some cases be modified (e.g., filtered / attenuated, amplified, etc.) before being used to control aspects of the video game. However, the techniques and systems described herein are directed to game controllers configured to transmit raw, unfiltered sensor data generated by one or more sensors of the game controller to a remote computing system (e.g., by transmitting the sensor data via an associated client machine). This raw sensor data may be transmitted along with game control data (e.g., data generated by button presses, joystick deflections, etc., and modified sensor data) that will be processed to control aspects of the video game.
[0008] It will be appreciated that over time, a vast collection of historical data tied to past sessions of a video game played by a player community may be available to a remote computing system. The remote computing system may then use a portion of the historical data as training data to train one or more machine learning models. For example, historical sensor data generated by one or more physical sensors of a game controller and received by the remote computing system during past sessions of the video game may be used as training data. As another example, historical game state data, including game states of a video game that occurred during past sessions of the video game, may be used as training data, alone or in combination with sensor data. The sensor data and / or game state data may be represented by a set of features and may be labeled to indicate one of a number of types of user inputs that caused corresponding game control data to be received within a period of time since the sensor data was received and / or since the game state occurred.
[0009] A machine learning model trained on this data can predict the type of user input provided to a player's game controller, such as by generating a score indicating the probability that a particular type of user input will be provided to the game controller. These machine learning scores can be used to proactively generate real-time game control data corresponding to the predicted type of user input such that the game control data can be input to the video game prior to receiving the actual game control data from the client machine, which compensates for latency since the computing system does not wait for the actual game control data to be received from the client machine. As discussed above, video game platforms may be subject to latency between a player's actions in a video game and player perception, such as when the video game is played in a multiplayer mode by multiple players over a computer network. For example, there may be a time delay between a first time that a first player provides a user input to a game controller to control a character in the video game and a second time that the remote computing system receives the corresponding game control data from the first player's client machine. If this time delay (or latency) is not compensated for, the corresponding movements of the player-controlled character presented on the display of the second player's client machine may be delayed relative to the actions of the first player controlling the video game character, potentially degrading the quality of video game play. The disclosed techniques for latency compensation using machine-learned predictions of user inputs to a player's game controller enable prediction of future player actions with relatively high accuracy compared to existing approaches such as dead reckoning. In this way, the remote computing system can predict with improved accuracy the type of user input provided by a player before the remote computing system receives actual game control data corresponding to the upcoming user input.This may allow the second player to perceive that the first player's character is performing actions that the first player would most likely control, and to perceive this before the remote computing system has even received actual game control data related to the first player's actions from the first player's client machine.
[0010] In an exemplary process, a computing system may receive sensor data from a client machine associated with a player of a video game while the video game is being played (e.g., in a multiplayer mode). The sensor data may include raw sensor data generated by one or more physical sensors of the game controller (e.g., raw gyroscope data, raw accelerometer data, and / or raw capacitive sensor data, etc.). The computing system may provide the sensor data as input to a trained machine learning model to predict a user input to the game controller that will cause receipt of corresponding game control data from the player's client machine within a period of time after receiving the sensor data. The trained machine learning model is configured to process the sensor data and generate a score as an output, the score indicating a probability that a type of user input will be provided to the game controller. Based at least in part on the machine learning score, the computing system may generate game control data on behalf of the player that corresponds to the type of user input, and may provide the actively generated game control data as input to the video game so that video game data output from the video game can be transmitted to the client machine (e.g., a second client machine of a second player of the video game) to compensate for latency of the video game platform.
[0011] In another exemplary process, a computing system may receive game state data from a video game running in a multiplayer mode. The game state data may include a game state of the video game (e.g., player movement, location in a game map, character class, weapon, etc.). The computing system may provide the game state data as input to a trained machine learning model to predict user inputs to a given player's game controller that will cause receipt of corresponding game control data from the player's client machine within a period of time after receiving the game state data. The trained machine learning model is configured to process the game state data and generate a score as an output, the score indicating a probability that a type of user input will be provided to the game controller. Based at least in part on the machine learning score, the computing system may generate game control data on behalf of the player that corresponds to the type of user input, and may provide the actively generated game control data as input to the video game so that the video game data output from the video game can be transmitted to a client machine (e.g., a second client machine of a second player of the video game) to compensate for latency of the video game platform. In some embodiments, both the sensor data and the game state data may be provided as inputs to a trained machine learning model to predict user inputs to the game controller for purposes of latency compensation.
[0012] The techniques and systems described herein may provide an improved gaming experience for players of a video game because the techniques and systems described herein may predict user inputs to game controllers and proactively generate corresponding game control data ahead of the time that the actual game control data generated by those controllers is received by the computing system. For example, the trained machine learning models described herein may learn to predict when a player is going to provide the type of user input to control a character in a video game to jump a wall and make a sharp left turn. This predictive ability not only compensates for latency so that consistency is maintained between multiple players of a multiplayer video game, but also allows for animation enhancements to be provided for a more realistic visual experience. For example, if a computing system predicts that a player is going to actuate the "A" button to control a character in a video game to jump over a wall, the animation of the player-controlled character may be enhanced based on the machine-learned prediction to render a "lead move" of the character, which may depict a shift in the player-controlled character's weight and / or a crouching motion of the character slightly prior to the jump. Meanwhile, the actual "jump" of the character may still be rendered in close synchronization with the player actuating the "A" button due to the predictive capabilities of the system. Without such animation enhancements and machine learning prediction of user input, a video game may output frames that exhibit snappy, responsive animation without any lead-in animation (e.g., a character suddenly transitions from running in an unrealistic manner to jumping over a wall), or the lead-in animation is shown over multiple frames but is slow and unresponsive (e.g., the jump occurs with a noticeable delay from the time of the button press).The techniques and systems described herein make it possible to provide a video game experience to player communities that includes the "best of both worlds" by rendering animation enhancements (e.g., lead-in animation) without noticeable time lag so that the actions of objects and characters in the video game appear realistic and remain responsive to user input provided by the player.
[0013] While many of the examples provided herein relate to trained machine learning models residing and executing on a remote computing system, it should be understood that the computing system utilizing the trained machine learning model to predict user inputs may be a client machine located with a user playing a video game. That is, the training may be performed at a remote computing system, but the trained machine learning model may be downloaded to a client machine for execution of the trained machine learning model on the client machine itself (instead of, or in addition to, executing the trained machine learning model on the remote computing system). In these embodiments, the client machines may utilize a common machine learning model, and / or multiple client machines may each utilize different machine learning models. For example, user-specific machine learning models may be trained on historical data specific to a particular user and downloaded to those users' client machines instead of, or in addition to, training machine learning models based on aggregate data collected from the entire player population.
[0014] Because the machine learning model can learn to identify complex relationships within sensor data, game state data, and / or game control data that may be utilized as training data, the techniques and systems described herein may enable accurate prediction of user inputs to a game controller, resulting in lower false positive rates and fewer instances of incorrectly predicting a player's actions. The techniques and systems described herein may also be more adaptable to variations in player behavior than existing systems (e.g., systems that use dead reckoning for forward prediction of player behavior) because the machine learning model can be retrained with new data to adapt the machine learning model to understanding player actions over time as the player's behavior changes. Furthermore, the machine learning model may account for more variables and more types of user behavior than can be reasonably accounted for in a manually constructed system. The techniques and systems described herein may further enable one or more devices to conserve resources in terms of processing resources, memory resources, networking resources, and the like, in various manners described herein.
[0015] 1 illustrates an example environment 100 including a computing system 102 configured to train and use a machine learning model 104 to predict user inputs to a player's video game controller 106 and proactively generate corresponding game control data 108 to compensate for latency. A community of players 110 (sometimes referred to herein as "users" 110) may be associated with respective client machines 112 and respective video game controllers 106 (sometimes referred to herein as "game controllers" 106). In the example of FIG. 1, a first client machine 112(A) is shown associated with and utilized by a first player 110(A), and a second client machine 112(B) is shown associated with and utilized by a second player 110(B). A first game controller 106(A) is also associated with and utilized by a first player 110(A), and a second game controller 106(B) is associated with and utilized by a second player 110(B). It should be understood that any number of players 110 and client machines 112 may be implemented in the environment 100, such as a single player 110 with a single client machine 112, or more than two players 110 with more than two client machines 112.
[0016] That is, the client machines 112 illustrated in FIG. 1 may be implemented as any suitable type of computing device configured to execute video games and / or render graphics on an associated display, including, without limitation, a personal computer (PC), a desktop computer, a laptop computer, a mobile phone (e.g., a smartphone), a tablet computer, a portable digital assistant (PDA), a wearable computer (e.g., a virtual reality (VR) headset, an augmented reality (AR) headset, smart glasses, etc.), an in-vehicle (e.g., in-car) computer, a television (smart TV), a set-top box (STB), a game console, and / or any similar computing device. Furthermore, the client machines 112 may vary with respect to their respective platforms (e.g., hardware and software). For example, the two client machines 112(A) and 112(B) illustrated in FIG. 1 may represent different types of client machines 112 having various capabilities with respect to processing capabilities (e.g., central processing unit (CPU) model, graphics processing unit (GPU) model, etc.), graphics driver versions, etc.
[0017] The client machine 112 may communicate with a remote computing system 102 (sometimes shortened herein to "computing system 102" or "remote system 102") via a computer network 114. This computer network 114 may represent or include, but is not limited to, the Internet, other types of data and / or voice networks, wired infrastructures (e.g., coaxial cable, fiber optic cable, etc.), wireless infrastructures (e.g., radio frequency (RF), cellular, satellite, etc.), and / or other connection technologies. The computing system 102 may, in some cases, be part of a network-accessible computing platform that is maintained and accessed via the computer network 114. Such network-accessible computing platforms may be referred to using terms such as "on-demand computing," "software as a service (SaaS)," "platform computing," "network-accessible platforms," "cloud services," "data centers," and the like.
[0018] It should be understood that the computer network 114 of FIG. 1 may represent a local area network (LAN) in some embodiments, which may be the case in an “in-home” video game streaming scenario. In this scenario, the computing system 102 may represent a host computing system located at or near the same geographic location as the client machine 112. In some embodiments, the computer network 114 may represent a wide area network (WAN), such as the Internet, and the client machines 112(A) and 112(B) may be located at different geographic locations. In some embodiments, the client machine 112 may be implemented as a thin client configured to send and receive data to and from the computing system 102 with minimal data processing on the client machine 112 itself (e.g., without having to run a video game on the client machine 112 itself). In a thin client implementation, the computing system 102 may receive, among other data, game control data from the client machine 112, and the computing system 102 may transmit video game data (e.g., frames of image data and audio data) to the client machine 112 for presentation on an associated display. In other implementations, the client machine 112 may be configured to download and run the video game locally, but data may still be transmitted over the computer network 114 for purposes of engaging in online multiplayer video gaming with other players. Additionally, it should be understood that the computer network 114 may be omitted in some scenarios, such as the functionality and components of the computing system 102 being implemented on the client machine 112.This is because even in examples where a network is not involved (e.g., a single player 110 playing a video game on a local client machine 112), latency between the player's actions and the player's perception may still arise due to other non-network factors, including, but not limited to, a relatively low-performance client machine 112 running a data-intensive video game with complex graphics. In this scenario, machine learning predictions of user inputs can be utilized to compensate for such latency.
[0019] In some embodiments, the computing system 102 serves as or has access to a video game platform that implements a video game service to distribute (e.g., download, stream, etc.) video games (and their contents) to the client machines 112. In one example, the client machines 112 may each install a client application thereon. The installed client application may be a video game client (e.g., game software for playing a video game). The client machines 112 with the client application installed may be configured to download, stream, or otherwise receive programs (e.g., video games, and their contents) from the computing system 102 over the computer network 114. To this end, any type of content distribution model may be utilized, such as a direct purchase model in which programs (e.g., video games) can be individually purchased for download and execution on the client machines 112, a subscription-based model, a content distribution model in which programs are rented or leased for a period of time, streamed, or otherwise made available to the client machines 102, etc. Thus, an individual client machine 112 may include one or more installed video games that are executable by loading a client application, or may have access to one or more video games running on a remote computing system 102 that are playable by loading a client application.
[0020] Client machines 112 may be used to register with and subsequently log into the video game service. Users 110 may create user accounts for this purpose and may specify / set credentials (e.g., passwords, PINs, biometric IDs, etc.) tied to the registered user accounts. As users 110 interact with the video game platform (e.g., by accessing their user / player profiles with the registered user accounts, playing video games on their respective client machines 112 using their respective game controllers 106, etc.), the client machines 112 transmit data to the remote computing system 102. For a given client machine 112, the data transmitted to the remote computing system 102 may include, but is not limited to, game control data 116 and sensor data 118, as well as video game data (e.g., game performance statistics uploaded to the remote system), social networking messages and related activities, identifiers (IDs) of video games played on the client machine 112, etc. At least a portion of this data may be streamed in real time (or substantially in real time) during the video game session, and a subset of the data may be transmitted to the remote system 102 at defined intervals and / or uploaded in response to an event (e.g., the end of a video game).
[0021] 1, the game controller 106(A) may be used by a first player 110(A) to interact with a video game platform as described herein. For example, the first player 110(A) may pair the game controller 106(A) with a client machine 112(A) running a video game client (e.g., game software for playing a video game) thereon. Once the game controller 106(A) is paired with the client machine 112(A) and is enabled to communicate with it (e.g., by sending / receiving data to / from the client machine 112(A)), the game controller 106(A) can be used to interact with the video game platform described herein, such as registering the game controller 106(A) with a user account, accessing video games available from the remote computing system 102, and playing video games using the game controller 106(A).
[0022] The game controller 106, in some embodiments, may represent a handheld game controller, such as the game controllers 106(A) and 106(B) shown in FIG. 1, designed to be held by one or both hands of the player 110 and configured with one or more finger-operated controls operated by the fingers and / or thumbs of the user's 110 hands. In some embodiments, the game controller 106 may be further configured to be operated by movement (e.g., panning, rotating, tilting, etc.) of the game controller 106 in three-dimensional (3D) space. It should be understood that the game controller 106 may represent any other suitable type of controller device, such as a mobile phone (e.g., a smartphone), a tablet computer, a portable digital assistant (PDA), a wearable computer (e.g., a smart watch, a head-mounted display (HMD), etc.), a portable game player, and / or any similar electronic device. The term "handheld," as used herein to describe the game controller 106, means a device that is configured to be held by a user 110, whether the device is held by the hand of the user 110 or by another part of the user's 110 body (e.g., a wearable device worn on the wrist, arm, leg, waist, head, etc., that is considered a "handheld" device as the term is used herein).
[0023] One or more sensors of the game controller 106 are configured to generate sensor data 118 related in some manner to the physical state of the game controller 106. For example, a touch sensor may generate sensor data 118 indicative of whether an object (e.g., a finger) is in contact with or near a portion of the game controller 106 that includes the touch sensor (e.g., a directional pad (D-pad), a joystick, a trigger button, a bumper button, a selector button, etc.). As another example, a pressure sensor may generate sensor data 118 indicative of whether an object is being pressed against a portion of the game controller 106 and whether that portion of the game controller 106 is being pressed lightly or heavily by the object. As yet another example, a motion sensor (e.g., a gyroscope and / or an accelerometer) may generate sensor data 118 indicative of whether the game controller 106 has changed in orientation and / or spatial position in 3D space and / or whether the game controller 106 is moving rapidly or slowly. Accordingly, the sensors of the game controller 106 are configured to generate sensor data 118 that is indicative of these and other types of physical conditions of the game controller 106 .
[0024] When a first player 110(A) utilizes a game controller 106(A) to interact with a video game platform, for example to play a video game 120 on a respective client machine 112(A), sensor data 118 and game control data 116 may be transmitted from the game controller 106(A) to the client machine 112(A) and forwarded from the client machine 112(A) to the remote computing system 102. The game control data 116 may be used to control aspects of the video game 120, thereby being processed by the video game 120 to determine how to render the next frame of the video game 120. Meanwhile, the sensor data 118 represents raw, unfiltered sensor data 118, such as raw data generated by a gyroscope, raw data generated by an accelerometer, and / or raw data generated by a touch sensor (e.g., a capacitive touchpad), etc.
[0025] In a streaming implementation, the video game 120 may be executed on a remote computing system 102, which may capture video game data 122 and transmit the video game data 122 to the client machine 112 over the network 114. This may involve the remote computing system 102 capturing the state of the video game 120, encoding the video and audio data into bits, and transmitting the encoded bits over the computer network 114 to the client machine 112, where an application (e.g., a video game client) executing on the client machine 112 may decode the bits to output an image via a display and an audio via a speaker of the client machine 112 for a given frame. The players 110 may react to the video and audio they see by manipulating their respective game controllers 106. For example, the player 110(A) may actuate controls of the game controller 106 (e.g., depress a directional pad (D-pad), deflect a joystick, swipe a finger on a trackpad, press a button) and / or tilt or move the game controller 106 in 3D space to control aspects of the video game 120. In response to the movements of the game controller 106, the game controller 106 may generate game control data 116 that is transmitted to the remote computing system 102 directly via a wireless access point, via the computer network 114, or via a client machine 112 with which the game controller 106 is associated. In either case, the game control data 116 may be transmitted in real time to the remote computing system 102 to control aspects of the video game 120. For example, the game control data 116 may be processed by the video game 120 to control the movement of a virtual object (e.g., a player-controlled character) of the video game 120 by moving the virtual object within a virtual world represented by a scene on the display of the client machine 112.In some embodiments, an instance of video game 120 may run locally on each client machine 112(A) and 112(B), and video game data 122 may include game control data 108 and / or 116 that is input to video game 120 running locally on a given client machine 112.
[0026] While the game control data 116 transmitted to the remote computing system 102 may include at least some sensor data 118 generated by sensors of the game controller 106, such as filtered / attenuated sensor data 118 and / or amplified sensor data 118, as depicted in FIG. 1 , the sensor data 118 transmitted by the game controller 106 to the remote computing system 102 represents raw, unfiltered sensor data 118 that is not used to control aspects of the video game 120, but is used to generate a machine-learned score indicative of predicted user inputs to the game controller 106. As described herein, machine-learned predictions of types of user inputs to the game controller 106 may be used to compensate for latency of the video game platform.
[0027] Thus, when the computing system 102 receives the game control data 116 and the sensor data 118 from the client machine 112, the computing system 102 may store that data, and possibly other data received from the client machine 112, in a data store 124 accessible to the remote computing system 102. The data collected by the remote computing system 102 may be organized in the data store 124 in any suitable manner to associate user accounts with relevant portions of the data and / or to associate video games and / or past sessions of the video games with relevant portions of the data. Over time, and given a large community of players 110 who frequently interact with the video game platform, sometimes over extended periods of time during a given session, a large amount of data may be collected and maintained in the data store 124.
[0028] The computing system 102 may train the machine learning model 104 using historical sensor data sampled from the data store 124. For example, the computing system 102 may access a portion of the historical data as sampled data and use the sampled data to train the machine learning model 104. In some embodiments, the portion of the data used as training data is represented by a set of features and labeled with a label indicative of one of multiple types of user input that corresponds to the historical game control data 116. In this manner, a supervised learning approach may be employed to train the machine learning model 104 to predict a particular type of user input that is likely to be provided to a player's game controller 106.
[0029] The machine learning model 104 may be trained using historical sensor data 118 collected in the data store 124. Additionally or alternatively, the machine learning model 104 may be trained using historical game state data 126 including game states of the video game 120 that occurred during past sessions of the video game 120. Examples of game states of the video game 120 include, but are not limited to, player movements, positions in a game map, character classes, weapons, etc. It should be understood that the machine learning model 104 may be trained using additional data in addition to or in lieu of the sensor data 118 and game state data 126 described above. For example, historical game control data 116 may be collected from the client machine 112 over time, the game control data 116 having been generated based on user inputs provided by human users to the game controller 106 to control aspects of the video game 120. These user inputs may include actuation of controls on the game controller 106, such as button presses, joystick deflections, finger swipes on a trackpad, etc., and / or tilt or movement of the game controller 106 in 3D space. Thus, the historical game control data 116 (e.g., data indicative of user inputs provided to the game controller 106, such as to control aspects of the video game 120) may be used as training data in conjunction with the historical sensor data 118 and / or the historical game state data 126, or in lieu of use of that historical sensor data. The use of the historical game control data 116 in conjunction with the historical sensor data 118 and / or the historical game state data 126 allows correlations to be made between the game control data 116 and the sensor data 118 and / or the game state data 126 at or near the time of the game control data 116.
[0030] In general, training data may include two components: features and labels. However, in some embodiments, the training data used to train the machine learning model 104 may be unlabeled. Thus, the machine learning model 104 may be trainable using any suitable learning technique, such as supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, etc. The features included in the training data may be represented in a set of features, such as in the form of an n-dimensional feature vector of quantifiable information regarding attributes of the training data. Exemplary features included in the training data may include, but are not limited to, values of sensor data 118 (e.g., capacitance values, resistance values, displacement values, velocity values, acceleration values, temperature values, humidity values, etc.), values of game state data 126 (e.g., values related to player movement, position in a game map, character class, weapon, etc.), values of game control data 116 (e.g., values generated by potentiometers and other switches of a finger-operated control), and / or any other suitable features that may be relevant to calculating a score indicative of the probability that a particular type of user input is provided to the game controller 106. In some embodiments, the values of the sensor data 118 may relate to data generated by one or more physical sensors (e.g., biofeedback sensors) placed on or in proximity to the player 110 to obtain biofeedback measurements / values. Examples of such sensors and sensor data are described in U.S. Pat. No. 9,511,289, which is incorporated herein by reference in its entirety. The term "biofeedback" as used herein refers to measurements of specific and quantifiable bodily functions of a game player. Such biofeedback measurements are typically referred to as measurements of unconscious or involuntary bodily functions.For example, the sensor data 118 generated by such biofeedback sensors may generate values related to the player's 110 blood pressure, blood oxygen levels, heart rate, respiratory rate, galvanic skin response, skin and / or body temperature, sweat gland activity, eye movement, pupil dilation, head, face, hand, or other body movement, gestures, position, facial expressions, posture, muscle tension, neuronal signals from the brain, and / or any other suitable non-verbal signals, values, and / or cues that provide indications of the player's 110 feelings, emotions, desires, and / or desires. These types of values may be predictions of future user inputs to the game controller 106 and thus are used as features to train the machine learning model 104.
[0031] As part of training the machine learning model 104, the training component may set weights for the machine learning. These weights may be applied to the set of features included in the training data as derived from historical data in the data store 124, including historical sensor data 118 and / or historical game state data 126. In some embodiments, the weights set during the training process may be applied to parameters internal to the machine learning model 104 (e.g., weights for neurons in a hidden layer of a neural network). These internal parameters of the machine learning model may or may not map one-to-one with individual input features of the set of features. The weights may indicate the influence of any given feature or parameter on the score output by the trained machine learning model 104.
[0032] Using the trained machine learning model 104, the computing system 102 may score different types of user inputs that may be provided to the game controller 106. For example, the computing system 102 may receive new sensor data 118 associated with one or more logged-in and registered user accounts from one or more client machines 112, and / or the computing system 102 may receive new game state data 126 from a running video game 120. In an illustrative example, the video game 120 may run on the client machine 112(A) / (B) via an installed client application, or the video game 130 may be streamed to the client machine 112(A) / (B) while running on the remote computing system 102. In any event, when a player 110 logs in with his or her user account, runs a particular video game 120, and requests to play in multiplayer mode, for example, each client machine 112 may provide information indicative of the same to the computing system 102, and the sensor data 118 and game control data 116 may be streamed in real time from the client machines 112(A) / (B) to the remote computing system 102 when the video game 120 is started and the player 110 begins gameplay.
[0033] In one example, the computing system 102 may provide new sensor data 118 received from a first client machine 112(A) as input to the trained machine learning model 104 and generate a score 128 associated with a predicted type of user input as output from the trained machine learning model 104. In the example of FIG. 1, the score 128 relates to the probability that a first player 110(A) will actuate the "A" button of the game controller 106(A) (the "A" button actuates a particular type of user input to the game controller 106(A)) within a period of time after receiving the new sensor data 118, thereby triggering receipt of corresponding game control data 116 from the first client machine 112(A). In this case, a high score 128 indicates a high probability that the player 110(A) will provide a particular type of user input, while a low score 128 indicates a low probability that the player 110(A) will provide a particular type of user input. In some embodiments, the score 128 is a variable that is normalized in the range of [0,1]. This score 128 may have a monotonic relationship to the probability that a particular type of user input will be provided to the game controller 106. The relationship between the score 128 and the actual probability may or may not be a monotonic, but linear, relationship. Of course, the scoring may be implemented in any suitable manner to predict the probability that a type of user input will be provided to the game controller 106.
[0034] 1 illustrates a trained machine learning model 104 residing and executing on a remote computing system 102, it should be understood that the trained machine learning model 104 may be provided to one or more of the client machines 112 located with the player 110 of the video game 120. That is, training may be performed at the remote computing system 102, but the trained machine learning model 104 may be downloaded to the client machine 112 for execution of the trained machine learning model 104 on the client machine 112 itself (instead of, or in addition to, executing the trained machine learning model 104 on the remote computing system 102). In these embodiments, the client machine 112 may utilize a common machine learning model 104 and / or multiple client machines 112 may each utilize a different machine learning model 104. For example, a first client machine 112(A) may run a first trained machine learning model 104, a second client machine 112(B) may run a second trained machine learning model 104 that is different from the first trained machine learning model 104, and so on for any number of client machines 112. In an illustrative example, different user-specific machine learning models 104 may be trained based on historical data specific to each user 110, and these respective user-specific machine learning models 104 may be downloaded to the respective client machines 112 of those users 110 instead of, or in addition to, training the machine learning models 104 based on aggregate data collected from the entire population of players 110. It should be understood that such user-specific machine learning models 104 may also be executed on a remote computing system 102 and utilized on a per-player 110 basis to predict user inputs associated with a particular player 110. Thus, any reference herein to a "computing system 102" that provides input data to a trained machine learning model 104 to predict user inputs may be replaced with a "client machine 112."In this manner, the trained machine learning model 104 may be executed on the client machine 112. Thus, the terms "computing system 102" and "client machine 112" may be used interchangeably in this context herein.
[0035] In some embodiments, the new game state data 126 may be provided as an additional or alternative input to the trained machine learning model 104 to generate a score 128. In this scenario, the score 128 (as shown in FIG. 1 ) may be related to the probability that the first player 110(A) will actuate the “A” button on the game controller 106(A), causing receipt of corresponding game control data 116 from the first client machine 112(A), within a certain period of time after receiving the new game state data 126. The period (or future time at which the user input is predicted to occur) may vary, and the trained machine learning model 104 may be trained to predict a user input provided at any suitable time after receiving the input data. For example, the machine learning model 104 may be trained to predict a user input provided 1 millisecond after the sensor data 118 and / or game state data 126 are received, 200 milliseconds after the sensor data 118 and / or game state data 126 are received, or any other suitable period of time. Although a single score 128 is shown in FIG. 1, it should be understood that the machine learning model 104 may be configured to output multiple scores 128 for multiple different types of user input (e.g., selection of the "B" button, the "X" button, the "Y" button, a joystick deflection, a rotation of the game controller 106, etc.).
[0036] In some embodiments, the new game control data 116 itself may be provided as an additional or alternative input to the trained machine learning model 104 to generate the score 128. For example, the player 110 may manipulate the game controller 106 during gameplay by actuating controls of the game controller 106 (e.g., pressing a button such as a directional pad (D-pad), deflecting a joystick, swiping a finger on a trackpad, etc.) and / or tilting or moving the game controller 106 in 3D space to control aspects of the video game 120. In response to a user's manipulation of the game controller 106, the game controller 106 may generate game control data 116. This new game control data 116 is indicative of the user input provided to the game controller 106 and may be provided as an input to the trained machine learning model 104. If the trained machine learning model 104 is running on the remote computing system 102, the game control data 116 may be transmitted to the remote computing system 102 for this purpose.
[0037] Based at least in part on the machine learning score 128 determined for the type of user input, the computing system 102 may generate game control data 108 corresponding to the type of user input, provide the game control data 108 as an input to the video game 120, and receive the video game data 122 as an output from the video game. If the video game data 122 is generated at the remote system 102, the remote system 102 may transmit the video game data 122 to the client machine 112(B) over the computer network 114. If the video game 120 is running on the client machine 112(B), the remote system 102 may transmit the game control data 108 to the client machine 112(B) over the computer network 114, and the client machine 112(B) may generate the video game data 122. If the trained machine learning model 104 is running on the client machine 112(B), the remote system 102 may transmit at least the sensor data 118 to the client machine 112(B) over the computer network 114, and the client machine 112(B) may provide the sensor data 118 and / or the game state data 126 as input for the trained machine learning model 104. If the video game 120 and the trained machine learning model 104 are running on the client machine 112(A), the video game data 122 may not be transmitted to any other computing device, but may be presented on a display of the client machine 112(A). In any event, a player 110, such as a second player 110(B), may perceive video game content that indicates something related to a player action (e.g., an action of a first player 110(A)) very close to the time that the player 110 (e.g., the first player 110(A)) provided user input to the game controller 106(A) (i.e., without any noticeable delay that could degrade the quality of play of the video game 120).For example, when a first player 110(A) selects the "A" button on the game controller 106(A), a second player 110(B) perceives the video game character 130 being controlled by the first player 110(A) to jump in close synchronization with the actuation of the "A" button by the first player 110(A). This is due to machine learning predictions that compensate for the latency of the video game platform and that the first player 110(A) will select the "A" button before the first player 110(A) actually selects the "A" button.
[0038] FIG. 2 illustrates a block diagram illustrating exemplary components of a game controller 106 configured to, among other things, provide sensor data 118 to a remote computing system 102 for the purpose of predicting user input to the game controller 106 using a trained machine learning model 104. The game controller 106 of FIG. 2 may be used by a player 110 to interact with a video game platform as described herein. For example, the player 110 may pair the game controller 106 with a client machine 112 running a video game client (e.g., game software for playing a video game) thereon. Once the game controller 106 is paired with the client machine 112 and the game controller 106 is enabled to communicate with it (e.g., by sending / receiving data to / from the client machine 112), the game controller 106 can be used to interact with the video game platform as described herein, such as registering the game controller 106 with a user account, accessing video games available from the remote computing system 102, and playing video games using the game controller 106.
[0039] The game controller 106 may represent a handheld game controller, such as the game controller shown in FIG. 1, designed to be held by one or both hands of the player 110 and configured with one or more finger-operated controls operated by the fingers and / or thumbs of the player 110's hands. In some embodiments, the game controller 106 may be further configured for operation by movement (e.g., panning, rotating, tilting, etc.) of the game controller 106 in three-dimensional (3D) space. It should be understood that the game controller 106 may represent any other suitable type of handheld device, wearable device, etc., such as a mobile phone (e.g., smartphone), tablet computer, portable digital assistant (PDA), wearable computer (e.g., smart watch, HMD), portable game player, and / or any similar handheld electronic device.
[0040] As shown in FIG. 2, the game controller 106 includes one or more input / output (I / O) devices 200, such as finger-operated controls (e.g., joystick, trackpad, triggers, depressible buttons, etc.), potentially other types of input or output devices, such as a touch screen, a microphone for receiving audio input, such as a user's voice input, a camera or other type of sensor (e.g., sensor 202) that can function as an input device for receiving gestural input, such as the movement of the game controller 106 and / or the hand of the user 110. In some embodiments, additional input devices may be provided in the form of a keyboard, keypad, mouse, touch screen, joystick, control buttons, etc. The input devices may further include control mechanisms, such as basic volume control buttons for increasing / decreasing the volume, as well as power and reset buttons. The input devices may facilitate input of biometric data of the user 110, such as acquiring a fingerprint or handprint, scanning the user's eyes and / or face, capturing the user's voice, etc., for biometric recognition / authentication of the user 110.
[0041] On the other hand, output devices may include displays, light emitting elements (e.g., LEDs), vibrators that create tactile perception, speakers (e.g., headphones), etc. There may also be simple light emitting elements (e.g., LEDs) to indicate a state, such as when the power is on. While some examples have been provided, the game controller 106 may additionally or alternatively include any other kind of output device. In some cases, output by one or more output devices may be based on input received by one or more of the input devices. For example, actuation of a control may result in output of a tactile response by a vibrator located adjacent to (e.g., directly below) the control or anywhere else.
[0042] Additionally, the game controller 106 may include one or more communication interfaces 204 to facilitate wireless connection to a network and / or one or more remote systems (e.g., a client machine 112 running an application, a game console, a wireless access point, etc.). The communication interface 204 may implement one or more of a variety of wireless technologies, such as Wi-Fi, Bluetooth, radio frequency (RF), etc. It should be appreciated that the game controller 106 may further include physical ports to facilitate a wired connection to a plug-in network device that communicates with the network, connected peripheral devices, or other wireless networks.
[0043] In the illustrated implementation, the game controller 106 further includes one or more processors 206 and computer-readable media 208. In some implementations, the processor 206 may include a central processing unit (CPU), a graphics processing unit (GPU), both a CPU and a GPU, a microprocessor, a digital signal processor, or other processing units or components known in the art. Alternatively or additionally, the functionality described herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and the like. In addition, each of the processors 206 may have its own local memory that may also store program modules, program data, and / or one or more operating systems.
[0044] The computer readable medium 208 may include removable and non-removable media implemented in any method or technology for storing information, such as volatile and non-volatile memory, computer readable instructions, data structures, program modules, or other data. Such memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, RAID storage systems, or any other medium that may be used to store the desired information and that may be accessed by a computing device. The computer readable medium 208 may be implemented as a computer readable storage medium ("CRSM"), which may be any available physical medium accessible by the processor 206 to execute instructions stored on the computer readable medium 208. In one basic implementation, the CRSM may include random access memory ("RAM") and flash memory. In other implementations, the CRSM may include, but is not limited to, read-only memory ("ROM"), electrically erasable programmable read-only memory ("EEPROM"), or any other tangible medium that can be used to store the desired information and that can be accessed by the processor 206.
[0045] Some modules, such as instructions, data storage, etc., can be stored in the computer readable medium 208 and configured to execute on the processor 206. Some exemplary functional modules are illustrated as being stored in the computer readable medium 208 and executed on the processor 206, although the same functions can alternatively be implemented in hardware, firmware, or as a system on a chip (SOC).
[0046] The operating system module 210 may be configured to manage hardware within and coupled to the game controller 106 for the benefit of other modules. Additionally, the computer-readable medium 208 may store a network communication module 212 that enables the game controller 106 to communicate with one or more other devices, such as a client machine 112 (e.g., PC) running an application (e.g., game application), a game console, a remote computing system 102, or the like, via the communication interface 204. The computer-readable medium 208 may further include a game session database 214 for storing data associated with the game controller 106, a client machine 112 connected to the game controller 106, or a game (or other application) running on the game controller 106 and / or the remote computing system 102 connected to the client machine 112. The computer-readable medium 208 may also include a device record database 216 that stores data associated with devices to which the game controller 106 couples, such as a client machine 112 (e.g., PC, game console, etc.), a remote computing system 102, and the like. The computer-readable medium 208 may further store game control instructions 218 for configuring the game controller 106 to function as a game controller, and general purpose control instructions 220 for configuring the game controller 106 to function as a controller for other non-gaming devices.
[0047] The game controller 106 is also shown to include one or more sensors 202. For example, the sensors 202 may include motion sensors such as an inertial measurement unit (IMU), which may include one or more gyroscopes, and / or accelerometers, and / or magnetometers, and / or compasses, or any other suitable motion sensors. In some embodiments, the sensors 202 may be implemented as independent gyroscopes, accelerometers, magnetometers, compasses, etc., but not necessarily as an IMU. In some embodiments, one or more of these sensors 202 may be utilized to provide six-component motion sensing. For example, an IMU may be configured to sense and generate sensor data 118 indicative of translational and / or rotational motion about 3D space. The sensor data 118 generated by such sensors 202 may relate to the degree, velocity, and / or acceleration of translational motion (X, Y, and Z motion) in 3D space, as well as the degree, velocity, and / or acceleration of rotational motion (roll, pitch, and yaw) in 3D space. The measurements may be generated in terms of a 3D coordinate system, such as a Cartesian (X, Y, and Z) coordinate system or a spherical coordinate system. The sensor data 118 may include measurements in terms of displacement (e.g., displacement from a previous time log), velocity, and / or acceleration of translational motion (represented by variables: d, v, a) and angular motion (represented by variables: θ, ω, α). The sensor data 118 may further include the time at which the sensor data 118 is generated and / or transmitted, for example, at any suitable time interval, such that a history of the sensor data 118 can be collected and stored temporarily or permanently in the game controller 106.
[0048] As another example, the sensor 202 may include a touch sensor configured to detect the proximity of an object, such as a finger, palm, or the like, to the touch sensor, which may be based on any suitable touch sensing technology, such as a capacitive touch sensor, a resistive touch sensor, an infrared touch sensor, a touch sensor that utilizes acoustic sound waves to detect the proximity of a finger, or any other type of touch sensor. For example, a touch sensor may be disposed under or on a surface of the device and / or in or on a finger-operated control to detect the proximity of a finger to the surface or finger-operated control. In response to detecting the proximity (e.g., a finger touching or hovering over a surface), the touch sensor may generate sensor data 118 indicative of the proximity of the finger. Touch sensors and the like may be embedded in the handle of the game controller 106 to detect the grip of a user 110, and / or in various controls, including trackpads, joysticks, buttons, and the like. In implementations utilizing capacitance-based sensing, the touch sensor may include electrodes (e.g., transmitter and receiver electrodes of a capacitive sensor) and a voltage may be applied to the electrodes such that the electrodes are configured to measure a capacitance change at the electrodes, which may be converted into sensor data 118 in the form of a capacitance value indicative of the proximity of an object to the sensor 202. For example, the capacitance change at the electrodes of a capacitance-based touch sensor may be affected by an object (such as a finger) being in proximity to the electrodes. The raw capacitance may be digitized into a proximity value to generate the sensor data 118.
[0049] As another example, the sensor 202 may include a pressure sensor such as a force sensing resistor (FSR). For example, the FSR may include a conductive material spaced apart from a resistive film (e.g., a semiconductor material such as an ink composition) and an actuator configured to transmit a force to the resistive film such that the resistive material contacts the conductive material under the application of a compressive force applied to the actuator. The FSR may exhibit a variety of resistances in response to the variable force and generate sensor data 118 corresponding to the resistance values. The FSR may be a "ShuntMode" FSR or a "ThruMode" FSR. In a shunt mode FSR, the conductive material spaced apart from the resistive film may be a plurality of interdigitated metal fingers. When a force is applied to the actuator of the FSR, the resistive film contacts some of the interdigitated metal fingers that shunt the metal fingers, thereby changing the resistance across the output terminals of the FSR, which may be digitized into an FSR value to generate the sensor data 118. In some embodiments, the pressure sensor may additionally or alternatively include other types of pressure sensing mechanisms, such as piezoelectric sensors, strain gauges, and the like.
[0050] Other examples of sensors 202 configured to generate corresponding sensor data 118 may include other biofeedback sensors such as those described in U.S. Pat. No. 9,511,289, including, but not limited to, temperature sensors, humidity sensors, cameras (including infrared (IR) or other non-visible spectrum cameras), eye tracking systems, weight / position sensor pads on which the player 110 may stand, heart rate sensors, galvanic skin response sensors, eye tracking sensors, facial expression sensors, muscle tension sensors, posture sensors, etc. As yet another example, the sensors 202 may include sensors 202 that detect signals indicative of neuronal activity, such as electroencephalography (EEG) sensors, quantitative electroencephalography (qEEG) sensors, functional near-infrared spectroscopy (fNIRS) sensors, functional magnetic resonance imaging (fMRI) sensors, electromyography (EMG) sensors, electro-ophthalmology (EOG) sensors, and / or other brain-computer interface sensors. A game controller 106 configured to be worn on the head of a player 110 (e.g., an HMD) is a natural facilitator for these types of brain-computer interface sensors since it is already in contact with the user's head. Although the sensor 202 is depicted as being embedded within the game controller 106 (e.g., integrated into the game controller 106, one or more finger-operated controls thereof, etc.) or on the surface of the game controller 106 (attached to the body of the game controller 106, its finger-operated controls, etc.), in some embodiments the sensor 202 may be outside the game controller 106 and connected as a peripheral thereto, or the sensor 202 may be outside the game controller 106 and communicatively coupled to the client machine 112 such that the sensor 202 is available as an independent sensor 202. For example, the sensor 202 may be placed on (e.g., attached to) or in close proximity to the player 110 to obtain biofeedback measures about the player 110 that may be input into the trained machine learning model 104.
[0051] Regardless of the type of sensor 202 of the game controller 106, the sensor 202 is configured in some manner to generate sensor data 118 related to the physical state of the game controller 106. For example, a touch sensor may generate sensor data 118 indicative of whether an object (e.g., a finger) is in contact with or near a portion of the game controller 106 that includes the touch sensor (e.g., a directional pad (D-pad), a joystick, a trigger button, a bumper button, a selector button, etc.). As another example, a pressure sensor may generate sensor data 118 indicative of whether an object is being pressed against a portion of the game controller 106 and whether that portion of the game controller 106 is being pressed lightly or heavily by the object. As yet another example, a motion sensor (e.g., a gyroscope and / or an accelerometer) may generate sensor data 118 indicative of whether the game controller 106 has changed in orientation and / or spatial position in 3D space and / or whether the game controller 106 is moving rapidly or slowly. Accordingly, the sensors 202 of the game controller 106 are configured to generate sensor data 118 indicative of these and other types of physical conditions of the game controller 106 .
[0052] 1 may be transmitted to the client machine 112 and forwarded from the client machine 112 to the remote computing system 102. The game control data 116 may be used to control aspects of the video game 120, and thereby processed by the video game 120 to determine how to render the next frame of the video game 120. The sensor data 118 represents raw, unfiltered sensor data 118, such as raw data generated by a gyroscope, raw data generated by an accelerometer, and / or raw data generated by a touch sensor (e.g., a capacitive touchpad).
[0053] FIG. 3 illustrates a block diagram showing exemplary components of a computing system 102 configured to predict user inputs to a game controller 106 using a trained machine learning model 104. In the illustrated implementation, the computing system 102 includes, among other components, one or more processors 300 (e.g., CPUs), memory 302 (or non-transitory computer-readable media 302), and a communication interface 304. The memory 302 (or non-transitory computer-readable media 302) may include volatile and non-volatile memory, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Such memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, RAID storage systems, or any other medium that may be used to store the desired information and that may be accessed by a computing device. The computer-readable medium 302 may be implemented as a computer-readable storage medium ("CRSM"), which may be any available physical medium that is accessible by the processor 300 to execute instructions stored on the memory 302. In one basic implementation, the CRSM may include random access memory ("RAM") and flash memory. In other implementations, the CRSM may include, but is not limited to, read-only memory ("ROM"), electrically erasable programmable read-only memory ("EEPROM"), or any other tangible medium that may be used to store desired information and that may be accessed by the processor 300. The video game service 306 may represent instructions stored in the memory 302 that, when executed by the processor 300, cause the computing system 102 to perform the techniques and operations described herein.
[0054] For example, the video game service 306 may include, among other possible components, a training component 308, a scoring component 310, and a game control data generator 312. The training component 308 may be configured to train a machine learning model using a portion of the data 314 in the data store 124 (sometimes referred to herein as “historical data” 314) as training data to obtain a trained machine learning model 104. The trained machine learning model 104 is usable by the scoring component 310 to determine a score 128 associated with a predicted user input to the game controller 106. The game control data generator 312 generates game control data 108 corresponding to a type of user input based at least in part on the machine learning score 128 generated for the type of user input. The game control data generator 312 may be further configured to provide the generated game control data 108 as input to a running video game 120 and to receive video game data 122 output from the video game 120 based at least in part on the game control data 108. As mentioned above, in some embodiments, the trained machine learning model 104 may reside and execute on one or more client machines 112. Thus, it should be understood that at least the scoring component 310, the trained machine learning model 104, and / or the game control data generator 312 may execute on the client machine 112 instead of, or in addition to, executing on a remote computing system 102, as shown in FIG. 3. In some embodiments, the computing system 102 may represent a client machine 112 that is local to a player 110 playing a video game 120.In an embodiment in which the computing system 102 represents a remote computing system 102 having the components shown in FIG. 3, the computing system 102 may transmit the video game data 122 (or game control data 108, or data input to the local machine learning model 104, such as sensor data 118) via the communication interface 304 and via the computer network 114 to a client machine 112 associated with a player 110 of the video game 120. Anticipating user inputs to the game controller 106 and proactively generating the game control data 108 in this manner may mitigate or compensate for the effects of latency of the video game platform. As an example, the trained machine learning model 104 may process as input the sensor data 118 received from a first client machine 112(A) and generate as output a score 128 related to the probability that a type of user input, such as, for example, the "A" button on the controller is actuated, is provided to the first game controller 106(A) of the first player 110(A) associated with the first client machine 112(A).
[0055] As mentioned above, the score 128 determined by the scoring component 310 (e.g., output by the trained machine learning model 104) is the machine learned score 128. Machine learning generally involves processing a set of examples (called "training data") to train a machine learning model. The machine learning model 104 is a learned mechanism that, once trained, can receive new data as input and estimate or predict an outcome as output. For example, the trained machine learning model 104 can include a classifier that is tasked with classifying an unknown input (e.g., an unknown image) as one of multiple class labels (e.g., labeling an image as a cat or a dog). In some cases, the trained machine learning model 104 is configured to implement a multi-label classification task (e.g., labeling an image as a "cat", "dog", "duck", "penguin", etc.). Additionally or alternatively, the trained machine learning model 104 can be trained to infer a probability or set of probabilities for a classification task based on unknown data received as input. In the context of the present disclosure, the unknown input may be sensor data 118 (e.g., raw sensor data) generated by the sensors 202 of the game controller 106 utilized by the player 110 while playing the video game 120, and the trained machine learning model 104 may be tasked with outputting a score 128 indicative of or otherwise associated with the probability that one of multiple classes or types of user input will be provided to the game controller 106, causing receipt of corresponding game control data 116 from the client machine 112, within a certain time period after receiving the sensor data 118.In some embodiments, the unknown input may be game state data 126 including game state of the running video game 120 (e.g., player movement, character position on a game map, character class, weapon, etc.), and the trained machine learning model 104 may be tasked with outputting a score 128 indicative of or otherwise associated with the probability that one of a plurality of classes or types of user inputs will be provided to the game controller 106 and cause receipt of corresponding game control data 116 from the client machine 112 within a certain time period after receiving the game state data 126. In some embodiments, the score 128 is a variable that is normalized in the range of [0,1]. This score 128 may have a monotonic relationship with the probability that a particular type of user input will be provided to the game controller 106. The relationship between the score 128 and the actual probability associated with a particular type of user input may be a monotonic, but linear, relationship or may not be a linear relationship. In some embodiments, the trained machine learning model 104 may output a set of probabilities (e.g., two probabilities) or associated scores, where one probability (or score) is associated with the probability that a type of user input is provided to the game controller 106 and the other probability (or score) is associated with the probability that a type of user input is not provided to the game controller. The score 128 output by the trained machine learning model 104 can be associated with either of these probabilities to proactively influence the type of game control data 108 that is generated (i.e., before future game control data 116 is received based on actual user input to the game controller 106).
[0056] The trained machine learning model 104 may represent a single model or an ensemble of base-level machine learning models and may be implemented as any type of machine learning model 104. For example, machine learning models 104 suitable for use in the techniques and systems described herein include, but are not limited to, neural networks, tree-based models, support vector machines (SVMs), kernel methods, random forests, splines (e.g., multivariate fitted regression splines), hidden Markov models (HMMs), Kalman filters (or extended Kalman filters), Bayesian networks (or Bayesian belief networks), expectation maximization, genetic algorithms, linear regression algorithms, nonlinear regression algorithms, logistic regression-based classification models, or ensembles thereof. An "ensemble" may include a collection of machine learning models 104 whose outputs (predictions) are combined, such as by using a weighted average or voting. Individual machine learning models in an ensemble may have different expertise, and the ensemble may operate as a committee of individual machine learning models that are collectively "smarter" than the individual machine learning models in the ensemble.
[0057] The training data used to train the machine learning model 104 may include various types of data 314. In general, as mentioned above, training data for machine learning may include two components: features and labels. However, in some embodiments, the training data used to train the machine learning model 104 may be unlabeled. Thus, the machine learning model 104 may be trainable using any suitable learning technique, such as supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, etc. The features included in the training data may be represented in a set of features, such as in the form of an n-dimensional feature vector of quantifiable information about the attributes of the training data. The following is a list of exemplary features that may be included in the training data for training the machine learning model 216 described herein. However, it should be understood that the list of features below is not exhaustive and the features used in training may include additional features not described herein, and in some cases, a portion, but not all, of the features described herein. Exemplary features included in the training data may include, but are not limited to, values of sensor data 118 (e.g., capacitance values, resistance values, displacement values, velocity values, acceleration values, temperature values, humidity values, biofeedback values, such as those described in U.S. Pat. No. 9,511,289), values of game state data 126 (e.g., values related to player movement, location in a game map, character class, weapon, etc.), values of game control data 116 (e.g., values generated by potentiometers and other switches of a finger-operated control), and / or any other suitable features that may be relevant to calculating a score indicative of the probability that a particular type of user input is provided to the game controller 106. Additionally, as part of the training process, the training component 308 may set weights for machine learning. These weights may be applied to the set of features included in the training data as derived from the historical data 314 in the data store 124. In some embodiments, the weights set during the training process may be applied to parameters internal to the machine learning model (e.g., weights for neurons in a hidden layer of a neural network).These internal parameters of the machine learning model may or may not map one-to-one with individual input features of the set of features. The weights may indicate the influence of any given feature or parameter on the score 128 output by the trained machine learning model 104.
[0058] FIG. 3 illustrates examples of types or classes of user inputs that may be scored by the trained machine learning model 104 as a means of predicting future user inputs to the game controller 106. For example, with respect to any finger-operated control of the game controller 106 (e.g., the "A" button, the "B" button, the "X" button, the "Y" button, a joystick, a D-pad, a trackpad, a bumper button, a trigger button, etc.), various types of user inputs may be scored, such as a finger hovering over the control without touching the control, a finger touching the control without pressing the control with a force above a threshold, or a finger pressing the control with a force above a threshold. If the finger-operated control is a mechanically actuated control (e.g., a button, a joystick, a D-pad, etc.), an additional type of user input that may be scored is the press and / or deflection of the finger-operated control. Another type of user input that may be scored is the rotation of the game controller 106 in 3D space (e.g., roll, pitch, yaw). Yet another type of user input that may be scored is translational movement of the game controller 106 in 3D space (e.g., translational movement in the X, Y, and / or Z directions). It should be understood that these examples are non-exhaustive and other examples of types of user inputs may be scored, as would be recognized by one of ordinary skill in the art. In any case, the game control data generator 312 may generate game control data 108 corresponding to the type of user input that is assigned a score that meets (e.g., exceeds, meets or exceeds) a threshold score associated with the type of user input. Different types of user inputs may be associated with different threshold scores for this purpose. For example, if the system wants to be more confident in its prediction that the player 110 will rotate the game controller 106 before generating the corresponding game control data 108, a relatively high threshold score may be used as a confidence level to reduce the false positive rate in predicting that particular type of user input.In contrast, if predicting that the player 110 will select the "X" button is incorrectly predicted such that it is unlikely to degrade play quality (e.g., if selecting the "X" button is not a performance-critical action in the video game 120), a relatively low threshold score may be used as the confidence level for generating game control data 108 for that particular type of user input with reasonable confidence that the prediction is accurate. Methods for dealing with incorrect predictions may include replacing the incorrectly predicted game state with a known correct game state (once that state is known), possibly with additional logic to make this replacement less noticeable / intrusive to the affected player (e.g., gradually interpolating from the incorrect position to the correct position instead of instantly teleporting a virtual object to the correct position).
[0059] The processes described herein are illustrated as a collection of blocks in a logical flow graph that represent a sequence of operations that may be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer-executable instructions that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any some of the described blocks may be combined in any order and / or in parallel to implement a process.
[0060] 4 is a flow diagram of an example process 400 for training a machine learning model 104 to predict the probability that a type of user input will be provided to a player's game controller. For purposes of discussion, process 400 will be described with reference to the previous figures.
[0061] At 402, computing system 102 may provide users 110 with access to a video game service. For example, computing system 102 may enable users to access and browse a catalog of video games, modify a user profile, perform transactions, engage in social media activity, and perform other similar actions. Computing system 102 may distribute video games (and their content) to client machines 112 as part of the video game service. In an illustrative example, users 110 accessing the video game service may load an installed client application, log in with a registered user account, select a desired video game 120, and run the video game 120 on their client machines 112 or stream the video game 120 through the client application.
[0062] At 404, the computing system 102 may collect and store data 314 associated with past sessions of the video game. This data 314 may be collected at block 404 whenever the user 110 accesses the video game service with a registered user account and uses the video game platform, such as when playing a video game 120 on the platform. The data 314 collected at block 404 may include, but is not limited to, sensor data 118 and game control data 116 received from the client machine 112 and generated by the game controller 106, game state data 126 of the video game, etc. It will be appreciated that over time, a vast collection of data 314 tied to a registered user account may be available to the computing system 106. It will be appreciated that entities providing video game services may require users 110 to provide informed consent for their data to be collected and used to train machine learning models for the purpose of reducing latency between the player's actions and the player's perception in their video game 120.
[0063] At 406, the computing system 102, via the training component 308, may access (historical) data 314 associated with past sessions of a video game 120 played via the video game service to be labeled for supervised machine learning. For example, the (historical) data 314 accessed at 406 may include historical sensor data 118 received from the client machine 112 during past sessions of a particular video game 120. Additionally or alternatively, the (historical) data 314 accessed at 406 may include historical game state data 126 including game states of the video game 120 that occurred during past sessions of the video game 120.
[0064] At 408, the computing system 102, via the training component 308, may access (historical) game control data 116 received from the client machines 112 within a period of time since (i) the historical sensor data 118 was received from those client machines 112, or (ii) the game state (included in the historical game state data 126) occurred during a past session of a video game played via the video game service. This period of time may be any suitable period of time, such as a period of time within a range of about 1 millisecond to about 200 milliseconds, depending on how far into the future the user input prediction is made. In an illustrative example, a particular player 110 about to press the "A" button on the game controller 106 may often twist the game controller 106 slightly to the right just before pressing the "A" button. In this example, the historical sensor data 118 to be labeled may be sensor data 118 generated by a gyroscope of the game controller 106, and the game control data 116 accessed in block 408 may include data corresponding to a press of the "A" button received from a client machine 112 of a player 110 a period of time (e.g., 100 milliseconds) after the gyroscope-based sensor data 118 is received from the same client machine 112. As another illustrative example, a particular player 110 may transition between selecting the "A" button and selecting the "B" button. In this example, the historical sensor data 118 to be labeled may be sensor data 118 generated by touch sensors (e.g., capacitance pads) associated with the “A” and “B” buttons that may indicate an increase in the capacitance value of the “A” button along with a decrease in the capacitance value of the “B” button after the “B” button is released, and the game control data 116 accessed in block 408 may include data corresponding to a press of the “A” button received from a client machine 112 of a player 110 after a period of time (e.g., 50 milliseconds) after the touch sensor-based sensor data 118 is received from the same client machine 112.In yet another example, the historical game state data 126 to be labeled may include a game state of the video game 120 where the game character is located next to a wall in the game map and the character is running directly towards the wall. In this example, the game control data 116 accessed in block 408 may include a "jump" action corresponding to a press of the "A" button received from the client machine 112 after a period of time (e.g., 20 milliseconds) after the game state occurred in the video game 120. This type of game control data 116 (e.g., a "jump" action of a player-controlled character) may be shown in the game state data 126 of the video game 120 for this particular game state over hundreds of millions of past game sessions, meaning that most players would always control their game character to jump over the wall if the character was running directly towards the wall and was next to the wall within a certain period of time of the jump action.
[0065] At 410, the computing system 102, via the training component 308, may label the data (e.g., the historical sensor data 118 and / or the historical game state data 126) with a label indicating one of multiple types of user inputs corresponding to the historical game control data 116 received within a period of time (i.e., the historical game control data 116 accessed at block 408). Examples of labels are described herein, such as the various types / classes of user inputs described herein with reference to FIG. 3. Referring to the example described in the previous paragraph, gyroscope-based sensor data 118 received 100 milliseconds prior to receiving game control data 116 corresponding to a press of the "A" button may be labeled with "A" button:pressed. Similarly, touch sensor-based sensor data 118 received 50 milliseconds prior to receiving game control data 116 corresponding to a press of the "A" button may be labeled with the same label "A" button:pressed. Similarly, game state data 126 containing game state that occurred 20 milliseconds prior to receiving game control data 116 corresponding to a press of the "A" button may be labeled with the same label, "A" button:pressed."
[0066] At 412, the computing system 102, via the training component 308, may train the machine learning model using the (historical) data 314 (e.g., the historical sensor data 118 and / or the historical game state data 126) as training data to obtain a trained machine learning model 104. As indicated by sub-block 414, training the machine learning model at block 412 may include setting weights for machine learning. These weights may be applied to a set of features derived from the historical data 314. Exemplary features are described herein. In some embodiments, the weights set at block 314 may be applied to parameters internal to the machine learning model (e.g., weights for neurons in a hidden layer of a neural network). These internal parameters of the machine learning model may or may not map one-to-one with individual input features of the set of features. As indicated by sub-block 416, training the machine learning model at block 412 may include determining a prediction error and training the model 104 using the prediction error. For example, this may be implemented as an error-driven learning technique (a type of reinforcement learning) in which the machine learning model being trained is tasked with predicting outputs based on sample training data provided as input, and the training component 308 determines (e.g., measures) a prediction error of the model's prediction, which may be based on labels (e.g., known correct output values, often referred to as ground truth values) from the training data. The training component 308 may then feed the prediction error back through the model to improve the model's performance during the training process. As indicated by the arrow from block 412 to block 404, the machine learning model 104 may be retrained using updated (historical) data 314 to obtain a newly trained machine learning model 104 adapted to recent player behavior. This allows the machine learning model 104 to adapt to changing player behavior over time.
[0067] It should be understood that the sampled historical data 314 used to train the machine learning model 104 may be any suitable data in addition to the training data described with reference to the process 400. For example, the user input data itself (e.g., buttons pressed, controls actuated, etc.) may be used as training data to predict the type of user input provided to the game controller 106. For example, the player 110 may actuate controls in a particular sequence to perform a special maneuver of the player-controlled character in the video game 120, such as a fighting / combat video game, where a particular sequence of button presses may cause the game character to fire a special weapon or maneuver in a particular way. Thus, the training module 308 may access game control data 116 received within a period of a particular sequence of button presses to train the machine learning model to predict when the player 110 is about to complete a special maneuver in the video game 120.
[0068] It should further be appreciated that the training of the machine learning model 104 by the process 400 may be, for example, per player and / or per player community. In a per player training example, the machine learning model 104 may be trained on data 314 associated with a user account of a particular player 110 to learn to predict user inputs that the particular player 110 provides to the game controller 106. In a player community training example, the machine learning model 104 may be trained on data 314 associated with multiple user accounts of the player community to learn to predict user inputs that a general player 110 provides to the game controller 106. This latter training approach may be useful when there is not enough data 314 associated with a single player to accurately predict the actions of that player 110.
[0069] 5 is a flow diagram of an example process 500 for utilizing a trained machine learning model 104 to predict user input to a player's game controller 106 and pre-generate game control data 108 using the machine learning predictions of user input to compensate for latency. For purposes of discussion, process 500 is described with reference to previous figures. Additionally, as indicated by external page reference "A" in FIGS. 4 and 5, process 500 may proceed from block 412 of process 400.
[0070] At 502, the computing system 102 may receive data to be provided to the trained machine learning model 104 as input data. This data received at block 502 may include sensor data 118 generated by the game controller 106. The sensor data 118 received at block 502 may include, but is not limited to, capacitance values, resistance values, displacement values, velocity values, acceleration values, temperature values, humidity values, etc. In some embodiments, the sensor data 118 is raw sensor data (e.g., raw gyroscope data, raw accelerometer data, raw capacitive sensor data, etc.) such that values in the sensor data 118 represent raw (e.g., unfiltered, unamplified) values generated by the physical sensors 202 of the game controller 106. Additionally, if the computing system 102 is a remote computing system 102, the sensor data 118 received in block 502 may be received from a first client machine 112(A) associated with a first player 110(A) of a video game 120 (e.g., a video game 120 being played in a multiplayer mode with a second player 112(B)). However, it should be understood that the client machine 112 itself may receive data in block 502 from the remote system 102 or from a local game controller 106.
[0071] Additionally or alternatively, the data received at block 502 may include game state data 126 received from a video game 120 (e.g., a video game running in a multiplayer mode for at least a first player 110(A) and a second player 110(B). The game state data 126 may include a current game state of the video game 120, such as a current player movement, a current position of a game character within a game map, a current character class, and a currently selected weapon.
[0072] At 504, the computing system 102 may provide at least a portion of the data received at block 502 as an input to the trained machine learning model 104 via the scoring component 310 for the purpose of predicting a user input that the player 110 will provide to the game controller 106 within a period of time from receipt of the data at block 502 that will cause receipt of corresponding game control data 116 from the player's client machine 112. For example, the sensor data 118 may be provided as an input to the trained machine learning model 104 to predict a type of user input that a first player 110(A) will provide to the game controller 106, thereby causing corresponding game control data 116 to be received from the first client machine 112(A) within the next N milliseconds, where N may be any suitable value, such as a value in the range of approximately 1 millisecond to 200 milliseconds. As another example, the game state data 126 may be provided as an input to the trained machine learning model 104, instead of or as an additional input along with the sensor data 118, to predict the type of user input that the first player 110 (A) will provide to the game controller 106. In some embodiments, the history of the sensor data 118 and / or the history of the game state data 126 (e.g., data received over the last P milliseconds) may be provided as input to the trained machine learning model 104 at block 504. For example, the trained machine learning model 104 may learn to predict that the player 110 will press the "B" button immediately after the player 110 presses the "A" button for a time period that exceeds a threshold and with an amount of force that exceeds a threshold. Similarly, ramping (e.g., decreasing or increasing) of the sensor value may indicate a future type of user input.
[0073] At 506, the computing system 102, via the scoring component 310, may generate a score 128 related to or indicative of the probability (or probability) that a type of user input, as output from the trained machine learning model 104, will be provided to a game controller 106 associated with a player 110 (e.g., a first player 110(A)) of the video game 120 to generate corresponding game control data 116 within a period (e.g., N milliseconds) of receiving the data at block 502. The types of user inputs for which a score is assigned in block 506 may include, but are not limited to, actuation of a particular finger-operated control of the game controller 106 (e.g., a button, joystick, D-pad, etc.), movement of the game controller 106 in a particular direction (e.g., the X, Y, or Z directions for translational motion, and / or the direction of roll, pitch, or yaw rotational motion (e.g., clockwise or counterclockwise)), an object hovering over or touching a particular portion of the game controller 106 (e.g., a handle, surface, finger-operated control, etc.), or an object pressing against a portion of the game controller 106 with a particular amount of force. For example, these types of user inputs may be associated with actions intended to cause a player-controlled character to jump (e.g., pressing the "A" button can cause the game character to jump), rotate left or right (e.g., pressing the D-pad, deflecting the joystick, or rotating the game controller 106 can cause the game character to rotate left or right), fire a weapon (e.g., pressing the "B" button can cause a weapon to fire), etc. In some embodiments, the score 128 is a variable that is normalized in the range of [0,1]. This score 128 may have a monotonic relationship with the probability that a type of user input is provided to the game controller 106. The relationship between the score 128 and the actual probability associated with a particular type of user input may be a monotonic, but linear, relationship or may not be a linear relationship.
[0074] At 508, a determination may be made as to whether one or more criteria are met. For example, the machine learning score 128 generated at block 506 may be evaluated against a threshold score to determine whether the score 128 meets (e.g., exceeds, meets or exceeds, etc.) the threshold score. Thus, if the score 128 meets the threshold score, the exemplary criteria is met at block 508. The threshold score may be associated with the particular type of user input to which the score 128 is assigned, such that different threshold scores are associated with different types of user input. This allows the system to aggressively generate game control data 108 in a conservative manner for more performance-critical player actions. For example, if the player 110 is currently controlling a character to hide behind a barrier in the video game 120 to avoid being killed by enemy fire, and the score 128 is related to the probability that the D-pad is pressed in a particular direction that places the game character in the line of fire and the game character may be killed in the video game 120, a relatively high threshold score may be associated with this performance-critical type of user input, causing the system to not aggressively generate corresponding game control data 108 to move the game character from cover unless the score 128 associated with that type of user input meets the relatively high threshold score. It may be appreciated that other types of user inputs that do not impact the game play quality as much may be associated with relatively low threshold scores, such that latency may be compensated for more frequently with those types of user inputs. However, in some embodiments, even performance-critical types of user inputs, such as firing a weapon to hit a target in a timely manner, may benefit from latency compensation. The evaluation at block 508 may consider accuracy statistics predicting the particular type of user input for this purpose.For example, if the system accurately predicts a particular type of user input 99.99% of the time, the threshold score in block 508 may be relatively low to allow for proactive generation of corresponding game control data 108 to reduce latency, while for other types of user input that the system is poor at predicting, a relatively high threshold score may be used in block 508 to mitigate the false positive rate of the prediction.
[0075] Another exemplary criterion that may be evaluated at block 508 is whether the type of user input in question is classified as a type of user input for which game control data 108 will be actively generated on behalf of the player to compensate for latency. For example, a data store 124 accessible to the computing system 102 may tag certain types of user inputs as those for which the system will actively generate game control data 108, and other types of user inputs may be tagged as those that are excluded from consideration. In other words, there may be certain types of user inputs that are deemed not beneficial to predict on behalf of the player 110, and it is better to wait for actual game control data 116 that corresponds to those types of user inputs. Thus, if the type of user input to be assigned the score 128 at block 506 is determined to be one of the types of user inputs for which game control data 108 will be actively generated on behalf of the player 110, then the exemplary criterion is met at block 508. In some embodiments, the threshold score and / or classification of the types of user inputs that are considered for actively generating game control data 108 may be dictated at least in part by user settings. For example, the user 110 may specify how much they want the system to compensate for latency and / or the types of user inputs they want the system to predict on their behalf. Additionally, the threshold scores and / or classifications of types of user inputs may be game specific, such that in some video games 120, certain types of user inputs are predicted and in other video games 120, those types of user inputs are not predicted.
[0076] If one or more criteria are not met at block 508, process 500 may follow a "no" path from block 508 to block 510, where the computing system 102 may wait for actual game control data 116 instead of actively generating game control data 108 on behalf of the user for the predicted type of user input. For example, this may be the case if the score 128 for the type of user input did not meet a threshold score associated with that type of user input and / or if the type of user input is not classified as one for which game control data 108 is actively generated. If one or more criteria are met at block 508, process 500 may follow a "yes" route from block 508 to block 512.
[0077] At 512, the computing system 102, via the game control data generator 312, may generate game control data 108 corresponding to the type of user input associated with the score 128 generated at block 506. The generation of the game control data 108 at block 512 occurs proactively before the actual scheduled game control data 116 is generated. The data store 124 may maintain a mapping from types of user input to game control data 108 such that the computing system 102 can generate game control data 108 that appears to have been generated by the game controller 106.
[0078] At 514, the computing system 102 may provide the game control data 108 as input to the video game 120 via the game control data generator 312. At 516, the computing system 102 may receive video game data 122 as output from the video game 120 via the game control data generator 312 based at least in part on the game control data 108 actively generated on behalf of the player 110. At 518, the remote computing system 102 may transmit the video game data 122 to the client machine 112 via the computer network 114. For example, in a multiplayer video game scenario, the game control data 108 generated in block 512 may be generated on behalf of the first player 110(A) based on a prediction that the first player 110(A) will provide a certain type of user input to the game controller 106(A), and the video game data 122 transmitted in block 518 may be transmitted to a second client machine 112(B) of a second player 110(B) who is playing the video game 120 in a multiplayer mode with the first player 110(A). It should be understood that the video game data 112 may be transmitted in this manner to any number of client machines 112. By transmitting the video game data 122 to the client machine 112 in block 518, the remote system 102 may be considered to be causing the presentation of the video game data 122 on the display of the client machine 112, since the client machine 112 receives the video game data 122 and presents the video game data 122 on the display of the client machine 112. However, it should be understood that the trained machine learning model 104 and / or the video game 120 may be executed locally on the client machine 112, and in these embodiments, the video game data 122 may be generated on the client machine 112, and thus, the process 500 may omit block 518 in these embodiments.In these embodiments, the remote system 102 may transmit other data, such as the data received at block 502, over the computer network 114 to the client machine 112, and the client machine 112(B) may generate the video game data 122 locally and present the video game data 122 on a display of the client machine 112(B). When the computer network 114 is omitted, the client machine 112 may receive data directly from the game controller 106 and / or components of the client machine 112 (e.g., from a video game 120 executing thereon) at block 502, and the client machine 112(A) may generate the video game data 122 and present the video game data 122 on a display of the client machine 112(A).
[0079] Thus, the process 500 illustrates a technique for predicting what a target player 110 is going to do in a video game 120, proactively generating corresponding game control data 108 on behalf of that target player 110, and using the game control data 108 to compensate for the latency of the video game platform such that the other players 110 see what the target player 110 will do when the target player 110 does it (assuming the prediction of the user inputs to the game controller 106 of the target player 110 is accurate). Using this technique, each client machine 112 receives data that mimics a setup in which the game controllers 106 of all players 110 connected via the network 114 are directly connected to the client machine 112. Thus, the trained machine learning model 104 is utilized to predict what other players are currently doing even though the actual game control data 116 has not yet been received from those players 110. This allows the game characters of the other players 110 to behave in a more realistic manner. For example, in the example of FIG. 1, the sensor data 118 input to the trained machine learning model 104 may indicate that the game controller 106(A) is beginning to twist to the right, and the prediction may be that the first player 110(A) is likely to press the "A" button in the next N milliseconds. As another example, if the game state data 126 input to the trained machine learning model 104 indicates that the first player's 110(A) character is running directly into a wall, and the system has not received game control data 116 in the past 200 milliseconds long enough for the character to run up and hit the wall, the prediction may be that the first player 110(A) is likely to press the "A" button in the next N milliseconds because game state data for millions of past game sessions indicates that nearly everyone jumps over the wall in the same game state as the first player 110(A).In some cases, compensation for latency on the order of 200 milliseconds or less can mean the difference between success or failure in a video game 120 (e.g., a game character lives or dies). Thus, latency compensation techniques using machine-learned predictions of user inputs to a game controller 106 properly regulate interactions between players 110 of a multiplayer game (e.g., players 110 are not unfairly penalized due to the latency of the video game platform). As previously mentioned, machine-learned predictions of user inputs also allow animation enhancements to be utilized to create a more realistic appearance of video game content. For example, a lead-in animation for a predicted "jump" of a game character can be rendered, such as a shift in the weight of the game character or a crouching movement just before the jump.
[0080] 6 is a flow diagram of an example process 600 for measuring latency of a computer network 114 and selecting one of a plurality of trained machine learning models 104 to be used to predict user input to a game controller 106 based on the measured latency. For purposes of discussion, process 600 is described with reference to the previous figures.
[0081] At 602, the computing system 102 may measure the latency of the computer network 114 to which the client machines 112 are connected to interact with the video game service. The network latency may be measured as a value indicating how much data is delayed in transit to obtain the latency value measured at block 602. This measurement may be performed periodically, at the start of a new game session and / or during a game session, or at any other suitable time (e.g., triggered by an event).
[0082] At 604, the computing system 102 may determine a period spanning a range of possible periods (e.g., a period of N milliseconds) based at least in part on the measured latency value. For example, a higher latency value, which may indicate that the network 114 is relatively slow and delays in transit are relatively high due to congestion, etc., may correspond to a longer period, while a lower latency value, which may indicate that the network 114 is relatively fast and delays in transit are relatively low, may correspond to a shorter period. The data store 124 may maintain a mapping from latency values to time periods for this purpose, or the time period may be calculated based on a function in which the latency is a variable of the function.
[0083] At 606, the computing system 102 may select a trained machine learning model 104 from the available trained machine learning models based at least in part on the time period determined at block 604. For example, the machine learning models 104 may be trained to predict future user inputs to the game controller 106 that will occur within different time periods. For example, a first trained machine learning model 104 may be trained to predict user inputs that will cause receipt of corresponding game control data 116 within the next 5 milliseconds, and a second trained machine learning model 104 may be trained to predict user inputs that will cause receipt of corresponding game control data 116 within the next 200 milliseconds. Thus, based on the measured latency, the system may dynamically select an appropriate machine learning model 104 for predicting a particular amount of time into the future. The distribution of how far ahead to predict may be based on the topology of the network (e.g., a machine learning model 104 that predicts far into the future may be used for a wide area network, while a machine learning model 104 that predicts over a shorter period of time may be used for an in-home streaming platform or local predictions when the machine learning model 104 runs on a client machine 112, for example to trigger a "lead-in" animation).
[0084] 7 is a flow diagram of an example process 700 for comparing actively generated game control data 108 based on machine-learned predictions of user inputs with actual game control data 116 to determine a prediction error and record the prediction error for purposes of validating the performance of the model 104. For purposes of discussion, the process 700 is described with reference to the previous figures. Additionally, as indicated by external page reference "B" in FIG. 5 and FIG. 7, the process 700 may proceed from block 518 of process 500.
[0085] At 702, after actively generating game control data 108 on behalf of player 110 in process 500, computing system 102 may subsequently receive actual game control data 116 associated with player 110 within a period of time after receiving the data input to trained machine learning model 104. If the prediction was accurate, the actual game control data 116 matches the actively generated game control data 108. If the prediction was inaccurate, the actual game control data 116 does not match the actively generated game control data 108.
[0086] At 704, the computing system 102 may determine a prediction error based at least in part on comparing the actual game control data 116 received at block 702 with the game control data 108 actively generated at block 512 of process 500 to compensate for latency.
[0087] At 706, the computing system 102 may record the prediction errors in a database of recorded prediction errors so that the prediction errors can be tracked over time and statistics can be generated for predictions of particular types of user inputs to the game controller 106. These recorded prediction errors may be utilized to validate the performance of the trained machine learning model 104, such as by verifying whether the model 104 is behaving as expected and performing at a desired level of performance.
[0088] Although the subject matter has been described in language specific to structural features, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features described. Rather, the specific features are disclosed as example forms of implementing the claims. The inventions described in the claims at the time of filing of this application are set forth below. [1] 1. A method comprising: receiving, by a computing system, sensor data associated with a player of the video game; a user input to a game controller associated with the player that causes receipt of corresponding game control data by the computing system within a period of time from the receipt of the sensor data; providing the sensor data as input to a trained machine learning model; generating a score as an output from the trained machine learning model, the score indicative of a probability that a type of user input will be provided to the game controller; and generating, by the computing system, game control data corresponding to a type of the user input based at least in part on the score; providing the game control data as input to the video game; receiving video game data as output from the video game based at least in part on the game control data; causing, by the computing system, presentation of the video game data on a display. [2] The method of any one of claims 1 to 5, wherein the sensor data includes raw sensor data generated by one or more physical sensors of the game controller, the raw sensor data including at least one of raw gyroscope data, raw accelerometer data, or raw capacitance sensor data. [3] The type of user input is: Activating a particular finger control of the game controller; moving the game controller in a particular direction; an object hovering over or touching a particular portion of the game controller; or The method of claim 1, further comprising at least one of: the object pressing against a portion of the game controller with a particular amount of force. [4] receiving, by the computing system, game state data from the video game prior to said predicting; In addition to providing the sensor data as the input to the trained machine learning model, providing the game state data as an additional input to the trained machine learning model; The method of any one of claims 1 to 5, wherein the score is generated based at least in part on the game state data provided as the additional input to the trained machine learning model. [5] Prior to said receiving of said sensor data, accessing, by the computing system, historical sensor data generated during a past session of the video game or a past session of a different video game; accessing, by the computing system, historical game control data generated within the period of time since the historical sensor data was received; labeling the historical sensor data with a label indicative of one of a plurality of types of user input corresponding to the historical game control data received within the period of time; The method of claim 1, further comprising: training a machine learning model using the historical sensor data as training data to obtain the trained machine learning model. [6] The method according to [1], wherein the certain period of time is within a range of about 1 millisecond to about 200 milliseconds. [7] the sensor data is received over a computer network from a first client machine, and the causing the presentation of the video game data on the display includes transmitting the video game data over the computer network to a second client machine, the method comprising: measuring the latency of the computer network to obtain a measured latency value; determining the period of time based at least in part on the measured latency value; and The method of any one of claims 1 to 5, further comprising: selecting the trained machine learning model from a plurality of trained machine learning models based at least in part on the period of time. [8] 1. A method comprising: receiving, by a computing system, game state data from a video game; providing said game state data as an input to a trained machine learning model; generating as an output from the trained machine learning model a score associated with a probability that a type of user input will be provided to a game controller associated with a player of the video game and corresponding game control data will be received by the computing system within a period of time following the receipt of the game state data; generating, by the computing system, game control data corresponding to a type of the user input; providing the game control data as input to the video game; receiving video game data as output from the video game; causing, by the computing system, presentation of the video game data on a display. [9] The type of user input is: Activating a particular finger control of the game controller; moving the game controller in a particular direction; an object hovering over or touching a particular portion of the game controller; or The method of claim 8, further comprising at least one of: the object pressing against a portion of the game controller with a particular amount of force.
[10] prior to said receiving said game state data; accessing, by the computing system, historical game state data including game states of the video game that occurred during past sessions of the video game; accessing, by the computing system, historical game control data generated within the period of time since the game state occurred during the past session; labeling the historical game state data with a label indicative of one of a plurality of types of user input corresponding to the historical game control data received within the period of time; The method of claim 8, further comprising training a machine learning model using the historical game state data as training data to obtain the trained machine learning model.
[11] receiving, by the computing system, sensor data associated with the player; In addition to providing the game state data as the input to the trained machine learning model, providing the sensor data as an additional input to the trained machine learning model; The method of claim 8, wherein the score is generated based at least in part on the sensor data provided as the additional input to the trained machine learning model.
[12] The method of any one of claims 1 to 11, wherein the sensor data includes raw sensor data generated by one or more physical sensors of the game controller, the raw sensor data including at least one of raw gyroscope data, raw accelerometer data, or raw capacitance sensor data.
[13] prior to said generating said game control data, accessing data indicative of a type of user input for which the game control data is being actively generated on behalf of a player to compensate for latency; determining that the type of user input is one of the types of user input; The method according to [8], wherein the generating of the game control data is in response to determining that the type of the user input is one of the types of user input.
[14] determining that the score satisfies a threshold score associated with the type of user input, the threshold score being one of a plurality of different threshold scores associated with different types of user input; The method of any one of claims 8 to 11, wherein the generating of the game control data is based at least in part on the score satisfying the threshold score.
[15] 1. A system comprising: one or more processors; and a memory storing computer-executable instructions that, when executed by the one or more processors, cause the system to: receiving sensor data associated with a player of a video game; providing the sensor data as an input to a trained machine learning model; generating as output from the trained machine learning model the score indicative of a probability that a type of user input will be provided to a game controller associated with the player and corresponding game control data will be received by the system within a period of time following the receipt of the sensor data; generating game control data corresponding to a type of the user input; providing the game control data as input to the video game; receiving video game data as output from the video game; causing presentation of said video game data on a display.
[16] The type of user input is: Activation of a particular finger-operated control of a game controller; moving the game controller in a particular direction; an object hovering over or touching a particular portion of the game controller; or The system of
[15] , wherein the object presses against a portion of the game controller with a particular amount of force.
[17] The computer-executable instructions, when executed by the one or more processors, cause the system to: determining that the score satisfies a threshold score associated with the type of user input, the threshold score being one of a plurality of different threshold scores associated with different types of user input; The system of any one of claims 1 to 15, wherein generating the game control data is based at least in part on the score satisfying the threshold score.
[18] The computer-executable instructions, when executed by the one or more processors, cause the system to: receiving actual game control data associated with the player within the period of time from the receipt of the sensor data; and determining a prediction error based at least in part on comparing the actual game control data to the game control data.
[19] The computer-executable instructions, when executed by the one or more processors, cause the system to: The system of
[18] , further comprising: recording the prediction error in a database of recorded prediction errors.
[20] The system of
[15] , wherein the sensor data includes raw sensor data generated by one or more physical sensors of the game controller, and the raw sensor data includes at least one of raw gyroscope data, raw accelerometer data, or raw capacitance sensor data.
Claims
1. 1. A method comprising: receiving, by a computing system, sensor data associated with a player of a video game, the sensor data generated by at least one of a gyroscope of a game controller associated with the player, an accelerometer of the game controller, a pressure sensor of the game controller, or a biofeedback sensor on or proximate to the player; a user input to the game controller that causes receipt of corresponding game control data by the computing system within a period of time from the receipt of the sensor data; providing the sensor data as input to a trained machine learning model; generating a score as an output from the trained machine learning model, the score indicative of a probability that a type of user input will be provided to the game controller; and after generating said score and prior to actively generating game control data corresponding to said type of user input, accessing data indicative of a type of user input for which each game control data is to be generated on behalf of a player to compensate for latency; determining that the type of the user input is one of the user input types; in response to determining that the type of user input is one of the types of user input, proactively generating, by the computing system, the game control data based at least in part on the score prior to receiving actual game control data from a client machine associated with the player; providing the game control data as input to the video game; receiving video game data as output from the video game based at least in part on the game control data; and causing, by the computing system, the presentation of the video game data on a display without waiting for the actual game control data from the client machine to be received by the computing system.
2. The type of user input is: Activating a particular finger control of the game controller; moving the game controller in a particular direction; an object hovering over or touching a particular portion of the game controller; or The method of claim 1 , further comprising at least one of: the object pushing against a portion of the game controller with a particular amount of force.
3. receiving, by the computing system, game state data from the video game prior to said predicting; In addition to providing the sensor data as the input to the trained machine learning model, providing the game state data as an additional input to the trained machine learning model; The method of claim 1 , wherein the score is generated based at least in part on the game state data provided as the additional input to the trained machine learning model.
4. Prior to said receiving of said sensor data, accessing, by the computing system, historical sensor data generated during a past session of the video game or a past session of a different video game; accessing, by the computing system, historical game control data generated within a second period of time after the historical sensor data was received; labeling the historical sensor data with a label indicative of one of a plurality of types of user input corresponding to the historical game control data received within the second time period; The method of claim 1 , further comprising: training a machine learning model using the historical sensor data as training data to obtain the trained machine learning model.
5. The method of claim 1 , wherein the period of time is in a range of 1 millisecond to 200 milliseconds.
6. The client machine is a first client machine, the sensor data is received from the first client machine over a computer network, and the causing the presentation of the video game data on the display includes transmitting, over the computer network, the video game data to a second client machine, the method comprising: measuring the latency of the computer network to obtain a measured latency value; determining the period of time based at least in part on the measured latency value; and 10. The method of claim 1, further comprising: selecting the trained machine learning model from a plurality of trained machine learning models based at least in part on the period of time.
7. 1. A method comprising: receiving, by a computing system, sensor data associated with a player of a video game, the sensor data generated by at least one of a gyroscope of a game controller associated with the player, an accelerometer of the game controller, a pressure sensor of the game controller, or a biofeedback sensor on or proximate to the player; providing the sensor data as an input to a trained machine learning model; generating as an output from the trained machine learning model a score associated with a probability that a type of user input will be provided to the game controller and corresponding game control data will be received by the computing system within a period of time following the receipt of the sensor data; after generating said score and prior to actively generating game control data corresponding to said type of user input, accessing data indicative of a type of user input for which each game control data is to be generated on behalf of a player to compensate for latency; determining that the type of the user input is one of the user input types; in response to determining that the type of user input is one of the types of user input, generating, by the computing system, the game control data prior to receiving actual game control data from a client machine associated with the player; providing the game control data as input to the video game; receiving video game data as output from the video game; and causing, by the computing system, the presentation of the video game data on a display without waiting for the actual game control data from the client machine to be received by the computing system.
8. The type of user input is: Activating a particular finger control of the game controller; moving the game controller in a particular direction; an object hovering over or touching a particular portion of the game controller; or The method of claim 7 , further comprising at least one of: the object pushing against a portion of the game controller with a particular amount of force.
9. Prior to said receiving of said sensor data, accessing, by the computing system, historical sensor data generated during a past session of the video game or a past session of a different video game; accessing, by the computing system, historical game control data generated within a second period of time after the historical sensor data was received; labeling the historical sensor data with a label indicative of one of a plurality of types of user input corresponding to the historical game control data received within the second time period; The method of claim 7 , further comprising: training a machine learning model using the historical sensor data as training data to obtain the trained machine learning model.
10. receiving, by the computing system, game state data from the video game; In addition to providing the sensor data as the input to the trained machine learning model, providing the game state data as an additional input to the trained machine learning model; The method of claim 7 , wherein the score is generated based at least in part on the game state data provided as the additional input to the trained machine learning model.
11. The method further comprising, prior to said actively generating said game control data, determining that said score satisfies a threshold score associated with said type of user input, wherein said threshold score is one of a plurality of different threshold scores associated with different types of user input; The method of claim 7 , wherein the actively generating the game control data is based at least in part on the score satisfying the threshold score.
12. 1. A system comprising: one or more processors; and a memory storing computer-executable instructions that, when executed by the one or more processors, cause the system to: receiving sensor data associated with a player of a video game, the sensor data generated by at least one of a gyroscope of a game controller associated with the player, an accelerometer of the game controller, a pressure sensor of the game controller, or a biofeedback sensor on or proximate to the player; providing the sensor data as an input to a trained machine learning model; generating as an output from the trained machine learning model a score indicative of a probability that a type of user input will be provided to the game controller and corresponding game control data will be received by the system within a period of time from the receipt of the sensor data; after generating said score and prior to actively generating game control data corresponding to said type of user input, accessing data indicative of a type of user input for which each game control data is to be generated on behalf of a player to compensate for latency; determining that the type of the user input is one of the user input types; in response to determining that the type of user input is one of the types of user input, proactively generating game control data prior to receiving actual game control data from a client machine associated with the player; providing the game control data as input to the video game; receiving video game data as output from the video game; causing the presentation of the video game data on a display without waiting for actual game control data from the client machine to be received by the system.
13. The type of user input is: Activation of a particular finger-operated control of a game controller; moving the game controller in a particular direction; an object hovering over or touching a particular portion of the game controller; or The system of claim 12 , wherein the object includes at least one of: pushing a portion of the game controller with a particular amount of force.
14. The computer-executable instructions, when executed by the one or more processors, cause the system to: determining, prior to said actively generating said game control data, that said score satisfies a threshold score associated with said type of user input, said threshold score being one of a plurality of different threshold scores associated with different types of user input; The system of claim 12 , wherein the actively generating the game control data is based at least in part on the score satisfying the threshold score.
15. The computer-executable instructions, when executed by the one or more processors, cause the system to: receiving actual game control data from the client machine within the period of time from the receipt of the sensor data; 13. The system of claim 12, further comprising: determining a prediction error based at least in part on comparing the actual game control data to the game control data.
16. The computer-executable instructions, when executed by the one or more processors, cause the system to:
16. The system of claim 15, further comprising: recording the prediction error in a database of recorded prediction errors.
17. The method of claim 1, wherein causing presentation of the video game data on the display includes causing presentation of at least one of an animation associated with a type of user input or a lead-in animation associated with the animation.
18. The method of claim 7, wherein causing presentation of the video game data on the display includes causing presentation of at least one of an animation associated with a type of user input or a lead-in animation associated with the animation.
19. The system of claim 12, wherein causing presentation of the video game data on the display includes causing presentation of at least one of an animation associated with a type of user input or a lead-in animation associated with the animation. receiving actual game control data from the client machine within the period of time from receiving the sensor data; The method of claim 7 , further comprising: determining a prediction error based at least in part on comparing the actual game control data to the game control data.
Citation Information
Patent Citations
Program, information storage medium, and game machine
JP2010022492A
Information processing apparatus, information processing method, and program
JP2013084175A
Previous image generation program, virtual machine control program, and computer-readable storage medium
JP2015212920A
Flexible computer gaming based on machine learning
WO2019075132A1
