Machine learning-based confidence scoring based on sensor data

Abstract

The trained machine learning model is used to determine scores for user accounts registered with the video game service, and the scores are used to pit players against each other in a multiplayer video game setting. For example, sensor data received from a client machine is input into the trained machine learning model, and the model generates as output a score related to the probability that game control data received from the client machine was generated by the handheld device, as opposed to being synthesized and / or modified using software. In this manner, a subset of logged-in user accounts running the video game can be assigned to different matches (e.g., isolating non-human players from human players) based at least in part on the scores determined for the logged-in user accounts, and the video game is played in the matches assigned to each logged-in user account.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a PCT application claiming priority to pending U.S. patent application Ser. No. 16 / 663,041, entitled "MACHINE-LEARNED TRUST SCORING BASED ON SENSOR DATA," filed Oct. 24, 2019, which claims priority under 35 U.S.C. § 120 as a continuation-in-part of pending U.S. patent application Ser. No. 16 / 125,224, entitled "MACHINE-LEARNED TRUST SCORING FOR PLAYER MATCHMAKING," filed Sep. 7, 2018. Each of Ser. Nos. 16 / 663,041 and 16 / 125,224 is incorporated herein by reference in its entirety. [Background technology]

[0002] The constant or near-constant availability of wide area network communications, coupled with increasing performance of client machines, has led to an increase in the popularity of online multiplayer video games. In these multiplayer video games, video game services may use matchmaking systems to match players in groups so that the grouped players may play the video game together in a multiplayer mode. One popular video game genre that is often played by multiple players in a multiplayer mode is the first-person shooter genre. In this exemplary genre, teams of two or more players may compete in multiple rounds with the goal of winning enough rounds to win the match. Players on the same team may work together to achieve objectives while competing against players on opposing teams.

[0003] While the majority of video game players do not engage in cheating behavior, there is often a small minority of players who cheat to gain an advantage over other players. Often, cheating players employ third-party software that provides them with an informational or mechanical advantage over other players. For example, the third-party software may be configured to extract location data regarding the locations of other players and present these locations to the cheating player. This information advantage allows the cheating player to ambush other players or otherwise exploit the location information revealed by the third-party software. A more prominent method of cheating is the use of third-party software that can detect the location of another player (e.g., by programmatically moving the mouse cursor to the target player and automatically firing a weapon) and automate the movement of the cheater's avatar. That is, some players cheat by having the third-party software play the video game on their behalf, using computer-controlled algorithms that mechanically improve the cheating player's performance.

[0004] Generally, bad player behavior, such as cheating, ruins the gameplay experience for players who wish to play the game legitimately. Thus, when a player who seeks to engage in bad behavior matches with other players who exhibit good behavior, the multiplayer video game experience is ruined for the good-behavior player. Currently used systems that attempt to identify players who are likely to engage in bad behavior in the future are somewhat inaccurate in their predictions. These systems are also generally static in the sense that they are based on statically prescribed rules and inputs; that is, manual adjustments are required to change the output of the system. The disclosure made herein is presented with respect to these and other considerations. [Brief description of the drawings]

[0005] The detailed description is now set forth with reference to the accompanying drawings. In each figure, the leftmost digit(s) of a reference number identifies the figure in which the reference number first appears. Use of the same reference number in different figures indicates similar or identical components or features.

[0006] [Figure 1] FIG. 1 illustrates an example environment including a remote computing system configured to train and use a machine learning model to determine confidence scores associated with likely player actions, and match players based on the machine learned confidence scores.

[0007] [Diagram 2] FIG. 2 shows a block diagram illustrating example components of the remote computing system of FIG. 1 and a diagram illustrating how machine-learned confidence scoring can be used for player matchmaking.

[0008] [Diagram 3] FIG. 1 is a flow diagram of an example process for training a machine learning model to predict the probability that a player will act or not act in accordance with a particular action.

[0009] [Figure 4] FIG. 2 is a flow diagram of an example process for utilizing a trained machine learning model to determine a trust score for a user account, where the trust score is related to the probability that a player will or will not act in accordance with a particular behavior.

[0010] [Diagram 5] FIG. 1 is a flow diagram of an example process for assigning user accounts to different matches of a multiplayer video game based on machine-learned confidence scores associated with likely player actions.

[0011] [Figure 6]FIG. 6 illustrates an example environment including a block diagram illustrating example components of a handheld device, and shows how sensor data received by a remote computing system can be used for machine learning confidence scoring.

[0012] [Figure 7] FIG. 13 is a flow diagram of an example process for training a machine learning model using data including sensor data to predict the probability that game control data received from a client machine was generated by a handheld device.

[0013] [Figure 8] FIG. 1 is a flow diagram of an example process for utilizing a trained machine learning model to determine a trust score for a user account, the trust score relating to the probability that the game control data was generated by the handheld device.

[0014] [Figure 9] FIG. 13 is a flow diagram of an example process for assigning user accounts to different matches of a multiplayer video game based on a machine-learned confidence score associated with the probability that the game control data was generated by a handheld device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Described herein, among other things, are techniques, devices, and systems for generating confidence scores using machine learning approaches and then using the machine-learned confidence scores to match players in a multiplayer video game setting. The disclosed techniques may be implemented, at least in part, by a remote computing system that distributes video games (and their contents) to client machines of a user community as part of a video game service. These client devices may individually install a client application configured to run video games received (e.g., downloaded, streamed, etc.) from the remote computing system. The video game platform enables registered users of the community to play the video games as "players." For example, a user may load the client application, log in with a registered user account, select a desired video game, and run the video game on his or her client machine through the client application.

[0016] Whenever such users access and use the video game platform, data may be collected by the remote computing system, and this data may be associated with the registered user accounts and maintained by the remote computing system. It will be appreciated that over time, a vast collection of historical data tied to the registered user accounts may be available to the remote computing system. The remote computing system may then use a portion of the historical data as training data to train one or more machine learning models. For example, the portion of the historical data associated with a sampled set of user accounts may be represented by a set of features and labeled to indicate players who have behaved in a particular way in the past while playing a video game. The machine learning models trained on this data may predict player behavior by outputting machine learned scores (e.g., confidence scores) for user accounts registered with the video game service. These machine learned scores may be used for player matchmaking, so that players who are likely to behave or not behave in accordance with a particular behavior may be grouped together in a multiplayer video game setting.

[0017] In one example process, a computing system may determine scores (e.g., confidence scores) for a plurality of user accounts registered with a video game service. The individual scores may be determined by accessing data associated with the individual user accounts, providing the data as input to a trained machine learning model, and generating a score associated with the individual user accounts as an output from the trained machine learning model. The score is related to a probability that a player associated with the individual user account will behave or not behave in accordance with a particular behavior while playing one or more video games in a multiplayer mode. The computing system may then receive information from a plurality of client machines indicating logged-in user accounts that are logged into a client application that executes the video games. Additionally, the computing system may define a plurality of matches in which players will be grouped to play the video games in a multiplayer mode. The plurality of matches may include at least a first match and a second match. The computing system may assign a first subset of the logged-in user accounts to the first match and a second subset of the logged-in user accounts to the second match based at least in part on the scores determined for the logged-in user accounts. Further, a first subset of client machines associated with a first subset of logged-in user accounts may be caused to run a video game in a first match, while simultaneously a second subset of client machines associated with a second subset of logged-in user accounts may be caused to run a video game in a second match.

[0018] Because human players who wish to play video games legitimately often utilize handheld devices (e.g., handheld game controllers) to play the video games, and because software configured to synthesize and / or modify game control data for purposes of cheating does not use a handheld device to generate the synthesized and / or modified game control data, sensor data associated with one or more sensors of the handheld device may be leveraged using machine learning scoring to distinguish legitimate human players from software programmed to cheat, such that players using software to cheat may be isolated from legitimate human players. Thus, described herein are techniques, devices, and systems for receiving game control data and sensor data from one or more client machines interacting with a video game platform and providing at least the sensor data as input to a machine learning model trained to generate a confidence score associated with a user account. In this instance, the confidence score may relate to a probability that game control data received from a given client machine was generated by a handheld device associated with the given client machine. Thus, the trust score may be utilized in player matchmaking to mitigate instances where a non-human player (e.g., software designed to cheat) is matched with a human player who legitimately wishes to play a video game with one or more other human players.

[0019] To illustrate, whenever a user plays a video game via a video game platform, the user may use a handheld device, such as a handheld game controller, to control aspects of the video game. A given handheld device may include one or more sensors, such as, but not limited to, a gyroscope, an accelerometer, and / or a touch sensor, among other possible sensors. A touch sensor (e.g., a capacitance pad) may be disposed, for example, under or on the surface of the device, and / or in or on a finger-operated control. The touch sensor may be configured to detect the proximity of a finger to the surface or finger-operated control, and in response, the touch sensor may generate sensor data indicative of the proximity of the finger to the touch sensor. As another example, a gyroscope and / or accelerometer attached to the housing of the handheld device may detect motion of the handheld device in different directions (e.g., movement by translation, rotation, and / or tilting, etc.), and in response, the gyroscope and / or accelerometer may generate sensor data indicative of characteristics of this motion. In general, sensor data can be utilized for a variety of purposes, such as to control aspects of a video game (e.g., to control a player-controlled character, to rotate a virtual camera that directs what is seen on a display, etc.). When sensor data is used as game control data in this manner, the sensor data may in some cases be modified (e.g., filtered / attenuated, amplified, etc.) before being used to control aspects of the video game. However, the techniques and systems described herein are directed to handheld devices configured to transmit raw, unfiltered sensor data generated by one or more sensors of the handheld device to a remote computing system (e.g., by transmitting the sensor data via an associated client machine).This raw sensor data may be transmitted along with game control data (e.g., data generated by button presses, joystick deflections, etc., as well as modified sensor data) that will be processed to control aspects of the video game.

[0020] Over time, as users interact with the video game platform using handheld devices having one or more sensors, a vast collection of historical sensor data and game control data may be associated with registered user accounts and used to train machine learning models, as described above and elsewhere herein. For example, the remote computing system may use a portion of the historical sensor data as training data to train one or more machine learning models. For example, a portion of the historical sensor data associated with a sampled set of user accounts may be represented by a set of features, and each user account in the sampled set may be labeled to indicate whether the historical game control data associated with the user account was generated by a real handheld device while a human player held and manipulated the handheld device to generate the game control data. A machine learning model trained on this sensor data may determine whether the game control data associated with a particular user account was generated by a physical handheld device, as opposed to being synthesized and / or modified by software. In this manner, the machine learning model may detect cheating behavior based on the sensor data provided as input to the machine learning model. For example, when a human operates a handheld device, such as a handheld game controller, there may be slight movements of the handheld device that are unintentional movements, but these slight movements may be indicated in sensor data generated by one or more physical sensors of the handheld device (e.g., touch sensors, gyroscopes, and / or accelerometers, etc.) A machine learning model may be trained using at least this sensor data as training data to distinguish between human players and non-human players.For example, by recognizing subtle nuances in new sensor data received from a client machine, the machine learning model can output a confidence score indicating that the corresponding game control data was generated by a physical sensor of a real handheld device, such as a device that is mechanically operated (e.g., by a human user). Conversely, by failing to receive sensor data from another client machine, or failing to recognize subtle nuances in sensor data received from other client machines that are expected to be indicated in the sensor data, the machine learning model can output a confidence score indicating that the corresponding game control data was synthesized and / or modified using software, such as, for example, game control data synthesized and / or modified by a so-called "aimbot" that programmatically generates game control data to automatically aim and fire a player-controlled character's weapon. Thus, using machine-learned scores for player matchmaking can help avoid matching human players who wish to play a video game legitimately with non-human players, such as software designed to cheat in a video game's multiplayer mode.

[0021] In an exemplary process, a computing system may receive game control data and sensor data from a client machine and determine a score (e.g., a confidence score) associated with the received data for a user account registered with the video game service. For example, the computing system may provide the received sensor data as an input to a trained machine learning model and generate a score associated with the user account as an output from the trained machine learning model. The score is related to a probability that the received game control data was generated by a handheld device associated with the client machine. Based on the score, the computing system may assign the user account to an assigned match of a plurality of matches, including at least one of a first match designated for a human player or a second match designated for a non-human player. Thus, a video game may be executed for the user account in the assigned match.

[0022] The techniques and systems described herein can provide an improved gaming experience for users who wish to play a video game in a multiplayer mode in the way that it was intended. This is because the techniques and systems described herein can match players (including non-human players such as "aimbots") who are more likely to perform malicious actions (e.g., cheat) and isolate these players from other trustworthy players who are more likely to play the video game legitimately. For example, a trained machine learning model can learn to predict which players are likely to cheat and which are less likely to cheat by attributing a trust score corresponding to each player's user account that indicates the player's tendency to cheat (or not cheat). In this way, players with low trust scores (e.g., below a threshold) can be matched and isolated from other players whose user accounts are ascribed high trust scores (e.g., above a threshold), allowing the trustworthy players to play in a match while leaving out the players who are more likely to cheat. Although the use of threshold scores has been described as one exemplary manner of providing match assignments, other techniques are contemplated, such as clustering algorithms or other statistical approaches that use confidence scores to preferentially match user accounts (players) with "similar" confidence scores (e.g., based on a similarity metric such as a distance metric, a dispersion metric, etc.).

[0023] The techniques and systems described herein also improve upon existing matchmaking techniques that use static rules to determine a user's trust level. Machine learning models can learn to identify complex relationships of player behavior to more accurately predict player behavior, which is not possible with static rule-based approaches. Thus, the techniques and systems described herein generate trust scores that more accurately predict player behavior, reduce false positive rates, and reduce instances of players being attributed inaccurate trust scores, compared to existing trust systems. The techniques and systems described herein are also more adaptable to fluctuations in player behavior than existing systems, since the machine learning models can be retrained with new data to adapt the machine learning models to understanding player behavior over time as player behavior changes. The techniques and systems described herein may further enable one or more devices to conserve resources in terms of processing resources, memory resources, networking resources, and the like, in various manners described herein.

[0024] While many of the examples described herein refer to "cheating" as a target behavior against which players can be scored and grouped for matchmaking purposes, it should be understood that the techniques and systems described herein may be configured to use machine learning scoring approaches to identify any type of behavior (good or bad) and predict the likelihood that a player will engage in that behavior for player matchmaking purposes. Thus, the techniques and systems may be extended beyond the notion of "trust" scoring in the context of bad behaviors such as cheating, and may more broadly attribute scores to user accounts that indicate compatibility or affinity between players.

[0025] FIG. 1 illustrates an example environment 100 including a remote computing system configured to train and use machine learning models to determine confidence scores associated with likely player actions and match players based on the machine-learned confidence scores. A community of users 102 may be associated with one or more client machines 104. The client machines 104(1)-(N) shown in FIG. 1 represent computing devices on which the users 102 may utilize to run programs, such as video games. For this reason, the users 102 shown in FIG. 1 may be referred to as "players" 102, and these names may be used interchangeably herein to refer to human operators of the client machines 104. That is, the client machines 104 may be implemented as any suitable type of computing device configured to execute video games and render graphics on an associated display, including, without limitation, a personal computer (PC), a desktop computer, a laptop computer, a mobile phone (e.g., a smartphone), a tablet computer, a portable digital assistant (PDA), a wearable computer (e.g., a virtual reality (VR) headset, an augmented reality (AR) headset, smart glasses, etc.), an in-vehicle (e.g., in-car) computer, a television (smart TV), a set-top box (STB), a game console, and / or any similar computing device. Furthermore, the client machines 104 may vary with respect to their respective platforms (e.g., hardware and software). For example, the multiple client machines 104 shown in FIG. 1 may represent different types of client machines 104 having various capabilities with respect to processing capabilities (e.g., central processing unit (CPU) model, graphics processing unit (GPU) model, etc.), graphics driver versions, etc.

[0026] The client machine 104 may communicate with a remote computing system 106 (sometimes shortened herein to "computing system 106" or "remote system 106") via a computer network 108. This computer network 108 may represent or include, but is not limited to, the Internet, other types of data and / or voice networks, wired infrastructures (e.g., coaxial cable, fiber optic cable, etc.), wireless infrastructures (e.g., radio frequency (RF), cellular, satellite, etc.), and / or other connection technologies. The computing system 106 may, in some cases, be part of a network-accessible computing platform that is maintained and accessed via the computer network 108. Such network-accessible computing platforms may be referred to using terms such as "on-demand computing," "software as a service (SaaS)," "platform computing," "network-accessible platforms," ​​"cloud services," "data centers," and the like.

[0027] In some embodiments, the computing system 106 serves as or has access to a video game platform that implements a video game service to distribute (e.g., download, stream, etc.) the video game 110 (and its content) to the client machines 104. In one example, the client machines 104 may each install a client application thereon. The installed client application may be a video game client (e.g., game software for playing the video game 110). The client machines 104 with the client application installed may be configured to download, stream, or otherwise receive programs (e.g., the video game 110, and its content) from the computing system 106 over the computer network 108. To this end, any type of content distribution model may be utilized, such as a direct purchase model in which the program (e.g., the video game 110) can be individually purchased for download and execution on the client machine 104, a subscription-based model, a content distribution model in which the program is rented or leased for a period of time, streamed, or otherwise made available to the client machine 104, etc. Thus, each client machine 104 may include one or more installed video games 110 that are executable by loading a client application.

[0028] As indicated by reference numeral 112 in FIG. 1, a client machine 104 may be used to register with and subsequently log into a video game service. A user 102 may create a user account for this purpose and may specify / set credentials (e.g., password, PIN, biometric ID, etc.) tied to the registered user account. As users 102 interact with the video game platform (e.g., by accessing their user / player profile with the registered user account, playing video games 110 on their respective client machines 104, etc.), the client machines 104 transmit data 114 to the remote computing system 106. For a given client machine 104, the data 114 transmitted to the remote computing system 106 may include, but is not limited to, user input data, video game data (e.g., game performance statistics uploaded to the remote system), social networking messages and related activities, an identifier (ID) of the video game 110 played on the client machine 104, etc. This data 114 may be streamed in real-time (or substantially real-time), transmitted to the remote system 106 at defined intervals, and / or uploaded in response to an event (e.g., the end of a video game).

[0029] 1 illustrates that the computing system 106 may store the data 114 collected from the client machines 104 in a data store 116, which may represent a data repository maintained by and accessible to the remote computing system 106. The data 114 may be organized in the data store 116 in any suitable manner, such as to associate user accounts with relevant portions of the data 114 associated with those user accounts. Over time, and given a large community of users 102 who frequently interact with the video game platform, sometimes over extended periods of time during a given session, a large amount of data 114 may be collected and maintained in the data store 116.

[0030] In step 1 of FIG. 1, the computing system 106 may train a machine learning model using historical data 114 sampled from the data store 116. For example, the computing system 106 may access a portion of the historical data 114 associated with a sampled set of user accounts registered with the video game service and may use the sampled data 114 to train a machine learning model. In some embodiments, the portion of the data 114 used as training data is represented by a set of features, and each user account in the sampled set is labeled with a label indicating whether the user account is associated with a player who acted in line with a particular behavior while playing at least one video game in the past. For example, if a player with a particular user account was banned by the video game service in the past for cheating, this "banned" may be used as one of multiple class labels for the particular user account. In this manner, a supervised learning approach may be employed to train a machine learning model to predict players who are likely to cheat in the future.

[0031] In step 2, the computing system 106 may score the multiple registered user accounts using the trained machine learning model. For example, the computing system 106 may access data 114 associated with the multiple registered user accounts from the data store 116, provide the data 114 as input to the trained machine learning model, and generate scores associated with the multiple user accounts as output from the trained machine learning model. These scores (sometimes referred to herein as "trust scores" or "trust coefficients") relate to the probability that players associated with the multiple user accounts will or will not behave in accordance with a particular behavior while playing one or more video games in a multiplayer mode. In the case of "bad" behaviors, such as cheating, the trust score may relate to the probability that the player will not cheat. In this case, a high trust score indicates a trustworthy user account, while a low trust score indicates an untrustworthy user account, which may be used as an indicator of players who are more likely to exhibit bad behaviors, such as cheating. In some embodiments, the scores are normalized variables in the range of [0,1]. This confidence score may have a monotonic relationship with the probability that the player will act (or possibly not act) along certain lines while playing the video game 110. The relationship between the score and the actual probability associated with a particular action may or may not be a linear relationship, although it is monotonic. Of course, the scoring may be implemented in any suitable manner to predict whether a player associated with a user account will or will not behave in a particular manner. FIG. 1 illustrates multiple user accounts 120 scored according to the techniques described herein. For example, a first score 118(1) (score=X) is attributed to a first user account 120(1), a second score 118(2) (score=Y) is attributed to a second user account 120(2), and so on, for any suitable number of registered user accounts 120.

[0032] With the machine-learned scores 118 determined for the multiple registered user accounts 120, the computing system 106 may be configured to match players in a multiplayer video game setting based at least in part on the machine-learned scores 118. For example, FIG. 1 illustrates that the computing system 106 may receive information 122 from multiple client machines 104 that have begun execution of a video game 110. This information 122 may indicate to the computing system 106 that a set of logged-in user accounts 120 are currently executing the video game 110 on each client machine 104 via an installed client application. That is, when a player 102 logs in with his or her user account 120, begins execution of a particular video game 110, and requests to play in multiplayer mode, each client machine 104 may provide information 122 indicative of parity to the computing system 106.

[0033] In step 3, the computing system 106, in response to receiving this information 122 from the client machine 104, may match the players 102 by defining a plurality of matches in which the players 102 will be grouped to play the video game 110 in a multiplayer mode, and by providing match assignments 124 to the client machine 104 for assigning a subset of the logged-in user accounts 120 to separate matches of the plurality of matches based at least in part on the machine-learned scores 118 determined for the logged-in user accounts 120. In this manner, if the trained machine learning model assigns a low (e.g., below a threshold) score 118 to a first subset of the logged-in user accounts 120 and a high (e.g., above a threshold) score 118 to a second subset of the logged-in user accounts 120, the first subset of the logged-in user accounts 120 may be assigned to a first match of the plurality of matches and the second subset of the logged-in user accounts 120 may be assigned to a second separate match of the plurality of matches. In this manner, subsets of players 102 having similar scores 118 can be grouped together and still be isolated from other players having scores 118 that differ from the subset of scores 118 of players 102.

[0034] In step 4, a client application on each machine 104 running the video game 110 may run the video game 110 in a match assigned to the logged-in user account 120 in question. For example, a first subset of client machines 104 may be associated with a first subset of logged-in user accounts 120 assigned to a first match, and this first subset of client machines 104 may run the video game 110 in the first match, while a second subset of client machines 104 associated with a second subset of logged-in user accounts 120 assigned to a second match may run the video game 110 in the second match. Grouping players into matches based at least in part on the machine-learned scores 118 may allow the system to group players who are predicted to behave badly (e.g., by cheating) together in the same match, thereby improving the in-game experience for at least a portion of the group of players 102, since the badly behaving players may be kept isolated from other players who legitimately wish to play the video game 110.

[0035] FIG. 2 shows a block diagram illustrating exemplary components of the remote computing system 106 of FIG. 1 and an illustration illustrating how machine learning confidence scoring may be used for player matchmaking. In the illustrated implementation, the computing system 106 includes, among other components, one or more processors 202 (e.g., central processing unit (CPU)), memory 204 (or non-transitory computer readable medium 204), and a communication interface 206. The memory 204 (or non-transitory computer readable medium 204) may include volatile and non-volatile memory, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules, or other data. Such memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, RAID storage systems, or any other medium that may be used to store the desired information and that may be accessed by a computing device. The computer-readable medium 204 may be implemented as a computer-readable storage medium ("CRSM"), which may be any available physical medium that is accessible by the processor 202 to execute instructions stored on the memory 204. In one basic implementation, the CRSM may include random access memory ("RAM") and flash memory. In other implementations, the CRSM may include, but is not limited to, read-only memory ("ROM"), electrically erasable programmable read-only memory ("EEPROM"), or any other tangible medium that may be used to store desired information and that may be accessed by the processor 202. The video game services 208 may represent instructions stored in the memory 204 that, when executed by the processor 202, cause the computing system 106 to perform the techniques and operations described herein.

[0036] For example, the video game service 208 may include, among other possible components, a training component 210, a scoring component 212, and a matchmaking component 214. The training component 210 may be configured to train a machine learning model using a portion of the data 114 in the data store 116 associated with the sampled set of user accounts 120 as training data to obtain a trained machine learning model 216. The trained machine learning model 216 is usable by the scoring component 212 to determine scores 118 (e.g., confidence scores 118) for the multiple registered user accounts 120. The matchmaking component 214 provides match assignments 124 based at least in part on the machine-learned scores 118 such that players are grouped into separate ones of the multiple matches 218 (e.g., a first match 218(1), a second match 218(2), etc.) for playing the video game 110 in a multiplayer mode. FIG. 2 illustrates how a first group of players 102 associated with a first subset of logged-in user accounts 120 (e.g., 10 players 102) may be assigned to a first match 218(1), and a second group of players 102 associated with a second subset of logged-in user accounts 120 (e.g., another 10 players 102) may be assigned to a second match 218(2). Of course, any number of matches 218 may be defined, which may depend on the number of logged-in user accounts 120 requesting to run the video game 110 in multiplayer mode, the capacity limitations of the network traffic that the computing system 106 can handle, and / or other factors. In some embodiments, other factors (e.g., skill level, geographic region, etc.) are considered in the matchmaking process, which may cause a further division and / or subdivision of players into fewer or more matches 218.

[0037] As mentioned above, the score 118 determined by the scoring component 212 (e.g., output by the trained machine learning model 216) is the machine learned score 118. Machine learning generally involves processing a set of examples (called "training data") to train a machine learning model. The machine learning model 216 is a learned mechanism that, once trained, can receive new data as input and estimate or predict an outcome as output. For example, the trained machine learning model 216 can include a classifier tasked with classifying an unknown input (e.g., an unknown image) as one of multiple class labels (e.g., labeling an image as a cat or a dog). In some cases, the trained machine learning model 216 is configured to implement a multi-label classification task (e.g., labeling an image as a "cat", "dog", "duck", "penguin", etc.). Additionally or alternatively, the trained machine learning model 216 can be trained to infer a probability or set of probabilities of a classification task based on unknown data received as input. In the context of the present disclosure, the unknown input may be data 114 associated with individual user accounts 120 registered with the video game service, and the trained machine learning model 216 may be tasked with outputting a score 118 (e.g., confidence score 118) indicative of or otherwise related to the probability that the individual user account 120 is in one of a number of classes. For example, the score 118 may be related to the probability that a player 102 associated with the individual user account 120 will act (or, as the case may be, will not act) along a particular course of action while playing the video game 110 in a multiplayer mode. In some embodiments, the score 118 is a variable that is normalized in the range of [0,1]. This confidence score 118 may be monotonically related to the probability that a player 102 will act (or, as the case may be, will not act) along a particular course of action while playing the video game 110. The relationship between the score 118 and the actual probability associated with a particular action may or may not be a linear relationship, although it is monotonically related.In some embodiments, the trained machine learning model 216 may output a set of probabilities (e.g., two probabilities) or scores associated therewith, where one probability (or score) is associated with the probability that the player 102 will act in accordance with a particular behavior, and the other probability (or score) is associated with the probability that the player 102 will not act in accordance with the particular behavior. The score 118 output by the trained machine learning model 216 may be associated with either of these probabilities to guide the matchmaking process. In an exemplary embodiment, the particular behavior may be cheating. In this embodiment, the score 118 output by the trained machine learning model 216 is associated with the likelihood that the player 102 associated with the individual user account 120 will or will not cheat while playing the video game 110 in a multiplayer mode. Thus, the score 118 may, in some embodiments, indicate a level of trustworthiness of the player 102 associated with the individual user account 120, which is why the score 118 described herein may be referred to as a "confidence score" 118.

[0038] The trained machine learning model 216 may represent a single model or an ensemble of base-level machine learning models and may be implemented as any type of machine learning model 216. For example, machine learning models 216 suitable for use in the techniques and systems described herein include, but are not limited to, neural networks, tree-based models, support vector machines (SVMs), kernel methods, random forests, splines (e.g., multivariate fitted regression splines), hidden Markov models (HMMs), Kalman filters (or extended Kalman filters), Bayesian networks (or Bayesian belief networks), expectation maximization, genetic algorithms, linear regression algorithms, nonlinear regression algorithms, logistic regression-based classification models, or ensembles thereof. An "ensemble" may include a collection of machine learning models 216 whose outputs (predictions) are combined, such as by using a weighted average or voting. Individual machine learning models in an ensemble may have different expertise, and the ensemble may operate as a committee of individual machine learning models that are collectively "smarter" than the individual machine learning models in the ensemble.

[0039] The training data used to train the machine learning model 216 may include various types of data 114. In general, training data for machine learning may include two components: features and labels. However, in some embodiments, the training data used to train the machine learning model 216 may be unlabeled. Thus, the machine learning model 216 may be trainable using any suitable learning technique, such as supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, etc. The features included in the training data may be represented in a set of features, such as in the form of an n-dimensional feature vector of quantifiable information about attributes of the training data. The following is a list of exemplary features that may be included in the training data for training the machine learning model 216 described herein. However, it should be understood that the following list of features is not exhaustive and that the features used in training may include additional features not described herein, and in some cases, a portion, but not all, of the features described herein.Exemplary features included in the training data include the amount of time a player spends playing the video game 110 in general, the amount of time a player spends playing a particular video game 110, the amount of time per day that a player logs into and plays the video game 110, historical data for a player, such as total score (per match, per round, etc.), headshot percentage, kill count, death count, assist count, player rank, etc., the number and / or frequency of reports of a player cheating, the number and / or frequency of a player not guilty of cheating, the number and / or frequency of a player guilty of cheating, a confidence value (score) output by a machine learning model that detected a player cheating during a video game, the number of user accounts 120 associated with a single player (which may be inferred from common addresses, phone numbers, payment methods, etc. tied to multiple user accounts 120). ), the length of time the user account 120 has been registered with the video game service, the number of previously banned user accounts 120 associated with the player, the number and / or frequency of the player's money transactions on the video game platform, dollar amount per transaction, the number of digital items of monetary value associated with the player's user account 120, the number of times the user account 120 has changed hands (e.g., transfers between different owners / players), how often the user account 120 is transferred between players, the geographic locations from which the player logged into the video game service, the number of different payment instruments, phone numbers, mailing addresses, etc. associated with the user account 120, and / or how frequently these items have been changed, and / or any other suitable features that may be relevant to the calculation of the confidence score 118 indicative of a player's propensity to engage in certain behaviors. Additionally, as part of the training process, the training component 210 may set weights for the machine learning. These weights may be applied to the set of features included in the training data as derived from the historical data 114 in the data store 116.In some embodiments, weights set during the training process may be applied to parameters internal to the machine learning model (e.g., weights for neurons in a hidden layer of a neural network). These internal parameters of the machine learning model may or may not map one-to-one with individual input features of a set of features. The weights may indicate the influence of any given feature or parameter on the score 118 output by the trained machine learning model 216.

[0040] With respect to cheating specifically, this is an illustrative example of a type of behavior that can be used as a basis for matching players, and there may be behaviors associated with the user accounts 120 of players who plan to cheat in the video game 110 that are different from the behaviors associated with the user accounts 120 of non-cheaters. Thus, the machine learning model 216 can learn to identify these behavioral patterns from the training data to identify with high confidence players who are likely to cheat and score them appropriately. It should be understood that outliers may exist in an ecosystem where the system may be configured to protect based on some known information about outliers. For example, professional players may exhibit different behaviors than average players, but these professional players may be at risk of being scored incorrectly. As another example, employees of a service provider of a video game service may log in with user accounts for research or quality control purposes and may behave in a manner different from that of the average player. These types of players / users 102 may be treated as outliers and may be proactively assigned a score 118 outside of the machine learning context that attributes high confidence to these players / users 102. In this manner, known professional players, employees of a service provider, etc. may be assigned authority scores 118 that cannot be altered by the scoring component 212 to prevent these players / users 102 from playing against players with bad behavior.

[0041] The training data may also be labeled for supervised learning approaches. Again, using cheating as an example type of behavior that can be used to match players, the labels in this example may indicate whether a user account 120 has been banned from playing the video game 110 via the video game service. The data 114 in the data store 116 may include some data 114 associated with players who have been banned for cheating and some data 114 associated with players who have not been banned for cheating. An example of this type of ban is the Valve Anti-Cheat (VAC) ban utilized by Valve Corporation of Bellevue, Washington. For example, the computing system 106, and / or an authorized user of the computing system 106 may be able to detect that unauthorized third-party software has been used to cheat. In these cases, after undergoing a rigorous verification process to ensure that the determination is correct, the cheating user account 120 may be banned by flagging it as banned in the data store 116. Thus, the status of the user account 120 in terms of whether the account is banned or not can be used as an example of positive and negative discipline.

[0042] It should be understood that past player behavior, such as past cheating, can be indicated in other ways. For example, a mechanism may be provided to the user 102, or even a separate machine learning model, to detect and report players suspected of cheating. These reported players may be placed before a jury of peers who review the reported player's gameplay and render a verdict (e.g., cheated or did not cheat). If enough other players determine that the reported player's behavior constitutes cheating, a high confidence threshold may be reached and the reported player may be convicted of cheating and receive a ban on the user account 120, which may also constitute a label for the associated training data.

[0043] FIG. 2 illustrates examples of other behaviors that can be used as a basis for player matchmaking, other than cheating. For example, the trained machine learning model 216 can be configured to output a confidence score 118 associated with the probability that a player will or will not act in line with a game abandoning behavior (e.g., by abandoning (or quitting) a video game mid-match). Game abandoning, like cheating, is a behavior that tends to ruin the gameplay experience of players who do not abandon. As another example, the trained machine learning model 216 can be configured to output a confidence score 118 associated with the probability that a player will or will not act in line with a griefing behavior. A "griefer" is a player in a multiplayer video game who intentionally annoys and harasses other players in the video game 110, which can ruin the gameplay experience of players who are not griefing. As another example, the trained machine learning model 216 can be configured to output a confidence score 118 associated with the probability that a player will or will not act in line with a vulgar language behavior. Often, multiplayer video games allow players to engage in chat sessions or other social networking communications within the video game 110 that are visible to other players, and if a player uses vulgar language (e.g., curse words, offensive language, etc.), this can ruin the gameplay experience for players who do not use vulgar language. As yet another example, the trained machine learning model 216 may be configured to output a confidence score 118 associated with the probability that a player will or will not behave in line with a "high-skill" behavior. In this manner, the scoring can be used to identify high-skill or novice players from a set of players. This can help prevent situations where an experienced gamer creates a new user account masquerading as a player at a novice skill level so that they can play with amateur players.Thus, a player played in a first match 218(1) may be a player who is more likely to behave in line with certain "bad" behaviors (as determined from the machine-learned scores 118), while a player played in another match, such as a second match 218(2), may be a player who is less likely to behave in line with certain "bad" behaviors.

[0044] The distribution of trust scores 118 output for multiple players (user accounts 120) may be generally bimodal. For example, one peak of the statistical distribution of scores 118 may be associated with players who are more likely to act along a particular bad behavior, while the other peak of the statistical distribution of scores 118 may be associated with players who are less likely to act along that bad behavior. That is, the populations of badly behaved players and well behaved players may be separated by a clear margin in the statistical distribution. In this sense, if a new user account is registered with the video game service and assigned a trust score that is between two peaks in the statistical distribution, the user account will be driven quickly in some manner as the player interacts with the video game platform. Due to this tendency, the matchmaking parameters used by the matchmaking component 214 may be adjusted to treat players with trust scores 118 that are not within the bottom peak of the statistical distribution in a similar manner, and the matchmaking component 214 may be primarily concerned with isolating / isolating badly behaved players with trust scores 118 that are within the bottom peak of the statistical distribution. Although using a threshold score is described herein as one exemplary manner of providing match assignments, other techniques are contemplated, such as clustering algorithms that use confidence scores (e.g., based on a similarity metric such as a distance metric, a dispersion metric, etc.) to preferentially match user accounts (players) with "similar" confidence scores, or other statistical approaches.

[0045] Additionally, the matchmaking component 214 may not use player matchmaking for new user accounts 120 that have recently registered with the video game service. Rather, a rule may be implemented that a player accumulates a certain level of experience / achievement in an individual mode of play before being scored and matched against other players in the multiplayer mode of the video game 110. It should be understood that a player's path from the time they first launch the video game 110 to the time they access player matchmaking may vary greatly from player to player. Some players may take a long time to access the multiplayer mode with matchmaking, while others will go through a certification process quickly and easily. Once this certification process is in place, a user account that is to be scored for matchmaking purposes will have played the video game to provide enough data 114 to accurately score 118 the user account 120.

[0046] Again, as mentioned above, while many of the examples described herein refer to "bad" behaviors such as cheating, game abandoning, griefing, foul language, etc. as target behaviors that players may be scored and grouped into for matchmaking purposes, the techniques and systems described herein may be configured to use machine learning scoring approaches to identify any type of behavior and predict the likelihood that a player will engage in that behavior for player matchmaking purposes.

[0047] The processes described herein are illustrated as a collection of blocks in a logic flow graph, which represent a sequence of operations that may be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer-executable instructions, which, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order and / or in parallel to implement a process.

[0048] 3 is a flow diagram of an example process 300 for training a machine learning model to predict the probability that a player will act or not act in accordance with a particular action. For discussion purposes, the process 300 is described with reference to the previous figure.

[0049] At 302, a computing system 106 may provide a user 102 with access to a video game service. For example, the computing system 106 may enable a user to access and browse a catalog of video games 110, modify a user profile, perform transactions, engage in social media activity, and perform other similar and / or related actions. The computing system 106 may distribute the video games 110 (and their content) to client machines 104 as part of the video game service. In an exemplary embodiment, a user 102 accessing the video game service can load an installed client application, log in with a registered user account, select a desired video game 110, and run the video game 110 on his or her client machine 104 through the client application.

[0050] At 304, the computing system 106 may collect and store data 114 associated with a user account 120 registered with the video game service. This data 114 may be collected at block 304 whenever a user 102 accesses the video game service with a registered user account 120 and uses the video game platform, such as when playing a video game 110 on the platform. It can be appreciated that over time, a vast collection of data 114 tied to a registered user account may be available to the computing system 106.

[0051] At 306, the computing system 106, via the training component 210, may access (historical) data 114 associated with the sampled set 120 of user accounts registered with the video game service. At least a portion of the (historical) data 114 may have been generated as a result of players playing one or more video games 110 on a video game platform provided by the video game service. For example, the (historical) data 114 accessed at 306 may represent match history data of players who played one or more video games 110 (e.g., by participating in a match in a multiplayer mode). In some embodiments, the (historical) data 114 may indicate whether the sampled set 120 of user accounts was transferred between players or other types of user activity related to the user accounts 120.

[0052] At 308, the computing system 106, via the training component 210, may label each user account 120 of the sampled set of user accounts 120 with a label indicating whether the user account is associated with a player who has behaved in accordance with a particular behavior during play of at least one video game 110 in the past. Examples of labels are described herein, such as whether the user account 120 has been banned in the past for cheating, which may, in some cases, be used as a label in the context of scoring players for their propensity to cheat or not. However, the labels may correspond to other types of behaviors, such as a label indicating whether the user account is associated with a player who has abandoned a game in the past, who has griefed during a game in the past, who has used vulgar language during a game in the past, etc.

[0053] At 310, the computing system 106, via the training component 210, may train the machine learning model using the (historical) data 114 as training data to obtain a trained machine learning model 216. As indicated by sub-block 312, training the machine learning model at block 310 may include setting weights for machine learning. These weights may be applied to a set of features derived from the historical data 114. Exemplary features are described herein, such as those described above with reference to FIG. 2. In some embodiments, the weights set at block 312 may be applied to parameters internal to the machine learning model (e.g., weights for neurons in a hidden layer of a neural network). These internal parameters of the machine learning model may or may not map one-to-one with individual input features of the set of features. As indicated by the arrow from block 310 to block 304, the machine learning model 216 may be retrained using the updated (historical) data 114 to obtain a newly trained machine learning model 216 adapted to recent player behavior. This allows the machine learning model 216 to adapt to changing player behavior over time.

[0054] 4 is a flow diagram of an example process 400 for utilizing a trained machine learning model 216 to determine a confidence score 118 for a user account 120, which confidence score 118 is related to (or indicative of) the probability that a player will or will not act in accordance with a particular behavior. For discussion purposes, the process 400 is described with reference to the previous figure. Additionally, as indicated by external page reference "A" in FIGS. 3 and 4, the process 400 may proceed from block 310 of process 300.

[0055] At 402, the computing system 106, via the scoring component 212, may access data 114 associated with multiple user accounts 120 registered with the video game service. This data 114 may include any information (e.g., quantifiable information) in a set of features, as described herein, that are used to train a machine learning model 216. This data 114 constitutes unknown inputs that will be input to the trained machine learning model 216.

[0056] At 404, the computing system 106 may provide the data 114 accessed in block 402 as input to the trained machine learning model 216 via the scoring component 212.

[0057] At 406, the computing system 106, via the scoring component 212, may generate a confidence score 118 associated with the plurality of user accounts 120 as an output from the trained machine learning model 216. Individually, a score 118 is associated with each individual user account 120 of the plurality of user accounts 120, and the score 118 relates to the probability that a player 102 associated with the individual user account 120 will or will not act in accordance with a particular behavior while playing one or more video games 110 in a multiplayer mode. In this context, the particular behavior may be any suitable behavior that is represented in the data 114 such that the machine learning model may be trained to predict players who are prone to engage in that behavior. Examples include, but are not limited to, cheating behavior, game abandoning behavior, griefing behavior, or vulgar language behavior. In some embodiments, the score 118 is a variable that is normalized in the range of [0,1]. The confidence score 118 may have a monotonic relationship to the probability that a player will or will not act in accordance with a particular behavior while playing the video game 110. The relationship between the score 118 and the actual probability associated with a particular action may or may not be a linear relationship, although it is monotonic.

[0058] Thus, process 400 represents a machine learning scoring approach, where a score 118 (e.g., confidence score 118) is determined for a user account 120, where the score indicates the probability that a player using the user account 120 will engage in a particular behavior in the future. The use of machine learning models in this scoring process can identify complex relationships in player behavior to more accurately predict player behavior compared to existing approaches that attempt to predict player behavior. This leads to more accurate predictions of player behavior with a more adaptive and versatile system that can adapt to fluctuations in player behavior without human intervention.

[0059] 5 is a flow diagram of an example process 500 for assigning user accounts 120 to distinct matches of a multiplayer video game based on machine-learned confidence scores 118 associated with likely player actions. For discussion purposes, the process 500 is described with reference to the previous figure. Additionally, as indicated by external page reference "B" in FIGS. 4 and 5, the process 500 may proceed from block 406 of process 400.

[0060] At 502, the computing system 106 may receive information 122 from multiple client machines 104, where the information 122 indicates logged-in user accounts 120 logged into client applications running the video game 110 on each client machine 104. For example, multiple players 102 may wish to play the video game 110 in multiplayer mode and may have begun execution of a particular video game 110 (e.g., a first-person shooter game). The information 112 received at block 502 may indicate at least the logged-in user accounts 120 for these players.

[0061] At 504, the computing system 106, via the matchmaking component 214, may define a number of matches in which players 102 will be grouped to play the video game 110 in a multiplayer mode. Any number of matches may be defined depending on a variety of factors, including demand, capacity, and other factors affecting the matchmaking process. In one embodiment, the number of matches defined at block 504 may include at least a first match 218(1) and a second match 218(2).

[0062] At 506, the computing system 106, via the matchmaking component 214, may assign a first subset of the logged-in user accounts 120 to a first match 218(1) and a second subset of the logged-in user accounts 120 to a second match 218(2) based at least in part on the scores 118 determined for the logged-in user accounts 120. As previously mentioned, at block 506, any number of matches may be defined such that further splits and additional matches may be assigned to the user accounts.

[0063] As indicated by subblock 508, the allocation of user accounts 120 to separate matches may be based on a threshold score. For example, matchmaking component 214 may determine that scores 118 associated with a first subset of logged-in user accounts 120 are below a threshold score and may further assign the first subset of logged-in user accounts 120 to a first match based on these scores being below the threshold score. Similarly, matchmaking component 214 may determine that scores 118 associated with a second subset of logged-in user accounts 120 are equal to or greater than a threshold score and may assign the second subset of logged-in user accounts 120 to a second match based on these scores being equal to or greater than the threshold score. However, this is only one exemplary manner for providing match allocation 124 and other techniques are contemplated. For example, clustering algorithms, or other statistical approaches may be used in addition to or as an alternative to using a threshold score. In one embodiment, confidence scores 118 may be used to prioritize matching user accounts (players) having “similar” confidence scores. Given the inevitable tendency for the distribution of confidence scores 118 to be generally bimodal across multiple user accounts 120, grouping confidence scores 118 together based on a similarity metric (e.g., distance metric, dispersion metric, etc.) may yield similar results as using a threshold score. However, using a similarity metric to match user accounts together for matchmaking purposes may be useful in cases where the matchmaking pool is small (e.g., players in a small geographic area who want to play less popular game modes), as it may provide a more fine-grained approach that allows for gradual adjustment of the relative importance of confidence scores to other matchmaking factors such as skill level. In some embodiments, multiple thresholds may be used to "bucket" user accounts 120 into multiple separate matches.

[0064] As indicated by sub-block 510, other factors besides the confidence score 118 may be considered in the matchmaking assignment in block 506. For example, the match assignment 124 determined in block 506 may be further based on the skill level of a player associated with the logged-in user account, the amount of time the logged-in user account has been waiting to be placed in one of a plurality of matches, a geographic region associated with the logged-in user account, and / or other factors.

[0065] At 512, the computing system 106 may (e.g., by providing control instructions) cause a client application running the video game 110 on each client machine 104 that sent the information 122 to start one of the defined matches (e.g., one of the first match 218(1) or the second match 218(2)) based at least in part on the logged-in user accounts 120 associated with the client machine 104. For example, once the first match 218(1) and the second match 218(2) have been defined, the computing system 106 may cause a first subset of the plurality of client machines 104 associated with a first subset of the logged-in user accounts 120 to run the video game 110 in the first match 218(1), and may further cause a second subset of the plurality of client machines 104 associated with a second subset of the logged-in user accounts 120 to run the video game 110 in the second match 218(2).

[0066] The machine-learned trust scores 118 are used as a factor in the matchmaking process, which may improve the gaming experience for users who wish to play the video game in multiplayer mode the way it was meant to be played, since the techniques and systems described herein may be used to match up players who are more likely to engage in malicious behavior (e.g., cheating) and isolate these players from other trustworthy players who are more likely to play the video game legitimately.

[0067] 6 is a diagram illustrating an example environment 600 including a block diagram illustrating example components of a handheld device 602, which shows how sensor data 604 received by a remote computing system 106 can be used for machine learning confidence scoring. The handheld device 602 of FIG. 6 can be used by a user 102 to interact with a video game platform as described herein. For example, the user 102 may pair the handheld device 602 with a client machine 104 running a video game client (e.g., game software for playing a video game 110) thereon. Once the handheld device 602 is paired with the client machine 104 and is able to communicate with it (e.g., by sending / receiving data to / from the client machine 104), the handheld device 602 can be used to interact with the video game platform described herein, such as by registering the handheld device 602 to a user account 120, accessing video games 110 available from the remote computing system 106, and playing video games 110 using the handheld device 602.

[0068] The handheld device 602 may represent a handheld game controller, such as a game controller designed to be held by one or both hands of the user 102 and configured with one or more finger-operated controls operated by the fingers and / or thumbs of the user's 102 hands. In some embodiments, the handheld device 602 may be further configured for operation by movement (e.g., panning, rotating, tilting, etc.) of the game controller in three-dimensional (3D) space. It should be understood that the handheld device 602 may represent any other suitable type of handheld device, such as a mobile phone (e.g., a smartphone), a tablet computer, a portable digital assistant (PDA), a wearable computer (e.g., a smart watch, a head-mounted display (HMD)), a portable game player, and / or any similar handheld electronic device. The term "handheld," as used herein to describe a handheld device 602, means a device that is configured to be held by a user 102, whether the device is held by the hand of the user 102 or by another part of the user's 102 body (e.g., a wearable device worn on the wrist, arm, leg, waist, head, etc., is considered a "handheld" device as the term is used herein).

[0069] As shown in FIG. 6, the handheld device 602 includes one or more input / output (I / O) devices 608 such as finger-operated controls (e.g., joystick, trackpad, trigger, depressible buttons, etc.), potentially other types of input or output devices, e.g., a touch screen, a microphone for receiving audio input such as a user's voice input, a camera or other type of sensor (e.g., sensor 610) that can function as an input device for receiving gestural input such as the movement of the handheld device 602 and / or the hand of the user 102. In some embodiments, additional input devices can be provided in the form of a keyboard, keypad, mouse, touch screen, joystick, adjustment buttons, etc. The input devices may further include control mechanisms such as basic volume buttons for increasing and decreasing the volume in addition to power and reset buttons. The input devices can facilitate input of biometric data of the user 102, such as acquiring a fingerprint or handprint, scanning the user's eyes and / or face, capturing the user's voice, etc. for biometric recognition / authentication of the user 102.

[0070] Output devices, on the other hand, may include displays, light emitting elements (e.g., LEDs), vibrators that create a tactile sensation, speakers (e.g., headphones), etc. There may also be simple light emitting elements (e.g., LEDs) to indicate a state, such as when the power is on. Although several examples have been provided, the handheld device 602 may additionally or alternatively include any other type of output device. In some cases, output by one or more output devices may be based on input received by one or more of the input devices. For example, activation of a control may result in output of a tactile response by a vibrator placed adjacent to (e.g., directly below) the control or anywhere else.

[0071] Additionally, the handheld device 602 may include one or more communication interfaces 612 to facilitate wireless connection to a network and / or one or more remote systems (e.g., a client machine 104 running an application, a game console, a wireless access point, etc.). The communication interface 612 may implement one or more of a variety of wireless technologies, such as Wi-Fi, Bluetooth, radio frequency (RF), etc. It should be appreciated that the handheld device 602 may further include physical ports to facilitate a wired connection to a network, connected peripheral devices, or plug-in network devices that communicate with other wireless networks.

[0072] In the illustrated implementation, the handheld device 602 further includes one or more processors 614 and computer-readable media 616. In some implementations, the processor 614 may include a central processing unit (CPU), a graphics processing unit (GPU), both a CPU and a GPU, a microprocessor, a digital signal processor, or other processing units or components known in the art. Alternatively or additionally, the functionality described herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and the like. In addition, each of the processors 614 may have its own local memory that may also store program modules, program data, and / or one or more operating systems.

[0073] The computer readable medium 616 may include removable and non-removable media implemented in any method or technology for storing information, such as volatile and non-volatile memory, computer readable instructions, data structures, program modules, or other data. Such memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, RAID storage systems, or any other medium that may be used to store the desired information and that may be accessed by a computing device. The computer readable medium 616 may be implemented as a computer readable storage medium ("CRSM"), which may be any available physical medium accessible by the processor 614 to execute instructions stored on the computer readable medium 616. In one basic implementation, the CRSM may include random access memory ("RAM") and flash memory. In other implementations, the CRSM may include, but is not limited to, read-only memory ("ROM"), electrically erasable programmable read-only memory ("EEPROM"), or any other tangible medium that can be used to store the desired information and that can be accessed by the processor 614.

[0074] A number of modules of instructions, data stores, etc., can be stored in the computer readable medium 616 and configured to execute on the processor 614. Some example functional modules are illustrated as being stored in the computer readable medium 616 and executed on the processor 614, although the same functionality can alternatively be implemented in hardware, firmware, or as a system on a chip (SOC).

[0075] The operating system module 618 may be configured to manage hardware within the handheld device 602 and coupled to the handheld device 602 for the benefit of other modules. Additionally, the computer readable medium 616 may store a network communication module 620 that enables the handheld device 602 to communicate with one or more other devices, such as a client machine 104 (e.g., PC) running an application (e.g., game application), a game console, a remote computing system 106, or the like, via the communication interface 612. The computer readable medium 616 may further include a game session database 622 for storing data associated with games (or other applications) executing on the handheld device 602 or on a client machine 104 connected to the handheld device 602. The computer readable medium 616 may also include a device record database 624 that stores data associated with devices to which the handheld device 602 couples, such as a client machine 104 (e.g., PC, game console, etc.), a remote computing system 106, and the like. The computer-readable medium 616 may further store game control instructions 626 for configuring the handheld device 602 to function as a game controller, and general purpose control instructions 628 for configuring the handheld device 602 to function as a controller for other non-gaming devices.

[0076] The handheld device 602 is also shown to include one or more sensors 610. For example, the sensors 610 may include motion sensors such as an inertial measurement unit (IMU), which may include one or more gyroscopes, and / or accelerometers, and / or magnetometers, and / or compasses, or any other suitable motion sensors. In some embodiments, the sensors 610 may be implemented as independent gyroscopes, accelerometers, magnetometers, compasses, etc., but not necessarily as an IMU. In some embodiments, one or more of these sensors may be utilized to provide six-component motion sensing. For example, an IMU may be configured to sense and generate sensor data 604 indicative of translational and / or rotational motion about 3D space. The sensor data 604 generated by such sensors 610 may relate to the degree, velocity, and / or acceleration of translational motion (X, Y, and Z motion) in 3D space, as well as the degree, velocity, and / or acceleration of rotational motion (roll, pitch, and yaw) in 3D space. The measurements may be generated in terms of a 3D coordinate system, such as a Cartesian (X, Y, and Z) coordinate system or a spherical coordinate system. The sensor data 604 may include measurements in terms of, for example, displacement (e.g., displacement from a previous time log), velocity, and / or acceleration of translational motion (represented by variables: d, v, a) and angular motion (represented by variables: θ, ω, α). The sensor data 604 may further include the time at which the sensor data 604 is generated and / or transmitted, for example, at any suitable time interval, such that a history of the sensor data 604 can be collected and stored temporarily or permanently on the handheld device 602.

[0077] As another example, the sensor 610 can include a touch sensor configured to detect the proximity of an object, such as a finger, palm, or the like, to the touch sensor, which may be based on any suitable touch sensing technology, such as a capacitive touch sensor, a resistive touch sensor, an infrared touch sensor, a touch sensor utilizing acoustic sound waves to detect the proximity of a finger 102, or any other type of touch sensor. For example, a touch sensor may be disposed under or on the surface of the device and / or in or on a finger-operated control to detect the proximity of a finger to the surface or finger-operated control. In response to detecting the proximity (e.g., a finger touching or hovering over a surface), the touch sensor may generate sensor data 604 indicative of the proximity of the finger. Touch sensors and the like may be embedded in the handle of the handheld device 602 to detect the grip of the user 102, and / or in various controls, including trackpads, joysticks, buttons, and the like. In implementations utilizing capacitance-based sensing, a touch sensor may include electrodes (e.g., transmitter and receiver electrodes of a capacitive sensor) and a voltage may be applied to the electrodes such that the electrodes are configured to measure a capacitance change at the electrodes, which may be converted into sensor data 604 in the form of a capacitance value indicative of the proximity of an object to the sensor 610. For example, the capacitance change at the electrodes of a capacitance-based touch sensor may be affected by an object (such as a finger) being in proximity to the electrode. The raw capacitance may be digitized into a proximity value to generate the sensor data 604.

[0078] As another example, the sensor 610 may include a pressure sensor such as a force sensing resistor (FSR). For example, the FSR may include a conductive material spaced apart from a resistive film (e.g., a semiconductor material such as an ink composition) and an actuator configured to transmit a force to the resistive film such that the resistive material contacts the conductive material under the application of a compressive force applied to the actuator. The FSR may exhibit a variety of resistances in response to the variable force and generate sensor data 604 corresponding to the resistance values. The FSR may be a "ShuntMode" FSR or a "ThruMode" FSR. In a shunt mode FSR, the conductive material spaced apart from the resistive film may be a plurality of interdigitated metal fingers. When a force is applied to the actuator of the FSR, the resistive film contacts some of the interdigitated metal fingers that shunt the metal fingers, thereby changing the resistance across the output terminals of the FSR, which may be digitized into an FSR value to generate the sensor data 604. In some embodiments, the pressure sensor may additionally or alternatively include other types of pressure sensing mechanisms, such as piezoelectric sensors, strain gauges, and the like.

[0079] Other examples of sensors 610 configured to generate corresponding sensor data 604 may include, but are not limited to, temperature sensors, humidity sensors, cameras, and the like. Regardless of the type of sensor 610 of the handheld device 602, the sensor 610 is configured to generate sensor data 604 related in some manner to the physical state of the handheld device 602. For example, a touch sensor may generate sensor data 604 indicative of whether an object (e.g., a finger) is in contact with or near a portion of the device 602 that includes the touch sensor (e.g., a directional pad (D-pad), joystick, trigger button, bumper button, selector button, etc.). As another example, a pressure sensor may generate sensor data 604 indicative of whether an object is being pressed against a portion of the device 602 and whether that portion of the device 602 is being pressed lightly or heavily by the object. As yet another example, motion sensors (e.g., gyroscopes and / or accelerometers) may generate sensor data 604 that indicates whether the handheld device 602 has changed in orientation and / or spatial position in 3D space and / or whether the handheld device 602 is moving rapidly or slowly. Thus, the sensors 610 of the handheld device 602 are configured to generate sensor data 604 that are indicative of these and other types of physical states of the device 602.

[0080] 6 illustrates that the handheld device 602 may further include encryption hardware 630. For example, a service provider of a video game service and / or a third party manufacturer may produce a "trusted" handheld device 602 that is manufactured with a tamper-resistant hardware component embedded therein at the time of manufacture. This embedded hardware component may include the encryption hardware 630 shown in FIG. 6, which may then store a private key 632. For example, the private key 632 may be held in a non-volatile memory of the handheld device 602.

[0081] In some embodiments, the encryption hardware 630 is tamper-resistant. For example, the encryption hardware 630 may be contained within or covered by a tamper-resistant casing, and the encryption hardware 630 may be configured to disable itself if the tamper-resistant casing is damaged. For example, the encryption hardware 630 may be configured to erase or delete the private key 632 and / or send an alert to the remote computing system 106 if the tamper-resistant casing is damaged or otherwise tampered with. This makes it difficult for a user to gain unauthorized access to the private key 632, or at least undetected access, thereby preventing sharing the private key 632 with other users or using the private key 632 to act as a "smurf" through another user account 120. In online gaming, "smurfing" is the act of a player of a particular skill level playing under another person's or secondary account to appear to be of a lower rank or skill level.

[0082] In some embodiments, the user 102 may initially associate or register the handheld device 602 with his / her user account 120 (and possibly additional user accounts 120, such as family member accounts). This may be accomplished by performing an initial registration procedure for the handheld device 602. The initial registration procedure may include an initial activation procedure using, for example, a two-factor authentication (2FA) step for added security. For example, the user 102 may pair the handheld device 602 with a client machine 104 in the vicinity of the handheld device 602, for example, using a short-range wireless protocol such as Bluetooth. The user 102 may log into his / her user account 120 using a client application (e.g., a video game client) running on the client machine 104 or on the user's 102 mobile device (e.g., a mobile phone). In response to the user 102 logging in, the remote computing system 602 may receive an indication of the paired but unregistered / unactivated device 602 and may send an activation code to the user 102, such as by sending a text to a mobile number or an email to an email address specified in the user account 120. The user 102 may access the message to obtain the activation code and may enter the activation code via the user's phone or a video game client running on the client machine 104 (e.g., by entering the activation code using the handheld device 602). If the remote system 106 receives the correct activation code within a predetermined time limit, the handheld device 602 may be associated with the user account 120 of the user 102 by indicating in the user account 120 that the handheld device 602 is a registered device for the user account 120, the time and date of registration, the location where the handheld device 602 was activated, and the like.The remote computing system 106 may also associate the user account 120 with the handheld device's 602 private key 632 and / or the user's 102 biometric data (e.g., fingerprint, eye print, face print, voice print, etc.) acquired by an input device of the handheld device 602 as a means of authenticating the user 102 in the future when the user 102 logs into his or her user account 120 and uses a handheld device 602 registered to that user account 120.

[0083] In some embodiments, additional security layers may be implemented to make switching of the handheld device 602 from one user account 120 to another user account 120 safer, which may help prevent unauthorized use of a trusted handheld device 602, for example, if the device 602 is stolen or lost. For example, the remote computing system 106 may enforce a waiting period (e.g., a waiting period of several days) as the device 602 can associate / register with the first user account 120 (e.g., by disassociating / deregistering the device 602 from the first user account 120 and associating / registering the device 602 with the second user account 120) before the device 602 is allowed to be switched to the second user account 120. The remote computing system 106 may additionally or alternatively limit the number of times a user 102 is permitted to switch the handheld device 602 between user accounts 120 within a given period of time (e.g., a user 102 may be permitted to switch the handheld device 602 from one user account 120 to another user account 120 no more than twice over the span of an hour). These additional layers may help to curb unauthorized use of trusted handheld devices 602.

[0084] Once the handheld device 602 is registered with the user account 120, the user 102 may use the device 602 to interact with the video game platform, such as by connecting the handheld device 602 to the client machine 104 (e.g., pairing the device 602 with the client machine 104 via a wireless communication link) (which may occur automatically after a first pairing procedure using the device record database 624) and playing the video game 110 using the device 602 as a handheld game controller. In this scenario, the client machine 104 may be used to display images of the video game 110 while the user 102 plays the video game 110 by manipulating the handheld device 602. At any appropriate point during interaction with the video game platform, the handheld device 602 may obtain biometric data (e.g., a fingerprint, an eye print, a face print, a voice print, etc.) of the user 102 via an input device of the handheld device 602 and may transmit the obtained biometric data to the remote computing system 106 via the communication interface 612 (e.g., by routing the biometric data to the remote computing system 106 via the client machine 104), where the biometric data may be used by the remote computing system 106 to authenticate the user account 120, to authenticate the user 102, and / or to authenticate the handheld device 602. Additionally, at any appropriate point during interaction with the video game platform, the private key 632 may be used by logic of the handheld device 602 to encrypt device output, such as game control data 606 and / or sensor data 604, transmitted by the handheld device 602 during use of the device 602.For example, the private key 632 may be used to sign the game control data 606 and / or the sensor data 604 to convert the data into ciphertext form using any suitable encryption algorithm, and the device 602 may transmit the encrypted data to the remote computing system 106 via the communication interface 612 (e.g., by routing the encrypted data to the remote computing system 106 via the client machine 104). The remote computing system 106 may maintain a repository of private keys in the data store 116 (or database 116) for trusted handheld devices, which may include a copy of the private key 632 used by the handheld device 602 to encrypt data it transmits to the remote computing system 106. The remote computing system 106 may have recorded the private key in the data store 116 at the time of manufacture of the trusted handheld device or thereafter. With the private key accessible to the remote computing system 106, the system 106 can decrypt encrypted data it receives from the handheld device 602 to verify that the received data was encrypted with the expected private key 632. Thus, the remote system 106 has knowledge of the private key 632 used by the handheld device 602 to encrypt the device output and can use its copy of the private key 632 to authenticate the user account 120, authenticate the user 102, and / or authenticate the handheld device 602, among other possible uses of the private key 632. In some embodiments, the remote computing system 106, in response to receiving data associated with a logged-in user account 120, may require the user 102 to enter a code (e.g., a personal identification number (PIN), passcode, passphrase, etc.) that authenticates the user 102 on each login of the user account 120.Such a request may be sent via the user's 102 2FA device (e.g., via a mobile phone, via the client machine 104, etc.), and a response to the request may be received by the remote system 106 from the same 2FA device of the user 102. In some embodiments, no additional user action is required to use an already registered handheld device 602 associated with a corresponding logged-in user account 120. That is, all that may be required to use a handheld device 602 associated with a user account 120 to which the device 602 is registered is for the device 602 to encrypt data using its private key 632 and send the encrypted data to the remote computing system 106, and for the remote computing system 106 to decrypt the data to verify that the received data was encrypted with the expected private key 632. For example, the remote system 106 may send a chunk of data to the handheld device 602, which may encrypt the chunk of data and return the encrypted data to the remote system 106, which can then verify that the encryption was performed correctly without having to send all of the keys 632 itself.

[0085] For example, a data store 116 (or database 116) accessible to the remote computing system 106 may maintain biometric data and / or multiple private keys associated with multiple trusted handheld devices, such as handheld device 602, and the remote computing system 106 can compare the biometric data and / or private keys, such as private key 632, used to encrypt data received from the client machine 104 and / or the handheld device to determine whether the received biometric data and / or private key matches any of the biometric data and / or private keys maintained in the database 116. These operations may be performed for authentication purposes and / or computing a confidence score, as described herein. In other words, receipt of the biometric data matching the biometric data maintained in the database 116 and / or encrypted data encrypted with the private key 632 matching the private key may be used to authenticate the user 102, the device 602, and / or the user account 120 associated therewith, and / or receipt of the matching biometric data and / or encrypted data encrypted with the matching private key 632 may be used to generate a trust score for the associated user account 120, which may be used for player matchmaking. In this sense, a user account 120 associated with a handheld device 602 that provides the remote computing system 106 with the matching biometric data and / or encrypted data encrypted with the matching private key 632 may be seamlessly authenticated (e.g., in some cases, without additional user action) and is more likely to match with user accounts 120 of other human players, as opposed to being matched with "untrusted" user accounts 120 that are more likely to be associated with non-human players (e.g., software designed for cheating).

[0086] Thus, as users 102 utilize handheld devices 602 to interact with a video game platform, for example to play one or more video games 110 on respective client machines 104, sensor data 604 and game control data 606 may be transmitted to client machines 104 and transferred from client machines 104 to remote computing system 106. Game control data 606 may be used to control aspects of video game 110, and thereby processed by video game 110 to determine how to render the next frame of video game 110. Sensor data 604 represents raw, unfiltered sensor data 604, such as raw data generated by a gyroscope, raw data generated by an accelerometer, and / or raw data generated by a touch sensor (e.g., a capacitive touchpad). In a streaming implementation, video game 110 may be executed on remote computing system 106, which may capture video game 110 data and transmit the video game 110 data over network 108 to client machine 104 of user 102. This may involve the remote computing system 106 capturing the state of the video game 110, encoding the video and audio data into bits, and transmitting the encoded bits over the computer network 108 to the client machine 104, where an application (e.g., a video game client) running on the client machine 104 may decode the bits to output an image via a display and an audio via a speaker of the client machine 104 for a given frame. The user 102 may react to the video and audio he or she sees by manipulating the handheld device 602. For example, the user 102 may actuate the controls of the handheld device (e.g., pressing down on a directional pad (D-pad), deflecting a joystick, swiping a finger on a trackpad) and / or tilt or move the handheld device 602 in 3D space to control aspects of the video game 110.In response to operation of the handheld device 602, the handheld device 602 may generate game control data 606 that is transmitted to the remote computing system 106 either directly via a wireless access point, via the computer network 108, or via the client machine 104 with which the handheld device 602 is associated. In either case, the game control data 606 may be transmitted in real time to the remote computing system 106 to control aspects of the video game 110. For example, the game control data 606 may be processed by the video game 110 to control the movement of a virtual object (e.g., a player-controlled character) of the video game 110 by moving the virtual object within a virtual world represented by a scene on the display of the client machine 104.

[0087] The game control data 606 may include at least some sensor data 604 generated by the sensors 610 of the device 602, such as filtered / attenuated sensor data 604 and / or amplified sensor data 604, however, as depicted in FIG. 6, the sensor data 604 transmitted by the device 602 to the remote computing system 106 represents raw, unfiltered sensor data 604 that is not used to control aspects of the video game 110 but is used to generate a machine-learned confidence score for the associated user account 120.

[0088] In step 1 of FIG. 6, the computing system 106 may train the machine learning model 216 using at least the historical sensor data 604 sampled from the data store 116. For example, the computing system 106 may access a portion of the historical sensor data 604 associated with the sampled set of user accounts 120 registered with the video game service and may use the sampled sensor data 604 to train the machine learning model 216. As described herein, the historical sensor data 604 may have been received from the client machines 104 along with the historical game control data 606 as a result of the users 102 interacting with the video game platform using their handheld devices 602. In some embodiments, the portion of the sensor data 604 used as training data is represented by a set of features, and each user account 120 of the sampled set is labeled with a label indicating whether the historical game control data 606 associated with the corresponding user account 120 was generated by the handheld device 602 while a human player was holding and operating the device 602, as opposed to being synthesized and / or modified using software. In other words, user accounts 120 determined to be associated with game control data 606 generated by a human user 102 operating a handheld device 602 may be labeled as such, while other user accounts 120 determined to be associated with synthetic and / or modified game control data (i.e., data synthesized and / or modified by software to control aspects of a video game) may be labeled as such. In this manner, a supervised learning approach may be employed to train the machine learning model 216 to predict players who are likely to be legitimate human players operating the handheld device 602, as opposed to non-human software designed to cheat.

[0089] For example, when a human operates the handheld device 602, there may be subtle movements of the handheld device 602 that are unintentional movements, which can be detected by one or more physical sensors 610 of the handheld device 602 and may be shown as a result in the sensor data 604. Based on the idea that such sensor data 604 indicates these nuances, the machine learning model 216 may be trained with the sensor data 604 as training data to distinguish between human players and non-human players by recognizing these nuances in the sensor data 604, e.g., the result of the handheld device 602 being mechanically operated (e.g., by the human user 102), the movements generated by the physical sensors 610 of the actual physical handheld device 602, the proximity of fingers, etc. Conversely, if sensor data 604 is received but does not exhibit the expected nuances resulting from human operation of handheld device 602, it may be inferred that the corresponding game control data 606 was not generated as a result of actual, tangible operation of handheld device 602, but instead was synthesized and / or modified using software. For example, a cheating player may utilize third party software, or develop their own software, configured to automate the movement of the cheater's avatar (e.g., by programmatically moving a mouse cursor to a target and automatically firing a weapon). This so-called "aimbot" software effectively plays video game 110 in place of a human player, such that the human player is not actually playing video game 110, and the software synthesizes and / or modifies game control data 606 that is processed by video game 110 in the same manner that the actual game control data 606 is processed, thereby giving the cheating player an advantage over legitimate, non-cheating human players.If such synthetic and / or modified game control data 606 were undetected, computerized algorithms could become ubiquitous and widely used to improve the performance of cheating players in multiplayer games, which could ruin the experience for human players who wish to play the video game 110 legitimately and who may be matched up against these computerized algorithms. In these instances, attempts to mimic the sensor data 604 generated by human operation of the handheld device 602 can be detected by machine learning models 216 trained to distinguish between human players and non-human players, such as aimbots, and these and other types of cheating behaviors can be remedied, such as by isolating the non-human players in their own matches of the multiplayer video game.

[0090] It should be appreciated that the machine learning model 216 may be trained using additional data in addition to or in lieu of the historical sensor data 604 described above. For example, historical game control data 606 may be collected from the client machine 104 over time, the game control data 606 being generated based on user inputs provided by human users to handheld devices to control aspects of the video game 110. These user inputs may include actuation of controls on the handheld device 602, such as button presses, joystick deflections, finger swipes on a trackpad, and / or tilt or movement of the handheld device 602 in 3D space. Thus, the historical game control data 606 (e.g., data indicative of user inputs provided to the handheld device 602, such as to control aspects of the video game 110) may be used as training data along with the historical sensor data 604 or in lieu of the use of that historical sensor data. Using both the historical sensor data 604 and the historical game control data 606 as training data allows correlations to be made between the game control data 606 and the contemporaneously generated sensor data 604. As yet another example, wireless communication data may be collected from the client machine 104 over time, and the historical wireless communication data may be used as training data along with or in lieu of using the historical sensor data 604. This wireless communication data may include, but is not limited to, wireless (e.g., WiFi, Bluetooth, etc.) signal strength values, data indicative of instances of dropped data packets transmitted from the handheld device 602 to the associated client machine 104, the number of dropped data packets transmitted from the handheld device 602 to the associated client machine 104, and the like. This wireless communication data may be obtained by the handheld device 602, by the client machine 104, and / or by any other suitable device local to the environment of the user 102.

[0091] As mentioned above, the training data may include two components: features and labels. However, in some embodiments, the training data used to train the machine learning model 216 may be unlabeled. Thus, the machine learning model 216 may be trainable using any suitable learning technique, such as supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, etc. The features included in the training data may be represented in a set of features, such as in the form of an n-dimensional feature vector of quantifiable information about the attributes of the training data. Exemplary features included in the training data may include, but are not limited to, sensor data 604 values ​​(e.g., capacitance values, resistance values, displacement values, velocity values, acceleration values, temperature values, humidity values, etc.), game control data 606 values ​​(e.g., values ​​generated by potentiometers and other switches of a finger-operated control), biometric data values, the value of a private key 632 associated with the handheld device 602 that encrypted data transmitted to the remote computing system 106 using the private key 632, the time the private key 632 (and thus the trusted handheld device 602) was associated / registered with a user account 120, the number of times the trusted handheld device 602 (and / or its private key 632) was used with the user account 120, wireless communication data values ​​(e.g., wireless signal strength values, handheld device a value indicative of instances of dropped data packets sent from the handheld device 602 to the associated client machine 104, a value indicative of the number or frequency of dropped data packets sent from the handheld device 602 to the associated client machine 104, the number of times the trusted handheld device 602 (and / or its private key 632) has been switched between different user accounts 120, the general amount of time the player spent playing the video game 110, the amount of time the player spent playing a particular video game 110, the amount of time each day the player logged into and played the video game 110, the player's match history data, such as total score (per match, per round, etc.), headshot percentage, kill count, death count, assist count, player rank, etc.the number and / or frequency of reports of a player cheating, the number and / or frequency of a player being innocent of cheating, the number and / or frequency of a player being guilty of cheating, the confidence value (score) output by a machine learning model that detected a player cheating during a video game, the number of user accounts 120 associated with a single player (which may be inferred from a common address, phone number, payment method, etc. tied to multiple user accounts 120), how long a user account 120 has been registered with the video game service, the number of previously banned user accounts 120 tied to a player, the number and / or frequency of a player's financial transactions on the video game platform, dollar amount per transaction, the player's user account The number of digital items of monetary value associated with the user account 120, the number of times the user account 120 has changed hands (e.g., been transferred between different owners / players), how often the user account 120 is transferred between players, the geographic locations from which the player logged into the video game service, a number of different payment instruments, phone numbers, mailing addresses, etc. associated with the user account 120, and / or how often these items were changed, and / or any other suitable characteristics that may be relevant to calculating a confidence score 118 indicative of the probability that the game control data associated with the user account was generated by a handheld device 602 held and operated by a human player, as opposed to being synthesized and / or modified using software.

[0092] Additionally, as part of the training of step 1 of FIG. 6, the training component 210 may set weights for machine learning. These weights may be applied to the set of features included in the training data as derived from historical data in the data store 116, including historical sensor data 604. In some embodiments, the weights set during the training process may be applied to parameters internal to the machine learning model (e.g., weights for neurons in a hidden layer of a neural network). These internal parameters of the machine learning model may or may not map one-to-one with individual input features of the set of features. The weights may indicate the influence of any given feature or parameter on the score 118 output by the trained machine learning model 216.

[0093] In step 2 of FIG. 6, the computing system 106 may use the trained machine learning model 216 to score one or more registered user accounts 120. For example, the computing system 106 may receive new sensor data 604 and new game control data 606 associated with one or more logged-in registered user accounts 120 from one or more client machines 104. In an exemplary embodiment, the computing system 106 may receive information 122 indicating that a logged-in user account 120 is currently playing or has started a video game 110. The video game 110 may run on the client machine 104 via an installed client application, or the video game 110 may be streamed to the client machine 104 while running on the remote computing system 106. In any event, when a player 102 logs in with his / her user account 120, executes a particular video game 110, and requests to play in multiplayer mode, each client machine 104 may provide information 122 indicative of the equivalent to the computing system 106, and the sensor data 604 and game control data 606 may be streamed in real-time from the client machine 104 to the remote computing system 106 as the video game 110 is started and the player begins gameplay. The computing system 106 may provide the new sensor data 604 it receives as input to the trained machine learning model 216, and as output from the trained machine learning model 216, may generate a score 118 associated with the individual user account 120 currently logged in and associated with the new game control data 606. The score 118, in this example, may relate to the probability that the new game control data 606 was generated by the handheld device 602 associated with the client machine.In this case, a high confidence score indicates a trustworthy user account 120 (e.g., one that is more likely to be associated with a human player operating the handheld device 602), while a low confidence score indicates an untrustworthy user account 120 (e.g., one that is more likely to be associated with a non-human player, such as software, that generates synthetic and / or modified game control data 606 that appears similar to actual game control data 606 generated by the actual handheld device 602). In some embodiments, the score 118 is a variable that is normalized in the range of [0,1]. This confidence score 118 may have a monotonic relationship with the probability that the new game control data 606 was generated by the handheld device 602. The relationship between the score 118 and the actual probability, while monotonic, may or may not be a linear relationship. Of course, the scoring may be implemented in any suitable manner to predict whether the new game control data 606 was generated by the handheld device 602 or was synthetic and / or modified using software.

[0094] In some embodiments, the new game control data 606 itself may be provided as an additional input to the trained machine learning model 216, which may generate the score 118 based at least in part on the game control data 606 provided as an additional input to the trained machine learning model 216. For example, the user 102 may manipulate the handheld device 602 during gameplay by actuating controls of the handheld device 602 (e.g., pressing a button such as a directional pad (D-pad), deflecting a joystick, swiping a finger on a trackpad, etc.) and / or tilting or moving the handheld device 602 in 3D space to control aspects of the video game 110. In response to a user's manipulation of the handheld device 602, the handheld device 602 may generate game control data 606 that is transmitted to the remote computing system 106. This new game control data 606 is indicative of the user input provided to the handheld device 602 and may be provided as an input to the trained machine learning model 216. It should be appreciated that the new game control data 606 may be provided as an input to the trained machine learning model 216 in addition to providing the sensor data 604 as an input to the trained machine learning model 216. An exemplary reason for providing the game control data 606 as an input to the trained machine learning model 216 is that a human providing user input to the handheld device 602 may be "noisier" or less clean than a computer program that mimics the same type of user input. Thus, the trained machine learning model 216 may be trained to distinguish between a human user and a non-human player (e.g., software designed to cheat) based at least in part on the game control data 606 received from the client machine 104 and provided as input to the trained machine learning model 216.

[0095] In some embodiments, new wireless communication data may be provided as additional input to the trained machine learning model 216, which may generate the score 118 based at least in part on the wireless communication data provided as additional input to the trained machine learning model 216. For example, a wireless (e.g., WiFi, Bluetooth, etc.) signal strength may be determined for the handheld device 602 relative to another device with which it communicates, such as the client machine 104, which signal strength may vary (e.g., increase or decrease) depending on wireless signal interference in the environment and / or the distance the handheld device 602 is spaced from another device with which it communicates. The presence of wireless signal strength values, and / or wireless signal strengths that vary over time (e.g., observing changing signal strength values), may indicate that a human player is using the handheld device 602, as these characteristics are commonly encountered in gaming system settings. In contrast, the absence of signal strength values, or constant signal strength values ​​that do not vary over time, may indicate a non-human player (e.g., software designed to cheat). As another example, wireless communication data indicative of instances of dropped data packets transmitted from the handheld device 602 to the associated client machine 104 and / or the number of dropped data packets transmitted from the handheld device 602 to the associated client machine 104 may be determined and used as input to the trained machine learning model 216. For example, the presence of one or more dropped packets and / or the number of dropped data packets that fluctuates over time may indicate that a human player is using the handheld device 602. In contrast, the absence of dropped packets in the wireless communication data and / or the number of dropped data packets that remains constant (e.g., does not fluctuate) over time may indicate a non-human player (e.g., software designed to cheat).

[0096] In some embodiments, determining the score 118 is further based on a determination that the biometric data received from the client machine 104 and / or the private key 632 used to encrypt the data received from the client machine 104 matches one of multiple private keys maintained in the database 116 that are associated with the biometric data and / or multiple trusted handheld devices 602. That is, the biometric data and / or the private key 632 may be used as a proxy for trust to prove to the remote computing system 106 that an actual handheld controller 602 trusted by the system 106 is sending the game control data 606.

[0097] In step 3 of FIG. 6, using the machine-learned scores 118 determined for the logged-in, registered user accounts 120 based on the new sensor data 604 received from the corresponding client machines 104, the computing system 106 can compete players 102 in multiple matches by providing match assignments 124 to the client machines 104 to play the video game 110 in multiplayer mode and to assign a subset of the logged-in user accounts 120 to separate matches of the multiple matches based at least in part on the machine-learned scores 118 determined for the logged-in user accounts 120. In this manner, if the trained machine learning model 216 assigns a low (e.g., below a threshold) score 118 to a first subset of logged-in user accounts 120 and a high (e.g., above a threshold) score 118 to a second subset of logged-in user accounts 120, the first subset of logged-in user accounts 120 may be assigned to a first match of the multiple matches and the second subset of logged-in user accounts 120 may be assigned to a second, separate match of the multiple matches. This may include assigning user accounts 120 to matches at the start of the video game 110 (e.g., before a match begins) and / or reassigning user accounts 120 to new matches midway through the video game 110 (e.g., after a match begins).For example, if a first logged-in user account 120 is assigned to a first match designated for a human player 102, and a machine-learned score 118 is then generated for that user account 120 based on real-time data provided as input to the trained machine learning model 216, and the score 118 indicates that the first user account 120 is likely associated with a non-human player (e.g., software) that is synthesizing and / or modifying the game control data 606, the computing system 106 may reassign the first user account to a second match designated for a non-human player 102 (e.g., a cheating player using software to synthesize and / or modify the game control data). In some embodiments, when a client machine 104 transmits game control data 606 and does not transmit raw, unfiltered sensor data 604 along with the game control data 606, the user account 120 associated therewith may be deemed untrustworthy based on the absence of the raw sensor data 604. In other embodiments, the game control data 606 may be provided as an input to the trained machine learning model 216 to generate a score 118 for a user account, even in the absence of raw sensor data 604, or a machine learning score 118 may be generated for such user account 120 in any other suitable manner described herein to determine the cheating propensity of a player associated with the user account 120.

[0098] In step 4 of FIG. 6, the computing system 106 may cause the execution of the video game 110 in the match assigned for each user account 120. In a streaming implementation, this may involve the remote computing system 106 capturing the state of the video game 110 running on the remote system 106, encoding the video and audio data into bits, and transmitting the encoded bits over the computer network 108 to the client machine 104, where an application (e.g., a video game client) running on the client machine 104 may decode the bits to output an image via a display and an audio via a speaker of the client machine 104 for a given frame. It should be understood that the computer network 108 of FIG. 1 may represent a local area network (LAN) in some embodiments, which may be the case in an "in-home" video game streaming scenario. In this scenario, the computing system 106 may represent a host computing system located at or near the same geographic location as the client machine 104. In either a wide area network (WAN) or LAN implementation, the client machine 104 may be implemented as a thin client configured to send and receive data to and from the computing system 106 with minimal data processing on the client machine 104 itself (e.g., without having to run the video game 110 on the client machine 104 itself). In a thin client implementation, game control data 606 and sensor data 604 may be received from the computing system 106, and the computing system 106 may transmit frames of image data and audio data to the client machine 104 for presentation on a display associated with the client machine 104.In implementations where a client application on each machine 104 is running the video game 110, in step 4 the information may be sent to each client application running on the client machine 104 to trigger execution of the video game 110 on the client machine 104 in the match assigned to the logged-in user account 120 in question. Grouping players into matches based at least in part on the machine-learned scores 118 may allow the system to group players predicted to behave badly (e.g., by cheating), including non-human players (e.g., software designed to cheat), in the same match, thereby improving the in-game experience for at least a portion of the group of players 102 by isolating the badly behaving players from other players who legitimately wish to play the video game 110.

[0099] 7 is a flow diagram of an example process 700 for training a machine learning model 216 using data including sensor data 604 to predict the probability that game control data 606 received from a client machine 104 was generated by a handheld device 602. For discussion purposes, the process 700 is described with reference to the previous figure.

[0100] At 702, a computing system 106 may provide a user 102 with access to a video game service. For example, the computing system 106 may enable a user to access and browse a catalog of video games 110, modify a user profile, perform transactions, engage in social media activity, and perform other similar actions. The computing system 106 may distribute the video games 110 (and their content) to client machines 104 as part of the video game service. In an exemplary embodiment, a user 102 with access to the video game service can execute the video game 110 on his / her client machine 104 via the client application by loading an installed client application, logging in with a registered user account, selecting a desired video game 110, streaming the video game 110, downloading the video game 110, and executing the game locally on the client machine 104, or any other suitable game distribution / execution platform.

[0101] At 704, the computing system 106 may collect and store data associated with the user account 120 registered with the video game service. This data may include at least the sensor data 604 described herein, but may also include the data 114 and game control data 606 described herein. The data may be collected at block 704 at any appropriate time, such as when the user 102 accesses the video game service with the registered user account 120 and uses the video game platform, e.g., when playing a video game 110 on the video game platform. Data generated locally at the client machine 104 and / or by the handheld device 602 may be collected in real time as the data is generated (e.g., by receiving real-time streaming data from the client machine 104), received at defined intervals, and / or in response to an event (e.g., when the user account 120 finishes a video game, when network bandwidth is sufficient to transmit the data, etc.). It can be appreciated that over time, a vast collection of data tied to the registered user account 120 may be available to the computing system 106.

[0102] At 706, the computing system 106, via the training component 210, may access (historical) data including (historical) sensor data 604 associated with the sampled set of user accounts 120 registered with the video game service. The (historical) sensor data 604 may have been received from the client machine 104 along with (historical) game control data 606 that was processed by the video game 110 when the game control data 606 was received, e.g., to control aspects of the video game 110. Thus, the (historical) sensor data 604 may be correlated with corresponding (historical) game control data 606 that was generated contemporaneously. In one example, the sensor data 604 accessed at block 706 may include raw sensor data 604 generated by one or more physical sensors 610 of the handheld devices 602 as the users 102 operated their handheld devices 602 to play the video game 110 on the video game platform.

[0103] At 708, the computing system 106, via the training component 210, may label each user account 120 in the sampled set of user accounts 120 with a label indicating whether the (historical) game control data 606 associated with the user account 120 was generated by the handheld device 602 while a human player held and operated the device 602 to generate the (historical) game control data 606 associated with the user account 120.

[0104] At 710, the computing system 106, via the training component 210, may train the machine learning model 216 using the (historical) sensor data 604, and possibly additional data as described herein (e.g., (historical) game control data 606, (historical) wireless communication data, etc.) as training data to obtain a trained machine learning model 216. As indicated by sub-block 712, training the machine learning model 216 at block 710 may include setting weights for machine learning. These weights may be applied to a set of features derived from the historical sensor data 604. In some embodiments, the weights set at block 712 may be applied to parameters internal to the machine learning model (e.g., weights for neurons in a hidden layer of a neural network). These internal parameters of the machine learning model 216 may or may not map one-to-one with individual input features of the set of features. As indicated by the arrow from block 710 to block 704, the machine learning model 216 can be retrained using updated (historical) data, including updated (historical) sensor data 604, to obtain a newly trained machine learning model 216 adapted to recent player behavior. This allows the machine learning model 216 to automatically adapt to changing player behavior over time.

[0105] 8 is a flow diagram of an example process 800 for utilizing a trained machine learning model 216 to determine a confidence score for a user account, which confidence score is related to the probability that the game control data 606 was generated by the handheld device 602. For discussion purposes, the process 800 is described with reference to the previous figure. Additionally, as indicated by external page reference "C" in FIGS. 7 and 8, the process 800 may proceed from block 710 of process 700.

[0106] At 802, the computing system 106 may receive game control data 606 and sensor data 604 from the client machine 104. The game control data 606 and sensor data 604 may be associated with a user account 120 that is logged into and registered with the video game service. The game control data 606 may be data that is processed by the video game 110 to control aspects of the video game 110. The sensor data 604 received at block 802 may include, but is not limited to, capacitance values, resistance values, displacement values, velocity values, acceleration values, temperature values, humidity values, and the like. In some embodiments, these values ​​represent raw (e.g., unfiltered, unamplified) values, assuming they are generated by actual physical sensors 610 of the handheld device 602. Additionally, some or all of the data received at block 802 may be encrypted (e.g., the device 602 may encrypt the data using a private key 632 before transmitting the data to the remote system 106).

[0107] At 804, the computing system 106 may also receive biometric data from the client machine 104. For example, an input device (e.g., a touch sensor, camera, microphone, etc.) of the handheld device 602 may acquire biometric data from a user of the handheld device 602, the handheld device 602 may transmit the acquired biometric data to the client machine 104, and the client machine 104 may forward the biometric data to the remote computing system 106 over the computer network 108. This biometric data may also be received as encrypted data (e.g., the device 602 may have used the private key 632 to encrypt the biometric data before transmitting the data to the remote system 106).

[0108] At 806, the computing system 106 may evaluate the private key 632 and / or the biometric data. As indicated by sub-blocks 808, 810, and 812, this evaluation may include a variety of operations. For example, the remote system 106 may decrypt any encrypted data received at blocks 802 and / or 804 to verify that the expected private key 632 was used to encrypt the received data. At sub-block 808, the computing system 106 may determine that the biometric data and / or the private key 632 is associated with the user account 120 associated with the game control data 606 and the sensor data 604 received at block 802. This may be determined by comparing the biometric data to the biometric data associated with the user account 120 and / or by determining that the encrypted data was indeed encrypted using the private key 632 associated with the user account 120.

[0109] In sub-block 810, the computing system 106 may determine that the biometric data matches biometric data maintained in the database 116 and / or that the private key 632 matches one of a plurality of private keys maintained in the database 116. The biometric data and / or the private keys maintained in the database 116 may be associated with a plurality of trusted handheld devices 602 such that if the private key 632 derived from the encrypted data matches one of the private keys in the database 116 and / or if the received biometric data matches any of the biometric data in the database 116, the computing system 106 may use the matching data as a proxy for trust, knowing that it was sent from a trusted handheld device 602.

[0110] In sub-block 812, the computing system 106 may determine data associated with the biometric data and / or the private key 632. For example, the computing system 106 may determine the length of time the biometric data and / or the private key 632 have been associated with the user account 120. The computing system 106 may additionally or alternatively determine the number of times the biometric data and / or the private key 632 has been used with or to log into the user account 120. The computing system 106 may additionally or alternatively determine the number of times the private key 632 has been switched between different user accounts 120, such as by being disassociated / deregistered from a first user account 120 and then associated / registered with a second user account 120.

[0111] At 814, the computing system 106 may provide data as input to the trained machine learning model 216 via the scoring component 212. As indicated by sub-blocks 816, 818, and 820, in some embodiments this may involve providing various types of data as input to the trained machine learning model 216 at block 806. For example, at sub-block 816, the sensor data 604 received at block 802 may be provided as input to the trained machine learning model 216. At sub-block 818, the game control data 606 received at block 802 may be provided as additional input to the trained machine learning model 216. At sub-block 820, private key 632 data and / or biometric data may be provided as further additional input to the trained machine learning model 216, such as the private key 632 value and / or any of the data determined at sub-block 812. In some embodiments, wireless communication data (e.g., wireless signal strength data, data related to dropped data packets, etc.) may be provided as an input to the trained machine learning model 216 at block 814. In some embodiments, data 114 may be provided as an input to the trained machine learning model 216 at block 814, as described herein. In general, any quantifiable data included in the set of features used to train the machine learning model 216, as described herein, may be provided as an input to the trained machine learning model 216. In embodiments in which sensor data 604 is provided as input data to the model 216, the sensor data 604 constitutes unknown input used to score the user account 120 in question.

[0112] At 822, the computing system 106, via the scoring component 212, may generate a confidence score 118 associated with the user account 120 associated with the game control data 606 received at block 802 as an output from the trained machine learning model 216. The score 118 relates to a probability that the game control data 606 was generated by the handheld device 602 associated with the client machine 104 that transmitted the game control data 606 to the remote computing system 106. This score 118 can also be considered to relate to a probability that the game control data 606 was "human generated," as opposed to being synthesized and / or modified by software (e.g., an aimbot). For various types of data that can be input into the trained machine learning model 216 to generate the score 118, determining the score 118 can be based at least in part on the sensor data 604, based at least in part on the game control data 606, based on determining that the private key matches a private key in the database 116, and / or based at least in part on the length of time that the private key 632 has been associated with the user account 120, etc. In some embodiments, the score 118 is a normalized variable in the range of [0,1]. This confidence score 118 may have a monotonic relationship with the probability that the game control data 606 was generated by the handheld device 602. The relationship between the score 118 and the actual probability associated with a particular action, while monotonic, may or may not be a linear relationship.

[0113] Thus, process 800 represents a machine learning scoring approach, where a score 118 (e.g., confidence score 118) is determined for a user account 120, where the score indicates the probability that the game control data 606 was generated by a handheld device 602. Thus, the confidence score 118 generated in process 800 may indicate the probability that a player will engage in non-cheating behavior by playing the video game 110 using an actual handheld device 602. The use of machine learning models 216 in this scoring process may distinguish between human players and non-human players (e.g., software designed to cheat).

[0114] 9 is a flow diagram of an example process 900 for assigning user accounts 120 to different matches of a multiplayer video game 110 based on machine-learned confidence scores 118 associated with the probability that the game control data 606 was generated by the handheld device 602. For discussion purposes, the process 900 is described with reference to the previous figure. Additionally, as indicated by external page reference "D" in FIGS. 8 and 9, the process 900 may proceed from block 822 of process 800.

[0115] At 902, for a logged-in user account 120 scored according to process 800, the computing system 106, via the matchmaking component 214, may assign the logged-in user account 120 to an assigned match 218 of the plurality of matches 218, the assignment based at least in part on the machine-learned score 118 generated for and associated with the user account 120. The plurality of matches 218 may include at least one match 218(1) designated for a human player and a second match 218(2) designated for a non-human player, such as one that uses software to synthesize and / or modify game control data 606. As previously discussed, at block 902, any number of matches may be defined such that further divisions and additional matches may be assigned to the user account 120. For example, the computing system 106, via the matchmaking component 214, may define any suitable number of matches in which players 102 will be grouped to play the video game 110 in multiplayer mode, depending on a variety of factors, including demand, capacity, and other factors that affect the matchmaking process.

[0116] As indicated by subblock 904, the assignment of user account 120 to the assigned match may be based on a threshold score. For example, matchmaking component 214 may determine that score 118 associated with logged-in user account 120 is below a threshold score and may assign logged-in user account 120 to a second match 218(2) designated for a non-human player based on the score being below the threshold score. Similarly, matchmaking component 214 may determine that score 118 associated with logged-in user account 120 is equal to or greater than a threshold score and may assign logged-in user account 120 to a first match 218(1) designated for a human player based on the score being equal to or greater than the threshold score. However, this is only one exemplary manner for providing match assignment 124, and other techniques are contemplated. For example, clustering algorithms, or other statistical approaches may be used in addition to or as an alternative to using a threshold score. In one embodiment, confidence score 118 may be used to prioritize matching user accounts (players) with “similar” confidence scores. Given the inevitable tendency for the distribution of confidence scores 118 to be generally bimodal across multiple user accounts 120, grouping confidence scores 118 together based on a similarity metric (e.g., distance metric, dispersion metric, etc.) may yield similar results as using a threshold score. However, using a similarity metric to match user accounts together for matchmaking purposes may be useful in cases where the matchmaking pool is small (e.g., players in a small geographic area who want to play less popular game modes), as it may provide a more fine-grained approach that allows for gradual adjustment of the relative importance of confidence scores to other matchmaking factors such as skill level. In some embodiments, multiple thresholds may be used to "bucket" user accounts 120 into multiple separate matches.

[0117] As indicated by sub-block 906, other factors besides the confidence score 118 may be considered in the matchmaking assignment in block 902. For example, the match assignment 124 determined in block 902 may be further based on the skill level of a player associated with the logged-in user account, the amount of time the logged-in user account has been waiting to be placed in one of a plurality of matches, a geographic region associated with the logged-in user account, and / or other factors.

[0118] As indicated by sub-block 908, the assignment operation at block 902 may include a reassignment operation in which a user account 120 currently assigned to a first match 218(1) is reassigned to a second match 218(2), or vice versa. For example, the reassignment at sub-block 908 may include removing the user account 120 from the first match 218 and reassigning the user account 120 to a second match 218 based on a trust score 118 generated for the user account 120. This means that the trust scoring 118 may occur at run-time while the video game is being played, and a player may be reassigned midway through playing a game if a reasonably high probability is predicted based on the machine-learned trust score 118 that the player is cheating (e.g., using “aimbot” software). Thus, while match assignments may be made based on the machine-learned confidence scores 118 before the multiplayer video game 110 begins, the computing system 106 may regenerate the confidence scores 118 at any appropriate time during game play to determine whether a player is to be reassigned to a different match.

[0119] At 910, the computing system 106 may cause the execution of the video game 110 in an assigned match (e.g., one of the first match 218(1) or the second match 218(2)) for a logged-in user account 120 associated with the client machine 104 utilized to play the video game 110. For example, the computing system 106 may cause the execution of the video game 110 in the first match 218(1) for a first user account 120(1) associated with a first trust score 118(1) and may cause the execution of the video game 110 in the second match 218(2) for a second user account 120(2) associated with a second trust score 118(2). In a streaming implementation, the video game 110 may be executed on the remote computing system 106 and streamed to the client machine 104 at block 910. In other implementations, the remote computing system 106 may provide control instructions to a client application running the video game 110 on the client machine 104 to execute the video game 110 in an assigned match.

[0120] The machine-learned confidence scores 118 are used as a factor in the matchmaking process, which may improve the gaming experience for users who wish to play the video game in a multiplayer mode the way it was meant to be played, because the techniques and systems described herein may be used to match up players (e.g., non-human players) who are more likely to engage in malicious behavior (e.g., cheating by using software to synthesize and / or modify the game control data 606) and isolate these players from other trustworthy players who are more likely to play the video game legitimately.

[0121] Although the present subject matter has been described in language specific to structural features, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the particular features described. Rather, the specific features are disclosed as exemplary forms of implementing the claims. The invention as originally claimed in the present application is set forth below. [Appendix 1] One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, receiving game control data and sensor data from a client machine, the game control data and the sensor data being associated with a user account registered with a video game service; providing the sensor data as an input to a trained machine learning model; generating, as an output from the trained machine learning model, a confidence score associated with the user account, the confidence score relating to a probability that the game control data was generated by a handheld game controller associated with the client machine; assigning the user account to an assigned match of a plurality of matches for playing a video game in a multiplayer mode based at least in part on the confidence score, the plurality of matches including at least a first match designated for a human player and a second match designated for a non-human player using software to generate synthetic or modified game control data; and causing the execution of the video game in the assigned match for the user account. [Appendix 2] 2. The one or more non-transitory computer-readable media of claim 1, wherein the sensor data includes raw sensor data generated by one or more physical sensors of the handheld game controller, and the assigned match is the first match assigned to the human player. [Appendix 3] At least one of the game control data or the sensor data is received as encrypted data, and the operation comprises: decrypting the encrypted data; determining that a private key used to encrypt the encrypted data is associated with the user account; determining that the private key matches one of a plurality of private keys maintained in a database, the plurality of private keys being associated with a plurality of trusted handheld game controllers; 2. The one or more non-transitory computer-readable media of claim 1, wherein the generation of the trust score is further based on the determination that the private key matches the one of the multiple private keys. [Appendix 4] said assigning said user account to said assigned match; removing the user account from the first match; and reassigning the user account to the second match. [Appendix 5] 1. A method comprising: receiving, by a computing system, game control data and sensor data from a client machine, the game control data and the sensor data being associated with a user account logged into a video game service, the game control data being processed by the video game to control aspects of the video game; determining, by a computing system, a score associated with the user account; providing the sensor data as input to a trained machine learning model; generating the score as an output from the trained machine learning model, the score indicating a probability that the game control data was generated by a handheld device associated with the client machine; assigning the user account to an assigned game of a plurality of games based at least in part on the score, the assigned game including at least one of a first game or a second game; causing, by the computing system, the execution of the video game in the assigned match for the user account. [Appendix 6] 6. The method of claim 5, wherein the sensor data includes raw sensor data generated by one or more physical sensors of the handheld device. [Appendix 7] 7. The method of claim 6, wherein the raw sensor data includes at least one of raw gyroscope data, raw accelerometer data, or raw capacitive sensor data. [Appendix 8] 7. The method of claim 6, wherein the handheld device is a handheld game controller. [Appendix 9] prior to said determining said score, accessing, by the computing system, historical sensor data associated with a sampled set of user accounts registered with the video game service, the historical sensor data received from client machines along with historical game control data processed by the video game or the different video game to control aspects of the video game or the different video game; labeling each user account of the sampled set of user accounts with a label indicating whether the historical game control data associated with the user account was generated by a handheld device while a human player held and manipulated the handheld device to generate the historical game control data associated with the user account; 6. The method of claim 5, further comprising training a machine learning model using the historical sensor data as training data to obtain the trained machine learning model. [Appendix 10] 10. The method of claim 9, wherein training the machine learning model includes setting weights on at least one of a set of features derived from the historical sensor data or parameters internal to the machine learning model. [Appendix 11] determining said score In addition to providing the sensor data as the input to the trained machine learning model, further comprising providing the game control data as an additional input to the trained machine learning model; 6. The method of claim 5, wherein the score is generated based at least in part on the game control data provided as the additional input to the trained machine learning model. [Appendix 12] At least one of the game control data or the sensor data is received as encrypted data, the method comprising: decrypting, by the computing system, the encrypted data; and determining, by the computing system, that a private key used to encrypt the encrypted data is associated with the user account; determining, by the computing system, that the private key matches one of a plurality of private keys maintained in a database, the plurality of private keys being associated with a plurality of trusted handheld devices; 6. The method of claim 5, wherein the determining the score is further based on the determining that the private key matches the one of the multiple private keys. [Appendix 13] 13. The method of claim 12, further comprising determining an amount of time the private key has been associated with the user account, and wherein the determining of the score is further based on the amount of time. [Appendix 14] 6. The method of claim 5, wherein allocating the user account to at least one of the first match or the second match based at least in part on the score includes determining whether the score meets a threshold score. [Appendix 15] 1. A system comprising: one or more processors; and a memory storing computer-executable instructions that, when executed by the one or more processors, cause the system to: receiving game control data and sensor data from a plurality of client machines, the game control data and the sensor data being associated with a logged-in user account of a plurality of user accounts registered with a video game service, the game control data being processed by the video game to control one or more aspects of the video game; determining a score for the logged in user account, each score of the scores comprising: providing the sensor data associated with an individual user account as input to a trained machine learning model; and generating, as an output from the trained machine learning model, a score associated with the individual user account, the score being related to a probability that the game control data associated with the individual user account was generated by a handheld device associated with a client machine of the plurality of client machines; assigning a first subset of the logged-in user accounts to a first match and a second subset of the logged-in user accounts to a second match based at least in part on the scores determined for the logged-in user accounts; causing execution of the video game in the first match for the first subset of the logged-in user accounts and in the second match for the second subset of the logged-in user accounts. [Appendix 16] 16. The system of claim 15, wherein the sensor data associated with the first subset of logged-in user accounts includes raw sensor data generated by one or more physical sensors of a handheld device. [Appendix 17] 17. The system of claim 16, wherein the raw sensor data includes at least one of raw gyroscope data, raw accelerometer data, or raw capacitive sensor data. [Appendix 18] The individual scores of the scores are: further determined by, in addition to providing the sensor data associated with the individual user account as the input to the trained machine learning model, providing the game control data associated with the individual user account as an additional input to the trained machine learning model; 16. The system of claim 15, wherein the score associated with the individual user account is generated based at least in part on the game control data associated with the individual user account provided as the additional input to the trained machine learning model. [Appendix 19] When at least one of the game control data or the sensor data is received as encrypted data and the computer-executable instructions are executed by the one or more processors, the system: decrypting the encrypted data; determining that a private key used to encrypt the encrypted data is associated with one of the logged-in user accounts; determining that the private key matches one of a plurality of private keys maintained in a database, the plurality of private keys being associated with a plurality of trusted handheld devices; 16. The system of claim 15, wherein an individual score associated with the one of the logged-in user accounts is further determined based on a determination that the private key matches the one of the plurality of private keys. [Appendix 20] 20. The system of claim 19, wherein the computer-executable instructions, when executed by the one or more processors, further cause the system to determine an amount of time that the private key has been associated with the one of the logged-in user accounts, and wherein the individual score associated with the one of the logged-in user accounts is further based on the amount of time.

Claims

1. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, receiving game control data and sensor data from a client machine, the game control data and the sensor data being associated with a user account registered with a video game service, where at least one of the game control data or the sensor data is received as data encrypted using a private key associated with the user account; determining private key data associated with the private key, the private key data the length of time that the private key has been associated with the user account; the number of times the private key has been used for the user account or to log into the user account; or The number of times the private key has been switched between different user accounts At least one of the following is provided: providing the sensor data and the private key data as inputs to a trained machine learning model; generating, as an output from the trained machine learning model, a confidence score associated with the user account, the confidence score relating to a probability that the game control data was generated by a handheld game controller associated with the client machine; assigning the user account to an assigned match of a plurality of matches for playing a video game in a multiplayer mode based at least in part on the confidence score, the plurality of matches including at least a first match designated for a human player and a second match designated for a non-human player using software to generate synthetic or modified game control data; and causing the execution of the video game in the assigned match for the user account.

2. 2. The one or more non-transitory computer-readable media of claim 1 , wherein the sensor data comprises raw sensor data generated by one or more physical sensors of the handheld game controller, and the assigned match is the first match assigned to the human player.

3. The operation, decrypting the encrypted data; determining that the private key is associated with the user account; determining that the private key matches one of a plurality of private keys maintained in a database, the plurality of private keys being associated with a plurality of trusted handheld game controllers; 10. The one or more non-transitory computer-readable media of claim 1 , wherein the generation of the trust score is further based on the determination that the private key matches the one of the multiple private keys.

4. said assigning said user account to said assigned match; removing the user account from the first match; and reassigning the user account to the second match.

5. 1. A method comprising: receiving, by a computing system, game control data and sensor data from a client machine, the game control data and the sensor data being associated with a user account logged into a video game service, the game control data being processed by the video game to control aspects of the video game, wherein at least one of the game control data or the sensor data is received as data encrypted using a private key associated with the user account; determining private key data associated with the private key, the private key data the length of time that the private key has been associated with the user account; the number of times the private key has been used for the user account or to log into the user account; or The number of times the private key has been switched between different user accounts from the user account. At least one of the following is provided: determining, by a computing system, a score associated with the user account; providing the sensor data and the private key data as inputs to a trained machine learning model; generating the score as an output from the trained machine learning model, the score indicating a probability that the game control data was generated by a handheld device associated with the client machine; assigning the user account to an assigned game of a plurality of games based at least in part on the score, the assigned game including at least one of a first game or a second game; causing, by the computing system, the execution of the video game in the assigned match for the user account.

6. The method of claim 5 , wherein the sensor data comprises raw sensor data generated by one or more physical sensors of the handheld device.

7. The method of claim 6 , wherein the raw sensor data comprises at least one of raw gyroscope data, raw accelerometer data, or raw capacitive sensor data.

8. The method of claim 6 , wherein the handheld device is a handheld game controller.

9. prior to said determining said score, accessing, by the computing system, historical sensor data associated with a sampled set of user accounts registered with the video game service, the historical sensor data received from client machines along with historical game control data processed by the video game or the different video game to control aspects of the video game or the different video game; labeling each user account of the sampled set of user accounts with a label indicating whether the historical game control data associated with the user account was generated by a handheld device while a human player held and manipulated the handheld device to generate the historical game control data associated with the user account; The method of claim 5 , further comprising: training a machine learning model using the historical sensor data as training data to obtain the trained machine learning model.

10. 10. The method of claim 9, wherein the training the machine learning model includes setting weights on at least one of a set of features derived from the historical sensor data or parameters internal to the machine learning model.

11. determining said score In addition to providing the sensor data and the secret key data as the inputs to the trained machine learning model, providing the game control data as an additional input to the trained machine learning model; The method of claim 5 , wherein the score is generated based at least in part on the game control data provided as the additional input to the trained machine learning model.

12. decrypting, by the computing system, the encrypted data; and determining, by the computing system, that the private key is associated with the user account; determining, by the computing system, that the private key matches one of a plurality of private keys maintained in a database, the plurality of private keys being associated with a plurality of trusted handheld devices; The method of claim 5 , wherein the determining the score is further based on the determining that the private key matches the one of the multiple private keys.

13. 6. The method of claim 5, wherein the assigning the user account to the at least one of the first match or the second match based at least in part on the score comprises determining whether the score meets a threshold score.

14. 1. A system comprising: one or more processors; and a memory storing computer-executable instructions that, when executed by the one or more processors, cause the system to: receiving game control data and sensor data from a plurality of client machines, the game control data and the sensor data being associated with logged-in user accounts of a plurality of user accounts registered with a video game service, the game control data being processed by the video game to control one or more aspects of the video game, wherein at least one of the game control data or the sensor data is received as data encrypted using a private key associated with each of the logged-in user accounts; determining private key data associated with the private key, the private key data the length of time that the private key is associated with each logged-in user account; the number of times the private key has been used for each of the currently logged-in user accounts or to log into each of the currently logged-in user accounts; or The number of times the private key has been switched between different user accounts At least one of the following is provided: determining a score for the logged in user account, each score of the scores comprising: providing the sensor data and the private key data associated with an individual user account as input to a trained machine learning model; and generating, as an output from the trained machine learning model, a score associated with the individual user account, the score being related to a probability that the game control data associated with the individual user account was generated by a handheld device associated with a client machine of the plurality of client machines; assigning a first subset of the logged-in user accounts to a first match and a second subset of the logged-in user accounts to a second match based at least in part on the scores determined for the logged-in user accounts; causing execution of the video game in the first match for the first subset of the logged-in user accounts and in the second match for the second subset of the logged-in user accounts.

15. The system of claim 14 , wherein the sensor data associated with the first subset of logged-in user accounts comprises raw sensor data generated by one or more physical sensors of a handheld device.

16. The system of claim 15 , wherein the raw sensor data includes at least one of raw gyroscope data, raw accelerometer data, or raw capacitive sensor data.

17. The individual scores of the scores are: further determined by providing the game control data associated with the individual user account as an additional input to the trained machine learning model in addition to providing a private key and the sensor data associated with the individual user account as the input to the trained machine learning model; 15. The system of claim 14, wherein the score associated with the individual user account is generated based at least in part on the game control data associated with the individual user account provided as the additional input to the trained machine learning model.

18. The computer-executable instructions, when executed by the one or more processors, provide the system with: decrypting the encrypted data; determining that a private key used to encrypt the encrypted data is associated with one of the logged-in user accounts; determining that the private key matches one of a plurality of private keys maintained in a database, the plurality of private keys being associated with a plurality of trusted handheld devices; 15. The system of claim 14, wherein an individual score associated with the one of the logged-in user accounts is further determined based on a determination that the private key matches the one of the plurality of private keys.

19. The individual scores of the scores are: and further determining, in addition to providing the private key data and the sensor data associated with the individual user account as inputs to the trained machine learning model, by providing wireless communication data associated with the individual user account as an additional input to the trained machine learning model, the wireless communication data indicative of at least one of a wireless signal strength associated with a handheld device or instances of dropped data packets transmitted from the handheld device.

15. The system of claim 14, wherein the score associated with the individual user account is generated based, at least in part, on the wireless communication data associated with the individual user account that is provided as the additional input to the trained machine learning model.

20. the determining of the score further comprises, in addition to providing the private key data and the sensor data as inputs to the trained machine learning model, providing wireless communication data as an additional input to the trained machine learning model, the wireless communication data indicative of at least one of a wireless signal strength associated with a handheld device or instances of dropped data packets transmitted from the handheld device; 6. The method of claim 5, wherein the score is generated at least in part based on the wireless communication data provided as the additional input to the trained machine learning model.

Images