System and method for cheat detection in video games

Abstract

A system and method may be implemented by from an entity (which may be a cheating person or a robot; the method includes receiving input to a computer game (e.g., a human), comparing the input from an entity to data relating to the input to the computer game from a human player, and based on the comparison, determining whether the entity is a human player or a cheating person to detect cheating in the computer game. The comparison may be based on performance metrics derived from the inputs, and may be made by statistical analysis or comparison of statistical distributions.

Description

Field of the Invention

[0001] The present invention generally relates to the field of detecting cheating in video games and esports, such as by analyzing the inputs of a purported human player and comparing them to the inputs from a real human player. Background of the Invention

[0003] In video games and esports (e.g., first-person shooter games), human players can interact with the game via input devices (e.g., computer mouse and keyboard, game controllers for game consoles such as Microsoft Xbox systems or Sony PlayStation systems). For example, a human player can perform coordinated movements of the input device to control an on-screen object or view (e.g., an avatar) in the virtual environment of the game. Performance metrics can be determined.

[0004] Human performance in first-person shooter (FPS) video games depends on learned perceptual and motor skills. Competitive players spend years improving these skills. Successful FPS games require effectively identifying and locating relevant visual stimuli and coordinating dynamic movements with precise shooting responses.

[0005] Some players cheat by running or executing a second computer program (e.g., an aim bot or a trigger bot) that operates in parallel with the computer program used to play the game (e.g., an esports game or a video game) on the same computer. A player or entity may be recognized by the game system as a human player, but their input may be provided or assisted by a robot. Some aim bot or trigger bot computer programs read from the computer memory to determine the state of the first computer program (e.g., the game), including the location of targets or opponents. Other aim bot or trigger bot computer programs read from the computer's frame buffer and apply computer vision and image processing algorithms to detect and locate targets or opponents. Then, some aim bot computer programs write to the memory in such a way as to automatically aim and shoot at the target without the player having to move or manipulate their input device. Some aim bot computer programs (e.g., via a USB connector) provide input signals to simulate an input device to automatically aim and shoot at the target without the player having to move or manipulate their input device. Some aim bot computer programs assist human players, automatically correcting the errors or inaccuracies of human players when aiming and shooting at a target, and such players may be considered cheaters. Some trigger bot computer programs cause the cheater's weapon to automatically fire when the target is in the crosshairs. These aim bot programs can be considered cheaters and may claim to provide inputs that are recognized as those of a human player.

[0006] In the gaming industry, it is well known that without the use of computer processes, humans do not have the ability to detect cheaters. Existing cheating detection techniques or software also do not have the ability to detect many advanced cheaters. Cheating detection techniques have difficulty distinguishing between external simulated inputs and legitimate professional-level human games. Summary of the Invention

[0008] A system and method can detect cheating in computer games and esports (e.g., first-person shooter games) by receiving an input to the computer game from an entity (which may be a cheater or a robot; or a human), comparing the input from the entity with data related to inputs to the computer game from human players (e.g., a verified database), and based on that comparison, determining whether the entity is a human player or a cheater. The comparison can be based on performance metrics derived from the input and can be by statistical analysis or comparison of statistical distributions. The process can obtain a verified database of performance metrics from human players, characterize the statistical distribution of these human performance metrics, and compare them with the performance metrics from a new player to determine whether the new player is human or a cheater (e.g., an aimbot or triggerbot in a first-person shooter game). Brief Description of the Drawings

[0010] The subject matter of the present invention is particularly pointed out and distinctly claimed at the end of the specification. However, the present invention as to the organization and method of operation thereof, and its objects, features and advantages, will be best understood from the following detailed description when read in conjunction with the Figure 1 accompanying drawings in which:

[0011] Figure 1 A screenshot of an Adaptive Reflexshot task scenario according to some embodiments is depicted.

[0012] Figure 2 An example of movement parsing utilizing changes in movement direction according to some embodiments is shown.

[0013] Figure 3 An illustration of a movement trajectory according to some embodiments is shown.

[0014] Figure 4 An example of a movement trajectory and model fitting according to some embodiments is shown.

[0015] Figure 5 Examples of firing speed, shot variability, and firing accuracy according to some embodiments are shown.

[0016] Figure 6It is a flowchart of a method according to an embodiment of the present invention.

[0017] Figure 7 It is an example curve graph of the accuracy of cheating detection according to some embodiments.

[0018] Figure 8 It is an example computer system for performing the methods discussed herein according to some embodiments.

[0019] Figure 9 It is an example computer system for performing the methods discussed herein according to some embodiments.

[0020] It will be understood that, for simplicity and clarity of illustration, the elements shown in the figures are not necessarily drawn to scale. For example, for clarity, the dimensions of some elements may be exaggerated relative to other elements. Additionally, where considered appropriate, reference numerals may be repeated in multiple figures to indicate corresponding or analogous elements.

[0021] Detailed Description of the Invention

[0022] Those skilled in the art will recognize that the present invention may be embodied in specific forms other than the examples given herein without departing from the spirit or essential characteristics of the present invention. Accordingly, the foregoing embodiments are considered illustrative in all respects and not restrictive of the invention described herein. Thus, the scope of the present invention is represented by the appended claims rather than the foregoing description.

[0023] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, those skilled in the art will understand that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention. Some features or elements described with respect to one embodiment may be combined with features or elements described with respect to other embodiments. For clarity, the same or similar features or elements may not be discussed repeatedly.

[0024] Although embodiments of the present invention are not limited in this regard, discussions using terms such as, for example, "processing", "computing", "calculating", "determining", "establishing", "analyzing", "inspecting", etc. may refer to operations and / or processes of a computer or other electronic computing device that manipulates and / or transforms data represented as physical (e.g., electronic) quantities within the registers and / or memories of the computer into other data similarly represented as physical quantities within the registers and / or memories of the computer or other non-transitory storage media that may store instructions for performing the operations and / or processes.

[0025] An embodiment of a cheating detector is incorporated into a first computer program (e.g., an esports game or a video game) that compares performance metrics of newly-seen entities or players to a database of performance metrics from human players (e.g., a verified database). The embodiment can determine whether a newly-added player is a genuine human player or a cheater. A cheater can be considered a robot or an automated process, or a person using such a robot or automated process or assisted by such a robot or automated process; a person who cheats can use a robot. A cheater can be detected when the performance metrics of a new player do not match the database of performance metrics from genuine human players. When discussing the analysis or processing of input, "player" or "entity" includes genuine human players, as well as newly-added players, which can be: a robot or an automated process (e.g., a cheater); a person assisted by or using a robot or an automated process (also a cheater); or a genuine human player (e.g., not a cheater).

[0026] Player inputs can be used to calculate performance and performance metrics and can include manipulating an input device (e.g., computer mouse, Wii controller, keyboard, mobile computing device (e.g., tablet), mobile phone, game controller, etc.) to move the player, or move the player's aim (or the aim of a weapon), or the player's view; and also trigger or provide a firing input to discharge in-game equipment; as well as inputs for moving the player within the game. Inputs other than moving the player, or aiming or performing a firing can be used. Inputs from a player or an entity (e.g., a robot) can include a series of data points such as mouse movement or game controller movement, and the input can also refer to data generated by such a device, such as a series of data points (or data purporting to be such data points) generated by the input device. Other inputs can be used. For example, a user can move a mouse to change the player's aim and field of view, and can click a mouse button to discharge a weapon in the game. Measurements of speed, accuracy, precision, swipiness, etc. can be based on the relationships between game entities (e.g., targets, player point of view (POV), aim of a weapon), the movement of these entities (e.g., movement of the aim of a weapon, target movement), and the timing of entity actions (e.g., weapon discharge, target appearance, etc.). For example, the accuracy of a firing triggered by a user input to a mouse or game controller can be determined by the aiming point (e.g., crosshair) relative to the point or location of the expected target at the time of firing. A user can move their avatar or player via, for example, keyboard input (e.g., the known arrow keys or WASD input method), which in turn can affect the game entity of the player's point of view, which typically co-extends with the crosshair of a virtual weapon or the aimed entity in a FPS game. A player can use an input device to aim a weapon, which typically means aiming the POV, and can provide a trigger input to the input device to discharge the weapon. Other methods of user input can be used. The trajectory or movement of the aim or crosshair can be used to measure player movement, such as swipiness. The crosshair or aiming position of a weapon can determine where a shot lands: for example, if the crosshair is pointing / showing above the center of a target when a shot is discharged, the shot is determined to be or is considered to be located at the center of the target. Other systems can have more complex relationships between the crosshair, the target, and the landing point of a shot, e.g., by considering simulated gravity that causes a simulated projectile to fall over a certain distance.

[0027] Evaluations, calculations, displays, etc. as described herein are typically performed automatically using a computer system as described herein, typically based on user (e.g., a player of a game) input and / or input from a cheating process (e.g., a bot or aimbot). Player activities as described herein (e.g., flicking, firing a shot) are typically provided as input to the computer system, and the player typically views a computer-generated display. A player or entity "moving" can refer to, for example, a user or entity providing input that causes some display representation of an object (e.g., an avatar or a crosshair) on the screen in a virtual environment of a game to move, or to move or reorient the field of view.

[0028] Performance metrics

[0029] In some embodiments, performance metrics can include movement kinematics. A player typically aims at a target by starting to move towards the target, increasing the movement speed and then decreasing the movement speed, and firing a shot after decelerating or coming to a complete stop. In some embodiments, the kinematics of these movements are characterized by fitting a sigmoid function or other function or line to a movement time series (e.g., based on player input or parsed input). The optimal fit parameter values of the sigmoid function or other function for the movement, player input, or parsed input can indicate movement amplitude (e.g., the amplitude of the sigmoid), movement accuracy (e.g., the angular distance between the target and the landing point of the movement), movement speed (e.g., the slope of the sigmoid), movement variability (e.g., the median of multiple movements of the absolute angular distance between the target and the landing point of the movement; other methods of calculating variability can be envisioned as discussed elsewhere herein), movement precision (e.g., 1 / movement variability), and reaction time (e.g., the time point at which the sigmoid movement starts).

[0030] In some embodiments, performance metrics can include slipperiness. In addition to the typical flick-and-land movement that is fast and accurately hits the target, a player sometimes chooses to maximize speed by "firing on the fly" rather than decelerating first and then firing, a tactic known as "swipe" movement. For a swipe, the end point of the movement far exceeds the target position. There is a continuum of swipiness, with an ideal flick-and-land and an ideal swipe corresponding to the two ends of the continuum. The kinematics of human movement depend on task demands: for game scenarios that incentivize speed rather than accuracy, the shorter the reaction time, the less precise the movement, and the more prone to swiping.

[0031] In some embodiments, performance metrics may include shooting performance, such as shooting time (e.g., the time interval between target appearance and the first shot directed at the target) and shooting error (e.g., the angular distance between the first shot and the center of the target), each of which is a function of target size and target angular distance (e.g., the angular distance between the player's orientation and the target position).

[0032] In some embodiments, the performance metric may characterize the number of corrective movements to hit a target. A player may fail to hit the target on their first attempt and need to make corrective movements. Sometimes there are multiple corrective movements. The performance metric may include the median number of corrective movements to hit the target (e.g., as a function of the number of shot attempts) and the proportion of targets that the player hits after N movements, where N is a positive integer.

[0033] In some embodiments, the performance metric may include the proportion of movements that do not have a shot associated with the movement.

[0034] Embodiments of the present invention may calculate or determine performance metrics, for example, based on player input (e.g., from a human player or a robot or a human assisted by a robot), which can be used to evaluate the player. These metrics may include movement kinematics, such as, for example, movement amplitude, movement speed, accuracy of movement landing point, reaction time between target appearance and movement start, each of which is a function of target size (e.g., which depends on the distance in the virtual environment of a first-person shooter game); the metrics may include the relationships between these example kinematics. The performance metric may include slipperiness (e.g., in a first-person shooter game, the tendency to shoot while moving rather than landing on the target first and then shooting). The performance metric may include shooting performance in a first-person shooter game, such as, for example, shooting time (e.g., the time interval between target appearance and the first shot directed at the target) and shooting error (e.g., the angular distance between the first shot and the center of the target), each of which is a function of target size and target angular distance (e.g., the angular distance between the player's orientation and the target position in the virtual environment of a first-person shooter game). The performance metric may include the number of corrective movements to hit the target in a first-person shooter game. Other performance metrics may be used.

[0035] Known human behavior exhibits a trade-off between these different performance metrics. Human performance shows a speed-accuracy trade-off or relationship, e.g., the speed of making a response or movement is negatively correlated with the accuracy or precision of that movement. One can be very fast but not very accurate, very accurate but slow, or somewhere in between. The speed-accuracy trade-off can be evident for different aspects of speed (e.g., reaction time, movement speed, firing time) and different aspects of accuracy (e.g., percentage of correct responses / decisions, movement accuracy and variability, firing error). The hallmark of the speed-accuracy trade-off is the ability of humans to adapt to current needs and prioritize speed and accuracy relative to each other. If a task requires a very rapid response or movement, a person may sacrifice accuracy to maximize speed. If the cost of a miss is high, a person may take longer to respond or move more slowly to maximize accuracy.

[0036] Breaking the speed-accuracy trade-off may indicate cheating. Some embodiments perform statistical analysis to characterize the multivariate probability density of a database of human performance metrics (e.g., a validated database). For example, given a target size and target angular distance, the probability density can characterize the likelihood that a movement is a real human movement given a particular combination of performance metrics (e.g., movement amplitude, movement speed, movement accuracy, movement reaction time, slipperiness, firing time, and firing error). Some embodiments perform statistical analysis to determine whether performance metrics from a new player are statistical outliers. After characterizing the multivariate probability density of performance metrics from a validated database of human movement, cheating can be detected by determining the likelihood that a new player is a human player (e.g., the likelihood of the performance metrics from the new player relative to the probability density of the validated database of human performance metrics).

[0037] The speed-accuracy trade-off in FPS performance can be characterized, for example, according to movement kinematics (e.g., reaction time to start moving, movement speed, movement accuracy, variability or precision of multiple movements). The speed-accuracy trade-off can be characterized according to firing performance (e.g., time taken to fire a shot and the error of that shot).

[0038] Different speed measurements can replace fire rate (e.g., shots per second) and can be considered performance metrics. Embodiments can calculate a speed performance metric as a hit rate (e.g., hits per second), movement per second, average or median movement speed (e.g., in centimeters per second, degrees per second, pixels per second, etc.), or peak speed during movement. For example, speed can be calculated by: 1) separately determining, for each target, the elapsed time between target spawn or appearance and the first shot fired (e.g., the input received from a game controller to fire a shot); 2) calculating the average or median elapsed time of this spawn-shot time for multiple targets; and 3) calculating 1 divided by the average or median elapsed time. In one embodiment, accuracy can be calculated by: 1) calculating the shot error of the first shot fired at each target (e.g., the distance between the landing point of the shot in the game and the center of the target); 2) calculating the average or median of the shot errors for multiple targets; and 3) calculating 1 divided by the average or median shot error. Other metrics of accuracy can be used.

[0039] Aim Lab

[0040] Embodiments can be used with the Aim Lab TM FPS video game, and the Aim Lab TM system can be used to validate embodiments of the disclosed invention. Aim Lab includes a set of tasks (e.g., mini-games) that replicate FPS game scenarios. To validate some aspects of the disclosed embodiments, data from human Aim Lab players is analyzed to characterize human performance. These human performance metrics are compared to the performance metrics of non-human aiming robots, which are computer programs that automatically aim at and shoot targets. Aiming robots or trigger bot programs may not mimic inputs corresponding to natural human movement and reaction times and may not exhibit the same performance metrics as human players. For example, aiming robot reaction times may be unrealistically short, movement speeds may be unrealistically fast, and accuracy may be unrealistically good. Additionally, aiming robots may not exhibit typical human trade-offs. For example, aiming robot movement accuracy may not be traded off against movement speed or amplitude.

[0041] The Aim Lab TM system is a commercial software product written in the C# programming language using the Unity game engine. Unity is a cross-platform video game engine used to develop digital games for computers, mobile devices, and game consoles. Players download the Aim Lab TM system to their desktop or laptop computers. While Aim Lab is one example system used with embodiments of the present invention, other systems can be used.

[0042] Players use a mouse and keyboard to control their virtual weapons in an Aim Lab TM task while viewing the game on a computer screen. Performance data is uploaded to an Aim Lab TM secure server. Aim Lab TM includes a large number of different task scenarios for skill assessment and training, each customized for an aspect of FPS games. These task scenarios evaluate and train visual detection, motion control, tracking moving targets, auditory spatial localization, change detection, working memory capacity, cognitive control, distraction, and decision-making. Tasks can be customized to prioritize accuracy, speed, and other fundamental performance components. In a round, players receive points for each target they successfully shoot and destroy. In some tasks, destroying a target more quickly rewards additional points. Players attempt to maximize their score per round by destroying as many targets as possible. Different task scenarios motivate players to prioritize accuracy over speed, or speed over accuracy.

[0043] A fundamental aspect of FPS games is called "flicking", quickly moving the crosshair to a target or opponent and firing a shot to damage or destroy the target. For example, when playing with a computer mouse and keyboard, flicking is mainly achieved through hand and arm (e.g., stretching) movement. Some embodiments utilize task scenarios that evaluate flicking skills, e.g., quick ballistic movement to destroy stationary targets. Specifically, embodiments utilize flicking tasks for different target sizes and / or different target presentation durations to characterize each player's performance metrics, including the speed-accuracy trade-off. Multiple ways can be used to calculate speed to determine flicking skills.

[0044] An example of a task scenario in Aim Lab TM is the adaptive reflection or reflex shot task (an example of which is shown in Figure 1 . In this task scenario, one target at a time appears in a random position, confined within an imaginary ellipse in front of the player's virtual avatar. The player has a limited time to destroy each target before it "times out" and disappears. If a target times out, the player does not receive any points for that target. To provide a visual cue of the remaining time, each target gradually becomes more transparent and then disappears. This motivates the player to be as fast as possible. Figure 1 The three panels in the top row show a time series of screenshots as the target gradually becomes more transparent. Figure 1The three panels in the bottom row show examples of three different target sizes. In some embodiments, the target presentation duration is adjusted according to performance: it decreases in the next test after the target is destroyed (the target becomes transparent faster), and it increases in the next test after the target times out (the target becomes transparent slower). In some embodiments, the target size changes from one round of the task to the next, and the target duration is adjusted separately for each target size. In some embodiments, different target sizes are staggered within each round of the game, and the target duration is adjusted separately for each target size. In other embodiments, the target size is adjusted within each round of the game: it decreases in the next test after the target is destroyed, and it increases in the next test after the target times out. In some embodiments, the target duration is constant during each round, the target duration changes from one round of the task to the next, and the target size is adjusted during each round. In some embodiments, different target durations are staggered within each round of the game, and the target size is adjusted separately for each target duration. In some embodiments, both different target durations and different target sizes are staggered within each round of the game, and based on whether the player has destroyed the target, both the target size and the target duration are adjusted differently for each test. In all of these embodiments of the adaptive reflex shooting task, the player is incentivized to adjust their speed and accuracy according to the target size and duration in order to maximize their score.

[0045] Input device calibration

[0046] In some embodiments, the orientation of the player's view of the environment controlled by the input device is recorded in Euler angles and sampled at 120 Hz. Other sampling rates can be used instead of 120 Hz; for example, the sampling rate ranges from 30 Hz to 240 Hz. Other representations of the player's orientation and movement can be used instead of Euler angles, such as pixels or quaternions. In some embodiments, a crosshair marked with a dot at the center of the screen corresponds to the direction in which the shot will be fired. When triggered, the shot moves in a straight line in the direction away from the player's virtual avatar.

[0047] Some embodiments measure the player's physical movement. For example, when the input device is a computer mouse, the orientation data is converted into the corresponding movement of the mouse on the mouse pad, in centimeters or inches. This requires additional information about the relationship between physical mouse movement and changes in the player's orientation, which may vary from player to player due to the player's hardware and software settings.

[0048] Some embodiments record in-game settings that control the field of view and the mouse sensitivity (the magnitude of the orientation change resulting from a single increment or count of the mouse). The mouse sensitivity for x-axis and y-axis movement can be recorded independently. Additionally, some embodiments record the player's mouse (e.g., hardware, software, or both) settings that determine the number of counts required to produce a one-inch distance traveled (counts per inch, CPI). Other metrics can be used.

[0049] In one embodiment, the orientation of the player's avatar in the virtual environment is converted to centimeters of physical mouse movement, for example, according to the following:

[0050] 1. Mouse sensitivity * 0.05 = Angle increment (degrees of rotation per count)

[0051] 2. Total degrees of rotation / Angle increment = Counts

[0052] 3. (Counts / CPI) * 2.54 = Physical distance traveled (cm)

[0053] The value 0.05 is an example constant used in the Unity software code to scale mouse counts to degree increments. Other constants and other formulas can be used.

[0054] Those skilled in the art will recognize that similar methods and calculations can be used to calibrate other input devices, and distances can be specified in metric units (e.g., centimeters) or imperial units (e.g., inches).

[0055] Movement Parsing

[0056] Some embodiments parse the time series of each player's movement or input obtained by the input device. In this way, a player's input (e.g., a mouse movement that may be combined with a shot (indicated, for example, by a mouse click)) can be converted into discrete and defined parsed input, and then the parsed input can have one or more metrics derived from the parsed input. The movement can be recorded as a change in orientation (e.g., Euler angles) in the virtual environment of the game. Movement parsing can label each time point as, for example, in motion or stationary; other labels can be used. An epoch (e.g., a time period corresponding to a series of consecutive time samples) can be labeled as in motion after a number of consecutive samples exceed a speed threshold. An epoch can be labeled as stationary after a number of consecutive samples drop below the speed threshold. The number of consecutive samples used and the value of the threshold can be determined from a large dataset that includes data from a large number of players. The duration of each consecutive epoch (e.g., in motion or stationary) can vary.

[0057] A player may change the direction of their movement without noticeably slowing down the speed of movement. This may occur, for example, when there are multiple targets in a virtual environment, and the player begins moving toward one target, and then decides to prioritize a different target without destroying or shooting the original target. As another example, the player may not reach a target quickly enough, and therefore never fire a shot at that target even though a ballistic movement is initiated in the direction of that target. Some embodiments utilize changes in the direction of movement to resolve movements. If a player moves in one direction and then changes to a substantially different direction, the time series may be segmented such that the period before the change in direction is associated with one movement, and the period after the change in direction is associated with a different movement. Some embodiments detect a change in direction when the angle of the speed of movement changes above a threshold in at least one time sample. Other embodiments detect a change in direction when the angle of the speed of movement changes above a threshold in multiple time samples. The number of consecutive samples and the value of the threshold may be determined from a large data set that includes data from a large number of players. In Figure 2 An example is shown in Figure 2 , the player's orientation is plotted in degrees on the y-axis over time in seconds along the x-axis. The thick and thin lines represent the x- and y-components of the player's orientation, respectively. The periods encompassed by the three-sided rectangular shape (missing the bottom) represent times when the player's movement is fast enough to be considered in motion. During the time period shown, the player moves in one direction and then quickly changes direction without slowing down. The points of direction change are resolved into separate moves, which can be referred to as resolved inputs, e.g., discrete inputs that can be analyzed by calculating an indicator that describes it. The solid line and the solid curve correspond to the first of the two moves. The dashed line and the dashed curve correspond to the second of the two moves.

[0058] Figure 3 1 shows an illustration of a movement trajectory according to some embodiments. A primary movement toward a target is often followed by a corrective movement. For example, a primary movement may be hypermetric, exceeding the target, and returning to the target by a corrective action in the opposite direction ( Figure 3 , C to D). Conversely, the primary move may instead be hypometric, falling short of the goal ( Figure 3 , A to B), a corrective movement in the same direction as the main movement is required. Sometimes there are multiple corrective movements. In some embodiments, a single main movement for each target can be identified as the movement of the largest magnitude (e.g., within ±45 degrees) in the direction of that target. In some embodiments, different types of movement components (main movement and corrective movement) can be analyzed separately.

[0059] In some embodiments, performance metrics can include the median number of corrective movements to hit a target and the proportion of targets that a player hits with N movements (where N is a positive integer).

[0060] Some embodiments associate each movement of a player with a shot (e.g., automatically using a computer system as described herein). A player typically makes a series of movements to destroy a target. To associate a shot with one or more movements, some embodiments first identify the time point at which the shot is fired and step back in time (e.g., make an assessment) from the time the shot is fired to identify the movement that began at a time point prior to that shot. Each shot can be associated with more than one movement. Each movement can be associated with more than one shot. For example, a player can fire a shot that misses the target during an initial sliding movement, followed by a corrective movement and a shot that destroys the target. In this example, the initial sliding movement can be associated with two shots. Some embodiments associate the movement with the corresponding target based on which shots are associated with the movement and which one or more targets are closest to the associated shots.

[0061] In some embodiments, performance metrics can include the proportion of movements that have shots not associated with the movement (e.g., if there are no shots between the start of one movement and the start of the next movement), which reflects the need for a player to make multiple movements to destroy any given target. Performance metrics can include the proportion of targets that a player hits with N movements (where N is a positive integer).

[0062] Movement kinematics

[0063] In some embodiments, performance metrics can include movement kinematics based on, for example, player input or parsed input. Movement kinematics can be measured by fitting a parametric function, curve, or line to the time series of each player's movement or to the parsed input obtained using an input device: for example, a function, curve, or line can be fit to the movement measurements, and the parameters that define the function, curve, or line can be used as performance metrics. Performance metrics can be based on the parameters that define the function, curve, or line. Some embodiments measure movement amplitude, speed, accuracy, precision, reaction time as metrics. For example, the player orientation during each time period labeled "in motion" (e.g., through a movement parsing process) can be fit, for example, according to Example Equations 1 to 3, with an S-shaped function:

[0064]

[0065] x(t) = f(t; p 1 , p 3 , p 4 ) (2)

[0066] y(t) = f(t; p2 , p 3 , p 4 ) (3)

[0067] Among them, Equation 1 defines the S-shaped function. In this function, c, b, and a are the parameters of the S-shaped function, which respectively determine the midpoint (which indicates the timing including lift off), the slope (which indicates the speed), and the amplitude (which indicates the accuracy). The value of x(t) in Example Equation 2 represents a model of the horizontal component (rotation around the y-axis) of the movement trajectory for each time sample, where p1 becomes a, p3 becomes b, and p4 becomes c. The value of y(t) in Equation 3 represents a model of the vertical component (rotation around the x-axis) of the movement trajectory for each time sample. Figure 4 Shows an example of a movement trajectory and model fitting according to some embodiments. Example illustrations of the model components x(t) and y(t) are shown in Figure 4 . Figure 4 The top row of [reference] shows two example movements in centimeters for calibrating the transformation using an input device (a mouse in this example). Figure 4 The bottom row of [reference] shows the same two example movements, scaled to normalized units such that 1 corresponds to the target position. Figure 4 The left column of [reference] shows an example of a fast movement and an accurate hit. The shot (vertical dashed line) occurs after the movement ends and the movement lands on the target position (horizontal dotted line). Figure 4 The right column of [reference] shows an example of a slide. The shot (vertical dashed line) occurs in the middle of the movement, and the movement lands far beyond the target position (horizontal dashed line). In this figure, the time series of the x- and y-mouse positions from each test are normalized, for example, divided by the x-component and y-component of the target position respectively. The values of the parameters (p 1 , p 2 , p 3 , p 4 ) are fitted to each individual movement trajectory. The time series of the player's mouse position is parsed into epochs corresponding to the period of the movement, the period before the movement, and the period after the movement. Figure 4 The circles in [reference] represent samples of the player's movement trajectory. Figure 4 The triangles and inverted triangles in [reference] represent samples of the player's mouse position before and after the movement. Figure 4 The curves in [reference] represent the model of the movement trajectory as represented by Equations 1, 2, and 3, with the best fit values of the parameters. In Equations 2 and 3, the x-component and y-component of the movement are fitted using the shared parameters p 3 and p 4 . In other embodiments, the x-component and y-component are independently fitted without shared parameters, such that the midpoints and speeds of the x-component and y-component can be different from each other.

[0068] In some embodiments, speed, accuracy, precision, and reaction time are then calculated based on or according to the best-fit parameter values, such as:

[0069] 1. Speed (e.g., centimeters per second or degrees per second): The peak speed at the midpoint of the movement (e.g., the movement of a user-controlled game entity such as a weapon, aiming point, or crosshair).

[0070] 2. Precision (e.g., percent distance of the landing point of the movement to the target): It can be described as the distance between the landing point of the movement and the center of the target (e.g., spatial error). For example, if a 20-degree movement is required, but the player moves 18 degrees, then the precision is -10% (10% less than the target), and if the player moves 22 degrees, then the precision is +10% (10% more than the target). The distance can be measured in, for example, degrees, pixels, virtual distances meaningful in the virtual environment of the game, the actual distance of the input controller relative to the distance of the controller corresponding to the target (e.g., the position of the mouse on the mouse pad), or other metrics.

[0071] 3. Reaction time (e.g., seconds): The time interval between the appearance of the target and the start of the movement (e.g., when the movement reaches 5% of its end point).

[0072] For example, the movement speed and precision of each parsed movement can be quantified according to, for example, Example Equations 4 to 10:

[0073]

[0074]

[0075] e = (p 1 - x t , p 2 - y t ) (9)

[0076]

[0077] The position of the player at time t (e.g., the position of the crosshair) is represented as x t and y t . The function f’(t) is the derivative of the sigmoid function (Equation 6). The value of a m represents the magnitude of the movement (Equation 7), and a tThe value represents the distance to the target position (Equation 8). Vector e represents the movement error (Equation 9), and vector u represents the unit vector in the direction of the target position (Equation 10). In some embodiments, the accuracy is calibrated to have units of centimeters and / or the movement speed is calibrated to have units of centimeters per second. In some embodiments, the accuracy is calibrated to have units of degrees (e.g., the rotation angle in the virtual environment of the game) and / or the movement speed is calibrated to have units of degrees per second.

[0078] Other units and other ways of calculating speed, accuracy, and reaction time can be used. Speed, accuracy, and reaction time can be measured based on multiple movements, e.g., by calculating the average or median of multiple movements.

[0079] Movement precision can be calculated as 1 divided by movement variability. Movement variability can be calculated as the standard deviation (across multiple movements) of the movement landing points. Movement variability can alternatively be calculated as the median (across multiple movements) of the absolute values of the spatial error (e.g., the distance between the movement landing point and the center of the target). Variability can be expressed in units of % distance, and precision can be expressed in units of 1 / % distance. Other measures of variability can be used instead of the standard deviation and median absolute error, and variability can be expressed in units other than percentage (e.g., angle, centimeter).

[0080] Error, precision, and variability

[0081] Shooting performance or movement kinematics can be used to determine precision, variability, or other metrics. In some embodiments, multiple metrics (typically errors) of shooting or movement kinematics (usually for a specific target type) can be used to calculate precision or variability. For example, multiple error metrics of an indicator (e.g., shooting error or movement error) can be taken; variability can be calculated for that error; and precision can be determined as 1 / variability (so variability can be 1 / precision). In some embodiments, the error can be the absolute value of the error, e.g., the absolute value of the distance between the target center and the movement or shooting landing point, or other errors. In one embodiment, precision (e.g., shooting precision, movement precision) can be calculated as 1 / variability (1 divided by variability) or a metric based on 1 / variability. Those skilled in the art recognize that there are multiple alternative ways to calculate shooting or movement variability. Some embodiments calculate shooting variability as the standard deviation of the shooting positions or movement landing points, or the standard deviation of the accuracy of multiple movements or shootings. Some embodiments alternatively calculate the median absolute deviation (MAD) of a metric such as movement or shooting accuracy, or the median of the absolute differences relative to that median. MAD is a robust measure of variability and can minimize the impact of outliers.

[0082] Precision or other measurements can be based on multiple shots, movements, or tests of a single player, or multiple shots, movements, or tests of multiple players, e.g., in order to compare a newly added player to multiple other players in a database, or to compare a player to the player's previous performance. A database of speed, precision or variability, accuracy or error, reaction time, and other performance metrics can be created for players (e.g., past players) for comparison with new or future players.

[0083] Various measures of variability can be used in place of the standard deviation or MAD. Some embodiments calculate the square root of the median squared difference based on, for example, the median of shot accuracy or movement landing accuracy, the median of the mean squared movement error, the square root of the median of the squared deviations, or the median of other player metrics. Embodiments calculate shot variability based on shot error, e.g., the distance between each shot location and the center of the target (as opposed to the mean or median of the shot locations). Embodiments can calculate the mean of the movement or shot errors of multiple shots or movements, the median of the errors, the square root of the mean, or the square root of the median, the square root of the mean squared error, or the square root of the median of the mean squared error. Embodiments can calculate movement variability based on movement error, e.g., the distance between the landing point of each movement and the center of the target (as opposed to the mean or median of the movement landing points). Embodiments can calculate the mean of the movement errors of multiple movements, the median of the movement errors, the square root of the mean of the mean squared movement errors, the square root of the median of the mean squared movement errors.

[0084] Slipperiness

[0085] In some embodiments, the performance metric can include slipperiness based on, e.g., player input or parsed input. To characterize the degree to which each ballistic movement or parsed movement resembles a slide (shooting while moving) relative to a quick movement and precise hit (decelerating and stopping before firing) movement, some embodiments compare the time of each shot to the time of the midpoint of the corresponding ballistic movement. Ideal slipperiness corresponds to firing a shot at the midpoint of the movement (e.g., the time point of maximum speed). Ideal quick movement and precise hit corresponds to firing a shot only after the end of the movement, after the midpoint of the movement. Some embodiments calculate slipperiness as the ratio of the shot time to the time of the midpoint of the movement (e.g., p4 in equations 2 and 3 in one embodiment) divided by 2. This results in a "slipperiness" value of 0.5 (example arbitrary units) for ideal slipperiness, and a "slipperiness" value greater than or equal to 1 for ideal quick movement and precise hit. For such embodiments, e.g., if there is no shot between the start of one movement and the start of the next movement, there is no slipperiness value calculated for that movement without an associated shot. Slipperiness is a granular measure of shooting speed because it is related to the movement trajectory, with lower slipperiness indicating that the firing of the shot occurs at an earlier position in the trajectory.

[0086] In some embodiments, a performance metric can include a proportion of moves that do not have a shot associated with the move. A lack of a slipperiness value can indicate that a move is not associated with a shot. Additionally, in a test within a particular context or task, the number of moves without a slipperiness value can reflect a player's need to make multiple moves to destroy any given target.

[0087] Shooting performance

[0088] Embodiments can measure the speed, accuracy, or precision of shooting performance; in different embodiments, other values based on shooting performance can be used. Some embodiments operate or calculate speed and precision based on the first shot fired at each target. For example, speed can be calculated or be a metric based on: 1) for each target, separately determining the elapsed time between when the target spawns or appears and the first shot fired (e.g., an input received from a game controller to fire a shot); 2) calculating the mean or median elapsed time of the spawn-shot times for multiple targets; and 3) calculating 1 divided by the mean or median elapsed time. Similarly, precision can be calculated by: 1) calculating the shot error of the first shot fired at each target (e.g., the distance between the landing point of the shot in the game and the center of the target); 2) calculating the mean or median of the absolute values of the shot errors for multiple targets; and 3) calculating 1 divided by the mean or median shot error. Other methods of calculating shooting speed, precision, or variability can be used for multiple shots.

[0089] Distance can be measured while a shot is being taken. Taking a shot can include a player providing an input to an input device such as a mouse or game controller indicating that the player wants to fire a virtual weapon (e.g., a rifle) at a target displayed in the game. The distance between the landing point of the virtual shot in the game (optionally taking into account bullet drop due to simulated gravity) and the center of the target can be measured.

[0090] Figure 5 An example of shooting performance for multiple targets is shown in terms of the time between target spawn and the first shot fired and the shot variability of the first shot fired, according to some embodiments. In Figure 5 , the Y-axis represents speed, and the X-axis represents variability (left graph) and (right graph) precision, respectively. In Figure 5 , shooting speed and shooting variability are analyzed, but the methods described with reference to Figure 5 can be used for other performance metrics, such as movement kinematic performance metrics. Figure 5 An example of a speed-accuracy tradeoff curve for three players (other numbers of players can be used) is illustrated, and Figure 5The dashed lines on the right illustrate examples of axes for skills such as quick movement. In some embodiments, the intersection of each curve with the diagonal (which can be a performance scale) indicates a quick movement skill performance metric. In Figure 5 the example of Figure 5 , the player represented by the diamond drawing symbol has the best quick movement skill, while the player represented by the circular drawing symbol has the worst skill. In other embodiments, there are alternatives for defining the skill axis, and task skills other than quick movement can be used.

[0091] Data Post-Processing

[0092] Data post-processing can be performed, for example, after obtaining data from real players, entities, games or gamified tasks or computer games, and, for example, before using the data to evaluate entities for cheat detection, for input into a database or for performance metrics, movement acuity, or skills. Some embodiments apply one or more steps of post-processing to movement kinematic metrics (e.g., speed, precision, accuracy, reaction time, and slipperiness) or shooting performance metrics to remove outliers and / or decorrelate these values. Some embodiments apply an upper band threshold and a lower band threshold to remove outliers from movement kinematic metrics (e.g., assumed to fail in test parsing or S-shaped fitting). For example, the lower band threshold can be 0, and the upper band threshold can be the 95th percentile. In some embodiments, if the S-shaped fitting is poor, e.g., the r-squared (coefficient of determination) value is less than 0.5, the movement is trimmed (e.g., ignored from further analysis).

[0093] In some embodiments, each accuracy value is multiplied by 100 to convert from a ratio to a percentage of the distance to the target. Some embodiments subtract 100 from each accuracy value so that under-movements (movements that end short of the target) have negative values, while over-movements (movements that end beyond the target) have positive values.

[0094] Some embodiments calculate the z-score (a statistical measure of the distance relative to the mean or average) for each movement kinematic metric of a player. Similar processing can be used for other player data, such as shooting performance metrics.

[0095] Embodiments may use residual correlation after z - scoring a player's locomotion kinematic performance metric or other performance metric and regressing out mouse sensitivity from the locomotion kinematic performance metric or other performance metric. Mouse sensitivity may be related to locomotion reaction time, locomotion speed, slipperiness, and other performance metrics. The correlation may be removed through a regression process. Embodiments may use correlation for primary locomotion and corrective locomotion before and after z - scoring and regressing mouse sensitivity, respectively.

[0096] Example regression operations sequentially include equations 11 through 13. First, the pseudo - inverse of the mouse sensitivity array a # may be calculated and multiplied by the matrix Y of z - scored kinematic metrics to produce an estimate

[0097]

[0098] Subsequently, embodiments may calculate an estimate by multiplying by the z - scored mouse sensitivity array A to produce an estimate

[0099]

[0100] where subtracting Y^ from y gives the residual (E) of the locomotion performance matrix:

[0101]

[0102] Density estimation

[0103] (For example, based on the input) a set or database of real human performance metrics can have a statistical distribution or probability density function computed for it; cheating detection can involve comparing a new set of one or more inputs (usually the same metrics or set of metrics as those used for the distribution) or metrics based on the inputs to the distribution or function to produce an output of the probability of the newly added inputs. Comparing one or more inputs from an entity to data related to inputs from previous human players can include potentially comparing the performance or other metrics based on the entity's inputs to metrics based on inputs from previous human players in ways such as comparing the statistical distribution of the metrics from the entity to the statistical distribution of the database of human players, or comparing the performance metrics of the entity to the statistical distribution based on such a database. For example, multiple samples of a firing speed metric all based on real human inputs can be used to create a probability distribution that measures the probability of observing a particular value for a newly added firing speed metric (e.g., the probability that the firing speed metric was generated by a human player (e.g., without assistance from a robot)). The resulting probability distribution can describe the probability that any point in the distribution belongs to the distribution or is observed in the distribution or the commonality that any point in the distribution has, which corresponds to the likelihood that such a point is a real human.

[0104] Various methods can be used to compare newly added inputs to a database of past real human inputs or performance metrics. For example, a probability density can be created such that for each instance or data item of past performance metrics, a probability can be created that represents the probability that a given newly added data item will be equal to or within a given range of that instance.

[0105] One embodiment can create a histogram from each data point of past human performance metrics and normalize each bar, count, or bin such that all bars, counts, or bins sum to 1. A newly added metric based on the input can be assigned to a bin, and the probability of that metric on a 0 to 1 scale is the magnitude or count of that bin.

[0106] Another embodiment can use a Gaussian distribution or another method such as machine learning to approximate the probability density using a set of inputs of past human performance metrics. In cases where each past human performance metric is multivariate (e.g., each point of the sample is a vector of different metrics), a kernel density function can be used.

[0107] Some embodiments approximate the multivariate distribution of performance metrics as a Gaussian (e.g., normal) distribution. The mean and covariance of the distribution can be estimated from the information collected, such as a database of human performance metrics. Those skilled in the art recognize that a variety of robust methods can be used to estimate the mean and covariance. For example, the median can be used as a robust estimate of the mean, and the median absolute deviation can be used as a robust estimate of the standard deviation. Optionally, principal component analysis (e.g., singular value decomposition) can be used to decorrelate or circularize the distribution to simplify subsequent processing steps and optionally reduce the dimensionality of the distribution.

[0108] Some embodiments employ methods of probability density estimation to characterize the multivariate distribution of a set of true human performance metrics. Such density estimation methods are designed to address the challenge of estimating the underlying distribution of a dataset, particularly in cases where the dataset is complex, high-dimensional, or of limited size. Density estimation methods that can be used include kernel density estimation, Gaussian mixture models, and nonparametric Bayesian methods, among others.

[0109] Kernel density estimation can be used, which involves assigning probability density values to each data point in the dataset using a kernel function. The resulting density values are then averaged to obtain a smoothed estimate of the underlying probability density function. The choice of kernel function can depend on the shape of the data distribution and the desired level of smoothing. Example kernel functions that can be used include the Gaussian kernel, the Epanechnikov kernel, and the triangular kernel, among other functions.

[0110] Gaussian mixture models can be used, which approximate the underlying probability density function as a mixture of several Gaussian distributions. The parameters of each Gaussian component, such as the mean and covariance, are estimated from the data. These parameters can be estimated from the data using an optimization algorithm such as the expectation maximization (EM) algorithm.

[0111] Nonparametric Bayesian methods can be used, which estimate the probability density function without making any assumptions about the functional form of the distribution. This method can be useful when the data distribution is unknown or highly variable. An example of a nonparametric Bayesian method is the Dirichlet process mixture model, which models the data distribution as a mixture of a large number of Gaussian distributions. The number of Gaussian components and their parameters, such as the mean and covariance, can be estimated from the data. The mixing proportions of the Gaussian components can be determined by the Dirichlet process, which is a probability distribution over probability distributions. Data points can be assigned to different clusters based on the mixing weights. The number of clusters can be automatically determined from the data.

[0112] Cheating Detection

[0113] Some embodiments detect cheating by comparing one or more inputs (or metrics based on these inputs) from a new entity to be evaluated with data related to inputs (or metrics based on these previous inputs) from previous human players of a computer game; based on this comparison, it can be determined whether the entity is a human player or a cheater. For example, determining whether an entity is human or a cheater can be performed by comparing performance metrics based on the inputs (e.g., by using statistical distributions). This can be done by calculating the probability or likelihood of performance metrics from a new player (which may be, for example, an executed computer program or a robot or a genuine non-cheating human player) relative to a database of previously collected human performance metrics (e.g., the probability density of a validated database of human performance metrics). Various performance metrics can be used, such as those described herein, or other metrics.

[0114] Embodiments can convert one or more inputs of a newly added set (e.g., mouse movement, shooting, or parsed input or parsed movement) into one or more metrics or performance metrics. Some metrics, such as movement speed, movement accuracy, or movement reaction time, can be based on one movement or parsed movement; other metrics such as movement variability or movement precision can be based on multiple movements. For example, each movement or parsed movement can be converted into a metric such as shooting speed (univariate) or multiple metrics such as shooting speed and shooting accuracy (multivariate, e.g., represented as a vector, e.g., an ordered list of numbers). The metrics of each sample can be compared to a distribution (e.g., a statistical distribution) or function of those same metrics or performance metrics (e.g., created based on previous genuine human samples) to produce a probability output. The probability can be that the newly added input falls within the source samples of the distribution, which can be considered the probability that the newly added input is an input of a genuine human: if the newly added metric has a low probability within the set of previous metrics and thus a low likelihood of occurrence, it is less likely to be an input of a human.

[0115] The comparison of the metrics of the newly added sample to the distribution depends on the type of distribution: compared to a histogram, the metric is simply assigned to a rectangular bar, producing a probability; compared to a Gaussian distribution, the metric is input into an appropriate function, producing a probability; a lookup table can be used to compare the new metric or metric vector to a kernel density function.

[0116] The probabilities of multiple (univariate or multivariate) samples from the newly added entity can have their probabilities combined, e.g., multiplied together, to produce the likelihood of multiple samples from the new entity. The resulting probability can be considered the probability that the entity is a genuine human; a low probability indicates that it may be a cheater or a robot.

[0117] A series of probabilities from a new entity can be combined to produce a combined probability. The input from a new player can be a set of inputs, such as a set of one or more input samples. Each input can be processed as described elsewhere herein (e.g., calibrated, parsed, converted to performance metrics, etc.). Given a set of samples X = {x1, x2, ..., xn} of performance metrics for each of a series of targets from a new player, in one embodiment, the probability that these performance metrics correspond to human input can be calculated as the product of the probabilities of observing each sample. For a continuous probability distribution (and for some other distributions, such as a discrete histogram), an example likelihood function for N sample metrics (Xi sampled from 1 to N) of an entity is given by example equation 14:

[0118] L(θ) = P(X|θ) = Πf(xi|θ) (14)

[0119] where L(θ) is the resulting probability; θ is the probability distribution based on known human players; X is the set of sample performance metrics such that xi is the i-th sample; P(X|θ) represents it as the probability of X given θ; f(xi|θ) is the probability density function of the distribution evaluated at the sample metric xi (univariate or multivariate; i = 1 to N, N is the number of sample metrics used for comparison for this player or entity) given the parameter θ that characterizes the probability density (e.g., based on a set or database of true human inputs or performance metrics); Π represents the multiplicative comparison of each resulting probability. For some embodiments, if the likelihood is less than a threshold, the new player is identified as a cheater. The probability that these performance metrics correspond to cheater input can be 1 minus the probability that the metrics correspond to human input.

[0120] For example, the sample or performance metric value can be the reaction time of a new player for each of a plurality of targets. These reaction times can form a set of samples and can be compared to the distribution of human reaction times. Extremely short reaction times (in the tail of the human reaction time distribution) are less likely to be from human input and are more likely to be from an aimbot input. If a large portion of the reaction times are extremely short, then it is almost certain that the new player is cheating.

[0121] It is possible to analyze more than one performance metric at a time. In some embodiments, each sample (e.g., each sample in a plurality of samples of newly added players) is a vector (e.g., an ordered sequence of numbers) of performance metrics corresponding to the same input or parsed movement, and the likelihood is calculated relative to the multivariate distribution of human performance. Samples can include, for example, vectors of performance metrics such as movement amplitude, movement speed, accuracy of movement landing point, movement reaction time, movement variability or precision, slipperiness, shooting time or speed, shooting variability or precision, and the number of corrective movements, or another set of metrics, each of which describes the same movement input. In cases where each sample consists of multiple different types of performance metrics, the similar performance metrics of newly added players from each sample are compared with the similar performance metrics in a database of real human player data. For example, the reaction time from a new player is compared with the reaction time from the real human database.

[0122] The analysis of performance metrics and the comparison with the real human database can be performed separately for each of a plurality of different conditions or environments to identify new players as cheaters. For example, the comparison can be performed separately for different target sizes or target distances.

[0123] Some embodiments detect cheating by calculating the Mahalanobis distance between the performance metrics of a new player (e.g., multiple vectors or sets of performance metrics) and the previously collected sample metrics (e.g., the probability density of a validated database of human performance metrics). The Mahalanobis distance is a measure of the distance between two points in a multi-dimensional space that takes into account the correlations between dimensions. Different from the Euclidean distance that assumes all dimensions are equally important and independent, the Mahalanobis distance takes into account the correlations between dimensions, thus giving a more meaningful representation of distance in a multi-dimensional space. The Mahalanobis distance between a point represented by the vector x and a multivariate probability density function with mean μ and covariance matrix ∑ is defined according to Example Equation 15:

[0124]

[0125] where T represents the transpose operation, and ∑^(-1) is the inverse of the covariance matrix. For some embodiments, if the proportion of Mahalanobis distances (e.g., each distance generated by comparing a set of performance data (e.g., vectors or metrics) from a new player with the database) that exceed a threshold is greater than a threshold, the new player is identified as a cheater. Other distance metrics can be used, such as the Euclidean distance.

[0126] Those skilled in the art will recognize that various other methods or calculations can be used in conjunction with embodiments of the present invention to compare performance metrics from new entities or players with the probability density of a validated database of human performance metrics. Examples of such methods include:

[0127] 1. Kullback-Leibler divergence (KL divergence): KL divergence is a measure of the difference between two probability distributions. KL divergence measures the amount of information lost when one distribution is approximated by another. KL divergence can be used to compare the fit of a data sample to a probability density.

[0128] 2. Wasserstein distance (Earth Mover's Distance): Wasserstein distance is a measure of the distance between two probability distributions based on the amount of "mass" that must be moved to transform one distribution into the other. It provides a way to compare the similarity of data samples to probability densities.

[0129] 3. Bures distance: Bures distance can be used to quantify the dissimilarity between symmetric positive definite matrices, such as covariance matrices. When considering zero-mean multivariate Gaussian distributions, Bures distance provides a way to compare these distributions based on their covariance matrices. Bures distance provides a measure of the work required to "reshape" one pile of ellipsoidal volumes of dirt into another. Singular value decomposition can be used to calculate Bures distance.

[0130] 4. Energy distance: Energy distance is a non-parametric distance used in various fields, including statistics and machine learning. Energy distance considers all possible pairwise differences between data points within and between distributions and calculates the average of the inter-distribution similarity terms and intra-distribution similarity terms. By subtracting the intra-distribution similarity term from the average inter-distribution similarity, energy distance captures the difference between two distributions while considering the location and distribution of the data.

[0131] 5. Anderson-Darling test: The Anderson-Darling test is a statistical test used to evaluate the goodness of fit of a sample to a theoretical distribution. The test statistically measures the difference between the cumulative distribution function of the data sample and the cumulative distribution function.

[0132] 6. Kolmogorov-Smirnov test: The Kolmogorov-Smirnov test is a non-parametric test used to compare the goodness of fit of a sample to a theoretical distribution. The test statistically measures the maximum difference between the empirical cumulative distribution function of the data sample and the cumulative distribution function.

[0133] 7. Chi-squared test: The chi-squared test is a statistical test used to evaluate the goodness of fit of a data sample to a probability density. This test statistically measures the difference between the observed and expected frequencies in a sample given a probability density.

[0134] Some embodiments may combine two or more of these methods or calculations to compare the performance metrics from new players to the probability density of a validated database of human performance metrics. For example, if the difference or distance measured by one of these metrics or other metrics between the performance metric or vector of performance metrics and the performance metric or vector of performance metrics of the database of real players is greater than a threshold, the entity may be considered a cheater.

[0135] Figure 6 is a flowchart of a method according to an embodiment of the present invention. Figure 6 The operations of can be performed using a system such as Figure 8 and Figure 9 but can be used with other devices. Although an exemplary method is depicted in the flowchart of for illustrative purposes, those skilled in the art will understand that, without departing from the scope of the present disclosure, features and operations from this process can be selectively combined with features and operations from alternative embodiments of the present invention. Additionally, although certain features and operations are explicitly included in the flowchart of Figure 6 those skilled in the art will understand that not all of the depicted features and operations are mandatory elements, and different embodiments may omit certain features or operations without departing from the scope of the present disclosure. Thus, embodiments that include combinations of the features and operations recited in Figure 6 are explicitly within the scope of the present disclosure and do not constitute an intermediate generalization of the present disclosure. Figure 6

[0136] In operation 1000, input can be obtained or received from each of a plurality of real human players via an input device such as a mouse or a game controller. Data preprocessing can be performed; for example, for each player, a move can be divided or parsed into separate parsed moves. In some embodiments, input is collected using a specific input device: since the distribution of performance metrics may be different for different input devices, in one embodiment, a database of performance metrics is collected using the same type of input device (e.g., a mouse) across different inputs, and the same type of input device is used to obtain the performance metrics from a new entity. Different devices can be calibrated to provide the same data for the same input, as the distribution of performance metrics may be different for players, e.g., when using mice with different mouse sensitivities. Move parsing can be performed.​

[0137] In operation 1010, performance metrics for each real human player can be calculated or analyzed. For example, a set of one or more metrics can be calculated for each move or parsed move. Embodiments can transform each input (e.g., mouse move or parsed move) into one or more metrics for each move. For example, each move can be transformed into a metric such as move speed (univariate) or multiple metrics such as move speed and move accuracy (multivariate, e.g., represented as a vector, e.g., an ordered list of numbers). In one embodiment, a metric vector can be generated for each parsed move. In this way, embodiments can obtain and analyze a set of past performance metrics from human players (e.g., a validated database). Optionally, post-processing can be performed; for example, outliers can be removed from the metrics. Such a database can include many measurements typically regarding many different real players, with each measurement having multiple associated data points in the database, such as data points for move kinematics, move amplitude, move speed, accuracy of move landing, move reaction time, reaction time between target appearance and move start, and slipperiness. In some embodiments, a metric vector can be generated for each shot, e.g., including shot time or shot speed and shot error. In some embodiments, the metric vector can include both move kinematic performance metrics and shot performance metrics. In some embodiments, the metric vector can be generated from multiple moves and / or shots, e.g., including move variability or precision, shot variability or precision, motion acuity, speed-accuracy trade-off, proportion of targets the player hits, proportion of moves without a shot, number of corrective moves that hit the target, and number of shots that hit the target. Although specific metrics are described in groups, different embodiments can use one or more metrics described in different combinations.

[0138] In operation 1020, statistical or machine learning analysis methods such as density estimation, pattern recognition, and pattern classification can be performed to characterize the multivariate distribution of human performance metrics from multiple human players. A statistical distribution or probability density function can be calculated for the metrics calculated or created in operation 1010.

[0139] In operation 1030, move inputs can be received or obtained from a new entity or player via the same type of input device used for the validated database for obtaining human performance metrics. The entity can be a real human, or can be identified as or logged in as a real human, but in the case of a cheater, a bot can be used to provide input in an attempt to mimic high-performance human input. Data preprocessing such as move parsing and input device calibration can be performed.

[0140] In operation 1040, the movement input of the newly added entity has a calculated performance metric, typically the same as the performance metric analyzed for real players in operation 1010. Optionally, post-processing is performed, such as outlier removal. Data from the newly added player does not need to be from the same game as the data collected from previous human players in operation 1000, but in some embodiments, the data should be from the same type of input device as used by the previous players. For example, human performance measured in certain ways using a mouse is the same for any game (with the same limitations and speed-accuracy trade-offs), but calibration may be required to compensate for different mouse sensitivity settings.

[0141] In operation 1050, the input or metrics of the newly added player can be compared with a database of real human input or past performance metrics. For example, statistical or machine learning analysis methods can be applied to determine whether the performance metrics from the new player are inconsistent with a validated database of human performance metrics. In some embodiments, the probability or likelihood that the performance metrics from the new player appear in the database of past human metrics can be determined, or the distance between the metrics of the new entity and past human metrics can be determined.

[0142] In operation 1060, it can be determined whether the entity is a cheater. For example, if the probability determined in operation 1050 is below a threshold, or the distance between the metrics of the new entity and the metrics of the real human database is above a threshold, the entity can be considered a cheater. In one embodiment, the entity is a human player or a robot providing input, and determining that the entity is a cheater indicates that the entity is a robot.

[0143] In operation 1070, if the entity is not considered a cheater, for example, the player is considered a real human or a real human providing their own input without the assistance of a robot, no corrective action is taken. For example, the player can be allowed to continue playing the game.

[0144] In operation 1080, if the entity is considered a cheater, for example, a player logging in to the system with a specific identity or recognized by the system as a robot, or whose input is provided by a robot, an action can be taken. For example, the process can notify or send an alert to other entities (such as players, game developers, or game platforms) that the analyzed entity (e.g., the player with input provided by a robot) is a cheater; stop the game of the entity; add the username or online name of the entity to the cheater list; remove the entity from the leaderboard or other registry or list of successful players; prohibit the entity from entering the game; freeze the entity's account; and revoke the impact of the entity's actions on other entities (e.g., increase the ranking of other players).

[0145] Other operations or a series of operations may be used.

[0146] Verification

[0147] To verify some aspects of the disclosed embodiments, data from human players of the Aim Lab system were analyzed to characterize the distribution of human performance metrics. These human performance metrics were compared with the performance metrics from non - human aiming robots. The aiming robots did not exhibit the same performance metrics as human players. For example, the aiming robots had reaction times that were unrealistically short, movement speeds that were unrealistically fast, and accuracies that were unrealistically good. Additionally, the aiming robots did not exhibit the typical trade - offs of humans. For example, the aiming robots' movement accuracy was not traded off against the magnitude of movement. Figure 7 The accuracy of cheat detection (e.g., the proportion of new players correctly classified as aiming robots or human players) was plotted against the sample size (e.g., the number of targets presented). With sufficient data (e.g., more than 50 targets), one embodiment could almost perfectly distinguish aiming robots from human players.

[0148] Example device

[0149] In some embodiments, human participants play a FPS task remotely using their own gaming set - up, which may be or include a computer system or a game console including a computer system (e.g., a Microsoft Xbox system or a Sony PlayStation system), as Figure 8 and Figure 9 shown. Figure 8 A computer according to an embodiment of the present invention is depicted. The various embodiments and operations discussed herein may be performed by a computing device such as Figure 8 shown in. For example, the user computer 900 ( Figure 9 )、the server 930、the cheat process 920、the computer game 910 or other processes described herein may be or be performed by a computer such as Figure 8 depicted in. Referring to Figure 8 , the computing device 100 may include a controller or a computer processor 105, which may be, for example, a central processing unit processor (CPU), a chip, or any suitable computing device, an operating system 115, a memory 120, a storage device 130, an input device 135, and an output device 140, such as a computer monitor or a display showing, for example, a computer desktop system.

[0150] The operating system 115 may be or may include code for performing tasks involving coordinating, scheduling, arbitrating, or managing the operations of the computing device 100 (e.g., execution of a scheduler). The memory 120 may be or may include, for example, random access memory (RAM), read-only memory (ROM), flash memory, volatile or non-volatile memory, or other suitable memory units or storage device units. The memory 120 may be or may include multiple different memory units. The memory 120 may store, for example, instructions (e.g., code 125) for performing the methods disclosed herein and / or data such as documents.

[0151] The executable code 125 may be any application, program, process, task, or script. The executable code 125 may be executed by the controller 105, possibly under the control of the operating system 115. For example, the executable code 125 may be one or more applications that perform the methods disclosed herein. One or more processors 105 may be configured to implement embodiments of the present invention by, for example, executing software or code.

[0152] The input device 135 may be or may include a game controller, gyroscope sensor, electroencephalogram (EEG) sensor, electromyogram (EMG) sensor, mouse, game input device, keyboard, touch screen or touchpad, mobile phone, tablet computer (e.g., iPad), or any suitable input device or combination of devices. The output device 140 may include one or more displays, speakers, and / or any other suitable output device or combination of output devices. Any applicable input / output (I / O) device may be connected to the computing device 100. For example, a wired or wireless network interface card (NIC), modem, printer, universal serial bus (USB) device, or external hard drive may be included in the input device 135 and / or the output device 140.

[0153] Embodiments of the present invention may include one or more articles (e.g., the memory 120 or the storage device 130) that encode, include, or store instructions (e.g., computer-executable instructions), such as a computer or processor non-transitory readable medium, or a computer or processor non-transitory storage medium, such as, for example, a memory, disk drive, or USB flash drive, that when executed by a processor or controller perform the methods disclosed herein.

[0154] Figure 9depicts a computer system according to an embodiment of the present invention. The user computer 900 can be a desktop computer, a laptop computer, a personal computer (PC), a cellular phone, a smart phone, or a gaming console or computer (e.g., Xbox, PlayStation, etc.), and can accept user (e.g., gamer) input to a video or computer game 910 via an input device 905 (e.g., a mouse, a Wii controller, an iPad, etc.), and display game output (e.g., Figure 8 the view shown in Figure 1 ) on a computer monitor or display (such as the output device 140 in

[0155] The game 910 can be executed in whole or in part on the computer 900 and / or a remote computer such as the server 930, which can be a computer operated by, for example, a game provider company, a cloud computing facility, etc. The computer 900 can provide outputs such as game displays, and can be connected to other computers such as the server 930 via one or more networks such as the Internet 990. The user computer 900 and / or the server 930 can detect cheating or provide other methods as discussed herein. The user computer 900, the server 930, and other systems can be, for example, computers including components in the system 100 ( Figure 8 ). Although one user computer 900 and one server 930 are shown, other embodiments can use, for example, multiple such computers connected via a network such as the Internet 990.

[0156] The hardware and settings used by different users may vary. Such variations include different hardware (such as a PC, a monitor, a mouse or other input device, and a mouse pad) and settings (such as monitor size, field of view, viewing distance, chair height, or counts per inch (CPI) of the mouse). There can be a wide variety of equipment combinations among players. In some embodiments, mouse acceleration is disabled. In some embodiments, mouse movement is sampled at 120 Hz, but other sampling rates can alternatively be used.

[0157] In some embodiments, the movement trajectory is determined by sampling input positions from an input device and is controlled by a person while the person is playing a game. The input device may control the player's interaction with the virtual environment of the game, including moving or reorienting the player's avatar and / or one or more actions of the player's avatar (e.g., use of a weapon). Examples of input devices (e.g., Figure 8 input device 135 of Figure 8 ) include a keyboard, a mouse, a gaming mouse, a game console, a joystick, an accelerometer, a gyroscope, a pointing device, a motion capture device, a Wii remote, an eye tracker, a computer vision system, or any one of a variety of methods, devices, apparatuses, and systems for sensing, measuring, or estimating human movement to provide an input signal to a computer. For example, the gyroscope in a smart phone or a tablet can be used as an input device for gamification tasks running on these devices. Examples of human movement include hand movement, arm movement, head movement, body movement, and eye movement. Examples of input devices also include brain-computer interface methods, devices, apparatuses, and systems for sensing, measuring, or estimating brain activity. Examples of brain-computer interfaces include, but are not limited to, devices that measure electrophysiological signals (e.g., using EEG, magnetoencephalography (MEG), microelectrodes) and optical signals (e.g., using voltage-sensitive dyes, calcium indicators, intrinsic signals, functional near-infrared spectroscopy imaging, etc.). Examples of brain-computer interfaces also include other neuroimaging techniques (e.g., functional magnetic resonance imaging). Examples of input devices also include methods, devices, apparatuses, or systems for sensing, measuring, or estimating physiological data. Physiological data includes, but is not limited to, EEG, EKG, EMG, EOG, pupil size, and biomechanical data related to breathing and / or respiration. Those skilled in the art recognize that any such input device or any combination of such input devices can be used. It should also be recognized that other methods, devices, apparatuses, or systems for sensing, measuring, or estimating human movement or physiological activity can be substituted, including those that have not yet been put into practice.

[0158] Embodiments can improve cheating detection or player identification techniques, for example, by detection methods for detecting advanced or subtle cheating, such as those using robots or computer programs to simulate human players.

[0159] Unless explicitly specified, the method embodiments described herein are not limited to a particular order or sequence. In addition, all formulas described herein are only intended as examples, and other or different formulas can be used. Additionally, some of the method embodiments described or their elements can occur or be executed at the same point in time.

[0160] Although certain features of the present invention have been illustrated and described herein, many modifications, alternatives, variations, and equivalents will occur to those skilled in the art. Accordingly, it is to be understood that the appended claims are intended to cover all such modifications and changes that fall within the true spirit of the present invention.

[0161] Various embodiments have been presented. Of course, each of these embodiments may include features of the other embodiments presented, and embodiments not specifically described may include various features described herein. Additionally, while certain operations and features are described with respect to certain embodiments, not all elements and operations are mandatory, and different embodiments may omit certain features.

Claims

1. A method for detecting cheating in a computer game, the method comprising: receiving, from an entity, one or more inputs to a computer game; comparing the one or more inputs from the entity with data related to inputs to the computer game from multiple human players; and based on the comparison, determining whether the entity is a human player or a cheater.

2. The method according to claim 1, wherein, the comparison includes comparing a performance metric based on the one or more inputs from the entity with a performance metric based on data related to inputs to the computer game from multiple human players.

3. The method according to claim 2, wherein, the performance metric is selected from the group consisting of: movement kinematics, movement amplitude, movement speed, accuracy of movement landing point, movement reaction time, reaction time between target appearance and movement start, slipperiness, movement variability, movement precision, movement acuity, speed-accuracy trade-off, proportion of movement without shooting, number of corrective movements to hit a target, shooting performance, shooting time, shooting speed, shooting error, shooting variability, shooting precision, number of shots hitting a target, and proportion of targets hit by the player.

4. The method according to any one of the preceding claims, wherein, the comparison includes comparing the statistical distribution of the performance metric of one or more inputs from the entity with the statistical distribution of the performance metric of one or more inputs from the multiple human players.

5. The method according to any one of the preceding claims, wherein, the one or more inputs are provided by an input device selected from the group consisting of: a computer mouse, a keyboard, a mobile computing device, a mobile phone, a mobile device, and a game controller.

6. The method according to any one of the preceding claims, comprising: if it is determined that the entity is a cheater, taking an action from the group consisting of: notifying another entity that the entity is a cheater; stopping the entity from playing the game; adding the username of the entity to a cheater list; removing the entity from a leaderboard; banning the entity from entering the game; freezing the entity's account; and undoing the impact of the entity's actions on other entities.

7. The method according to any one of the preceding claims, wherein, the entity is a human player or a robot providing an input, and if it is determined that the entity is a cheater, indicating that the entity is a robot.

8. A system for detecting cheating in a computer game, the system comprising: a memory; and a processor for: receiving, from an entity, one or more inputs to a computer game; comparing the one or more inputs from the entity with data related to inputs to the computer game from multiple human players; and based on the comparison, determining whether the entity is a human player or a cheater.

9. The system according to claim 8, wherein, The comparison includes comparing a performance metric based on one or more inputs from the entity with a performance metric based on data related to inputs from multiple human players to a computer game.

10. The system according to claim 9, wherein, the performance metric is selected from the group consisting of: movement kinematics, movement amplitude, movement speed, accuracy of movement landing point, movement reaction time, reaction time between target appearance and movement start, slipperiness, movement variability, movement precision, movement acuity, speed-accuracy trade-off, proportion of movements without shooting, number of corrective movements hitting the target, shooting performance, shooting time, shooting speed, shooting error, shooting variability, shooting precision, number of shots hitting the target, and proportion of targets hit by the player.

11. The system according to any one of claims 8 or 10, wherein, the comparison includes comparing the statistical distribution of the performance metric of one or more inputs from the entity with the statistical distribution of the performance metric of one or more inputs from the multiple human players.

12. The system according to any one of claims 8 to 11, wherein, the one or more inputs are provided by an input device selected from the group consisting of: a computer mouse, a keyboard, a mobile computing device, a mobile phone, a mobile device, and a game controller.

13. The system according to any one of claims 8 to 12, wherein, the processor is configured to: if it is determined that the entity is a cheater, take an action from the group consisting of: notify another entity that the entity is a cheater; stop the entity from playing the game; add the username of the entity to a cheater list; remove the entity from the leaderboard; prohibit the entity from entering the game; freeze the entity's account; and revoke the impact of the entity's actions on other entities.

14. The system according to any one of claims 8 to 13, wherein, the entity is a human player or a robot providing an input, and determining that the entity is a cheater indicates that the entity is a robot.

15. A method for detecting cheating in a computer game, the method comprises: receiving an input to a computer game from an entity; comparing a performance metric based on the input from the entity with a database of inputs from human players; and based on the comparison, determining whether the entity is a human player or a player using a robot.

16. The method according to claim 15, wherein, the performance metric is selected from the group consisting of: movement kinematics, movement amplitude, movement speed, accuracy of movement landing point, movement reaction time, reaction time between target appearance and movement start, slipperiness, movement variability, movement precision, movement acuity, speed-accuracy trade-off, proportion of movements without shooting, number of corrective movements hitting the target, shooting performance, shooting time, shooting speed, shooting error, shooting variability, shooting precision, number of shots hitting the target, and proportion of targets hit by the player.

17. The method according to any one of claims 15 to 16, wherein, the comparison includes comparing statistical distributions.

18. The method according to any one of claims 15 to 17, wherein, the input is provided by an input device selected from the group consisting of: a computer mouse, a keyboard, a mobile computing device, a mobile phone, a mobile device, and a game controller.

19. The method according to any one of claims 15 to 18, comprising: if it is determined that the entity is a player using a robot, taking an action from the group consisting of: notifying another entity that the entity is a cheater; stopping the entity from playing the game; adding the entity's username to a cheater list; removing the entity from a leaderboard; prohibiting the entity from entering the game; freezing the entity's account; and undoing the effect of the entity's actions on another entity.

20. The method according to any one of claims 15 to 19, wherein, the comparison includes a comparison with a probability distribution.

Images