Systems and methods for predicting dynamics of sports games based on transformations and linear operators

Abstract

A non-transitory, processor-readable medium stores instructions that, when executed by a processor, cause the processor to receive (1) actor position data associated with a set of actors, (2) object position data associated with an object of interest, and (3) an indication of a designated actor from the set of actors. Designated actor position data is identified within the actor position data based on the indication of the designated actor, and a transformation is applied to the actor position data, the object position data, and the designated actor position data, to produce first dynamics data associated with a first time. A linear operator is applied to the first dynamics data to produce second dynamics data associated with a second time that is after the first time. The instructions also cause the processor to cause display of a representation of the set of actors and a representation of the object of interest.

Description

Attorney Docket No.: KHSL-001 / 01WO 354667-2003 SYSTEMS AND METHODS FOR PREDICTING DYNAMICS OF SPORTS GAMES BASED ON TRANSFORMATIONS AND LINEAR OPERATORS CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 701,225, filed September 30, 2024, and titled “SYSTEMS AND METHODS FOR PREDICTING DYNAMICS OF SPORTS GAMES BASED ON TRANSFORMATIONS AND LINEAR OPERATORS,” which is incorporated herein by reference in its entirety. FIELD

[0002] One or more embodiments described herein relate to systems and computerized methods for predicting dynamics of sports actors and / or objects of interest. BACKGROUND

[0003] Some known methods of analytics (e.g., sports analytics) apply statistical techniques to forecast performance. However, some known methods are more useful in certain domains (e.g., baseball) than other domains (e.g., basketball, soccer, American football, etc.). A need exists, therefore, for systems and methods configured to predict dynamics of actors (e.g., sports participants) and objects of interests (e.g., a gameplay object, such as a ball, a puck, etc.). SUMMARY

[0004] According to an embodiment, a non-transitory, processor-readable medium stores instructions that, when executed by a processor, cause the processor to receive (1) actor position data associated with a set of actors, (2) object position data associated with an object of interest, and (3) an indication of a designated actor. Designated actor position data is identified within the actor position data based on the indication of the designated actor. The instructions also cause the processor to apply a transformation to the actor position data, the object position data, and the designated actor position data, to produce first dynamics data associated with a first time, the transformation being associated with a linear dynamics domain. A linear operator is applied to theAttorney Docket No.: KHSL-001 / 01WO 354667-2003 first dynamics data to produce second dynamics data associated with a second time that is after the first time. The instructions also cause the processor to cause display of a representation of the set of actors and a representation of the object of interest, based on the second dynamics data.

[0005] According to an embodiment, a method includes receiving, at a processor, (1) actor position data associated with a set of actors in a sports game and (2) object position data associated with an object of interest in the sports game. The method further includes performing, via the processor, dynamic mode decomposition jointly on the actor position data and the object position data, to produce first dynamics data associated with a first time. A density-over-time map is generated, via the processor, for the set of actors and the object of interest, based on historical data associated with at least one of the set of actors or the sports game. The method also includes generating, via the processor, a linear operator based on the density-over-time map, and applying, via the processor, the linear operator to the first dynamics data to predict second dynamics data associated with a second time that is after the first time. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] FIG. 1 shows a system block diagram of a dynamics prediction system, according to an embodiment.

[0007] FIG. 2 shows a system block diagram of a compute device included in a dynamics prediction system, according to an embodiment.

[0008] FIG. 3 shows a system block diagram of components included in a dynamics prediction system, according to an embodiment.

[0009] FIG. 4 shows a system block diagram of components included in a dynamics prediction system, according to an embodiment.

[0010] FIG.5 shows a flow diagram illustrating a method for causing display of representations of at least one actor and an object of interest based on dynamics data, according to an embodiment.

[0011] FIG. 6 shows a flow diagram illustrating a method for generating and applying a linear operator to first dynamics data to predict second dynamics data, according to an embodiment.Attorney Docket No.: KHSL-001 / 01WO 354667-2003

[0012] FIG. 7 shows a visualization of prediction data for an American football game and generated by a dynamics prediction system, according to an embodiment.

[0013] FIG. 8 shows a visualization of prediction data for a soccer game and generated by a dynamics prediction system, according to an embodiment.

[0014] FIG. 9 shows a visualization of a plurality of ball positions predicted by a dynamics prediction system, according to an embodiment.

[0015] FIG.10 shows a visualization of prediction data over time for an offensive maneuver in a soccer game, according to an embodiment. DETAILED DESCRIPTION

[0016] A scenario (e.g., a game) can include a sports and / or electronic sports (e.g., esports) activity having defined rules and a defined goal that determines wins, draws, and losses. A game can be played by one or more actors against one or more opponents. An actor can be a human player, a human-controlled player (e.g., a videogame character), and / or a bot (e.g., an artificial intelligence (AI) entity). A sports activity can be associated with, for example, American football, European football (e.g., soccer), basketball, hockey, auto racing (e.g., Formula 1), and / or the like. An electronic sports activity can be associated with, for example, fantasy football, fantasy Premier League, e-football, e-soccer, and / or the like.

[0017] At least some systems and methods described herein can collect data from a physical game, a computer game, and / or a simulation of a physical and / or computer game. The data can be provided as input to predict the outcome of the game for a fixed set of players or a set of players that includes replacement of a single or multiple players. At least some systems and methods described herein can be further configured to instruct a sports player or team to play the game in an improved manner for a given opponent. At least some systems and methods described herein can indicate tactical and / or strategic decisions to game personnel, such as coaches and players, to cause a change in outcome of the game. These decisions can include, among others, play calls and / or player substitutions.Attorney Docket No.: KHSL-001 / 01WO 354667-2003

[0018] System inputs can be connected to system outputs via a structured model, such as, forexample, the following linear model:x = + ,= , where is the state of the system and is a linear vector of observables.

[0019] In static machine learning problems, there can also be “inputs” and “outputs,” such asand and / or more complicated input and / or output spaces. Inputs and outputs can beconnected by a map learned from a measured subset of input-output pairs ( , ),= 1, ..., .

[0020] Let ( ) be the image of the map , and ( ) its domain. Provided ( ) ( ),can be considered a dynamical system, since ( ) can be defined for any ( ). In thiscase, the data pairs ( , ) can be obtained as successive points along the trajectory of :

[0021] Dynamic learning can provide an advantage in that data can be sampled along a trajectoryadvancing in time. To illustrate, assume, for example, a discrete-time dynamical system has an-dimensional linear representation ( , ), such thatTaking samples of along a trajectory( ), ( ), ..., ( ) and obtaining a sequence of snapshots 1 = ( ), ( ) = 2, ..., ( )= , the following results:

[0022] ( ) = ( 1), = 1, ..., . The following data matrices can be formed:

[0023] Note that each row of data matrices , can be an evaluation of the function on thetrajectory of starting at . Setting , can be the companion matrixAttorney Docket No.: KHSL-001 / 01WO 354667-2003The solution of this equation, provided= and is nonsingular, is = 1 can then be considered an approximationto the Koopman operator acting on the space of functions on the set of points = ( ,, ..., ).

[0024] Applied to a use case such as, for example, sports, the Koopman operator approximationcan represent learned dynamics of a game and can be efficiently estimated using previouslyobserved game data. Further, because the Koopman operator is linear, the operator can becomputationally efficient and can be used to efficiently simulate large numbers of game instances and / or improve (e.g., optimize) game scenarios according to a specified objective function. Asdescribed further herein, at least some systems and methods can observe game dynamics, estimatedynamical models, and / or analyze, predict and / or control game situations. Game observations can,in some instances, be coarse, including, for example, a single position measurement of each playeron the field for each instance in time. In some instances, game observations can be detailed, including, for example, full three-dimensional captures of aspects of the game and / or players.

[0025] FIG. 1 shows a system block diagram of a dynamics prediction system 100, according toan embodiment. The dynamics prediction system 100 includes a compute device 110, a computedevice 120, a sensor(s) 130, and a network N1. The dynamics prediction system 100 can includealternative configurations, and various steps and / or functions of the processes described below canbe shared among the various devices of the dynamics prediction system 100 or can be assigned tospecific devices (e.g., the compute device 110, the compute device 120, and / or the like). For example, in some configurations, a user can provide inputs directly to the compute device 120 rather than via the compute device 110, as described herein.

[0026] In some embodiments, the compute device 110 and / or the compute device 120 can includeany suitable hardware-based computing devices and / or multimedia devices, such as, for example, a server, a desktop compute device, a smartphone, a tablet, a wearable device, a laptop and / or thelike. In some implementations, the compute device 110 and / or the compute device 120 can beimplemented at an edge node or other remote computing facility. In some implementations, eachAttorney Docket No.: KHSL-001 / 01WO 354667-2003 of the compute device 110 and / or compute device 120 can be a data center or other control facility configured to run and / or execute a distributed computing system and can communicate with other compute devices.

[0027] The compute device 110 can implement a user interface 112. The user interface 112 can be a graphical user interface (GUI) configured to receive user-defined data and / or display a representation of a scenario prediction that is generated by a dynamics prediction application 122 (described further herein). The user interface 112 can be implemented via software (e.g., that is executed via a processor that is functionally and / or structurally similar to the processor 220 of FIG. 2, described herein) and / or hardware.

[0028] The compute device 120 can be, for example, a server(s) configured to execute (e.g., via a processor structurally and / or functionally similar to the processor 220 of FIG.2, described herein) the dynamics prediction application 122, which can be stored at a memory functionally and / or structurally similar to the memory 210 of FIG.2. The dynamics prediction application 122 can be configured to predict dynamics of actors (e.g., sports participants) and / or objects of interests (e.g., a gameplay object, such as a ball, a puck, etc.), as described further herein.

[0029] The sensor(s) 130 can include, for example, a position sensor (e.g., a radio receiver associated with a satellite navigation system, such as the Global Positioning System (GPS)), an accelerometer, an inertial measurement unit (IMU), and / or the like. The sensor(s) 130 can be associated with an actor(s) and / or an object of interest. For example, the sensor(s) 130 can be included in a wearable (e.g., a smartwatch, wristband, integrated sensor in equipment such as shoulder pads, etc.) and worn by an actor(s). Alternatively or in addition, the sensor(s) 130 can be coupled to and / or embedded in an object of interest, such as a ball, puck, shuttlecock, etc.

[0030] The compute device 110, the compute device 120, and / or the sensor(s) 130 can be networked and / or communicatively coupled to at least one of the remaining compute device 110, compute device 120, and / or sensor(s) 130, via the network N1, directly using wired connections and / or wireless connections. The network N1 can include various configurations and protocols, including, for example, short range communication protocols, Bluetooth®, Bluetooth® LE, the Internet, World Wide Web, intranets, virtual private networks, wide area networks, local networks, private networks using communication protocols proprietary to one or more companies, Ethernet,Attorney Docket No.: KHSL-001 / 01WO 354667-2003 WiFi and / or Hypertext Transfer Protocol (HTTP), cellular data networks, satellite networks, free space optical networks and / or various combinations of the foregoing. Such communication can be facilitated by any device capable of transmitting data to and from other compute devices, such as a modem(s) and / or a wireless interface(s).

[0031] In some implementations, although not shown in FIG.1, the dynamics prediction system 100 can include multiple compute devices 110 and / or compute devices 120. For example, in some implementations, the dynamics prediction system 100 can include a plurality of compute devices 110, where each compute device 110 can be associated with a different user from a plurality of users. In some implementations, a plurality of compute devices 110 can be associated with a single user, where each compute device 110 can be associated with, for example, a different input modality (e.g., text input, audio input, image input, video input, location input, etc.).

[0032] FIG. 2 shows a system block diagram of a compute device included in a dynamics prediction system, according to an embodiment. The compute device 201 can be structurally and / or functionally similar to, for example, the compute device 120 of the dynamics prediction system 100 shown in FIG. 1. The compute device 201 can be a hardware-based computing device, a multimedia device, or a cloud-based device such as, for example, a computer device, a server, a desktop compute device, a laptop, a smartphone, a tablet, a wearable device, a remote computing infrastructure, and / or the like. The compute device 201 includes a memory 210, a processor 220, and a network interface 230 operably coupled to a network N2.

[0033] The processor 220 can be, for example, a hardware-based integrated circuit (IC), or any other suitable processing device configured to run and / or execute a set of instructions or code (e.g., stored in memory 210). For example, the processor 220 can be a general-purpose processor, a central processing unit (CPU), an accelerated processing unit (APU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic array (PLA), a complex programmable logic device (CPLD), a graphics processing unit (GPU), a programmable logic controller (PLC), a remote cluster of one or more processors associated with a cloud-based computing infrastructure and / or the like. The processor 220 is operatively coupled to the memory 210 (described herein). In some embodiments, for example, the processor 220 canAttorney Docket No.: KHSL-001 / 01WO 354667-2003 be coupled to the memory 210 through a system bus (for example, address bus, data bus and / or control bus).

[0034] The memory 210 can be, for example, a random-access memory (RAM), a memory buffer, a hard drive, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), and / or the like. The memory 210 can store, for example, one or more software modules and / or code that can include instructions to cause the processor 220 to perform one or more processes, functions, and / or the like. In some implementations, the memory 210 can be a portable memory (e.g., a flash drive, a portable hard disk, and / or the like) that can be operatively coupled to the processor 220. In some instances, the memory can be remotely operatively coupled with the compute device 201, for example, via the network interface 230. For example, a remote database server can be operatively coupled to the compute device 201.

[0035] The memory 210 can store various instructions associated with processes, models, and / or data, including machine learning models, as described herein. The memory 210 can further include any non-transitory computer-readable storage medium for storing data and / or software that is executable by processor 220, and / or any other medium which may be used to store information that may be accessed by processor 220 to control the operation of the compute device 201. For example, the memory 210 can store data associated with a dynamics prediction application 212. The dynamics prediction application 212 can be functionally and / or structurally similar to dynamics prediction application 122 of FIG.1.

[0036] The dynamics prediction application 212 includes a dynamics estimator 214 (e.g., that is structurally and / or functionally similar to the dynamics estimator 425 of FIG. 4, described in further detail herein), a predictor 216 (e.g., that is structurally and / or functionally similar to the predictor 430 of FIG. 4, described in further detail herein), and / or a visualizer 218 (e.g., that is structurally and / or functionally similar to the visualizer 470 of FIG.4, described in further detail herein).

[0037] The network interface 230 can be configured to connect to the network N2, which can be functionally and / or structurally similar to the network N1 of FIG.1. For example, network N2 can use any of the communication protocols described above with respect to network N1 of FIG.1.Attorney Docket No.: KHSL-001 / 01WO 354667-2003

[0038] In some instances, the compute device 201 can further include a display, an input device, and / or an output interface (not shown in FIG.2). The display can be any display device by which the compute device 201 can output and / or display data (e.g., via a user interface that is structurally and / or functionally similar to the user interface 112 of FIG. 1). The input device can include a mouse, keyboard, touch screen, voice interface, and / or any other hand-held controller or device or interface via which a user may interact with the compute device 201. The output interface can include a bus, port, and / or other interfaces by which the compute device 201 may connect to and / or output data to other devices and / or peripherals.

[0039] FIG.3 shows a system block diagram of components 300 included in a dynamics prediction system, according to an embodiment. The components 300 can implement a method of determining actor (e.g., player) position data based on video data (and / or, although not shown in FIG. 3, sensor data collected by a sensor(s) that is functionally and / or structurally similar to the sensor(s) 130 of FIG.1). The components 300 can be associated with and / or executed by a compute device (e.g., a compute device that is structurally and / or functionally similar to the compute device 201 of FIG. 2 and / or the compute devices 110 and / or 120 of FIG. 1). In some instances, for example, the components 300 can be included in the dynamics prediction application 122 and / or the dynamics prediction application 212, and / or can include software stored in memory 210 and configured to execute via the processor 220 of FIG.2. In some instances, for example, at least a portion of the components 300 can be implemented in hardware. The components 300 include video data 301, a marker detector 305, a marker classifier 310, a line detector 315, a reference selector 320, a homography matrix generator 325, an actor detector 330, an actor classifier 335, and an actor location determinator 340.

[0040] The components 300 can implement a method for automatically (e.g., without human intervention) estimating a location(s) of an agent(s) relative to a defined area (e.g., a field, a court, a rink, a pool, etc.). In some instances, the components 300 can implement the method without camera information (e.g., configuration information, performance information, etc.) and / or camera position(s) being known and / or provided as input to the components 300. Alternatively or in addition, in some instances, the components 300 can implement the method based on known camera information and / or know camera position(s).Attorney Docket No.: KHSL-001 / 01WO 354667-2003

[0041] Video data 301 can include at least one frame from a series of video frames depicting a scenario (e.g., a game). The at least one frame can be captured by a camera and / or rendered by a videogame application. The marker detector 305 can identify known markers on a field depicted in the at least one frame, by using a color mask consistent with field markers on the field to define field marker regions of interest (ROIs). ROIs can be classified by the marker classifier 310 to determine ROIs that include field markers that are unique to a location of the playing field. These markers can be unique patterns, shapes, alpha numeric symbols, etc. For alpha numeric symbols used as unique field markers, the marker classifier 310 can be configured to perform optical character recognition (OCR) by providing the alpha numeric symbols as input to a deep learning network to generate a classification. For example, the marker classifier 310 can perform OCR to detect a specific yard line on an American football field based on a number painted on the field.

[0042] The line detector 315 can be configured to perform, for example, a Hough transform of each frame included in the video data 301 to detect lines (e.g., area boundaries, such as field boundaries, zone boundaries, 3-point lines, etc.). The line detector 315 can detect lines by receiving indications of classified field markers from the marker classifier 310. Alternatively or in addition, the line detector 315 can detect lines within the video data 301 without receiving indications of classified field markers as input from the marker classifier 310. The output (e.g., an indication(s) of a detected line(s)) of the line detector 315 and the output (e.g., the indication(s) of a classified field marker(s)) of the marker classifier 310 can be provided as input to the reference selector 320 to determine at least one reference point (e.g., three or more reference points). A reference point can be, for example, a coordinate, such as a Cartesian coordinate and / or the like. At least three reference points can be provided as input to the homography matrix generator 325 to generate a homography matrix, which can represent a projective transformation between two planes (e.g., between a first plane associated with a camera perspective and a second plane associated with the defined area).

[0043] The actor detector 330 can include a machine learning model (e.g., a general deep learning detection model, such as a You Only Look Once (YOLO) model) that is trained to detect at least one actor and / or object depicted in the video data 301 (e.g., to generate actor detection data and / or object detection data). The machine learning model can be further trained to detect a specific actor and / or a type of actor (e.g., a goalie, quarterback, etc.) based on the actor’s stances, movements,Attorney Docket No.: KHSL-001 / 01WO 354667-2003 features (e.g., facial features, height, hair color), jersey number, position on a playing surface relative to other actors, and / or equipment. The actor classifier 335 can receive an indication of a detected actor from the actor detector 330 and can classify the detected actor by group (e.g., by team). For example, the actor classifier 335 can classify the detected actor based on a color mask(s) that is consistent with the team’s jerseys. To classify an attribute(s) of a detected color mask, the actor classifier 335 can be robust to varying lighting conditions on the field and / or throughout the scenario (e.g., the game). The actor classifier 335 can also be robust to a change in uniform color during a game (e.g., pigment changes caused by soils deposited on the uniform). For example, a team’s jersey can be sampled in various lighting conditions (e.g., over a plurality of games), and a threshold value(s) associated with an RGB color model (and / or other color model) can be defined to be inclusive of the observed samples. The actor classifier 335 can then use this threshold value(s) to identify an actor of the team.

[0044] The actor classifier 335 can further generate a bounding box(es) for an actor detected by the actor detector 330 and / or classified by the actor classifier 335. A bounding box can be positioned within a frame such that the bounding box surrounds the depiction of the actor within the frame. A portion of the bounding box (e.g., the middle of the base of the bounding box) can designate a relative position (e.g., a frame-specific position) of the agent.

[0045] The actor location determinator 340 can determine a position of at least one actor depicted in a video frame from the video data 301 relative to a known field reference frame. More specifically, the actor location determinator 340 can be configured to apply a projective transform to the video frame (and / or the bounding box associated with an actor) using the homography matrix generated by the homography matrix generator 325. Each frame (or a selection of frames) from video data 301 (e.g., that spans an entire game) can be processed by the components 300 to determine actor data for at least a portion (e.g., the entirety) of the game. The actor location determinator 340 can be further configured to perform post processing to reduce noise in the player position location estimates.

[0046] FIG.4 shows a system block diagram of components 400 included in a dynamics prediction system, according to an embodiment. The components 400 can implement a method of predicting an outcome of a scenario (e.g., a game). The components 400 can be associated with a computeAttorney Docket No.: KHSL-001 / 01WO 354667-2003 device (e.g., a compute device that is structurally and / or functionally similar to the compute device 201 of FIG. 2 and / or the compute devices 110 and / or 120 of FIG. 1). In some instances, for example, the components 400 can be included in the dynamics prediction application 122 (FIG.1) and / or the dynamics prediction application 212 (FIG. 2), and / or can include software stored in memory 210 and configured to execute via the processor 220 of FIG. 2. In some instances, for example, at least a portion of the components 400 can be implemented in hardware. The components 400 include input data 410, a pre-processor 420, a scenario generator 445, an engine 460, and a post-processor 465.

[0047] The input data 410 can include video data 401 (which can be similar to the video data 301 of FIG. 3) and / or sensor data 405. The sensor data 405 can be generated by a sensor(s) that is structurally and / or functionally similar to the sensor(s) 130 of FIG.1. The sensor data 405 can be associated with an actor (e.g., and can be generated by a wearable sensor) and / or an object of interest (e.g., and can be generated by a sensor embedded in that object of interest). Alternatively or in addition, the input data 410 can include data that is derived from the video data 401 and / or the sensor data 405. For example, the input data 410 can include scenario (e.g., gameplay) observation data that is represented by a numerical value and / or a set of numerical values, and this scenario observation data can include position location data and / or other measure of gameplay (e.g., velocity data) for at least some of the scenario participants (e.g., players) and / or the object of interest. In some instances, the input data 410 can be sourced from a rendering (e.g., for esports and / or other simulated scenarios). As described further below, the input data 410 can be used to train, via a dynamics estimator 425, a model to be included in a predictor 430.

[0048] The input data 410 can be pre-processed by the pre-processor 420 to apply a standardization and / or normalization to the input data 410, to reformat the input data 410, and / or to reduce measurement noise included in the input data 410. The pre-processor 420 can also include a parameter estimator 415, which can be configured to determine a parameter(s) of interest to the scenario prediction. For example, if a prediction of a ball location following (1) a play and / or (2) a predetermined time is desired, the parameter estimator 415 can determine, based on the input data 410, parameters associated with ball locations throughout the game, and the parameter estimator 415 can further order these parameters in time. As another example, if the scenario prediction includes a prediction of an actor velocity, the parameter estimator 415 can determineAttorney Docket No.: KHSL-001 / 01WO 354667-2003 and order in time parameters associated with actor velocity during the scenario. If a desired prediction of interest is not directly representable by an individual parameter, the parameter estimator 415 can determine multiple parameters to support the prediction. For example, if a prediction of a team scoring points is desired, parameters of both the ball location and the player velocities can be calculated and ordered in time by the parameter estimator 415.

[0049] The engine 460 (which can be an artificial intelligence (AI) engine) can include a dynamics estimator 425, a predictor 430, a library 455, and / or a configurator 450. Pre-processed data, including parameters, generated by the pre-processor 420 can be provided as input to the dynamics estimator 425, which can be configured to estimate spectral and / or geometrical dynamical properties of the scenario. More specifically, the dynamics estimator 425 can be configured to perform dynamic mode decomposition (e.g., Koopman mode decomposition) to produce at least one mode (e.g., at least one Koopman mode) associated with the game. This dynamic mode decomposition is also referred to herein as a transformation associated with a linear dynamics domain. Dynamical properties (e.g., dynamical modes) of the game can define a dynamical model of the scenario, and the predictor 430 can include this dynamical model. In some instances (e.g., where the input data 410 that represents a large number of game observations and / or where sufficient computational resources are available), the dynamics estimator 425 can include, for example, a deep neural network(s), a generative adversarial network(s) (GAN(s)), and / or a similarly suited machine learning model. These models can be trained using supervised and / or unsupervised techniques to estimate geometrical dynamical properties of the scenario to generate a dynamical model of the scenario. In some instances, the dynamics estimator 425 can include a machine learning model that is configured to output an observed metamodel.

[0050] Alternatively or in addition, in some instances, where the input data 410 includes limited game observations (e.g., a reduced training dataset), where computational resources are constrained, and / or where a more comprehensive dynamical game representation is desired, the dynamics estimator 425 can include more computationally efficient operator methods, such as Koopman operator and / or a similarly configured linear operator, to estimate the spectral dynamical properties and / or the geometrical dynamical properties of the game. Operator methods can generate the dynamical model of the game and / or provide measures on model stability (e.g., by determining eigenvalues that can be real and / or imaginary and can indicate a level of stability).Attorney Docket No.: KHSL-001 / 01WO 354667-2003

[0051] The predictor 430 can be trained based on scenario observations (e.g., historical position data) included in the input data 410 to predict the outcome of game scenarios represented by scenario data received from the scenario generator 445 during inferencing, as described further herein. As described above, the predictor 430 can include a stable dynamical model of the scenario to predict the outcome of a scenario. A scenario (e.g., a game scenario) can be a particular physical or simulated configuration of, for example, game players and / or entities. A scenario can also include an associated context of the scenario, such as game score, time constraints, etc.

[0052] The scenario generator 445 can include an observed scenario generator 435 and / or a synthetic scenario generator. The observed scenario generator 435 can derive a representation of an observed scenario based on pre-recorded and / or live data (e.g., game data) included in the input data 410. Pre-recorded data can include, for example, historical position data previously collected for a sport, team, player, etc., that is associated with a game to be analyzed. The synthetic scenario generator 440 can generate synthetic scenarios based on simulations (e.g., Monte Carlo simulation) to examine a game scenario that is not represented by observed data included in the input data 410. As described further below, the scenario generator 445 can provide scenario data as input to the predictor 430 during inferencing to generate predictions (e.g., future position data).

[0053] The predictor 430 can predict an outcome of a single scenario, multiple independent scenarios, and / or multiple scenarios that are temporally and / or dynamically related. For example, in American football, temporally and / or dynamically related scenarios can include, for example, a first down play that results in 3 yards of gain can change the dynamics of the following second down play, where the team now has to advance the ball only 7 yards instead of 10 yards. Similarly, a first down play that results in no forward progress can change the dynamics of a following play, since the ball needs to be advanced 10 yards in the three remaining downs instead of 10 yards in four downs. In some instances, more than two successive scenarios can be temporally and / or dynamically related. Dynamics of a play can also be based on a temporal order of a plurality of plays that includes this play. The predictor 430 can predict game dynamics a period of time into the future, where this period of time (e.g., 5 seconds, etc.) can be determined based on the stability of the dynamical model included in the predictor 430. For a dynamical model generated by an operator like a Koopman operator or a similarly configured linear operator, model stability can be estimated at the time of model generation based on eigenvalues associated with the dynamicalAttorney Docket No.: KHSL-001 / 01WO 354667-2003 model. For example, an eigenvalue having a negative real part can indicate a stable model, whereas an eigenvalue having a positive real part can indicate an unstable model. Similarly stated, real poles of the dynamical model that are more positive can be more unstable, which can limit how far into the future the dynamical model can predict before the prediction is corrupted by the instability. For a dynamical model generated by a machine learning model, dynamical model stability can be estimated based on, for example, controlled experimentation. The dynamical model can be executed (e.g., via a processor) iteratively, where each iteration can generate a prediction associated with a future time that can be based on the stability of the linear operator.

[0054] The library 455 can include data that can improve the prediction performance of the predictor 430. This data can include, for example, an individual player performance measure(s) (e.g., top speed), an environmental condition(s) (e.g., wind, temperature, humidity, etc.), a pre- coordinated and / or predetermined team behavior(s) (e.g., plays in a playbook, infraction history, etc.), a coach and / or captain preference(s) and / or pattern(s), an injury probability associated with a play (e.g., a passing play to the center of the field), and / or other information specific to a game, team, and / or condition that could be used to improve the prediction. In some instances, the library 455 can be precluded from directly influencing the dynamics of the dynamics model of the predictor 430; in other instances, the library 455 can extend the stability of the dynamics model. For example, if a team’s playbook is included in the library and the dynamics model can identify the movement of the team to be a variation of an established play in the playbook, the dynamics model can constrain the prediction to be ‘close’ to the pre-determined play in the playbook. As a result, stability of the dynamics model can be improved.

[0055] The configurator 450 can receive a goal, a request, and / or a desired outcome. An example of a goal can be “minimize time to get a score” or “minimize an injury.” A request can include, for example, “display three play options in order of scoring likelihood.” A desired outcome can include, for example, “turnover on downs.” The configurator 450 can include an interface (e.g., that is functionally and / or structurally similar to the user interface 112 of FIG. 1), such as a graphical user interface (GUI), to receive the goal, request, and / or desired outcome from the user. In some implementations, the configurator 450 can include a machine learning model (e.g., a large language model (LLM)) that can generate a machine-readable representation of the goal, request, and / or desired outcome defined by the user. If, for example, a user defines a time constraint and / orAttorney Docket No.: KHSL-001 / 01WO 354667-2003 an end condition to be met, the configurator 450 can define an objective function(s) that can facilitate a search for a team behavior(s) (e.g., dynamics of an agent, a plurality of agents, and / or an object of interest) that satisfy the constraint and / or condition while maintaining dynamics that are consistent with the dynamical model within the predictor 430. The library 455 and / or the configurator 450 can improve quality and / or accuracy of the predictor 430.

[0056] In some instances, the engine 460 can be configured to generate a density map (e.g., via the dynamics estimator 425), which can be a grid having a plurality of cells. The grid can represent a defined area (e.g., a field or other playing surface), and each cell from the plurality of cells can represent a segment of the defined area. Each cell can be associated with a density, which can indicate a number of times a team, player, and / or gameplay object was located within the playing surface segment associated with that cell for a given time period (e.g., that spans a portion of a game, a full game, a plurality of games, a season, etc.). Historical position data (e.g., that is associated with a previous game(s)) density map can be associated with a density map that can be used to train (e.g., parameterize, generate, etc.) a linear operator. This density map can be associated with a plurality of games and / or a plurality of plays, such that the density map is a density-over-time map (e.g., a density map that spans a period of time). Following training, the linear operator can map the dynamics of an actor(s), team(s), and / or gameplay object, such that the linear operator can predict (e.g., via the predictor 430) a future position(s) of the actor(s), team(s), and / or gameplay object, where the future position(s) is associated with a second (e.g., future) time. The linear operator can predict the future position based on a current position associated with the first time that is before the second time. The engine 460 can execute the linear operator iteratively (e.g., via the predictor 430) to make incremental predictions (e.g., predictions at one second intervals, five second intervals, etc.). Following an incremental prediction, the engine 460 can receive additional (e.g., current) game data, which can be used to determine a subsequent incremental prediction.

[0057] In some implementations, the input data 410 can be used to train the linear operator (e.g., via the dynamics estimator 425), and the trained linear operator can be received by the predictor. The scenario generator can then provide new data (e.g., from a current game for which a prediction is to be made and / or from a user-defined scenario) to the predictor to generate a prediction.Attorney Docket No.: KHSL-001 / 01WO 354667-2003

[0058] In some embodiments, a user can provide an indication of a designated actor via a user interface (e.g., that is structurally and / or functionally similar to the user interface 112 of FIG.1). The designated actor can be, for example, a standout player, a player having a significant ability as compared to other players (e.g., speed, dribbling, etc.), a player associated with a specific position (e.g., a quarterback), and / or the like. The engine 460 can be configured to generate a density-over-time map for the designated actor based on position data for that designated actor, and both this density-over-time map and a density-over-time map for remaining actors (e.g., from a team that includes the designated actor) can be used to train the linear operator to jointly estimate future positions for the designated actor and the remaining actors. In some instances, a designated actor can be modeled as a particle, and a plurality of actors (e.g., a team, an offense, a defense, a powerplay unit, etc.) can be modeled as a fluid, and the dynamics estimator 425 can combine a particle model(s) and a fluid model(s) to jointly simulate the dynamics of designated actors and a plurality of actors. Similar to a designated actor, an object of interest (e.g., a gameplay object) can also be modeled as a particle (e.g., using a density-over-time map that is specific to the object of interest).

[0059] The post-processor 465 can generate data and / or representations to aid a user’s interpretation of scenario outcome data generated by the predictor 430. For example, the post- processor 465 can generate numerical data associated with a request received via the configurator 450. This numerical data can indicate, for example, predicted yardage following a play, a likelihood that a play ends in a score, etc. The post-processor 465 can include a visualizer 470 that can be configured to cause display of the numerical estimates and / or generate and / or cause display of a static image and / or video data. The static image and / or the video data can represent predicted dynamics of an agent(s) and / or an object of interest.

[0060] To illustrate the components 400 in use, an example scenario can include a play within an American football game that involves an offense from a first team and a defense from a second team competing against the first team. The offense can include, for example, a quarterback and a plurality of other players (e.g., linemen, receivers, at least one tight end, at least one running back, etc.). A coach of the defense from the second team might desire to predict a play run by the offense from the first team based on (1) positioning of (a) the players from the offense and / or (b) the players from the defense, time left in the game, score, field position, etc. Position data for offenseAttorney Docket No.: KHSL-001 / 01WO 354667-2003 players and / or defense players can be derived from the video data 401 and / or the sensor data 405 using, for example, a method implemented by components that are structurally and / or functionally similar to the components 300 of FIG.3. Position data can also be collected for a football during gameplay. The position data can be collected from previous games (e.g., a game that involves both the first team and the second team and / or a game that involves one of the first team or the second team) or the current game between the first team and the second team. The position data can be provided as input to the pre-processor 420 to estimate, via the parameter estimator 415, parameters associated with a player(s) on the offense, a player(s) on the defense, and / or the football, such as a velocity parameter(s).

[0061] The position data and / or the parameters can be provided as input to the dynamics estimator 425 to train a dynamic model, such as a linear operator. More specifically, the dynamics estimator 425 can generate one or more density maps based on the position data and / or the parameters data. In some instances, the dynamics estimator 425 can receive an indication of one or more designated player(s). For example, a user can designate the quarterback, a running back, a specific receiver of concern, a specific defensive lineman of concern, etc. as a designated player. In response to the user designating the quarterback as a designated player, the dynamics estimator 425 can generate a density map that is specific to the quarterback. The dynamics estimator 425 can also generate an additional density map for the football (e.g., a plurality of footballs used in a plurality of games represented by the position data) and yet another density map for non-designated players from the offense and / or defense. Based on the heatmap(s), the dynamics estimator can train a model (e.g., a linear operator) to jointly predict the motion of designated players on the offense and / or defense, non-designated players on the offense and / or defense, and / or the football.

[0062] The predictor 430 can receive the model that is trained by the dynamics estimator 425. The predictor 430 can also receive game data from the scenario generator 445. This game data can include, for example, real-time data (or substantially real-time data) from the observed scenario generator 435 and can be associated with the game currently being played between the first team and the second team. For example, the real-time data, which can include position data for the football, defensive players, and / or offensive players, can be received after players have lined up at the line of scrimmage but before the football is snapped. Alternatively, the game data can include synthetic game data that can be defined by a user via a user interface (e.g., that is functionallyAttorney Docket No.: KHSL-001 / 01WO 354667-2003 and / or structurally similar to the user interface 112 of FIG.1) and the synthetic scenario generator 440. For example, a user (e.g., a coach) associated with the defense of the second team can test a hypothetical defensive formation (e.g., a prevent defense formation, a 4-3 defense formation, a nickel defense formation, etc.). As yet another example, a user (e.g., a general manager) associated with the defense can insert a desired player (e.g., a player to be signed or traded for) into the set of defense players.

[0063] The predictor 430 can predict an outcome by providing the game data from the scenario generator 445 as input to the trained model. For example, the predictor 430 can predict the movement of the football, players on the offense, and / or players on the defense based on the game data. The predictor 430 can also receive a constraint from the configurator 450. This constraint can be associated with a goal defined by the user. For example, the constraint can indicate the user’s desire to prevent the offense from completing a long pass. In response, the predictor 430 can predict positioning and / or movement of defensive players (individually and / or collectively) that can reduce the likelihood of the offense completing a long pass.

[0064] The visualizer 470 can cause display of a predicted play (e.g., players and / or the football in motion), such that, for example, a coach can cause their players to position themselves and / or run routes in manner that achieves a desired goal (e.g., stopping the offense, scoring a touchdown, minimizing injury, etc.), as suggested by the predictor 430. The predictor 430 and the visualizer 470 can also cause a coach to select one player over another player based on a comparison of synthetic games involving those respective players. Because the predictor 430 can use real-time (or near real-time) data and / or historical data during inferencing, a user can also use the predictor 430 to generate predictions during a game and / or during practice in preparation for a game. The historical data can include, for example, position data from previous games involving the opposing team that the user is preparing to play.

[0065] FIG. 5 shows a flow diagram illustrating a method 500 for causing display of representations of at least one actor and an object of interest based on dynamics data, according to an embodiment. The method 500 can be implemented by a dynamics prediction system described herein (e.g., the dynamics prediction system 100 of FIG. 1). Portions of the method 500 can beAttorney Docket No.: KHSL-001 / 01WO 354667-2003 implemented using a processor (e.g., the processor 220 of FIG.2) of any suitable compute device (e.g., the compute device 201 of FIG.2 and / or the compute devices 110 and / or 120 of FIG.1).

[0066] The method 500 at 502 includes receiving (1) actor position data associated with at least one actor, (2) object position data associated with an object of interest, and (3) an indication of a designated actor. At 504, designated actor position data is identified within the actor position data based on the indication of the designated actor, the designated actor being from the at least one actor. The method 500 at 506 includes applying a transformation to the actor position data, the object position data, and the designated actor position data, to produce first dynamics data associated with a first time, the transformation being associated with a linear dynamics domain. At 508, a linear operator is applied to the first dynamics data to produce second dynamics data associated with a second time that is after the first time. At 510, the method 500 includes causing display of a representation of the at least one actor and a representation of the object of interest, based on the second dynamics data.

[0067] FIG.6 shows a flow diagram illustrating a method 600 for generating and applying a linear operator to first dynamics data to predict second dynamics data, according to an embodiment. The method 600 can be implemented by a dynamics prediction system described herein (e.g., the dynamics prediction system 100 of FIG.1). Portions of the method 600 can be implemented using a processor (e.g., the processor 220 of FIG.2) of any suitable compute device (e.g., the compute device 201 of FIG.2 and / or the compute devices 110 and / or 120 of FIG.1).

[0068] The method 600 at 602 includes receiving, at a processor, (1) actor position data associated with a set of actors in a sports game and (2) object position data associated with an object of interest in the sports game. The method 600 further includes, at 604, performing, via the processor, dynamic mode decomposition jointly on the actor position data and the object position data, to produce first dynamics data associated with a first time. At 606, a density-over-time map is generated, via the processor, for the set of actors and the object of interest, based on historical data associated with at least one of the set of actors or the sports game. The method 600 at 608 includes generating, via the processor, a linear operator based on the density-over-time map, and at 610, applying, via the processor, the linear operator to the first dynamics data to predict second dynamics data associated with a second time that is after the first time.Attorney Docket No.: KHSL-001 / 01WO 354667-2003

[0069] FIG. 7 shows a visualization 700 of prediction data for an American football game and generated by a dynamics prediction system, according to an embodiment. The visualization 700 can be depicted by a graphical user interface (GUI) that is functionally and / or structurally similar to the interface 112 of FIG. 1. The prediction data can be generated by a dynamics prediction system described herein (e.g., the dynamics prediction system 100 of FIG.1).

[0070] The prediction data can be generated based position location data for players (e.g., offense and / or defense players) and a football, and the prediction data can indicate predictions of future ball positions. The prediction data can be generated by training the dynamics prediction system on a number of past games (e.g., 5 games, 10 games, 15 games, and / or etc.). For example, training position location data can be associated with passing plays within a plurality of past games. After training, pre-snap position location information of a passing play not represented in the training data can be evaluated by the dynamics prediction system.

[0071] More specifically, the dynamics prediction system can identify a region on the field and an associated likelihood that the football will be within that region at the end of a play. For example, as shown in FIG.7, starting positions of the offensive players and defensive players are depicted as ‘o’s and ‘x’s respectively. A probability heat map is overlayed on the depiction of the field shown in FIG.7, and the paths that the offensive players take are also illustrated. The probabilities associated with four outcomes are shown on the right of the visualization 700: (i) probability of no or negative forward progress (28%), (ii) probability of forward progress short of a first down (29%), (iii) probability of a first down but short of the endzone (4%), and (iv) probability of a touchdown (40%). As described herein, the dynamics prediction system can generate these probabilities by learning predictable patterns in the game of American football (e.g., with limited training data), and the probabilities can represent detailed and accurate predictions of game positions that extend beyond the training data.

[0072] FIG.8 shows a visualization 800 of prediction data for a soccer game and generated by a dynamics prediction system, according to an embodiment. The visualization 800 can be depicted by a graphical user interface (GUI) that is functionally and / or structurally similar to the interface 112 of FIG. 1. The prediction data can be generated by a dynamics prediction system described herein (e.g., the dynamics prediction system 100 of FIG.1).Attorney Docket No.: KHSL-001 / 01WO 354667-2003

[0073] The dynamics prediction system can be applied to soccer position location data to make accurate predictions of future ball positions. In some instances, the dynamics prediction system can be trained on a single previous game (and not other games). In some embodiments, the dynamics prediction system used to analyze the soccer game can be similar to the dynamics prediction system used to evaluate, for example, American football (e.g., as described in relation to FIG.7), with minor adjustments, for example, to permit ingestion of different data formats.

[0074] In the example illustrated in FIG.8, the dynamics prediction system evaluates possible ball location approximately 34 seconds into the future (from the time of making the prediction) and produces a heatmap of possible outcomes and / or probabilities for the future ball location. The heatmap depiction shows darker regions where the probabilities are higher and lighter regions where the probabilities are lower. The numerical values shown in FIG.8 represent the sum of the probabilities of an associated region. The region in and around the left penalty box, for example, shows a 42% probability, which is higher than other locations on the field, indicating a predicted scoring opportunity, even though the ball at the time of the prediction is on the opposite side of the field.

[0075] FIG.9 shows a visualization 900 of a plurality of ball positions predicted by a dynamics prediction system, according to an embodiment. The visualization 900 can be depicted by a graphical user interface (GUI) that is functionally and / or structurally similar to the interface 112 of FIG. 1. The predictions can be generated by a dynamics prediction system described herein (e.g., the dynamics prediction system 100 of FIG.1).

[0076] The visualization 900 represents a progression of play that follows the prediction represented by the visualization 800 of FIG. 8. More specifically, at 902, the ball is passed to player 13, who then passes to player 27 at 904, who at 906 dribbles the ball down field and passes at 908 to player 23, who shoots on the goal at 910. The goalie stops the shot, as shown at 912, but the prediction of the scoring opportunity that was predicted 34 seconds beforehand (as described above in relation to FIG.8) is consistent with the preceding play. This example illustrates that the dynamics prediction system can learn predictable patterns in the game of soccer with limited training data and is capable of summarizing non-linear dynamical predictions in terms ofAttorney Docket No.: KHSL-001 / 01WO 354667-2003 probability heatmaps and making accurate predictions of complex sequences and team interactions.

[0077] FIG.10 shows a visualization 1000 of prediction data over time for an offensive maneuver in a soccer game, according to an embodiment. The visualization 1000 can be depicted by a graphical user interface (GUI) that is functionally and / or structurally similar to the interface 112 of FIG.1. The prediction data can be generated by a dynamics prediction system described herein (e.g., the dynamics prediction system 100 of FIG.1). The visualization 1000 shows a plurality of heatmaps 1002-1012. In some implementations not shown in FIG. 10, the heatmaps 1002-1012 can be implied (e.g., represented as matrix data but not rendered in the visualization 1000).

[0078] The dynamics prediction system can generate a series of predictions associated with specific points in time. The example illustrated in FIG.10 shows a time evolution of predictions and probabilities associated with a complex offensive soccer maneuver and / or play. As described above, darker regions depicted in the visualization 1000 designate areas of higher probability for predicted ball location (e.g., predicted 36 seconds before the actual play proceeds), solid arrows indicate the path that the ball actuality takes, and dashed arrows show the path that selected players take. As play progresses, the probability distribution also progresses. Each heatmap shown in this example can be generated before play occurs, and each heatmap can be associated with a particular instance in time. For example, the heatmap 1002 can be associated with a first time, the heatmap 1004 can be associated with a second time that is after the first time, the heatmap 1006 can be associated with a third time that is after the second time, etc. The dynamics prediction system can therefore not only predict spatial information (e.g., where the ball will be) but can also predict temporal information (e.g., when the ball will arrive at a given location).

[0079] In some embodiments, a method for creating an automated metamodel of system dynamic simulation (e.g., to represent a sports game, an esports game, etc.) includes collecting game data, generating a machine-learned model of the game, and causing results to be presented in visual, audio, and / or text format. In some implementations, the game data can be collected via video recordings, audio recordings, and / or wearable and / or object (e.g., ball, puck, etc.) embedded sensors. In some implementations, the machine-learned model can include at least one Koopman operator model (e.g., from a family of Koopman operator models) that can be parameterized byAttorney Docket No.: KHSL-001 / 01WO 354667-2003 input parameters. In some implementations, the machine-learned model can be a single Koopman operator model. In some implementations, the machine-learned model can be a deep learning mode, a support vector machine (SVM), and / or the like. In some implementations, the method can further include predicting an outcome a play within a game, predicting the outcome of a game, and / or informing a player and / or a team. In some implementations, the system dynamic simulation can include or exclude controls.

[0080] In some embodiments, a method for determining a sensitivity (e.g., a scenario sensitivity, a game sensitivity, etc.) to agent (e.g., player) replacement includes selecting an agent in a scenario (e.g., a game, such as a sports game), selecting a replacement for that agent, performing sensitivity analysis to agent change (e.g., from the agent to the replacement) by executing a model of the scenario, and presenting sensitivity analysis results in a tabular representation, a graphical representation, an audio representation, etc. In some implementations, the sensitivity analysis can be performed by executing a learned model that is (1) a family of Koopman operator models parametrized by input parameters, (2) a single Koopman operator model, (3) a deep learning model, a support vector machine (SVM), and / or (4) a similarly suited model.

[0081] Examples of computer code include, but are not limited to, micro-code or micro- instructions, machine instructions, such as produced by a compiler, code used to produce a web service, and files containing higher-level instructions that are executed by a computer using an interpreter. For example, embodiments can be implemented using Python, Java, JavaScript, C++, and / or other programming languages and development tools. Additional examples of computer code include, but are not limited to, control signals, encrypted code, and compressed code.

[0082] The drawings primarily are for illustrative purposes and are not intended to limit the scope of the subject matter described herein. The drawings are not necessarily to scale; in some instances, various aspects of the subject matter disclosed herein can be shown exaggerated or enlarged in the drawings to facilitate an understanding of different features. In the drawings, like reference characters generally refer to like features (e.g., functionally similar and / or structurally similar elements).

[0083] The acts performed as part of a disclosed method(s) can be ordered in any suitable way. Accordingly, embodiments can be constructed in which processes or steps are executed in an orderAttorney Docket No.: KHSL-001 / 01WO 354667-2003 different than illustrated, which can include performing some steps or processes simultaneously, even though shown as sequential acts in illustrative embodiments. Put differently, it is to be understood that such features can not necessarily be limited to a particular order of execution, but rather, any number of threads, processes, services, servers, and / or the like that can execute serially, asynchronously, concurrently, in parallel, simultaneously, synchronously, and / or the like in a manner consistent with the disclosure. As such, some of these features can be mutually contradictory, in that they cannot be simultaneously present in a single embodiment. Similarly, some features are applicable to one aspect of the innovations, and inapplicable to others.

[0084] Where a range of values is provided, it is understood that each intervening value, to the tenth of the unit of the lower limit unless the context clearly dictates otherwise, between the upper and lower limit of that range and any other stated or intervening value in that stated range is encompassed within the disclosure. That the upper and lower limits of these smaller ranges can independently be included in the smaller ranges is also encompassed within the disclosure, subject to any specifically excluded limit in the stated range. Where the stated range includes one or both of the limits, ranges excluding either or both of those included limits are also included in the disclosure.

[0085] The phrase “and / or,” as used herein in the specification and in the embodiments, should be understood to mean “either or both” of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with “and / or” should be construed in the same fashion, i.e., “one or more” of the elements so conjoined. Other elements can optionally be present other than the elements specifically identified by the “and / or” clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to “A and / or B”, when used in conjunction with open-ended language such as “comprising” can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.

[0086] As used herein in the specification and in the embodiments, “or” should be understood to have the same meaning as “and / or” as defined above. For example, when separating items in a list,Attorney Docket No.: KHSL-001 / 01WO 354667-2003 “or” or “and / or” shall be interpreted as being inclusive, i.e., the inclusion of at least one, but also including more than one of a number or list of elements, and, optionally, additional unlisted items. Only terms clearly indicated to the contrary, such as “only one of” or “exactly one of,” or, when used in the embodiments, “consisting of,” will refer to the inclusion of exactly one element of a number or list of elements. In general, the term “or” as used herein shall only be interpreted as indicating exclusive alternatives (i.e., “one or the other but not both”) when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.” “Consisting essentially of,” when used in the embodiments, shall have its ordinary meaning as used in the field of patent law.

[0087] As used herein in the specification and in the embodiments, the phrase “at least one,” in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements. This definition also allows that elements can optionally be present other than the elements specifically identified within the list of elements to which the phrase “at least one” refers, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, “at least one of A and B” (or, equivalently, “at least one of A or B,” or, equivalently “at least one of A and / or B”) can refer, in one embodiment, to at least one, optionally including more than one, A, with no B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, with no A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, and at least one, optionally including more than one, B (and optionally including other elements); etc.

[0088] In the embodiments, as well as in the specification above, all transitional phrases such as “comprising,” “including,” “carrying,” “having,” “containing,” “involving,” “holding,” “composed of,” and the like are to be understood to be open-ended, i.e., to mean including but not limited to. Only the transitional phrases “consisting of” and “consisting essentially of” shall be closed or semi-closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures, Section 2111.03.Attorney Docket No.: KHSL-001 / 01WO 354667-2003

[0089] Some embodiments described herein relate to a computer storage product with a non- transitory computer-readable medium (also can be referred to as a non-transitory processor- readable medium) having instructions or computer code thereon for performing various computer- implemented operations. The computer-readable medium (or processor-readable medium) is non- transitory in the sense that it does not include transitory propagating signals per se (e.g., a propagating electromagnetic wave carrying information on a transmission medium such as space or a cable). The media and computer code (also can be referred to as code) can be those designed and constructed for the specific purpose or purposes. Examples of non-transitory computer- readable media include, but are not limited to, magnetic storage media such as hard disks, floppy disks, and magnetic tape; optical storage media such as Compact Disc / Digital Video Discs (CD / DVDs), Compact Disc-Read Only Memories (CD-ROMs), and holographic devices; magneto-optical storage media such as optical disks; carrier wave signal processing modules; and hardware devices that are specially configured to store and execute program code, such as Application-Specific Integrated Circuits (ASICs), Programmable Logic Devices (PLDs), Read- Only Memory (ROM) and Random-Access Memory (RAM) devices. Other embodiments described herein relate to a computer program product, which can include, for example, the instructions and / or computer code discussed herein.

[0090] Some embodiments and / or methods described herein can be performed by software (executed on hardware), hardware, or a combination thereof. Hardware modules can include, for example, a processor, a field programmable gate array (FPGA), and / or an application specific integrated circuit (ASIC). Software modules (executed on hardware) can include instructions stored in a memory that is operably coupled to a processor and can be expressed in a variety of software languages (e.g., computer code), including C, C++, Java™, Ruby, Visual Basic™, Python, and / or other object-oriented, procedural, scientific, or other programming language and development tools. Examples of computer code include, but are not limited to, micro-code or micro-instructions, machine instructions, such as produced by a compiler, code used to produce a web service, and files containing higher-level instructions that are executed by a computer using an interpreter. For example, embodiments can be implemented using imperative programming languages (e.g., C, Fortran, etc.), functional programming languages (Haskell, Erlang, etc.), logical programming languages (e.g., Prolog), object-oriented programming languages (e.g., Java, C++, etc.) or other suitable programming languages and / or development tools. Additional examples ofAttorney Docket No.: KHSL-001 / 01WO 354667-2003 computer code include, but are not limited to, control signals, encrypted code, and compressed code.

Claims

Attorney Docket No.: KHSL-001 / 01WO 354667-2003 CLAIMS What is claimed is:

1. A non-transitory, processor-readable medium storing instructions that, when executed by a processor, cause the processor to: receive (1) actor position data associated with a set of actors in a sports game, (2) object position data associated with an object of interest in the sports game, and (3) an indication of a designated actor from the set of actors; identify designated actor position data within the actor position data based on the indication of the designated actor; apply a transformation to the actor position data, the object position data, and the designated actor position data, to produce first dynamics data associated with a first time, the transformation being associated with a linear dynamics domain; apply a linear operator to the first dynamics data to predict second dynamics data associated with a second time that is after the first time; and cause display of a predicted location of the set of actors and a predicted location of the object of interest, based on the second dynamics data.

2. The non-transitory, processor-readable medium of claim 1, further storing instructions to cause the processor to: receive video data depicting (1) the set of actors and (2) the object of interest; and generate the actor position data and the object position data, based on the video data.

3. The non-transitory, processor-readable medium of claim 1, further storing instructions to cause the processor to: receive sensor data associated with at least one of (1) the set of actors or (2) the object of interest; and generate at least one of the actor position data or the object position data, based on the sensor data.

4. The non-transitory, processor-readable medium of claim 1, wherein the set of actors includes a game player and the object of interest is a gameplay object.Attorney Docket No.: KHSL-001 / 01WO 354667-2003 5. The non-transitory, processor-readable medium of claim 1, wherein the set of actors includes a videogame character.

6. The non-transitory, processor-readable medium of claim 1, wherein the first dynamics data is representable by a density map.

7. The non-transitory, processor-readable medium of claim 1, wherein the transformation includes a dynamic mode decomposition.

8. The non-transitory, processor-readable medium of claim 7, wherein the dynamic mode decomposition is a Koopman mode decomposition.

9. The non-transitory, processor-readable medium of claim 3, further storing instructions to cause the processor to: receive an indication of a constraint from a user compute device, the second dynamics data being based on the indication of the constraint.

10. The non-transitory, processor-readable medium of claim 1, further storing instructions to cause the processor to: receive historical position data associated with (1) the set of actors and (2) the object of interest; generate a density-over-time map for the set of actors and the object of interest; and generate the linear operator based on the density-over-time map.

11. The non-transitory, processor-readable medium of claim 1, wherein: the instructions to cause the processor to apply the transformation include instructions to cause the processor to: generate a first density map based on the designated actor position data, generate a second density map based on the object position data, and generate a third density map based on the actor position data; andAttorney Docket No.: KHSL-001 / 01WO 354667-2003 the instructions to cause the processor to apply the linear operator include instructions to cause the processor to apply the linear operator to jointly predict future position for (1) the set of actors and (2) the object of interest, based on the first density map, the second density map, and the third density map.

12. The non-transitory, processor-readable medium of claim 1, further storing instructions to cause the processor to: determine the second time based on an eigenvalue associated with the linear operator.

13. The non-transitory, processor-readable medium of claim 1, wherein the object position data is first object position data, the non-transitory, processor-readable medium further storing instructions to cause the processor to: cause display, based on the second dynamics data, of a first heatmap that represents a first location probability distribution for the object of interest at the second time; receive second object position data associated with the object of interest and for the second time; produce, based on the second object position data, third dynamics data associated with a third time that is after the second time; and cause display, based on the third dynamics data, of a second heatmap that is different from the first heatmap and that represents a second location probability distribution for the object of interest at the third time.

14. The non-transitory, processor-readable medium of claim 1, wherein: the set of actors includes (1) a first subset of players associated with a first team and (2) a second subset of player associated with a second team different from the first team.

15. The non-transitory, processor-readable medium of claim 1, wherein: the sports game includes a ball game; and the object of interest includes a ball associated with the ball game.Attorney Docket No.: KHSL-001 / 01WO 354667-2003 16. The non-transitory, processor-readable medium of claim 1, further storing instructions to cause the processor to: receive image data depicting (1) the set of actors and (2) the object of interest; identify at least one reference point for a playing surface associated with the sports game, based on the image data; provide the image data as input to a machine learning model to produce (1) actor detection data associated with the set of actors and (2) object detection data associated with the object of interest; generate a homography matrix based on the at least one reference point, the actor detection data, and the object detection data; and generate the actor position data and the object position data, based on the homography matrix.

17. A method, comprising: receiving, at a processor, (1) actor position data associated with a set of actors in a sports game and (2) object position data associated with an object of interest in the sports game; performing, via the processor, dynamic mode decomposition jointly on the actor position data and the object position data, to produce first dynamics data associated with a first time; generating, via the processor, a density-over-time map for the set of actors and the object of interest, based on historical data associated with at least one of the set of actors or the sports game; generating, via the processor, a linear operator based on the density-over-time map; and applying, via the processor, the linear operator to the first dynamics data to predict second dynamics data associated with a second time that is after the first time.

18. The method of claim 17, wherein: the dynamic mode decomposition includes a Koopman mode decomposition.

19. The method of claim 17, further comprising: determining, via the processor, the second time based on an eigenvalue that indicates a level of stability for the linear operator.Attorney Docket No.: KHSL-001 / 01WO 354667-2003 20. The method of claim 17, further comprising: receiving, at the processor, image data depicting (1) the set of actors and (2) the object of interest; identifying, via the processor, at least one reference point for a playing surface associated with the sports game, based on the image data; providing, via the processor, the image data as input to a machine learning model to produce (1) actor detection data associated with the set of actors and (2) object detection data associated with the object of interest; generating, via the processor, a homography matrix based on the at least one reference point, the actor detection data, and the object detection data; and generating, via the processor, the actor position data and the object position data, based on the homography matrix.

Images