Systems and methods for identity analysis in sports video

Abstract

Systems and methods are described for identity analysis using video coverage of sporting events. The system receives a plurality of video feeds of a sporting event and detects a plurality of objects of interest across the video feeds. An identity data structure stores data identifying the objects of interest across a plurality of camera views of the video feeds. Upon a change to the current identity information for a given object of interest of the plurality of objects of interest, the system updates the identity information forward and backward over the time course of the video feeds, thereby retroactively correcting past identity assignments when new evidence becomes available. The system transmits a signal encoding data of the identity data structure through a communication interface for consumption by downstream applications such as statistics engines, broadcast graphics systems, highlight generators, and coaching analytics platforms.

Description

SYSTEMS AND METHODS FOR IDENTITY ANALYSIS IN SPORTS VIDEOCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims all benefit, including priority to, U.S. Provisional Patent Application 63 / 788,428, filed April 14, 2025, and entitled "MULTI-CAMERA OBJECT PERMANENCE AND BACK-PROPAGATION”; the entire content of which is incorporated by reference herein.FIELD

[0002] This disclosure relates to video analysis systems, and more particularly relates to systems and methods for identity analysis in sports video.BACKGROUND

[0003] Sports analysis and broadcasting increasingly rely on automated systems to track players, balls, and other objects of interest throughout sporting events. The accuracy and utility of all downstream applications depend fundamentally on the ability to consistently and correctly identify which tracked object corresponds to which real-world entity throughout the entire duration of an event.

[0004] In a typical multi-camera sports broadcast, players move between the fields of view of different cameras, become temporarily occluded by other players or equipment, and present varying visual appearances due to changes in viewing angle, lighting, distance, and physical state. Identifiable features may be only intermittently visible, partially obscured, presented at unfavorable angles, or at insufficient resolution for reliable recognition during substantial portions of a sporting event.

[0005] There is therefore a need for systems that can maintain persistent object identity across multiple camera views and extended time periods throughout the duration of a sporting event.SUMMARY

[0006] In accordance with an aspect, there is provided a computer-implemented method for identity analysis in sports video, the method comprising: receiving a plurality of video feeds of a sporting event; upon processing data from the plurality of video feeds: detecting a plurality of objects of interest across the video feeds; maintaining an identity data structure that stores data identifying the objects of interest across a plurality of camera views of the video feeds, the identity data structure including current and historical identity information for the objects of interest over a time course of the video feeds; upon a change to the current identity information for a given object of interest of the plurality of objects of interest, updating the historical identity information forward and backward over the time course of the video feeds; and transmitting a signal encoding data of the identity data structure through a communication interface.

[0007] In some embodiments, the method further comprises associating with an item of the identity information a confidence metric reflecting a certainty in that item of the identity information.

[0008] In some embodiments, the detecting comprises detecting at least one player, official, coach, equipment, or sport-specific objects within video frames of the video feeds.

[0009] In some embodiments, the method further comprises mapping positions of detected objects of interest across the camera views to a unified spatial coordinate system.

[0010] In some embodiments, the method further comprises extracting from the video feeds identifying features including at least one of jersey number recognitions, facial recognition embeddings, team color classifications, body morphology descriptors, or uniform detail patterns from detected objects of interest.

[0011] In some embodiments, the maintaining utilizes a hybrid memory-augmented transformer.

[0012] In some embodiments, the hybrid memory-augmented transformer communicates with a non-transitory memory that stores identity information to be persistent beyond an attention window of the hybrid memory-augmented transformer.

[0013] In some embodiments, the non-transitory memory implements a memory architecture providing a short-term buffer for recent identity information, a medium term store for session-level identity information, and a long-term store for identity information that spans substantially the sporting event.

[0014] In some embodiments, the method further comprises transferring an item of identity information to long-term storage when confidence in that item exceeds a threshold.

[0015] In some embodiments, the hybrid memory-augmented transformer includes a temporal graph that represents the objects of interest as nodes and their potential identity associations across the time course and the camera views as edges.

[0016] In some embodiments, the updating the historical identity information forward and backward over the time course includes traversing the temporal identity graph.

[0017] In some embodiments, the identity information for the objects of interest includes, for each object of interest, an identity probability distribution of possible identity associations for that object of interest.

[0018] In some embodiments, the maintaining the identity data structure includes updating the identity probability distributions for the objects of interest.

[0019] In some embodiments, the updating the historical identity information forward and backward over the time course includes updating the identity probability distributions by applying a Bayesian update in reverse temporal order.

[0020] In some embodiments, the method further comprises generating a multi-modal signature that combines at least two of appearance feature, spatio-temporal trajectory information, team affiliation indicators, role and position characteristics, and interaction patterns.

[0021] In accordance with another aspect, there is provided a computer-implemented system for identity analysis in sports video, the system comprising: a communication interface; a processing subsystem that includes one or more processors and one or more memories coupled with the one or more processors, the processing subsystem configured to cause the system to: receive a plurality of video feeds of a sporting event; upon processing data from the plurality of video feeds: detect a plurality of objects of interest across the video feeds; maintain an identity data structure that stores data identifying the objects of interest acrossa plurality of camera views of the video feeds, the identity data structure including current and historical identity information for the objects of interest over a time course of the video feeds; upon a change to the current identity information for a given object of interest of the plurality of objects of interest, update the historical identity information forward and backward over the time course of the video feeds; and transmit a signal encoding data of the identity data structure through the communication interface.

[0022] In some embodiments, the computer-implemented system further comprises a video input interface interconnected with at least one camera, and wherein the plurality of video feeds includes a video feed from the at least one camera.

[0023] In some embodiments, the plurality of video feeds includes live footage of the sporting event.

[0024] In some embodiments, the identity data structure is maintained using a memory-augmented transformer.

[0025] In accordance with another aspect, there is provided a non-transitory computer readable medium storing machine interpretable instructions, which when executed by one or more processors, cause the one or more processors to perform a method for identity analysis in sports video, the method comprising: receiving a plurality of video feeds of a sporting event; upon processing data from the plurality of video feeds: detecting a plurality of objects of interest across the video feeds; maintaining an identity data structure that stores data identifying the objects of interest across a plurality of camera views of the video feeds, the identity data structure including current and historical identity information for the objects of interest over a time course of the video feeds; upon a change to the current identity information for a given object of interest of the plurality of objects of interest, updating the historical identity information forward and backward over the time course of the video feeds; and transmitting a signal encoding data of the identity data structure through a communication interface.

[0026] Many further features and combinations thereof concerning embodiments described herein will appear to those skilled in the art following a reading of the instant disclosure.BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In the figures,

[0028] FIG. 1 is a network diagram of a network environment of an example identity analysis system, in accordance with an embodiment;

[0029] FIG. 2 is a schematic diagram of an identity analysis system, in accordance with an embodiment;

[0030] FIG. 3 is a schematic diagram of an example detection subsystem of the identity analysis system, in accordance with an embodiment;

[0031] FIG. 4 is a schematic diagram of an example identification subsystem of the identity analysis system, in accordance with an embodiment;

[0032] FIG. 5 is a block diagram illustrating an example hybrid memory-augmented transformer architecture, in accordance with an embodiment;

[0033] FIG. 6 is a diagram illustrating an example temporal identity graph structure, in accordance with an embodiment;

[0034] FIG. 7 is a block diagram illustrating an example retroactive identity resolution pipeline, in accordance with an embodiment;

[0035] FIG. 8 is a block diagram illustrating an example multi-modal feature association system, in accordance with an embodiment;

[0036] FIG. 9 is a block diagram illustrating an example cross-view identity association mechanism, in accordance with an embodiment;

[0037] FIG. 10 is a block diagram illustrating an example probabilistic identity propagation process, in accordance with an embodiment;

[0038] FIG. 11 is a flowchart of an example method for identity analysis in sports video, in accordance with an embodiment;

[0039] FIG. 12 is a block diagram of an example real-time processing pipeline from multi-camera input through detection, tracking, identity assignment, retroactive resolution, and analytics output, in accordance with an embodiment; and

[0040] FIG. 13 is an example computing device, in accordance with an embodiment.

[0041] These drawings depict exemplary embodiments for illustrative purposes, and variations, alternative configurations, alternative components and modifications may be made to these exemplary embodiments.DETAILED DESCRIPTION

[0042] Embodiments disclosed in the present disclosure address certain technical challenges in maintaining persistent object identity across multiple camera views and extended time periods in sports video analysis. In a multi-camera coverage of a sporting event, a plurality of objects of interest, such as players, officials, coaches, equipment, and sport-specific objects, must be tracked and identified accurately across a plurality of video feeds from different camera views. Maintaining consistent object identity, particularly player identity, across different camera angles, occlusions, and extended time periods presents significant technical challenges that existing systems address inadequately.

[0043] Existing systems that process each video feed independently do not leverage complementary identity information available across multiple camera views and are unable to retroactively correct identity assignments when new identifying information becomes available at a later point in time of the video feeds. Object extraction systems using background subtraction and difference calculation do not address the persistent identity problem across extended durations or multiple views. Ball trajectory tracking systems do not address the challenge of maintaining identity across multiple visually similar entities (e.g., players on the same team). Frame-by-frame tracking approaches accumulate drift errors over time, leading to progressive degradation of identity accuracy. Detection-and-recognition approaches that assign identity only when high-confidence features are visible create temporal gaps in identification and do not improve the identity assignments for periods when such features become unavailable. Identity assignment in existing systems additionally fails during occlusions or when players / objects leave and re-enter camera views, producing fragmented tracks with inconsistent or missing identity labels, hindering any meaningful downstream processing of the video data.

[0044] Embodiments described herein provide systems and methods for identity analysis using multicamera video coverage of sporting events that may address some of the technical problems discussed above. In an example embodiment, the system receives a plurality of video feeds of a sporting event and, upon processing data from the plurality of video feeds, detects a plurality of objects of interest across the video feeds. The system maintains an identity data structure that stores data identifying the objects of interest across a plurality of camera views of the video feeds. The identity data structure includes current and historical identity information for the objects of interest over a time course of the video feeds. Upon a change to the current identity information for a given object of interest of the plurality of objects of interest, the system updates the identity information forward and backward over the time course of the video feeds, thereby retroactively correcting past identity assignments when new evidence becomes available. The system transmits a signal encoding data of the identity data structure through a communication interface for consumption by downstream applications such as statistics engines, broadcast graphics systems, highlight generators, and coaching analytics platforms.

[0045] Some embodiments employ a multi-component architecture as further described below. In an example embodiment, a detection subsystem detects the plurality of objects of interest within video frames of the video feeds and extracts identifying features including jersey number recognitions, facial recognition embeddings, team color classifications, body morphology descriptors, and uniform detail patterns when such features are available with sufficient quality. An identification subsystem maintains the identity data structure using complementary identity processing approaches including a hybrid memory-augmented transformer for long-horizon identity persistence, a multi-modal signature generator for constructing cross-view identity signatures, and a probabilistic identity tracker for uncertainty-aware identity management. A bidirectional identity propagator enables the updating of the historical identity information forward and backward over the time course of the video feeds when new evidence indicating object identity arrives at any point during the sporting event. An output interface transmits the signal encoding data of the identity data structure through the communication interface to downstream consumers.

[0046] FIG. 1 is a network diagram of a network environment of an example identity analysis system, in accordance with an embodiment. As depicted, this network environment includes an identity analysis system 100, interconnected with a plurality of cameras 80, one or more downstream consumers 10, and one or more raw footage sources 90, by way of a communication network 50.

[0047] Identity analysis system 100 is configured to receive the plurality of video feeds of a sporting event from cameras 80 and / or raw footage sources 90, to detect a plurality of objects of interest across the video feeds. The identity analysis system 100 maintains an identity data structure that stores data identifying the objects of interest across a plurality of camera views of the video feeds. Signals encoding data of the identity data structure are transmitted through a communication interface to the downstream consumers 10.

[0048] Each of cameras 80 is configured to provide raw video footage covering the sporting event to the identity analysis system 100. In some embodiments, cameras 80 comprise one or more physically articulating cameras, including pan-tilt-zoom (PTZ) cameras. Alternatively, cameras 80 may comprise one or more wide-angle or panoramic cameras positioned to capture an entire playing field, court, track, rink, pool, or a substantial portion thereof. In some configurations, the cameras 80 are not physically articulated; instead, the identity analysis system 100 extracts a virtual viewport from the wide-angle imagery for subsequent processing.

[0049] The identity analysis system 100 may additionally or alternatively receive, from a raw footage source 90, pre-recorded or externally sourced video delivered over the communication network 50 using a supported streaming protocol, including RTSP, RTMP, SRT, or other supported protocols. The raw footage source 90 may originate from cameras not directly controlled by the identity analysis system 100 or from archived video recordings of previous sporting events. The embodiments disclosed herein are applicable to processing either live footage from cameras 80, raw footage source 90, or combinations thereof.

[0050] Using the video data from cameras 80 and / or raw footage sources 90, the identity analysis system 100 processes the plurality of video feeds to detect and identify the objects of interest, to maintain the identity data structure with current and historical identity information over the time course of the video feeds, and to update the historical identity information forward and backward in time when a change to the current identity information occurs. The identity analysis system 100 transmits a signal encoding data of the identity data structure through the communication interface to the downstream consumers 10, which may include statistics engines, highlight generators, broadcast graphics systems, and coaching analytics platforms. The communication network 50 may carry data using protocols including Internet, Ethernet, PSTN, ISDN, DSL, coaxial cable, fiber optics, satellite, or wireless (e.g., Wi-Fi, WiMAX, Li-Fi) protocols, or any combination thereof.

[0051] FIG. 2 is a schematic diagram of the identity analysis system 100, in accordance with an embodiment. As depicted, the identity analysis system 100 includes a video input interface 102, a detection subsystem 104, an identification subsystem 106, a bidirectional identity propagator 108, an output interface 110, and an electronic datastore 112.

[0052] The video input interface 102 receives the plurality of video feeds of the sporting event from the cameras 80 and / or the raw footage stream sources 90 described with reference to FIG. 1. The video input interface 102 decodes and buffers incoming video streams, preparing the video data for processing by thedetection subsystem 104. The video input interface 102 supports multiple streaming protocols including RTSP, RTMP, and SRT, and processes video feeds from cameras with different resolutions, frame rates, and encoding formats. By centralizing video ingestion, the video input interface 102 facilitates generation of normalized, synchronized video data from disparate camera sources and video streams for processing at subsequent processing stages.

[0053] The detection subsystem 104 processes video frames from each camera view to detect the plurality of objects of interest across the video feeds and to extract identifying features from the detected objects of interest. The detection subsystem 104 employs trained object detection models optimized for sports environments to identify and localize objects of interest, from which identifying features are extracted. A multi-camera registrator within the detection subsystem 104 maps positions of detected objects of interest across the camera views to a unified spatial coordinate system, enabling the system to reason about the same physical entity observed from different viewpoints. The subcomponents of the detection subsystem 104 and their functionalities are described in further detail below with reference to FIG. 3.

[0054] The identification subsystem 106 receives detection data from the detection subsystem 104 and maintains the identity data structure identifying objects of interest across the plurality of camera views of the video feeds. The identity data structure maintained by the identification subsystem 106 includes current and historical identity information for the objects of interest over the time course of the video feeds. The identification subsystem 106 combines contextual understanding from a transformer architecture with explicit memory mechanisms for long-horizon identity persistence, storing identity embeddings, physical characteristics, and accumulated identity evidence in a hierarchical external memory that persists beyond the transformer's native attention window. The identification subsystem 106 creates persistent object signatures by fusing visual appearance features, spatio-temporal trajectory information, team affiliation indicators, role and position characteristics, and interaction patterns into unified multi-modal object signatures that remain discriminative even when individual feature components are transiently unavailable. The identification subsystem 106 maintains identity probability distributions over possible identities for each tracked object of interest, rather than making deterministic assignments, preserving multiple hypotheses when identity is ambiguous and sharpening the distributions as further evidence accumulates. The subcomponents of the identification subsystem 106 and their functionalities are described in further detail below with reference to FIG. 4.

[0055] The bidirectional identity propagator 108 implements the back-propagation capability of the identity analysis system 100. The bidirectional identity propagator 108 continuously monitors for changes to the current identity information and, upon a change to the current identity information for a given object of interest, updates the historical identity information forward and backward over the time course of the video feeds. The bidirectional identity propagator 108 maintains a temporal identity graph that represents the complete history of object detections, tracking associations, and identity assignments throughout the sportingevent, as described in further detail with reference to FIG. 6 and FIG. 7. When new identity evidence arrives, such as a jersey number recognition, a facial recognition match, or a broadcast graphic confirmation, the bidirectional identity propagator 108 evaluates the significance of the evidence, inserts it into the temporal identity graph, and traverses the graph both forward and backward from the point of evidence insertion. Forward propagation updates identity assignments for subsequent time steps; while backward propagation retroactively corrects identity assignments for preceding time steps that were uncertain or incorrect before the new identity evidence became available.

[0056] The output interface 110 transmits a signal encoding data of the identity data structure through the communication interface to downstream consumers. The output interface 110 provides downstream applications with access to the identity-resolved tracking data produced by the identification subsystem 106 and updated by the bidirectional identity propagator 108. The output interface 110 expresses confidence measures for each identity assignment, enabling downstream applications to appropriately handle uncertain identities. When identity is confident, statistics are attributed deterministically to the identified player; when identity is uncertain, statistics may be attributed proportionally to the possible identities according to their probabilities. By expressing both the identity-resolved data and the associated confidence measures through the communication interface, the output interface 110 enables downstream consumers to produce analytics that incorporate identity uncertainty rather than presenting potentially incorrect deterministic attributions.

[0057] Electronic datastore 112 includes a non-transitory computer-readable storage medium that stores data and machine-interpretable instructions used by various components of the identity analysis system 100. The electronic datastore 112 stores, for example, the plurality of artificial intelligence models disclosed herein, including, without limitation, the per-camera object detector 302, the identity feature extractor 304, the hybrid memory-augmented transformer 402, the multi-modal signature generator 404, and the probabilistic identity tracker 406, and their associated trained model weights; sport-specific profiles encapsulating the plurality of adaptation parameters for each supported sport, including detection model configurations with class sets, confidence thresholds, and spatial masks, tracking parameters, substitution rules and constraints defining entry and exit points, positional expectations based on tactical formations, game phase definitions that influence identity reasoning, and expected numbers of entities of each type; calibration data and camera parameters used by the multi-camera registrator 306; and configuration settings for the video input interface 102 and output interface 110 including supported streaming protocol configurations for RTSP, RTMP, and SRT.

[0058] The electronic datastore 112 may be implemented as one or more of local solid-state storage, network-attached storage, or cloud-based storage accessible via the communication network 50. In the depicted embodiment, at least part of electronic datastore 112 is organized as a database. Such database may be a conventional relational database such as a SQL-based database, an object-oriented database, a NoSQL database, or the like.

[0059] In some embodiments, the video input interface 102, the detection subsystem 104, the identification subsystem 106, the bidirectional identity propagator 108, and the output interface 110 may be implemented using a suitable combination of software and hardware components. Software components may be implemented using conventional programming languages such as Java, J#, C, C++, C#, Perl, Visual Basic, Ruby, Scala, etc. These software components may be in the form of one or more executable programs, scripts, routines, statically / dynamically linkable libraries, or the like.

[0060] FIG. 3 is a schematic diagram of an example detection subsystem 104 within the identity analysis system, in accordance with an embodiment. The detection subsystem 104 includes a per-camera object detector 302, an identity feature extractor 304, and a multi-camera registrator 306.

[0061] The per-camera object detector 302 processes video frames from each of the plurality of video feeds and detects the plurality of objects of interest across the video feeds using trained object detection models optimized for sports environments. The per-camera object detector 302 identifies and localizes the objects of interest within each frame, where the objects of interest may include at least one of player, official, coach, equipment, or sport-specific objects within video frames of the video feeds. For each detected object, the per-camera object detector 302 extracts spatial features such as bounding box positions, size, and estimated body pose, and produces per-frame detections with class labels and confidence scores. Detection models deployed by the per-camera object detector 302 may operate on batched frames for throughput optimization and may employ sport-specific detection configurations such as class sets, confidence thresholds, and spatial masks that restrict detection to the playing surface ( / '.e. , the specific court, rink, field, or pool of interest).

[0062] The identity feature extractor 304 extracts identifying features including at least one of jersey number recognitions, facial recognition embeddings, team color classifications, body morphology descriptors, or uniform detail patterns from the detected objects of interest when such features are visible with sufficient quality. The identity feature extractor 304 operates opportunistically by extracting the identifying features whenever they are available but does not require their presence for basic tracking by the detection subsystem 104. For each extracted identifying feature, the identity feature extractor 304 produces a confidence score reflecting the estimated reliability of the extraction given the current viewing conditions. For example, a clearly visible jersey number observed from a close-up camera produces a higher confidence score than a partially visible number observed from a distant camera.

[0063] Embodiments of the identity feature extractor 304 operates through several complementary mechanisms. In one example mechanism, the identity feature extractor 304 applies quality-gated extraction, in which computationally expensive feature extraction processes — such as jersey number optical character recognition or facial recognition embedding generation — are triggered only when frame-level quality metrics at the detected object location, including resolution, occlusion degree, viewing angle, and motion blur, exceed modality-specific thresholds. For example, jersey number recognition may require a minimumbounding box height of 80 pixels and an estimated occlusion degree below 30%, while facial recognition embedding may additionally require a frontal or near-frontal viewing angle.

[0064] In another example mechanism, the identity feature extractor 304 implements camera-priority scheduling, in which close-up and high-resolution cameras are prioritized for identity feature extraction while wide-angle cameras contribute primarily spatial and motion features, reflecting the differing utility of each camera type for identity-discriminative feature extraction.

[0065] In yet another example mechanism, the identity feature extractor 304 recognizes natural extraction opportunity windows during which players are stationary and cameras provide stable, high-quality views — such as game stoppages, free throws, corner kicks, faceoffs, and penalty situations — and intensifies feature extraction during these periods.

[0066] In yet another example mechanism, the identity feature extractor 304 maintains a per-entity feature recency index that tracks when each feature modality was last successfully extracted for each tracked entity, and prioritizes extraction for entities whose identifying features are stale or have never been successfully extracted, thereby progressively building a complete feature profile for each entity over the course of the sporting event.

[0067] The multi-camera registrator 306 establishes the geometric relationship between each camera's image coordinate system and a unified spatial coordinate system. The multi-camera registrator 306 maps positions of detected objects of interest across the camera views to the unified spatial coordinate system, which in some embodiments is a ground-plane coordinate system aligned with the playing field. Using known or estimated camera parameters such as focal length, lens distortion ( / .e., intrinsic parameters), position, and orientation (7.e., extrinsic parameters), the multi-camera registrator 306 projects detected object positions from each camera view into the unified spatial coordinate system. This projection by the multicamera registrator 306 enables cross-view association of detections: when the same physical entity is visible in multiple cameras simultaneously, the projected positions in the unified spatial coordinate system will be spatially proximate, providing a strong association signal independent of appearance.

[0068] The multi-camera registrator 306 handles both static cameras with fixed calibration and dynamic scenarios where camera parameters may change, such as with PTZ cameras or cameras with automatic focus. For static installations, calibration is performed by the multi-camera registrator 306 during setup and refined periodically. For dynamic cameras, the multi-camera registrator 306 employs continuous calibration refinement using detected field markings, known object sizes, and other visual references to maintain accurate spatial projection.

[0069] FIG. 4 is a schematic diagram of an example identification subsystem 106 within the identity analysis system, in accordance with an embodiment. The identification subsystem 106 includes a hybrid memory-augmented transformer 402, a multi-modal signature generator 404, and a probabilistic identitytracker 406. These components of the identification subsystem 106 operate in concert, each contributing identity evidence that is fused into a unified identity representation for each tracked object of interest.

[0070] The hybrid memory-augmented transformer 402 combines contextual understanding from a transformer architecture with explicit memory mechanisms for long-horizon identity persistence, enabling identity tracking across durations that exceed the transformer's native attention window. The hybrid memory-augmented transformer 402 communicates with a non-transitory memory that stores identity information to be persistent beyond an attention window of the hybrid memory-augmented transformer 402. The hybrid memory-augmented transformer 402 is described in further detail with reference to FIG. 5.

[0071] The multi-modal signature generator 404 creates persistent object signatures by fusing visual, spatial, temporal, and contextual features from the plurality of camera views. The multi-modal signature generator 404 generates a multi-modal signature that combines at least two of appearance features, spatiotemporal trajectory information, team affiliation indicators, role and position characteristics, and interaction patterns. The multi-modal signature generator 404 is described in further detail below with reference to FIG.8.

[0072] The probabilistic identity tracker 406 maintains identity probability distributions over possible identities for each tracked object of interest, rather than making deterministic identity assignments. The identity information for the objects of interest maintained by the probabilistic identity tracker 406 includes, for each object of interest, an identity probability distribution of possible identity associations for that object of interest. The probabilistic identity tracker 406 additionally associates each item of the identity information with a confidence metric reflecting a certainty in that item of the identity information. The probabilistic identity tracker 406 is described in further detail below with reference to FIG. 10.

[0073] FIG. 5 is a block diagram illustrating an example architecture of the hybrid memory-augmented transformer 402, in accordance with an embodiment. The hybrid memory-augmented transformer 402 receives a multi-camera detection stream 502 from the detection subsystem 104 and produces an identity-resolved tracking output 590. The hybrid memory-augmented transformer 402 includes a multi-modal identity embedding mechanism 510, a feature fusion mechanism 520, and a memory consolidation process 550, operating together within a transformer architecture that includes multi-head self-attention and feed-forward layers.

[0074] The multi-modal identity embedding 510 creates robust identity representations by fusing multiple information sources, including but not limited to visual appearances, motion patterns, positional data, identity markers, and game context. Visual appearance features — extracted by convolutional or vision transformer networks from the detected object's image crop — capture the entity's visual presentation including uniform appearance, exposed skin tone, hair characteristics, and equipment details. Motion pattern features capture characteristic movement dynamics such as gait, running style, positional tendencies, and interaction patterns. Positional data encodes the entity's location within the playing field in the unified spatial coordinatesystem. Identity markers such as jersey number recognitions and facial recognition embeddings provide direct identity evidence when available. Contextual game knowledge, including the current game phase, tactical formation, and expected positions, provides prior information about which identity is likely at which location. The multi-modal identity embedding 510 fuses the above information sources into a unified embedding, creating identity representations that remain discriminative even when individual feature components are temporarily unavailable or unreliable.

[0075] The feature fusion mechanism 520 within the hybrid memory-augmented transformer 402 integrates the multi-modal identity embedding 510 outputs with information from an external memory architecture 560. The external memory architecture 560 stores, for each tracked entity, an identity embedding that encodes the system's current understanding of the entity's identity, a set of physical characteristic descriptors (e.g., estimated height, build, team affiliation, position tendencies), and accumulated identity evidence (e.g., jersey number observations, facial recognition matches, behavioral signatures). The non-transitory memory implements a memory architecture providing a short-term buffer for recent identity information with full detail, a medium-term store for session-level identity information with moderate compression, and a long-term store for identity information that spans a length of time up to the sporting event. A memory attention mechanism enables the transformer to selectively read from and write to the external memory based on the current identity processing context, retrieving relevant identity information for the entities currently under consideration. Write operations update the memory with new identity observations and refined identity estimates.

[0076] The memory consolidation process 550 progressively transfers identity information from the shortterm buffer to the long-term store as identity confidence accumulates, while pruning low-confidence entries that have not been reinforced by subsequent observations. An item of identity information is transferred to long-term storage when confidence in that item exceeds a threshold. The memory consolidation 550 process subsequently removes entries from the short-term buffer that have been consolidated into longer-term stores, thus freeing up short term memory capacity. The memory consolidation process 550 thereby influences the feature fusion mechanism 520 by determining the content and structure of the external memory architecture 560 from which the feature fusion mechanism 520 retrieves identity information via the memory attention mechanism.

[0077] The hybrid memory-augmented transformer 402 further includes a temporal graph attention network that represents the objects of interest as nodes and their potential identity associations across the time course and the camera views as edges. At each time step, nodes correspond to currently tracked objects of interest across all cameras, and edges connect nodes that may represent the same physical entity, which can be either the same object observed in consecutive frames within a camera, or the same object observed simultaneously in different cameras, or an object that has disappeared from one camera view and reappeared in another after an occlusion or camera transition. The attention mechanism weights edgesbased on spatial proximity in the unified spatial coordinate system, temporal recency, appearance similarity based on the multi-modal identity embedding 510, and contextual consistency. The temporal graph attention network resolves identity ambiguities by propagating identity information through the graph, leveraging collective evidence from multiple time steps and camera views rather than making identity decisions from individual observations in isolation. An example temporal identity graph structure is described in further detail below with reference to FIG. 6.

[0078] The identity-resolved tracking output 590 produced by the hybrid memory-augmented transformer 402 includes the objects of interest with their associated identity probability distributions. The hybrid memory-augmented transformer 402 implements uncertainty-aware identity management by maintaining probability distributions over possible identities for each tracked object of interest rather than making deterministic hard assignments. When a new detection is associated with an existing track, the hybrid memory-augmented transformer 402 updates the identity probability distribution for that track using the new observation as evidence. When identity is ambiguous — for example, after two players from the same team cross paths and separate — the hybrid memory-augmented transformer 402 maintains multiple identity hypotheses, each with an associated probability, rather than committing prematurely to a single assignment that may be incorrect. As further evidence accumulates (e.g., a jersey number becomes visible, one player moves to an expected position, occlusion is resolved after player movement), the probability distribution sharpens toward the correct identity. In some embodiments, this soft assignment approach may reduce cascading errors characteristic of hard-assignment systems, where an incorrect identity commitment propagates through subsequent frames.

[0079] For example, during a basketball game, two teammates cross paths near the three-point line, causing temporary tracking ambiguity. The hybrid memory-augmented transformer 402 maintains two hypotheses for each resulting track (Player A continuing left and Player B continuing right, or vice versa), with probabilities reflecting the motion and appearance evidence. When one player subsequently takes a free throw and a close-up camera captures a clear jersey number, the system confirms the identity for that track with high confidence, uses the mutual exclusivity constraint to resolve the other track's identity, updates the external memory with the confirmed identities, and the bidirectional identity propagator 108 retroactively corrects any uncertain assignments from the period of ambiguity. The identity-resolved tracking output 590 for both players reflects complete, identity-resolved tracking data throughout the ambiguous period, despite the delayed availability of the confirming evidence.

[0080] FIG. 6 is a diagram illustrating an example temporal identity graph structure, which underlies the bidirectional identity propagator 108, in accordance with an embodiment.

[0081] The temporal identity graph structure is a directed graph with temporal ordering, representing the complete history of object detections, tracking associations, and identity assignments for all tracked objects of interest throughout the sporting event. Nodes in the temporal identity graph represent individual objectdetections at specific time steps in specific cameras, and directed edges represent tracking associations between consecutive detections within a camera or video feed (labeled "T rack") and cross-view associations between simultaneous detections across cameras (labeled "Cross-view"). FIG. 6 depicts the example temporal identity graph structure across five successive time steps labeled 610, 620, 630, 640, and 650, corresponding to times T1 through T5 respectively. Two tracked objects of interest — a first detection (Det A) and a second detection (Det B) — are tracked across these five time steps, with a second camera (Cam 2) providing additional cross-view observations at certain time steps.

[0082] At time step 610 (Ti), the example temporal identity graph structure shows a first detection Det A in Cam 1 and a second detection Det B in Cam 1. Simultaneously, entity A is also detected by Cam 2. A Cross-view edge connects Det A in Cam 1 to Det A in Cam 2 at time step 610, indicating that the cross-view identity association mechanism has determined that the two detections from different cameras correspond to the same physical entity, using the geometric projection, appearance matching, and motion consistency criteria described further below with reference to FIG. 9.

[0083] The two detected entities persist into time step 620 (T2), where Track edges connect Det A and Det B at time step 610 to establish that each detection is being tracked as a distinct object of interest over time, maintaining tracking continuity across consecutive frames within the same cameras. At time step 630 (T3), Det A in Cam 1 and Det B in Cam 1 are again present, however, entity A is no longer detected on Cam 2, indicating potential occlusion of the entity from Cam 2's point of view. Track edges continue to link the detections at time step 630 back to those at time steps 620 and 610.

[0084] At time step 640 (T4), the temporal identity graph structure receives new identity evidence: jersey #7 is recognized on Det A in Cam 1, triggering the bidirectional identity propagator 108. The identity resolution pipeline 700 (described further below with reference to FIG. 7) inserts this high-confidence jersey number evidence into the temporal identity graph at the node corresponding to Det A in Cam 1 at time step 640.

[0085] The identity evidence triggers a propagation decision, which determines that jersey #7 recognition constitutes evidence with a significant expected impact, warranting a full bidirectional propagation. Backward propagation traverses the temporal identity graph from time step 640 back through time steps 630, 620, and 610, updating the identity assignments for Det A in Cam 1 at each preceding time step to reflect the confirmed identity of Player #7. At time steps 620 and 610, the Cross-view edge connecting Det A in Cam 1 to Det A in Cam 2 enables the backward propagation to update the identity of Det A in Cam 2 as well, confirming that the detection in the second camera also corresponds to Player #7. Forward propagation traverses from time step 640 to time step 650 (T5), maintaining the confirmed Player #7 identity through subsequent tracking associations. At time step 650, Det A in Cam 1 and Det B in Cam 1 continue to be tracked, with the Player #7 identity propagated through all forward detections of entity A. In some embodiments, a mutual exclusivity constraint ensures that no other simultaneous track is assigned the identity of Player #7, and a rosterconstraint confirms that jersey number 7 does in fact exist and additionally corresponds to a specific named player on the team roster. In parallel, entity B continues to be tracked as unidentified Det B, pending emergence of future identity evidence.

[0086] FIG. 6 illustrates how the temporal identity graph structure enables the system to update the historical identity information forward and backward over the time course of the video feeds. Before the jersey #7 recognition at time step 640, the identity assignments for Det A across time steps 610, 620, and 630 reflected uncertainty — the system tracked entity A as a distinct object of interest but had not yet confirmed which specific player Det A represented. After the jersey #7 recognition, backward propagation through the temporal identity graph retroactively corrects those earlier identity assignments, and forward propagation ensures the confirmed identity persists in subsequent time steps. The cross-view edges at time steps 610 and 620 demonstrate that identity evidence propagates not only through within-camera Track edges but also through inter-camera cross-view associations, so that a detection confirmed in one camera is simultaneously resolved in all other cameras observing the same entity.

[0087] For example, during a soccer match, a player may be initially tracked without a confirmed identity for twenty minutes due to the jersey number being obscured from the available camera angles. When the player takes a corner kick and a close-up camera clearly captures jersey number 7, the identity resolution pipeline 700 inserts this evidence at the corresponding node of the graph structure, executes backward propagation through the preceding twenty minutes of tracking data, enforces the roster constraint and the mutual exclusivity constraint, and updates the multi-resolution identity cache. As a result, performance statistics, heat maps, and tactical analysis for this player may be available for the full match, including the period before the jersey number was confirmed.

[0088] FIG. 7 is a block diagram illustrating an example retroactive identity resolution pipeline 700, in accordance with an embodiment.

[0089] The retroactive identity resolution pipeline 700 enables the back-propagation capability of the bidirectional identity propagator 108. New identity evidence 702 enters the pipeline as input, where the evidence may comprise one or more of a jersey number recognition (such as the example illustrated in FIG.6), a facial recognition match, a broadcast graphic confirmation, or a roster-based inference. An evidence significance estimator 710 evaluates the potential impact of the new identity evidence 702 before triggering propagation. The evidence significance estimator 710 determines whether the new identity evidence 702 is expected to materially alter existing identity assignments and therefore warrant propagation. For example, when the evidence significance estimator 710 determines that the new identity evidence 702 confirms an already confident assignment, only a localized update is triggered; however, when the new identity evidence 702 contradicts a previous assignment or resolves an uncertain identity, the propagation decision 720 triggers full bidirectional propagation.

[0090] The retroactive identity resolution pipeline includes a backward propagation path 722 and a forward propagation path 724. The backward propagation 722 updates identity assignments for preceding time steps, propagating the new identity evidence 702 through reverse tracking associations in the temporal identity graph to retroactively correct historical assignments throughout a set of video streams that were uncertain or incorrect. The forward propagation 724 updates identity assignments for subsequent time steps where the same entity is tracked, which corresponds to updating the identity information forward over the time course. The propagation algorithm attenuates the influence of the new identity evidence 702 as it propagates further from the insertion point, reflecting decreasing certainty that temporally distant detections correspond to the same entity. The attenuation rate depends on the quality of the intervening tracking associations, namely, identity propagates with less attenuation through chains of high-confidence tracking associations and attenuates more rapidly through chains with weak associations or gaps. In some embodiments, as the video coverage progresses, new identity evidence with a corresponding confidence level is added in real time and dynamically influences subsequent propagation treatment of the resulting identification at propagation decision point 720.

[0091] During propagation, the retroactive identity resolution pipeline 700 enforces identity constraints satisfaction 730 to maintain physical and game-rule consistency. For example, mutual exclusivity constraints prevent the assignment of the same physical identity to multiple simultaneous tracks, enforcing that a single physical entity cannot be in two places at once; roster constraints bound the set of possible identities to the known team rosters, preventing assignment of identities not participating in the sporting event; entry / exit event constraints incorporate information about player substitutions, removing substituted players from the set of possible on-field identities after the substitution point and adding incoming players; and position constraints encode sport-specific spatial expectations — for example, an identified goalkeeper is expected to remain near the goal area.

[0092] A change propagation optimization mechanism 750 minimizes computational resources during identity updates by determining which portions of the temporal identity graph require reconsideration based on the significance and reach of the new identity evidence 702, avoiding reprocessing of unaffected portions. The change propagation optimization mechanism 750 estimates a propagation boundary representing the temporal extent beyond which the new identity evidence 702 is unlikely to alter existing identity assignments, and limits propagation to within this boundary. In some embodiments, the propagation boundary is estimated by the change propagation optimization mechanism 750 based on the current confidence levels and tracking association quality in the temporal identity graph.

[0093] In some embodiments, the retroactive identity resolution pipeline maintains a multi-resolution identity cache 740 at multiple temporal resolutions for efficient access and updates: frame-level data for recent portions of the sporting event, segment-level data for intermediate history, and event-level data for long-term patterns. The result is updated identity-resolved tracking data 790 that reflects the correctedidentity assignments. In some embodiments, the updated identity-resolved tracking data 790 is provided to the output interface 110 for transmission through the communication interface to downstream consumers.

[0094] As an example, during a basketball game, a substitution occurs but the incoming player's identity is initially uncertain. The system detects the substitution event (one player leaves, one enters), updates the roster constraint to remove the departing player and add possible incoming players, and initially assigns a probability distribution over the plausible incoming identities. When a close-up camera subsequently captures the new player's number with a high degree of confidence, the system retroactively resolves the identity from the point of substitution entry, using the backward propagation 722 path and forward propagation 724 path to correct any uncertain attributions from the period between substitution and confirmation. Embodiments of the updated identity-resolved tracking data 790 may reflect complete identity resolution for the substituted player and would be updated for the remainder of the duration that the substituted player is on the court.

[0095] FIG. 8 is a block diagram illustrating an example multi-modal feature association system 800, in accordance with an embodiment. The multi-modal feature association system implements the multi-modal signature generator 404 described with reference to FIG. 4.

[0096] A per-detection feature extraction module 810 extracts multiple feature modalities from each detected object of interest. In some embodiments, the feature modalities comprise appearance features, spatio-temporal trajectory, team affiliation, role & position, and interaction patterns. Appearance features are extracted by vision models (convolutional networks or vision transformers) from the object's image crop, capturing visual characteristics such as uniform appearance, visible numbers or markings, body proportions, hair and skin features, and equipment details. Spatio-temporal trajectory features encode the object's movement history in the unified spatial coordinate system, capturing position, velocity, acceleration, and movement patterns. Team affiliation features classify the object's team membership based on uniform color analysis, enabling the fundamental discrimination between players on opposing teams. Role & position characteristics capture the object's behavioral tendencies, including positional heatmap, typical movement patterns, and interaction frequencies with specific field zones. Interaction patterns capture the object's relationships with other tracked entities, such as clustering behavior, passing relationships, and defensive matchup patterns. The per-detection feature extraction 810 generates a multi-modal signature of at least two of the feature modalities above, combining the separate features into a unified component.

[0097] A confidence-weighted feature fusion mechanism 820 dynamically adjusts the weighting of different feature components produced by the per-detection feature extraction 810 based on their estimated reliability under current conditions. The confidence-weighted feature fusion 820 increases reliance on spatial and motion features when visual appearance is degraded (due to distance, partial occlusion, or unfavorable viewing angle) and increases reliance on appearance features when spatial information is ambiguous (such as during player clustering). In some embodiments, feature quality indicators used by the confidence-weighted feature fusion 820 include detection confidence, image resolution at the object location, occlusion degree, and viewing angle. The confidence-weighted feature fusion 820 produces a unified multi-modal object signature 830 that persists even when individual feature components are transiently unavailable. In some embodiments, the unified multi-modal object signature 830 may enable robust identity maintenance through challenging tracking conditions, such as during a live sports broadcasting setting.

[0098] The multi-modal feature association system 800 further includes a temporal signature evolution framework 850 that models how the unified multi-modal object signature 830 changes over time, distinguishing between expected appearance drift (e.g., gradual changes due to perspiration, lighting changes, accumulated dirt, viewing angle variations) and identity-breaking changes indicating a completely different entity. The temporal signature evolution framework 850 maintains an expected appearance drift model of each object's signature evolution, predicting expected appearance at future time steps and tolerating drift within expected bounds while flagging unexpected changes for identity verification. This temporal modeling enables robust re-identification after extended occlusions — when an object of interest reappears after a period of invisibility, the system compares its current appearance not against the last observed appearance, which may be outdated, but against the predicted current appearance accounting for expected temporal evolution.

[0099] In some embodiments, context-aware identity disambiguation 860 resolves identity ambiguities by incorporating game-specific context. For example, when two players from the same team are in close proximity and their visual features are insufficient for visual discrimination, the system leverages contextual information such as the tactical formation being played (which implies expected positions for specific roles), the game phase (which constrains likely player locations — attackers in the offensive zone, defenders in the defensive zone), historical movement patterns specific to each player identity, and sport-specific spatial constraints. This contextual reasoning provides identity evidence that is independent of visual appearance, enabling discrimination in scenarios where appearance-only methods could fail.

[0100] As depicted in FIG. 8, embodiments of the multi-modal feature association system 800 may additionally employ cross-view association 920 and one-to-one assignment optimization 930, which are described below with reference to FIG. 9.

[0101] A bidirectional feature propagation mechanism 870 propagates newly discovered identifying features both forward and backward through the time course of the video feeds. In the forward direction, the bidirectional feature propagation 870 updates future matching by enriching the unified multi-modal object signature 830 with the new features. In the backward direction, the bidirectional feature propagation 870 resolves past ambiguities by propagating the new features to earlier time steps. The bidirectional feature propagation 870 produces persistent identity-resolved signatures 890 that reflect the most current understanding of each tracked object's identity, incorporating all discovered features across the entire timecourse. In some embodiments, the persistent identity-resolved signatures 890 are provided to the identification subsystem 106 for fusion with other identity evidence.

[0102] FIG. 9 is a block diagram illustrating an example cross-view identity association pipeline 900, in accordance with an embodiment. The cross-view identity association pipeline 900 matches object signatures across the plurality of camera views to establish that detections in different cameras correspond to the same physical entity ( / .e., a cross-view edge between nodes of the temporal identity graph).

[0103] FIG.9 shows an example multi-camera configuration including a first camera (Cam 1, wide angle) capturing detections Det Ai, Det Bi, and Det Ci , a second camera (Cam 2, sideline) capturing detections Det A2 and Det B2, and a third camera (Cam 3, close-up) capturing detection Det A3. A multi-camera calibration stage 910 projects detected object positions from each camera view into the unified spatial coordinate system using camera calibration parameters (intrinsic and extrinsic), such as values indicating focal length, lens distortion, camera position, and camera orientation. The multi-camera calibration 910 establishes ground-plane coordinates aligned with the court / field / rink / pool, enabling the system to map positions of detected objects of interest across the camera views to the unified spatial coordinate system. The multi-camera calibration 910 may be a coordinate-specific embodiment of the unified multi-modal object signature 830 as depicted in FIG. 8.

[0104] In some embodiments, the cross-view identity association mechanism 920 employs three complementary association criteria: geometric projection matching measures spatial proximity between detections projected into the unified spatial coordinate system by the multi-camera calibration 910, determining whether projected positions fall within a threshold distance; appearance feature matching measures similarity between the multi-modal identity embeddings extracted from each camera's view of the detected objects of interest; and motion trajectory consistency verifies that the motion vectors of the matched detections are consistent across camera views. Identity associations that satisfy all three criteria receive high confidence; associations that satisfy only a subset receive proportionally lower confidence.

[0105] In some embodiments, a one-to-one assignment stage 930 enforces the constraint that each detection in one camera can be matched to at most one detection in another camera. The one-to-one assignment 930 uses an assignment algorithm that maximizes global association quality across all camera pairs, preventing duplicate matches. The one-to-one assignment 930 produces a unified entity representation 990 for each associated object of interest. The unified entity representation 990 combines the best features from all observing cameras into a single cross-view fused identity signature with a complete spatial and appearance profile. Embodiments of the unified entity representation 990 enable the identification subsystem 106 to reason about each physical entity using complementary information from all available camera views. For example, the availability of a clear identification in one camera may improve identity assignments across other cameras observing the same entity via cross-view association.

[0106] For example, during a soccer match captured by eight cameras, a player moves between multiple camera views as play develops across the field. The system projects the player's detected position from each camera into the unified spatial coordinate system via the multi-camera calibration 910, identifies that detections at approximately the same position in different cameras correspond to the same entity, and associates their signatures through the one-to-one assignment. The unified entity representation 990 for this player combines appearance features from the highest-resolution camera view, spatial trajectory from the continuous position stream, and team affiliation from uniform color analysis. When the player's jersey number becomes clearly visible in a close-up camera, the high-confidence identification propagates to all camera views and through the temporal history via the bidirectional identity propagator 108, creating consistent identification across the entire field and throughout the game duration.

[0107] FIG. 10 is a block diagram illustrating an example probabilistic identity propagation process 1000, in accordance with an embodiment. The probabilistic identity propagation process 1000 implements the probabilistic identity tracker 406 described with reference to FIG. 4.

[0108] The probabilistic identity tracker 406 represents the identity of each tracked object of interest as a probability distribution over possible identities rather than a deterministic assignment. For a team sport with N players per team, each tracked object of interest maintains a probability vector of length 2N (for both teams) plus entries for officials, coaches, and unidentified entities. The identity probability distribution reflects the system's current belief about the object's identity given all evidence observed so far. High-entropy distributions, where probability is spread across multiple identities, indicate uncertain identity. Low-entropy distributions, where probability is concentrated on one identity, indicate confident identity. This current belief is illustrated by the initial state 1004, where each detected object is associated with its identity distribution.

[0109] When new identity evidence 1002 becomes available, a Bayesian update 1010 mechanism updates the identity probability distribution using Bayesian inference. The Bayesian update 1010 characterizes the new identity evidence 1002 by a likelihood function that specifies the probability of observing this evidence given each possible identity — for example, observing jersey number "23" has high likelihood given the identity of the player who wears number 23 and low likelihood given other identities. The Bayesian update 1010 combines this likelihood with the prior identity distribution from the initial state 1004 to produce an updated identity distribution 1012 (the posterior belief incorporating the evidence). The Bayesian update 1010 incorporates evidence reliability, ensuring that the strength of the update reflects the quality of the evidence. For example, a clearly visible jersey number observed from a close-up camera produces a sharper likelihood function than a partially visible number observed from a distant camera.

[0110] In some embodiments, the probabilistic identity propagation process 1000 maintains multiple tracking and identity hypotheses simultaneously in a multi-hypothesis management module 1020 when the tracking situation is ambiguous. Hypothesis generation creates multiple hypotheses representing different possible identity assignments when an identity-switching event may have occurred, such as two playerscrossing paths or a player emerging from a group. Parallel propagation propagates each hypothesis forward through time. Strategic pruning manages computational complexity by eliminating hypotheses whose cumulative probability falls below a configurable threshold, while preserving hypotheses that remain plausible until sufficient evidence accumulates. The pruning threshold may be adaptive, tightening when computational resources are constrained and relaxing when more resources are available.

[0111] The system resolves identities across multiple tracks and camera views jointly rather than independently. Joint identity resolution 1030 enforces global constraints through probabilistic inference, such as the constraint that a single physical entity cannot be assigned to multiple simultaneous tracks. In some embodiments, the joint identity resolution 1030 normalizes identity probability distributions across all active tracks so that the total probability assigned to each identity does not exceed the expected number of instances of that identity (e.g., one, for individual players). In some embodiments, this joint resolution may prevent the failure mode where two tracks are each independently assigned the same identity with moderate confidence, which would be flagged by the joint resolution mechanism as a violation of the mutual exclusivity constraint.

[0112] In some embodiments, retroactive probability propagation 1040 propagates newly discovered identity evidence backward through time using the Bayesian update mechanism 1010 applied in reverse temporal order. At each preceding time step, the new identity evidence serves as additional information that the system incorporates into the historical identity probability distribution, producing corrected distributions that reflect the hindsight knowledge of the later evidence. The backward propagation respects the same constraints and attenuation as forward propagation, producing probability updates that diminish as the temporal distance from the evidence increases and the intervening tracking associations weaken. The result is corrected identity probability histories produced without requiring complete reprocessing of the source video.

[0113] The probabilistic identity tracker 406 further includes a joint identity resolution mechanism 1030 that monitors the entropy of identity probability distributions across active tracks and triggers focused resolution processing when distribution entropy exceeds configurable thresholds. This mechanism concentrates computational resources on the most uncertain identity assignments, ensuring that the system applies additional identity resolution effort where it is most needed rather than uniformly across all tracked objects of interest.

[0114] Confidence-based analytics 1050 provides downstream analytics that incorporate identity uncertainty. When the identity is confident (the updated identity distribution 1012 has low entropy), statistics are attributed deterministically to the identified player through deterministic attribution. When identity is moderately confident, statistics are attributed proportionally to the possible identities according to their probabilities through proportional attribution — for example, if a detection is 70% likely to be Player A and 30% likely to be Player B, a shot from that detection is attributed 0.7 shots to Player A and 0.3 shots to PlayerB. This probabilistic attribution prevents the all-or-nothing error of deterministic systems, where the entire shot would be attributed to whichever player the system estimated, potentially incorrectly, and provides honest uncertainty quantification for downstream consumers. When identity is uncertain (the updated identity distribution 1012 has high entropy), statistics may be withheld with a confidence interval. The confidencebased analytics 1050 thus associates with each item of the identity information a confidence metric reflecting a certainty in that item of the identity information, enabling downstream consumers to appropriately interpret data from periods of identity uncertainty.

[0115] For example, during a basketball game, two similarly-appearing teammates are initially confused after crossing paths. The system maintains identity probability distributions: Track 1 is 55% Player A and 45% Player B; Track 2 is 45% Player A and 55% Player B. Statistics during this uncertain period are attributed proportionally to both players via the confidence-based analytics 1050. Three minutes later, when Player A takes a free throw and the camera captures a clear jersey number, the Bayesian update 1010 sharply concentrates Track Ts distribution on Player A (now 99% Player A). The mutual exclusivity constraint forces Track 2's distribution to concentrate on Player B (now 99% Player B). Retroactive probability propagation updates the distributions for the three-minute uncertain period, and the confidence-based analytics 1050 retroactively re-attributes statistics to reflect the corrected identities. The final output shows continuous, identity-resolved tracking data for both players throughout the ambiguous period, with appropriate confidence measures that indicate the period was initially uncertain and was subsequently resolved through back-propagation.

[0116] FIG. 11 is a flowchart of an example method 1100 for identity analysis in sports video, in accordance with an embodiment.

[0117] The method 1100 begins at step 1102 with receiving a plurality of video feeds of a sporting event. The plurality of video feeds may originate from cameras 80 and / or raw footage stream sources 90 described with reference to FIG. 1 and are received by the video input interface 102 of the identity analysis system 100. At step 1104, upon processing data from the plurality of video feeds, the method 1100 includes detecting a plurality of objects of interest across the video feeds. Detecting the plurality of objects of interest at step 1104 may comprise detecting at least one player, official, coach, equipment, or sport-specific objects within video frames of the video feeds, as performed by the per-camera object detector 302 of the detection subsystem 104.

[0118] At step 1106, the method 1100 includes maintaining an identity data structure that stores data identifying the objects of interest across a plurality of camera views of the video feeds. The identity data structure maintained at step 1106 includes current and historical identity information for the objects of interest over a time course of the video feeds. The maintaining at step 1106 may utilize the hybrid memory-augmented transformer 402, the multi-modal signature generator 404, and the probabilistic identity tracker406 of the identification subsystem 106. The maintaining at step 1106 may include updating the identity probability distributions for the objects of interest as new evidence is observed.

[0119] At step 1108, upon a change to the current identity information for a given object of interest of the plurality of objects of interest, the method includes updating the historical identity information forward and backward over the time course of the video feeds. The updating at step 1108 may include traversing the temporal identity graph described with reference to FIG. 6 and FIG.7. The updating at step 1108 may further include updating the identity probability distributions by applying a Bayesian update in reverse temporal order, as described with reference to FIG. 10. At step 1110, the method includes transmitting a signal encoding data of the identity data structure through a communication interface. The transmitting at step 1110 may be performed by the output interface 110 of the identity analysis system 100, providing the identity-resolved tracking data to downstream consumers including statistics engines, broadcast graphics systems, highlight generators, and coaching analytics platforms.

[0120] It should be understood that steps of one or more of the blocks depicted in FIG. 11 may be performed in a different sequence or in an interleaved or iterative manner. Further, variations of the steps, omission or substitution of various steps, or additional steps may be considered.

[0121] FIG. 12 is a block diagram of an example real-time processing pipeline 1200 from multi-camera input through detection, tracking, identity assignment, retroactive resolution, and analytics output, in accordance with an embodiment. The real-time processing pipeline 1200 transforms multi-camera video input into identity-resolved tracking output with sufficiently low latency for live applications while supporting asynchronous retroactive updates.

[0122] The pipeline 1200 employs a multi-process architecture with sequential stages communicating through queued data structures. Video streams from all cameras and / or other sources (Video Stream 1 through Video Stream N) are received and processed through a video decoding stage 1210 that decodes the incoming video data from each camera. A frame buffer 1220 receives the decoded frames and organizes them into per-camera frame buffers for processing. Object detection and identity feature extraction are performed on the buffered frames by the detection subsystem 104.

[0123] A cross-camera synchronization and projection stage 1230 aligns detections across cameras based on timestamps and projects detected positions into the unified spatial coordinate system. In some embodiments, the detection alignment is conducted by the multi-camera registrator 306. The cross-camera synchronization and projection stage 1230 enables the system to reason about the same physical entity observed from different camera views simultaneously.

[0124] The identification subsystem 106 consumes the synchronized, projected detections and performs within-camera tracking, cross-camera association, multi-modal signature construction, and probabilistic identity assignment. In some embodiments, within-camera tracking associates detections across consecutive frames in each camera, creating track edges within the temporal identity graph; the retroactiveidentity resolution pipeline 700 subsequently updates identity assignments across these associations when new evidence becomes available; cross-camera association matches detections across cameras at each time step using the cross-view identity association mechanism 900; multi-modal signature construction builds and updates the unified multi-modal object signatures using the multi-modal signature generator 404; and probabilistic identity assignment maintains and updates the identity probability distributions using the probabilistic identity tracker 406.

[0125] When the identification subsystem 106 generates new identity evidence, such as a clear jersey number recognition or a facial recognition match, an identity evidence trigger initiates the retroactive resolution process. The bidirectional identity propagator 108 evaluates the evidence significance, determines the appropriate propagation scope, executes bidirectional propagation through the temporal identity graph as described with reference to FIG. 7, enforces constraints including mutual exclusivity, roster constraints, entry / exit event constraints, and position constraints, and publishes updated identity assignments. The retroactive updates are applied without interrupting the forward processing pipeline, so that live output continues with minimal latency while historical corrections are applied in the background. The output interface 110 provides the updated identity-resolved tracking data to downstream consumers, including confidence measures and update notifications.

[0126] The real-time processing pipeline supports configurable processing modes to accommodate different computational environments. A high-performance mode processes all cameras at full frame rate with full multi-modal feature extraction and maintains the complete temporal identity graph in memory. A balanced mode subsamples frames for identity feature extraction while maintaining full-rate detection and tracking. A lightweight mode reduces the number of active cameras for identity processing and employs more aggressive hypothesis pruning. The processing mode can be selected based on available hardware and adjusted dynamically based on system load.

[0127] The identity analysis system 100 supports sport-specific adaptation through configurable profiles that adjust detection, tracking, identity processing, and constraint parameters for different sporting contexts. Each sport profile defines: the set of object classes to detect and track, the relevant identifying features for the sport (e.g., jersey numbers, helmet numbers, team colors, equipment markings), the expected number of entities of each type (e.g., players per team, officials), substitution rules and constraints, characteristic spatial patterns and positional expectations, and game phase definitions that influence identity reasoning.

[0128] For example, a basketball profile defines five players per team on the court, frequent substitutions with known entry / exit points (scorer's table), jersey numbers as the primary identifying feature, facial recognition as a secondary feature available during free throws and stoppages, and positional expectations based on common offensive and defensive formations. A soccer profile defines eleven players per team, three to five substitutions per team with sideline entry points, jersey numbers visible primarily from elevated or close-up cameras, and positional expectations based on tactical formation (4-3-3, 4-4-2, etc.). An icehockey profile defines six players per team on the ice, frequent line changes with bench-area entry / exit zones, jersey numbers and helmet markings as identifying features, and the additional challenge of similarappearing equipment reducing the discriminative power of appearance features.

[0129] The identity analysis system 100 may incorporate an adaptive learning capability that enables progressive improvement through operational experience. Identity feature extraction models may be incrementally refined based on confirmed identity assignments: when a track's identity is confirmed with high confidence (through clear jersey number recognition or facial recognition matching), the confirmed appearance features serve as additional training examples for the appearance model, improving future recognition of that player and similar players. Sport-specific models — appearance encoders, motion pattern models, contextual reasoning models — may be specialized through accumulated experience across multiple sporting events, progressively improving performance for each sport. Venue-specific adaptation learns optimal camera calibration refinements, lighting compensation parameters, and detection configurations for specific installations. Embodiments of team-specific and player-specific model refinement improve recognition accuracy for frequently observed teams and players, building detailed appearance and behavioral models that enhance identity persistence in subsequent events.

[0130] The identity analysis system 100 may support a range of applications in sports video analysis and broadcasting. For example, in some embodiments, combination with player statistics data may provide continuous, identity-resolved player statistics throughout entire games despite intermittent visibility of identifying features, with retroactive correction ensuring that statistics attributed during uncertain periods are updated when identity is subsequently confirmed. For example, in some embodiments, enhanced broadcast graphics may provide consistent player identification across multiple camera views for player name overlays, statistics displays, and tracking visualizations that maintain correct identity labels through camera transitions. For example, in some embodiments, automated highlight generation may identify and attribute actions to specific players even when identity was initially uncertain, producing personalized highlight reels with accurate player identification. For example, in some embodiments, tactical and performance analysis may provide metrics for coaches and analysts that correctly attribute actions, positions, and interactions to specific players throughout the entire duration of a match, supporting detailed post-game review and strategic planning. For example, in some embodiments, multi-entity tracking may provide consistent identification and tracking of not only players but also game officials, coaching staff, and sport-specific equipment, supporting comprehensive game data collection for league operations, officiating review, and equipment tracking. For example, in some embodiments, historical data correction may automatically and retroactively correct tracking data when new identification evidence becomes available, ensuring that the quality of the tracking record improves continuously throughout and after a sporting event without requiring manual review or reprocessing. For example, in some embodiments, uncertainty-aware analytics mayexpress the certainty of identity assignments, enabling downstream consumers including broadcasters, coaches, analysts, and fans to appropriately interpret data from periods of identity uncertainty.

[0131] For example, in some embodiments, the identity analysis system 100 provides several advantages over existing approaches to object identity tracking in sports video. For example, in some embodiments, retroactive identity correction may enable the propagation of newly discovered identity information backward through time to correct historical identity assignments without reprocessing entire video sequences, producing an identity record that improves continuously as more evidence becomes available. For example, in some embodiments, multi-camera identity fusion may exploit complementary information across multiple camera views, where a clear identification in one camera immediately improves identity assignments in all other cameras observing the same entity via tracking and cross-view links. For example, in some embodiments, persistent identity through occlusions may be maintained using the multimodal signatures that remain discriminative when individual feature components are unavailable due to occlusions, camera transitions, player clustering, or extended periods of limited visibility. For example, in some embodiments, principled uncertainty handling through probability distributions may avoid the cascading errors of premature hard assignments and may enable downstream analytics to appropriately incorporate identity confidence. For example, in some embodiments, constraint-based consistency may enforce logical constraints including mutual exclusivity, roster membership, and substitution rules during identity propagation, ensuring that identity assignments are physically possible and game-rule compliant. For example, in some embodiments, computational efficiency for updates may be achieved through optimized change propagation that avoids reprocessing unaffected portions of the video when performing retroactive identity updates, enabling efficient correction even for sporting events spanning hours of multicamera footage. For example, in some embodiments, sport-specific intelligence may enable configurable sport profiles that adapt detection, tracking, identity processing, and constraints to the characteristics of each sport, recognizing that the identity challenges in basketball (frequent substitutions, close contact) differ from those in soccer (large field, fewer camera views per player) or hockey (similar equipment among players, frequent line changes). For example, in some embodiments, adaptive quality improvement may enable continuous learning that improves identity recognition accuracy through accumulated operational experience, building sport-specific, venue-specific, team-specific, and player-specific models that enhance performance over time.

[0132] FIG. 13 is a schematic diagram of a computing device 1300 that may be used to implement the identity analysis system 100, in accordance with an embodiment.

[0133] As depicted, computing device 1300 includes a processing subsystem with one or more processors 1302, one or more memories 1304, and a communication subsystem with one or more I / O interfaces 1306 and one or more network interfaces 1308.

[0134] Each processor 1302 may be, for example, any type of microprocessor or microcontroller, a digital signal processing (DSP) processor, an integrated circuit, a field programmable gate array (FPGA), a reconfigurable processor, a programmable read-only memory (PROM), or any combination thereof. Processor 1302 may, for example, include one or more central processing units (CPUs), a graphics processing unit (GPU), a quantum processing unit (QPU), or the like. In some embodiments, the processor 1302 includes a GPU configured to execute the object detection models deployed by the per-camera object detector 302, the hybrid memory-augmented transformer 402, the multi-modal signature generator 404, and the probabilistic identity tracker 406 described with reference to FIG. 4.

[0135] Memory 1304 may include a suitable combination of any type of computer memory that is located either internally or externally, such as, for example, random-access memory (RAM), read-only memory (ROM), compact disc read-only memory (CDROM), electro-optical memory, magneto-optical memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferroelectric RAM (FRAM), or the like.

[0136] Memory 1304 may store machine-interpretable instructions that, when executed by the processor 1302, cause the computing device 1300 to perform the methods described herein. Memory 1304 may further store, as non-limiting examples, trained model weights for the object detection models deployed by the per-camera object detector 302, feature extraction models used by the identity feature extractor 304, the external memory architecture of the hybrid memory-augmented transformer 402 including the short-term buffer, medium-term store, and long-term store, the temporal identity graph maintained by the bidirectional identity propagator 108, the identity probability distributions maintained by the probabilistic identity tracker 406, and the multi-modal object signatures generated by the multi-modal signature generator 404.

[0137] Each I / O interface 1306 enables computing device 1300 to interconnect with one or more input devices, such as a keyboard, mouse, camera, touch screen, and a microphone, or with one or more output devices such as a display screen and a speaker.

[0138] I / O interface 1306 may include a user interface providing operators with a visual dashboard displaying the current identity tracking status, the detected game state, the identity data structure, and system status information including per-module latency metrics. Through I / O interface 1306, an operator may configure processing parameters, specify sport profile settings, adjust confidence thresholds, or manually intervene in identity assignments when desired.

[0139] Each network interface 1308 enables computing device 1300 to communicate with other components via the communication network 50 described with reference to FIG. 1, to exchange data with other components, to access and connect to network resources, to serve applications, and to perform other computing applications by connecting to a network (or multiple networks) capable of carrying data including the Internet, Ethernet, plain old telephone service (POTS) line, public switched telephone network (PSTN), integrated services digital network (ISDN), digital subscriber line (DSL), coaxial cable, fiber optics, satellite,mobile (e.g., Wi-Fi, WiMAX, Li-Fi), SS7 signaling network, fixed line, local area network, wide area network, and others, including any combination of these. Network interface 1308 enables the computing device 1300 to receive the plurality of video feeds from the cameras 80 and to transmit the signal encoding data of the identity data structure through the communication interface to the downstream consumers 10.

[0140] The foregoing discussion provides many example embodiments of the inventive subject matter. Although each embodiment represents a single combination of inventive elements, the inventive subject matter is considered to include all possible combinations of the disclosed elements. Thus if one embodiment comprises elements A, B, and C, and a second embodiment comprises elements B and D, then the inventive subject matter is also considered to include other remaining combinations of A, B, C, or D, even if not explicitly disclosed.

[0141] The embodiments of the devices, systems and methods described herein may be implemented in a combination of both hardware and software. These embodiments may be implemented on programmable computers, each computer including at least one processor, a data storage system (including volatile memory or non-volatile memory or other data storage elements or a combination thereof), and at least one communication interface.

[0142] Program code is applied to input data to perform the functions described herein and to generate output information. The output information is applied to one or more output devices. In some embodiments, the communication interface may be a network communication interface. In embodiments in which elements may be combined, the communication interface may be a software communication interface, such as those for inter-process communication. In still other embodiments, there may be a combination of communication interfaces implemented as hardware, software, and combination thereof.

[0143] Throughout the foregoing discussion, numerous references will be made regarding servers, services, interfaces, portals, platforms, or other systems formed from computing devices. It should be appreciated that the use of such terms is deemed to represent one or more computing devices having at least one processor configured to execute software instructions stored on a computer readable tangible, non-transitory medium. For example, a server can include one or more computers operating as a web server, database server, or other type of computer server in a manner to fulfill described roles, responsibilities, or functions.

[0144] The technical solution of embodiments may be in the form of a software product. The software product may be stored in a non-volatile or non-transitory storage medium, which may be a compact disk read-only memory (CD-ROM), a USB flash disk, or a removable hard disk. The software product includes a number of instructions that enable a computer device (personal computer, server, or network device) to execute the methods provided by the embodiments.

[0145] The embodiments described herein are implemented by physical computer hardware, including computing devices, servers, receivers, transmitters, processors, memory, displays, and networks. Theembodiments described herein provide useful physical machines and particularly configured computer hardware arrangements.

[0146] The above-described embodiments are intended to be illustrative only and in no way limiting. The described embodiments are susceptible to many modifications of form, arrangement of parts, details, and order of operation. The disclosure is intended to encompass all such modifications within its scope, as defined by the claims.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method for identity analysis in sports video, the method comprising:receiving a plurality of video feeds of a sporting event;upon processing data from the plurality of video feeds:detecting a plurality of objects of interest across the video feeds;maintaining an identity data structure that stores data identifying the objects of interest across a plurality of camera views of the video feeds, the identity data structure including current and historical identity information for the objects of interest over a time course of the video feeds; upon a change to the current identity information for a given object of interest of the plurality of objects of interest, updating the historical identity information forward and backward over the time course of the video feeds; andtransmitting a signal encoding data of the identity data structure through a communication interface.

2. The computer-implemented method of claim 1, further comprising: associating with an item of the identity information a confidence metric reflecting a certainty in that item of the identity information.

3. The computer-implemented method of claim 1 , wherein said detecting comprises detecting at least one player, official, coach, equipment, or sport-specific objects within video frames of the video feeds.

4. The computer-implemented method of claim 1, further comprising: mapping positions of detected objects of interest across the camera views to a unified spatial coordinate system.

5. The computer-implemented method of claim 1, further comprising: extracting from the video feeds identifying features including at least one of jersey number recognitions, facial recognition embeddings, team color classifications, body morphology descriptors, or uniform detail patterns from detected objects of interest.

6. The computer-implemented method of claim 1 , wherein said maintaining utilizes a hybrid memory- augmented transformer.

7. The computer-implemented method of claim 6, wherein the hybrid memory-augmented transformer communicates with a non-transitory memory that stores identity information to be persistent beyond an attention window of the hybrid memory-augmented transformer.

8. The computer-implemented method of claim 7, wherein the non-transitory memory implements a memory architecture providing a short-term buffer for recent identity information, a medium term store for session-level identity information, and a long-term store for identity information that spans substantially the sporting event.

9. The computer-implemented method of claim 8, further comprising: transferring an item of identity information to long-term storage when confidence in that item exceeds a threshold.

10. The computer-implemented method of claim 6, wherein the hybrid memory-augmented transformer includes a temporal identity graph that represents the objects of interest as nodes and their potential identity associations across the time course and the camera views as edges.

11. The computer-implemented method of claim 10, wherein said updating the historical identity information forward and backward over the time course includes traversing the temporal identity graph.

12. The computer-implemented method of claim 1, wherein the identity information for the objects of interest includes, for each object of interest, an identity probability distribution of possible identity associations for that object of interest.

13. The computer-implemented method of claim 12, wherein said maintaining the identity data structure includes updating the identity probability distributions for the objects of interest.

14. The computer-implemented method of claim 12, wherein said updating the historical identity information forward and backward over the time course includes updating the identity probability distributions by applying a Bayesian update in reverse temporal order.

15. The computer-implemented method of claim 1, further comprising: generating a multi-modal signature that combines at least two of appearance feature, spatio-temporal trajectory information, team affiliation indicators, role and position characteristics, and interaction patterns.

16. A computer-implemented system for identity analysis in sports video, the system comprising:a communication interface;a processing subsystem that includes one or more processors and one or more memories coupled with the one or more processors, the processing subsystem configured to cause the system to:receive a plurality of video feeds of a sporting event;upon processing data from the plurality of video feeds:detect a plurality of objects of interest across the video feeds;maintain an identity data structure that stores data identifying the objects of interest across a plurality of camera views of the video feeds, the identity data structure including current and historical identity information for the objects of interest over a time course of the video feeds;upon a change to the current identity information for a given object of interest of the plurality of objects of interest, update the historical identity information forward and backward over the time course of the video feeds; andtransmit a signal encoding data of the identity data structure through the communication interface.

17. The computer-implemented system of claim 16, further comprising a video input interface interconnected with at least one camera, and wherein the plurality of video feeds includes a video feed from the at least one camera.

18. The computer-implemented system of claim 16, wherein the plurality of video feeds includes live footage of the sporting event.

19. The computer-implemented system of claim 16, wherein the identity data structure is maintained using a memory-augmented transformer.

20. A non-transitory computer readable medium storing machine interpretable instructions, which when executed by one or more processors, cause the one or more processors to perform a method for identity analysis in sports video, the method comprising:receiving a plurality of video feeds of a sporting event;upon processing data from the plurality of video feeds:detecting a plurality of objects of interest across the video feeds;maintaining an identity data structure that stores data identifying the objects of interest across a plurality of camera views of the video feeds, the identity data structure including current and historical identity information for the objects of interest over a time course of the video feeds;upon a change to the current identity information for a given object of interest of the plurality of objects of interest, updating the historical identity information forward and backward over the time course of the video feeds; andtransmitting a signal encoding data of the identity data structure through a communication interface.

Images