System, apparatus and method for training a transformer encoder neural network

Abstract

A method for training a transformer encoder to analyse time series multivariate data. The transformer encoder is first pre-trained on simulated event data, 126, and then fine-tuned, 128, on real-world

Description

Field One or more embodiments of the present invention relate to a system, apparatus and method for a transformer encoder neural network, in particular, but not exclusively, to real-time analysis of event outcomes using a transformer-based machine learning architecture. More particularly, one or more embodiments of the invention relate to adapting a transformer encoder neural network, commonly used for natural language processing tasks, to analyse time series data representing the state of a real-world event at various times during the event’s progression. In one or more embodiments, a transformer encoder, adapted in accordance with one or more embodiments of the present invention, is first pre-trained on simulated event data and then fine-tuned on real-world historical data to enable accurate real-time analysis of discrete and continuous outcomes in a real-world event, such as a sporting event. Background Determining a likelihood or probability (often termed “predicting”) an event conclusion, whether in-play events and / or final conclusion, of sporting competitions conventionally uses complex mathematical modelling and Monte Carlo simulations run over millions of iterations to generate probabilistic representations of event conclusions. This approach is performed both prior to the competition and while the competition is happening (in-play). The generated outcome probabilities are then converted to odds for sports books, allowing bets to be placed on the predicted event outcomes before and during the competition. Sports books, also known as sportsbooks or bookmakers, are entities that accept and pay out sports bets. They set odds, take wagers, and pay out winnings for various sporting events. In-play bets account for a significant portion (up to 70-80%) of all bets placed. A major challenge in providing several probabilistic outcomes is that each market, i.e., each offered type of bet, is dependent upon other bets and / or types of bets. As a result, extremely large hierarchical Markov chains are created to provide accurate probabilities for multiple betting markets. The computations associated with this approach are extremely large, resulting in a significant carbon footprint for the organizations utilizing them and increased operational costs. This approach also has significant limitations, with a very significant disadvantage being the computational time required to run such simulations, particularly for in-play betting when the competition is occurring, and probabilities need to be updated after each event. The processing time can take several seconds to minutes following each event, resulting in sports books suspending markets to prevent gamblers from placing bets on the outcome of an event when the event has just happened. Some sports, such as boxing and mixed martial arts, offer almost no in-play betting due to the speed at which events occur. Analysis of in-play markets in other sports, such as American football, has highlighted that markets are suspended for 18-41% of the game duration. This can lead to reduced consumer engagement and, ultimately, fewer bets being placed. Machine learning methods can significantly reduce overall computational usage but lack the complexity of the mathematical approach in providing multiple parlay market options. A parlay is a type of bet that combines multiple individual bets into a single wager. To win a parlay bet, all the individual bets within it must be successful. Parlays offer higher potential payouts due to their increased difficulty but also carry a higher risk. Machine learning methods typically only offer micro-markets in real-time, i.e., the next event, without considering how everything else is impacted, limiting their use cases. Aspects and embodiments in accordance with the present invention were devised with the foregoing in mind. Summary According to a first aspect of the present invention, there is provided a system for training a transformer encoder neural network for analysing time series multivariate data. The system comprises a data store for storing time series multivariate data representing a state relationship between a first entity and a second entity at respective times in a prior event. The system further comprises a processor configured to invoke a pre-training module. The pre-training module is configured to retrieve the time series multivariate data from the data store, invoke an event simulation process, and input the time series multivariate data into the event simulation process to generate synthetic event data. The synthetic event data comprises a matrix representing a full span of a simulated event, with the matrix comprising data for a plurality of time steps. The pre-training module then invokes a transformer encoder neural network and inputs the synthetic event data generated by the event simulation process to the transformer encoder neural network to train it for building a pre-trained transformer encoder neural network. By using synthetic event data generated from real historical data, the system is configured to create a large volume of diverse training scenarios, which improves the robustness and generalisation capabilities of the transformer encoder neural network. The use of time series multivariate data allows the system to capture complex relationships and dependencies between different entities over time, enabling more accurate predictions in real-world scenarios. Additionally, the pretraining approach allows the network to learn general patterns and features from a wide range of simulated events before fine-tuning on specific real-world data, improving overall performance and reducing the amount of real-world data needed for training. Optionally, the system further comprises a data pre-processing module configured to apply a max-aggregate gate over a time dimension of the matrix to determine a maximum value within a defined window size, generating aggregated synthetic event data. By applying a max-aggregate gate, the system is configured to reduce the dimensionality of the input data, which can lead to faster training and inference times. It also helps to capture the most salient features within each time window, improving the model's ability to detect important events or changes in the data. This approach can also act as a form of noise reduction, focusing the model's attention on the most significant data points within each window. Optionally, the pre-training module is further configured to generate an encoded matrix comprising an embedding for each time step in the aggregated synthetic event data, where the embedding encodes information relating to the progress of the simulated event. It then applies a pooling mechanism over a time dimension of the encoded matrix to generate a pre-training event evolution vector representative of an evolution of the simulated event over a respective time step. This pre-training event evolution captures encoded information relating to the evolution of the simulated event. Optionally, the pre-training module is further configured to concatenate the encoded pre-training entity vector with the pre-training event evolution vector to generate a composite pre-training vector. This concatenation step combines the information from both the event evolution (temporal dynamics) and the entity-specific features into a single, comprehensive representation. This composite vector provides a, multifaceted input for subsequent tasks, allowing the model to consider both the progression of the event and the characteristics of the entities involved when outputting likely outcomes for events. This approach can lead to more accurate and contextually relevant predictions compared to models that consider only one aspect of the data. Optionally, the pre-training module is further configured to use the composite pre-training input vector to pre-train one or more output neural networks configured to generate an output indicative of a likely outcome of the simulated event. By using the composite vector as input for pre-training output neural networks, the system is configured to learn to make predictions that consider both the temporal evolution of the event and the specific characteristics of the entities involved. This approach leads to more accurate and nuanced predictions of event outcomes, as it leverages the, multi-faceted information captured in the composite vector. The pre-training of these output networks on simulated data allows the system to learn general patterns and relationships before fine-tuning on real-world data, potentially improving overall predictive performance. Optionally, the system further comprises a fine-tuning module, and the processor is further configured to invoke the fine-tuning module to fine-tune the pretrained transformer encoder neural network using real-world time series multivariate data, where the real-world time series multivariate data represents a full span of a real-world event at respective time steps. By adapting the pre-trained network to real-world data, the system can be configured to adjust its learned representations and predictions to better match actual events. This two-stage approach of pre-training on synthetic data followed by fine-tuning on real data combines the advantages of large-scale training data (from simulations) with the specificity and nuance of real-world scenarios. This can result in a model that is both robust and highly accurate for real-world applications. Optionally, the fine-tuning module is configured to invoke the data preprocessing module to apply a max-aggregate function over a time dimension of the real-world time series multivariate data to generate aggregated real-world event data, with the max-aggregate function taking a maximum value within a defined window size. It then processes the aggregated real-world event data using the pre trained transformer encoder neural network to generate a fine-tuned event evolution vector. The fine-tuning module also invokes the pre-trained neural network to process the time series multivariate data to generate a fine-tuned entity vector, where the fine-tuned event evolution vector is representative of an evolution of the real-world event up to a current time step. This approach to fine-tuning maintains consistency with the pre-training process while adapting to real-world data. The use of the same max-aggregate function ensures that the input data structure remains similar, allowing for effective transfer of learned features. The generation of fine-tuned event evolution and entity vectors allows the model to adapt its representations to the nuances of real-world data, potentially improving its performance on actual events. This process helps bridge the gap between simulated and real-world scenarios, leading to a model that can make more accurate predictions in practical applications. Optionally, the fine-tuning module is further configured to combine the finetuned event evolution vector and the fine-tuned entity vector to generate a composite fine-tuning input vector representing the evolution of the real-world event based on the entity's state in response to previous events. It then uses this composite fine-tuning input vector to fine-tune the one or more pre-trained output neural networks, where the one or more fine-tuned output neural networks generate an output indicative of the likely outcomes of one or more real-world events. By combining the fine-tuned event evolution and entity vectors, the system is configured to capture both the temporal dynamics of the event and the specific characteristics of the entities involved. Using this composite vector to fine-tune the output networks configures the system to output outcomes that are tailored to real-world scenarios. This approach can lead to more accurate and contextually relevant predictions for actual events, improving the system's practical utility. Optionally, the transformer encoder comprises a plurality of transformer encoder layers, each layer comprising an attention mechanism and a feedforward layer configured to extract short-term, long-term, and higher-order relationships between the entities' states across the time sequence. Each transformer encoder layer is configured to take as input the aggregated synthetic event data matrix of dimension (T, F), where T is the number of time steps and F is the number of features per time step in the aggregated synthetic event data. It successively extracts temporal relationships between the entities' time-varying state data and applies a linear transformation to the aggregated synthetic event data, preferably using a feedforward layer. The attention mechanism allows the model to focus on relevant parts of the input sequence when making predictions, capturing complex dependencies across different time steps. The ability to extract short-term, long-term, and higher-order relationships enables the model to capture a wide range of temporal patterns, from immediate effects to long-range dependencies. This can lead to more sophisticated and accurate modelling of complex time series data. The successive extraction of temporal relationships through multiple layers allows for hierarchical feature learning, potentially capturing increasingly abstract and complex patterns in the data. The use of feedforward layers for transformation adds non-linearity to the model, increasing its capacity to learn complex functions. Overall, this architecture enhances the model's ability to capture and leverage the intricate temporal dynamics present in the data. The process of generating embeddings and applying a pooling mechanism allows the system to create compact, informative representations of the event's progression over time. The embeddings capture complex relationships and dependencies within the data, while the pooling mechanism helps to summarise this information across the time dimension. This approach configures the model to learn and represent long-term dependencies and patterns in the event data, which is useful for accurate prediction and analysis of time series data. The pre-training module is also configured to invoke a neural network configured as a feedforward neural network to process the time series multivariate data to generate an encoded pre-training entity vector. The neural network is designated as a pre-trained neural network after processing the time series multivariate data. By incorporating a separate feedforward neural network to process the entityspecific data, the system can be configured to capture and encode salient features and patterns related to the individual entities involved in the event. This allows the model to consider both the temporal dynamics of the event (captured by the transformer encoder) and the specific characteristics of the entities involved. This dual approach can lead to more nuanced and accurate predictions, as it considers both the evolving state of the event and the inherent properties of the participating entities. Optionally, the processor is further configured to apply a binning process to convert continuous outcome variables into discrete bins representing ranges of values, and wherein the one or more output neural networks output a value representative of the probability of the outcome falling within each discrete bin. The processor is further configured to apply a calibration process to the value for enhancing the predicted probability to reflect the frequency of the outcomes. By converting continuous variables into discrete bins, the system can be configured to handle a wide range of outcome values without the need for precise point predictions. This approach is particularly useful for outcomes that may have a non-linear or complex distribution. Additionally, the binning process offers several advantages including allowing the model to capture and represent uncertainty more effectively. Instead of predicting a single point value, the model can express the likelihood of the outcome falling within different ranges. Additionally, it can help mitigate the impact of outliers or extreme values in the training data, potentially leading to more robust predictions. Moreover, the discrete bins can be more interpretable and actionable in many real-world scenarios, where exact precision may not be necessary or practical. The calibration process further enhances the reliability of the predictions. By adjusting the predicted probabilities to better reflect the true frequency of outcomes, the system can provide more accurate probability estimates. This is particularly important in applications where the predicted probabilities are used for decisionmaking or risk assessment. Optionally, the time series multivariate data represents a state relationship between a first participant and a second participant at respective times in a sporting event, wherein the sporting event is selected from the group consisting of: rugby, American football, basketball, baseball, soccer, hockey, tennis, boxing, cricket and martial arts including mixed martial arts. This enhanced analysis and prediction capabilities is useful for sports analytics, which can be valuable for teams, coaches, and sports betting industries. Additionally, by processing time series data from these events, the system could potentially identify patterns or strategies that might not be apparent through traditional analysis methods. Viewed from a second aspect, the present invention provides a system for analysing real-time events using a pre-trained transformer encoder neural network. The system comprises a memory for storing a fine-tuned transformer encoder neural network, a fine-tuned neural network, and one or more fine-tuned output neural networks. It also includes an interface configured to receive real-time multivariate data representing a state relationship between a first entity and a second entity at respective times in a real-time event. Additionally, the system includes a processor configured to invoke a data pre-processing module to process the real-time multivariate data to generate aggregated real-time event data. By utilising pre-trained and fine-tuned neural networks, it can quickly process and analyse incoming data without the need for time-consuming training during the event. The use of aggregated real-time event data allows for efficient processing of large volumes of incoming information, enabling rapid insights and predictions. Optionally, the processor is further configured to invoke the fine-tuned transformer encoder neural network and input the aggregated real-time event data to generate a real-time event evolution vector. This vector captures encoded information relating to the progression of the real-time event. The processor also invokes the fine-tuned neural network and inputs the real-time multivariate data to generate a real-time entity vector, which represents characteristics of the entities involved in the real-time event. This feature enhances the system's ability to capture both the temporal dynamics of the event and the specific characteristics of the entities involved. By generating separate vectors for event evolution and entity characteristics, the system can provide a more nuanced and comprehensive analysis of the real-time situation. Optionally, the processor is configured to concatenate the real-time event evolution vector and the real-time entity vector to generate a combined real-time input vector. It then invokes the one or more fine-tuned output neural networks and inputs the combined real-time input vector to generate an output indicative of one or more likely outcomes of the real-time event. This concatenation step allows the system to consider both the event progression and entity characteristics simultaneously when making predictions. By combining these different aspects of the data, the system can potentially make more accurate and context-aware predictions about the likely outcomes of the event. Optionally, the processor is further configured to apply a binning process to convert continuous outcome variables into discrete bins representing ranges of values. The one or more fine-tuned output neural networks output a value representative of the outcome falling within each discrete bin. The processor is further configured to apply a calibration process to the value for enhancing the predicted probability to reflect the frequency of the outcomes. This binning and calibration process improves the system's ability to handle continuous outcomes and provide more reliable probability estimates. By converting continuous variables into discrete bins, the system can better manage uncertainty and provide more interpretable predictions. The calibration process further enhances the reliability of these predictions by adjusting them to better reflect real-world outcome frequencies. Optionally, the communication interface is configured to receive the real-time multivariate data as a live stream during the real-time event, and the processor is configured to repeatedly process the real-time multivariate data at defined intervals during the real-time event to generate updated outputs indicative of the one or more likely outcomes. This feature enables real-time analysis and prediction, allowing the system to continuously update its outputs as new data becomes available. This can be particularly valuable in fast-paced environments where conditions can change rapidly, enabling users to make informed decisions based on the most up-to-date information available. Viewed from a third aspect, the present invention provides a method for training a transformer encoder neural network for analysis of time series multivariate data. The method comprises storing, in a data store, time series multivariate data representing a state relationship between a first entity and a second entity at respective times in a prior event. It involves retrieving the time series multivariate data from the data store, invoking an event simulation process, and inputting the time series multivariate data into the event simulation process to generate synthetic event data. The synthetic event data comprises a matrix representing a full span of a simulated event, with the matrix comprising data for a plurality of time steps. The method then involves invoking a transformer encoder neural network and inputting the synthetic event data generated by the event simulation process to the transformer encoder neural network to train it for building a pre-trained transformer encoder neural network. Optionally, the method further comprises invoking a data pre-processing module and applying, by the data pre-processing module, a max-aggregate gate over a time dimension of the matrix to determine a maximum value within a defined window size to generate aggregated synthetic event data. Optionally, the method further comprises generating an encoded matrix comprising an embedding for each time step in the aggregated synthetic event data, wherein the embedding encodes information relating to the progress of the simulated event. It also involves applying a pooling mechanism over a time dimension of the encoded matrix to generate a pre-training event evolution vector representative of an evolution of the simulated event over a respective time step, wherein the pre-training event evolution captures encoded information relating to evolution of the simulated event. Optionally, the method further comprises invoking a neural network configured as a feedforward neural network, processing the time series multivariate data using the neural network to generate an encoded pre-training entity vector, and designating the neural network as a pre-trained neural network after processing the time series multivariate data. Optionally, the method further comprises concatenating the encoded pre-training entity vector with the pre-training event evolution vector to generate a composite pre-training vector. Optionally, the method further comprises using the composite pre-training input vector to pre-train one or more output neural networks configured to generate an output indicative of a likely outcome of the simulated event. Optionally, the present invention provides a method for analysing real-time events using a pre-trained transformer encoder neural network. The method comprises storing, in a memory store, the pre-trained transformer encoder neural network and a pre-trained neural network. It involves receiving, via a communication interface, real-time multivariate data representing a state relationship between a first entity and a second entity at respective times in a real-time event. The method then proceeds with invoking the pre-trained transformer encoder neural network and inputting the real-time multivariate data to generate a real-time event evolution vector. Optionally, the method further comprises invoking the pre-trained neural network and inputting the real-time entity data to generate a real-time entity vector. It then involves combining the real-time event evolution vector and the real-time entity vector to generate a combined real-time input vector. Optionally, the method includes invoking one or more output neural networks and inputting the combined real-time input vector to generate an output indicative of one or more likely outcomes of the real-time event. Optionally, the real-time event is a sporting event, and the first entity and the second entity are participants in the sporting event. The sporting event may be selected from the group consisting of: rugby, American football, basketball, baseball, soccer, hockey, tennis, boxing, and mixed martial arts. Brief description of the figures One or more embodiments in accordance with the invention will now be described, by way of non-limiting example only, and with reference to the accompanying drawings, in which: Figure 1 is a high-level system diagram illustrating the components of a realtime sports event analysis system according to an embodiment of the present invention; Figure 2 is a process flow diagram showing the key steps in a method for pre-training the transformer encoder neural network using synthetic event data according to an embodiment of the present invention; Figure 3 is a process flow diagram depicting the fine-tuning stage of a transformer encoder neural network using real-world historical event data according to an embodiment of the present invention; Figure 4 illustrates the binning process for converting continuous outcomes into discrete categories in accordance with an embodiment of the present invention; Figure 5 illustrates a calibration curve for assessing the quality of model predictions in accordance with an embodiment of the present invention. Detailed Description In the following description, for the purposes of non-limiting explanation only, numerous specific details are set forth to provide a thorough understanding of the present disclosure. In other instances, well-known structures and devices are shown in block diagram form to avoid unnecessarily obscuring the present disclosure. In one embodiment, the system facilitates real-time analysis of sporting event outcomes. This approach is particularly effective for fast-paced sports such as Mixed Martial Arts (MMA), where traditional methods may not be able (struggle) to provide timely in-play betting options. System Overview Fig. 1 is a high-level system diagram illustrating the components of the realtime sports event analysis system, according to an embodiment of the present invention. The system 100 comprises a processor 110, a memory 112, a simulation module 114, a datastore 116, a data input module 118, a data pre-processing module 120, an output analysis module 130, and a server 122 comprising a transformer encoder neural network 124, a pre-training module 126, a fine-tuning module 128, and a statistics neural network 132. It should be appreciated that the elements stored in the server 122 can be stored in any other suitable location such as the memory 112, which can act as a memory store for the transformer encoder neural network 124, a pre-trained / fine-tuned transformer encoder neural network, a pre-trained / fine-tuned statics neural network and other pre-trained / fine-tuned neural networks. Alternatively, or additionally the components may be stored or an external storage resource accessible to the system via a network connection. The processor 110 is configured to execute executable instructions (e.g., invoking the data input module 118). Datastore The datastore 116 comprises multivariate data elements comprising player statistics data amongst other data elements. The player statistics data may also be stored in the server 122 or any other, e.g. external, storage resource. The player statistics data stored in the datastore 116 may comprise a variety of information related to the past performance, characteristics, and attributes of the players involved in the sporting event. In one embodiment, for a Mixed Martial Arts (MMA) sporting event, the player statistics data may include, but is not limited to: historical fight records, strike accuracy and volume, grappling and takedown statistics, fight duration and round data, and finish rate in each round. Additionally, the player statistics data may include date elements comprising physical attributes and anthropometric data. These data elements comprise information about each player's physical characteristics, such as age, height, reach, weight, and weight class. In one or more embodiments the player statistics data may be obtained through a process known as ‘web scraping ‘or ‘data scraping’ from publicly available sources. Web scraping refers to the automated extraction of data from websites or other digital sources using software tools or scripts. This process involves programmatically accessing web pages, parsing their content, and collecting specific data elements of interest. In addition to the player statistics data, the datastore 116 may also store real-world historical event data. This real-world historical event data comprises timeseries information captured from actual past sporting events, representing the state of the event at various time steps throughout its duration. In one embodiment, for a Mixed Martial Arts (MMA) sporting event, the real-world historical event data may include, but is not limited to: - time-stamped strike data elements comprising information about the strikes thrown by each player during the event, including the type of strike (e.g., jab, cross, kick), its target (e.g., head, body, legs), and the time at which it was thrown; - time-stamped grappling data elements comprising information about grappling exchanges during the event, such as takedown attempts, successful takedowns, submission attempts, and positional changes (e.g., guard, half-guard, mount), along with their corresponding timestamps; - scoring and judging data elements which comprise the official scores awarded by the judges at the end of each round, as well as any point deductions or penalties issued during the event; and - contextual data elements comprising additional contextual information about the event, such as the venue, crowd reactions, and commentary from experts. For other sports, time-stamped “play / event characterisation” data is grouped into data element characterisation types. In one or more embodiments the real-world historical data is time-series data obtained from publicly available videos of fights. A manual encoding process may be employed, where a human annotator watches each fight video and records the occurrence of specific events at different time steps throughout the fight. This manually encoded data captures the real-world dynamics and progression of the sporting event. Alternatively, or additionally, the encoding process may be automated using specialised software or artificial intelligence techniques. These automated methods can employ computer vision algorithms and machine learning models trained to recognize and classify specific events in fight videos, potentially increasing the speed and consistency of data encoding while reducing the reliance on human annotators. Such Al-driven approaches can process large volumes of video data efficiently, extracting relevant time-series information about fight dynamics and events. The scraped data referred to above may undergo further processing to calculate several additional variables, enhancing the abundance and relevance of the player statistics dataset. This processing involves applying mathematical operations and algorithms to the raw scraped data to calculate derived statistics that provide deeper insights into player performance and match dynamics. In one or more embodiments in the context of MMA, the system may scrape raw data comprising: 1. The total number of strikes: This represents all offensive striking attempts by a fighter, including punches, kicks, elbows, and knees, and whether they land or not; 2. Number of ground strikes: These are strikes attempted or landed while at least one fighter is on the ground. For example, punches thrown by a fighter in a dominant position on top of their opponent; 3. The total amount of control time for each fighter. This is the cumulative duration a fighter maintains a dominant position over their opponent; Control time may encompass both: a) Ground control: When a fighter is in a dominant position on the ground (e.g., full mount, side control) b) Standing control: Clinch positions where one fighter controls the other against the cage or in a dominant standing position. The control time may encompass both ground control and standing control (e.g., clinch positions). To generate a more nuanced understanding of the fight dynamics, the system calculates additional derived variables. To generate a more nuanced understanding of the fight dynamics, the system calculates additional derived variables. One such derived variable may be the amount of ground control time, which is not directly available from the scraped data. The system may calculate this using a formula such as: ground_control = total_control * (num_ground_strikes / total_strikes) where: ground_control represents the derived estimate of time spent in ground control, total_control is the scraped data for total control time, num_ground_strikes is the scraped data for number of strikes executed on the ground, and total_strikes is the scraped data for total number of strikes. This formula estimates ground control time by assuming the proportion of ground strikes to total strikes is roughly equivalent to the proportion of ground control time to total control time. It should be appreciated that this is a simplified example and in practice the calculation may involve additional variables and more complex mathematical relationships to improve accuracy. For instance, the system might incorporate: 1. Takedown data: Number of successful takedowns could influence the ground control time estimate. 2. Submission attempts: These could indicate periods of ground control not captured by strike data. 3. Position transitions: Frequency of position changes on the ground could refine the control time estimate. The system may employ similar derivation techniques for various other statistics relevant to the sport being analysed. For example: 1. Striking efficiency: (Significant strikes landed / Total strikes attempted) * 100 2. Takedown defense: (Takedowns defended I Takedown attempts against) * 100 3. Average fight time: Total fight time across all fights I Number of fights It should be appreciated that this is a simplified example and in practice the calculation may involve additional variables and more complex mathematical relationships to improve accuracy. The system may employ similar derivation techniques for various other statistics relevant to the sport being analysed. This process of deriving additional variables is applied across the scraped dataset to generate an augmented player statistics dataset. This augmented dataset, enriched with both directly scraped and derived variables, serves as the foundation for generating synthetic event data using a Monte Carlo simulation approach, as will be described in further detail in the following sections. Simulation module The simulation module 114 is configured to generate synthetic event data based on the player statistics data stored in the datastore 116. The synthetic data generated by the simulation module 114 is utilised during a pre-training stage to facilitate the learning of general patterns and dynamics of the sporting event by the transformer encoder neural network 124, prior to fine-tuning on the real-world historical event data. Fine-tuning, in this context, refers to the process of further training the pre-trained model on a dataset of real-world historical event data. This process allows the model to adapt its learned general patterns to the specific nuances and characteristics of actual sporting events, thereby improving its accuracy to output real world on real-world data. In one or more embodiments of the present invention, the simulation module 114 employs a combination of Bayesian modelling and Monte Carlo Markov Chain (MCMC) techniques to generate the synthetic event data. Bayesian modelling, in this context, involves using probability distributions to represent uncertainties in the model parameters and updating these distributions as new data is observed. This approach allows the model to incorporate prior knowledge and update its beliefs based on observed data. A simplified illustration of Bayesian modelling in this context might look like: Prior belief: P(0) - Initial probability distribution of model parameters Likelihood: P(D|0) - Probability of observing the data given the parameters Posterior: P(0|D) oc P(D|0) * P(0) - Updated belief about parameters after observing data The process begins by retrieving the player statistics data from the datastore 116 and using a Bayesian approach to generate models for predicting individual aspects of the event. For example, in the context of Mixed Martial Arts (MMA), a 'strike rate model' may be developed to estimate the number of strikes per second that could explain the observed results from historical fights. This Bayesian approach might be illustrated as follows: 1. Prior belief: Initial assumption about strike rate distribution 2. Data: Observed strike rates from historical fights 3. Likelihood: Probability of observing these strike rates given different possible true rates 4. Posterior: Updated distribution of strike rates after considering the data By applying the Bayesian approach to various aspects of the event, such as grappling exchanges, takedowns, and submissions, the simulation module 114 generates a set of coefficients that define the probabilistic relationships between different event variables and the observed outcomes. These coefficients are then utilised in a Monte Carlo Markov Chain (MCMC) model to simulate the progression of the sporting event. In the described embodiment the MCMC model operates on a time-step basis, where each time step represents a one-second interval. At each time step, the model starts in a particular node, representing the current state of the event, and probabilistically transitions to another node based on the coefficients derived from the one or more Bayesian models. The sequence of nodes visited during the simulation is recorded, capturing the evolution of the event over time. The simulation continues until the end of the event is reached, as determined by predefined criteria such as the duration of the fight or a specific outcome being achieved. Once the event simulation is complete, the recorded sequence of nodes is processed to calculate the relevant variables and generate synthetic time-series data. These variables may include the number of strikes thrown, the number of grappling exchanges, the duration of each phase of the fight, and other pertinent metrics. The synthetic time-series data generated by the MCMC model serves as a dataset for pre-training the transformer encoder neural network 124. By exposing the network to a wide range of simulated event scenarios, it can learn the general patterns, dependencies, and dynamics that govern the sporting event. The combination of Bayesian modelling and MCMC techniques employed by the simulation module 114 allows for the generation of realistic and varied synthetic event data. Data input module The data input module 118 is responsible for receiving and handling the input data required for the real-time analysis of sporting event outcomes. This module is configured to accept data from various sources, including live event data feeds, manual input from operators, and data retrieved from the datastore 116 and simulation model 114. For example, the data input module 118 can retrieve data from the datastore 116, which contains player statistics data and historical event data in addition to receiving the synthetic event data output from the simulation module. This data will typically be used for pre-training or for fine-tuning the transformer encoder neural network 124 and the statistics neural network 132. In one embodiment, the data input module 118 is designed to process data specific to Mixed Martial Arts (MMA) events, but it can be adapted to handle data from other sporting events as well. For instance, the system can be extended to other combat sports, such as boxing. In a boxing context, the data input module 118 may be configured to process data including punch count data for various types of punches, round-by-round scoring, knockdown occurrences, clinch durations, and footwork patterns. These data points, when processed as time-series events, allow the system to analyse boxing matches in a manner similar to MMA bouts. In each case, the sport-specific events are treated as a series of time-stamped occurrences, allowing the transformer encoder neural network 124 to process and analyse the data in a manner analogous to the MMA implementation. This demonstrates the system's versatility and potential for application across a diverse range of sporting events, each with its unique set of variables and event types. The key lies in the system's ability to interpret and process these varied inputs as sequential, time-based data, regardless of the specific sport or event type. For example, the system can be configured to extend beyond combat sports to team sports. For example: 1. Rugby: The data input module could process events such as passes, tackles, rucks, mauls, scrums, lineouts, and kicks. Each event would be time-stamped and could include additional data like field position, player involvement, and outcome. 2. American Football (NFL): The system could handle data including play types (run, pass, special teams), yards gained / lost, downs, penalties, turnovers, and time management events. Each play would be treated as a discrete event with associated time and outcome data. 3. Soccer: The data input module could process events such as passes, shots, tackles, fouls, comers, and possession changes. Continuous data like player positioning and ball movement could be discretised into time-stamped events for analysis. In one or more embodiments the data input module 118 may work in conjunction with the communication interface 134, which is connected to the data input module 118 by a dotted line in Figure 1, indicating its use only when the system is fully trained, configured and operational. During real-time operation, the communication interface 134 receives live event data feeds, which may include real-time information about the ongoing sporting event, such as strike data, grappling data, and scoring data. These feeds can be provided by external sources, such as official event broadcasters or third-party data providers. The communication interface 134 employs various technologies such as APIs, websockets, or other data streaming protocols to ensure efficient and timely reception of event data. The data input module 118 then processes this live data, parsing and interpreting these feeds to Identify and categorise different types of events (e.g., strikes, takedowns, scoring events), extract relevant attributes for each event (e.g., type of strike, success / failure, associated players), assign accurate timestamps to each event, validate data consistency and handle any anomalies or errors in the incoming data stream, standardise the data format across different input sources for consistent downstream processing. After this processing, the data input module converts the parsed and interpreted data into a format suitable for further processing by the system, particularly the transformer encoder neural network 124. In addition to the live event data feeds, the data input module 118 may also accept manual input from human operators through the communication interface 134. This allows for the incorporation of subjective observations, contextual information, or data points that may not be captured by the automated feeds. The manual input can be provided through a user interface or an API, which the data input module 118 can process and integrate with the other input data. The Data Pre-processing Module 120 is configured to refine and transform the input data received from the Data Input Module 118 to optimize it for the Transformer Encoder Neural Network 124. Data pre-processing module In one or more embodiments, the Data Pre-Processing Module 120 is configured to apply a max-aggregate function over the time dimension of the event data, wherein the event data may comprise the live data, the synthetic event data, or the real-world historical event data. In one or more embodiments the max-aggregate gate reduces the dimensionality of the matrix by a factor equal to the defined window size. For example, in Mixed Martial Arts (MMA), a round typically lasts 300 seconds. If 1-second time steps and a 5-second window are used, there would be 300 / 5 = 60 data points per round after aggregation. The max-aggregate function operation is designed to reduce the granularity of the time-series data and generate aggregated input sequences. By aggregating the data over time, the module can capture the most significant events or changes within each time window, effectively summarizing the relevant information for the analysis and thereby providing opportunities for reducing the amount of information that must be analysed. In some embodiments, the size of each time window is found experimentally and typically the size of each time window ranges from 5 to 15 seconds. To illustrate the max-aggregate function, there is provided the following example: Timestep 1 2| 3 4 5 6] 7 8 9 lol 11 12 13 14 15 value 0 0| 1 1 1 2| 3 3 3 4| 4 5 N / A N / A N / A window value 1 4 5 In this table, the first row represents the timesteps, the second row shows the value of a given variable at each timestep, and the third row displays the max-aggregated value within each window. The example depicts a scenario 12 timesteps into an event. It should be appreciated that within each window, the max-aggregated value corresponds to the maximum value that exists within that window. For instance, in the window spanning timesteps 1 to 5, the maximum value is 1, which is reflected in the corresponding window value. As the event progresses to the next timestep (timestep 13), the variable increases by 1. Consequently, the max-aggregated value in the corresponding window also increases by 1, as shown in the window value row. By applying the max-aggregate function, the model takes input at each timestep but aggregates the data over time windows to reduce the problem dimensionality. This approach helps to filter out noise and minor fluctuations in the data, focusing on the key moments or peaks that are likely to have a greater impact on the outcome of the sporting event. The resulting aggregated input sequences provide a more compact and informative representation of the event data, which can improve the efficiency and accuracy of the subsequent analysis performed by the Transformer Encoder Neural Network 124. In addition to the max-aggregation operation, the Data Pre-processing Module 120 may perform other data transformations and feature engineering tasks to further enhance the quality and relevance of the input data. This can include normalization techniques to scale the data into a consistent range, handling missing or incomplete data points, and encoding categorical variables into numerical representations suitable for the neural network. After applying the necessary transformations and aggregations, the Data Preprocessing Module 120 outputs the processed data in a format that is optimized for input into the Transformer Encoder Neural Network 124. In one or more embodiments this format may be a numerical vector, where each element represents a specific feature or variable of the sporting event at a given time window. The general format of this output vector offers several benefits for the subsequent analysis. Firstly, the aggregation process, as discussed earlier, reduces the dimensionality of the problem. This reduction in data points improves the overall speed and performance of the model by focusing on the most significant changes within each time window. Secondly, the Data Pre-processing Module 120 may apply standardisation to the values within the vector. This process transforms the variable distributions such that they all have a mean of 0 and a standard deviation of 1. Standardisation aids in training and ensures numerical stability of the transformer encoder. Such scaling helps to prevent issues that can arise from features having widely different scales. For example, consider a vector representing various features of a Mixed Martial Arts match at a given time window. The vector may include standardised values for the number of strikes landed, the duration of ground control, and the number of submission attempts. By standardising these values, the model can process them more effectively, regardless of their original scales or units of measurement. Statistics Neural Network The server 122 may also comprise a statistics neural network 132, which is used for processing player-specific data. This may be a feedforward neural network which is configured to analyse the player statistics data stored in the datastore 116 and encode it into a compact vector representation. The statistics neural network 132 takes as input various player-related features such as historical fight records, strike accuracy and volume, grappling and takedown statistics, physical attributes, and anthropometric data. It then processes this data through one or more neural network layers, configured to identify patterns and relationships within the player statistics. The output of this network is an entity vector that encapsulates relevant information about the players' historical performance and current state. Transformer Architecture The Transformer Encoder Neural Network 124 is responsible for processing the pre-processed input data and extracting meaningful patterns and relationships to facilitate real-time analysis of sporting event outcomes. This refers to the network's ability to identify statistically significant correlations, trends, and inter-dependencies in the input data that are predictive of event outcomes. These patterns may include, but are not limited to, sequences of actions that frequently lead to specific results, combinations of player statistics that correlate with certain performances, or temporal trends that indicate momentum shifts in the sporting event. This neural network architecture is based on the transformer model, which was originally designed for natural language processing tasks but has been adapted in this invention to handle time-series sporting event data (Vaswani et al., "Attention is all you need", 2017). Firstly, in one or more embodiments, the Input Embedding differs from language models that use token embeddings. An embedding, in this context, refers to a dense vector representation of input data that captures a semantic meaning. This adapted architecture employs a Multi-Layer Perceptron (MLP) for embedding the input features. This MLP takes the standardised numerical vectors from the Data Pre-processing Module 120 and transforms them into a higher-dimensional space suitable for the transformer layers. Secondly, the architecture focuses on Temporal Relationship Modelling. While traditional transformer models focus on relationships between language tokens, the transformer architecture in accordance with one or more embodiments of the invention is designed to capture relationships between different time steps in the sporting event data. The self-attention mechanism in the transformer encoder layers allows the model to weigh the importance of different time steps when analysing the current state of the sporting event. In one or more embodiments the input to the Transformer Encoder Neural Network 124 is a matrix of shape (T, F), where T represents the number of time steps and F represents the number of features per time step. These features are relevant to the sport being analysed and may include various aspects of player performance and game state. For example, in a combat sport such as MMA, the features might include positional data of the fighters, the number of strikes landed in a given time period, grappling control time, and other relevant metrics. In a team sport like soccer, features could include player positions, ball possession statistics, shot attempts, and tactical formations. This input matrix is first passed through an embedding layer, which transforms the input features into a higher-dimensional space, resulting in a matrix of shape (T, E), where E is the embedding dimension. The embedding layer is configured to create a more expressive representation of the input data, capturing complex relationships between features. The embedding for each timestep is generated via a multi-layered neural network, which takes as inputs the current variables which relate to the players. The embedded input matrix is then processed by a stack of transformer encoder layers. Each encoder layer consists of two main components: a multi-head self-attention mechanism and a position-wise feed-forward network. The multi-head self-attention mechanism allows the model to attend to different parts of the input sequence, capturing dependencies and relationships between time steps. This is achieved by computing attention weights between each pair of time steps, which determine the importance of one time step with respect to another. The self-attention mechanism is applied multiple times in parallel, with each head attending to different aspects of the input data. After the self-attention mechanism, the output is passed through a positionwise feed-forward network, which applies a series of non-linear transformations to each time step independently. Specifically, for each time step: 1. the input is multiplied by a weight vector; 2. a bias vector is added to the result; and 3. a non-linear activation function is applied In one or more embodiments a ReLU (Rectified Linear Unit) activation function is used, although it should be appreciated that other functions such as sigmoid or tanh could also be used. This feed-forward network helps the model capture complex patterns and relationships within each time step. It does this by allowing each node (time step) to update its representation based on its relationships with other nodes. For example, in a combat sport context, this could help extract meaningful relationships such as: • How the number of strikes in one time step might affect the probability of a knockout in a future time step • How the positioning of two players in one time step might indicate an advantage for an upcoming offensive push The output of each encoder layer is a matrix of shape (T, E), which serves as the input to the next encoder layer in the stack, where T represents the number of time steps and E represents the embedding dimension. The number of encoder layers is a hyperparameter that is configured during the model design phase, prior to training and task execution. This configuration is based on the complexity of the task and the available computational resources. By stacking multiple encoder layers, the model can learn hierarchical representations of the input data, capturing both local and global dependencies. One of the key advantages of the transformer architecture is its ability to handle variable-length input sequences. This is particularly useful in the context of natural language processing (NPL), and it has been recognised by the inventors as something particularly suitable for sporting event analysis where the duration of events may vary. The Transformer Encoder Neural Network 124 can process input sequences of different lengths by utilizing a technique called padding and masking. Padding involves appending dummy values to shorter sequences to make them the same length as the longest sequence in the batch, while masking ensures that the model does not attend to these dummy values during the self-attention computation. After the input data passes through the stack of encoder layers, the final output is a matrix of shape (T, E), where T represents the number of time steps and E represents the embedding dimension. This matrix is the final embedding, which contains information about all the time steps and their relationships. This output matrix can be further processed by pooling operations or additional feed-forward layers to obtain a fixed-size representation of the entire input sequence, which can be used for downstream tasks such as classification or regression. In one or more embodiments, global attention pooling is employed, which uses an attention mechanism to combine the values in all nodes. This pooling mechanism aggregates information across multiple time steps to create a single, compact representation. It should be appreciated that other pooling mechanisms can be employed. Such other mechanisms may include, but are not limited to: max pooling, wherein the maximum value across all time steps for each feature dimension is taken; average pooling, wherein the mean value across all time steps for each feature dimension is computed; and other variants of attention-based pooling. Such pooling operations serve to reduce the temporal dimension of the output, thereby rendering it suitable for tasks that require a fixed-size input. Examples of such tasks include sentiment analysis, which involves determining the emotional tone of a piece of text (e.g., positive or negative) such as may be used in natural language processing, and classification of time-series data with different durations, such as identifying patterns in variable-length sequences of sports event data. The Transformer Encoder Neural Network 124 is trained using a combination of synthetic data generated by the simulation module 114 and real-world historical event data stored in the datastore 116. The pre-training module 126 is responsible for training the model on the synthetic data This process configures the model to recognize general patterns and statistical relationships present in the simulated sporting events. Subsequently, the fine-tuning module 128 adapts the pre-trained model to real-world data, enabling it to capture the nuances and specific characteristics of actual sporting events. In one or more embodiments the pre-training module comprises a processor configured to: retrieve the player-statistics data from the datastore 116, invoke the simulation module 114, input the player statistics data into the simulation module 114 to generate synthetic event data; wherein the synthetic event data comprises a matrix representing a full span of a simulated event, the matrix comprising data for a plurality of time steps. The processor in the pre-training module in one or more embodiments can invoke the transformer encoder neural network 124; and input the synthetic event data generated by the simulation module 114 to the transformer encoder neural network 124 to train the transformer encoder neural network for building a pre-trained transformer encoder neural network. Output analysis module The Output Analysis Module 130 comprises task-specific neural network(s) 136 which are configured to interpret and process the combined real-time input vector and generate specific predictions or analyses. These networks are trained to perform particular output or classification tasks related to sporting event outcomes. For example, one task-specific network might predict the likely winner of the event, another could estimate the margin of victory, while a third might forecast the probability of specific game events occurring. In the context of Mixed Martial Arts (MMA) events, this may include predicting the likelihood of each fighter winning the match, the probability of the fight ending in a specific round, or the odds of different methods of victory (e.g., knockout, submission, or decision). The input to these networks will be described in more detail below. The Task-specific Neural Network(s) 136 receive as input the combined realtime input vector, which is formed by concatenating the real-time event evolution vector from the Transformer Encoder Neural Network 124 and the real-time entity vector from the statistics neural network 132. The output provided by the one or more task specific neural network(s) 136 is value representative of the probability of one or more event outcomes. This is achieved by applying a set of output layers, such as softmax or sigmoid activation functions, to the learned representations obtained from the Transformer Encoder Neural Network 124. The Output Analysis Module 130 may also incorporate domain-specific knowledge and rules to refine and contextualize the generated predictions. For example, in MMA events, the module may consider factors such as weight class, fighting style, or historical performance of the fighters to adjust the predictions accordingly. By integrating domain expertise, the module can provide more nuanced and reliable assessments of the event outcomes. The outputs of the Output Analysis Module 130 can be presented in various formats, depending on the specific application or user requirements. This may include visual representations such as probability charts, heat maps, or interactive dashboards that allow users to explore the predicted outcomes and underlying factors. The module may also generate textual summaries or reports that provide a narrative explanation of the predictions and insights. Pre-training the transformer encoder neural network Fig. 2 is a process flow diagram showing the key steps in the method for pre-training the transformer encoder neural network using synthetic event data governed by the pre-training module 126, implemented in the real-time sports event analysis system, according to an embodiment of the present invention. The pre-training process commences by retrieving player statistics data 210 from a datastore or an external source such as datastore 116. The player statistics data may include, but are not limited to, historical performance metrics, physical attributes, and playing styles of the athletes participating in the sporting event. The pre-training module 126 invokes the simulation module 114 and passes the player statistics data through the simulation module to generate synthetic event data 212. The Simulation module 114 may be configured to employ techniques such as Monte Carlo simulations to generate a large volume of simulated event data based on the retrieved player statistics data. This synthetic event data serves as a training dataset that captures a wide range of possible event scenarios and outcomes. The retrieved player statistics data and the generated synthetic event data are then subjected to data pre-processing methods 214. In this step, the pre-training module 116 may invoke the data pre-processing module 120 which may be configured to perform various data transformations and aggregations to prepare the synthetic data for input into the transformer encoder neural network. This includes but is not limited to applying a max-aggregate function over the time dimension of the synthetic data to generate aggregated synthetic event data. The pre-processing methods may also include data cleaning, normalization, feature scaling, and feature engineering techniques. The purpose of the data pre-processing stage is to transform the raw data into a format suitable for input into the transformer encoder neural network 124. The aggregated synthetic event data is then utilized as inputs to the transformer encoder neural network 124 at step 216. This generates a matrix of embeddings per aggregated time-step, within which is encoded information about the progress of the event at step 218. At step 220 a pooling mechanism is then applied over the time series dimension to generate a vector embedding that encodes information about the evolution of the event. In parallel to the transformer encoder, the initial player statistics data is used within the statistics neural network 132 at step 224 to generate a pre-training entity vector which comprises an embedding encoding the performance of players in previous events. The output of the pooling mechanism and the output of the statistics neural network are then concatenated at step 228 to form a single embedding. This embedding is now a vector that fully encodes the progression of an event, including both the past performance of players and their performance in the current event. The concatenated representation serves as input to one or more task-specific neural network(s) 136 at step 230. Each task-specific network is designed to perform a specific output or classification task related to the sporting event outcomes. For example, Task 1 may output the likely winner of the event, Task 2 may output the margin of victory, and Task 3 may output the likelihood of specific game events occurring. These task-specific networks are trained jointly with the transformer encoder neural network 124 and the statistics network 132 using appropriate loss functions and optimization techniques. Specifically, a loss is calculated for all tasks, and these individual losses are summed to produce a total loss. This total loss is then used in backpropagation to train the model. The optimization process employs a standard optimizer called AdamW, which is an adaptive learning rate optimization algorithm commonly used in deep learning applications. The weights of the statistics network, transformer encoder, and each task network are optimised via gradient descent, with the training loss calculated as the sum of the loss of each final task. The pre-processing steps applied to the player statistics data and the synthetic event data are specifically designed for this application case. While some of the individual techniques used, such as data cleaning, normalization, and feature scaling, may be conventional, the overall pre-processing pipeline, including the max-aggregate function and the specific transformations applied, is tailored to the requirements of this invention and the characteristics of the sporting event data being analysed. Fine-tuning the transformer encoder Once the pre-training stage using synthetic event data is completed, the transformer encoder neural network 124 undergoes a fine-tuning process to optimize it for real-world historical event data. This fine-tuning stage is implemented in order to improve the model's performance and its ability to analyse real-world sporting event outcomes. Figure 3 illustrates the steps involved in the fine-tuning process. The fine-tuning stage utilises the pre-trained weights of the trainable components, including the statistics neural network 132, the transformer encoder neural network 124 and the task-specific output networks 136. These components have been configured with initial weight values derived from patterns and relationships in the synthetic data during the pre-training stage. By utilising these pre-trained weights, the fine-tuning process can efficiently optimise the model to real-world data without reinitializing all parameters. In the initial phase of fine-tuning, all the pre-trained weights are kept fixed, except for the weights of the first layer in the transformer encoder. As the fine-tuning process progresses, the restrictions on the weights are gradually lifted, moving toward allowing all the trainable components to update their weights based on the real-world data. This enables the model to fully adapt to the nuances and specific characteristics of the real-world sporting events, refining its predictions and analysis capabilities. The fine-tuning stage utilises the same player statistics data 310 that was used to generate the synthetic data during pre-training. This data provides information about the players' historical performance, physical attributes, and other relevant characteristics. In addition to the player statistics data, the fine-tuning stage incorporates real-world historical event data 312. Such data contains time-series information captured from actual past sporting events, representing the state of the event at various time steps throughout its duration. To prepare the real-world event data for input into the transformer encoder, data pre-processing methods 314 are applied governed by the data pre-processing module 120. These methods may include aggregation techniques, such as the maxaggregate function, to generate fixed-length input sequences. The pre-processing steps reduce the dimensionality of the data and isolate the most relevant information for analysis. The pre-processed real-world event data is then fed into the pre-trained transformer encoder neural network 316, which generates an embedding matrix. This matrix encodes the temporal dependencies and relationships within the real-world event data. A pooling mechanism 318 is applied to the embedding matrix to generate a fine-tuned event evolution vector embedding 324, which represents a compact representation of the event's evolution over time. Concurrently, the player statistics data 310 is passed through the pre-trained statistics neural network 320, which generates a fine-tuned entity vector embedding 322. This embedding captures the relevant information about the players' historical performance and characteristics. The fine-tuned event evolution vector from the pooling mechanism 318 and the fine-tuned entity evolution vector from the statistics network 320 are then concatenated 326 to form a single comprehensive embedding. This embedding provides a holistic representation of the event, incorporating both the real-world event dynamics and the players' historical performance. Finally, the comprehensive embedding is used as input to the task-specific output networks 328. Each output network is responsible for predicting a specific aspect or outcome of the event, such as Task 1, Task 2, and Task 3. The output networks generate predictions based on the fine-tuned model's understanding of the real-world event data. During the fine-tuning process, the model's weights are optimised using gradient descent, with the objective of minimizing the sum of the losses from each task-specific output network. This optimisation allows the model to learn from the real-world data and improve its predictive accuracy. By fine-tuning the transformer encoder neural network using real-world historical event data, the model is adapted to the specific characteristics and dynamics of actual sporting events. This enables the model to provide more accurate and reliable predictions and analyses, taking into account both the pre-trained knowledge from the synthetic data and the nuances learned from the real-world data. Discrete and Continuous Predictions One or more embodiments in accordance with the present invention address the challenge of predicting both discrete and continuous outcomes in sporting events. Discrete outcomes are those with a limited predefined number of possible results, such as whether a team wins or loses. Continuous outcomes, on the other hand, can theoretically take any value within a range, such as the number of fouls committed by a team. In the context of machine learning approaches, continuous outcomes present a particular challenge due to the tendency of models to regress towards the mean of a distribution. This tendency is especially problematic in sporting events where certain outcomes can be significantly rarer than others. To address this issue, the present invention employs a novel “binning” approach to transform continuous outcomes into discrete categories. Figure 4 illustrates the binning process. Figure 4a shows the relative frequency distribution of a possible summary statistic at the end of an event. In this distribution, traditional models would tend to predict towards the mean, and re-weighting is challenging due to limited data availability. Figure 4b demonstrates the application of the binning approach, where similar outcomes are combined into discrete categories or "bins." The binning method offers several advantages including mitigating the model's bias towards predicting mean values. Furthermore, it allows for improved uncertainty quantification compared to continuous predictions and enables the use of classification techniques for outcomes that were originally continuous. In the implementation of one or more embodiments in accordance with the invention, for a given prediction task, the model outputs a vector of size B, where B represents the number of bins for the classification task. The components of this vector (b_0, b_1, ..., b_B-1) sum to 1 and are interpreted as probabilities for each bin. One or more embodiments in accordance with the invention implement calibration techniques for improving the accuracy of these probability outputs. These techniques adjust the raw model outputs to align more closely with observed frequencies of outcomes. The quality of model calibration is assessed through the use of calibration curves, an example of which is shown in Figure 5. A calibration curve is generated by placing or assigning the probability of each model class prediction to a value (“binning”) and comparing it to the frequency of correct classifications. For instance, all model predictions for a particular class where the model predicts a probability between 0% and 20% should, on average, be correct 10% of the time. This relationship is represented on the calibration curve by a line for each possible class. In a perfectly calibrated model, these lines would align with the y=x diagonal, which is represented by 510 in Figure 5. Figure 5 displays three distinct lines: - A dashed diagonal line 510 representing perfect calibration - A darker solid line 520 representing the calibration for one class - A lighter solid line 530 representing the calibration for the complementary class. The darker solid line 520 and the lighter solid line 530 each represent the calibration for a different class in a binary classification task. For example, the darker line 520 may show the calibration for predicting a win, while the lighter line 530 shows the calibration for predicting a loss. The symmetry of these curves around the diagonal (510) is expected in a binary classification task, as the probabilities for the two classes must sum to 1. When a curve is above the diagonal 510, it indicates that the model is underconfident for that class, predicting probabilities lower than the actual observed frequencies. Conversely, when a curve is below the diagonal 510, it suggests that the model is overconfident, predicting probabilities higher than the actual observed frequencies. The proximity of both curves to the diagonal 510 in Figure 5 indicates that the model is well-calibrated across most of the probability range, with slight deviations at very low and very high probabilities. By employing this binning approach and ensuring proper calibration, one or more embodiments of the present invention provide a robust method for predicting both discrete and originally continuous outcomes in sporting events. This approach enables more accurate and reliable predictions across a wide range of possible outcomes, enhancing the overall performance and applicability of the machine learning model in sports analytics and betting scenarios. The use of calibration curves allows for a nuanced understanding of the model's performance across different probability ranges and for different outcome classes. This information is valuable for further refining the model, interpreting its predictions in practical applications, and understanding any biases the model might have towards certain outcomes or probability ranges. By ensuring proper calibration, the system provides more reliable probability estimates, which is crucial in the context of sports betting where accurate risk assessment is paramount. Example of a Real-Time Event Analysis System Using a Pre-Trained Transformer Encoder Neural Network in accordance with an embodiment of the present invention Figure 1 illustrates a high-level system diagram of a real-time sports event analysis system (100) in accordance with an embodiment of the present invention. The system (100) demonstrates how the pre-trained components are utilised for realtime event analysis, building upon the pre-training and fine-tuning processes described earlier. In one or more embodiments of the present invention, a system (100) for analysing real-time events using a pre-trained transformer encoder neural network (124) is provided. This system is particularly well-suited for applications in sports analytics, and real-time decision-making scenarios. The system comprises a memory store (112) for storing the pre-trained transformer encoder neural network 124, a pre-trained statistics neural network 132 and one or more task specific neural network(s) 136. These networks have been previously trained and fine-tuned using the methods described earlier, incorporating both synthetic and real-world historical data to promote robust performance across a wide range of scenarios. A communication interface (134) is included in the system, configured to receive real-time multivariate data representing the state relationship between a first entity and a second entity at respective times in a real-time event. In the context of a sporting event, these entities may be individual players or teams participating in the event. The multivariate data comprises time series information capturing various aspects of the event as it unfolds. The system further includes a processor (110) configured to execute several operations. For example, the processor (110) invokes the pre-trained transformer encoder neural network (124) and inputs the real-time multivariate data. This network processes the time series data, capturing the temporal dynamics and relationships within the event. The output of this process is a real-time event evolution vector, which encodes the progression of the event up to the current moment. Concurrently, the processor (110) invokes the pre-trained statistics neural network 136, which is typically a feedforward neural network. This network processes the same real-time multivariate data to generate a real-time entity vector, capturing relevant information about the entities involved in the event. The processor (110) then combines the real-time event evolution vector and the real-time entity vector to create a combined real-time input vector. This combination is typically achieved through concatenation, resulting in a comprehensive representation of both the event's progression and the entities' characteristics. Finally, the processor (110) invokes one or more pre-trained task specific neural networks and inputs the combined real-time input vector. These networks generate outputs indicative of one or more likely outcomes of the real-time event. In a specific implementation for sporting events, the system (100) can be applied to a variety of sports, including but not limited to rugby, American football, basketball, baseball, soccer, hockey, tennis, boxing, and mixed martial arts. The flexibility of the system allows it to be adapted to different sports by adjusting the input features and fine-tuning the networks accordingly. The real-time multivariate data typically comprises time series data representing the state of the event at multiple time steps. To optimize processing, the data pre-processing module (120) may apply a max-aggregate gate over the time dimension of this data, taking the maximum value within a defined window size to generate aggregated real-time event data. This aggregation helps to focus on significant changes and reduces computational load. When processing the aggregated data through the transformer encoder neural network (124), the system generates an encoded matrix comprising an embedding for each time step. These embeddings encode information about the progress of the real-time event. A pooling mechanism is then applied over the time dimension of this encoded matrix to produce the real-time event evolution vector. To handle continuous outcome variables, the output analysis module (130) employs a binning process, converting these variables into discrete bins representing ranges of values. The task-specific output neural networks (136) then generate predictions indicating the likelihood of the outcome falling within each discrete bin. This approach allows for more nuanced predictions and better handling of rare events. To ensure the accuracy of the probability predictions, the output analysis module (130) applies a calibration process. This process enhances the output to accurately reflect the true frequency of the outcomes, improving the reliability of the predictions for decision-making purposes. The data input module (118) works in conjunction with the communication interface (134) to receive and process live event data feeds during real-time operation. The processor (110) is configured to repeatedly process this data at defined intervals, generating updated predictions throughout the course of the event. This continuous updating allows for dynamic decision-making and real-time adjustments in applications such as live betting or in-game strategy formulation. It will be understood by those skilled in the art that the drawings are merely diagrammatic and that further items of equipment may be required in a commercial apparatus. The position of such ancillary items of equipment forms no part of the present invention and is in accordance with conventional practice in the art. Insofar as embodiments of the invention described above are implementable, at least in part, using a software-controlled programmable processing device such as a general purpose processor or special-purposes processor, digital signal processor, microprocessor, or other processing device, data processing apparatus or computer system it will be appreciated that a computer program for configuring a programmable device, apparatus or system to implement methods and apparatus is envisaged as an aspect of the present invention. The computer program may be embodied as any suitable type of code, such as source code, object code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. The instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled and / or interpreted programming language, such as, Liberate, OCAP, MHP, Flash, HTML and associated languages, JavaScript, PHP, C, C++, Python, Nodejs, Java, BASIC, Perl, Matlab, Pascal, Visual BASIC, ActiveX, assembly language, machine code, and so forth. A skilled person would readily understand that term “computer” in its most general sense encompasses programmable devices such as referred to above, and data processing apparatus and computer systems. Suitably, the computer program is stored on a carrier medium in machine readable form, for example the carrier medium may comprise memory, removable or non-removable media, erasable or non-erasable media, writeable or re-writeable media, digital or analogue media, hard disk, floppy disk, Compact Disk Read Only Memory (CD-ROM), Company Disk Recordable (CD-R), Compact Disk Rewriteable (CD-RW), optical disk, magnetic media, magneto-optical media, removable memory cards or disks, various types of Digital Versatile Disk (DVD) subscriber identity module, tape, cassette solid-state memory. The computer program or elements thereof may be stored transiently as part of their execution or implementation. The computer program may be supplied from a remote source embodied in the communications medium such as an electronic signal, radio frequency carrier wave or optical carrier waves. Such carrier media are also envisaged as aspects of the present invention. As used herein any reference to “one embodiment” or “an embodiment” means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment. As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, unless expressly stated to the contrary, “or” refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present). In addition, use of the “a” or “an” are employed to describe elements and components of the invention. This is done merely for convenience and to give a general sense of the invention. This description should be read to include one or at least one and the singular also includes the plural unless it is obvious that it is meant otherwise. In view of the foregoing description, it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention. The following alternatives and variations are provided as examples and are not intended to be exhaustive. Sequential Processing: While the described embodiment processes the realtime multivariate data concurrently through the pre-trained transformer encoder neural network and the pre-trained statistics neural network, these operations could be performed sequentially. Sequential processing might be beneficial in systems with limited parallel processing capabilities or when prioritizing certain analyses over others. Alternative Neural Network Architectures: The pre-trained statistics neural network, described as typically being a feedforward neural network, could be implemented using other architectures such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), or convolutional neural networks (CNNs), depending on the specific requirements of the sport or event being analysed. Variable Window Sizes: The max-aggregate function could be applied with variable window sizes rather than a fixed window size. This could allow for more granular analysis in critical portions of an event while using larger windows for less eventful periods, potentially optimizing both processing efficiency and predictive accuracy. Different Pooling Mechanisms: While the described embodiment uses a specific pooling mechanism to generate the real-time event evolution vector, alternative pooling methods such as average pooling, max pooling, or attentionbased pooling could be employed, potentially offering different trade-offs between computational efficiency and information retention. Multi-modal Input Data: The system could be extended to incorporate additional types of input data beyond the described multivariate time series data. For example, video feeds, audio data, or external factors like weather conditions could be integrated to provide a more comprehensive analysis. Ensemble Methods: Instead of relying on a single pre-trained transformer encoder neural network, an ensemble of multiple networks could be used, potentially improving robustness and accuracy through techniques like bagging or boosting. Adaptive Binning: The binning process for continuous variables could be made adaptive, adjusting bin sizes and boundaries based on the distribution of observed outcomes in real-time, potentially improving the system's ability to handle evolving event dynamics. Transfer Learning: The pre-training process could be modified to incorporate transfer learning techniques, allowing the system to leverage knowledge gained from one sport to improve performance in analysing another related sport. Online Learning: While the described system uses pre-trained networks, an online learning component could be added to allow the system to continuously refine its models based on real-time outcomes, potentially improving accuracy overtime. Alternative Calibration Techniques: Beyond the described calibration process, other techniques such as Platt scaling, isotonic regression, or beta calibration could be employed to ensure the accuracy of probability predictions. Hierarchical Prediction: The system could be extended to provide hierarchical predictions, offering both high-level outcomes (e.g., match winner) and more granular predictions (e.g., specific play outcomes) simultaneously. Custom Loss Functions: The training process could utilize custom loss functions tailored to the specific requirements of different sports or betting scenarios, potentially improving the system's performance for particular use cases. Privacy-Preserving Techniques: For applications where data privacy is a concern, the system could incorporate federated learning or differential privacy techniques to protect sensitive information while still benefiting from diverse data sources. The scope of the present disclosure includes any novel feature or combination of features disclosed therein either explicitly or implicitly or any generalisation thereof irrespective of whether or not it relates to the claimed invention or mitigate against any or all of the problems addressed by the present invention. The applicant hereby gives notice that new claims may be formulated to such features during prosecution of this application or of any such further application derived therefrom. In particular, with reference to the appended claims, features from dependent claims may be combined with those of the independent claims and features from respective independent claims may be combined in any appropriate manner and not merely in specific combinations enumerated in the claims.

Claims

1. A system for training a transformer encoder neural network for analysing time series multivariate data, the system comprising:a data store for storing time series multivariate data representing a state relationship between a first entity and a second entity at respective times in a prior event; anda processor configured to:invoke a pre-training module configured to:retrieve the time series multivariate data from the data store;invoke an event simulation process;input the time series multivariate data into the event simulation process to generate synthetic event data; wherein the synthetic event data comprises a matrix representing a full span of a simulated event, the matrix comprising data for a plurality of time steps;invoke a transformer encoder neural network; andinput the synthetic event data generated by the event simulation process to the transformer encoder neural network to train the transformer encoder neural network for building a pretrained transformer encoder neural network.

2. A system according to claim 1, wherein the pre-training module is further configured to invoke a data pre-processing module, wherein the data preprocessing module is configured to apply a max-aggregate gate over a time dimension of the matrix to determine a maximum value within a defined window size to generate aggregated synthetic event data.

3. A system according to claim 2, wherein the pre-training module is further configured to:generate an encoded matrix comprising an embedding for each time step in the aggregated synthetic event data, wherein the embedding encodes information relating to the progress of the simulated event; andapply a pooling mechanism over a time dimension of the encoded matrix to generate a pre-training event evolution vector representative of an evolution of the simulated event over a respective time step, wherein the pre-training event evolution captures encoded information relating to evolution of the simulated event.

4. A system according to claim 3, wherein the pre-training module is further configured to:Invoke a neural network configured as a feedforward neural network to process the time series multivariate data to generate an encoded pre-training entity vector and designate the neural network as a pre-trained neural network after processing the time series multivariate data.

5. A system according to claim 4, wherein the pre-training module is further configured to concatenate the encoded pre-training entity vector with the pre-training event evolution vector to generate a composite pre-training vector.

6. A system according to claim 5, wherein the pre-training module is further configured to use the composite pre-training input vector to pre-train one or more output neural networks configured to generate an output indicative of a likely outcome of the simulated event.

7. A system according to claim 6, further comprising a fine-tuning module, and wherein the processor is further configured to invoke the fine-tuning module to fine-tune the pre-trained transformer encoder neural network using real-world time series multivariate data, the real-world time series multivariate data representing a full span of a real-world event at respective time steps.

8. A system according to claim 7, wherein the fine-tuning module is configured to: invoke the data pre-processing module configured to:apply a max-aggregate function over a time dimension of the real-world time series multivariate data to generate aggregated real-world event data, the max-aggregate function taking a maximum value within a defined window size;process the aggregated real-world event data using the pretrained transformer encoder neural network to generate a fine-tuned event evolution vector; andinvoke the pre-trained neural network to process the time series multivariate data to generate a fine-tuned entity vector, wherein the fine-tuned event evolution vector is representative of an evolution of the real-world event up to a current time step.

9. A system according to claim 8, wherein the fine-tuning module is further configured to:combine the fine-tuned event evolution vector and the fine-tuned entity vector to generate a composite fine-tuning input vector representing the evolution of the real-world event based on the entity’s state in response to previous events; anduse the composite fine-tuning input vector to fine-tune the one or more pre-trained output neural networks, wherein the one or more fine-tuned output neural networks generate an output indicative of the likely outcomes of one or more real-world events.

10. A system according to any of claim 2 to claim 9, wherein the transformer encoder comprises a plurality of transformer encoder layers, each layer comprising:an attention mechanism and a feedforward layer configured to extract short-term, long-term, and higher-order relationships between the entities' states across the time sequence;wherein each transformer encoder layer is configured to:take as input the aggregated synthetic event data matrix of dimension (T, F), where T is the number of time steps and F is the number of features per time step in the aggregated synthetic event data,successively extract temporal relationships between the entities' time-varying state data, andapply a linear transformation to the aggregated synthetic event data, preferably using a feedforward layer.11 .A system according to claim 9 , wherein the processor is further configured to apply a binning process to convert continuous outcome variables into discrete bins representing ranges of values, and wherein the one or more output neural networks output a value representative of the probability of the outcome falling within each discrete bin; and the processor is further configured to apply a calibration process to the value for enhancing the predicted probability to reflect the frequency of the outcomes.

12. A system according to any preceding claim, wherein the time series multivariate data represents a state relationship between a first participant and a second participant at respective times in a sporting event, wherein the sporting event is selected from the group consisting of: rugby, American football, basketball, baseball, soccer, hockey, tennis, boxing, cricket and martial arts including mixed martial arts.

13. A system for analysing real-time events using a pre-trained transformer encoder neural network, the system comprising:a memory for storing a fine-tuned transformer encoder neural network, a fine-tuned neural network, and one or more fine-tuned output neural networks;a interface configured to receive real-time multivariate data representing a state relationship between a first entity and a second entity at respective times in a real-time event; anda processor configured to:invoke a data pre-processing module to process the real-time multivariate data to generate aggregated real-time event data;invoke the fine-tuned transformer encoder neural network;input the aggregated real-time event data to the fine-tuned transformer encoder neural network to generate a real-time eventevolution vector, wherein the real-time event evolution vector captures encoded information relating to the progression of the real-time event;invoke the fine-tuned neural network;input the real-time multivariate data to the fine-tuned neural network to generate a real-time entity vector, wherein the real-time entity vector captures information relating the performance and characteristics of the first entity and the second entity together;concatenate the real-time event evolution vector and the realtime entity vector to generate a combined real-time input vector, wherein the combined real-time entity vector represents a encoding of the current state and evolution of the real-time event and captures information relating the performance and characteristics of the first entity and the second entity;invoke the one or more fine-tuned output neural networks; and input the combined real-time input vector to the one or more fine-tuned task specific neural networks to generate an output indicative of one or more likely outcomes of the real-time event.

14. A system according of claim 13, wherein the data pre-processing module is further configured to apply a max-aggregate gate over a time dimension of the real-time multivariate data to determine a maximum value within a defined window size to generate aggregated real-time event data.

15. A system according to claim 13 or claim 14, wherein the processor is further configured to apply a binning process to convert continuous outcome variables into discrete bins representing ranges of values, and wherein the one or more fine-tuned output neural networks a value representative of the outcome falling within each discrete bin, wherein the processor is further configured to apply a calibration process to the value for enhancing the predicted probability to reflect the frequency of the outcomes.

16. A system according to any of claim 13 to claim 15, wherein the communication interface is configured to receive the real-time multivariate data as a live stream during the real-time event, and the processor is configured torepeatedly process the real-time multivariate data at defined intervals during the real-time event to generate updated outputs indicative of the one or more likely outcomes.

17. A system according to any of claim 13 to claim 16, wherein the real-time event is a sporting event, and the first entity and the second entity are participants in the sporting event, wherein the sporting event is selected from the group consisting of: rugby, American football, basketball, baseball, soccer, hockey, tennis, boxing, and mixed martial arts.

18. A method for training a transformer encoder neural network for analysis of time series multivariate data, the method comprising:storing, in a data store, time series multivariate data representing a state relationship between a first entity and a second entity at respective times in a prior event;retrieving the time series multivariate data from the data store;invoking an event simulation process;inputting the time series multivariate data into the event simulation process to generate synthetic event data; wherein the synthetic event data comprises a matrix representing a full span of a simulated event, the matrix comprising data for a plurality of time steps;invoking a transformer encoder neural network; andinputting the synthetic event data generated by the event simulation process to the transformer encoder neural network to train the transformer encoder neural network for building a pre-trained transformer encoder neural network.

19. A method according to claim 18, further comprising:invoking a data pre-processing module;applying, by the data pre-processing module, a max-aggregate gate over a time dimension of the matrix to determine a maximum value within a defined window size to generate aggregated synthetic event data.

20. A method according to claim 19, further comprising:generating an encoded matrix comprising an embedding for each time step in the aggregated synthetic event data, wherein the embedding encodes information relating to the progress of the simulated event; andapplying a pooling mechanism over a time dimension of the encoded matrix to generate a pre-training event evolution vector representative of an evolution of the simulated event over a respective time step, wherein the pre-training event evolution captures encoded information relating to evolution of the simulated event.21 .A method according to claim 20, further comprising:invoking a neural network configured as a feedforward neural network;processing the time series multivariate data using the neural network to generate an encoded pre-training entity vector; anddesignating the neural network as a pre-trained neural network after processing the time series multivariate data.

22. A method according to claim 21, further comprising:Concatenating the encoded pre-training entity vector with the pre-training event evolution vector to generate a composite pre-training vector.

23. A method according to claim 22, further comprising:using the composite pre-training input vector to pre-train one or more output neural networks configured to generate an output indicative of a likely outcome of the simulated event.

24. A method according to claim 23, further comprising:fine-tuning the pre-trained transformer encoder neural network using real-world time series multivariate data, the real-world time series multivariate data representing a full span of a real-world event at respective time steps.

25. A method according to claim 24, wherein the fine-tuning comprises:applying a max-aggregate function over a time dimension of the real-world time series multivariate data to generate aggregated real-world event data, the max-aggregate function taking a maximum value within a defined window size;processing the aggregated real-world event data using the pre-trained transformer encoder neural network to generate a fine-tuned event evolution vector; andprocessing the time series multivariate data using the pre-trained neural network to generate a fine-tuned entity vector, wherein the fine-tuned event evolution vector is representative of an evolution of the real-world event up to a current time step.

26. A method according to claim 25, further comprising:combining the fine-tuned event evolution vector and the fine-tuned entity vector to generate a composite fine-tuning input vector representing the evolution of the real-world event based on the entity’s state in response to previous events; andusing the composite fine-tuning input vector to fine-tune the one or more pre-trained output neural networks, wherein the one or more fine-tuned output neural networks generate an output indicative of the likely outcomes of one or more real-world events.

27. A method according to claim 19, wherein the transformer encoder comprises a plurality of transformer encoder layers, each layer comprising an attentionmechanism and feedforward layers configured to extract short-term, long-term, and higher-order relationships between the entities' states across the time sequence; wherein each transformer encoder layer is configured to:take as input the aggregated synthetic event data matrix of dimension (T, F), where T is the number of time steps and F is the number of features per time step in the aggregated synthetic event data,successively extract temporal relationships between the entities' timevarying state data, andapply a linear transformation to the aggregated synthetic event data using a feedforward layer.

28. A method according to claim 25, further comprising:applying a binning process to convert continuous outcome variables into discrete bins representing ranges of values;predicting, by the one or more output neural networks, a probability of the outcome falling within each discrete bin; andapplying a calibration process to the predicted probabilities for enhancing the predicted probabilities to accurately reflect the true frequency of the outcomes.

29. A method according to claim 18, wherein the time series multivariate data represents a state relationship between a first participant and a second participant at respective times in a sporting event, wherein the sporting event is selected from the group consisting of: rugby, American football, basketball, baseball, soccer, hockey, tennis, boxing, cricket and martial arts including mixed martial arts.

Images