Systems and methods for controlling interactive hybrid environments representing motorized sports events on a track.

Abstract

A computer implementation method for controlling an interactive hybrid environment representing motorized sports events on a track is described. The interactive hybrid environment includes representations of real vehicles and virtual vehicles on the track. The method includes: receiving a stream of real sensor data, including: real kinematic data of real vehicles on the track and real control data regarding driver control of the real vehicles, the real kinematic data being captured by infrared sensors at the track, and the real control data being captured by vehicle sensors and obtained via a telemetry system from the real vehicles; using the real kinematic data to determine the position and kinematic behavior of the real vehicle representations within the interactive hybrid environment; using the real control data and the real kinematic data to create a black-box determination of the position of the real vehicles on the track based on the real control data; receiving a computer-generated control data stream obtained through user interaction with a computer presenting the interactive hybrid environment to the user and capturing user input to control the kinematic behavior of the virtual vehicle representations; and determining the position and kinematic behavior of the virtual vehicle representations within the interactive hybrid environment using the black-box determination and the computer-generated control data.

Description

Technical Field

[0001] This invention relates to systems and methods for controlling interactive hybrid environments representing motorized sports events on a racetrack. More specifically, and not exclusively, this invention relates to methods and systems for enabling remote gaming computers and potentially other entertainment devices to participate at scale in realistic motorized sports events. The invention also extends to the real-time capture and live streaming of highly accurate real-time vehicle tracking and control data from live motorized sports events, and the use of this data to provide a new form of gaming or viewing experience in which remote players can compete against actual participants in the motorized sports event in a more interactive manner, and remote spectators can participate in the event. The capture of accurate kinematic data enables the system and method to be applied to a range of other types of indoor and outdoor sports events with large-scale participation, such as football, basketball, cycling, and skiing. Background Technology

[0002] Various methods have been used to provide realistic systems and approaches for virtual representations of athlete control in motorized sports events. Most of these methods are entirely virtual, with complex models determining the kinematic behavior of virtual vehicles in response to user actuator input, attempting to provide a realistic appearance. Realism is typically achieved by building models using real kinematic data obtained by recording the vehicle's movement during a race. However, these models are not only complex and difficult to build, but they are also often based on inaccurate kinematic data. Furthermore, this existing approach is usually geared towards a single or a few athletes, meaning the scope of interaction with other athletes is very limited. All of this results in unscalable and unrealistic simulations and gaming experiences.

[0003] Existing technologies that attempt to limit simulations for player control to interact with real live racing have not extended to providing practical methods for enabling large numbers of players and followers worldwide to simultaneously engage with real drivers, cars, racing teams, tracks, off-track and event environments in a way that demonstrates the satisfaction and challenge of computer game players. This would support large-scale competition in esports leagues and provide non-game followers with features that enhance their immersion and enjoyment of live racing events. Achieving any of these goals would broaden and enhance the appeal of racing, for example, to a fan base that extends from traditional television viewers or individual driver fans to core fans of racing computer games and users of “top” racing and technical data services.

[0004] The inventors have drafted prior patent GB2518602B, which describes a system, method, and technique for, among other applications, to passively and accurately track vehicles in real-world motorized vehicle racing (such as Formula 1) under all conditions except the most extreme weather conditions, providing computer players with the ability to replace one of these vehicles with a virtual car and compete in a live race. The data provided by GB2518602B pertains to the positional information of motorized vehicles on the track, which is sensed non-invasively by a single IR tracking sensor located at an effective height (1-2 km) above the track (e.g., on a helicopter, drone, or lighter-than-air aircraft). This relies on the field of view (FOV) of a single sensor encompassing the entire set area. Using Formula 1 as an example, this method is impractical when the FOV is obstructed by vegetation (overhanging trees), grandstands, or various buildings and other structures on urban tracks. Therefore, this prior art method has some limitations in its practical application.

[0005] Video games that interact with live events (described in-game methods where a player-controlled virtual car interacts with a representation of a real car in a limited and somewhat artificial manner via data streamed from live events to a traditional computer racing game) are known, see, for example, US2010 / 0271367. The performance of the virtual car is determined by a combination of the player input and the software model of the car and its environment. Physics-based models, which are part of video game software, are subject to many limitations in terms of the fidelity they can achieve when attempting to simulate the very complex and dynamic typical scenarios of racing events. This results in poor quality of interaction between the virtual car and the representation of the real car.

[0006] Other existing technical documents (e.g., GB2365360A) have attempted to address this major deficiency, suggesting that the physical model simulation of virtual vehicle dynamics and its environment in computer games can be achieved by preprocessing data collected from practice runs or by real-time processing of performance data transmitted from real vehicles and their environments, thereby creating an "optimal physical model" of the vehicle and its environment that can then be controlled by the player. This approach suffers from the same fundamental limitations because no practical means of achieving very high-performance tracking are described. Furthermore, software models involving very complex scenarios of motor vehicles operating in very complex environments involve so many variables and complex relationships between them that even if some of the more obvious variables are continuously measured and used, the model either: a) quickly deviates from reality due to numerous approximations and flaws in the model; or b) requires enormous computational power, making real-time operation impossible even on the largest computers.

[0007] Other existing technical documents (such as US6155927) describe systems in an abstract form that enable computer game players to compete against live and recorded real matches, but do not provide specific methods and system descriptions for actually implementing the abstract concepts described in GB2518602B.

[0008] GB2585165A, also co-authored by the inventors, describes a method for infrared (IR) tracking of ordinary vehicles, trucks, etc., on highways, roads, and street traffic networks. The tracking device is located on low-level infrastructure, such as lampposts, and an IR transmitter is placed on the vehicle for detection, or an IR reflector is placed on the vehicle. Optionally, the tracking device includes IR lights. The tracking device then transmits high-precision, real-time tracking data of the vehicle and its neighbors to the vehicle to assist in navigation and autonomous or semi-autonomous driving. The tracking devices can be linked together to form a linear, high-integrity network, enabling vehicle tracking along continuous, uniform road segments. The arrangement of the tracking devices in GB2585165A has features unrelated to this invention (e.g., the aggregation and transmission of high-integrity data from the tracking devices to the motor vehicle itself). Furthermore, motorized sports events typically require tracking high-performance vehicles, such as motor vehicles and motorcycles, on closed, non-uniform tracks with challenging geometry (sharp turns, bends, ramps, peaks, pit stops, etc.), where motor vehicles travel at extremely high accelerations, speeds, turning speeds, etc. The systems and methods described in GB2585165A are inadequate because they are designed to monitor vehicles that are typically moving at slower speeds.

[0009] Therefore, the object of this invention is to overcome the limitations of the aforementioned prior art documents. Furthermore, in various embodiments, it is desirable to overcome the limitations of the precise tracking capabilities defined in GB2518602B and described above, and to provide, for the first time, improved, practically implementable systems and methods for high-performance motorsports events involving large-scale interactive virtual racing with numerous players and followers worldwide in complex environments. In various embodiments, it is also desirable to clearly indicate how virtual drivers can interact with real races and real drivers to enhance the player experience and enable fair operation of local, regional, national, or global esports or competitive racing events while being highly integrated with real motorsports events and races. This invention is also applicable to the creation and use of recorded data regarding real motorsports events, wherein kinematic data and vehicle control data (possibly driver input) of competing vehicles are recorded or accurately reconstructed to meet the accuracy and latency requirements for computer games specified in GB2518602B. Summary of the Invention

[0010] According to one aspect of the present invention, a computer-implemented method is provided for controlling an interactive hybrid environment representing motorized sports events on a track, the interactive hybrid environment including representations of real vehicles and virtual vehicles on the track, the method comprising: receiving a real data stream, the real data including real kinematic data of real vehicles on the track and real control data regarding control of the real vehicles by a driver, the real kinematic data being captured by infrared sensors on the track, the real control data being captured by vehicle sensors and obtained via a telemetry system from the real vehicles; using the real kinematic data to determine the position and kinematic behavior of the real vehicle representations in the interactive hybrid environment; using the real control data and the real kinematic data to create a black-box determination of the position of the real vehicles on the track based on the real control data; receiving a computer-generated control data stream obtained through user interaction with a computer that presents the interactive hybrid environment to the user and captures user input to control the kinematic behavior of the virtual vehicle representations; and determining the position and kinematic behavior of the virtual vehicle representations in the interactive hybrid environment using the black-box determination and the computer-generated control data.

[0011] Using real data, including kinematic and control data, in an interactive hybrid environment enables precise black-box determination. This, in turn, provides a reference for received computer-generated control data, allowing for precise and realistic kinematic control of the virtual vehicle relative to a representation of a real vehicle on the track. For example, the degree to which a user operates the actuators is precisely reflected in its effect on the virtual vehicle in the same way that a driver's maneuvers in a real vehicle affect the kinematic data (e.g., speed, direction, acceleration, and position) of the real vehicle. This provides a sense of realism impossible to achieve with existing methods and accurately reflects movement in both the virtual and real domains. Furthermore, this combination of features overcomes the interoperability problem of providing vehicle representations from different domains in the same hybrid environment in a realistic and accurate manner. For example, when the real data is live data—data streamed from events occurring simultaneously with control in the interactive hybrid environment—the invention enables a virtual driver in the virtual domain to compete in real time with a real driver in the real domain, which was previously impossible.

[0012] Preferably, the real sensor data includes real kinematic data of multiple real vehicles on the track and real control data regarding the control exerted by the respective drivers on each of the multiple real vehicles. Embodiments of the invention are designed to capture, for example, the competitive behavior of multiple vehicles holistically (as seen in Formula One racing). A real data stream representing the behavior of multiple real vehicles can create such a competitive environment. Each real data stream can be processed individually and may also include a unique vehicle identifier.

[0013] Similarly, computer-generated control data streams can include multiple computer-generated data streams, each generated through interactions between different users and their respective computers, and the capture of corresponding user input. This allows multiple users to be associated with sporting events and advantageously enables large-scale gaming. Each computer-generated data stream can be processed individually and may also include a unique computer device identifier.

[0014] Preferably, in cases where there are fewer computer-generated data streams with multiple real vehicles than multiple streams, the method may further include linking two or more subsets of multiple virtual vehicle representations with a single real vehicle representation to create a linked representation. Clearly, in large-scale gaming environments, the number of users participating via gaming computers / devices far exceeds the number of vehicles in a race (because there are typically physical limitations on the number of vehicles that can participate in a race for security reasons). Therefore, by linking two or more virtual vehicle representations to a single representation of a real vehicle, any number of users can be accommodated. This allows the method to scale to large-scale gaming scenarios where hundreds of thousands of users in a virtual domain can compete simultaneously with drivers in a real domain.

[0015] In some embodiments, the method further includes using a linked representation to represent a subset of multiple virtual vehicle representations within an interactive hybrid environment, wherein the positions of the virtual vehicles in the subset are within the tolerances of the real vehicles.

[0016] In many embodiments, the multiple computer-generated data streams are typically several times larger than the number of real vehicles, and the linking step may include linking each of the multiple computer-generated data streams to multiple real vehicle representations in a uniformly distributed manner. In other embodiments where the multiple computer-generated data streams are several times larger than the number of real vehicles, the linking step may include linking each of the multiple computer-generated data streams to multiple real vehicle representations in a logarithmic distribution. Regardless of the approach, it is possible to accommodate a large number of players / users in an interactive hybrid environment, where there may be a small number of real vehicle representations.

[0017] In one embodiment, there may be a central game server that creates the interactive hybrid environment and then provides it to all gaming devices via a communication network. In this case, the method may include updating the interactive hybrid environment using new positions of the real and virtual vehicle representations determined by received real sensor data and computer-generated data; generating the updated interactive hybrid environment; and broadcasting the updated interactive hybrid environment from the central server to multiple remote computers. A centralized solution may require greater processing power but is relatively easy to update and control.

[0018] In an alternative embodiment, each user / player's gaming device can generate an interactive hybrid environment locally, creating multiple such local environments. In this case, the method may include broadcasting black-box determination and real sensor data from a central server to multiple remote computers; generating an interactive hybrid environment on each remote computer; updating the interactive hybrid environment with new positions of the real vehicle representation and virtual vehicle representation determined by the received real sensor data and computer-generated data; and sending the new positions of the virtual vehicle representation to the central server. This distributed solution may require more management but is less susceptible to potential bottlenecks and is generally less affected by time delays and lags in the generation of interactive hybrid environments.

[0019] In some embodiments, the method further includes using an artificial intelligence engine to alter the association between computer-generated control data and the synthetic position of the virtual vehicle, the AI ​​engine being determined with reference to a black box. The use of such an AI engine can assist each player in controlling their virtual vehicle. This assistance can provide a handicap factor, which allows weaker virtual players to compete more fairly with professional drivers in the real domain.

[0020] In most embodiments, the received real kinematic data includes longitudinal position data relative to the track, lateral position data relative to the track, and vehicle orientation data relative to the track. These types of data enable the vehicle's kinematic behavior to be accurately mapped into the virtual domain.

[0021] In some embodiments, the real-world control data includes one or more of the following: the steering wheel position, accelerator position, brake pedal position, and gear selection of the real vehicle. These are typical control data, and the telematics system provides information on these typical control data. This typical control data helps determine the driver control input, which determines the real vehicle's track position. Furthermore, advantageously, this data can be readily associated with corresponding actuators that can be controlled by a driver in the virtual world.

[0022] Different embodiments generate an interactive hybrid environment from different real-world data sources. In one embodiment, the source is a storage of previously recorded real-world data. Therefore, the method may also include retrieving real-world sensor data from a data store that already stores copies of the real-world sensor data when generating the real-world sensor data. In an alternative embodiment, the source is the sporting event itself, and in this case, the receiving step includes receiving the real-world sensor data substantially in real-time as the sporting event occurs.

[0023] In one embodiment, the real sensor data stream has a sampling rate of at least 25 Hz, capturing the position of a real vehicle at a given time point and providing that position to the interactive hybrid environment within 40 milliseconds of capture. This enables the real domain to be implemented in real time within a virtual domain operating at a minimum refresh rate of 25 Hz. More preferably, in some embodiments, the real sensor data stream has a sampling rate of at least 60 Hz, capturing the position of a real vehicle at a given time point and providing that position to the interactive hybrid environment within 16.7 milliseconds of capture. This refresh rate is typically present in most computer monitors, thus supporting high-quality representation of real-time events within the virtual domain.

[0024] To aid in the generation of interactive virtual environments, the method may also include the use of stored data models. This can make the virtual environment more realistic for the players.

[0025] In addition, video and audio data streams from real vehicles can be received to enhance the interactive hybrid environment. In this case, the method may include receiving video or audio data streams from real vehicles and including the video or audio data streams in the interactive hybrid environment.

[0026] To facilitate large-scale gaming, in some embodiments, the method further includes linking one virtual vehicle representation among multiple virtual vehicles to one real vehicle representation among multiple real vehicles at a time point where the location of the virtual vehicle representation is within a predetermined threshold of the location of the real vehicle representation, and using that real vehicle representation as the virtual vehicle representation in the interactive hybrid environment. This advantageously allows a large number of virtual vehicles to be included in the interactive hybrid environment without cluttering the screen with numerous virtual vehicle representations. In fact, this representation solves the technical problem of how to present motorsports events that may involve millions of players within a limited screen size. Furthermore, some embodiments of the invention address the challenge of linking millions of computer game players and other fans worldwide to live motorsports events, providing players and viewers with a challenging, satisfying, and entertaining interactive experience, and, crucially, enabling the fair management and ranking of all players as part of a real virtual motorsports event, which is increasingly attracting professional gamers.

[0027] In some embodiments, where audio and / or video data is provided from a real vehicle, a linking step (also referred to herein as “snapping”) can activate the delivery of the audio or video data stream received from the real vehicle to a computer presenting the interactive hybrid environment to the user. This allows the vision and sound experienced by a particular real vehicle to be delivered into the virtual domain, making the interactive hybrid environment more realistic.

[0028] In some embodiments, the method further includes disconnecting the link between one virtual vehicle representation from one real vehicle representation among a plurality of virtual vehicles at a time point where the location of the virtual vehicle representation is outside a predetermined threshold of the location of the real vehicle representation, and presenting the virtual vehicle representation and the real vehicle representation separately in an interactive hybrid environment. This allows the virtual vehicle representation to be displayed even when it does not match the real vehicle representation, enabling players to see the actual position of their virtual vehicle relative to the real vehicle representation and to switch between real vehicle representations in the interactive hybrid environment.

[0029] In some embodiments, aspects of the interactive hybrid environment may also be forwarded back to the team associated with the real vehicle, as described below. In this case, the method may also include providing details of any virtual vehicles linked to the real vehicle representation to a remote third-party computer.

[0030] In some embodiments, the performance of a virtual vehicle can be matched with the performance of a close localization representation of a real vehicle. This allows for a fairer game, especially when each real vehicle has a different set of performance features. In this embodiment, the method further includes determining the localization representation of the closest virtual vehicle representation among a plurality of real vehicles, and using the performance feature set of the closest real vehicle representation as the performance features of the virtual vehicle.

[0031] Some embodiments also include the steps of using infrared sensors to capture the position data of real vehicles on the track, converting the position data over time into a real kinematic data stream and sending it to a central server in real time.

[0032] Preferably, the capture step involves capturing position data using a group of sensors monitoring different sections of the track, wherein each sensor in each group detects infrared radiation reflected or transmitted from one or more vehicles running on the track within the sensor's field of view (FOV). This arrangement is particularly advantageous for providing accurate real-time information for interactive hybrid environments, and will be discussed later.

[0033] In some embodiments, the method further includes processing infrared radiation detected by infrared sensors to determine kinematic data for one or more real vehicles running on a track. Preferably, this processing is performed at each sensor and is able to transmit a smaller amount of data for use in an interactive hybrid environment.

[0034] According to another aspect of the invention, a computer system is provided for controlling an interactive hybrid environment representing motorized sports events on a track, the interactive hybrid environment including representations of real vehicles and virtual vehicles on the track, the system comprising: a receiver, a virtual race command processor, and a virtual race simulation engine, wherein the receiver is configured to receive a real data stream, the real data including real kinematic data of real vehicles on the track and real control data regarding driver control of the real vehicles, the real kinematic data being captured by infrared sensors on the track, and the real control data being captured by vehicle sensors and obtained via telemetry systems from the real vehicles; the virtual race command processor is configured to receive a computer-generated control data stream, the computer-generated control data stream being transmitted via a user and... The virtual race simulation engine comprises: a race simulation output engine, a reference black-box model generator, and a game black-box implementation engine, wherein the race simulation output engine is used to determine the position and kinematic behavior of a real vehicle representation in the interactive hybrid environment using real kinematic data; the reference black-box model generator is configured to use real control data and real kinematic data to create a black-box determination of the position of a real vehicle on a track based on real control data; and the game black-box implementation engine is configured to determine the position and kinematic behavior of a virtual vehicle representation within the interactive hybrid environment by using black-box determination and computer-generated control data.

[0035] In some embodiments, the computer system may further include an artificial intelligence engine configured to alter the association between the synthetic locations of the virtual vehicles. In some embodiments, the artificial intelligence engine may be configured to broaden the thresholds required to generate a given location of the virtual vehicles from the received computer-generated control data.

[0036] As will be described in more detail below, some embodiments of the present invention relate to improved systems and methods, or systems and methods associated therewith, for any entertainment purpose (including but not limited to computer games, esports leagues, streaming, and viewing) and for generally enhancing fan engagement with motorsports, enabling millions of computer game motorsports players worldwide to simultaneously interact with real, live motorsports events. One embodiment enables a computer game player to begin a race by digitally pairing with one of the real cars and to switch from one car to another, or to run as an add-on car, based on certain parameters. It should be understood that in this specification, the terms “car” and “vehicle” are used interchangeably but have the broader meaning of any vehicle. When paired with a car, dynamic, black-box simulation methods can be used to ensure that the performance of the virtual car in its environment closely matches reality, thereby providing a realistic, challenging, enjoyable, and fair competition between computer game players and real drivers. This system and method for a single player then forms the basis for extended systems and methods for enabling millions of players to simultaneously and competitively interact with real events. The system and method can also be used to enable participants who are merely watching motorsports races rather than playing computer games to have a more engaging interaction with motorsports events. This system and method can also be used to enhance the participation of spectators and computer gamers in a range of other sports.

[0037] Generally, current embodiments of the present invention relate to improvements over known prior art for capturing and live-broadcasting high-precision, real-time vehicle tracking data in motor vehicle racing (e.g., Formula One), thereby enabling truly fair, competitive, and enjoyable racing between computer gamers and professional drivers, as well as a wide range of other viewing and entertainment enhancements. These improvements include capturing and broadcasting tracking data from a large number of high-performance vehicles competing in live events from any racing circuit, making the broadcast data sufficiently accurate, real-time, and formatted to allow computer games and other entertainment media to incorporate and utilize the data, whether live or previously recorded, thereby enhancing the computer game experience and / or providing additional benefits to motorsport organizers and fans, such as user-selectable viewing points, customized streaming, user-targeted advertising, etc. In some embodiments, when competing in live-linked races, players are allowed to change the real car selected at the start of the race to a different real car during the race, or operate as an add-on car given certain parameters, thus allowing players to find their place during the race and match their abilities with other racing drivers. When players choose to switch to a new real car, their car's capabilities are matched to the new real car, thus establishing a fair and competitive match against the new real driver. The system and method described in this paper are then extended to allow an unlimited number of players to participate in a participatory, fair, and competitive manner. In some cases, players may also see and interact with other virtual cars, but in all cases, the system and method described in this paper enables a fair and competitive match between an unlimited number of players in a virtual environment and real drivers in a real environment; this combination is referred to as an interactive hybrid environment. The system and method described in this paper can also be applied to other sporting events, and many representative examples are described.

[0038] Therefore, embodiments of the present invention conveniently provide systems and methods for operating interactive real-virtual racing events, whereby, when the system and methods are in use, any number of participants using a virtual environment (interactive hybrid environment) can interact with real live events and real drivers in one of the following ways: a) by competing in a live, compatible racing event as individual virtual drivers with all levels of experience and skill between real drivers and a possibly limited number of other virtual drivers (the total number of virtual drivers does not exceed the number of real drivers) in a real, satisfying, challenging and fair manner; or b) by participating in a large-scale, multi-player organized e-sports computer game event or any other computer game event or activity highly integrated with a live, compatible racing event (typically organized by friends or contacts) in a real, satisfying, challenging and fair manner; or c) by participating as observers, using the systems and methods of the present embodiments to enhance their live viewing or live streaming experience.

[0039] According to another aspect of the invention, a sensing system is provided for providing location data of one or more mobile entities operating in a defined area to a central server. The sensing system includes multiple sensor groups, each configured to monitor a portion of the defined area, and each sensor group includes: multiple position sensing devices positioned around the defined area, each position sensing device configured to monitor different portions of the defined area from a relatively high position, wherein each position sensing device includes: an infrared sensor having a field of view (FOV), a transmitter, and a communication device. The infrared sensor with the FOV is used to detect infrared radiation reflected or transmitted from one or more mobile entities operating in the defined area within the FOV and generate a sensor output. The transmitter is configured to send the sensor output of the infrared sensor or information derived from the infrared sensor to another position sensing device in its sensor group, the other position sensing device serving as a communication node of the sensor group. The communication device is communicatively coupled to the position sensing device serving as a communication node within the sensor group, and the communication device is configured to send the sensor output of each infrared sensor in the sensor group or information derived therefrom to a central calibration server.

[0040] In some embodiments, each sensor group includes 10 or fewer position sensing devices. Minimizing the number of position sensing devices advantageously ensures an optimal balance between data transmission latency and system complexity due to the need for additional communication paths.

[0041] In some embodiment groups, at least one of the position sensing devices in the sensor group includes a processor configured to determine current kinematic data of one or more moving entities operating within a defined area of ​​field of view (FOV) in at least two dimensions, based on sensor output or information derived from the at least one position sensing device. This feature can significantly reduce the amount of data to be transmitted around the system, thereby increasing the speed of information transmission because the sensor output is processed before being transmitted to the communication node.

[0042] In some embodiments, a first sensor group among a plurality of sensor groups is configured to forward sensor outputs or information derived therefrom as determined by the first sensor group to a second sensor group among the plurality of sensor groups.

[0043] In various embodiments, one or more of the plurality of position sensing devices may include a long-wave infrared (LWIR) microthermometer or a mid-wave infrared (MWIR) photon detection camera configured to detect thermal IR emitted by one or more real moving entities within a defined area. Additionally, one or more of the plurality of sensing devices may include one of a short-wave infrared (SWIR) or near-infrared (NIR) photon detection camera for detecting broadband or narrow-band light emitted or reflected from moving entities.

[0044] Preferably, in order to accurately detect very fast-moving vehicles, one or more of the multiple location sensing devices may have a frame rate of at least 60Hz, more preferably at least 100Hz. This is advantageous compared to the lowest game refresh rates, which are typically 25Hz to 60Hz.

[0045] Preferably, in some embodiments, one or more of the multiple position sensing devices are configured to detect unique identifiers of moving entities based on infrared features. This is particularly useful for tracking multiple moving entities within the same field of view (FOV) where position sensing devices are present. The unique identifier can be a modulated IR signal from a transmitter on the moving entity, with each different entity having a different modulated signal.

[0046] Other optional features of one or more of the multiple position sensing devices include LED floodlights pointing towards a portion of the track, with one or more of the multiple position sensing devices configured to detect reflected light originating from the LED floodlights. This realistic vehicle illumination provides greater resilience in low-light and inclement weather conditions. In some embodiments, one or more of the multiple position sensing devices are configured to detect the infrared signature of the vehicle, consisting of modulated infrared light. This modulation provides further resilience to variable environmental factors and also advantageously makes distance measurements more accurate.

[0047] To aid in relative position determination, one or more of a plurality of position sensing devices may be configured to detect infrared radiation reflected or emitted at the edge of a defined area, and the system may be configured to use the detected information as a reference frame to determine the lateral position of a moving entity.

[0048] In some embodiments, the position sensing devices can be guided to face an oncoming real vehicle. To capture the infrared signature of the real vehicle, they can be positioned at angles to the horizontal and vertical directions. More specifically, in these embodiments, at least some position sensing devices are arranged with their field of view (FOV) at an acute angle to either the horizontal or vertical plane, facing the oncoming moving entity in use as they travel through a designated area. Different configurations are possible; thus, in some embodiments, at least some of the multiple position sensing devices include an FOV of 20 to 30 degrees and a detection range of up to 50 meters. In other embodiments, at least some of the multiple position sensing devices include an FOV of 70 degrees and a detection range of up to 15 meters.

[0049] In some embodiments, the system further includes a GPS receiver. The GPS receiver provides timestamps for information output from or derived from the sensors, and the system is configured to use the timestamps to establish a common time reference for data from sensor outputs or derived from at least some location sensing devices.

[0050] As described above, in some embodiments, the communication device is configured to operate at a minimum refresh rate of 25 Hz to provide the sensor outputs or information derived therefrom of one or more moving entities operating within a defined area to a central server. Using this refresh rate provides sufficient data resolution to track high-speed vehicles (e.g., operating at speeds up to 220 mph) and provide their kinematic data to the interactive hybrid environment. However, in a more preferred embodiment, the communication device is configured to operate at a minimum refresh rate of 60 Hz to provide the sensor outputs or information derived therefrom of one or more moving entities operating within a defined area to a central server. This refresh rate matches the refresh rates of most computer game displays, thus contributing to providing a realistic representation of the movement of entities in, for example, an interactive hybrid environment.

[0051] In some embodiments, the processor of each position sensing device may be configured to determine the longitudinal position along a set area, the lateral position across the set area, and the rotational direction of the moving entity.

[0052] In some embodiments, multiple position sensing devices of the sensor group are arranged sequentially, and the position sensing device located at the midpoint of the sequence serves as the communication node of the sensor group. This arrangement minimizes the number of communication hops between the position sensing devices and the communication node. Thus, for example, for a group of nine sensors where the fifth sensor is the communication node, the maximum number of hops for the sensor output or information derived from any position sensing device to reach the communication node is four.

[0053] In embodiments of motorized sports events, the moving entities include vehicles, and the set area includes a track.

[0054] According to another aspect of the present invention, a system is provided for generating and controlling an interactive hybrid environment representing motorized sports events on a track, the interactive hybrid environment including representations of real vehicles and virtual vehicles on the track, the system comprising: the aforementioned computer system combined with the aforementioned sensing system.

[0055] According to another aspect of the invention, a computer-implemented method is provided for determining the updated positions of one or more user-operated virtual vehicles on a virtual representation of a track using data from one or more physical vehicles operating on a track. The method includes: at a processor, associating each of one or more user-operated virtual vehicles with one of one or more physical vehicles; at the processor, receiving initial position data of one or more physical vehicles indicating the positions of the one or more physical vehicles on the track at a first time; at the processor, determining initial position data of each of the one or more user-operated virtual vehicles on the virtual representation of the track based on the initial position data of the associated physical vehicles; subsequently, at the processor, receiving: position data of one or more physical vehicles indicating the positions of the one or more physical vehicles on the track at a second time, driver input data of each of the one or more physical vehicles, and user input for controlling the operation of one or more user-operated virtual vehicles; and determining the position of one or more user-operated virtual vehicles at a third time based on the subsequently received position data, driver input data, and user input.

[0056] Some embodiments of the present invention provide tracking capabilities, particularly in terms of data latency and data coordination from multiple vehicles, for tracking vehicles traveling at high speeds around non-uniform tracks in motorsports events held globally in any type of environment—whether urban, town, or suburban. Specifically, these embodiments explicitly demonstrate how to simultaneously track real drivers around complex, cluttered, urban-based tracks with tunnels, adjacent buildings, and other obstacles that hinder tracking by single-line-of-sight, high-altitude IR sensors. Systems embodying the present invention are characterized by a specific architecture comprising computer software, computer, communication, and sensor hardware, configured individually or configurably and readily deployable across a wide range of motorsports stadiums, enabling simultaneous tracking of all competing vehicles around complex tracks in varying configurations and environments tailored to push vehicles to their performance limits. Each configuration architecture of the equipment and software tailored for a single track is capable of providing a real-time, single dynamic data stream representing the precise location of all real competing vehicles in a computer-based model that is highly representative of the topology of at least a real racing track. This data stream is suitable in terms of accuracy, latency, and representativeness fidelity, and can be simultaneously distributed to millions of computer gaming devices via the internet or other communication technologies. Computer gaming devices embody some or all of the methods described below and generally involve any range of computer hardware that participants traditionally use to play motorsport-related computer games or experience motorsport events or recordings via digital data streams provided by commercial or other vendors.

[0057] This embodiment provides enhancements to the techniques described in GB2,585,165A, thereby arranging the tracking device into an architecture capable of handling the extreme challenges of tracking high-performance motor racing vehicles in complex environments and constructing a single, integrated, real-time data stream that includes simultaneous, high-precision, real-time tracking data from all competitors, suitable for broadcast and use in gaming and other entertainment environments. The IR sensing capabilities of the tracking device described herein include those described in GB2,585,165A; however, this application includes the thermal IR tracking techniques described in GB2,585,165A, exceeding the techniques described in GB2,518,602B, and allows for tracking motor racing vehicles based solely on their thermal IR characteristics.

[0058] The features described above in the embodiments can be combined in different ways. If not specifically described in the embodiments of the present invention, these features can be added to the following detailed description of the embodiments of the present invention. Attached Figure Description

[0059] To facilitate a clearer understanding of the invention, reference will now be made to the accompanying drawings by way of example, wherein: Figure 1 This is a schematic diagram illustrating a three-part game environment including an embodiment of the present invention; the three parts include multiple game devices, a game server, and a live event data capture system; Figure 2 To show in more detail Figure 1 A diagram of the game server; Figure 3a It shows from Figure 1 A schematic block diagram illustrating the format of the real data received by the live event data capture system; Figure 3b It shows from Figure 1 A schematic block diagram illustrating the format of virtual match commands received by the game computer / device; Figure 4 It is shown Figure 1 A flowchart illustrating the operation of a game server; Figure 5 It shows an alternative Figure 1 A schematic diagram of a game entertainment server according to another embodiment of the present invention; Figure 6 This is a plan view showing the locations of different types of IR sensors along the Monaco Grand Prix route; Figure 7a and Figure 7b This is a schematic diagram illustrating two possible strategies for creating a highly representative computer-based data model of a racing track, as used in this embodiment; Figure 8 This is a set of three-dimensional diagrams illustrating the sensor arrangement according to an embodiment of the invention and its corresponding field of view around the race track; Figure 9 It is shown Figure 1 A three-dimensional map of the sensor array, the communication arrangement between the sensor arrays, and their respective fields of view around the race track. Figure 10a This is a schematic floor plan of the starting configuration for a Formula One racing competition involving 24 real cars and drivers linked to a video game with only one driver, where the real cars have been assigned to their starting positions or chosen by the driver as their starting positions. Figure 10b It is a schematic plan view of the real vehicle representation and the driver's simulated vehicle on the track, where the driver's simulated vehicle deviates from the real vehicle representation by known amounts in instantaneous position and possible other attributes (such as speed, rate and acceleration). Figure 11aIt is a schematic plan view of the race situation where the simulated car is positioned between the three nearest real car representations; Figure 11b yes Figure 11a The game situation is shown in a schematic diagram of the continuation of the race at a later time, in which the simulator has caught up with and is approaching the advancing real car. Figure 11c yes Figure 11b The race situation is shown at a longer time interval and in a schematic plan view of another part of the track, where the simulator car is close to the leading real car. Figure 12a It is a schematic plan view of the starting grid of a race involving 24 real cars, which is live-linked to and combined with esports or online game competitions, with ten million video game players evenly distributed in the starting positions. Figure 12b It is shown Figure 12a The diagram shows a schematic plan view of the starting grid, which can be used as an alternative starting configuration. More than 100 million video game players are distributed in a logarithmic manner, so that only the highest-ranked virtual car is linked to the leading real car. Figure 13 The video shows a match in progress with 24 real cars and a large number of computer game players linked to those real cars. Figure 14 It is a graph showing how various parameters change over time, sent by a live event data capture server, illustrating the principle of the black-box dynamic simulation method; Figure 15 To show in more detail Figure 2 A schematic diagram of the virtual competition simulation engine; Figure 16 This illustrates a distributed processing system according to another alternative embodiment. Figure 1 A schematic block diagram of a gaming device; Figure 17 It is shown that... Figure 16 Distributed processing systems used in gaming devices Figure 1 A schematic diagram of a game server; Figure 18a It shows from in Figure 16 A schematic block diagram illustrating the format of the real data received by the live event data capture system used in gaming devices; Figure 18b It shows from Figure 16 A schematic diagram illustrating the format of virtual vehicle location data received by the gaming device; Figure 19 To show in more detail Figure 17 A schematic block diagram of a virtual competition management engine; and Figure 20 It is shown Figure 16 A flowchart showing how to operate a gaming device. Detailed Implementation

[0060] The computer gaming methods described below all assume that each computer gaming element of the system (including the player's computer / tablet / phone / ... and / or remote server hardware and all associated software) can provide the conventional functions typical of motorsport-related games, whereby the player can provide input to the computer system to control the progress of the simulated motor vehicle around a detailed data model of a real race track. The player's simulated vehicle can interact with the track model, as well as with other simulated vehicles controlled by the game's physics model and artificial intelligence (AI) functions or other types of simulation, or with other simulated vehicles controlled by other players. These other players can be present and provide input to the same computer system, or they can be located elsewhere, using a remote computer system communicating with the game server of one or more original players (sometimes with the degree of involvement of the game server) via the Internet or other suitable network to provide realistic interaction between all vehicles and realistic interaction between all vehicles and the track. The methods of this embodiment described below can also add to, modify, or replace those conventional functions.

[0061] Considered as a whole, the new environment to which the system embodying the present invention belongs can be considered a multi-component infrastructure that supports the large-scale participation of computer game players and other interactive spectators in live motorsports and other types of sporting events. The novel environment to which this embodiment pertains comprises three elements: a) multiple devices used by players and spectators (e.g., game controllers, personal computers, simulators, arcade game equipment, satellite receiving and processing equipment, smartphones, tablets, etc.), including possible camera and audio capture devices, to obtain streaming media from participants located around the world; b) the Internet and associated computing devices (e.g., servers, real-time data centers, real-time server clouds, and possibly dedicated gaming and entertainment servers) to support the real-time reception, manipulation, combination, storage, and dissemination of data originating from players and spectators, which may include, but is not limited to, audio, video, game data, and control inputs from their driving of virtual cars; and c) real, live motorsports or other sporting events, particularly including systems for GB2518602B and GB2365360A, as well as other systems for collecting and providing additional data about real, live events when they occur, and enhancements and developments described herein of systems for storing data related to live events for future use.

[0062] Specific embodiments will now be described with reference to the accompanying drawings. Figure 1 The diagram schematically illustrates the three basic elements described above, including non-limiting embodiments of the invention. More specifically, Figure 1 Several different gaming devices 2 are shown connected to the game server 4 via a communication network 6. The game server also includes a local data storage 5 for calibrating different types of kinematic and control data. Figure 1 The circuit 8 of the motorized sports event 10 is also shown, which includes real vehicles 12 located at various positions around the circuit 8. Each vehicle is equipped with a vehicle telemetry system that wirelessly provides a data stream to the corresponding team telemetry data capture system 14 via the communication infrastructure around the circuit, so that control data (driver control of the vehicle) and other data related to the operation of the real vehicle 12 can be analyzed. The telemetry data stream may also include live video data from onboard cameras on the real vehicle 12 and audio data from microphones provided on the real vehicle 12. The entire telemetry system ensures that each team's data is kept in a limited manner: a) team-only, b) limited portions provided to organizations and management, and c) limited portions available for broadcast on television and other media streams.

[0063] Track 8 is also equipped with multiple sensor groups 16 (nine such groups are shown schematically, although this is not limiting and different numbers may be provided in other embodiments). Each sensor group 16 consists of multiple track-side infrared sensors 18 configured to detect infrared radiation emitted from the real vehicles 12 as they race around track 8, and to convert the sensor data into real kinematic data about each real vehicle 12 within the sensor field of view (FOV). Each sensor group 16 can transmit this real kinematic data to a position data capture system 20, which calibrates the individual kinematic data streams and provides a kinematic data stream 22 about all real vehicles 12 on track 8 to a live event data capture server 24. A live telemetry data stream 26 from a team telemetry data capture system 14 is also provided to the live event data capture server 24.

[0064] The live event data capture server 24 verifies the information received from the location data capture system 20, which includes multiple tracking sensor groups 16 and a team telemetry data capture system 14. In this embodiment, all this information (sensor live data / data stream 28) is provided to the game server 4, which provides data orchestration and generates a virtual match simulation. In this embodiment, the virtual match simulation is provided to all gaming devices. However, in (see later) Figures 16 to 20In another embodiment (described below), a virtual match simulation is generated on each gaming device / computer 2, where the game server 4 simply orchestrates the entire game process.

[0065] See later Figure 5 In another embodiment described, the live event data capture system also calibrates other information received from the team, including all data generated by the team, and the live event data capture system has a general event data capture system that covers all other data associated with motorsport events that may be related to entertainment and gaming features. In this other embodiment, all information is provided to an entertainment and gaming server, which may provide entertainment feeds to non-gaming fans via licensed resellers, ensuring data security needs for all stakeholders. Telemetry data will be described later.

[0066] Figure 2 The composition is schematically shown. Figure 6 The game server of the system shown has game-related components. The game server has a real data processor 30 for processing real data (sensor data streams) 28 received from a live event data capture server 24, as detailed below. The live event data capture server 24 has ensured the synchronization of all its contributing data sources (kinematic data stream 22 and live telemetry data stream 26). The received real data 28 corresponding to the live event is stored in a data storage 5 as real data from the current record of the live event 32 and is also passed to a virtual race simulation engine 36, which generates a reference black-box model generator (described later) that maps the real driver input (control data 26) of each real vehicle 12 to the kinematic position data 22 of that real vehicle 12. The term "black box" is used in the art in its conventional sense, i.e., a system or engine characterized by its response to signals applied to its input ports. The output of the reference black-box model generator is then used within the virtual race engine 36 to evaluate the driver virtual race commands (driving inputs) 38 (received from the gaming device 2 via the communication network 6) to determine where each virtual vehicle should ultimately be positioned. In this embodiment, a virtual race command processing engine 40 is provided to receive these driver virtual race command data streams 38, convert them into a common format if necessary, and provide them to the virtual race simulation engine 36.

[0067] As an alternative to the real data 28 from the live competition, real data 42 from previously recorded live events stored in the data memory 5 can also be used to generate the virtual black box functionality. The virtual competition simulation engine 36 also uses the data model 44 stored in the data memory to generate the virtual competition simulation 46. Finally, optionally, an AI engine 48 is provided to assist in adjusting the virtual car's response to driver inputs (virtual competition commands 38) by referencing a black box model created by a reference black box model generator (described later), for example, by referencing the margins obtained by the virtual driver, possibly assisting the virtual car's driver (player) to varying degrees to make the competition between virtual drivers with different skill and experience levels and real drivers more equitable. For example, this assistance could be to broaden the thresholds around each player input (virtual competition command 38) to make them equal to the optimal position of the virtual vehicle. Thus, while using the reference black box model generator to determine the optimal set of control data inputs to produce a given vehicle position, the player controls the virtual inputs 38 to produce the given position of the car within the thresholds of the specific control data 26 of those live inputs. The degree to which these thresholds are adjusted determines the margin applied by the AI ​​engine 48 to a given driver and the control of their corresponding virtual car. It should be understood that the AI ​​engine 48 is used because it can be trained on an array of complex combinations of control data inputs to produce a specific vehicle position output.

[0068] Figure 3a The illustration schematically shows real data 28 corresponding to real events of real vehicles received from the live event data capture server 24 in this embodiment. It can be seen that the real data 28 corresponding to a particular vehicle's live event is linked to a vehicle ID 50 and includes real vehicle kinematic data 22 (e.g., longitudinal position 52b, lateral position 52c, and directional data 52a) and real driver input data (control data) 26, such as steering position 54a, braking position 54b, accelerator position 54c, gear selection 54d, and possibly other actuator control inputs (not shown). This is provided for each real vehicle 12 racing around track 8.

[0069] Figure 3bThe diagram schematically illustrates virtual vehicle control data (virtual race commands) 38 received at game server 4 from game device / emulator / computer 2 in this embodiment. This virtual control data 38 includes a game computer ID 56 and driver actuator inputs 58 for controlling the virtual vehicle and possibly other data. The vehicle control driver actuator inputs 58 include steering position data 58a, braking position data 58b, accelerator position data 58c, gear selection data 58d, and possibly other actuator control inputs (not shown), and generally match or differ from measurements taken by a real driver, thereby allowing a reference black box to be used to determine the virtual vehicle's position based on these inputs.

[0070] Figure 4 The overview flowchart illustrates the general operation of the system on game server 4. Method 60 begins at step 62, entering the setup phase, where all virtual participants are assigned to one of the real vehicles 12 participating in the race. As described in detail later, a single real vehicle 12 may have multiple participants assigned to it. Subsequently, at step 64, a virtual race simulation environment model 46 is generated, which includes a race track data model and the initial positions of virtual representations of the real vehicles. As previously mentioned, this can be based on real data 28 corresponding to the live event from live event 10 or pre-recorded data 42 from previously recorded live events. The track data model and possibly other data models 44 are stored in local data storage 5 to generate the virtual race simulation environment 46. Once the competition begins at step 66, at step 68, data streams 28, 42 are received from live events or from pre-recorded events. Data streams 28, 42 include precise kinematic data 22 of all real vehicles 12 measured and synchronized at a series of precise time points spanning the competition. Using data streams 28, 42, the positions of the representations of the real competition vehicles 12 can be changed. These new positions are used at step 70 to generate new positions for the representations of the real vehicles in the virtual competition simulation environment 46, and then these new positions are sent (not shown) to each game device 2 registered for the competition for display to the relevant players. In response, the game server 4 receives game control data 38 at step 72, which instructs players to input user data to control their virtual vehicles. These player user virtual inputs 38 are then compared with inputs from a reference black-box model generator, which is used at step 74 to generate subsequent positions of the virtual vehicles as a result of the player user virtual inputs 38. In some embodiments, these new positions of the virtual vehicles may then be sent (not shown) to each game device 2 (see detailed description of another embodiment below). Continue with steps 68 through 74 until the match is confirmed to be over at step 76.

[0071] The system operation method of this embodiment is characterized by the following advantageous features: a) simultaneously providing millions of players and followers worldwide with accurate and timely live data of real motorsport events from any type of environment; b) providing computer game players of all skill and experience levels who compete in the following scenarios: i) as a player competing against real drivers and a limited number (at most one less than the number of real drivers) of other players, all of whom are represented in the game; or ii) as a player and competitor in an organized esports event or other computer game event involving any number (potentially millions) of other players and possible real drivers, some or all of whom are represented in the game; c) selectively providing real teams and real drivers with data of any number of players and followers from around the world for any purpose during the competition (including but not limited to presenting teams (even drivers) with information about the players participating in the event in real time); and d) selectively providing motorsport followers with any data from real events and / or all players and / or all esports competitions linked to the live events, so that they can interact and enhance their participation and enjoyment of the events.

[0072] In another embodiment, the above reference Figure 2 The described game server 4 can be provided by Figure 5 The Game and Entertainment Server 4a shown here is an alternative. Game and Entertainment Server 4a operates in the same way as the Game Server, but with added functionality. Therefore, for the sake of brevity, only the following will be described here. Figure 2 Game server 4 and Figure 5 The differences between the game and entertainment servers 4a.

[0073] refer to Figure 5 Among them, the components of the game and entertainment server 4a are... Figure 2 The game server components are identical, and the same reference figures are used. Where there are any differences, new reference figures have been introduced. Therefore, it can be seen that in this embodiment, the game and entertainment server 4a further includes a match simulation and entertainment engine 36a, which receives virtual match commands, audience commands, and other data 38a from the game server 4 and other entertainment devices (not shown). This data is collected and routed to the match simulation and entertainment engine 36a via an entertainment / match command and data processing engine 40a. Furthermore, an advertising engine 80 is provided, which also provides data to the match simulation and entertainment engine 36a.

[0074] Leveraging the availability of precise tracking data from location data capture systems, numerous entertainment enhancements can be provided; three will be explained below with examples. The first is a fluid perspective, based on viewer commands from the entertainment device, such as a live video stream from the driver's perspective in a real car, which can be seamlessly switched or panned to a perspective generated by software features within the entertainment engine, such as a computer-generated perspective above the car. Figure 2 The virtual race simulation engines in this example are the same as or very similar to those in the previous example. A second example is the use of computer-generated displays or overlays to enhance live video streams. These computer-generated displays or overlays are convincingly integrated into the live video based on live technology data from sensors or other sources originating from the vehicle, such as temperature-dependent shading of tires or “under the hood” graphics of vehicle mechanics. This can be extended, for example, to the enhancement of live video streams from the perspective of a driver in a real car, convincingly and seamlessly integrating advertising images onto the sidewalks or bridges or on the vehicle itself. Computer-generated advertisements from advertising engine 80 move with the trackside scenery or vehicle dynamics so that they are indistinguishable from advertisements appearing on the live video stream. Because the advertisements are combined and streamed to specific, known users, the advertisements can be customized for users based on the data received in the entertainment command and data processing capabilities, as well as the algorithms contained in advertising engine 80.

[0075] The specific operation of the location data acquisition system 20, which includes the tracking sensor group 16 described in the above non-limiting embodiments, is described below by way of example of the system and method provided by the present invention. To better understand the location data acquisition system 20 of the above embodiments, first refer to a demanding racing track.

[0076] refer to Figure 6 The image illustrates an example of a particularly demanding race track 8, which cannot be monitored from above by a single high-performance IR sensor and requires a novel system of sensor 18 and communication equipment to provide the data needed for real-time interactive gameplay. An illustrative example of a live motorsports event 10 is an international Formula One race involving 24 real cars from 12 teams, each with its own pit lane, garage, and other facilities. The race location is... Figure 6 The track is exceptionally complex. It includes the locations of tall buildings, bridges, and tunnels, and features various types of permanent street and transportation infrastructure as well as temporary infrastructure such as spectator stands. It is subject to a wide range of atmospheric, weather, and lighting conditions, depending on the time of year and the location of the event.

[0077] Figure 7a and Figure 7bTwo possible strategies used in this embodiment are illustrated for creating a highly representative, computer-based data model 44 for the race track 8. This highly representative, computer-based data model 44 is designed to ensure that a real-time link can be established between the measured kinematic data of the real vehicle 12 and the virtual kinematic data of the player's virtual vehicle in a manner that overcomes the problem of verifying whether the fidelity of the track representation is sufficiently accurate and correct. This model 44 is a crucial link between the real environment and real events 10 and the computer game environment and computer game event virtual race simulation 46, because each tracking sensor 18 in the network of tracking sensors will have a portion of the real field of view (FOV) of the track 8 and the relationship between the real FOV in its memory and a portion of the data model 44 of the race track 8. In this way, images of IR reflectors or IR emitters on the vehicle, or images of natural thermal infrared radiation from the vehicle, are converted within the tracking sensor 18 into kinematic data represented according to the race track data model 44. Note that in another embodiment, the raw data from the tracking sensor 18 may also be transmitted first to another component of the system, where the relationship between the raw data and the track data model is calculated before being forwarded to the game server 4 for processing.

[0078] For more specific reference Figure 7a The example base of data model 44 is shown, comprising an orthogonal grid 82 of closely spaced latitude and longitude lines. The spacing of the grid lines is suitable for the tracking accuracy achievable by the tracking sensor 18 and desired by the computer game software. The kinematic data in this model typically includes at least longitude data 52b, lateral data 52c, and rotational direction data 52a for each real vehicle 12. The track model may also include precise elevation data for each cell in the latitude-longitude grid 82 to enable the virtual interactive hybrid environment to simulate vertical gravity, acceleration, etc.

[0079] Figure 7b An alternative model that facilitates computational simplicity is shown; the track is modeled as a series of lateral strips, each with the geographic location (latitude, longitude, and altitude) of its lateral center point, orientation, lateral width, and (in some embodiments) possible lateral slope angles or elevation profiles. Then, the kinematic data for each vehicle includes at least (e.g., Figure 3a The data reflected in the figure are: longitudinal position 52b along track 8, lateral position 52c across the width of track 8, and rotation direction data 52a.

[0080] Establishing a high-precision relationship between the computer game data model of a racetrack and the terrain of a real-world racetrack has historically been surprisingly difficult. For the entire system of the current embodiment, this can be addressed, for example, by calibrating at each sensor 18 during system setup, in a way that creates a very accurate physical-to-data-model mapping by placing static IR reflection / emission markers (not shown) at precise measurement locations along the outer edge of the racetrack 8 within the FOV of each camera. In some embodiments, dynamic calibration is achieved by, for example, driving or moving a dedicated calibration vehicle in stages along each edge of the racetrack 8 to calibrate tracking performance and achieve continuity between the sensors 18.

[0081] The inventors have realized that, as Figure 6 The topology of this complex environment precludes connecting the required 100 or more tracking sensors in a "star" configuration to a central location data acquisition system so that they all communicate in parallel. For wireless communication, it would be difficult to find 100 independent, reliable signal paths, and wired connections would involve approximately 100 km of cables radially distributed across the urban landscape, which is highly disadvantageous. The only reliable path for either technology is to link the tracking sensors 18 around track 8 in a closed-loop geometry—in which case at least one point on or near the track must accumulate data in real time. If it's just a single point, then some sensors would require at least 50 separate data hops from one device to another, and the accumulated data transmission latency would be unacceptable.

[0082] A realistic interactive game between two geographically separated players / computers A and B requires gaming device B to receive and process the impact of player input on gaming device A within 20-40 milliseconds, and vice versa, an impact determined by a typical computer game update rate of 25-50 Hz. It should be understood that the term "computer game" is synonymous with simulated environments, including interactive hybrid environments. In the current embodiment, the entire position data capture system, including tracking sensor 18, can be considered to provide equivalent player input for all 24 real cars on the track and must provide this data to the remote gaming device 2 within a similar potential time delay. Synchronization could be ensured by using data time stamping (described in more detail below as an option), introducing an overall time delay (e.g., 1 second) into the data stream, but this would lag the entire live game experience behind other information streams (e.g., live video), which is undesirable. Therefore, for an infrared camera frame rate of 100 Hz that allows 5 ms for further processing of the raw image signal (maximum 10 ms delay), only about 20-30 ms is allowed to aggregate the data from sensor 18 around track 8 and for subsequent communication. For both wired and wireless technologies, the inventors have determined that, taking this into account, 50 hops along the tracking sensor track cannot be tolerated in this embodiment, and even considering anticipated developments in communication technology, no more than 10 hops along the tracking sensor track is necessary.

[0083] therefore, Figure 1 The interpretive sensor group 16 shown is converted into an IR sensor network architecture in which a small portion of the sensor group 16 is arranged in a high-performance local network, but taking into account a number of other factors, including view geometry, line of sight, obstructions (between the actual vehicles 12 and between vehicles and infrastructure), available mounting points, IR sensor performance (e.g., image resolution (typically 640×512 or 1280×1024) and frame rate (typically 100Hz)), image processing parameters (e.g., the number of “target pixels” (typically required >4)), required tracking accuracy (10cm or better laterally, less stringent longitudinally due to high speed, but much lower relative speed between vehicles), maximum vehicle speed in the FOV of each sensor, execution time for handling worst-case scenarios, etc., for Figure 6 The sensor array was custom-designed for the actual race track, with the overall goal of making... Figure 1 The output of the location data capture system 20 (which includes data from each of the 24 real vehicles 12, for example). Figure 3a The kinematic data is synchronized to a common, accurate time base with an update frequency of at least 25 Hz, and the overall real-time delay does not exceed 1 second if real-time delay is necessary (note that if it is sensitive and confidential to the team...). Figure 3aThe real driver input data is sent to the game device, or used on the central game server without being sent to the game device at all, and is protected by fixed or dynamic encryption.

[0084] To illustrate the architectural constraints of a particular track 8, several possible sensor groups 16 are described in more detail below. It must be noted that these examples rely to some extent on the capabilities of today’s available technologies, such as IR camera image resolution, frame rate, LED intensity, etc., and that these technologies are rapidly evolving and improving.

[0085] Figure 8 A representative configuration of a specific sensor 18 arrangement is shown, suitable for monitoring a specific type of race track segment with a specific class of real race cars 12. These specific (F1) real cars 12 have an aerodynamic, forward-leaning, and low profile, meaning that if the tracking sensors 18, positioned relatively high relative to the track, are oriented towards oncoming vehicles, they are most likely to be reliably detected at high speeds, i.e., when facing an oncoming vehicle, the field of view (FOV) is at an acute angle to the horizontal plane. The FOV's field of view is considered to be the central axis of the FOV. This arrangement also means that the sensors 18 can be conveniently positioned closer to the ground and have an increased FOV compared to the purely downward-facing sensors in GB2585165A. In some cases, a higher pole or other structural mounting point 84 may be suitable, and in some cases, a higher pole or other structural mounting point 84 may be more suitable for situations requiring fewer sensors 18 to monitor the track, and the angle of the field of view relative to the horizontal plane will be increased. Alternative configurations can be similarly defined for other types of locations or other types of racing events, such as speedboat racing or NASCAR racing.

[0086] For details, please refer to the following: Figure 6 The example track shown now describes how sensors 18 can be arranged to provide location data capture system 20.

[0087] First, for those equipped with standard light poles Figure 6 The total 2km section of the track, Figure 8One of many possible arrangements of the sensor group 16 of the IR sensor 18 installed on lamppost 84 in this section of track 8 is shown. Each sensor (sensing device) has an interpretable resolution of 640 × 512 pixels, a field of view of approximately 20° × 30° (see views A and B), and a detection range of up to 50m. Using this geometry, reliable detection with a resolution as low as 10 cm can be achieved by using any of the following arrangements of sensor devices 18: a) a long-wave infrared (LWIR) microthermometer or mid-wave infrared (MWIR) photon detection (or other suitable technology) camera that detects thermal IR emitted by the vehicle (its tires, engine, exhaust system, etc.) or alternatively detects thermal IR “cold spots” created by coating or attaching IR non-emitting marks or materials on the vehicle; b) a short-wave infrared (SWIR) or near-infrared (NIR) photon detection (or other suitable technology) camera that detects broadband (or narrow-band) light emitted by at least two small LEDs or LED clusters mounted on the vehicle or detects natural (sunlight-derived) IR radiation from reflectors on the vehicle or detects cold spots from IR non-emitters on the vehicle. The light emitted by the LED can be constant or time-modulated; c) Detecting broadband (or narrowband) SWIR or NIR light provided by an LED floodlight located near the IR camera and reflected by at least two small reflectors mounted on the vehicle, or detecting cold spots in reflected images from IR non-emitters on the vehicle using a SWIR or NIR camera. This will enable nighttime racing, and the LED floodlight can be pulse-modulated and synchronized with image capture in the camera to improve detection performance in extreme weather conditions.

[0088] As described above, the positions of all individual targets (LED emitters, reflectors, non-emitters, or vehicles / tires) in the raw image on the camera's focal plane array are converted into precise positions within the track model through signal processing and geometric calculations. These signal processing and geometric calculations are tailored to the specific location, configuration, and orientation of the tracking sensors. This precise position may involve the relative position of each of the two emitters (or reflectors or non-emitters) on the real vehicle 12 relative to its external physical envelope. Such techniques are entirely within the capabilities of those skilled in the art and therefore do not need to be further described herein.

[0089] The continuity of identification and tracking of a single real vehicle 12 can be achieved (as in GB2585165A) by each tracking sensor transmitting the vehicle's position to the next sensor as the vehicle leaves its FOV. Note that regarding the arrangement of the SWIR or NIR cameras as described in b) above, the LED light can be constant or modulated to emit different patterns (or IR features) for each real vehicle 12. This will enable each tracking sensor to track and identify simultaneously, without requiring a tracking sensor to "hand over" vehicle identification to the next tracking sensor.

[0090] Secondly, there may be some locations around track 8 (e.g.) Figure 6 The black squares 81 in the diagram represent locations suitable for mounting a single IR camera sensor 18 at a relatively high altitude. This single IR camera sensor 18 is mounted on the top of a building, on a mobile cantilever lift (up to 45m), on a access platform (up to 100m), or carried by an aircraft such as a drone or tethered drone (which can provide power for extended flight time and transmit data to a ground station). At a height of 100m, the FOV of the single IR sensor 18 equipped with a wide-angle lens extends to approximately 140m × 140m. Figure 6 The black square 81 on the map is representative of the scale. On some tracks 8, it may be preferable to deploy several drones equipped with sensors 18 around the track to form a complete closed-loop network. Each drone will use its own onboard GPS to stay at the site and provide precise image references by placing fixed IR beacons on the ground, thus enabling absolute tracking and detection accuracy. This is an efficient deployment because fewer sensors are needed to cover the entire track, and the ease of deployment is excellent for tracks without permanently installed sensors. A safe case for flying drones above high-performance racing is feasible because the drones do not need to be directly above the track or the spectator area. Tethered drones may have a particular advantage because a malfunctioning drone can quickly rotate and descend to a defined crash zone around the base station.

[0091] third, Figure 8 The short lampposts 84a can be equipped with wide FOV (70°) IR sensors, each covering a short section approximately 15m in length. The same applies to the sensor 18, which will be attached to the top of tunnel 83. In another alternative embodiment, it may be more cost-effective to forgo the short lampposts 84a and deploy mobile masts up to 30m high, each covering a section up to 50m long. In summary, this complex 3.3km track example requires 50 standard sensors, 50 wide-angle sensors, and 2 sensors mounted on a remote high platform or drone.

[0092] It is important to note that on many race tracks, light poles may be absent for safety reasons, or they may be spaced further apart and taller, or they may need to be protected so as not to pose a safety risk to the vehicles. In different embodiments, temporarily mounting tracking sensors on structures or using appropriately positioned mobile masts may be necessary or preferred.

[0093] Finally, to achieve the aforementioned real-time data streaming performance standards, the tracking sensors in this example, based on ground and / or drone-mounted installations, are organized into multiple sensor groups 16, such as... Figure 1 As generally shown, but more specifically as Figure 9 As shown. Figure 9 An illustrative example is shown of how sensor 18 is arranged in sensor group 16 to achieve the overall real-time performance of location data acquisition system 20 and how sensor 18 and sensor group 16 can transmit data to provide a single, complete and continuously synchronized real data stream 28 with appropriate latency suitable for transmission to a real-time interactive hybrid i.e., real-virtual game simulation environment 46. Figure 9 A sensor group 16 with seven sensors 18 is shown mounted on a lamppost, although other configurations are possible in different embodiments. Multiple sensor groups 16 are created around track 8 to provide the required full coverage. In this way, the entire suite of tracking sensors 18 set up around the race track 8 can be presented as a single group, thereby creating a single data stream for all real cars in parallel, while still achieving the low latency and concurrency required for interactive gameplay.

[0094] For more specific reference Figure 9Each sensor group 16 is connected to a high-performance, low-latency communication link 86. This communication link 86 uses a suitable communication technology (for ground-based sensors, this could be via a wired connection, such as Gigabit Ethernet—data rate >900 Mbit / s, latency 100-500 μs; or for ground-based or drone-mounted sensors, it could be wireless, such as dual-band Wi-Fi—data rate >90 Mb / s, latency 1-2 ms). Each sensor group 16 is equipped with a forwarding communication node 88, which is typically used for fiber optic (“wired”) or wireless networks and is frequently used for streaming telemetry data in motorsport. The forwarding communication node 88 is configured to communicate with a live event data capture server, either wired or wirelessly, where the position data of all 24 vehicles is accumulated at a refresh rate of at least 25 Hz for forwarding transmission. Furthermore, based on the actual communication time delay around the network from one track to another, the entire sensor group, or each sensor group 16, or in some embodiments, each sensor 18, can be equipped with a GPS receiver so that a universal, accurate time reference can be established across the sensor network. The precise position measurement of the real car 12 can then be time-stamped, enabling the live event data capture server 24 to aggregate the car position data into the desired simultaneous set, even though this introduces a small time delay (<1 second) in the forwarding transmission to the game server 4.

[0095] The following further describes in detail, using the systems and methods related to the interactive hybrid real-virtual game simulation environment 46 provided by the present invention as examples, the referenced... Figures 10a to 14 The specific operation of the above-provided non-limiting embodiments.

[0096] In this example, Figure 10a The diagram illustrates the starting configuration for a Formula One racing competition involving 24 real cars and drivers. Representations of these real cars 12 (89a, 89b, 89c, 89d, ... 89x) are linked to a computer game with only one driver, where the real car representation 89f has been assigned to a starting position or selected by the driver as their starting position, such that the representation of their simulated car 90 accurately covers the selected real car representation 89f.

[0097] according to Figure 10aThe first scenario described above involves a single computer game player controlling a simulator car 90. At the start of the race, the simulator car 90's position in the computer game is the same as the real car 12f's position in the real event. After the race begins, the simulator car 90's progress is determined by the player's input to their computer device (game device / computer 2) and the game server 4's simulation of the car and its environment. The real car representation 89's progress is determined by the driver's input to the real car 12 and the extremely complex physical behavior of the real car 12 in its extremely complex environment. Due to i) insufficient fidelity in the simulation of the car and its environment or ii) differences in input between the player and the driver, the simulator car 90's progress will deviate from the real car 12f's progress. This embodiment minimizes the former cause, thereby making the competition between the real driver and the virtual player as fair, realistic, and enjoyable as possible.

[0098] One of the key features of the method in the described embodiments is that, while the simulator 90 remains sufficiently close to the real car representation 89, within tolerances that can be defined in various ways (the following description will refer to the simulator 90 being “captured” by the real car representation 89), the computer game will exhibit specific behaviors and increase interaction with the real car 12, which may include, but is not limited to: a) as part of the captured player's game, the real car representation 89 will not be displayed and will not interact with the player's simulator 90; b) the captured player's simulator 90 will be “digitally paired” with the real car representation 89, with the simulator 90 replicating the real car representation 89 sufficiently closely. The performance of the real car represents 89 to ensure fair competition between drivers and players; c) Inputs (control data 26) from the real driver (steering, accelerator, gear selection, etc.) will be able to be streamed in real time and applied to the interactive hybrid environment generated in real time by the game server; d) Audio, video and other data feeds from the real car 12, its driver and the wider racing team can be streamed in real time to the computer game (interactive hybrid environment) generated by the game server; e) Audio, video and other data feeds from the captured players can be fed back to the real car 12 and / or the team and / or other computer players.

[0099] Figure 10bA racing scenario is illustrated where the driver's simulator car 90 deviates from the real car representation 89f by known amounts in its instantaneous position and possibly other attributes (such as speed, rate, and acceleration). A simple, non-restrictive method for defining when the driver's simulator car 90 is captured by the real car representation 89f is explained here. Within the computer game software running on game server 4, the position of a reference point 94f on the real car representation 89f on the race track is used to define a neighboring region 96f, which in this case is a rectangle. If the position of the equivalent reference point 91 on the simulator car 90 falls within the neighboring region 96f, then the driver's car is "captured" by the real car representation 89f.

[0100] Many other possible capture methods could depend on other parameters, such as how far behind or ahead a driver's simulated car representation 90 is in time relative to the real car representation 89 (as used in F1's "Drag Reduction System (DRS)," which allows a "speed boost" if the chasing car is within a specified range of the car in front). Other capture methods might include velocity, rate vector, angular momentum, etc., but the principles are clear. Furthermore, the computer game AI engine 48 could use algorithms or other strategies to implement a degree of "hysteresis," which provides more flexible tolerances, making captures less affected by jitter, and, in the case of a handicap system, higher-ranked drivers might receive less "AI assistance" than lower-ranked drivers. Additionally, when a capture is "cancelled" from the real car representation 89, the game software running on the game server could consider the proximity of other drivers' virtual cars.

[0101] When the player's simulator car 90 does not capture any representations of the 24 real cars in the race, all car representations are displayed in the computer game in the conventional way. When the player's simulator car 90 captures a representation of a real car 89, only the representations of the other 23 real cars 89 are displayed to the player, and the player may feel as if they are driving real cars and having very engaging interactions with real drivers and their racing team, especially when the team provides live audio feeds to the player.

[0102] This capture method allows players to move throughout the race by avoiding real car representation 89 or by moving from real car representation 89 to the real car representation 89 that is capturing them (or not according to the player's operation selection). Figure 11aThis shows the race situation at a certain point in time, shortly after the start of the race. Here, simulator car 90 is positioned to some extent between the three closest real car representations 89c, 89b, and 89e. The driver's simulator car 90 has already outperformed its original paired car (real car representation 89f) and is now in the lead. The real car representations closest to the driver's simulator car are now real car representations 89b, 89c, and 89e. The positional relationships 98b, 98c, and 98e will rapidly evolve based on the skill and actions of the driver and the three real drivers. Figure 11a At that time, the computer game (virtual racing simulation) displays the real car representation 89c in front of the player's simulated car 90, and the real car representation 89e in its rearview mirror.

[0103] Figure 11b It shows Figure 11a The race continued later in the game, with simulator car 90 catching up to and approaching real car representation 89c. Simulator car 90 was shown as having entered the vicinity 96c of real car representation 89c and was now captured. The game server is now preventing real car representation 89c from appearing on the player's screen and has enabled the aforementioned feature.

[0104] Figure 11c It shows Figure 11b The race situation continues after a longer time interval and at another part of track 8, now designating simulator car 90, which is approaching the leading real car representation 89a. Here it can be seen that the computer game player's simulator car 90 is far ahead of real car representation 89c and is more than halfway between real car representation 89c and the leading real car representation 89a. However, the player's virtual car 90 has not yet reached the capture area 96a of real car representation 89a and is chasing it. Two additional areas are introduced around each real car representation 89 in the diagram. The first area 98a is in front of real car representation 89a and is referred to as the "chasing area," while the second area 100a is behind real car representation 89a and is referred to as the "chasing area." Figure 11cAs shown, the three regions 96a, 98a, and 100a together form a continuous set, so that when the simulated car 90 is in any of the three regions associated with the real car representation 89, it is digitally paired in terms of performance, with the performance of the simulated car 90 closely matching the performance of the real car representation. This feature balances the timing of the virtual car 90 chasing the real car representation 89 and vice versa, and can be naturally integrated with real-world activity features such as DRS. Therefore, the simulated car 90 is now located in the chasing region 100a of the real car representation 89a. Once the simulated car 90 moves from the chased region 98c of the real car representation 89c to the chasing region 100a of the virtual representation 89a of the real car, its performance characteristics switch between the performance characteristics of the real car representation 89c and the performance characteristics of the real car representation 89a. This means that the performance of the simulated car 90 is always matched with the real car representation 89 with which it interacts most closely and competes, thus ensuring fair competition between the computer game player and the real driver based solely on driving skill. This partitioning method is an interpretation of the partitioning principle, and there are several possible variations of this method. Figure 11c The diagram also shows a real vehicle representation 89b with a capture area 96b and a pursuit area 100b, and a real vehicle representation 89c with a capture area 96c and a pursuit area 100c.

[0105] By allowing individual virtual drivers of all experience and skill levels to compete in live, compatible racing events in a realistic, satisfying, challenging, and fair manner between real drivers and a potentially limited number of other virtual drivers (the total number of virtual drivers not exceeding the number of real drivers), more than one computer game player can be involved in a race simulation linked to a live or recorded race. In existing motorsport computer games, it is common for many players to compete online simultaneously, regardless of where they are located in the world (based on more than one real, normal participant in a real race). Thus, in the relative prior art implementation of the Formula One racing example described above, typically a maximum of 24 players participate in the race, each occupying a position on the starting line, with any unfilled positions occupied by simulated cars controlled by the game software's AI and algorithms. In the current embodiment, this type of limited-player game mode remains possible. However, in other embodiments described below, it is possible to implement other game modes that are not limited by the number of players. According to the above embodiments of the invention, it is possible to implement the above-described limited-player game mode, wherein unfilled player positions are assigned to real car representations 89 in the real race, whose real-time race data stream will control their behavior in the game. In this scenario, there will always be 24 drivers and / or players in the computer game, with many computer players, such as "n" computer players, competing against "24-n" real cars. All aspects of the aforementioned "chase and capture" method will apply, and the performance of each simulated car 90 will be paired with the performance of the real car representation 89 it replaces at the starting line, except when it represents the "chase and capture" zone of the real car representation 89 during the race.

[0106] According to an alternative embodiment of the finite-player game mode, there will be 24 players and 24 real drivers. Whenever a player's virtual car 90 is not captured by a real car representation 89, their virtual car 90 will appear in the computer game environment, visible to all other players' virtual cars 90, and interact with all other players' virtual cars 90. Therefore, there will be 24 real car representations and an arbitrary number of uncaptured simulated cars 90 visible at any given time in the computer game simulation. These two variations of the method in this game mode are illustrative; many other possible variations exist, but the principle of the method is clear.

[0107] In many types of motorsports (Formula One is merely an example of the more general applicability of this invention), real cars may spend time during the race performing maintenance or repairs, such as damage repairs, refueling, or installing new tires. Therefore, in this embodiment, the computer game player can choose to take the same amount of time off the race for a simulated version of the same maintenance or repairs performed on a real car, in which case the player's simulator 90 inherits the modified performance characteristics of the real car representation 89. Alternatively, when the computer game player's simulator 90 is not captured by the real car representation 89, the player can choose to take time off the race for simulated maintenance or repairs, which, upon returning to the real race, will qualify the player's simulator 90 to link to any real car representation 89 with similar or compatible maintenance or repairs. This is one of many possible implementations of a method to ensure fair competition when real cars can undergo maintenance or repairs during the race.

[0108] As mentioned above, it is possible to implement other game modes without being limited by the number of players. This can be achieved through participation in large-scale, multiplayer organized esports computer game events or any other computer game event or activity (often self-organized by friends or contacts) that are highly integrated with live, compatible racing events in a realistic, satisfying, challenging, and fair manner, where there are a very large number of computer game players participating in the game, and the game is linked to a live or recorded real event. Just one example of this is in large-scale esports competitions, where each player needs to interact with real competition in a realistic, engaging, and challenging way, and where a fair method is needed to rank players based on their performance and final position in a large number of esports events.

[0109] Figure 12a and Figure 12b Two examples of possible starting configurations for a competition are shown, involving 24 real cars that are live-linked to and interact with an esports or online game involving tens of millions of players, whose starting positions are distributed among the starting positions of the real cars. More specifically, Figure 12a The initial configuration of the competition is shown, in which 24 real cars are linked to ten million video game players evenly distributed between the starting positions. Figure 12b It shows Figure 12aAlternative starting configurations to the shown starting configurations include a logarithmic distribution of over 100 million video game players across 24 real car representations, such that only the highest-ranked virtual car 90 is linked to the leading real car representation 89. These game modes are referred to as the infinite player game mode. These examples are just two of many similar options; for the purposes of the method in this embodiment, it is only necessary to assign each participating computer game player to one of these cars. Thus, each player is captured with a real car representation when the game begins. Each player captured with the same real car representation 89 is paired with that real car number and has the same performance model for real car representation 89 in their environment.

[0110] Figure 13 The text shows the competition in progress, which involves 24 real cars and a large number of computer game players. Each player's virtual car 90 is linked to a real car representation 89. Figure 13 The actual match situation is similar to Figure 11c In the scenario described above, real car representation 89a is in the lead, followed by real car representation 89c in second place and real car representation 89b in third place. Each computer game player experiences the game in almost the same way as the single player interacting with representations of 24 real cars described above. Figure 13 The scenario shown is that the simulated cars 90 of three competitors 102a (3) are in area 98a in front of the leader (real car representation 89a). The simulated cars 90 are shown by a single dashed car on the track and are displayed in the extended view as three separate simulated cars 90c, 90f, and 90z located in specific, precise, and closely spaced positions. In this leading area, each competitor can be shown information and / or representations of some or all other competitors in that area to facilitate interactive competition for the leading position. Four cars 104a (4) are in the capture area 96a of the real car representation 89a and are captured in the real car representation 89a. They are in Figure 13The car shown as a single dashed line on track 8 is shown in the extended view as four separate dashed line car representations 90b, 90m, 90w, and 90s. Each of the four separate dashed line car representations 90b, 90m, 90w, and 90s is located at a precise position in the game relative to the real car representation 89a. Nine cars 106a (9) are in the pursuit zone 100a of the real car representation 89a. All simulated cars in zones 96a, 98a, and 100a are digitally paired with the real car representation 89a. The real car representation 89c has a capture zone 96c, in which 25 cars 104c (25) are currently captured, the pursued zone 98c includes 18 cars 102c (18), and the pursuit zone 100c includes 57 cars 106c (57), all of which are digitally paired with the real car representation 89c. In the chased and pursued zones, each player's virtual car 90 can be displayed with information and / or representations about some or all other players with their virtual cars 90 in the same zone to facilitate interactive competition between virtual players in the chased and pursued zones 98, 100. The real car representation 89b has a capture zone 96b and a chased zone 98b, in which 97 cars 104b (97) are currently captured, and the chased zone 98b contains 73 players 106b (73), all of whom are digitally paired with the real car representation 89b. This representation method advantageously allows for the provision of a large amount of player information within a small screen size, enabling gaming devices with small screen sizes to maximize the amount of information displayed. If a real car crashes in a real race, the captured virtual cars 90 will mimic this, provided that the player's input commands are not as mitigating and avoiding the crash as determined by the reference black-box model; otherwise, they will continue as an uncaptured simulated car 90 from where they successfully deviated.

[0111] Therefore, in this unlimited player mode, the above method conveniently provides any number of computer game players with the opportunity to interact with the live race, and each player experiences Event 10, competing against real cars across the entire arena in an engaging and exciting manner. This is achieved by each player's gaming device 2 receiving a live data stream of the real cars' progress around the track and driver input (in the virtual race simulation), as well as utilizing other data streams (such as team audio and driver video) where appropriate. Furthermore, in the case of players participating in large-scale live-linked races, the progress of each player around the track is typically transmitted and aggregated on the game server 4. This allows... Figure 13The viewpoint shown can be assembled and analyzed in real time over the entire track 8, allowing continuous monitoring of the relative positions of a large number of virtual cars 90 of various players during the race, with any level of precision (e.g., computer game software simulations can represent the instantaneous position of any simulated car at any time with any desired precision). This enables: a) the final positions of all players at the end of the race to be determined clearly and fairly, and b) to present images of the cars of other relevant players in the vicinity to a single player, where this can enhance competition among computer game players.

[0112] Furthermore, each player's computer game can showcase behavior and interaction with real-world events 10, but this behavior and interaction can be enhanced through the large-scale participation of many players. For example, in addition to streaming live audio feeds of real drivers and teams to all captured players, the number of captured, chased, and pursued players can also be presented to the real teams, or a large number of observers, or even the real drivers, along with the names of each player, so that real drivers can comment on any or all captured drivers, and selected captured drivers can comment in response (which, of course, is acceptable from a security perspective). There may be many other similar immersive entertainment features that can be implemented by this embodiment.

[0113] In a live-linked event 10 with large-scale participation of computer game players, the skill levels within the group of players may vary depending on their organization. As mentioned above, the current embodiment can be easily extended to allow "all-inclusive" or "open" events (based on a fair assessment of prior experience, as is the case with all handicap systems). For example, each player may have achieved a handicap between 0 and 100, where 0 represents a pro player's status and 100 represents a novice player's status. A handicap of 0 means that the AI ​​engine 48 of the game server 4 does not provide support in the game when accepting the players' input and translating it into their virtual car's progress on track 8, while a handicap of 5 might mean that if the players' virtual data input 38 is within 5% of the error of the real driver's input 26, they are considered a match, and so on.

[0114] All of the above methods advantageously present the characteristic that computer game simulations of a real car's performance in its environment can represent not only fair and enjoyable competition between gamers, but also fair and enjoyable competition between gamers and real drivers. In other words, when a gamer makes the same simulated input 38 to the simulated car as a real driver makes the same simulated control input 26 to the real car, the progress of the simulated car 90 will match the progress of the real car representation 89 with an acceptable accuracy. Achieving a high level of fidelity has been a goal of racing computer game development and professional simulators for many years, and significant capabilities have been achieved. However, for live events with large-scale participation, many factors are at play in any event, and these factors need to be measured and transmitted to the computer game and actually considered in the computer game algorithms and AI. As mentioned above, the prior art suggests sending many such measurements to the computer game in real time to enhance traditional physics-based simulations. An alternative and preferred method is implemented in the final embodiment described below, and this alternative and preferred method is essentially summarized as a method of "real-time, dynamic black-box simulation for real-virtual interaction".

[0115] This embodiment can implement a dynamic black-box simulation model within a computer game linked to live events. In these embodiments, in addition to via... Figure 1 In addition to providing the computer game with precise kinematic position data 22 of the real car in real time, the position data capture system 20 shown provides the computer game with precise kinematic position data 22 of the real car, and the real control input 26 of the real driver to the real car 12 can also be measured in real time and sent to the game server 4. Figure 14 It is a graph showing the changes over time of various real data 28 sent by the live event data capture server 24, which is used to explain the principle of the black-box dynamic simulation method.

[0116] refer to Figure 14 This embodiment uses a dynamic black-box simulation model in real time to interpret and simplify a set of driver inputs (control inputs) for a real car—namely, steering wheel position 54a, accelerator position 54c, and brake pedal position 54b—in reality, there may be many more driver inputs, such as gear shifting. Figure 14 (Not shown in the image). These are the dynamic data inputs to the black-box model. The dynamic outputs from the black-box model are a virtual representation 89 of the real car based on the progress of its position changing over time; in this example model, they are shown as longitudinal position data (longitudinal distance) 52b and lateral position 52c while driving on track 8. Figure 5This demonstrates how changes in driver input can be matched over time with changes in the output of the virtual representation 89 of the real car's track position / location. Recording and correlating the time history of input and output for one lap provides a simplified version of the black-box dynamic model for one lap, which can then be used to determine the possible track positions of the virtual car 90 based on the user's computer-generated virtual input 38.

[0117] This embodiment is based on a match simulation that takes place at game server 4. In this context, it will now be described and... Figure 15 The diagram illustrates an example of how the virtual race simulation engine 36 implements dynamic black-box simulation. Kinematic race position data 22 is matched against real driver input 26 from live or recorded events 10 via a (previously mentioned) reference black-box model generator 110. Its output is provided to a game black-box implementation generator 112, which in turn uses the reference to determine the impact of player input data (driving commands) 38 on the position of the virtual vehicle 90 they control in the race. In effect, the reference black-box generator defines the input-to-output transfer function, which the game black-box implementation engine 112 can use. As mentioned above, an AI engine 48 is also provided to assist players in competing against professional drivers of real vehicles. Once the position of the virtual vehicle 90 has been determined by the game black-box implementation engine 112, it is passed to the race simulation output engine 114, which uses the position of the virtual vehicle 90 to generate a representation 90 of the vehicle in the race. The interactive hybrid racing environment for all virtual vehicles 89 and 90 is generated by the racing simulation output engine 114, which uses the stored data model 44 and outputs the virtual racing simulation environment 46 to all relevant gaming devices 2.

[0118] The black-box dynamic model minimizes the need for sophisticated white-box models (i.e., models that combine more detailed models based on complex physics, which in turn combine even more detailed models...) to create highly complex and dynamically changing vehicles in highly complex and dynamically changing environments. Therefore, complex factors that cannot be dynamically measured and modeled (such as machine wear, tire compound deposits affecting wheel grip on the track surface, and aerodynamic gusts affecting the car's trajectory (to name just three)) are all included in the reality of the measured black-box model. The measured black-box model, then created by the reference black-box generator 110, serves as a reference model for the AI ​​algorithm of the game's AI engine 48, taking into account the specified virtual input 38 of the computer game player, which differs from the input of the reference model, and accordingly, proportionally, and realistically alters the output.

[0119] Due to significant variations in driver input from one lap to another, as well as variations in other physical factors (typically continuous and asymptotic, occasionally discrete), a car's black-box model is different on every lap. However, there are numerous detailed strategies and techniques that can be used with this embodiment to ensure that the method provides the best, dynamically accurate simulation throughout the race, including but not limited to: i. averaging or otherwise combining black-box model data from multiple laps of practice and real-race sessions to create a reference black-box lap model that best represents the driver, car, and track for the day; ii. performing the same operation as i, but on shorter sections of the track (e.g., specific corners or straights); iii. using the lap with the best single-lap time as the reference black-box model for the final performance of the driver, car, and track for the day; iv. Perform the same operation as iii, but on shorter sections of the track (e.g., specific corners or straights); v. Build a black-box model second by second, comparing the virtual input 38 of the game player with the input 26 of the real driver, and adjust the output with direct and immediate reactions (this method is particularly useful on the first lap when no historical records are available, or on the first lap after a "short stop," thus modifying the real car); vi. Supplement the measured black-box model with modeling of known physical laws that apply with known continuity and relative simplicity; vii. Model known limitations of the car's performance (e.g., maximum linear acceleration, maximum linear speed, etc.).

[0120] There are many other techniques, but the essence of this approach is to model and simulate variations around one or more known, valid dynamic black-box reference models, which is far more practical than creating white-box models of extremely complex physical systems in real time.

[0121] In addition to linking to a live event 10 in a real motorsport event, other embodiments of the invention are also applicable to situations where a computer game player interacts with a recorded version 42 of a previous live race, the previous record 42 including a high-precision virtual representation 89 of the real car in the environment of the previous race, measured kinematic performance data 22, and input 26 from the real driver to the real car throughout the race. In these embodiments, all of the above descriptions can be applied to the participation of a computer game player in a recorded race.

[0122] Alternatively, in another embodiment, if the previous real-world race record 42 only contains high-precision, measured kinematic performance data 22 of the virtual representation 89 of the real car in the environment, but no record of the real driver's input 26, then digital pairing can be any other method based on ensuring that the simulated performance of the player's simulator 90 in its environment is a sufficiently close representation of the performance of the virtual representation 89 of the real car in its environment, thereby creating a level playing field between the player and the real car or autonomous vehicle. For example, this could involve reverse engineering the real driver's input 26 using a high-fidelity racing simulator, having the driver practice, and then recording the input history, which would allow the driver to reproduce the performance of the virtual representation 89 of each real car in the recorded race. This is an illustrative example of a method for reconstructing the real driver's input 26 into a dynamic black-box model when data from the original event is unavailable.

[0123] Now for reference Figures 16 to 20 Another embodiment of the invention is described, which differs primarily from the embodiments described above in that it is implemented in a more distributed manner. Many components of this other embodiment are related to... Figures 1 to 4 The embodiments operate in a similar manner, and to avoid unnecessary repetition, this document will only discuss the differences. References Figure 16 The configuration of game device 2 is shown. Here, the virtual race simulation engine 36 is set in game device 2 instead of game server 4, so that player input commands 38 derived from player input can be directly input into the virtual race simulation engine 36 without having to be sent to game server 4. This greatly improves efficiency and bandwidth consumption. The virtual race simulation engine 36 is provided with at least kinematic position data 22 and real driver input data 26 from the virtual representation 89 of the real car in live event 10, record reference black box data 120 and supplementary data 122 from game server 4 via communication engine 124. The received data is stored in local data storage 5 and communicated by the virtual race simulation engine 36 with... Figure 2 The received data is used in a similar manner to that described in the embodiments. Similarly, data model 44 provides further support for the generation of the virtual race simulation environment 46. An AI engine 48 is provided to assist the players in competing with real drivers, as explained earlier. One result of adopting a more distributed approach to the virtual race simulation is that the game server 4 must be updated using the position of each player's virtual vehicle 90. Therefore, the communication engine 124 is configured to send not only the car selection 126 used at the start of the race and any changes made to it during the race 126, but also the virtual vehicle position data 126 determined by the virtual race simulation engine 36 during the race to the game server 4.

[0124] Now for reference Figure 17 The diagram illustrates a game and entertainment server 4a in this distributed embodiment. The game and entertainment server 4a has a virtual vehicle position processor 128 that calibrates the position of each virtual vehicle 90 and provides the position of the virtual vehicle 90 to a virtual match management engine 130. The virtual match management engine 130 then provides each game device 2 with information about their position and relative performance in the match, as well as information about the possible positions of other virtual vehicles 90 in the match (if it determines that the match between virtual players will be enhanced or become more competitive), to complete the match environment. The virtual match management engine 130 can also generate a reference black box for previously recorded data 42 and provide this reference black box to all game devices 2. Because this is pre-recorded data 42, this information can be provided in a time-critical manner not associated with the real-time match. Then the game and entertainment server 4a only handles the monitoring of the game, because it sees the location of all competitors—virtual and real competitors, and may feed telemetry and other data back to the "captured" car (therefore requiring a link from the live event data capture server 24), may feed selected competitive visuals back to each virtual competitor, may feed virtual game statistics back to the real teams, and so on.

[0125] Furthermore, in this embodiment, the real data 28 corresponding to the live event is routed from the live event data capture server 24 to the game and entertainment server 4a, and then distributed from the game and entertainment server 4a to the game devices 2. This facilitates synchronization. However, in another embodiment, the real live event data can be routed directly to each game device 2, which has the advantage of reducing latency and reducing the processing power required at the central server.

[0126] Figure 18a Data from the live event data capture server 24 and virtual vehicle location data received from the game device 2 at the game and entertainment server 4a are shown. Although Figure 8 a and Figure 13 While very similar to version a, in this embodiment, additional data 132 is also provided to the game and entertainment server 4a. This additional data 132 may be live telemetry data, radio communication data between teams and drivers, etc. As described above, these additional data streams 132 can make the game more realistic or more engaging.

[0127] Figure 18bThe diagram shows virtual car location data 134 sent from gaming device 2 to gaming and entertainment server 4a. This data also includes a game computer identifier 56. This location data 134 is used to update the location of virtual vehicle 90 so that, as determined by gaming and entertainment server 4a, this location can be transmitted to other participants in the same game using other gaming devices 2. Other data fields 136 relate to other data, such as the selection of a real vehicle to be associated with or handicap data.

[0128] Now for reference Figure 19 The diagram illustrates a virtual match management engine 130 within a game and entertainment server 4a. The virtual match management engine 130 determines which game devices 2 will receive which updated location data. At its core is a match engine management processor 140, which can combine match frames based on the different data streams provided to the game and entertainment server 4a. Furthermore, the virtual match management engine 130 includes a reference black-box model generator 142, which is related to the aforementioned reference... Figure 15 The same implementation is described, except that it operates only on previously recorded event data 42. Live data reference black-box model generation is passed to each gaming device 2. Communication engine 144 provides location data and reference black-box model data 146 to each appropriate gaming device 2. Furthermore, communication engine 144 can also provide each gaming device 2 with other match data 132 (as described above), and can also route live event data if it is not directly provided to each gaming device 2.

[0129] refer to Figure 20The diagram illustrates a method 150 of system operation at gaming device 2 according to this embodiment. Method 150 begins at step 152 where the gaming device sends a request to a gaming and entertainment server 4a to participate in a race. This request may include specific configuration details that enable the gaming and entertainment server 4a to interact with the gaming device 2, and may also explicitly specify the virtual representation 89 of the real vehicle to which the player on device 2 will be linked. For example, these game options may be explicitly specified if the gaming device 2 requires a direct feed of live data and / or a live telematics feed from a live event data capture server 24. In the next step, at step 154, the gaming server provides the required information, such as links to live data feeds from the live event data capture server 24 and the positions of other virtual player cars 90 in the race, as determined by the virtual race management engine 130. Although not shown, the gaming and entertainment server 4a can utilize this information to create an overall view of the active gaming device 2 assigned to a specific event 10. As with the embodiments described above, a single virtual representation 89 of a real vehicle may have multiple players assigned to it. Subsequently, at step 156, a virtual racing environment is locally generated on the gaming device, which includes the initial position of the virtual representation 89 of the real vehicle. As previously mentioned, this can be based on real data 28 corresponding to a live event or pre-recorded data 42 from a previously recorded live event. A data model 44 stored in the local data storage 5 is used to generate the virtual racing environment. Once the race begins at step 158, data streams 28, 42 from the live event or from the pre-recorded event are received at step 160, and using data streams 28, 42, the position of the real car representation can be changed at step 162. These new positions are used to generate new positions for the representation 89 of the real vehicle 12 in the virtual racing environment at step 162, and then this new position is presented to the player on the gaming device 2. In response, at step 164, the gaming device generates game control data (virtual racing commands) 38 derived from the player's user input to control their virtual vehicle. These virtual race commands are then compared with the output of a reference black box within the virtual race simulation engine 36, and this is used at step 166 to generate the subsequent positions of the virtual vehicles as a result of the players' user input (it should be understood that the real-time black box generation in this embodiment is as described with reference to FIG. 10). The new positions of the virtual vehicles can then be sent to the game and entertainment server 4a at step 168, and the positions of the virtual vehicles 90 of other players are received from the game and entertainment server 4a, which determines the relevant positions of the virtual vehicles 90 of the other players. Then, at step 170, the new positions of the relevant virtual vehicles are presented to the players on the game device 2. This process continues until the race is determined to end at step 172.

[0130] Finally, it should be clear that the features of this invention can be extended to a wide range of sporting events in which the moving objects to be tracked in real time within a defined competition area are not motor vehicles, but can be, for example, players on a football or basketball court, participants in a downhill skiing competition, or racing dogs on a track. These are all examples of moving entities that are not set areas of a track and are not vehicles. The location data capture system will be a tailored architecture of sensors, computing, and communication devices. Infrared non-emissive structures or small infrared emitting devices (which can emit constant or modulated light and may be triggered) can be incorporated into the clothing of real competitors, thereby providing cold spots or hot spots suitable for tracking and identification. While interactive competition between real athletes or teams and virtual athletes or teams may not be as practical as motorsports, the immersive entertainment and viewing enhancements described above are practical. Sports statistics and analysis can be automatically collected in a simple and understandable manner. Furthermore, (for example) a football or basketball can be designed to include infrared reflective or non-emissive markers invisible to the human eye, and the performance of the tracking device will allow the spin on the ball to be continuously measured. For example, this could also be applied to the cue ball in snooker or billiards, where a single sensor is located above the table. A common, distinctive, and novel feature is the use of wide-angle infrared sensors and illuminators, along with associated computing and communication equipment, to track moving objects (vehicles, players, balls, etc.) within a relatively short range (<100m) to detect passive or active infrared markers, reflectors, absorbers, or transmitters. The real-time data thus obtained is used to enhance various interactive entertainment and gaming experiences.

[0131] The features of this embodiment can be represented and described by the following terms: 1) A system and method are described for computer game players to interact with live real-world competitions involving real cars with drivers or real cars that are autonomous, whereby a single player can start a game by linking their simulated car to one of the real cars and can move forward by maintaining the link to the initial car or by linking from one real car to another real car based on a defined proximity parameter, thereby competing in a competitive manner as a virtual driver against a real driver in a real-world competition. 2) A system and method are described in which all real cars appear in the game when a computer game player's simulator car is not linked to a real car, and the computer game player can choose to avoid the real cars and compete as independent entities, or choose to link with a real car based on a defined proximity parameter. When the player's simulator car is linked to a real car, the real car will not appear as a competing car in the player's computer game. 3) A system and method are described in which the simulation of a computer game player's car is a digital pairing of the real car when the player's car is linked to a real car. This means that the performance of the player's car in its environment is a sufficiently close simulation of the performance of the real car in its environment to establish fair competition between the player and a real driver or autonomous vehicle. 4) A system and method are described in which digital pairing is based on dynamic black-box simulation of the kinematic performance of a real vehicle in its environment with high precision measurements and inputs from a real driver to the real vehicle. 5) Describes a system and method in which a computer game dynamically associates virtual areas with each real car, such that when a computer game player's car is in these areas, it either becomes a digital pairing with a real car or inherits specific performance characteristics associated with the real car. 6) A system and method are described in which, when a computer game player's car is linked to a real car and the real car takes time off from the race for any maintenance or repair activity (damage repair, refueling, new tires, etc.), the computer game player can choose to take the same amount of time off from the race (possibly delayed by 1 lap), in which case their simulated car inherits the changed performance characteristics of the real car. 7) Describes a system or method in which, when a computer game player's car is not linked to a real car, the computer game player can choose to take time off from the game for simulated maintenance or repair activities, and upon returning to the real game, the computer game player's car will be eligible to link to any real car with similar compatible maintenance or repair activities. 8) Describes a system and method in which many computer game players (up to and including the number of real cars in the game) can be linked with real cars throughout the game, thus competing against each other and against real cars in a live, realistic game. 9) A system and method are described for large-scale interaction between a large number of computer game players and live real-world competitions involving real cars with drivers or real cars with autonomous driving capabilities, whereby each player can start a game by linking their simulated car to one of the real cars and, by linking from one real car to another based on defined parameters, compete in a competitive manner as a virtual driver against all other computer game players in the real-world competition. 10) describes a system and method in which game players compete in large-scale esports competitions, where each player needs to interact with real-world competition in a realistic, engaging, and challenging manner, and a fair method is needed to rank players based on their performance and final position in a large number of esports events, based on the highly accurate position of computer game players' simulated cars relative to each other. 11) A system or method is described in which the interaction of a computer game player is performed with a recorded version of a previous game, the record including highly accurate, measured kinematic performance of a real car in the environment of the previous game and inputs from the real driver to the real car throughout the game. 12) As an alternative, a system or method is described in which the interaction between a computer game player and a player is performed against a recorded version of a previous match, the record containing only highly accurate, measured kinematic performance of a real car in the environment of that previous match. In this case, digital pairing is based on a digital record of the kinematic performance of a real car in its environment, combined with any other method, which ensures that the simulated performance of the player's car in its environment is a sufficiently close representation of the performance of a real car in its environment to create level competition between the player and a real driver or autonomous vehicle. 13) describes a system and method in which, when a computer game player's simulator car is replaced by a real car, there is an exchange of video, audio, or any other technical data in any direction and for any purpose between the player's computer system and the real car and / or its associated team and facilities. 14) describes a system and method in which each computer game player is assigned a handicap based on a fair assessment of their prior experience and accumulated skills, and the computer game uses the handicap to moderate the car's response to the player's input in its environment so that players with different levels of experience and skill have roughly equal chances of competing with and / or competing against real drivers in real cars and winning the game. 15) A system and method are described in which non-computer game racing fans can participate in an interactive viewing experience and receive technical data, video streams, audio streams, or any other data involved in the previously described system and method by arbitrarily linking to any number of real cars and computer game cars using any of the methods described above. 16) describes a system and method in which a virtual vehicle of a gamer can be associated with a representation of a real motor vehicle, and takes positional data of the real vehicle in the race and input from the real driver as inputs, and uses these inputs to generate a reference black box representation, which can be used to evaluate the gamer's input during the race to determine a simulated game environment in which the virtual player's vehicle can be matched with the representation of the real vehicle to simulate a real racing environment. 17) describes a system and method in which a black-box modeling approach is used to capture and analyze dynamic input and output data of real vehicles, the results of which can be fed back to participating teams in real time to help them understand the performance of the driver and the vehicle in their environment and, in particular, to provide teams with highly accurate kinematic data on the performance of their vehicles that would have been impossible to obtain until now.

[0132] A system and method are described in which vehicle movement is sensed by a group of position sensing devices positioned around a race track. Each sensing device includes an IR sensor and a communication device. The IR sensor detects IR radiation emitted, reflected, or transmitted from the vehicle to determine the vehicle's tracking position. The communication device transmits the position data to a position data capture system, where the data can be calibrated and provided as live position data regarding racing events involving these vehicles to a gaming device or server. However, it should be understood that the motion tracking system is not limited to vehicle movement as described above. Precise position detection of moving objects is possible using appropriately configured groups of infrared sensors and connections to other groups of infrared sensors, and can be applied to non-motorized sports, such as those involving the movement of humans or animals.

[0133] It should also be understood that various modifications to the embodiments are possible, and elements of one embodiment can be combined with elements of other embodiments without difficulty. Therefore, it should be understood that the methods and systems described herein are non-limiting examples of how different aspects of the invention can be implemented, and the invention will be determined by the spirit and scope of this disclosure.

Claims

1. A computer-implemented method for controlling an interactive hybrid environment representing motorized sports events on a track, the interactive hybrid environment including real vehicle representations and virtual vehicle representations on the track, the method comprising: Receive real sensor data stream, the real sensor data stream including: a. Real kinematic data of actual vehicles on the track, wherein the real kinematic data is captured by infrared sensors at the track, and b. Regarding the real control data of the driver's control of the real vehicle, the real control data is captured by vehicle sensors and obtained from the real vehicle via a telemetry system; The real kinematic data is used to determine the position and kinematic behavior of the real vehicle representation within the interactive hybrid environment; The real control data and the real kinematic data are used to create a black box determination of the position of the real vehicle on the track based on the real control data; Receive computer-generated control data streams, which are obtained through user-computer interaction, wherein the computer presents the interactive hybrid environment to the user and captures user input, the user input instructing the user to control the kinematic behavior represented by the virtual vehicle; and The position and kinematic behavior of the virtual vehicle representation within the interactive hybrid environment are determined by using the black box and the computer-generated control data stream.

2. The computer implementation method according to claim 1, wherein, The real sensor data stream includes real kinematic data of multiple real vehicles on the track and real control data about the control of each of the multiple real vehicles by the corresponding driver.

3. The computer implementation method according to claim 2, wherein, The real sensor data stream for each of the plurality of real vehicles includes a vehicle identifier.

4. The computer implementation method according to claim 2, wherein, The computer-generated control data stream includes multiple computer-generated data streams, each generated by different users interacting with the corresponding computer and capturing the input of the corresponding users.

5. The computer-implemented method according to claim 4, wherein, Each of the plurality of computer-generated data streams includes a computer device identifier.

6. The computer-implemented method according to claim 4, wherein, The number of the plurality of real vehicles is less than the number of the plurality of computer-generated data streams, and the method further includes linking a subset of the plurality of virtual vehicle representations with a real vehicle representation to create a linked representation.

7. The computer implementation method of claim 6 further includes using the link representation to represent a subset of the plurality of virtual vehicle representations within the interactive hybrid environment, wherein the positions of the virtual vehicles in the subset are within the tolerance limits of the real vehicles.

8. The computer-implemented method according to claim 6, wherein, The number of the plurality of computer-generated data streams is several times greater than the number of the plurality of real vehicles, and the linking step includes linking each of the plurality of computer-generated data streams to the plurality of real vehicle representations in a uniformly distributed manner.

9. The computer-implemented method according to claim 6, wherein, The number of the plurality of computer-generated data streams is several times greater than the number of the plurality of real vehicles, and the linking step includes linking each of the plurality of computer-generated data streams to the plurality of real vehicle representations in a logarithmic distribution.

10. The computer-implemented method according to claim 4, further comprising: The interactive hybrid environment is updated using the new positions of the real vehicle representation and the virtual vehicle representation determined by the received real sensor data stream and the multiple computer-generated data streams; Generate updated interactive hybrid environments; and The updated interactive hybrid environment is broadcast from the central server to multiple remote computers.

11. The computer-implemented method according to claim 4, further comprising: The black box determination and the real sensor data stream are broadcast from the central server to multiple remote computers; The interactive hybrid environment is generated at each remote computer; The interactive hybrid environment is updated using the new positions of the real vehicle representation and the virtual vehicle representation determined by the received real sensor data stream and the plurality of computer-generated data streams; and The new location represented by the virtual vehicle is sent to the central server.

12. The computer-implemented method according to claim 1, further comprising: An artificial intelligence engine is used to alter the relationship between the computer-generated control data stream and the synthesized position of the virtual vehicle, the artificial intelligence engine referring to the black box for determination.

13. The computer-implemented method according to claim 1, wherein, The received real kinematic data includes longitudinal position data relative to the track, lateral position data relative to the track, and vehicle orientation data relative to the track.

14. The computer-implemented method according to claim 1, wherein, The real control data includes one or more of the following: the steering wheel position, accelerator position, brake pedal position, and gear selection of the real vehicle.

15. The computer-implemented method according to claim 1, further comprising: Retrieve the real sensor data stream from a data storage that has stored a copy of the real sensor data stream when it was generated.

16. The computer implementation method according to claim 1, wherein the receiving step comprises: When the sports event occurs, the real sensor data stream is received in near real-time.

17. The computer implementation method according to claim 1, wherein, The real sensor data stream has a sampling rate of at least 25 Hz and captures the position of the real vehicle, providing the position of the real vehicle to the interactive hybrid environment within 40 milliseconds of being captured.

18. The computer implementation method according to claim 1, wherein, The real sensor data stream has a sampling rate of at least 60 Hz and captures the position of the real vehicle, providing the position of the real vehicle to the interactive hybrid environment within 16.7 milliseconds of being captured.

19. The computer implementation method according to claim 1, further comprising: The interactive hybrid environment is generated using a stored data model.

20. The computer-implemented method according to claim 1, further comprising: Receive video data streams from the real vehicle and include the video data streams in the interactive hybrid environment.

21. The computer-implemented method according to claim 1, further comprising: Receive audio data streams from the real vehicle and include the audio data streams in the interactive mixing environment.

22. The computer-implemented method according to claim 2, further comprising: At a time point when the location of the virtual vehicle representation is within a predetermined threshold of the location of the real vehicle representation, the representation of one of the plurality of virtual vehicles is linked to the representation of one of the plurality of real vehicles, and the representation of the real vehicle is used as the representation of the virtual vehicle in the interactive hybrid environment.

23. The computer-implemented method according to claim 22, further comprising: Receive audio or video data streams from the real vehicles and include the audio or video data streams in the interactive hybrid environment, wherein a representation of one of the plurality of virtual vehicles is linked to a representation of one of the plurality of real vehicles, and activate the operation of providing the audio or video data streams received from the real vehicles to the computer presenting the interactive hybrid environment to the user.

24. The computer-implemented method according to claim 22, further comprising: At a time point when the location of the virtual vehicle representation is outside a predetermined threshold of the location of the real vehicle representation, the link between the representation of one of the multiple virtual vehicles and the representation of one of the multiple real vehicles is broken, and the virtual vehicle representation and the real vehicle representation are displayed separately within the interactive hybrid environment.

25. The computer-implemented method according to claim 22, further comprising: Details of any virtual vehicles linked to the real vehicle representation are provided to a remote third-party computer.

26. The computer-implemented method according to claim 2, wherein, Each real vehicle has a set of performance features. The method further includes: determining the location representation of the real vehicle that is closest to the virtual vehicle representation among the plurality of real vehicles, and using the set of performance features of the closest representation of the real vehicle as the performance features of the virtual vehicle.

27. The computer-implemented method according to claim 1, further comprising: The infrared sensor is used to capture the position data of the real vehicles on the track, the position data that changes over time is converted into the real kinematic data stream, and the real kinematic data stream is sent to the central server in real time.

28. The computer-implemented method according to claim 27, wherein, The capture step includes: using multiple sensor groups to monitor different sections of the track to capture the location data, wherein each sensor in each sensor group detects emitted, reflected, or transmitted infrared radiation from one or more vehicles running on the track within the field of view of the sensor.

29. The computer-implemented method according to claim 28, further comprising: The infrared radiation detected by the infrared sensor is processed to determine the kinematic data of one or more real vehicles running on the track.

30. A computer system for controlling an interactive hybrid environment representing motorized sports events on a track, the interactive hybrid environment including real vehicle representations and virtual vehicle representations on the track, the system comprising: A receiver is configured to receive a real sensor data stream, the real sensor data stream including real kinematic data of a real vehicle on the track and real control data about the control of the real vehicle by the driver, the real kinematic data being captured by an infrared sensor at the track and the real control data being captured by a vehicle sensor and obtained from the real vehicle via a telemetry system; A virtual race command processor is configured to receive a computer-generated control data stream, which is obtained through user-computer interaction, wherein the computer presents the interactive hybrid environment to the user and captures user input, which instructs the user to control the kinematic behavior represented by the virtual vehicle. as well as A virtual match simulation engine, comprising: A race simulation output engine, which uses the real kinematic data to determine the position and kinematic behavior of the real vehicle representation within the interactive hybrid environment; A reference black-box model generator is configured to use the real control data and the real kinematic data to create a black-box determination of the position of the real vehicle on the track based on the real control data; and A game black-box implementation engine is configured to determine the position and kinematic behavior of a virtual vehicle representation within the interactive hybrid environment by using the black-box determination and the computer-generated control data stream.

31. The computer system according to claim 30, further comprising: An artificial intelligence engine configured to alter the association between the synthetic locations of the virtual vehicles.

32. The computer system according to claim 31, wherein, The artificial intelligence engine is configured to broaden the threshold required to receive the computer-generated control data stream in order to generate a given location for the virtual vehicle.

Images