System for a mixed-reality racing game and broadcast

Abstract

A computer system for a mixed-reality racing game, wherein the computer system comprises one or more computing devices and instructions which, when executed on at least one processor of the computer system, cause the computer system to perform a mixed-reality racing game, and wherein the computer system can be connected to a user device, and wherein the computer system is configured for the following steps: Providing a virtual racing environment with a virtual race track, wherein the virtual race track is a virtual representation of a real race track; Providing a virtual race car model that represents a real race car; Providing a controllable virtual race car; Receiving telemetry data from the real racing car moving on the real race track, wherein the telemetry data includes at least position data and movement data of the real racing car, and matching the telemetry data of the real racing car to the virtual racing car model; Received from the user device, control data associated with the controllable virtual race car, and user data including geolocation information of the user device, the geolocation information defining a geographic location of the user device; Obtaining a video element that is associated with the geolocation information and profile of the real race car; Adapting the video element to the virtual race car model, which represents the real race car; Calculating the position and speed of the virtual race car model on the virtual race track based on telemetry data; Calculating the position and speed of the controllable virtual race car on the virtual race track based on the control data; Generating a visual representation of the mixed-reality racing game, comprising the virtual racing environment together with the controllable virtual race car on the virtual race track, wherein the visual representation is generated at least partially based on the determined position and speed of the virtual race car model and the determined position and speed of the controllable virtual race car.

Description

TECHNICAL AREA

[0001] The present invention relates to a system for a mixed-reality racing game and the provision of real-world racing data for the game. BACKGROUND OF THE INVENTION

[0002] In racing games played on a computer system, players are set up to compete against each other in a multiplayer game. This can require multiple players to participate simultaneously. Furthermore, the game content may be the same for all players, regardless of their location. Therefore, greater flexibility and customization options would be beneficial while still providing a good user experience for each player. INVENTION SUMMARY

[0003] The scope of protection claimed for the various embodiments is defined in the independent claims. Further embodiments that fall within the scope of protection are defined in the dependent claims. Exemplary embodiments that do not fall within the scope of protection defined in the claims are to be considered as examples that contribute to understanding the scope of protection.

[0004] According to a first embodiment, a computer-implemented method is provided that includes a mixed-reality racing game, wherein the method comprises: providing a virtual racing environment with a virtual racetrack, wherein the virtual racetrack is a virtual representation of a real racetrack; providing a virtual race car model representing a real race car; providing a controllable virtual race car; receiving telemetry data from the real race car moving on the real racetrack, wherein the telemetry data includes at least position and motion data of the real race car; and associating the telemetry data of the real race car with the virtual race car model; and receiving, from a user device, control data associated with the controllable virtual race car and user data including geolocation information of the user device.where the geolocation information defines a geographic location of the user device, obtaining a video element that is associated with the geolocation information and the profile of the real race car, adapting the video element to the virtual race car model representing the real race car, calculating a position and speed of the virtual race car model on the virtual race track based on the telemetry data, calculating a position and speed of the controllable virtual race car on the virtual race track based on the control data, generating a visual representation of the mixed-reality racing game that includes the virtual racing environment along with the controllable virtual race car on the virtual race track,where the visual representation is generated at least partially based on the determined position and speed of the virtual racing car model and the determined position and speed of the controllable virtual racing car.

[0005] According to a second embodiment, a computer program product is provided which includes instructions which, when executed by a computer system, cause the computer device to carry out a computer-implemented method according to the first embodiment.

[0006] According to a third embodiment, a non-volatile, computer-readable medium is provided which includes program instructions stored on it which, when executed on a computer system, cause the computer system to carry out a computer-implemented procedure according to the first embodiment.

[0007] According to a fourth embodiment, a system for a mixed-reality racing game is provided, the system comprising a computer system which includes instructions which, when executed on at least one processor of the computer system, cause the computer system to perform a mixed-reality racing game, and a user device which can be connected to the computer system, wherein the computer system is configured to perform the following steps: providing a virtual racing environment with a virtual racetrack, wherein the virtual racetrack is a virtual representation of a real racetrack; providing a virtual race car model which represents a real race car; providing a controllable virtual race car; receiving telemetry data from the real race car which is moving on the real racetrack, wherein the telemetry data includes at least position data and motion data of the real race car.and mapping the telemetry data of the real race car to the virtual race car model, receiving control data from a user device that is associated with the controllable virtual race car, and user data that includes geolocation information of the user device, where the geolocation information defines a geographic location of the user device, obtaining a video element that is associated with the geolocation information and the profile of the real race car, adapting the video element to the virtual race car model that represents the real race car, calculating a position and speed of the virtual race car model on the virtual race track based on the telemetry data, calculating a position and speed of the controllable virtual race car on the virtual race track based on the control data, generating a visual representation of the mixed-reality racing game,which comprises the virtual racing environment together with the controllable virtual race car on the virtual race track, wherein the visual representation is generated at least partially based on the determined position and speed of the virtual race car model and the determined position and speed of the controllable virtual race car.

[0008] According to a fifth embodiment, a computer system for a mixed-reality racing game is provided, wherein the computer system comprises one or more computing devices and instructions which, when executed on at least one processor of the computer system, cause the computer system to perform a mixed-reality racing game, and wherein the computer system can be connected to a user device, the computer system being configured for the following steps: providing a virtual racing environment with a virtual racetrack, wherein the virtual racetrack is a virtual representation of a real racetrack; providing a virtual race car model representing a real race car; providing a controllable virtual race car; and receiving telemetry data from the real race car moving on the real racetrack.wherein the telemetry data includes at least position and motion data of the real racing car, and mapping telemetry data of the real racing car to the virtual racing car model, receiving from a user device control data associated with the controllable virtual racing car and user data including geolocation information of the user device, wherein the geolocation information defines a geographic location of the user device, receiving a video element associated with the geolocation information and profile of the real racing car, adapting the video element to the virtual racing car model representing the real racing car, calculating a position and speed of the virtual racing car model on the virtual race track based on the telemetry data,Calculating the position and speed of the controllable virtual race car on the virtual race track based on the control data; generating a visual representation of the mixed-reality racing game, which includes the virtual racing environment together with the controllable virtual race car on the virtual race track, wherein the visual representation is generated at least partially based on the determined position and speed of the virtual race car model and the determined position and speed of the controllable virtual race car. BRIEF DESCRIPTION OF THE DRAWING Fig. Figure 1 schematically represents an example of a real racing event, e.g. a car race. Fig. Figure 2 schematically represents an example of a real racing car. Fig. Figure 3 schematically represents the race data unit. Fig. Figure 4 schematically presents examples of different user devices. Fig. Figure 5 schematically presents examples of different user devices. Fig. Figure 6 represents an exemplary embodiment of a system. Fig. Figure 7 represents an exemplary embodiment of a system. Fig. Figure 8 shows a schematic configuration example of the software module with which the computer system can be operated. Fig. Figure 9 schematically represents an example of a two-dimensional visual representation of a mixed-reality racing game. Fig. Figure 10 schematically represents an example of a three-dimensional visual representation of a mixed-reality racing game. Fig. Figure 11 presents a flowchart according to an exemplary embodiment. Fig. Figure 12 schematically represents an exemplary embodiment of a system. Fig. Figure 13 schematically represents an exemplary embodiment of a system. Fig. Figure 14 schematically represents an exemplary implementation of a database structure. Fig. 15 represents the adaptation of a generated video element to a representation of a real racing car. Fig. Figure 16 represents an exemplary embodiment in which the video element is a three-dimensional image element configured to conform to the shape of the racing car or part of the shape of the racing car. Fig. Figure 17 presents another exemplary embodiment in which the racing car database and the vehicle profile data comprise the three-dimensional vehicle model representing the racing car. DETAILED DESCRIPTION OF THE EXECUTION EXAMPLES

[0009] Mixed reality offers many possibilities for immersive user experiences. For example, a user can experience more than just watching a racing event, such as a car race. For instance, a user can experience a car racing game by inputting a real-world racing event, creating a more immersive experience and making the user feel like they are part of the actual race. The exemplary implementations described below demonstrate how to realize the benefits of an enhanced mixed reality user experience.

[0010] Fig. Figure 1 schematically represents an example of a real racing event, such as a car race. The real racing event takes place on a real race track 10. One or more real racing cars 11, 12 move on or along the real race track 10 during the car race. The real race track 10 can be any race track with a specific physical shape and length, such as a motor racing circuit. The real race track 10 can be an infinite race track forming a loop or a lap, or alternatively, a race track that includes a start and finish line provided at different geographical locations.

[0011] The real racing car 11, 12 includes or is equipped with one or more sensors configured to acquire parameters of the real racing car 11, 12 in order to generate telemetry data based on the acquired parameters during the movement of the real racing car 11, 12 on or along the real race track 10. The acquired parameters include at least one position and one movement of the real racing car 11, 12, as recorded by the one or more sensors. In the context of this application, telemetry data can be understood as data of the real racing car acquired by the one or more sensors provided on or associated with the real racing car 11, 12 during the movement of the real racing car 11, 12 on the real race track 10.The telemetry data may include at least position and movement data of the real racing car 11, 12, measured by the one or more sensors provided on or associated with the real racing car 11, 12.

[0012] Fig. Figure 2 schematically represents an example of a real racing car 11. The real racing car 11 includes a racing data unit 20. The racing data unit 20 is configured to generate or measure telemetry data and to transmit this telemetry data to an external computer system. The computer system may comprise one or more computing devices configured to perform tasks involving computer instructions as a combined unit. The computer system may be a server, back-end, edge computing system, and / or cloud-based computing system. The racing data unit 20 includes the one or more sensors configured to record the parameters of the real racing car 11 during its movement.It should be noted that one or more sensors and other components of the racing data unit 20 may be distributed at different locations in the real racing car 11 or provided in a structurally compact unit.

[0013] As in Fig. As shown in Figure 1, the real racetrack 10 is also equipped with one or more environmental sensors 90, 91. The one or more environmental sensors 90, 91 are configured to generate or measure environmental measurement data of the real racetrack 10. The environmental sensors 90, 91 can include at least one or more humidity sensors configured to detect and / or measure humidity on the real racetrack 10, or one or more rain sensors configured to detect and / or measure rain on the real racetrack 10.In some exemplary embodiments, the environmental sensors 90, 91 may additionally or instead of humidity and / or rain sensors include one or more wind sensors configured to detect or measure wind speed or wind speed and direction on the actual racetrack 10, and / or one or more temperature sensors configured to measure the temperature of the actual racetrack 10 and / or the temperature of the atmosphere on the actual racetrack 10. In some exemplary embodiments, the environmental sensors 90, 91 may additionally or instead of the aforementioned sensors include one or more cameras configured to generate image or video data of the actual racetrack 10.

[0014] Fig. Figure 3 schematically represents the race data unit 20. The race data unit 20 comprises a sensor module 30, which includes one or more sensors configured to acquire and measure parameters of the real race car 11, 12 and to generate telemetry data based on the acquired parameters during the movement of the real race car 11, 12 on or along the real race track 10. The sensor module 30 can include at least sensors for acquiring the position and movement of the real race car 11, 12 on or along the real race track 10. The sensor module 30 is configured to generate position and movement data of the real race car 11, 12 during its movement on the real race track 10. The sensor module 30 can include one or more position sensors 31 configured to acquire the position of the real race car 11, 12 on or along the real race track 10.The position sensor 31 can be a navigation satellite receiver configured to detect or measure the position of the real race car 11, 12, such as a GPS (Global Positioning System) sensor, or one or more optical position sensors configured to detect or measure the position of the real race car 11, 12. It should be noted that the position sensor(s) 31 can be any type of position sensor capable of detecting or measuring the position of the real race car 11, 12 on or along the real race track 10. The one or more position sensors are configured to continuously detect or measure the position of the real race car 11, 12 on or along the real race track 10.The one or more position sensors are configured to continuously detect or measure the instantaneous position of the real race car 11, 12 on or along the real race track 10. The one or more position sensors 31 can be configured to continuously detect and / or measure the instantaneous position of the real race car 11, 12 on or along the real race track 10 at time intervals of 1 second or less. Accordingly, the one or more position sensors 31 are configured to detect or measure an instantaneous position of the real racing car 11, 12 on or along the real race track 10 with an output rate of at least 20 Hz (50 ms update rate) or with an output rate of at least 60 Hz (16.7 ms update rate) or with an output rate of at least 120 Hz (8.3 ms update rate) or with an output rate of at least 200 Hz (5 ms update rate).

[0015] The one or more position sensors 31 are configured to generate position data. The telemetry data includes the position data. The one or more position sensors 31, or their output, can be configured to measure or calculate a speed of the real race car 11, 12 on or along the real race track 10 based on the detected or measured position of the real race car 11, 12 or based on the position data. Accordingly, the position data can be used to calculate a speed of the real race car 11, 12 on or along the real race track 10. The sensor module 30 can include one or more motion sensors 32, which are configured to detect movement of the real race car 11, 12 on or along the real race track 10.The one or more motion sensors 32 are configured to detect at least the speed of the real race car 11, 12. Alternatively, the one or more motion sensors 32 are configured to detect at least the speed and acceleration of the real race car 11, 12. The one or more motion sensors 32 may include one or more of the following elements: accelerometer, gyroscope, position sensor, optical sensor, configured to detect or measure the speed or speed and acceleration of the real race car 11, 12. It should be noted that the motion sensor(s) 32 may be any type of motion sensor capable of detecting or measuring the speed and / or speed and acceleration of the real race car 11, 12 on or along the real race track 10.The one or more motion sensors 32 can be configured to continuously detect or measure the movement of the real race car 11, 12 on or along the real race track 10. The one or more motion sensors 32 can be configured to continuously detect or measure an instantaneous speed or speed and acceleration of the real race car 11, 12 on or along the real race track 10.

[0016] The one or more motion sensors 32 can be configured to continuously detect or measure an instantaneous speed or speed and acceleration of the real race car 11, 12 on or along the real race track 10 at time intervals of 1 s or less. Accordingly, the one or more motion sensors 32 are configured to detect or measure an instantaneous speed or speed and acceleration of the real race car 11, 12 on or along the real race track 10 at an output rate of at least 20 Hz (50 ms update rate), or at an output rate of at least 60 Hz (16.7 ms update rate), or at an output rate of at least 120 Hz (8.3 ms update rate), or at an output rate of at least 200 Hz (5 ms update rate). The one or more motion sensors 32 can be configured to generate motion data.Telemetry data can include motion data. This motion data can include speed data, or speed and acceleration data.

[0017] In some exemplary embodiments, the sensor module 30 may further comprise one or more orientation sensors or alignment sensors 33, which are configured to detect or measure an orientation or alignment, or a change in the orientation or alignment, of the real racing car 11, 12. The orientation or alignment of the real racing car 11, 12 can be understood as its orientation or alignment with respect to the real race track 10. Alternatively or additionally, the orientation or alignment of the real racing car 11, 12 can be understood as its orientation or alignment with respect to the real race track 10 and with respect to the cardinal directions (or north).The one or more orientation sensors 33 may comprise one or more of the following elements: accelerometer, gyroscope, optical orientation sensor, configured to detect or measure the orientation or direction of the real race car 11, 12. It should be noted that the orientation sensor(s) 33 may be any type of orientation sensor capable of detecting or measuring the orientation or direction of the real race car 11, 12 on or along the real race track 10. The one or more orientation sensors 33 may be configured to continuously detect or measure the orientation of the real race car 11, 12 on or along the real race track 10.The one or more orientation sensors 33 can be configured to continuously detect or measure the instantaneous orientation of the real racing car 11, 12 on or along the real race track 10.

[0018] The one or more orientation sensors 33 can be configured to continuously detect or measure the instantaneous orientation of the real race car 11, 12 on or along the real race track 10 at time intervals of 1 s or less. Accordingly, the one or more orientation sensors 33 are configured to detect or measure the instantaneous orientation of the real race car 11, 12 on or along the real race track 10 at an output rate of at least 20 Hz (50 ms update rate), or at an output rate of at least 60 Hz (16.7 ms update rate), or at an output rate of at least 120 Hz (8.3 ms update rate), or at an output rate of at least 200 Hz (5 ms update rate). The one or more orientation sensors 33 can be configured to generate orientation data or alignment data. The telemetry data can include the alignment data.In some embodiments, the sensor module 30 may further comprise one or more instrumentation sensors 34 which are configured to detect or measure technical racing characteristics of the real racing car 11, 12 during movement on or along the real race track 10.

[0019] The one or more instrumentation sensors 34 can be configured to detect or measure technical characteristics of the real racing car 11, 12, such as the operation of technical racing systems and / or inputs from a human driver of the real racing car 11, 12 via racing input devices, such as the accelerator pedal, brakes, and / or steering system. It should be noted that the instrumentation sensor(s) 34 can be any type of instrumentation sensor capable of detecting or measuring technical characteristics of the real racing car 11, 12 on or along the real race track 10. The one or more measuring sensors 34 can be configured to continuously detect or measure the technical characteristics or instantaneous technical characteristics of the real racing car 11, 12 on or along the real race track 10.The one or more instrumentation sensors 34 can be configured to continuously record or measure instantaneous technical characteristics of the real racing car 11, 12 on or along the real race track 10. The one or more instrumentation sensors 34 can be configured to continuously record or measure instantaneous technical characteristics of the real racing car 11, 12 on or along the real race track 10 at time intervals of 1 s or less.Accordingly, one or more measuring sensors 34 can be configured to detect or measure instantaneous technical characteristics of the real racing car 11, 12 on or along the real race track 10 with an output rate of at least 20 Hz (50 ms update rate) or with an output rate of at least 60 Hz (16.7 ms update rate) or with an output rate of at least 120 Hz (8.3 ms update rate) or with an output rate of at least 200 Hz (5 ms update rate).

[0020] One or more instrumentation sensors 34 can be configured to generate instrumentation data. The telemetry data can include the instrumentation data. The actual race car 11, 12 and its sensor module 30 can also include additional sensors configured to generate additional measurement data. The telemetry data can include the additional measurement data.

[0021] The racing data unit 20 may further comprise a racing communication module 40, which is configured to provide a communication link with a computer system. The communication module 40 may be configured to transmit telemetry data from the real racing car 11, 12 to the external computer system. The racing data unit 20 may further comprise a memory 42 containing instructions for operating the sensor module 30 and the sensors 31, 32, 33, 34, for processing the sensor measurement data and / or the telemetry data, and for operating the communication module 40 to transmit the telemetry data. The racing data unit 20 may also include one or more processors 44 for executing the instructions stored in the memory 42.

[0022] The communication module 42 can be any known communication module configured to perform data transmission to the external computer system or data transmission between the external computer system and the race data unit 20. For example, the communication module could be an internet communication module, a mobile network communication module, a local area network (LAN) communication module, an ultra-wideband (UWB) or wide area network (WAN) communication module, or any other suitable communication module.

[0023] Fig. 4 and Fig. Figure 5 schematically illustrates examples of various user devices, such as a personal computer, a laptop, a mobile device, a game console, a virtual reality device, or any other device suitable for playing the racing game. It should be noted that the user device can also be connected to other devices such as headphones, head-mounted displays, etc. Fig. Figure 4 presents an example of a user device, which in this example is a game console 60. The user device 60 comprises a display device 62 and a user input device 64 with one or more user input elements 63. The user input device 64 is configured to generate control data in response to user input. The user device 60 further comprises a display device 62, which is configured to display a virtual racing environment and / or a visual representation of the mixed-reality racing game according to the invention. It should be noted that the display device may include a head-mounted display that is contained in or connected to the user device. The user device 60 further comprises a central processing unit 61. The display device 62 is connected to the central processing unit 61, either wired or wirelessly, to receive the visual representation from the central processing unit 61.The user input device 64 is connected to the central processing unit 61 either wired or wirelessly and is configured to supply control data to the central processing unit 61. The central processing unit 61 may include at least one memory containing instructions for operating the display device 62 and the user input device 64. The central processing unit 61 may also include at least one user device processor configured to execute the instructions. The display device 62 may be any type of display device, such as a computer monitor or a television. The user input device 64 may be any type of user input device, such as a keyboard, a computer mouse, a game controller, or the like.

[0024] Fig. Figure 5 presents another example of a user device 65, which in this example includes an integrated display device 66. The user device 65 further includes a central processing unit (CPU) 67 as an integral part of the user device 65. The CPU 67 includes at least one user device memory containing instructions for operating the display device 66 and the user input device. The CPU 67 may also include at least one user device processor configured to execute the instructions. The display device 66 is provided as a touchscreen configured to also serve as the user input device. The display device 66 is configured to generate control data in response to user input via the touchscreen.Alternatively or additionally, the central unit 67 includes one or more motion sensors configured to generate motion data based on the movement or orientation of the user device 65. The motion sensors may include one or more of the following: accelerometers, gyroscopes, and magnetometers, or other motion sensors capable of detecting movement or orientation of the user device 65. Accordingly, the user device 65 itself or its motion sensors constitute the user input device. The user device 65 is configured to generate control data in response to movement of the user device 65 by the user. The user device 65 may be a mobile user device, such as a mobile phone.

[0025] The user devices 60, 65 or their central processing unit 61, 67 can include a user device communication module for data transmission with an external computer system. The user device communication module can be any known communication module configured to perform data transmission between the external computer system and the user device 60, 65. For example, the user device communication module can be an internet communication module, a cellular network communication module, a LAN communication module, a WAN communication module, a Wi-Fi communication module, a Bluetooth communication module, an ultra-wideband (UWB) communication module, or any other suitable communication module. The present invention is not limited to any type of user device communication module.

[0026] Fig. Figure 6 presents an exemplary embodiment of the system. In this exemplary embodiment, the system comprises a computer system 50. The computer system 50 can be configured to receive telemetry data from one or more real racing cars 11 or their racing data units 20 via a first network connection 101. The computer system 50 is also configured to receive control data from the user device 60, 65 via a second network connection 201. The first and second network connections 101, 201 can be, for example, the Internet, a mobile network, a local area network (LAN), a wide area network (WAN), an ultra-wideband (UWB) network, or another communication network. Furthermore, the first and second network connections 101, 201 can be implemented by a combination thereof.The present invention is not limited to a particular type of communication network or network connection.

[0027] The computer system 50 can be configured to generate a visual representation of a mixed-reality racing game. The user device 60, 65 can be configured to receive the visual representation via the second network connection 201. The user device 60, 65 can further be configured to display the visual representation using the display device 62, 66. The computer system 50 comprises one or more processors and one or more memories. A software module is stored in the one or more memories. The software module comprises instructions to be executed by the one or more processors of the computer system 50.

[0028] Fig. Figure 7 represents another exemplary embodiment of the system. In this embodiment, the method and the processing of the tasks are distributed between the external computer 50 and the user device 60, 65. Accordingly, the computer system comprises the external computer system 50, as shown in Figure 7. Fig. 6, and the user device 60, 65 or its central processing unit 61, 67. The central processing unit 61, 67 may be configured to provide an internal computer system 50'. In the exemplary embodiment of Fig. 7 The external computer system 50 is configured to receive telemetry data from the one or more real racing cars 11 or their racing data units 20 via the first network connection 101. The user device 60, 65 or the central unit 61, 67 or the internal computer system 50' is configured to receive control data from the user device 60, 65 or from its user input device 64, 66. The external computer system 50 and the internal computer system 50' can together form the computer system 50, 50'. The telemetry data can also be received in the user device 60, 65 via the second network connection 201 from the external computer system 50. Alternatively, the calculations based on the telemetry data can be performed in the external computer system 50 and the calculation results received in the user device 60, 65 or the internal computer system 50' for further processing.

[0029] Fig. Figure 8 shows a schematic configuration example of the software module that can operate the computer system 50, 50'. The computer system 50, 50' can be configured to perform the method steps of the present invention using the software module of the computer system 50, 50'. The computer system 50 and its software module can include an input unit 51. The input unit 51 can be configured to receive the telemetry data and the control data. The input unit 51 can be configured to receive the telemetry data from one or more real racing cars 11, 12. The input unit 51 can be configured to receive the control data from the user device 60, 65. The computer system 50, 50' and its software module can include a processing unit of the real racing car 53, which is configured to connect the telemetry data with a virtual racing car model 72, 73.to assign this and to calculate a position and speed of the virtual race car model 72, 73 on a virtual race track 71 in a virtual racing environment 70 based on the telemetry data assigned to the virtual race car model 72, 73. The virtual race car model 72, 73 can be configured to represent the real race car 11, 12, as in the . Fig. 9 and Fig. 10 is shown.

[0030] The virtual race track 71 can be configured to represent and correspond to the real race track 10 in the virtual racing environment 70. The virtual race track 71 can be a digital twin, a digital representation, or a digital replica of the real race track 10. The computer system 50, 50' and its software module can include a steerable virtual race car processing unit 54 configured to provide a steerable virtual race car 80, to link or assign the control data to the steerable virtual race car 80, and to calculate a position and speed of the steerable virtual race car 80 on the virtual race track 71 in the virtual racing environment 70 based on the control data linked to or assigned to the virtual race car model 72, 73.

[0031] The computer system 50, 50' and its software module can include a virtualization unit 55 configured to provide the virtual racing environment 70 and the virtual race track 71. The virtualization unit 55 can further be configured to generate the visual representation of the racing game, which includes the virtual racing environment 70, representing the virtual race car model 72, 73 together with the controllable virtual race car 80 on the virtual race track 71 based on the calculated position and speed of the virtual race car model 72, 73 and the calculated position and speed of the controllable virtual race car 80.

[0032] The computer system 50, 50' and its software module may include an output unit 52 configured to transmit the visual display as output data from the computer system 50, 50'. The output unit 52 may be configured to transmit the visual display from the computer system 50, 50' to the user device 60, 65. The computer system 50, 50' and its software module may include a virtual racing car model database 56. The virtual racing car model database 56 is configured to store one or more virtual racing car models 72, 73, which are configured to represent one or more real racing cars 11, 12. The computer system 50, 50' and its software module may include a telemetry database 57. The telemetry database 57 may be configured to store telemetry data received by the computer system 50, 50'.The computer system 50, 50' and its software module can include a virtual racetrack database 58. The virtual racetrack database 58 can be configured to store one or more virtual racetracks 71 or one or more virtual racing environments 70 that include the virtual racetrack 71. The computer system 50, 50' and its software module can include a database for controllable virtual race cars 59. The database for controllable virtual race cars 59 is configured to store one or more controllable virtual race cars 80.

[0033] Fig. Figure 9 schematically represents an example of a two-dimensional visual representation of a mixed-reality racing game, comprising the virtual racing environment 70, which represents the virtual racing car model 72, 73 together with the controllable virtual racing car 80 on the virtual race track 71 based on the calculated position and speed of the virtual racing car model 72, 73 and the calculated position and speed of the controllable virtual racing car 80.

[0034] Fig. Figure 10 schematically presents an example of a 3-dimensional visual representation of a mixed-reality racing game, comprising the virtual racing environment 70, which depicts the virtual racing car model 72 together with the controllable virtual racing car 80 on the virtual race track 71 based on the calculated position and speed of the virtual racing car model 72 and the calculated position and speed of the controllable virtual racing car 80. The 2-dimensional visual representation and the 3-dimensional visual representation can be configured to be displayed by the display device 62, 66 or the user devices 60, 65.

[0035] Fig. Figure 11 presents a flowchart according to an exemplary embodiment in which a mixed-reality racing game combines elements from the real world with the mixed-reality racing game itself, which is generated using software algorithms. A user can provide user input for a virtual representation of a real race car, and the game then modifies the behavior of the controllable virtual race car and / or the game according to the received user input. The virtualized real race track could, for example, be a race track where a real car race takes place, and the car race is then broadcast. The broadcast, orBroadcasting can be used as input, based on which the mixed-reality racing game is generated in such a way that the virtual race car model corresponds to the real race car participating in the race, the virtual racing environment corresponds to the environment of the race, the virtual racetrack is a representation of the real racetrack on which the participating race cars drive, and the user can have a user experience of participating in the race by providing input that controls the controllable virtual race car in the mixed-reality game. This enables a user experience in which the user feels like one of the drivers of the race car and competes against the real race cars and their drivers.

[0036] The flowchart can be implemented using a computer-implemented method, which in this exemplary embodiment includes providing the virtual racing environment 70, which comprises the virtual racetrack 71. The virtual racetrack 71 is a virtual image / representation of a real racetrack 10. The virtual racetrack 71 is provided by the virtual racetrack database 58. The method in this exemplary embodiment further includes providing a virtual race car model 72, 73, which represents a real race car 11, 12. The virtual race car model 72, 73 is a virtual representation of a real race car 11, 12. The virtual race car model 72, 73 is provided by the virtual race car model database 56.

[0037] The procedure further includes receiving telemetry data from the real racing car 11, 12. The telemetry data is linked to or associated with the virtual racing car model 72, 73. Each real racing car 11, 12 provides individual telemetry data, and the telemetry data of each of the real racing cars 11, 12 is linked to a virtual racing car model 72, 73 that represents the real racing car 11, 12. The telemetry data may include a racing car identifier, and the telemetry data may be linked to the specific virtual racing car model 72, 73 based on the racing car identifier. The virtual racing car model 72, 73 in the virtual racing car model database 56 may include a corresponding model identifier, and the telemetry data may be linked to the specific virtual racing car model 72, 73 based on the racing car identifier and the model identifier.

[0038] The procedure further includes calculating the position and movement data of the virtual race car model 72, 73 or the virtualized race car on the virtual race track 71 based on the telemetry data. Accordingly, the position and movement of the virtual race car model 72, 73 on the virtual race track 71 are calculated such that they correspond to the position and movement of the real race car 11, 12 on the real race track 10. Therefore, the virtual race car model 72, 73 on the virtual race track 71 becomes a virtualized race car that represents the movement of the real race car 11, 12 on the real race track 10.

[0039] The telemetry data can be received as continuous telemetry data, which continuously define at least the position and speed of the real racing car 11, 12, or continuously define at least the instantaneous position and instantaneous speed of the real racing car 11, 12 on the real race track 10. Accordingly, the method comprises a continuous calculation of an instantaneous position and speed of the virtual racing car model 72, 73 on the virtual race track 71 based on the telemetry data. The telemetry data can be received as real-time telemetry, and the continuous calculation of the position and speed of the virtual racing car model 72, 73 can be performed continuously in real time based on the received real-time telemetry data.Alternatively, the telemetry data can be stored in the telemetry database 57, and the continuous calculation of the position and speed of the virtual racing car model 72, 73 can be performed based on the telemetry data stored in the telemetry database 57.

[0040] The method also includes providing the steerable virtual race car 80. The virtual race car 80 can be provided from the database for steerable virtual race cars 59. The virtual race car 80 can be configured to be steerable by the user with the user device 60, 65 or the user input device 64 or their control unit. The method further includes receiving control data from the user device 60, 65 and assigning the control data to the steerable virtual race car 80. The method further includes calculating a position and speed of the steerable virtual race car 80 on the virtual race track 71 based on the received control data associated with the steerable virtual race car 80.The control data can be received as continuous control data, which continuously define at least the position and speed of the controllable virtual race car 80, or at least the instantaneous position and speed of the controllable virtual race car 80 on the virtual race track 71. Accordingly, the method comprises a continuous calculation of an instantaneous position and speed of the controllable virtual race car 80 on the virtual race track 71 based on the control data.

[0041] The method further comprises generating a visual representation of the mixed-reality racing game, which includes the virtual racing environment 70, representing the virtual race car models 72 and 73 together with the controllable virtual race car 80 on the virtual race track 71, based on the calculated position and speed of the virtual race car models 72 and 73 and the calculated position and speed of the controllable virtual race car 80. Accordingly, generating the visual representation includes generating a visual representation that depicts the virtual race car models 72 and 73 on the virtual race track 71, based on the calculated position and speed of the virtual race car models 72 and 73.Accordingly, the visual representation is configured to depict the virtual race car model 72, 73 on the virtual race track 71 with speed and position corresponding to the real race car 11, 12 on the real race track 10 with real speed and position. Furthermore, generating the visual representation can include generating a visual representation that includes a visual representation of the controllable virtual race car 80 on the virtual race track 71 based on the calculated position and speed of the controllable virtual race car 80. Accordingly, the visual representation is configured to depict the controllable virtual race car 80 on the virtual race track 71 with the speed and position calculated based on the control data.Accordingly, the visual representation can be set up to display the virtual racing car model 72, 73 and the controllable virtual racing car 80 on the virtual race track 71 simultaneously.

[0042] It should be noted that, at least in some variants of the exemplary embodiment described above, environmental sensors, as described above, can be present in the real environment. These sensors provide sensor information based on which the virtual racing environment for the mixed-reality racing game can be generated, as well as information about the conditions of the virtualized racetrack. This ensures that the environmental conditions generated for the mixed-reality racing game correspond to those of the real world when the race takes place. The environmental sensors can be used in addition to, or as an alternative to, software algorithms that can be used to determine the weather conditions in the real environment and / or on the real racetrack from the ambient light.This allows the user to experience driving under real conditions, and the mixed-reality game can imitate the real environmental conditions based on inputs received from the environmental sensors and / or software algorithms set up to detect the real environmental conditions from the transmission.

[0043] Fig. Figure 12 schematically represents an exemplary embodiment of a system that can be used to provide an input video stream, which can be understood as the broadcast that can be used as the basis for generating the mixed-reality user experience discussed above. The system in this exemplary embodiment comprises at least one imaging device 400, such as a digital camera device, configured to generate the input video stream of a car race, comprising one or more race cars 1, 2, 3, which can also be understood as real race cars, such as the real race cars 11, 20 discussed above. Therefore, the input video stream in this exemplary embodiment comprises video images of the one or more race cars 1, 2, 3.The motor race could be, for example, a Formula 1, IndyCar, or NASCAR race, a rally race, or another type of motor race. The imaging device 400 is configured to generate at least a portion of the input video stream or input video data of the motor race. The system further comprises a computer system 50, as described above. In this exemplary embodiment, the computer system 50 is configured to receive the generated input video stream via a first communication link 42. The computer system 50 and its software module include an input unit configured to receive the input video stream. The input unit is further configured to receive a broadcast request from at least one user device. It should be noted that the broadcast request may include user data, which has been discussed above.The input unit is further configured to receive two or more input video streams from two or more imaging devices 400. The computer system 50 and its software module include an identification unit configured to identify the one or more racing cars 1, 2, 3 in the input video stream.

[0044] The computer system further comprises an identification unit, which includes an object detection algorithm trained to detect and identify the race car 1, 2, 3 in the input video stream. The input video stream is used as input data for the object detection algorithm to detect and identify the race car 1, 2, 3 in the input video stream. In the context of this application, detecting the race car 1, 2, 3 in the input video stream can be understood as detecting its existence. Within the scope of the present invention, identifying the race car 1, 2, 3 in the input video stream can be understood as specifically identifying which race car 1, 2, 3 is detected in the input video stream. It should be noted that the race cars 1, 2, 3 may differ in their external shape or appearance.Therefore, it may be necessary to identify race car 1, 2, 3, i.e., which race car or race cars are present in the input video stream.

[0045] The object detection algorithm can be configured to identify race cars 1, 2, and 3 in the input video stream. The object detection algorithm can be any known type, such as a trained machine learning algorithm, a neural network, a statistical detection algorithm, or similar. The object detection algorithm can be trained using images, videos, or digital models of race cars 1, 2, and 3. Optionally, the object detection algorithm can also be configured to detect the orientation of the detected race cars 1, 2, and 3 in the input video stream.

[0046] It should be noted that in some exemplary embodiments, the object detection algorithm may be a single algorithm configured to detect race car 1, 2, 3 in the input video stream, identify the detected race car 1, 2, 3, and also capture the orientation of the identified race car 1, 2, 3. Alternatively, the object detection algorithm may be provided in the form of two, three, or more different algorithms that are jointly configured to detect race car 1, 2, 3 in the input video stream, identify the detected race car 1, 2, 3, and also capture the orientation of the identified race car 1, 2, 3. In some embodiments, the object detection algorithm may also not be configured to capture the orientation of race car 1, 2, 3 in the input image.

[0047] The computer system 50 and its software module further include a content generation unit configured to generate a video element for the input video stream. The content generation unit can be configured to generate the video element based on the identified race car 1, 2, 3 and geolocation information of the user device. The computer system 50 and its software module further include a video processing unit configured to adapt the generated video element to the identified race car 1, 2, 3 in the input video stream in order to provide manipulated video data that can be output. Thus, the generated video element can be superimposed on the identified race car 1, 2, 3.Adapting the generated video element to the identified race car in the input video stream includes, for example, providing a video element overlay or a video element layer on the input video stream to provide the manipulated video data. Computer System 50 and its software module include an output unit configured to broadcast the manipulated video data as an output video stream from Computer System 50 to the user device in response to the broadcast request. Computer System 50 and its software module further include a race car database containing vehicle profile data for each of the race cars 1, 2, 3 in the race. Accordingly, each of the race cars 1, 2, 3 in the race can be provided with separate vehicle profile data, or race car profiles, representing that specific race car 1, 2, 3. The vehicle profile data includes information about the specific race car.

[0048] Computer system 50 and its software module also include a content database. The content database comprises video content elements, each of which is associated with or includes geolocation data that defines a geographic area. It should be noted that the video content element can also be referred to as video content, video element, video content element, or video element. Each video content element can further be associated with vehicle profile data of at least one race car 1, 2, 3. Accordingly, each video content element in the content database can be associated with or provided with geolocation data or geolocation information and vehicle profile data. Thus, the video content elements are race car-specific and geographically area-specific video content elements.

[0049] As in Fig. 12 and Fig. As shown in Figure 13, the input video stream can be received in the input unit 51 of the computer system 50 via the first network connection 420. Furthermore, separate broadcast requests for the output video stream of the car race in the computer system 50 are received from user devices at various geographical locations 103, 203, 303 via second network connections 107, 207, 307 or communication networks. In the exemplary embodiment of Fig. 12. There is an input video stream that is received in the computer system 50 via the first network connection 420 from the imaging device 400, and in the exemplary embodiment of Fig. 13 Three input video streams are received in the computer system 50 via the first network connection 42 from imaging devices 400. Thus, one or more input video streams can be received in the computer system 50 from one or more imaging devices 400.

[0050] The broadcast request can include a request to receive the output video stream of the car race on the user device. Each broadcast request can include user data, and the user data can include geolocation information of the user device. The geolocation information defines a geographic location (103, 203, 303) of the user device at the time the broadcast request is transmitted. Accordingly, each received broadcast request can be associated with or include the geographic location (103, 203, 303) of the user device. The user device's geolocation information includes, for example, the user device's IP address, the user device's communication network node data defining the network node to which the user device is connected, or the user device's navigation satellite coordinates.In some exemplary embodiments, the geolocation information may also include other information defining the geographic location 103, 203, 303 of the user devices. It should be noted that the computer system 50 may receive one or more broadcast requests. The computer system 50 may be configured to process each broadcast request independently. In some exemplary embodiments, the computer system 50 may be configured to group received broadcast requests that include corresponding or identical geographic information defining the corresponding or identical geographic location of the user devices. The computer system may be configured to process the grouped broadcast requests together or as a single broadcast request.

[0051] In this exemplary embodiment, the imaging device 400 and the computer system 50 are connected to or arranged in communication with the first communication link or first communication network 420. Furthermore, the computer system 50 and the user devices can be connected to or arranged in communication with the second communication links or with the second communication network(s) 107, 207, 307. It should be noted that the first and second communication links or networks 420, 107, 207, 307 can be separate communication links or networks or parts of the same communication network. The communication network 420, 107, 207, 307 can be, for example, the Internet, a mobile network, a local area network (LAN), a wide area network (WAN), an ultra-wideband (UWB) network, or another communication network.Furthermore, the communication network 42, 101, 201, 301 can be implemented by a combination thereof. The present invention is not limited to a particular type of communication network. In some exemplary embodiments, the first and second communication links or networks 420, 107, 207, 307 can be arranged such that they are parts of a combined communication network. Accordingly, the computer system 50 can include a system communication element configured to receive the input video stream(s) and the broadcast request(s), as well as to broadcast the output video stream. Thus, the system communication element can be configured to provide a connection to the first communication network 420 and the second communication network 107, 207, 307.

[0052] Furthermore, the imaging device 400, or an imaging system comprising the imaging device 400, may include an imaging device communication element configured to transmit or send the input video stream to the computer system 50. The imaging device communication element may be configured to provide a connection to the first communication network 420. The user device may include a user device communication element configured to transmit the broadcast request to the computer system 50 and to receive the output video stream from the computer system 50. Thus, the user device communication element may be configured to provide a connection to the second communication network 107, 207, 307.

[0053] Fig. Figure 14 schematically represents an exemplary embodiment of a database structure. In this exemplary embodiment, the database structure comprises the race car database 56, which contains separate vehicle profile data 1', 2', 3' for each of the race cars 1, 2, 3 in the car race. The race car profile data 1', 2', 3' each contain vehicle information for the specific race car 1, 2, 3. The database in this exemplary embodiment also includes a content database, which contains one or more specific video content elements 111, 112, 113, 211, 212, 213, 311, 312, 313, each of which can be linked to the specific vehicle profile data 1', 2', 3', which can also be understood as assigned, linked, or associated, as shown in Figure 14. Fig. Figure 5 shows that each specific video content element 111, 112, 113, 211, 212, 213, 311, 312, 313, which is linked to the specific vehicle profile data 1', 2', 3', can be linked to different geolocation data. "Linked to" can also be understood as "associated with". In this exemplary embodiment, the geolocation data defines a specific geographic area 100, 200, 300. Accordingly, each specific video content element 111, 112, 113, 211, 212, 213, 311, 312, 313 can be linked to a specific geographic area 100, 200, 300. For example, each specific video content element 111, 112, 113, 211, 212, 213, 311, 312, 313, which is associated with specific vehicle profile data 1', 2', 3', can be associated with a different geographical area 100, 200, 300.Therefore, the individual vehicle profile data 1, 2', 3' and thus the identified racing cars 1, 2, 3 can be provided with one or more geographically oriented or limited video content elements 111, 112, 113, 211, 212, 213, 311, 312, 313.

[0054] For example, in the case of Fig. 14. The first vehicle profile data 1' can be associated with first video content elements 111, 112, 113. Each of the first video content elements 111, 112, 113 can be associated with different first geolocation data, and each of the different first geolocation data can be configured to define a different first geographic area 100, 200, 300. Similarly, the second vehicle profile data 2' can be associated with second video content elements 211, 212, 213, with each of the second video content elements 211, 212, 213 being associated with different second geolocation data, and each of the different second geolocation data can be configured to define a different second geographic area 100, 200, 300. Furthermore, the third vehicle profile data 3' can be linked to third video content elements 311, 312, 313.Each of the third video content elements 311, 312, 313 can be associated with different third geolocation data, the different third geolocation data being set up to define a different third geographic area 100, 200, 300.

[0055] The geographic area 100, 200, 300 of the geolocation data can be any defined geographic area, such as a continent, a country, a city, a part of a continent, a country, or a city, or any other geographic area. In the exemplary embodiments shown in the figures, the first geographic area 100 is North America, the second geographic area 200 is Europe, and the third geographic area 300 is Asia. The computer system 50 can be configured to receive broadcast requests from user devices located at various geographic locations 103, 203, 303. The broadcast requests can include user data. The user data can include the geolocation information of the user device, and the geolocation information is configured to define the geographic location 103, 203, 303 of the user devices.For example, the first user device can include initial geolocation information in the broadcast request. This initial geolocation information can be configured to define a first geographic location for the first user device. This first geographic location lies within the first geographic area. The second user device can include second geolocation information in the broadcast request. This second geolocation information can be configured to define a second geographic location for the second user device. This second geographic location lies within the second geographic area. Furthermore, the third user device can include third geolocation information in the broadcast request.The third geolocation information can be configured to define a third geographic location for the third user device. This third geographic location lies within the third geographic area 300.

[0056] In this exemplary embodiment, each of the first video content elements 111, 112, 113, which are linked to the first vehicle profile data 1', is in the content database linked to different geolocation data and, furthermore, to a different geographic area 100, 200, 300. The first video content element 111 is linked to the geolocation data configured to define the first geographic area 100. The first video content element 112 is linked to the geolocation data configured to define the second geographic area 200. Furthermore, the first video content element 113 is linked to the geolocation data configured to define or represent the third geographic area 300.Similarly, in the content database, each of the second video content elements 211, 212, 213, which are linked to the second vehicle profile data 2', is linked to different geolocation data and, furthermore, to a different geographic area 100, 200, 300. Second video content element 211 is linked to the geolocation data configured to define the first geographic area 100. Second video content element 212 is linked to the geolocation data configured to define the second geographic area 200. Furthermore, second video content element 213 is linked to the geolocation data configured to define the third geographic area 300.Furthermore, in the content database, each of the third video content elements 311, 312, and 313, which are linked to the third vehicle profile data 3', is linked to different geolocation data and also to a different geographic area 100, 200, and 300, respectively. Third video content element 311 is linked to the geolocation data configured to define or represent the first geographic area 100. Third video content element 312 is linked to the geolocation data configured to define the second geographic area 200. Third video content element 313 is linked to the geolocation data configured to define the third geographic area 300.

[0057] After receiving the input video stream from the imaging device 400 via the input unit 51 of the computer system 50, the input video stream can be provided as input to the identification unit 53. The identification unit 53 is configured to detect and identify the specific race car 1, 2, 3 in the input video stream. In response to the detection and identification of the specific race car 1, 2, 3 in the input video stream, the computer system 50 is configured to associate or link the identified race car 1, 2, 3 with the specific vehicle profile data 1', 2', 3' corresponding to the identified race car 1, 2, 3. It should be noted that the input video stream can also be used as input to provide the virtual race track and the virtualized race car in the virtual environment for the mixed-reality racing game discussed above.In such a use case, the race track of the real race is then represented as a virtual race track, and the virtual environment can be created in such a way that it corresponds to the environment of the car race.

[0058] Linking the identified race car 1, 2, 3 with the specific vehicle profile data 1', 2', 3' corresponding to the identified race car 1, 2, 3 can be done, for example, based on the identification output of the identification unit 53 and the vehicle profile data 1', 2', 3', or based on the output of the object detection algorithm and the vehicle profile data 1', 2', 3'. The computer system 50 can be configured to receive the broadcast request from one or more user devices. Each broadcast request can be provided with user data, including geolocation information of the user device. The geolocation information defines the geographic location 103, 203, 303 of the user devices.

[0059] The racing car 1, 2, 3 can be detected and identified by the identification unit 53 of the computer system 50.

[0060] It can be defined below that the detected and identified race car is the second race car 2. However, it should be noted that the identification unit 53 can also detect and identify two or more race cars 1, 2, 3 simultaneously, or each of the race cars 1, 2, 3 in the car race. The identified race car can then be virtualized so that it can be provided as a virtualized race car in the mixed-reality racing game. The second race car can be configured to provide telemetry data that allows it to be included in the mixed-reality racing game.

[0061] The identified second race car 2 can be linked to the second vehicle profile data 2' based on the identification of the second race car 2 and the second vehicle profile data 2'. The computer system 50 can be configured to generate different output video streams for different geographic areas 100, 200, 300 based on the broadcast requirements and the geolocation information of the broadcast requirements. For example, the computer system 50 and its content generation unit 54 can first be configured to select a second video content element 211, 212, 213 that is linked to the second vehicle profile data 2' of the identified second race car 2.The video content generation unit 54 can then further be configured to select the second video content element 211, 212, 213, which is linked to geolocation data that defines the geographic area 100, 200, 300, within which the geographic location of the user device is determined based on the broadcast request.

[0062] Accordingly, the content generation unit 54 can be configured to select video element 211 for the first broadcast request from the first user device based on the geographic location 103 of the first user device, which lies within the first geographic area 100. Similarly, the content generation unit 54 can be configured to select video element 212 for the second broadcast request from the second user device based on the fact that the geographic location 203 of the second user device lies within the second geographic area 200. Furthermore, the content generation unit 54 can be configured to select video element 213 for the third broadcast request from the third user device based on the fact that the geographic location 303 of the third user device lies within the third geographic area 200.It should be noted that when selecting video element 211, 212 and / or 213, the video element can be modified using one or more suitable software algorithms so that its appearance corresponds to the environmental conditions of the environment in which the car race takes place. For example, the appearance can reflect rainy or sunny conditions according to the weather and / or lighting conditions in the environment of the car race.

[0063] The computer system 50 and its video processing unit 55 can be configured to adapt the generated video element 211 to the identified second race car 2 in the input video stream in order to provide initial manipulated video data. The computer system 50 and its output unit 52 are configured to broadcast the initial manipulated video data as an initial output video stream from the computer system 50 to the first user device in response to the initial broadcast request. Thus, the first user device can be used to play the mixed-reality racing game, so that the game includes the initial manipulated video data.

[0064] Similarly, the computer system 50 and its video processing unit 55 can be configured to adapt the generated video element 212 to the identified second race car 2 in the input video stream to provide second manipulated video data. The computer system 50 and its output unit 52 are further configured to broadcast the second manipulated video data as a second output video stream from the computer system 50 to the second user device in response to the second broadcast request. Thus, the second user device can be used to play the mixed-reality racing game, so that the game incorporates the second manipulated video data.

[0065] Furthermore, the computer system 50 and its video processing unit 55 can be configured to adapt the generated video element 213 to the identified second race car 2 in the input video stream in order to provide third manipulated video data. The computer system 50 and its output unit 52 are further configured to broadcast the third manipulated video data as a third output video stream from the computer system 50 to the third user device in response to the third broadcast request. Thus, the third user device can be used to play the mixed-reality racing game, so that the game includes the third manipulated video data.

[0066] Adapting the generated video element to the detected and identified race car can be performed using an adaptation algorithm configured to adapt the generated video element to the race car based on the detection of the race car in the input video stream, or based on the output of the identification unit 53, or based on the output of the object detection algorithm. In some exemplary embodiments, the identification unit 53 or its object detection algorithm is configured to detect the boundary lines or surfaces of the race car in the input video stream.The adaptation of the generated video element to the captured and identified race car is then carried out using an adaptation algorithm that is set up to adapt the generated video element to the identified race car based on the boundary lines or surfaces of the race car captured by the identification unit 53 or the object capture algorithm.

[0067] In some exemplary embodiments, the adaptation of the generated video element to the captured and identified race car by the computer system 50 comprises providing a video element layer that includes the generated video element and combining the video element layer and the input video stream to adapt the generated video element to the race car, thus providing the manipulated video data. In some exemplary embodiments, the adaptation of the generated video element to the captured and identified race car by the computer system comprises splitting the input video stream into a race car layer and a background layer, wherein the race car layer includes the captured race car and the background layer includes image data outside of the captured race car.The adaptation further includes adjusting the generated video element to the captured race car in the race car layer and combining the background layer and the race car layer to provide the manipulated video data.

[0068] In some exemplary embodiments, the computer system's adaptation of the generated video element to the captured and identified race car involves splitting the input video stream into a first race car layer, a second race car layer, and a background layer. The first race car layer comprises the first captured race car, the second race car layer comprises the second captured race car, and the background layer comprises image data outside of the captured first and second race cars. The adaptation further includes adapting the first generated video element to the first captured race car in the first race car layer, adapting the second generated video element to the second captured race car in the second race car layer, and combining the background layer, the first race car layer, and the second race car layer to provide the manipulated video data.

[0069] The orientation of the race car can vary in the input video stream. Accordingly, the race car can be captured from different or varying viewpoints in the input video stream, since race cars 1, 2, 3 frequently move relative to the imaging device 400. Therefore, it is advantageous to capture the orientation of the race car in the video stream so that the generated video element can be adapted to the identified race car in a suitable orientation. In this application, the orientation of the race car can be understood as a viewpoint of race car 1, 2, 3 in the input video stream. Accordingly, the computer system 50 and its identification unit 53 or content generation unit 54 can be configured to capture the orientation of race car 1, 2, 3 in the input video stream.

[0070] In some exemplary embodiments, identifying the race car 1, 2, 3 in the input video stream in the identification unit 53 can include detecting the orientation of the race car 1, 2, 3 in the input video stream. Thus, identifying the race car 1, 2, 3 in the input video stream in the identification unit 53 can include providing the detection algorithm trained to detect the orientation of the race car in the input video stream and using the input video stream as input data into the object detection algorithm to detect the orientation of the race car in the input video stream. Detection of the orientation can be performed using the same or a separate object detection algorithm as the detection of the race car in the input video stream and / or the identification of the race car 1, 2, 3 in the input video stream.Alternatively, the identification unit 53 can include a separate object orientation detection algorithm. In some other exemplary embodiments, the generation of the video element in the content generation unit 54 includes detecting the orientation of the race car 1, 2, 3 in the input video stream.

[0071] Thus, generating the video element in content generation unit 54 can involve providing the orientation detection algorithm, which is trained to detect the orientation of the race car in the input video stream, and using the input video stream as input data into the orientation detection algorithm to detect the orientation of the race car in the input video stream. The generated video element can then be oriented according to the orientation of the race car. Accordingly, generating the video element in content generation unit 54 can involve calculating an orientation for the generated video element based on the detected orientation of the identified race car and generating an oriented video element based on this calculation.

[0072] In some exemplary embodiments, the generation of the video element in the content generation unit 54 includes calculating an orientation for the generated video element based on an output from the object detection algorithm or the orientation detection algorithm, and generating the aligned video element based on this calculation. Accordingly, the detected orientation of the race car can be used to calculate the orientation of the video element in order to provide the aligned video element. The orientation of the aligned video element can be set to match the orientation of the race car in the input video stream and be adapted to the identified race car in the input video stream to provide the manipulated video data. Therefore, the video element can be adapted to the same orientation in which race car 1, 2, 3 is detected.

[0073] The video element can be a separate video element 105, which is set up to be adapted to a part of the race car 1, 2, 3 or its outer surface, as shown in Fig. 15 is shown. Fig. Figure 16 presents an exemplary embodiment in which the video element 205 is a three-dimensional image element configured to correspond to the shape of the race car or a part thereof. For this purpose, the video element 205 can be configured to form part of the outer surface of the race car 1, 2, 3 in the output video stream. Accordingly, the video element 205 can be a three-dimensional vehicle model representing the race car, as shown in Figure 16. Fig. 16 is shown. Accordingly, there can be two or more three-dimensional vehicle models 205 than the video elements with different geolocation information.

[0074] Fig. Figure 17 presents a further exemplary embodiment in which the racing car database 56 and the vehicle profile data comprise the three-dimensional vehicle model 205, which represents the racing car. The content database also includes separate video elements. The three-dimensional vehicle model 205 is provided with an associated video element section 115, as shown in Figure 17. Fig. Figure 17 shows that in some embodiments, the identification in the identification unit 53 involves comparing the race car in the input video stream with the three-dimensional vehicle model 205 to identify the race car in the input video stream. Accordingly, the three-dimensional vehicle model can be used to identify the race car.

[0075] In some further exemplary embodiments, detecting the orientation of the identified race car in the input video stream involves determining the orientation of the race car based on the detected race car in the input video stream and the three-dimensional model 205 of the identified race car. Accordingly, the orientation of the three-dimensional model 205 can be adjusted so that its orientation matches the orientation of the race car in the input video stream. Thus, the three-dimensional model 205 can be adapted to the race car 1, 2, 3 in the input video stream by adjusting its orientation to match that of the race car 1, 2, 3 in the input video stream. Therefore, the three-dimensional vehicle model can be used to efficiently determine the orientation of the race car in the input video.

[0076] In some exemplary embodiments, the orientation of the generated video element can be calculated based on the specified three-dimensional vehicle model 205, and the three-dimensional model 205 can be adapted to the race car in the input video stream to provide the manipulated video data. Alternatively, the video element 105 can be adapted to the three-dimensional model 205. For example, the video element 105 can be adapted to the three-dimensional model 205 and the associated video element section 115 of the three-dimensional model 205. The orientation of the race car in the input video can be determined by adapting the three-dimensional vehicle model to the identified race car, and thus the orientation of the adapted three-dimensional vehicle model can represent the orientation of the race car in the input video.

[0077] The manipulated video data, as an output video stream, can be broadcast by the computer system 50 via the output unit 52 based on the broadcast request to the user device. The computer system can also use the manipulated video data to generate the mixed-reality content, which includes the virtual racing environment, corresponding to the racing environment of the captured input video stream, and the virtual race track, representing the real-world race track captured in the input video stream. The generated video element includes at least one race car, created based on the location data of the broadcast request received from the user device. The user device can then also be used by the user to provide the control data for the controllable virtual race car model, which is then made available in the mixed-reality racing game.

[0078] The user device can be configured to receive the broadcast output video stream. The user device can further be configured to display the output video stream as part of the mixed-reality game on a display or display device of the user device at the defined geographic location 103, 203, 303 of the user device 102, 202, 302. Accordingly, the generated output video with the video element is displayed at the geographic location of the user device, and the video element is specific to the geographic location of the user device.

[0079] It should be noted that the exemplary implementations discussed above can be combined and that the units discussed can be understood as logical units whose implementation can vary. The exemplary implementations discussed above can be used to provide a user experience in which the user can interact with a mixed-reality game that represents a real-world racing game, and the user can participate in the game by providing control data for a controllable virtual race car. In this way, the user can also experience a multiplayer game without necessarily having other players present at the time of the game.Furthermore, at least one of the real-world race cars in the mixed-reality game can be displayed in such a way that its visual appearance is adapted to the geographic location of the user's device. This enables targeted messaging, for example, for the geographic location of the user's device and / or for the user of the device. Targeted messaging can be used, for example, to ensure that the visual content is appropriate for that geographic location and / or that user.

Claims

[1] Computer system for a mixed-reality racing game, wherein the computer system comprises one or more computing devices and instructions which, when executed on at least one processor of the computer system, cause the computer system to perform a mixed-reality racing game, and wherein the computer system can be connected to a user device, wherein the computer system is configured for the following steps: Providing a virtual racing environment with a virtual race track, wherein the virtual race track is a virtual representation of a real race track; Providing a virtual race car model that represents a real race car; Providing a controllable virtual race car; Receiving telemetry data from the real racing car moving on the real race track, wherein the telemetry data includes at least position data and movement data of the real racing car, and matching the telemetry data of the real racing car to the virtual racing car model; Received from the user device, control data associated with the controllable virtual race car, and user data including geolocation information of the user device, the geolocation information defining a geographic location of the user device; Obtaining a video element that is associated with the geolocation information and profile of the real race car; Adapting the video element to the virtual race car model, which represents the real race car; Calculating the position and speed of the virtual race car model on the virtual race track based on telemetry data; Calculating the position and speed of the controllable virtual race car on the virtual race track based on the control data; Generating a visual representation of the mixed-reality racing game, comprising the virtual racing environment together with the controllable virtual race car on the virtual race track, wherein the visual representation is generated at least partially based on the determined position and speed of the virtual race car model and the determined position and speed of the controllable virtual race car. [2] Computer system according to claim 1, wherein the virtual racetrack includes positioning data configured to define positions on the virtual racetrack, wherein the positioning data includes real-world coordinates configured to map corresponding positions on the real-world racetrack to the defined positions on the virtual racetrack; or The virtual racetrack includes positioning data that defines positions on the virtual racetrack, wherein the positioning data includes virtual coordinates configured to represent positions on the virtual racetrack and real coordinates configured to represent positions on the real racetrack, wherein the positioning data is configured to map virtual coordinates to a corresponding real racetrack, and The virtual racetrack includes positioning data, which comprises spatial mapping data designed to define positions on the virtual racetrack and corresponding real-world positions on the real racetrack. [3] Computer system according to claim 1 or 2, wherein the real racing car comprises one or more sensors configured to detect parameters of the real racing car and to generate telemetry data based on the detected parameters, and wherein the detected parameters include at least one position and movement of the real racing car detected by the one or more sensors. [4] Computer system according to any one of claims 1 to 3, wherein the computer system is further configured for the following steps: - Receiving the telemetry data as continuous telemetry data from the actual race car; or - Receiving telemetry data as streaming telemetry data from the actual race car; or - Receiving telemetry data as continuous streaming telemetry data from the real race car. [5] Computer system according to any one of claims 1 to 4, wherein the computer system is further configured to receive the control data as continuous control data. [6] Computer system according to any one of claims 1 to 5, wherein the control data includes at least motion control data, wherein the motion control data defines a movement of the controllable virtual race car in the virtual racing environment. [7] Computer system according to any one of claims 1 to 6, wherein the telemetry data are received as real-time telemetry data, and the computer system is further configured to continuously calculate a real-time position and speed of the virtual race car model on the virtual race track based on the real-time telemetry data. [8] Computer system according to any one of claims 1 to 7, wherein the computer system is further configured to continuously generate a visual representation of the virtual environment, which simultaneously represents an instantaneous position and speed of the virtual racing car model and the controllable virtual racing car on the virtual race track based on the calculated position and speed of the virtual racing car model and the calculated position and speed of the controllable virtual racing car. [9] Computer system according to any one of claims 1 to 8, wherein the computer system is further configured to receive environmental measurement data from one or more environmental sensors provided in connection with the real race track, to calculate race track data using the environmental measurement data and to determine virtual race environment properties of the virtual race environment or virtual race track properties of the virtual race track based on the calculated race track data. [10] Computer system according to any one of claims 1 to 9, wherein the computer system is further configured for the following steps: Receiving an input video stream of a motor racing event taking place on the real race track, wherein the input video stream includes the real racing car, and wherein the virtual racing environment is provided based on, at least partially, the received input video stream; and Receiving, from the user device, a broadcast request for an output video stream of the motoring event, wherein the request includes the user data. [11] Computer system according to claim 10, wherein the computer system is further configured for the following steps: Identifying the real-world racing car in the input video stream, wherein identification includes defining profile data of the identified real-world racing car; Determine that the defined profile data corresponds to profile data representing the real race car stored in a race car database; and Broadcasting, in response to the request, the output video stream, which includes the virtual race car model with the adapted video element, where the virtual race car model represents the real race car. [12] Computer system according to any one of claims 10 to 11, wherein the computer system is further configured for the following steps: Capturing the orientation of the real race car in the input video stream; Calculating an orientation for the generated video element based on the captured orientation of the real race car; Creating an aligned video element by applying alignment to the video element; and Adjusting the aligned video element to the input video stream in order to provide the output video stream. [13] Computer system according to any one of claims 1 to 12, wherein the computer system is further configured to perform the following steps: Providing the video element as a unique non-fungible token; or Linking the video element to a unique non-fungible token; or Storing the video element with a unique non-fungible token on a blockchain. [14] Computer system according to any one of claims 1 to 13, wherein the computer system further comprises a real racing car comprising one or more sensors configured to detect parameters of the real racing car and to generate telemetry data based on the detected parameters, wherein the detected parameters include at least a position and movement of the real racing car detected by the one or more sensors. [15] Computer system according to any one of claims 1 to 14, wherein the computer system further comprises the user device and the user device comprises one or more input devices configured to generate control data in response to user input. [16] Computer system according to any one of claims 1 to 15, wherein - the computer system further comprises one or more environmental sensors, which are provided in conjunction with the actual race track and are configured to generate environmental measurement data, and - the computer system is further configured to receive environmental measurement data from the one or more environmental sensors, to calculate race track data using the environmental measurement data, and to determine virtual race environment properties of the virtual race environment or virtual race track properties of the virtual race track based on the calculated race track data. [17] System according to any one of claims 1 to 16, wherein - the computer system further comprises a display device designed to display the visual representation of the mixed-reality racing game; or - the user device includes a display device configured to display the visual representation of the mixed-reality racing game.

Images