Method and system for providing a virtual competition environment, motor vehicle with such a system

Abstract

Method for providing a virtual competition environment for a real participant of a virtual competition, wherein the virtual competition environment comprises a virtual replica of a real competition route, wherein the real participant first moves along the real competition route in a training run, wherein - during the training session, position data and / or orientation data of the real participant along the real competition route and movement dynamics data of the real participant along the real competition route are recorded by an environment sensor system (14) and transmitted as a time series to a computing device (16), wherein for each recorded data set of position and / or orientation data and / or movement dynamics data, a time stamp is recorded, which is transmitted with the respective data set to the computing device (16), so that the data sets can be sorted according to their chronological order and displayed as a simulation or simulation sequence consisting of successive frames, - the simulation is created as a virtual image of the real participant and his movement along the competition route by the computing device (16) on the basis of the position data and / or the orientation data and the movement dynamics data and is integrated into the virtual competition environment, and the virtual competition environment with the integrated simulation is transmitted to a display device (18), and wherein - the virtual competition environment with the simulation integrated therein is displayed for the real participant by the display device (18), while the real participant moves again along the real competition course in a competition round, wherein the display device (18) is designed as augmented reality glasses, wherein the virtual competition environment with the simulation integrated therein is displayed by the augmented reality glasses (18) as augmented reality display content for the real participant, wherein the augmented reality display content is displayed as display content superimposed on the real competition route, wherein the participant is the driver of a motor vehicle (10), wherein each frame of the simulation is always rendered with the current position and / or orientation of the motor vehicle (10) including the augmented reality glasses (18) during the competition run, wherein the current position and / or orientation of the motor vehicle (10) relative to the augmented reality glasses (18) is taken into account when displaying the simulation during the competition run, wherein, in order for the simulation to always be synchronous with the current temporal position and / or orientation of the motor vehicle (10) during the competition run, the motor vehicle (10) localizes itself on the competition track and the augmented reality glasses (18) localize themselves or are localized within the motor vehicle.

Description

[0001] The invention relates to a method and a system for providing a virtual competition environment for a driver of a motor vehicle as a participant in a virtual competition. Further aspects of the invention relate to a system for providing such a competition environment and a motor vehicle with such a system.

[0002] The use of virtual competition environments is known, for example, from the training of racing drivers, whether in motorsports, cycling, skiing, or similar. In such a virtual competition environment, for example, a virtual competition track can be provided along which the athletes or racing drivers can move virtually in a racing simulator. Such virtual or simulated training offers a wide range of opportunities to improve one's own driving style. Such virtual competition environments are also known from the field of entertainment electronics, for example in the computer games industry. For example, computer games often offer a "ghost mode" which, in racing games, offers the option of competing against "one's old self," i.e. against a previous recording of a game play as a "shadow vehicle" (hence the name "ghost mode").The aim is to improve your own driving style or to carry out training based on recorded data in the form of a shadow vehicle.

[0003] DE 100 31 026 A1 discloses a tracking system for racing cars. Tracking data is collected from this data, from which a racing car's position along a racetrack can be derived. The tracking data, together with the racing cars' operating data, can also form the boundary conditions for a computer simulation game. In the simulation game, a course can then be driven against comparison vehicles.

[0004] A disadvantage of the applications described so far is that they are only available virtually. In other words, there is no way to implement a "ghost mode" in real life, for example, in a physical racing car and use it on a real racetrack.

[0005] DE 10 2016 216 601 A1 discloses a method for assisting a driver of a motor vehicle when driving on a racetrack. During a first drive along the racetrack, the position of the motor vehicle is continuously recorded, and during a subsequent drive along the racetrack, a driving line determined from the recorded position is displayed to the driver in the motor vehicle. During the subsequent drive, the driver is thus shown his previous driving history for those sections of the route he is driving through again, allowing him to compare whether he is driving along the same driving line as before. A local comparison is thus provided. Unfortunately, the driver is unable to see how his driving behavior differs from the first drive when driving along the racetrack again.

[0006] EP 4 242 035 A1 describes a method and a device for conducting a virtual car race. In this case, virtual images of other vehicles competing with the driver of a real vehicle are displayed on a display in the vehicle, for example, on the windshield, a head-up display, or a portable visor. Due to the display options, the method described here does not provide a truly immersive racing experience for the driver of the real vehicle.

[0007] DE 10 2018 007 668 A1 also describes a method and device for conducting virtual vehicle races between a user and, for example, professional racing drivers. Here, the user is shown a 360-degree video in which they can see their own vehicle in the form of a "ghost vehicle" driving along the race track. The 360-degree video can also display another race participant, for example, a virtual representation of the professional racing driver. Here, too, the 360-degree video representation is not sufficiently immersive to evoke a true racing sensation in the user.

[0008] The invention is based on the object of providing an improved virtual competition environment for a participant in a virtual competition. In particular, the temporal difference between a competition round and a training round should also be made tangible for the participant.

[0009] The problem is solved by the subject matter of the independent patent claims. Advantageous developments of the invention are described by the dependent patent claims, the following description, and the figures.

[0010] The invention provides a method for providing a virtual competition environment for a real participant in a virtual competition, wherein the participant is a driver of a motor vehicle. The virtual competition environment comprises a virtual replica of a real competition course. The competition course can be, for example, a race track for car races or motorcycle races. However, the competition course can also be a racing course in general, and in particular also a cycling course, a marathon course, a cross-country skiing course, a downhill course in alpine skiing, a regatta course or even a swimming course. The participant therefore moves along the competition course in or with a motor vehicle or racing car or drives along it.

[0011] The method according to the invention initially comprises a training session, during which the real participant moves along the real competition track. To continue with the example of a car race as a competition, it can therefore be provided that the participant initially physically drives the competition track in the training session as the driver of a real racing car.

[0012] According to the invention, during the training session, an environmental sensor system records position data and / or orientation data of the real participant along the real competition route and movement dynamics data of the real participant along the real competition route. In other words, the participant's journey along the competition route is recorded or logged.

[0013] For this purpose, the environmental sensor system can comprise one or more sensors that continuously detect the position and / or orientation of the participant along the competition route. In the context of the present invention, orientation refers in particular to the orientation of the participant, preferably of the participant's motor vehicle. For example, the orientation of a longitudinal axis of the motor vehicle relative to the competition route or relative to predetermined reference points along the competition route can be determined. The sensors can be configured, for example, as GPS sensors or DGPS sensors (DGPS - differential GPS), which continuously detect the participant's position based on data from a satellite-based positioning system, in particular including possible correction data.Alternatively or additionally, further sensors may be provided for position detection and / or orientation detection, for example camera sensors and / or radar sensors and / or lidar sensors and / or ultrasonic sensors and / or sensors from the field of near-field communication, for example Bluetooth sensors and / or UWB sensors (UWB - ultra wide band).

[0014] Furthermore, the environmental sensor system can include driving dynamics sensors for recording the participant's movement dynamics data. The movement dynamics or driving dynamics sensors can, for example, be designed to record a speed and / or acceleration and / or deceleration and / or steering interventions of the participant or a participant's vehicle. The driving dynamics sensors can also record external forces acting on the participant or the vehicle, for example, centrifugal forces that can act on the participant or the vehicle when negotiating a curve along the competition course. An inertial measurement unit (IMU for short), which is usually already part of the vehicle, can also be used to determine acceleration, angular velocity, and orientation data.

[0015] The environmental sensor system can be designed as part of the racing car and include on-board sensors. However, the environmental sensor system can also be designed entirely or partially outside the racing car and, for example, include sensors integrated into a road surface of the competition track. Alternatively or additionally, the environmental sensor system can include light barriers that register the passage of the racing car or motor vehicle, based on which lap times or speeds of the motor vehicle can be determined.

[0016] According to the invention, the acquired position and / or orientation data and the acquired movement dynamics data are transmitted to a computing device for further processing. For this purpose, a communication unit of the environment sensor system can, for example, transmit the acquired data as data packets or data sets to the computing device. The transmission can, for example, be wireless, in particular internet-based via a WLAN network or via a mobile network. However, the communication unit can also operate a wired communication connection to the computing device.

[0017] The computing device can comprise a data processing apparatus and can be embodied, for example, as an internet-based cloud server. However, the computing device can also be embodied locally, for example, as part of a control device of the participant's sports equipment. Since the participant is a driver of a motor vehicle, the computing device can also be part of a control device of the motor vehicle. If the participant is a marathon runner, the computing device can be, for example, a processor device arranged in a wearable of the runner. Such a wearable can be, for example, a smartwatch of the runner. Such a smartwatch can also be wirelessly connected to the internet and thus operate a connection to an internet-based cloud server of the type described above.

[0018] The position and / or orientation data and the movement dynamics data are transmitted to the computing device as a time series. This means that a timestamp is recorded for each acquired data set of position and / or orientation data and / or movement dynamics data, which is transmitted to the computing device with the respective data set. This allows the data sets to be sorted according to their chronological order and displayed as a simulation or simulation sequence consisting of consecutive frames.

[0019] According to the invention, the computing device creates a virtual image of the real participant and their movement along the competition course based on the position data and / or the orientation data and the movement dynamics data. In other words, the computing device creates a simulation based on the position data and / or the orientation data and the movement dynamics data, in which an avatar or vehicle of the participant moves along the competition course. This simulation, i.e. the virtual image of the real participant and their movement, is then integrated into the virtual competition environment by the computing device. For this purpose, the computing device can be operated with a simulation environment (e.g., Unity) in which the real competition course is recreated in the virtual competition environment.The recorded time series of the acquired position data and / or orientation data and movement dynamics data is fed into this simulation environment. This results in the simulation with the recorded data. In the case of the racing driver, this results in a recorded race within the simulation environment, preferably in 3D.

[0020] The simulation, i.e., the virtual competition environment with the integrated virtual image of the real participant and their movements, is then transmitted from the computing device to a display device. The computing device can therefore comprise a suitable communication interface for transmitting the simulation to the display device. This is preferably a wireless interface for wirelessly transmitting or transmitting the simulation to the display device.

[0021] The display device then presents the simulation to the real participant, while the real participant moves along the real competition course again in a competition round. For this purpose, the display device can have one or more screen units, which can display the simulation, for example, as pixel-based display content. In other words, the participant sees on the display device or screen units during the competition round how they themselves moved along the competition course during the training round. The participant therefore not only sees their movement track along the competition course recorded during the training round, but also sees their avatar moving along the movement track, in a realistic temporal reproduction.This creates the experience for the participant during the real-life competition ride of following their "old self" along the competition course or of moving along the competition course together with their "old self." The participant can be overtaken by their avatar or overtake them.

[0022] In particular, in the case according to the invention where the participant is the driver of a motor vehicle, the simulation is not simply played in the motor vehicle. Rather, each frame of the simulation is always rendered with the current position and / or orientation of the motor vehicle, including the display device, during the competition run. In other words, the view of the simulation is always calculated from the motor vehicle or from the display device in the motor vehicle. This means that the current position and / or orientation of the motor vehicle relative to the display device is taken into account when displaying the simulation during the competition run.To ensure that the simulation is always synchronized with the current temporal position and / or orientation of the vehicle during the competition, the vehicle itself and the display device within the vehicle, such as AR (augmented reality) glasses, must localize. This ensures that the position and / or orientation are correctly represented within the simulation.

[0023] The invention provides that the virtual competition environment, with the integrated virtual image of the real participant and their movement—i.e., the simulation described above—is presented by the display device as augmented reality display content for the real participant. The augmented reality display content is presented as a display content superimposed on the real competition course. In other words, the participant can see the real competition course through the display content. This can be achieved by means of a head-up display of the display device, through which the display content is presented while the participant is driving along the competition course during the competition.

[0024] According to the invention, the display device is designed as augmented reality glasses or AR glasses. In other words, the simulation is displayed on AR glasses and thus visualized for the participant. This results in a particularly immersive experience for the participant. To ensure that the simulation is always synchronized with the current temporal position and / or orientation of the vehicle during the competition, the vehicle must localize itself and, within the vehicle, the AR glasses. This ensures that the position and / or orientation within the simulation is accurately reproduced.

[0025] The simulation preferably starts at the moment the participant crosses a starting line on the competition course at the beginning of the competition round. The crossing can be detected, for example, using one of the light barriers described above. Detection using sensors integrated into a surface of the competition course, which can operate inductively, for example, is also conceivable. Alternatively, the simulation can also be started manually.

[0026] The participant, i.e., a racing driver, can race against himself in real time. To ensure the simulation is always synchronized with the participant's current position in time, the participant, i.e., the racing car, must locate itself along the race track and the AR glasses within the racing car.

[0027] The integration of the AR glasses in the vehicle is designed as follows: The vehicle locates itself (globally) on the racetrack, for example, using GPS or high-precision DGPS (real-time kinematics, RTK). The AR glasses locate themselves within the vehicle. This can be achieved using the well-known "inside-out tracking" or "outside-in tracking" methods, whereby the AR glasses locate themselves within the vehicle, or the AR glasses are located within the vehicle, for example, using a camera and / or an infrared camera and / or a TOF camera (time of flight).

[0028] The invention provides the advantage of providing an improved virtual competition environment for a participant in a virtual competition. In particular, the invention also allows the participant to experience the time difference between a competition round and a training round.

[0029] The invention also includes embodiments which provide additional advantages.

[0030] Preferably, the augmented reality display content is adapted during the competition run to a current pose of the real participant along the real competition course, whereby the current pose comprises a current position of the real participant along the competition course and a current orientation of the real participant at the current position. For this purpose, the position and / or orientation of the participant must be calculated, including the position and / or orientation of the participant's AR glasses, for example in the racing car. This summed position and / or the summed orientation (from the vehicle and AR glasses) can then be incorporated into the simulation in real time, whereby the virtual orientation of the vehicle or the AR glasses can be calculated in the vehicle. In other words, the view within the simulation can be adjusted based on the correct position and / or orientation of the real vehicle or the AR glasses.

[0031] In a specific example, the computing device described above can be located in a motor vehicle, for example, as part of a racing car's control device. The simulation can be transferred or flashed to the computing device. The localization of the racing car and the AR glasses located in the racing car can then be summed within the computing device. Based on this localization, the position / orientation can be calculated within the simulation. The simulation (e.g., Unity) can then be streamed / transferred to the AR glasses. This variant has the advantage that the computing capacity of the motor vehicle's control device can be used, rather than the limited computing capacity of the AR glasses.

[0032] However, the computing device can also be implemented as the computing device of the AR glasses. In other words, the simulation can be transferred directly to a computing device of the AR glasses. The summation of the localization of the vehicle and the AR glasses can then be performed directly on the AR glasses. The position and / or orientation in the simulation is calculated on the AR glasses, with the output (stream) subsequently being provided on the AR glasses.

[0033] The two variants differ in that in one case, the simulation is run on a computer in the vehicle, and the AR glasses are used solely for the output (stream). In the other case, the entire simulation, including the output (stream), is run directly on the AR glasses.

[0034] Alternatively or additionally, the simulation can be run directly by a computing device in a network (cloud), with the network regularly receiving the location of the vehicle and the AR glasses. From this, the correct display within the simulation can be calculated and then streamed directly to the AR glasses in the vehicle or to a receiving unit in the vehicle, which then forwards the stream to the AR glasses. The transmission could be carried out, for example, via a mobile network, particularly using 5G transmission technology.

[0035] A further embodiment provides for the virtual competition environment to be enhanced with additional track data, wherein the additional track data is displayed on the display device during the competition run. In other words, the recorded training run can be edited. Here, for example, a racing trainer or coach can analyze the recorded training run and incorporate the recorded movement dynamics parameters into the recording. In other words, braking points, acceleration points, steering interventions and / or steering points can be inserted into the simulation. The simulation can also include an audio track, wherein the racing trainer can provide a commentary or driving recommendation for certain sections of the training run, wherein the recorded text can be played back in the simulation when the participant approaches or drives through the sections.Alternatively or additionally, a difference to an ideal line, i.e., an optimal driving line or movement path for the competition course, can be calculated and incorporated into the simulation. In other words, through processing, for example offline, a concrete improvement potential for the recorded data (or the course driven) can be incorporated.

[0036] A further embodiment provides that the virtual competition environment includes virtual images of other virtual participants and their movement along the competition course, which are also displayed by the display device. This creates the illusion for the participant that they are moving along the competition course with the other participants.

[0037] According to a preferred development, a virtual contact between the real participant and at least one of the other virtual participants is detected and converted into a haptic signal that the real participant can experience. For this purpose, the computing device can, for example, detect a contact or an overlap of display areas of the virtual participants in the display device based on image data analysis and generate a corresponding signal for an actuator. The actuator can receive this signal and act on the real participant according to the signal. If the participant moves along the competition course with / in a piece of sports equipment, the actuator can be designed as part of the sports equipment. For example, it is conceivable that the actuator is designed as part of the sole of a running shoe and can cause this to vibrate depending on the signal.Alternatively or additionally, the actuator can be designed as part of a wearable, e.g. a participant's smartwatch, and generate a vibration there depending on the signal.

[0038] In the case according to the invention that the participant is the driver of a motor vehicle (here and hereinafter also referred to as an EGO vehicle), for example a racing car, the haptic experience of such a virtual contact or collision can be created by detecting the collision as an event in the course of the virtual competition. The computing device can then be designed to generate a signal describing the course of the virtual competition and to transmit it to the control device of the motor vehicle. The control device can be designed to control a drive unit and / or a braking device and / or a longitudinal guidance system and / or a lateral guidance system of the motor vehicle depending on the signal. In this way, a "vehicle bump," which frequently occurs in a race, can be simulated in a virtual representation with multiple vehicles (virtual racing cars).

[0039] This means that if a collision occurs between the virtual vehicles (EGO vehicle and racing vehicles) during the live simulation ("playing back the recording"), a (simplified) crash calculation or momentum conservation calculation is carried out. The output of this calculation is essentially a resulting total force (magnitude and direction) on the EGO vehicle, which is divided into a longitudinal and lateral acceleration acting on the EGO vehicle. The longitudinal acceleration should cause the EGO vehicle to decelerate, and the lateral acceleration a (moderate) steering impulse / steering angle. The longitudinal acceleration can also result in acceleration, for example, if the virtual contact consists of another virtual participant virtually driving into the EGO vehicle.

[0040] Braking, acceleration, and steering angle are fed into the EGO vehicle during the live simulation of a collision, resulting in a real physical braking, acceleration, and steering movement of the EGO vehicle. Simply put, in the case of a virtual collision (EGO vehicle and racing cars), a real collision (or "vehicle collision") is simulated in the EGO vehicle during the live simulation.

[0041] A further aspect of the invention relates to a system for providing a virtual competition environment for a real participant in a virtual competition, comprising an environment sensor system, a computing device and a display device.

[0042] A further aspect of the invention relates to a motor vehicle with such a system.

[0043] A further development of the motor vehicle provides that the computing device is configured to transmit a signal describing the course of the virtual competition, for example, a signal describing a collision with other virtual participants in the competition, to a control device of the motor vehicle, wherein the control device is configured to control a drive unit and / or a braking device and / or a longitudinal guidance system and / or a lateral guidance system of the motor vehicle depending on the signal. In other words, a vehicle movement or vehicle dynamics is influenced depending on the signal, whereby the course of the competition can be experienced by the driver of the motor vehicle.

[0044] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0045] The invention also includes the control device for the motor vehicle. The control device can have a data processing device or a processor device that is configured to carry out an embodiment of the method according to the invention. For this purpose, the processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor device can have program code that is configured to carry out the embodiment of the method according to the invention when executed by the processor device. The program code can be stored in a data memory of the processor device.The processor device can be based, for example, on at least one circuit board and / or on at least one SoC (System on Chip).

[0046] The invention also includes further developments of the system according to the invention and / or the motor vehicle according to the invention that have features already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments of the system according to the invention and / or the motor vehicle according to the invention are not described again here.

[0047] The motor vehicle according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle.

[0048] As a further solution, the invention also encompasses a computer-readable storage medium comprising program code which, when executed by a computer or computer network, causes the computer to carry out an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data memory (e.g. as a flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data memory (e.g. as a RAM - random access memory). The storage medium can be arranged in the computer or computer network. However, the storage medium can also be operated on the Internet, for example, as a so-called app store server and / or cloud server. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.

[0049] A further aspect of the invention relates to the program code, which can be provided as binary code and / or as assembly code and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).

[0050] The invention also encompasses combinations of the features of the described embodiments. The invention therefore also encompasses implementations that each comprise a combination of the features of several of the described embodiments, unless the embodiments are described as mutually exclusive.

[0051] Exemplary embodiments of the invention are described below. Shown are: Fig. 1 is a schematic representation of a motor vehicle with a system for providing a virtual competition environment according to a first embodiment of the invention; Fig. 2 is a schematic representation of a motor vehicle with a system for providing a virtual competition environment according to another embodiment of the invention; Fig. 3 a schematic representation of a method for providing a virtual competition environment according to an embodiment of the invention.

[0052] The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that can be considered independently of one another, each of which also develops the invention independently of one another. Therefore, the disclosure is intended to encompass combinations of the features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0053] In the figures, the same reference symbols designate elements with the same function.

[0054] Fig. 1 shows a schematic representation of a motor vehicle 10 with a system 12 for providing a virtual competition environment. The system 12 shown as an example comprises an environmental sensor system 14, a computing device 16, and a display device 18. In the system 12 shown, the computing device 16 is designed as part of the display device 18, i.e., the simulation can be transferred directly to the computing device 16 of the display device 18, which can, for example, comprise one on the AR glasses (e.g., possible with Microsoft HoloLens). The summation of the localization of the motor vehicle 10 and the AR glasses can therefore be carried out directly on the AR glasses. The position / orientation in the simulation can also be calculated on the AR glasses, and the output (stream) can then be carried out on the AR glasses.

[0055] Fig. Figure 2 shows a further schematic representation of a motor vehicle 10 with a system 12 for providing a virtual competition environment. In the system 12 shown here as an example, which also includes an environment sensor system 14, a computing device 16, and a display device 18, the computing device 16 is designed as part of a control device of the motor vehicle 10. The computing device 16 is therefore not integrated into the display device 18, as in connection with Fig. 1, but in the control device of the motor vehicle 10.

[0056] The computing device 16 according to the Fig. The example shown in Figure 2 is therefore located in the motor vehicle 10, and can therefore be designed, for example, as part of a control device of a racing car 10. The simulation can be transferred or flashed to the computing device 16. In other words, a hard copy of the simulation is transferred to the motor vehicle 10. The localization of the racing car 10 and the AR glasses located in the racing car 10 can then be summed within the computing device 16. Based on this localization, the position / orientation within the simulation can be calculated. The simulation (e.g., Unity) can then be streamed / transferred to the AR glasses. This variant has the advantage that the computing capacity of the control device of the motor vehicle 10 can be used, and the limited computing capacity of the AR glasses or the display device 18 does not have to be used.

[0057] Fig.3 shows a schematic representation of a method for providing a virtual competition environment.

[0058] First, a participant drives a competition course, for example, in a motor vehicle 10 during a training session. In a step S1, vehicle data, such as position data and / or driving dynamics parameters (such as speed, acceleration / deceleration, and steering interventions) of the motor vehicle 10 are recorded or captured using sensors of an environment sensor system 14.

[0059] In step S2, the vehicle data is transmitted as a time series to a network / computing unit (cloud) or a computing device 16. In the network, the computing device 16 is operated with a simulation environment (e.g., Unity) in which the physical environment (race track) is recreated. The recorded time series of the recorded vehicle data is thus fed into the simulation environment in step S2. This results in a simulation with the recorded vehicle data, essentially a recorded 3D race within the simulation environment.

[0060] In a further, optional step S3, the recorded race can now be edited, if necessary. A racing trainer / coach can analyze the recorded race and incorporate recorded driving dynamics parameters into the recording. This means that braking points, acceleration points, or steering interventions / steering points can be plotted / visualized. In this step S3, a difference to an ideal line (optimal driving line for the competition track) can also be calculated and plotted / visualized in the simulation / 3D representation. In other words, through offline editing, concrete potential for improvement to the recorded data (or the course driven) can be incorporated into the simulation.

[0061] In a step S4, the recording (within the simulation) can be transferred or flashed to the motor vehicle 10, e.g., as a time series and with features such as plotted driving dynamics parameters and / or suggested improvements. This means that a hard copy is transferred to the motor vehicle 10. The transfer can occur, for example, when the motor vehicle 10 crosses the starting line of the competition course in order to drive along it in a competition run.

[0062] The simulation (e.g., in Unity) is then run in the motor vehicle 10 using the previous recording, possibly supplemented by the recorded driving dynamics parameters and / or suggestions for improvement from offline processing (step S5). The simulation can be displayed on an AR device, such as AR glasses (e.g., Microsoft HoloLens), and thus visualized for the driver.

[0063] In concrete terms, for example, the motor vehicle 10 starts the simulation by crossing the starting line of the race track and thus receives the recorded previous vehicle data (including any processing, if applicable). To ensure that this data is always synchronized with the current temporal position of the motor vehicle 10, the motor vehicle 10 must localize itself and, within the motor vehicle 10, the AR glasses. This ensures that the position is accurately reproduced within the simulation.

[0064] According to a concrete example, the live simulation sequence can be designed as follows. To start the competition run, the motor vehicle 10 crosses the starting line of the competition course, thereby starting the simulation. Alternatively, the simulation can also be started manually. Starting the simulation means that the display or playback of the simulation or the recording of the real competition environment begins with the movement of the "old self" along the competition course. As soon as the simulation is played back by the display device 18, for example, the AR glasses, the current position / orientation of the motor vehicle 10 is continuously calculated, including the position / orientation of the AR glasses or a display of the display device 18 in the motor vehicle 10.This summed position and orientation (from motor vehicle 10 and AR glasses) then flows into the live simulation and serves as the basis for calculating the virtual orientation of motor vehicle 10 or the AR glasses in motor vehicle 10. Based on the correct orientation of the real motor vehicle 10 or the AR glasses in the simulation, the view within the simulation is transferred to the AR glasses. In other words, the rendering of the simulation (i.e., the AR display) always occurs relative to the current pose of motor vehicle 10 and display device 18 in motor vehicle 10 during the competition run. The rendering preferably also takes into account the pose of the camera used to record the training run—in other words, relative to the view in the simulation.

[0065] In other words, a method is provided for providing a virtual competition environment for a real participant in a virtual competition, wherein the virtual competition environment comprises a virtual replica of a real competition course, wherein the real participant initially moves along the real competition course in a training session. In the method, an environmental sensor system 14 records position and / or orientation data and movement dynamics data of the participant along the competition course during the training session and transmits them as a time series to a computing device 16. This computing device creates a virtual image of the participant and their movement along the competition course based on the position and / or orientation data and the movement dynamics data and integrates this into the virtual competition environment.This simulation is then transmitted to a display device 18 and displayed by it based on the current position and orientation of the real participant as he moves again along the real competition course in a competition run.

[0066] The simulation is not simply played back; rather, the simulation is played back or rendered based on the position and orientation of the vehicle 10 or the display device 18 (display or AR glasses). In the simulation environment, for example, Unity, the "camera" or the view of the simulation must always be rendered based on the position and orientation of the vehicle 10 or the display device 18.

[0067] Overall, the examples show how an augmented reality “ghost mode” can be provided directly in the (racing) vehicle for racing training or for AR participation in recorded racing events.

Claims

[1] Method for providing a virtual competition environment for a real participant of a virtual competition, wherein the virtual competition environment comprises a virtual replica of a real competition route, wherein the real participant first moves along the real competition route in a training run, wherein - during the training session, position data and / or orientation data of the real participant along the real competition route and movement dynamics data of the real participant along the real competition route are recorded by an environment sensor system (14) and transmitted as a time series to a computing device (16), wherein for each recorded data set of position and / or orientation data and / or movement dynamics data, a time stamp is recorded, which is transmitted with the respective data set to the computing device (16), so that the data sets can be sorted according to their chronological order and displayed as a simulation or simulation sequence consisting of successive frames, - the simulation is created as a virtual image of the real participant and his movement along the competition route by the computing device (16) on the basis of the position data and / or the orientation data and the movement dynamics data and is integrated into the virtual competition environment, and the virtual competition environment with the integrated simulation is transmitted to a display device (18), and wherein - the virtual competition environment with the simulation integrated therein is displayed for the real participant by the display device (18), while the real participant moves again along the real competition course in a competition round, wherein the display device (18) is designed as augmented reality glasses, wherein the virtual competition environment with the simulation integrated therein is displayed by the augmented reality glasses (18) as augmented reality display content for the real participant, wherein the augmented reality display content is displayed as display content superimposed on the real competition route, wherein the participant is the driver of a motor vehicle (10), wherein each frame of the simulation is always rendered with the current position and / or orientation of the motor vehicle (10) including the augmented reality glasses (18) during the competition run, wherein the current position and / or orientation of the motor vehicle (10) relative to the augmented reality glasses (18) is taken into account when displaying the simulation during the competition run, wherein, in order for the simulation to always be synchronous with the current temporal position and / or orientation of the motor vehicle (10) during the competition run, the motor vehicle (10) localizes itself on the competition track and the augmented reality glasses (18) localize themselves or are localized within the motor vehicle. [2] The method of claim 1, wherein the augmented reality display content is adapted to a current pose of the real participant along the real competition course during the competition run. [3] The method of claim 2, wherein the current pose comprises a current position of the real participant along the competition course and a current orientation of the real participant at the current position. [4] Method according to one of the preceding claims, wherein the virtual competition environment is enriched by additional route data, wherein the additional route data is displayed during the competition run through the augmented reality glasses (18). [5] Method according to one of the preceding claims, wherein the virtual competition environment comprises virtual images of other virtual participants and their movement along the competition route, which are also displayed by the augmented reality glasses (18). [6] Method according to claim 5, wherein a virtual contact of the real participant with at least one of the further virtual participants is detected and converted into a haptically perceivable signal for the real participant. [7] System (12) for providing a virtual competition environment for a real participant of a virtual competition according to a method according to one of the preceding claims, comprising an environment sensor system (14), a computing device (16) and augmented reality glasses (18). [8] Motor vehicle (10) with a system (12) according to claim 7. [9] Motor vehicle (10) according to claim 8, wherein the computing device (16) is designed to transmit a signal describing a course of the virtual competition to a control device of the motor vehicle (10), wherein the control device is designed to control a drive unit and / or a braking device and / or a longitudinal guidance system and / or a lateral guidance system of the motor vehicle (10) in dependence on the signal.

Images