Racing Simulation

Abstract

To combine automobile races in the real world and the virtual world.SOLUTION: A method for displaying a virtual vehicle includes steps of identifying a position of a physical vehicle in a race field, identifying a position of a viewpoint in the race field, and providing a part of a virtual vehicle seen from a virtual position of a viewpoint. The method is operated by calculating a virtual position within the virtual world on the basis of the position of the viewpoint. A system for displaying a virtual vehicle includes: a first sensor for detecting a position of a physical vehicle in a race field; a second sensor for detecting a position of a viewpoint in the race field; and a simulation system for providing a part of a virtual vehicle seen from a virtual position of a viewpoint. The simulation system is configured so as to calculate the virtual position of the viewpoint within the virtual world on the basis of the position of the viewpoint.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 62 / 530,037, filed July 7, 2017, entitled "Racing Simulation," which claims the benefit of U.S. Nonprovisional Patent Application No. 15 / 813,662, filed November 15, 2017, each of which is incorporated herein by reference in its entirety.

[0002] Field FIELD OF THE DISCLOSURE This disclosure relates generally to vehicle simulation, and more specifically to combining real-world and virtual-world auto racing. [Background technology]

[0003] background Almost every automobile has a racing competition. The media is full of car racing, bike racing, and truck racing on race tracks. Each race produces a champion, and each racing series produces a season champion.

[0004] Auto racing is not limited to the real world. In recent years, virtual auto racing has become increasingly popular. Virtual race champions can win hundreds of thousands of dollars per race. Seasonal virtual champions can win millions of dollars. Summary of the Invention

[0005] overview In some aspects, a method for displaying a virtual vehicle includes identifying a location of a physical vehicle at a racetrack, identifying a location of a viewpoint at the racetrack, and providing a display system with a portion of the virtual vehicle visible from a virtual position of the viewpoint calculated within the virtual world based on the location of the viewpoint at the racetrack. As an exemplary advantage, the aspects described herein may enable a physical vehicle operator to compete against a virtual vehicle operator. Furthermore, by displaying a representation of the portion of the virtual vehicle visible from the virtual position of the viewpoint, a race between the physical vehicle operator and the virtual vehicle operator may be more realistically depicted.

[0006] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes the portion of the virtual vehicle that is not obscured, as viewed from the virtual position of the viewpoint, by a representation of the physical vehicle at the virtual position of the physical vehicle in the virtual world.

[0007] In some aspects, the method further includes a step in which the simulation system simulates an interaction between the virtual vehicle and a representation of the physical vehicle in the virtual world, and a portion of the virtual vehicle visible from a virtual position of the viewpoint is calculated based on the interaction.

[0008] In some aspects, the position of the viewpoint at the racetrack includes a viewpoint of an operator of the physical vehicle, and identifying the position of the viewpoint at the racetrack includes detecting a viewpoint of the operator of the physical vehicle with a sensor, and the method further includes identifying a position of the physical object, receiving kinematic information of the virtual vehicle, generating, at a display system, a representation of the virtual vehicle based on the positions of the physical objects, the position of the viewpoint at the racetrack, and the kinematic information, and displaying the representation of the virtual vehicle such that the virtual vehicle is aligned with the physical object as viewed from the position of the viewpoint at the racetrack.

[0009] In some aspects, the method further includes generating, on a display system, a representation of the portion of the virtual vehicle that is visible from the virtual position of view.

[0010] In some aspects, the method further includes the display system displaying a series of representations of the virtual vehicle over a period of time to simulate the virtual vehicle's trajectory on a racetrack, the series of representations including representations of portions of the virtual vehicle as seen from a virtual position of view. In some aspects, a predicted trajectory of the virtual vehicle is displayed. The prediction may be based on a current trajectory, acceleration, current vehicle parameters, etc. This may allow an audience member to predict whether the virtual vehicle is likely to overtake the physical vehicle. The predicted trajectory may be presented as a line, such as a yellow line. Other indications may also be included, such as "About to overtake!" or "About to crash!".

[0011] In some aspects, the method further includes the display system storing a digital 3D model of the virtual vehicle used to generate each representation from the series of representations, each representation being generated by the display system based on the digital 3D model.

[0012] In some aspects, the method further includes receiving a digital 3D model of the virtual vehicle used to generate each representation from the series of representations, each representation being generated by the display system based on the digital 3D model.

[0013] In some embodiments, the kinematic information includes one or more vectors of motion, one or more scalars of motion, a position vector, a GPS position, velocity, acceleration, orientation, or a combination thereof of the virtual vehicle.

[0014] In some embodiments, identifying the position of the physical vehicle includes detecting one or more vectors of motion, one or more scalars of motion, a position vector, a GPS position, a velocity, an acceleration, an orientation, or a combination thereof, of the virtual vehicle.

[0015] In some aspects, identifying a position of the viewpoint on the racetrack includes detecting a spatial position of a head of an operator of the physical vehicle.

[0016] In some aspects, the method further includes a telemetry system coupled to the physical vehicle sending the spatial position to the simulator system, receiving information at the telemetry system related to a portion of the virtual vehicle visible from the virtual position of the viewpoint, and displaying a representation of the portion of the virtual vehicle to an operator of the physical vehicle based on the information.

[0017] In some aspects, the method further includes: displaying a representation of the portion of the virtual vehicle includes converting information into a set of graphical elements; and displaying the representation of the portion displays the set of graphical elements. In some aspects, the method further includes calculating, at the simulation system, information related to the portion that is visible from the virtual position of view.

[0018] In some embodiments, the step of displaying the sequence of representations of the virtual vehicle includes displaying the sequence of representations on a display of the physical vehicle, the display being a transparent organic light emitting diode (T-OLED) display that passes light through a T-OLED to display a field of view to an operator.

[0019] In some aspects, displaying the sequence of representations of the virtual vehicle includes displaying the sequence of representations on a display of a physical vehicle, the display being an LCD display, and the method further includes a camera coupled to the physical vehicle capturing an image representing a field of view of the physical world as seen by an operator on a display within the physical vehicle, and outputting the image to one side of the LCD display for displaying the field of view to the operator, the sequence of representations being overlaid on the image displayed by the LCD display.

[0020] In some aspects, the step of displaying the sequence of representations of the virtual vehicle includes displaying the sequence of representations on a display of a physical vehicle, the display including a windshield of the physical vehicle, one or more side windows of the physical vehicle, a rear window of the physical vehicle, one or more side mirrors, a rearview mirror, or a combination thereof.

[0021] In some aspects, displaying the sequence of representations of the virtual vehicle includes displaying the sequence of representations on a display of a headset worn by the operator. In some aspects, the headset is a helmet.

[0022] In some embodiments, identifying the location of the viewpoint at the racetrack includes detecting one or more of the spatial position of the user's eyes, the gaze direction of the user's eyes, or the focus of the user's eyes.

[0023] In some aspects, the method further includes providing a simulation system with a position of the physical vehicle at the racetrack and a position of a viewpoint, the simulation system calculating a virtual world including the virtual vehicle and a representation of the physical vehicle, the simulation system calculating a virtual position of a viewpoint within the virtual world based on the position of the viewpoint at the racetrack, and the simulation system calculating a portion of the virtual vehicle visible from the virtual position of the viewpoint, wherein providing the portion of the virtual vehicle visible from the virtual position of the viewpoint to the display system includes the simulation system outputting the portion of the virtual vehicle visible from the virtual position of the viewpoint.

[0024] In some aspects, identifying the location of the physical vehicle includes receiving a location of each of two portions of the vehicle.

[0025] In some aspects, identifying a position of the physical vehicle includes receiving a position of a portion of the vehicle and an orientation of the vehicle. In some aspects, receiving the orientation of the vehicle includes receiving gyroscope data.

[0026] In some aspects, the viewpoint positions at the race track include viewpoint positions of an operator of a physical vehicle at the race track, hi some aspects, the viewpoint positions at the race track include viewpoint positions of audience members present at the race track and viewing the physical vehicle at the race track.

[0027] In some aspects, the position of the viewpoint at the race track includes the position of a camera that is at the race track and imaging the physical vehicles at the race track. In some aspects, the camera images the portion of the race track where the physical vehicles are racing. When the physical vehicles are traveling across the portion of the race track captured by the camera, the camera may capture the physical vehicles in its video feed. When the physical vehicles are not traveling across the portion of the race track captured by the camera, the camera may still capture the portion of the race track.

[0028] In some embodiments, identifying a position of a viewpoint at the racetrack includes at least one of measuring an eye gaze point, tracking eye movement, tracking head position, identifying a vector from one or both eyes to a fixed point on the physical vehicle, identifying a vector from a point on the head to a fixed point on the physical vehicle, identifying a vector from a point on the eyewear to a fixed point on the physical vehicle, identifying a vector from a point on the headgear to a fixed point on the physical vehicle, identifying a vector from one or both eyes to a fixed point in the venue, identifying a vector from a point on the head to a fixed point in the venue, identifying a vector from a point on the eyewear to a fixed point in the venue, or identifying a vector from a point on the headgear to a fixed point in the venue. In some embodiments, identifying a position of a viewpoint at the racetrack includes measuring an eye gaze point, and measuring includes measuring a reflection or refraction of light from the eye.

[0029] In some embodiments, providing the position of the physical vehicle at the racetrack and the position of the viewpoint includes wirelessly transmitting at least one of the positions.

[0030] In some aspects, computing the virtual world includes transforming physical coordinates of the physical vehicle into virtual world coordinates, and the virtual position of the physical vehicle includes the virtual coordinates.

[0031] In some aspects, computing the portion of the virtual vehicle visible from the virtual position of the viewpoint includes computing a representation of the physical vehicle in the virtual world, computing representations of physical objects in the virtual world between the viewpoint and the virtual vehicle in the virtual world, and extracting the portion of the virtual vehicle that is not obscured by the representation of the physical vehicle and the representation of the physical objects as seen from the virtual position of the viewpoint. In some aspects, the portion of the virtual vehicle in the virtual world that is visible from the virtual position of the viewpoint includes the unobscured portion.

[0032] In some aspects, extracting the portion of the virtual vehicle may include determining which pixels are occluded by other representations and displaying only those pixels that are not occluded by other representations. In some aspects, extracting the portion of the virtual vehicle may include setting a pixel alpha value of zero percent (in RGBA space) for all pixels that are occluded by other representations. For example, the portion of the virtual vehicle may be occluded by other virtual representations, such as another virtual vehicle, or a representation of a physical object, such as an object in a physical vehicle or the physical vehicle itself. Any visible (from the virtual position of the viewpoint) pixel value may be used to provide the portion of the virtual vehicle that is visible from the virtual position of the viewpoint. In some aspects, the pixels of the non-obscured visible portion of the virtual vehicle may each be set to include an alpha value greater than zero percent (in RGBA space) to indicate that those non-obscured pixels are visible and should be displayed. In contrast, pixels set to an alpha value of zero percent indicate that those pixels are completely transparent, i.e., should not be visible and displayed.

[0033] In some aspects, computing the representation of the physical object between the virtual position of the viewpoint and the representation of the physical vehicle includes accessing a database of representations to obtain the virtual position of the physical object.

[0034] In some aspects, the portions of the virtual vehicle that are visible from the virtual position of the viewpoint consist of the portions of the virtual vehicle that are not obscured by other representations in the virtual world.

[0035] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes a virtual shadow in the virtual world. In some aspects, the virtual shadow is at least one of a shadow cast by the virtual vehicle and a shadow projected onto the virtual vehicle. In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes a virtual reflection. In some aspects, the virtual reflection is at least one of a reflection of the virtual vehicle and a reflection on the virtual vehicle.

[0036] In some aspects, the step of the simulation system calculating a portion of the virtual vehicle within the virtual world that is visible from the virtual position of the viewpoint includes calculating a field of view from the virtual position of the viewpoint, and the step of providing to the display system the portion of the virtual vehicle that is visible from the virtual position of the viewpoint includes displaying the portion of the virtual vehicle within the field of view.

[0037] In some aspects, the step of the simulation system calculating a portion of the virtual vehicle in the virtual world that is visible from a virtual viewpoint position includes calculating a field of view from the virtual viewpoint position, and the step of providing to the display system the portion of the virtual vehicle that is visible from the virtual viewpoint position comprises displaying the portion of the virtual vehicle that is visible within the field of view.

[0038] In some aspects, a method for displaying a virtual vehicle includes means for identifying a position of a physical vehicle at a racetrack, means for identifying a position of a viewpoint at the racetrack, and means for providing to a display system a portion of the virtual vehicle visible from a virtual position of the viewpoint calculated within the virtual world based on the position of the viewpoint at the racetrack. As an exemplary advantage, the aspects described herein may enable a physical vehicle operator to compete against a virtual vehicle operator. Further, by displaying a representation of the portion of the virtual vehicle visible from the virtual position of the viewpoint, a race between the physical vehicle operator and the virtual vehicle operator may be more realistically depicted.

[0039] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes a portion of the virtual vehicle that is not obscured, as viewed from the virtual position of the viewpoint, by a representation of the physical vehicle at the virtual position of the physical vehicle in the virtual world. In some aspects, the method further includes means for the simulation system to simulate an interaction between the virtual vehicle and the representation of the physical vehicle in the virtual world, wherein the portion of the virtual vehicle visible from the virtual position of the viewpoint is calculated based on the interaction.

[0040] In some aspects, the position of the viewpoint at the racetrack includes a viewpoint of an operator of the physical vehicle, and the means for identifying the position of the viewpoint at the racetrack includes means for detecting a viewpoint of the operator of the physical vehicle with a sensor, and the method further includes means for identifying a position of the physical object, means for receiving kinematic information of the virtual vehicle, means for generating, at a display system, a representation of the virtual vehicle based on the positions of the physical objects, the position of the viewpoint at the racetrack, and the kinematic information, and means for displaying the representation of the virtual vehicle such that the virtual vehicle is aligned with the physical object as viewed from the position of the viewpoint at the racetrack.

[0041] In some aspects, the method further includes means for generating, at the display system, a representation of the portion of the virtual vehicle that is visible from the virtual position of view.

[0042] In some aspects, the method further includes means for the display system to display a series of representations of the virtual vehicle over a period of time to simulate the virtual vehicle's trajectory on a racetrack, the series of representations including representations of portions of the virtual vehicle as seen from a virtual position of view. In some aspects, a predicted trajectory of the virtual vehicle is displayed. The prediction may be based on a current trajectory, acceleration, current vehicle parameters, etc. This may allow an audience member to predict whether the virtual vehicle is likely to overtake the physical vehicle. The predicted trajectory may be presented as a line, such as a yellow line. Other indications may also be included, such as "About to overtake!" or "About to crash!".

[0043] In some aspects, the method further includes means for the display system to store a digital 3D model of the virtual vehicle used to generate each representation from the series of representations, each representation being generated by the display system based on the digital 3D model.

[0044] In some aspects, the method further includes means for receiving a digital 3D model of the virtual vehicle used to generate each representation from the series of representations, each representation being generated by the display system based on the digital 3D model.

[0045] In some embodiments, the kinematic information includes one or more vectors of motion, one or more scalars of motion, a position vector, a GPS position, velocity, acceleration, orientation, or a combination thereof of the virtual vehicle.

[0046] In some embodiments, the means for identifying the position of the physical vehicle includes means for detecting one or more vectors of motion, one or more scalars of motion, a position vector, a GPS position, a velocity, an acceleration, an orientation, or a combination thereof, of the virtual vehicle.

[0047] In some aspects, the means for identifying a position of the viewpoint at the racetrack includes means for detecting a spatial position of a head of an operator of the physical vehicle. In some aspects, the method further includes means for a telemetry system coupled to the physical vehicle to send the spatial position to the simulator system, means for receiving, at the telemetry system, information related to a portion of the virtual vehicle that is visible from the virtual position of the viewpoint, and means for displaying a representation of the portion of the virtual vehicle to the operator of the physical vehicle based on the information.

[0048] In some aspects, the method further includes the means for displaying a representation of the portion of the virtual vehicle includes means for converting information into a set of graphical elements, and the means for displaying the representation of the portion includes means for displaying the set of graphical elements. In some aspects, the method further includes means for calculating, at the simulation system, information related to the portion visible from the virtual position of view.

[0049] In some aspects, the means for displaying the sequence of representations of the virtual vehicle includes means for displaying the sequence of representations on a display of the physical vehicle, the display being a transparent organic light emitting diode (T-OLED) display that passes light through a T-OLED to display a field of view to the operator.

[0050] In some aspects, the means for displaying the sequence of representations of the virtual vehicle includes means for displaying the sequence of representations on a display of the physical vehicle, the display being an LCD display, and the method further includes means for a camera coupled to the physical vehicle to capture an image representing a field of view of the physical world as seen by an operator on a display within the physical vehicle, and means for outputting the image to one side of the LCD display for displaying the field of view to the operator, the sequence of representations being overlaid on the image displayed by the LCD display.

[0051] In some aspects, the means for displaying the sequence of representations of the virtual vehicle includes means for displaying the sequence of representations on a display of the physical vehicle, the display including a windshield of the physical vehicle, one or more side windows of the physical vehicle, a rear window of the physical vehicle, one or more side mirrors, a rearview mirror, or a combination thereof.

[0052] In some aspects, the means for displaying the sequence of representations of the virtual vehicle includes means for displaying the sequence of representations on a display of a headset worn by the operator. hi some aspects, the headset is a helmet.

[0053] In some embodiments, the means for identifying a position of the viewpoint at the racetrack includes means for detecting one or more of a spatial position of the user's eyes, a gaze direction of the user's eyes, or a focus of the user's eyes.

[0054] In some aspects, the method further includes means for providing to the simulation system a position of the physical vehicle at the racetrack and a position of a viewpoint, means for the simulation system to calculate a virtual world including the virtual vehicle and a representation of the physical vehicle, means for the simulation system to calculate a virtual position of the viewpoint within the virtual world based on the position of the viewpoint at the racetrack, and means for the simulation system to calculate a portion of the virtual vehicle visible from the virtual position of the viewpoint, wherein the means for providing to the display system the portion of the virtual vehicle visible from the virtual position of the viewpoint includes means for the simulation system to output the portion of the virtual vehicle visible from the virtual position of the viewpoint.

[0055] In some aspects, the means for identifying a location of the physical vehicle includes means for receiving a location of each of two portions of the vehicle.

[0056] In some aspects, the means for identifying a position of the physical vehicle includes means for receiving a position of a portion of the vehicle and an orientation of the vehicle. In some aspects, the means for receiving the orientation of the vehicle includes means for receiving gyroscope data.

[0057] In some aspects, the viewpoint location at the racetrack includes a viewpoint location of an operator of the physical vehicle at the racetrack. In some aspects, the viewpoint location at the racetrack includes a viewpoint location of an audience member present at the racetrack and viewing the physical vehicle at the racetrack. In some aspects, the viewpoint location at the racetrack includes a position of a camera present at the racetrack, and the method further includes imaging the physical vehicle at the racetrack. In some aspects, the camera images a portion of the racetrack where the physical vehicle is racing. When the physical vehicle is traveling across the portion of the racetrack captured by the camera, the camera may capture the physical vehicle in its video feed. When the physical vehicle is not traveling across the portion of the racetrack captured by the camera, the camera may still capture the portion of the racetrack.

[0058] In some embodiments, the means for identifying a position of a viewpoint at the racetrack includes at least one of: a means for measuring an eye gaze point, a means for tracking eye movement, a means for tracking head position, a means for identifying a vector from one or both eyes to a fixed point on the physical vehicle, a means for identifying a vector from a point on the head to a fixed point on the physical vehicle, a means for identifying a vector from a point on the eyewear to a fixed point on the physical vehicle, a means for identifying a vector from a point on the headgear to a fixed point on the physical vehicle, a means for identifying a vector from one or both eyes to a fixed point in the venue, a means for identifying a vector from a point on the head to a fixed point in the venue, a means for identifying a vector from a point on the eyewear to a fixed point in the venue, or a means for identifying a vector from a point on the headgear to a fixed point in the venue. In some embodiments, the means for identifying a position of a viewpoint at the racetrack includes a means for measuring an eye gaze point, and the means for measuring includes a means for measuring a reflection or refraction of light from the eye.

[0059] In some aspects, the means for providing the position of the physical vehicle at the racetrack and the position of the viewpoint includes means for wirelessly transmitting at least one of the positions.

[0060] In some aspects, the means for computing the virtual world includes means for transforming physical coordinates of the physical vehicle into virtual world coordinates, and the virtual position of the physical vehicle includes the virtual coordinates.

[0061] In some aspects, the means for computing the portion of the virtual vehicle visible from the virtual position of the viewpoint includes means for computing a representation of the physical vehicle in the virtual world, means for computing representations of physical objects in the virtual world between the viewpoint and the virtual vehicle in the virtual world, and means for extracting the portion of the virtual vehicle that is not obscured by the representation of the physical vehicle and the representation of the physical objects as viewed from the virtual position of the viewpoint. In some aspects, the portion of the virtual vehicle in the virtual world that is visible from the virtual position of the viewpoint includes the unobscured portion.

[0062] In some aspects, the means for extracting the portion of the virtual vehicle may include means for determining which pixels are occluded by other representations and displaying only pixels that are not occluded by other representations. In some aspects, the means for extracting the portion of the virtual vehicle may include means for setting a pixel alpha value of zero percent (in RGBA space) for all pixels that are occluded by other representations. For example, the portion of the virtual vehicle may be occluded by other virtual representations, such as another virtual vehicle, or a representation of a physical object, such as an object in a physical vehicle or the physical vehicle itself. Any visible (from the virtual position of the viewpoint) pixel value may be used to provide the portion of the virtual vehicle that is visible from the virtual position of the viewpoint. In some aspects, the pixels of the non-obscured visible portion of the virtual vehicle may each be set to include an alpha value greater than zero percent (in RGBA space) to indicate that the non-obscured pixels are visible and should be displayed. In contrast, pixels set to an alpha value of zero percent indicate that the pixels are completely transparent, i.e., should not be visible and displayed.

[0063] In some aspects, the means for computing a representation of a physical object between the virtual position of the viewpoint and the representation of the physical vehicle includes means for accessing a database of representations to obtain the virtual position of the physical object.

[0064] In some aspects, the portions of the virtual vehicle that are visible from the virtual position of the viewpoint consist of the portions of the virtual vehicle that are not obscured by other representations in the virtual world.

[0065] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes a virtual shadow in the virtual world. In some aspects, the virtual shadow is at least one of a shadow cast by the virtual vehicle and a shadow projected onto the virtual vehicle. In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes a virtual reflection. In some aspects, the virtual reflection is at least one of a reflection of the virtual vehicle and a reflection on the virtual vehicle.

[0066] In some aspects, the means for the simulation system to calculate a portion of the virtual vehicle within the virtual world that is visible from the virtual position of the viewpoint includes means for calculating a field of view from the virtual position of the viewpoint, and the means for providing to the display system the portion of the virtual vehicle that is visible from the virtual position of the viewpoint includes displaying the portion of the virtual vehicle within the field of view.

[0067] In some aspects, the means for the simulation system to calculate a portion of the virtual vehicle in the virtual world that is visible from a virtual viewpoint position includes means for calculating a field of view from the virtual viewpoint position, and the means for providing to the display system the portion of the virtual vehicle that is visible from the virtual viewpoint position comprises means for displaying the portion of the virtual vehicle that is visible within the field of view.

[0068] In some embodiments, a system for displaying a virtual vehicle includes a first sensor that detects a position of a physical vehicle on a racetrack, a second sensor that detects a position of a viewpoint on the racetrack, and a simulation system that outputs a portion of the virtual vehicle that is visible from a virtual position of the viewpoint. As an exemplary advantage, the embodiments described herein may enable a physical vehicle operator to compete with a virtual vehicle operator. Furthermore, by displaying a portion of the virtual vehicle that is visible from a viewpoint position on the racetrack, a competition between the physical vehicle operator and the virtual vehicle operator may be more realistically represented.

[0069] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes the portion of the virtual vehicle that is not obscured, as viewed from the virtual position of the viewpoint, by a representation of the physical vehicle at the virtual position of the physical vehicle in the virtual world.

[0070] In some aspects, the system further includes a simulation system configured to simulate an interaction between the virtual vehicle and a representation of the physical vehicle in the virtual world, where a portion of the virtual vehicle visible from a virtual position of the viewpoint is calculated based on the interaction.

[0071] In some aspects, the system further includes a first sensor coupled to the physical vehicle, a second sensor configured to detect an eye position of an operator of the physical vehicle, and a display system coupled to the physical vehicle and configured to receive kinematic information of the virtual vehicle, identify positions of physical objects in the operator's field of view, generate a representation of the virtual vehicle based on the positions of the physical objects, a viewpoint, and the kinematic information, and display the representation of the virtual vehicle such that the virtual vehicle is aligned with the physical objects as viewed from the viewpoint.

[0072] In some aspects, the system further includes a display system configured to generate a representation of the portion of the virtual vehicle that is visible from the virtual position of view.

[0073] In some aspects, displaying a representation of the virtual vehicle includes displaying a representation of a physical vehicle on the display screen, and the display system is further configured to display on the display screen a series of representations of the virtual vehicle over a period of time to simulate the trajectory of the virtual vehicle on a race track, the series of representations including a representation of a portion of the virtual vehicle as seen from a virtual position of view. In some aspects, a predicted trajectory of the virtual vehicle is displayed. The prediction may be based on a current trajectory, acceleration, current vehicle parameters, etc. This may allow an audience member to predict whether the virtual vehicle is likely to overtake the physical vehicle. The predicted trajectory may be presented as a line, such as a yellow line. Other indications may also be included, such as "About to overtake!" or "About to crash!".

[0074] In some aspects, displaying the series of representations of the virtual vehicle includes displaying the series of representations on a display of the physical vehicle, and the display system is further configured to store a digital 3D model of the virtual vehicle used to generate each representation from the series of representations, and each representation is generated by the display system based on the digital 3D model.

[0075] In some aspects, the display system is further configured to receive a digital 3D model of the virtual vehicle used to generate each representation from the series of representations, each representation being generated by the display system based on the digital 3D model.

[0076] In some embodiments, the kinematic information includes one or more vectors of motion, one or more scalars of motion, a position vector, a GPS position, velocity, acceleration, orientation, or a combination thereof of the virtual vehicle.

[0077] In some aspects, the first sensor is configured to detect the position of the physical vehicle by detecting one or more vectors of motion, one or more scalars of motion, a position vector, a GPS position, a velocity, an acceleration, an orientation, or a combination thereof, of the virtual vehicle.

[0078] In some embodiments, the first sensor is configured to detect an operator's gaze point by detecting a spatial position of the operator's head.

[0079] In some aspects, the display system is further configured to send the spatial position to the simulation system, receive information related to portions of the virtual vehicle that are visible from the virtual position of the viewpoint, and display the portions of the virtual vehicle to an operator of the physical vehicle.

[0080] In some aspects, the display system is further configured to display a series of representations of the virtual vehicle by converting the information into a set of graphical elements, and displaying the representations of the portions includes displaying the set of graphical elements. In some aspects, the information related to the portions visible from the virtual position of view is calculated by the simulation system.

[0081] In some embodiments, the system further includes a transparent organic light emitting diode (T-OLED) display that passes light through a T-OLED to display a field of view to an operator, and the display system is configured to display a sequence of representations on the T-OLED display to display a representation of the virtual vehicle.

[0082] In some aspects, the system further includes an LCD display and a camera coupled to the physical vehicle and configured to capture images representing a field of view of the physical world visible to an operator on the LCD display in the physical vehicle, the display system configured to output the image to one side of the LCD display for displaying the field of view to the operator and to overlay the series of representations on the image displayed by the LCD display.

[0083] In some aspects, the system further includes a display including a windshield of the physical vehicle, one or more side windows of the physical vehicle, a rear window of the physical vehicle, or a combination thereof, and the display system is configured to display a representation of the virtual vehicle on the display.

[0084] In some aspects, the system includes a headset worn by the operator, the headset including a display, and the display system configured to display a sequence of representations of the virtual vehicle on the display. In some aspects, the headset is a helmet.

[0085] In some aspects, the second sensor detects one or more of a spatial position of the user's eye, a gaze direction of the user's eye, or a focus of the user's eye.

[0086] In some aspects, the system includes a simulation system configured to receive a position of the physical vehicle and a position of a viewpoint at the racetrack, compute a virtual world including a virtual vehicle and a representation of the physical vehicle, compute a virtual position of the viewpoint within the virtual world based on the position of the viewpoint at the racetrack, compute a portion of the virtual vehicle that is visible from the virtual position of the viewpoint, and output to a display system the portion of the virtual vehicle that is visible from the virtual position of the viewpoint.

[0087] In some aspects, the first sensor receives a position of each of two portions of the vehicle.

[0088] In some aspects, the first sensor receives a position of a portion of the vehicle and an orientation of the vehicle. In some aspects, receiving the orientation of the vehicle includes receiving gyroscope data.

[0089] In some aspects, the viewpoint positions at the race track include viewpoint positions of an operator of a physical vehicle at the race track, hi some aspects, the viewpoint positions at the race track include viewpoint positions of audience members present at the race track and viewing the physical vehicle at the race track.

[0090] In some aspects, the position of the viewpoint at the race track includes the position of a camera that is at the race track and imaging the physical vehicles at the race track. In some aspects, the camera images the portion of the race track where the physical vehicles are racing. When the physical vehicles are traveling across the portion of the race track captured by the camera, the camera may capture the physical vehicles in its video feed. When the physical vehicles are not traveling across the portion of the race track captured by the camera, the camera may still capture the portion of the race track.

[0091] In some embodiments, the second sensor is configured to detect a position of the eye gaze at the racetrack by at least one of measuring an eye gaze point, tracking eye movement, tracking head position, identifying a vector from one or both eyes to a fixed point on the physical vehicle, identifying a vector from a point on the head to a fixed point on the physical vehicle, identifying a vector from a point on the eyewear to a fixed point on the physical vehicle, identifying a vector from a point on the headgear to a fixed point on the physical vehicle, identifying a vector from one or both eyes to a fixed point in the venue, identifying a vector from a point on the head to a fixed point in the venue, or identifying a vector from a point on the eyewear to a fixed point in the venue, identifying a vector from a point on the headgear to a fixed point in the venue. In some embodiments, identifying the position of the eye gaze at the racetrack includes measuring an eye gaze point, and measuring includes measuring a reflection or refraction of light from the eye.

[0092] In some aspects, receiving the position of the physical vehicle at the racetrack and the position of the viewpoint includes wirelessly receiving at least one of the positions.

[0093] In some aspects, computing the virtual world includes transforming physical coordinates of the physical vehicle into virtual world coordinates, and the virtual position of the physical vehicle includes the virtual coordinates.

[0094] In some aspects, calculating the portion of the virtual vehicle visible from the virtual position of the viewpoint includes calculating a representation of the physical vehicle in the virtual world, calculating representations of physical objects in the virtual world between the viewpoint and the virtual vehicle in the virtual world, and extracting the portion of the virtual vehicle that is not obscured by the representation of the physical vehicle and the representation of the physical objects as viewed from the virtual position of the viewpoint. In some aspects, the portion of the virtual vehicle in the virtual world that is visible from the virtual position of the viewpoint includes the unobscured portion.

[0095] In some aspects, extracting the portion of the virtual vehicle may include determining which pixels are occluded by other representations and displaying only those pixels that are not occluded by other representations. In some aspects, extracting the portion of the virtual vehicle may include setting a pixel alpha value of zero percent (in RGBA space) for all pixels that are occluded by other representations. For example, the portion of the virtual vehicle may be occluded by other virtual representations, such as another virtual vehicle, or a representation of a physical object, such as an object in a physical vehicle or the physical vehicle itself. Any visible (from the virtual position of the viewpoint) pixel value may be used to provide the portion of the virtual vehicle that is visible from the virtual position of the viewpoint. In some aspects, the pixels of the non-obscured visible portion of the virtual vehicle may each be set to include an alpha value greater than zero percent (in RGBA space) to indicate that those non-obscured pixels are visible and should be displayed. In contrast, pixels set to an alpha value of zero percent indicate that those pixels are completely transparent, i.e., should not be visible and displayed.

[0096] In some aspects, computing the representation of the physical object between the virtual position of the viewpoint and the representation of the physical vehicle includes accessing a database of representations to obtain the virtual position of the physical object.

[0097] In some aspects, the portions of the virtual vehicle that are visible from the virtual position of the viewpoint consist of the portions of the virtual vehicle that are not obscured by other representations in the virtual world.

[0098] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes a virtual shadow in the virtual world. In some aspects, the virtual shadow is at least one of a shadow cast by the virtual vehicle and a shadow cast onto the virtual vehicle. In some aspects, the portion of the virtual vehicle visible from the position of the viewpoint at the racetrack includes a virtual reflection. In some aspects, the virtual reflection is at least one of a reflection of the virtual vehicle and a reflection on the virtual vehicle.

[0099] In some aspects, the system further includes a simulation system configured to calculate a portion of the virtual vehicle visible from the virtual position of the viewpoint by calculating a field of view from the virtual position of the viewpoint, and providing the display system with the portion of the virtual vehicle visible from the virtual position of the viewpoint includes displaying the portion of the virtual vehicle within the field of view.

[0100] In some aspects, the system further includes a simulation system configured to calculate a portion of the virtual vehicle visible from the virtual position of the viewpoint by calculating a field of view from the virtual position of the viewpoint, and providing the display system with the portion of the virtual vehicle visible from the virtual position of the viewpoint comprises displaying the portion of the virtual vehicle visible within the field of view. [The present invention 1001] 1. A method for displaying a virtual vehicle, comprising the steps of: identifying a location of the physical vehicle on the racetrack; identifying a location of a viewpoint on the racetrack; providing a simulation system with the position of the physical vehicle on the racetrack and the position of the viewpoint; said simulation system computing a virtual world including said virtual vehicle and a representation of said physical vehicle in a virtual location; the simulation system calculating a virtual position of the viewpoint within the virtual world based on the position of the viewpoint at the racetrack; the simulation system calculating a portion of the virtual vehicle in the virtual world that is visible from the virtual position of the viewpoint, the portion of the virtual vehicle that is visible from the virtual position of the viewpoint including a portion of the virtual vehicle that is not obscured, as viewed from the virtual position of the viewpoint, by the representation of the physical vehicle at the virtual position of the physical vehicle; the simulation system outputting the portion of the virtual vehicle that is visible from the virtual position of view; providing to a display system the portion of the virtual vehicle that is visible from the virtual position of view; generating, on the display system, a representation of the portion of the virtual vehicle as seen from the virtual position of view; and displaying a series of representations of the virtual vehicle over a period of time to simulate a trajectory of the virtual vehicle around the racetrack, the series of representations including the representation of the portion of the virtual vehicle visible from the virtual position of view. [The present invention 1002] The method of the present invention 1001, wherein identifying the position of the viewpoint on the racetrack includes detecting the spatial position of a user's eye, the gaze direction of the user's eye, or the focus of the user's eye. [The present invention 1003] The method of claim 1001, wherein the position of the viewpoint at the racetrack includes a position of a viewpoint of an operator of the physical vehicle at the racetrack. [The present invention 1004] The method of claim 1001, wherein the location of the viewpoint at the racetrack includes a location of a viewpoint of an audience member present at the racetrack and viewing the physical vehicle at the racetrack. [The present invention 1005] The method of claim 1001, wherein the position of the viewpoint at the racetrack includes a position of a camera that is at the racetrack and is imaging the physical vehicle at the racetrack. [The present invention 1006] The method of claim 1001, wherein the step of calculating the virtual world includes transforming physical coordinates of the physical vehicle to virtual coordinates of the virtual world, and the virtual position of the physical vehicle includes the virtual coordinates. [The present invention 1007] computing a representation of physical objects in the virtual world between the virtual position of the viewpoint and the virtual vehicle within the virtual world; Further comprising: the portion of the virtual vehicle in the virtual world that is visible from the virtual position of viewpoint includes a portion of the virtual vehicle that is not obscured by a representation of the physical vehicle and the representation of the physical object, as viewed from the virtual position of viewpoint. The method of the present invention 1001. [The present invention 1008] The method of the present invention 1001, wherein the portion of the virtual vehicle visible from the virtual position of the viewpoint comprises a portion of the virtual vehicle that is not obscured from the virtual position of the viewpoint by other representations within the virtual world. [The present invention 1009] the step of the simulation system calculating the portion of the virtual vehicle in the virtual world that is visible from the virtual position of viewpoint includes calculating a field of view from the virtual position of viewpoint; providing, to the display system, the portion of the virtual vehicle that is visible from the virtual position of view, comprising providing the portion of the virtual vehicle within the field of view. The method of the present invention 1001. [The present invention 1010] the simulation system simulating an interaction between the virtual vehicle and the representation of the physical vehicle in the virtual world, the portion of the virtual vehicle that is visible from the virtual position of view being calculated based on the interaction; The method of the present invention 1001 further comprises: [The present invention 1011] the virtual world includes virtual objects, and the method further comprises: the simulation system determining contact between the virtual object in the virtual world and the representation of the physical vehicle. The method of the present invention 1001 further comprises: [The present invention 1012] The method of claim 1011, wherein the virtual object comprises a simulated obstacle, smoke, a wall, an explosion, or debris. [The present invention 1013] the simulation system calculating a portion of the virtual object in the virtual world that is visible from the virtual position of the viewpoint, the portion of the virtual object that is visible from the virtual position of the viewpoint including a portion of the virtual object that is not obscured, as viewed from the virtual position of the viewpoint, by the representation of the physical vehicle at the virtual position of the physical vehicle; the simulation system outputting the portion of the virtual object that is visible from the virtual position of view; providing to the display system the portion of the virtual object that is visible from the virtual position of the viewpoint; generating, at the display system, a representation of the portion of the virtual object that is visible from the virtual position of view; and displaying the representation of the portion of the virtual object that is visible from the virtual position of view. The method of the present invention 1011 further comprises: [The present invention 1014] Calculating the portion of the virtual object in the virtual world that is visible from the virtual position of the viewpoint comprises: Access 3D models, including models of living or inanimate objects; The method of the present invention 1013, comprising: [The present invention 1015] The method of the present invention 1011, wherein the virtual objects are simulated in the virtual world and do not physically exist on the racetrack. [The present invention 1016] the simulation system generating audio information based on the determined contact; providing the audio information to a speaker within the physical vehicle; and playing said audio information using said speaker. The method of the present invention 1011 further comprises: [The present invention 1017] the simulation system generating force information based on the determined contact; providing the force information to a force controller of the physical vehicle; and the force controller controlling mechanical or electrical components of the physical vehicle based on the force information. The method of the present invention 1011 further comprises: [The present invention 1018] receiving information associated with an input from an operator of the physical vehicle; Further comprising: the force information is provided to the force controller in response to receiving the information associated with the input. The method of the present invention 1017. [The present invention 1019] The method of claim 1018, wherein the information associated with the input is received from the operator. [The present invention 1020] controlling the mechanical or electrical components of the physical vehicle, Controlling the brakes or steering column of said physical vehicle. The method of the present invention 1017, comprising: [The present invention 1021] controlling the mechanical or electrical components of the physical vehicle, Controlling the power provided to motors that control the wheels or axles of said physical vehicle. The method of the present invention 1017, comprising: [The present invention 1022] controlling the mechanical or electrical components of the physical vehicle, controlling the mechanical or electrical components to reduce power supplied to the physical vehicle to affect the speed of the physical vehicle; The method of the present invention 1017, comprising: [The present invention 1023] controlling the mechanical or electrical components of the physical vehicle, Controlling the mechanical or electrical components to increase power supplied to the physical vehicle to affect the speed of the physical vehicle. The method of the present invention 1017, comprising: [The present invention 1024] receiving information associated with the physical vehicle input; Further comprising: the force information is provided to the force controller in response to receiving the information associated with the input. The method of the present invention 1017. [The present invention 1025] the simulation system generating force information based on the determined contact; receiving information associated with an input from an operator of the physical vehicle; providing the force information to a force controller of another vehicle in response to receiving the information associated with the input; and the force controller controlling a mechanical or electrical component of the other vehicle based on the force information. The method of the present invention 1011 further comprises: [The present invention 1026] The method of claim 1025, wherein the information associated with the input is received from the operator. [The present invention 1027] controlling the mechanical or electrical component of the other vehicle, Controlling the brakes or steering column of said other vehicle. The method of the present invention 1025, comprising: [The present invention 1028] controlling the mechanical or electrical component of the other vehicle, Controlling the power provided to a motor that controls the wheels or axles of said other vehicle. The method of the present invention 1025, comprising: [The present invention 1029] controlling the mechanical or electrical component of the other vehicle, Controlling the mechanical or electrical components to reduce power supplied to the other vehicle to affect the speed of the other vehicle. The method of the present invention 1025, comprising: [The present invention 1030] controlling the mechanical or electrical component of the other vehicle, Controlling the mechanical or electrical component to increase power supplied to the other vehicle to affect the speed of the other vehicle. The method of the present invention 1025, comprising: [The present invention 1031] A system for displaying a virtual vehicle, comprising: A first sensor for detecting a position of the physical vehicle on the racetrack; a second sensor for detecting a position of a viewpoint on the racetrack; 1. A simulation system comprising: receiving the position of the physical vehicle from the first sensor and the position of the viewpoint on the racetrack from the second sensor; Computing a virtual world including a virtual vehicle and a representation of the physical vehicle at a virtual location; calculating a virtual position of the viewpoint within the virtual world based on the position of the viewpoint at the racetrack; calculating a portion of the virtual vehicle in the virtual world that is visible from the virtual position of the viewpoint, the portion of the virtual vehicle that is visible from the virtual position of the viewpoint including a portion of the virtual vehicle that is not obscured, as viewed from the virtual position of the viewpoint, by the representation of the physical vehicle at the virtual position of the physical vehicle; outputting the portion of the virtual vehicle that is visible from the virtual position of view. Simulation systems and; 1. A display system comprising: receiving from the simulation system the portion of the virtual vehicle that is visible from the virtual position of view; generating a representation of the portion of the virtual vehicle as seen from the virtual position of view; displaying a series of representations of the virtual vehicle over a period of time to simulate a trajectory of the virtual vehicle around the racetrack, the series of representations including the representation of the portion of the virtual vehicle visible from the virtual position of view. Display system. [The present invention 1032] The system of the present invention 1031, wherein the second sensor detects the position of the viewpoint on the racetrack by detecting the spatial position of the user's eye, the gaze direction of the user's eye, or the focus of the user's eye. [The present invention 1033] The system of the present invention 1031, wherein the position of the viewpoint on the racetrack includes the position of the viewpoint of an operator of the physical vehicle on the racetrack. [The present invention 1034] The system of the present invention 1031, wherein the position of the viewpoint at the racetrack includes a position of a viewpoint of an audience member present at the racetrack and viewing the physical vehicle at the racetrack. [The present invention 1035] The system of the present invention 1031, wherein the position of the viewpoint at the racetrack includes the position of a camera present at the racetrack and imaging the physical vehicle at the racetrack. [The present invention 1036] The system of the present invention 1031, wherein calculating the virtual world includes transforming physical coordinates of the physical vehicle to coordinates of the virtual world, and the virtual position of the physical vehicle includes the virtual coordinates. [The present invention 1037] The simulation system comprises: Compute a representation of physical objects in the virtual world between the virtual position of the viewpoint and the virtual vehicle in the virtual world. It is configured as follows: The system of the present invention 1031, wherein the portion of the virtual vehicle in the virtual world visible from the virtual position of the viewpoint includes portions of the virtual vehicle that are not obscured by the representation of the physical vehicle and the representation of the physical object as viewed from the virtual position of the viewpoint. [The present invention 1038] The system of the present invention 1031, wherein the portion of the virtual vehicle visible from the virtual position of the viewpoint comprises a portion of the virtual vehicle that is not obscured from the virtual position of the viewpoint by other representations within the virtual world. [The present invention 1039] calculating the portion of the virtual vehicle in the virtual world that is visible from the virtual position of viewpoint includes calculating a field of view from the virtual position of viewpoint; outputting the portion of the virtual vehicle visible from the virtual position of view includes outputting the virtual portion of the virtual vehicle within the field of view. The system of the present invention 1031. [The present invention 1040] The system of the present invention 1031, wherein the simulation system simulates an interaction between the virtual vehicle and the representation of the physical vehicle in the virtual world, and the portion of the virtual vehicle visible from the virtual position of the viewpoint is calculated based on the interaction. [The present invention 1041] The system of claim 1031, wherein the virtual world includes a virtual object and the simulation system determines contact between the virtual object in the virtual world and the representation of the physical vehicle. [The present invention 1042] The system of the present invention 1041, wherein the virtual object includes a simulated obstacle, smoke, a wall, an explosion, or debris. [The present invention 1043] The simulation system comprises: calculating a portion of the virtual object in the virtual world that is visible from the virtual position of the viewpoint, the portion of the virtual object that is visible from the virtual position of the viewpoint including a portion of the virtual object that is not obscured, as viewed from the virtual position of the viewpoint, by the representation of the physical vehicle at the virtual position of the physical vehicle; outputting the portion of the virtual object that is visible from the virtual position of the viewpoint; The display system comprises: providing the portion of the virtual object that is visible from the virtual position of the viewpoint; generating a representation of the portion of the virtual object as seen from the virtual position of the viewpoint; displaying the representation of the portion of the virtual object as seen from the virtual position of view. The system of the present invention 1041. [The present invention 1044] calculating the portion of the virtual object in the virtual world that is visible from the virtual position of the viewpoint, Access 3D models, including models of living or inanimate objects; The system of the present invention 1043, comprising: [The present invention 1045] The system of the present invention 1041, wherein the virtual objects are simulated in the virtual world and do not physically exist on the racetrack. [The present invention 1046] The simulation system comprises: generating audio information based on the determined contact; providing the audio information to the physical vehicle; The system of the present invention 1041. [The present invention 1047] The simulation system comprises: generating force information based on the determined contact; providing the force information to a force controller of the physical vehicle; The system of the present invention 1041. [The present invention 1048] The system of the present invention 1047, wherein the force information is provided to the force controller in response to receiving information associated with an input from an operator of the physical vehicle. [The present invention 1049] The system of the present invention 1048, wherein the information associated with the input is received from the operator. [The present invention 1050] controlling the mechanical or electrical components of the physical vehicle; Controlling the brakes or steering column of said physical vehicle. The system of the present invention 1047, comprising: [The present invention 1051] controlling the mechanical or electrical components of the physical vehicle; Controlling the power provided to motors that control the wheels or axles of said physical vehicle. The system of the present invention 1047, comprising: [The present invention 1052] controlling the mechanical or electrical components of the physical vehicle; controlling the mechanical or electrical components to reduce power supplied to the physical vehicle to affect the speed of the physical vehicle; The system of the present invention 1047, comprising: [The present invention 1053] controlling the mechanical or electrical components of the physical vehicle; Controlling the mechanical or electrical components to increase power supplied to the physical vehicle to affect the speed of the physical vehicle. The system of the present invention 1047, comprising: [The present invention 1054] The system of the present invention 1047, wherein the force information is provided to the force controller in response to receiving information associated with an input of the physical vehicle. [The present invention 1055] The simulation system comprises: generating force information based on the determined contact; receiving information associated with an input from an operator of the physical vehicle; providing the force information to a force controller of another vehicle in response to receiving the information associated with the input. The system of the present invention 1041. [The present invention 1056] The system of the present invention 1055, wherein the information associated with the input is received from the operator. [The present invention 1057] controlling the mechanical or electrical component of the other vehicle; Controlling the brakes or steering column of said other vehicle. The system of the present invention 1055 includes: [The present invention 1058] controlling the mechanical or electrical component of the other vehicle; Controlling the power provided to a motor that controls the wheels or axles of said other vehicle. The system of the present invention 1055 includes: [The present invention 1059] controlling the mechanical or electrical component of the other vehicle; Controlling the mechanical or electrical components to reduce power supplied to the other vehicle to affect the speed of the other vehicle. The system of the present invention 1055 includes: [The present invention 1060] controlling the mechanical or electrical component of the other vehicle; Controlling the mechanical or electrical component to increase power supplied to the other vehicle to affect the speed of the other vehicle. The system of the present invention 1055 includes: [Brief description of the drawings]

[0101] [Figure 1] FIG. 2 is an illustration of a physical vehicle, according to some aspects. [Diagram 2] 2A-C are diagrams illustrating how one or more virtual vehicles may be displayed on one or more displays, according to some embodiments. [Diagram 3] 3A-D are diagrams illustrating how the visible portion of a virtual vehicle may be depicted on a display, according to some embodiments. [Figure 4] 4A-D are diagrams illustrating how the visible portion of a virtual vehicle may be depicted on a display, according to some embodiments. [Diagram 5] FIG. 1 is a diagram of a system for simulating a virtual race between a live system and a simulated system, according to some aspects. [Figure 6] 1 is a flow diagram illustrating a method for displaying a virtual vehicle on a display, according to some aspects. [Figure 7] 1 is a flow diagram illustrating a method for providing an operator of a physical vehicle with two-way interaction between a virtual vehicle and a physical vehicle, according to some aspects. [Figure 8]1 is a flow diagram illustrating a method for simulating a race between a virtual vehicle and a physical vehicle to provide two-way interaction, according to some aspects. [Figure 9] 1 is a flow diagram illustrating a method performed by a simulation system to enable display of a virtual vehicle, according to some aspects. [Figure 10] 1 is a flow diagram illustrating a method for enabling display of a virtual vehicle, according to some aspects. [Figure 11] FIG. 1 is a functional block diagram of a computer according to some aspects. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0102] Detailed Description Aspects described herein integrate real-world and virtual-world racing competitions. For example, real-world and virtual-world racing champions can compete to determine an overall champion. Advantageously, each champion can compete against champions from other "worlds" while remaining within their respective "worlds." In effect, the aspects described herein allow live participants to compete against virtual participants.

[0103] The terms "physical" and "real world" are used interchangeably herein in contrast to "virtual world." For example, a "physical vehicle" or "real world vehicle" may be physically present on or at a racetrack. A "virtual vehicle" may not be physically present at the same racetrack. For example, a "virtual vehicle" may be a graphically generated vehicle shown on a display. In some aspects, a "virtual vehicle" is a representation in a software-based environment.

[0104] In some aspects, a method for displaying a virtual vehicle includes identifying a location of a physical vehicle on a racetrack, identifying a location of a viewpoint on the racetrack, and providing a display system with a portion of the virtual vehicle that is visible from the virtual location of the viewpoint. Problems solved by aspects disclosed herein can include overcoming a lack of realism experienced by users of conventional solutions. In some aspects herein, providing a user with a visible portion of the virtual vehicle enhances the realism experienced by the user. The enhanced realism provides a reliable and repeatable user experience by providing a real-world race involving virtual vehicles.

[0105] In some aspects, the visible portion of the virtual vehicle is calculated based on a virtual position of the physical vehicle in the virtual world, a virtual position of the viewpoint in the virtual world, and a virtual position of the virtual vehicle in the virtual world. Problems solved by aspects disclosed herein can include how to provide the visible portion of the virtual vehicle. In some aspects herein, providing the visible portion of the virtual vehicle through virtual calculation of the visible portion increases the accuracy of the visible portion determination. The increased accuracy provides a reliable and repeatable user experience by providing a real-world race involving the virtual vehicle. In some aspects herein, providing the visible portion through virtual calculation increases the efficiency of the computation. The increased efficiency reduces power usage and increases the rendering speed for a more seamless user experience.

[0106] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes the portion of the virtual vehicle that is not obscured, as viewed from the virtual position of the viewpoint, by a representation of the physical vehicle at the virtual position of the physical vehicle in the virtual world.

[0107] In some aspects, the method further includes a step in which the simulation system simulates an interaction between the virtual vehicle and a representation of the physical vehicle in the virtual world, and a portion of the virtual vehicle visible from a virtual position of the viewpoint is calculated based on the interaction.

[0108] In some aspects, the position of the viewpoint at the racetrack includes a viewpoint of an operator of the physical vehicle, and identifying the position of the viewpoint at the racetrack includes detecting a viewpoint of the operator of the physical vehicle with a sensor, and the method further includes identifying a position of the physical object, receiving kinematic information of the virtual vehicle, generating, at a display system, a representation of the virtual vehicle based on the positions of the physical objects, the position of the viewpoint at the racetrack, and the kinematic information, and displaying the representation of the virtual vehicle such that the virtual vehicle is aligned with the physical object as viewed from the position of the viewpoint at the racetrack.

[0109] In some aspects, the method further includes generating, on a display system, a representation of the portion of the virtual vehicle that is visible from the virtual position of view.

[0110] In some aspects, the method further includes the display system displaying a series of representations of the virtual vehicle over a period of time to simulate the virtual vehicle's trajectory on a racetrack, the series of representations including representations of portions of the virtual vehicle as seen from a virtual position of view. In some aspects, a predicted trajectory of the virtual vehicle is displayed. The prediction may be based on a current trajectory, acceleration, current vehicle parameters, etc. This may allow an audience member to predict whether the virtual vehicle is likely to overtake the physical vehicle. The predicted trajectory may be presented as a line, such as a yellow line. Other indications may also be included, such as "About to overtake!" or "About to crash!".

[0111] In some aspects, the method further includes the display system storing a digital 3D model of the virtual vehicle used to generate each representation from the series of representations, each representation being generated by the display system based on the digital 3D model.

[0112] In some aspects, the method further includes receiving a digital 3D model of the virtual vehicle used to generate each representation from the series of representations, each representation being generated by the display system based on the digital 3D model.

[0113] In some embodiments, the kinematic information includes one or more vectors of motion, one or more scalars of motion, a position vector, a GPS position, velocity, acceleration, orientation, or a combination thereof of the virtual vehicle.

[0114] In some embodiments, identifying the position of the physical vehicle includes detecting one or more vectors of motion, one or more scalars of motion, a position vector, a GPS position, a velocity, an acceleration, an orientation, or a combination thereof, of the virtual vehicle.

[0115] In some aspects, identifying a position of the viewpoint at the racetrack includes detecting a spatial position of a head of an operator of the physical vehicle. In some aspects, the method further includes a telemetry system coupled to the physical vehicle sending the spatial position to the simulator system, receiving at the telemetry system information related to a portion of the virtual vehicle visible from the virtual position of the viewpoint, and displaying a representation of the portion of the virtual vehicle to the operator of the physical vehicle based on the information.

[0116] In some aspects, the method further includes: displaying a representation of the portion of the virtual vehicle includes converting information into a set of graphical elements; and displaying the representation of the portion displays the set of graphical elements. In some aspects, the method further includes calculating, at the simulation system, information related to the portion that is visible from the virtual position of view.

[0117] In some embodiments, the step of displaying the sequence of representations of the virtual vehicle includes displaying the sequence of representations on a display of the physical vehicle, the display being a transparent organic light emitting diode (T-OLED) display that passes light through a T-OLED to display a field of view to an operator.

[0118] In some aspects, displaying the sequence of representations of the virtual vehicle includes displaying the sequence of representations on a display of a physical vehicle, the display being an LCD display, and the method further includes a camera coupled to the physical vehicle capturing an image representing a field of view of the physical world as seen by an operator on a display within the physical vehicle, and outputting the image to one side of the LCD display for displaying the field of view to the operator, the sequence of representations being overlaid on the image displayed by the LCD display.

[0119] In some aspects, the step of displaying the sequence of representations of the virtual vehicle includes displaying the sequence of representations on a display of a physical vehicle, the display including a windshield of the physical vehicle, one or more side windows of the physical vehicle, a rear window of the physical vehicle, one or more side mirrors, a rearview mirror, or a combination thereof.

[0120] In some aspects, displaying the sequence of representations of the virtual vehicle includes displaying the sequence of representations on a display of a headset worn by the operator. In some aspects, the headset is a helmet.

[0121] In some embodiments, identifying the location of the viewpoint at the racetrack includes detecting one or more of the spatial position of the user's eyes, the gaze direction of the user's eyes, or the focus of the user's eyes.

[0122] In some aspects, the method further includes providing the simulation system with a position of the physical vehicle at the racetrack and a position of a viewpoint, the simulation system computing a virtual world including the virtual vehicle and a representation of the physical vehicle, the simulation system computing a virtual position of a viewpoint in the virtual world based on the position of the viewpoint at the racetrack, and the simulation system computing a portion of the virtual vehicle visible from the virtual position of the viewpoint, and providing the portion of the virtual vehicle visible from the virtual position of the viewpoint to the display system includes the simulation system outputting the portion of the virtual vehicle visible from the virtual position of the viewpoint. Problems solved by aspects disclosed herein can include how to compute the visible portion of the virtual vehicle. In some aspects herein, computing the visible portion of the virtual vehicle in the virtual world increases the accuracy of the visible portion determination. The increased accuracy provides a reliable and repeatable user experience by providing a real-world race including the virtual vehicle. In some aspects herein, providing the visible portion by virtual computation increases the efficiency of the computation. The increased efficiency reduces power usage and increases the speed of the rendering to provide a more seamless user experience.

[0123] In some aspects, identifying the location of the physical vehicle includes receiving a location of each of two portions of the vehicle. In some aspects, identifying the location of the physical vehicle includes receiving a location of one portion of the vehicle and an orientation of the vehicle. In some aspects, receiving the orientation of the vehicle includes receiving gyroscope data. Problems solved by aspects disclosed herein can include how to properly position a physical vehicle in a virtual world to determine a visible portion of a virtual vehicle. In some aspects herein, using an orientation measure provides accurate placement of the physical vehicle in the virtual world. Greater accuracy provides a more reliable representation of the visible portion of the vehicle, improving the user experience.

[0124] In some aspects, the viewpoint location at the race track includes a viewpoint location of an operator of a physical vehicle at the race track. In some aspects, the viewpoint location at the race track includes a viewpoint location of an audience member at the race track viewing the physical vehicle at the race track. In some aspects, the viewpoint location at the race track includes a position of a camera at the race track imaging the physical vehicle at the race track. In some aspects, the camera images a portion of the race track where the physical vehicle is racing. When the physical vehicle is traveling across the portion of the race track captured by the camera, the camera may capture the physical vehicle in its video feed. When the physical vehicle is not traveling across the portion of the race track captured by the camera, the camera may still capture the portion of the race track.

[0125] In some embodiments, identifying a position of a viewpoint at the racetrack includes at least one of measuring an eye gaze point, tracking eye movement, tracking head position, identifying a vector from one or both eyes to a fixed point on the physical vehicle, identifying a vector from a point on the head to a fixed point on the physical vehicle, identifying a vector from a point on the eyewear to a fixed point on the physical vehicle, identifying a vector from a point on the headgear to a fixed point on the physical vehicle, identifying a vector from one or both eyes to a fixed point in the venue, identifying a vector from a point on the head to a fixed point in the venue, identifying a vector from a point on the eyewear to a fixed point in the venue, or identifying a vector from a point on the headgear to a fixed point in the venue. In some embodiments, identifying a position of a viewpoint at the racetrack includes measuring an eye gaze point, and measuring includes measuring a reflection or refraction of light from the eye.

[0126] In some embodiments, providing the position of the physical vehicle at the racetrack and the position of the viewpoint includes wirelessly transmitting at least one of the positions.

[0127] In some aspects, computing the virtual world includes transforming physical coordinates of the physical vehicle into virtual world coordinates, and the virtual position of the physical vehicle includes the virtual coordinates.

[0128] In some aspects, calculating the portion of the virtual vehicle visible from the virtual position of the viewpoint includes calculating a representation of the physical vehicle in the virtual world, calculating a representation of the physical objects in the virtual world between the viewpoint and the virtual vehicle in the virtual world, and extracting the portion of the virtual vehicle that is not obscured by the representation of the physical vehicle and the representation of the physical objects as seen from the virtual position of the viewpoint. In some aspects, the portion of the virtual vehicle in the virtual world that is visible from the virtual position of the viewpoint includes the portion that is not obscured. Problems solved by aspects disclosed herein can include how to calculate the visible portion of the virtual vehicle, including more than just the portion that is not obscured by the physical vehicle. In some aspects herein, calculating the visible portion in the virtual world including the real-world physical objects increases the accuracy of the visible portion determination. The increased accuracy provides a reliable and repeatable user experience by providing a real-world race including the virtual vehicle. In some aspects herein, providing the visible portion by virtual calculation increases the efficiency of the calculation. The increased efficiency reduces power usage and increases the speed of the rendering to provide a more seamless user experience.

[0129] In some aspects, extracting the portion of the virtual vehicle may include determining which pixels are occluded by other representations and displaying only those pixels that are not occluded by other representations. In some aspects, extracting the portion of the virtual vehicle may include setting a pixel alpha value of zero percent (in RGBA space) for all pixels that are occluded by other representations. For example, the portion of the virtual vehicle may be occluded by other virtual representations, such as another virtual vehicle, or a representation of a physical object, such as an object in a physical vehicle or the physical vehicle itself. Any visible (from the virtual position of the viewpoint) pixel value may be used to provide the portion of the virtual vehicle that is visible from the virtual position of the viewpoint. In some aspects, the pixels of the non-obscured visible portion of the virtual vehicle may each be set to include an alpha value greater than zero percent (in RGBA space) to indicate that those non-obscured pixels are visible and should be displayed. In contrast, pixels set to an alpha value of zero percent indicate that those pixels are completely transparent, i.e., should not be visible and displayed.

[0130] In some aspects, computing the representation of the physical object between the virtual position of the viewpoint and the representation of the physical vehicle includes accessing a database of representations to obtain the virtual position of the physical object.

[0131] In some aspects, the portions of the virtual vehicle that are visible from the virtual position of the viewpoint consist of the portions of the virtual vehicle that are not obscured by other representations in the virtual world.

[0132] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes a virtual shadow in the virtual world. In some aspects, the virtual shadow is at least one of a shadow cast by the virtual vehicle and a shadow projected onto the virtual vehicle. In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes a virtual reflection. In some aspects, the virtual reflection is at least one of a reflection of the virtual vehicle and a reflection on the virtual vehicle.

[0133] In some aspects, the step of the simulation system calculating a portion of the virtual vehicle within the virtual world that is visible from the virtual position of the viewpoint includes calculating a field of view from the virtual position of the viewpoint, and the step of providing to the display system the portion of the virtual vehicle that is visible from the virtual position of the viewpoint includes displaying the portion of the virtual vehicle within the field of view.

[0134] In some aspects, the step of the simulation system calculating a portion of the virtual vehicle in the virtual world that is visible from a virtual viewpoint position includes calculating a field of view from the virtual viewpoint position, and the step of providing to the display system the portion of the virtual vehicle that is visible from the virtual viewpoint position comprises displaying the portion of the virtual vehicle that is visible within the field of view.

[0135] In some embodiments, the method may facilitate a race between two virtual vehicles on a physical racetrack. In a scenario where two virtual vehicles compete on a physical racetrack without physical vehicles, then "identifying the location of the physical vehicle" would not be necessary. The method could include identifying a viewpoint location on the racetrack and providing a display system with the portion of the virtual vehicle visible from the viewpoint location on the racetrack. In such embodiments, all aspects of the above method that do not relate to the location of the physical vehicle could be applied. In some embodiments, the virtual vehicle is given special attributes and a video game appearance. In some embodiments, video game attributes (i.e., virtual objects) can be applied to the physical vehicle as well by overlaying those video game attributes on the physical vehicle. For example, a car could be given a boost, a machine gun, a missile (another graphical virtual object added to the real world view), a virtual jump, etc. The racetrack and at-home spectators could view the virtual competitors on the physical racetrack as if they were competing in the real world.

[0136] In some aspects, a method for displaying a virtual vehicle includes means for identifying a location of a physical vehicle at a racetrack, means for identifying a position of a viewpoint at the racetrack, and means for providing to a display system a portion of the virtual vehicle that is visible from a virtual position of the viewpoint calculated within the virtual world based on the position of the viewpoint at the racetrack. Problems solved by aspects disclosed herein can include overcoming a lack of realism experienced by users of conventional solutions. In some aspects herein, providing a user with a visible portion of the virtual vehicle enhances the realism experienced by the user. The enhanced realism provides a reliable and repeatable user experience by providing a real-world race involving virtual vehicles.

[0137] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes the portion of the virtual vehicle that is not obscured by a representation of the physical vehicle at the virtual position of the physical vehicle in the virtual world as viewed from the virtual position of the viewpoint. Problems solved by aspects disclosed herein can include how to provide the visible portion of the virtual vehicle. In some aspects herein, the visible portion of the virtual vehicle is provided by a virtual calculation of the visible portion, thereby increasing the accuracy of the visible portion determination. The increased accuracy provides a reliable and repeatable user experience by providing a real-world race involving the virtual vehicle. In some aspects herein, the visible portion is provided by a virtual calculation, thereby increasing the efficiency of the computation. The increased efficiency reduces power usage and increases the rendering speed for a more seamless user experience.

[0138] In some aspects, the method further includes means for the simulation system to simulate an interaction between the virtual vehicle and a representation of the physical vehicle in the virtual world, and a portion of the virtual vehicle visible from a virtual position of the viewpoint is calculated based on the interaction.

[0139] In some aspects, the position of the viewpoint at the racetrack includes a viewpoint of an operator of the physical vehicle, and the means for identifying the position of the viewpoint at the racetrack includes a means for detecting a viewpoint of the operator of the physical vehicle with a sensor, and the method further includes a means for identifying a position of the physical object, a means for receiving kinematic information of the virtual vehicle, a means for generating, at a display system, a representation of the virtual vehicle based on the position of the physical object, a position of the viewpoint at the racetrack, and the kinematic information, and a means for displaying the representation of the virtual vehicle such that the virtual vehicle is aligned with the physical object as viewed from the position of the viewpoint at the racetrack. Problems solved by aspects disclosed herein can include how to calculate the visible portion of the virtual vehicle. In some aspects herein, the step of calculating the visible portion of the virtual vehicle in the virtual world increases the accuracy of the visible portion determination. The increased accuracy provides a reliable and repeatable user experience by providing a real-world race involving the virtual vehicle. In some aspects herein, the efficiency of the calculation increases by providing the visible portion by virtual calculation. The increased efficiency reduces power usage and increases the speed of the presentation to provide a more seamless user experience.

[0140] In some aspects, the method further includes means for generating, at the display system, a representation of the portion of the virtual vehicle that is visible from the virtual position of view.

[0141] In some aspects, the method further includes means for the display system to display a series of representations of the virtual vehicle over a period of time to simulate the virtual vehicle's trajectory on a racetrack, the series of representations including representations of portions of the virtual vehicle as seen from a virtual position of view. In some aspects, a predicted trajectory of the virtual vehicle is displayed. The prediction may be based on a current trajectory, acceleration, current vehicle parameters, etc. This may allow an audience member to predict whether the virtual vehicle is likely to overtake the physical vehicle. The predicted trajectory may be presented as a line, such as a yellow line. Other indications may also be included, such as "About to overtake!" or "About to crash!".

[0142] In some aspects, the method further includes means for the display system to store a digital 3D model of the virtual vehicle used to generate each representation from the series of representations, each representation being generated by the display system based on the digital 3D model.

[0143] In some aspects, the method further includes means for receiving a digital 3D model of the virtual vehicle used to generate each representation from the series of representations, each representation being generated by the display system based on the digital 3D model.

[0144] In some embodiments, the kinematic information includes one or more vectors of motion, one or more scalars of motion, a position vector, a GPS position, velocity, acceleration, orientation, or a combination thereof of the virtual vehicle.

[0145] In some embodiments, the means for identifying the position of the physical vehicle includes means for detecting one or more vectors of motion, one or more scalars of motion, a position vector, a GPS position, a velocity, an acceleration, an orientation, or a combination thereof, of the virtual vehicle.

[0146] In some aspects, the means for identifying a position of the viewpoint at the racetrack includes means for detecting a spatial position of a head of an operator of the physical vehicle. In some aspects, the method further includes means for a telemetry system coupled to the physical vehicle to send the spatial position to the simulator system, means for receiving, at the telemetry system, information related to a portion of the virtual vehicle that is visible from the virtual position of the viewpoint, and means for displaying a representation of the portion of the virtual vehicle to the operator of the physical vehicle based on the information.

[0147] In some aspects, the method further includes the means for displaying a representation of the portion of the virtual vehicle includes means for converting information into a set of graphical elements, and the means for displaying the representation of the portion includes means for displaying the set of graphical elements. In some aspects, the method further includes means for calculating, at the simulation system, information related to the portion visible from the virtual position of view.

[0148] In some aspects, the means for displaying the sequence of representations of the virtual vehicle includes means for displaying the sequence of representations on a display of the physical vehicle, the display being a transparent organic light emitting diode (T-OLED) display that passes light through a T-OLED to display a field of view to the operator.

[0149] In some aspects, the means for displaying the sequence of representations of the virtual vehicle includes means for displaying the sequence of representations on a display of the physical vehicle, the display being an LCD display, and the method further includes means for a camera coupled to the physical vehicle to capture an image representing a field of view of the physical world as seen by an operator on a display within the physical vehicle, and means for outputting the image to one side of the LCD display for displaying the field of view to the operator, the sequence of representations being overlaid on the image displayed by the LCD display.

[0150] In some aspects, the means for displaying the sequence of representations of the virtual vehicle includes means for displaying the sequence of representations on a display of the physical vehicle, the display including a windshield of the physical vehicle, one or more side windows of the physical vehicle, a rear window of the physical vehicle, one or more side mirrors, a rearview mirror, or a combination thereof.

[0151] In some aspects, the means for displaying the sequence of representations of the virtual vehicle includes means for displaying the sequence of representations on a display of a headset worn by the operator. hi some aspects, the headset is a helmet.

[0152] In some embodiments, the means for identifying a position of the viewpoint at the racetrack includes means for detecting one or more of a spatial position of the user's eyes, a gaze direction of the user's eyes, or a focus of the user's eyes.

[0153] In some aspects, the method further includes means for providing to the simulation system a position of the physical vehicle at the racetrack and a position of a viewpoint, means for the simulation system to calculate a virtual world including the virtual vehicle and a representation of the physical vehicle, means for the simulation system to calculate a virtual position of the viewpoint within the virtual world based on the position of the viewpoint at the racetrack, and means for the simulation system to calculate a portion of the virtual vehicle visible from the virtual position of the viewpoint, wherein the means for providing to the display system the portion of the virtual vehicle visible from the virtual position of the viewpoint includes means for the simulation system to output the portion of the virtual vehicle visible from the virtual position of the viewpoint.

[0154] In some aspects, the means for identifying a location of the physical vehicle includes means for receiving a location of each of two portions of the vehicle.

[0155] In some aspects, the means for identifying the location of the physical vehicle includes means for receiving a location of a portion of the vehicle and an orientation of the vehicle. In some aspects, the means for receiving the orientation of the vehicle includes means for receiving gyroscope data. Problems solved by aspects disclosed herein can include how to properly position a physical vehicle in a virtual world to determine visible portions of a virtual vehicle. In some aspects herein, the use of an orientation measure provides accurate placement of the physical vehicle in the virtual world. Greater accuracy provides a more reliable representation of the visible portions of the vehicle, improving the user experience.

[0156] In some aspects, the viewpoint location at the racetrack includes a viewpoint location of an operator of the physical vehicle at the racetrack. In some aspects, the viewpoint location at the racetrack includes a viewpoint location of an audience member present at the racetrack and viewing the physical vehicle at the racetrack. In some aspects, the viewpoint location at the racetrack includes a position of a camera present at the racetrack, and the method further includes imaging the physical vehicle at the racetrack. In some aspects, the camera images a portion of the racetrack where the physical vehicle is racing. When the physical vehicle is traveling across the portion of the racetrack captured by the camera, the camera may capture the physical vehicle in its video feed. When the physical vehicle is not traveling across the portion of the racetrack captured by the camera, the camera may still capture the portion of the racetrack.

[0157] In some embodiments, the means for identifying a position of a viewpoint at the racetrack includes at least one of: a means for measuring an eye gaze point, a means for tracking eye movement, a means for tracking head position, a means for identifying a vector from one or both eyes to a fixed point on the physical vehicle, a means for identifying a vector from a point on the head to a fixed point on the physical vehicle, a means for identifying a vector from a point on the eyewear to a fixed point on the physical vehicle, a means for identifying a vector from a point on the headgear to a fixed point on the physical vehicle, a means for identifying a vector from one or both eyes to a fixed point in the venue, a means for identifying a vector from a point on the head to a fixed point in the venue, a means for identifying a vector from a point on the eyewear to a fixed point in the venue, or a means for identifying a vector from a point on the headgear to a fixed point in the venue. In some embodiments, the means for identifying a position of a viewpoint at the racetrack includes a means for measuring an eye gaze point, and the means for measuring includes a means for measuring a reflection or refraction of light from the eye.

[0158] In some aspects, the means for providing the position of the physical vehicle at the racetrack and the position of the viewpoint includes means for wirelessly transmitting at least one of the positions.

[0159] In some aspects, the means for computing the virtual world includes means for transforming physical coordinates of the physical vehicle into virtual world coordinates, and the virtual position of the physical vehicle includes the virtual coordinates.

[0160] In some aspects, the means for calculating the portion of the virtual vehicle visible from the virtual position of the viewpoint includes means for calculating a representation of the physical vehicle in the virtual world, means for calculating a representation of a physical object in the virtual world between the viewpoint and the virtual vehicle in the virtual world, and means for extracting the portion of the virtual vehicle that is not obscured by the representation of the physical vehicle and the representation of the physical object as seen from the virtual position of the viewpoint. In some aspects, the portion of the virtual vehicle in the virtual world that is visible from the virtual position of the viewpoint includes the portion that is not obscured. Problems solved by aspects disclosed herein can include how to calculate the visible portion of the virtual vehicle, including more than just the portion that is not obscured by the physical vehicle. In some aspects herein, the process of calculating the visible portion in the virtual world including the real-world physical objects increases the accuracy of the visible portion determination. The increased accuracy provides a reliable and repeatable user experience by providing a real-world race including the virtual vehicle. In some aspects herein, the efficiency of the calculation increases by providing the visible portion by virtual calculation. The increased efficiency reduces power usage and increases the speed of the rendering to provide a more seamless user experience.

[0161] In some aspects, the means for extracting the portion of the virtual vehicle may include means for determining which pixels are occluded by other representations and displaying only pixels that are not occluded by other representations. In some aspects, the means for extracting the portion of the virtual vehicle may include means for setting a pixel alpha value of zero percent (in RGBA space) for all pixels that are occluded by other representations. For example, the portion of the virtual vehicle may be occluded by other virtual representations, such as another virtual vehicle, or a representation of a physical object, such as an object in a physical vehicle or the physical vehicle itself. Any visible (from the virtual position of the viewpoint) pixel value may be used to provide the portion of the virtual vehicle that is visible from the virtual position of the viewpoint. In some aspects, the pixels of the non-obscured visible portion of the virtual vehicle may each be set to include an alpha value greater than zero percent (in RGBA space) to indicate that the non-obscured pixels are visible and should be displayed. In contrast, pixels set to an alpha value of zero percent indicate that the pixels are completely transparent, i.e., should not be visible and displayed.

[0162] In some aspects, the means for computing a representation of a physical object between the virtual position of the viewpoint and the representation of the physical vehicle includes means for accessing a database of representations to obtain the virtual position of the physical object.

[0163] In some aspects, the portions of the virtual vehicle that are visible from the virtual position of the viewpoint consist of the portions of the virtual vehicle that are not obscured by other representations in the virtual world.

[0164] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes a virtual shadow in the virtual world. In some aspects, the virtual shadow is at least one of a shadow cast by the virtual vehicle and a shadow cast onto the virtual vehicle. In some aspects, the portion of the virtual vehicle visible from the position of the viewpoint at the racetrack includes a virtual reflection. In some aspects, the virtual reflection is at least one of a reflection of the virtual vehicle and a reflection on the virtual vehicle.

[0165] In some aspects, the means for the simulation system to calculate a portion of the virtual vehicle within the virtual world that is visible from the virtual position of the viewpoint includes means for calculating a field of view from the virtual position of the viewpoint, and the means for providing to the display system the portion of the virtual vehicle that is visible from the virtual position of the viewpoint includes displaying the portion of the virtual vehicle within the field of view.

[0166] In some aspects, the means for the simulation system to calculate a portion of the virtual vehicle within the virtual world that is visible from a virtual viewpoint position includes means for calculating a field of view from the virtual viewpoint position, and the means for providing to the display system the portion of the virtual vehicle that is visible from the virtual viewpoint position comprises means for displaying the portion of the virtual vehicle within the field of view.

[0167] In some aspects, a system for displaying a virtual vehicle includes a first sensor that detects a position of a physical vehicle on a racetrack, a second sensor that detects a position of a viewpoint on the racetrack, and a simulation system that outputs a portion of the virtual vehicle that is visible from a virtual position of the viewpoint. Problems solved by aspects disclosed herein can include overcoming a lack of realism experienced by users of conventional solutions. In some aspects herein, by providing a user with a visible portion of the virtual vehicle, the realism experienced by the user is enhanced. The enhanced realism provides a reliable and repeatable user experience by providing a real-world race involving virtual vehicles.

[0168] In some aspects, the simulation system determines the visible portion of the virtual vehicle based on a virtual position of the physical vehicle in the virtual world, a virtual position of the viewpoint in the virtual world, and a virtual position of the virtual vehicle in the virtual world. Problems solved by aspects disclosed herein can include how to provide the visible portion of the virtual vehicle. In some aspects herein, the visible portion of the virtual vehicle is provided by a virtual calculation of the visible portion, thereby increasing the accuracy of the visible portion determination. The increased accuracy provides a reliable and repeatable user experience by providing a real-world race involving the virtual vehicle. In some aspects herein, the visible portion is provided by a virtual calculation, thereby increasing the efficiency of the computation. The increased efficiency reduces power usage and increases the rendering speed for a more seamless user experience.

[0169] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes the portion of the virtual vehicle that is not obscured by the representation of the physical vehicle at the virtual position of the physical vehicle, as viewed from the virtual position of the viewpoint.

[0170] In some aspects, the system further includes a simulation system configured to simulate an interaction between the virtual vehicle and a representation of the physical vehicle in the virtual world, where a portion of the virtual vehicle visible from a virtual position of the viewpoint is calculated based on the interaction.

[0171] In some aspects, the system includes a simulation system configured to receive a position of a physical vehicle on a racetrack and a position of a viewpoint, compute a virtual world including a virtual vehicle and a representation of the physical vehicle, compute a virtual position of a viewpoint in the virtual world based on the position of the viewpoint on the racetrack, compute a portion of the virtual vehicle visible from the virtual position of the viewpoint, and output to a display system the portion of the virtual vehicle visible from the virtual position of the viewpoint. Problems solved by aspects disclosed herein can include how to compute a visible portion of a virtual vehicle. In some aspects herein, computing a visible portion of a virtual vehicle in a virtual world increases accuracy of the visible portion determination. Higher accuracy provides a reliable and repeatable user experience by providing a real-world race including virtual vehicles. In some aspects herein, providing the visible portion by virtual computation increases computation efficiency. Higher efficiency reduces power usage and increases rendering speed for a more seamless user experience.

[0172] In some aspects, the first sensor receives a position of each of two parts of the vehicle. In some aspects, the first sensor receives a position of one part of the vehicle and an orientation of the vehicle. In some aspects, receiving the orientation of the vehicle includes receiving gyroscope data. Problems solved by aspects disclosed herein can include how to properly position a physical vehicle in a virtual world to determine a visible portion of a virtual vehicle. In some aspects herein, the use of an orientation measure provides accurate placement of the physical vehicle in the virtual world. Higher accuracy provides a more reliable display of the visible portion of the vehicle, improving the user experience.

[0173] In some aspects, the viewpoint positions at the race track include viewpoint positions of an operator of a physical vehicle at the race track, hi some aspects, the viewpoint positions at the race track include viewpoint positions of audience members present at the race track and viewing the physical vehicle at the race track.

[0174] In some aspects, the position of the viewpoint at the race track includes the position of a camera that is at the race track and imaging the physical vehicles at the race track. In some aspects, the camera images the portion of the race track where the physical vehicles are racing. When the physical vehicles are traveling across the portion of the race track captured by the camera, the camera may capture the physical vehicles in its video feed. When the physical vehicles are not traveling across the portion of the race track captured by the camera, the camera may still capture the portion of the race track.

[0175] In some embodiments, the second sensor is configured to detect the position of the viewpoint at the racetrack by at least one of measuring eye gaze points, tracking eye movement, tracking head position, identifying a vector from one or both eyes to a fixed point on the physical vehicle, identifying a vector from a point on the head to a fixed point on the physical vehicle, identifying a vector from a point on the eyewear to a fixed point on the physical vehicle, identifying a vector from a point on the headgear to a fixed point on the physical vehicle, identifying a vector from one or both eyes to a fixed point in the venue, identifying a vector from a point on the head to a fixed point in the venue, identifying a vector from a point on the eyewear to a fixed point in the venue, or identifying a vector from a point on the headgear to a fixed point in the venue. In some embodiments, the second sensor is configured to detect the position of the viewpoint at the racetrack by measuring reflection or refraction of light from the eye.

[0176] In some aspects, receiving the position of the physical vehicle at the racetrack and the position of the viewpoint includes wirelessly receiving at least one of the positions.

[0177] In some aspects, computing the virtual world includes transforming physical coordinates of the physical vehicle into virtual world coordinates, and the virtual position of the physical vehicle includes the virtual coordinates.

[0178] In some aspects, calculating the portion of the virtual vehicle visible from a viewpoint position on the racetrack includes calculating a representation of the physical vehicle in the virtual world, calculating representations of physical objects in the virtual world between the viewpoint and the virtual vehicle in the virtual world, and extracting the portion of the virtual vehicle that is not obscured by the representation of the physical vehicle and the representation of the physical objects as seen from the virtual position of the viewpoint. In some aspects, the portion of the virtual vehicle in the virtual world that is visible from the virtual position of the viewpoint includes the portion that is not obscured. Problems solved by aspects disclosed herein can include how to calculate the visible portion of the virtual vehicle, including more than just the portion that is not obscured by the physical vehicle. In some aspects herein, calculating the visible portion in the virtual world including real-world physical objects increases the accuracy of the visible portion determination. The increased accuracy provides a reliable and repeatable user experience by providing a real-world race including virtual vehicles. In some aspects herein, providing the visible portion by virtual calculation increases the efficiency of the calculation. The increased efficiency reduces power usage and increases the speed of the rendering to provide a more seamless user experience.

[0179] In some aspects, extracting the portion of the virtual vehicle may include determining which pixels are occluded by other representations and displaying only those pixels that are not occluded by other representations. In some aspects, extracting the portion of the virtual vehicle may include setting a pixel alpha value of zero percent (in RGBA space) for all pixels that are occluded by other representations. For example, the portion of the virtual vehicle may be occluded by other virtual representations, such as another virtual vehicle, or a representation of a physical object, such as an object in a physical vehicle or the physical vehicle itself. Any visible (from the virtual position of the viewpoint) pixel value may be used to provide the portion of the virtual vehicle that is visible from the virtual position of the viewpoint. In some aspects, the pixels of the non-obscured visible portion of the virtual vehicle may each be set to include an alpha value greater than zero percent (in RGBA space) to indicate that those non-obscured pixels are visible and should be displayed. In contrast, pixels set to an alpha value of zero percent indicate that those pixels are completely transparent, i.e., should not be visible and displayed.

[0180] In some aspects, computing the representation of the physical object between the virtual position of the viewpoint and the representation of the physical vehicle includes accessing a database of representations to obtain the virtual position of the physical object.

[0181] In some aspects, the portions of the virtual vehicle that are visible from the virtual position of the viewpoint consist of the portions of the virtual vehicle that are not obscured by other representations in the virtual world.

[0182] In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes a virtual shadow in the virtual world. In some aspects, the virtual shadow is at least one of a shadow cast by the virtual vehicle and a shadow projected onto the virtual vehicle. In some aspects, the portion of the virtual vehicle visible from the virtual position of the viewpoint includes a virtual reflection. In some aspects, the virtual reflection is at least one of a reflection of the virtual vehicle and a reflection on the virtual vehicle.

[0183] In some aspects, calculating the portion of the virtual vehicle within the virtual world that is visible from the virtual position of the viewpoint includes calculating a field of view from the virtual position of the viewpoint, and providing the display system with the portion of the virtual vehicle that is visible from the virtual position of the viewpoint includes displaying the portion of the virtual vehicle within the field of view.

[0184] In some aspects, calculating a portion of the virtual vehicle within the virtual world that is visible from a viewpoint position at the racetrack includes calculating a field of view from the virtual position of the viewpoint, and providing the display system with the portion of the virtual vehicle that is visible from the virtual position of the viewpoint comprises displaying the portion of the virtual vehicle that is visible within the field of view.

[0185] In some embodiments, the system may facilitate a race between two virtual vehicles on a physical racetrack. In a scenario where two virtual vehicles compete on a physical racetrack without a physical vehicle, then it may be unnecessary for the first sensor to detect the position of the physical vehicle. The system in such an embodiment may include a sensor that detects the position of a viewpoint on the racetrack and a display system that provides the portion of the virtual vehicle that is visible from the viewpoint position on the racetrack. In such an embodiment, all aspects of the above system that do not relate to the position of the physical vehicle could be applied. In some embodiments, the virtual vehicles are given special attributes and video game appearances. For example, cars can be given boosts, machine guns, missiles (other graphic virtual objects added to the real world view), virtual jumps, etc. In some embodiments, physical vehicles can be given similar video game attributes. For example, graphic virtual objects such as machine guns and missiles can be rendered visible on a display and overlaid on the physical vehicles. Spectators at the racetrack and at home should be able to see the virtual competitors on the physical racetrack as if they were racing in the real world.

[0186] As used herein, a "viewpoint" may be understood to be a real-world location from which a virtual vehicle would be viewed. For example, an operator of a physical vehicle (e.g., a driver within a physical vehicle) viewing his or her surrounding environment. A display may be used to augment the operator's view by introducing the virtual vehicle into the view. Because the virtual vehicle is attached to the real-world viewpoint, if a display system is not provided or is disconnected, the virtual vehicle will not be visible from the real-world viewpoint.

[0187] Figure 1 is a diagram 100 of a physical vehicle 101, according to some embodiments. Figure 1 illustrates an example of the perspective of an operator (e.g., a driver) of the physical vehicle 101. Thus, the location of the operator's perspective is the position and direction (gaze) of the operator's eyes (or some vicinity of the position and direction of the operator's eyes). Although Figure 1 is illustrated with respect to the perspective of an operator of a physical vehicle, these teachings apply equally to other perspectives, such as an audience member of a racetrack where the physical vehicle is traveling, or a camera at the racetrack.

[0188] The physical vehicle 101 includes a display system 102 (including a rendering component 107), a simulation component 106, a telemetry system 104 (including a sensor 108), an RF circuit 105, and a force controller 112. The physical vehicle 101 also includes an eye position detector 110, a windshield 120, a rearview mirror 122, a rear window 124, side windows 126A and 126B, side mirrors 128A and 128B, a seat and head brace 130, a speaker 132, and brakes 134. FIG. 1 also includes a vehicle operator 114. In FIG. 1, the vehicle operator 114 is shown wearing a helmet 116, a visor covering the eyes 117, and a haptic suit 118. In some embodiments, the visor worn over the eyes 117 is a component of the helmet 116.

[0189] As shown in FIG. 1, the physical vehicle 101 is an automobile. In some aspects, devices in the physical vehicle 101 communicate with a simulation system 140 to simulate one or more virtual vehicles in the field of view of the vehicle operator 114. The simulation system 140 may be onboard the physical vehicle 101. In some aspects, as shown in FIG. 100, the simulation system 140 may be remote from the physical vehicle 101, as described elsewhere. In some aspects, functions performed by the simulation system 140 may be distributed across systems both onboard the physical vehicle 101 and remote from the physical vehicle 101. In some aspects, the simulation system 140 generates and maintains a racing simulation 141 between one or more live participants (i.e., the vehicle operator 114 operating the physical vehicle 101) and one or more remote participants (not shown).

[0190] Simulating the virtual vehicle in real time enhances the racing experience of the vehicle operator 114. Executing a simulation capability within the physical vehicle 101 allows the vehicle operator 114, who is a live participant, to compete against remote participants operating virtual vehicles within the racing simulation 141. The field of view of the vehicle operator 114 is the observable world as seen by the vehicle operator 114 augmented with the virtual vehicle. In some aspects, the augmentation can be provided by displays housed in or combined with one or more of the windshield 120, the rearview mirror 122, the rear window 124, the side windows 126A and 126B, and the side mirrors 128A and 128B.

[0191] In some embodiments, the augmentation can be provided by a holographic device or a 3D display system. In these embodiments, the windshield 120, the rearview mirror 122, the rear window 124, the side windows 126A and 126B, or the side mirrors 128A and 128B can be T-OLED displays that allow 3D images to be displayed and utilize cameras to capture the surrounding environment that is displayed with the 3D images overlaid on the non-transparent display.

[0192] In some aspects, the augmentation may be provided by a head mounted display (HMD) worn by the vehicle operator 114 over the eyes 117. The HMD may be worn as part of the helmet 116. In some aspects, the HMD is incorporated into a visor, glasses, goggles, or other device worn in front of the eyes of the vehicle operator 114. Similar to the displays described above, the HMD may operate to augment the field of view of the vehicle operator 114 by rendering one or more virtual vehicles on one or more displays within the HMD.

[0193] In other embodiments, the HMD implements retinal projection techniques to simulate one or more virtual vehicles. For example, the HMD may include a virtual retinal display (VRD) that projects images to the left and right eyes of the vehicle operator 114 to create a three-dimensional (3D) image of one or more virtual vehicles within the field of view of the vehicle operator 114.

[0194] In some aspects, the augmentation can be provided by one or more displays housed in the physical vehicle 101 (e.g., the windshield 120 and rearview mirror 122) as described above, a display worn by the vehicle operator 114 (e.g., an HMD) as described above, a holographic device as described above, or a combination thereof. An advantage of simulating a virtual vehicle on multiple types of displays (e.g., on displays housed in the physical vehicle 101 and on an HMD worn by the vehicle operator 114) is that the augmented reality experience can be maintained when the vehicle operator 114 removes the HMD. Additionally, multiple participants in the physical vehicle 101 can share the augmented reality experience regardless of whether each participant is wearing an HMD.

[0195] In some embodiments, multiple virtual vehicles are simulated for the vehicle operator 114. For example, multiple virtual vehicles are displayed in front of and / or behind and / or to the side of the operator. For example, one or more virtual vehicles may be displayed on the windshield 120 (an example of a display) and one or more virtual vehicles may be displayed on the rearview mirror 122 (an example of a display). In addition, one or more virtual vehicles may also be displayed on the HMD. Similarly, the physical vehicle 101 may be one of multiple physical vehicles in close proximity to each other. In some embodiments, the virtual vehicle being simulated for the vehicle operator 114 may be another physical vehicle running on a physical race track in a different physical location than the one being run by the vehicle operator 114. For example, one driver may operate a vehicle on a race track in Monaco, and another driver may operate a vehicle on a replica race track in Los Angeles. Aspects herein contemplate presenting one or both of the drivers in Monaco and Los Angeles with a virtual vehicle representing the other driver.

[0196] Returning to the simulation system 140, as described above, the simulation system 140 can include a racing simulation 141, which simulates a race between the physical vehicle 101 and one or more virtual vehicles on a virtual racetrack. In some aspects, the virtual racetrack is generated and stored by the simulation system 140 to correspond to a physical racetrack on which the vehicle operator 114 is operating, e.g., driving, the physical vehicle 101. In some aspects, the virtual racetrack is generated using 360-degree laser-scanned video recording or similar technology. Thus, as the vehicle operator 114 controls the physical vehicle 101 in real time on the physical racetrack, the virtual trajectory of the physical vehicle 101 in the racing simulation 141 is simulated by the simulation system 140 to mimic the physical, real-world trajectory of the physical vehicle 101 on the physical racetrack.

[0197] In some aspects, the physical vehicle 101 includes a telemetry system 104 to enable the simulation system 140 to simulate the physical vehicle 101 on the virtual racetrack in the racing simulation 141. The telemetry system 104 includes sensors 108 to detect data associated with the physical vehicle 101. The sensors 108 include one or more devices to detect kinematic information of the physical vehicle 101. In some aspects, the kinematic information includes one or more vectors of motion, one or more scalars of motion, orientation, a Global Positioning System (GPS) position, or a combination thereof. For example, the vectors of motion may include a velocity, a position vector, or an acceleration. For example, the scalars of motion may include a velocity. Thus, the sensors 108 may include one or more accelerometers to detect acceleration, one or more GPS (or GLONASS or other navigation system) receivers to detect a GPS position, one or more motion sensors, one or more orientation sensors, or a combination thereof. In some aspects, the real-time data collected by the sensors 108 is transmitted to the simulation system 140. Other real-time data may include measurements of the vehicle, heat, tire temperature, etc. In some embodiments, one or more of the kinematic information and vehicle measurements are used for simulation predictability. For example, some embodiments may include a predictive simulation engine that pre-builds scenes based on these other measurements and speed and acceleration information.

[0198] In some embodiments, the physical vehicle 101 includes radio frequency (RF) circuitry 105 for transmitting data, e.g., telemetry information generated by the telemetry system 104, to the simulation system 140. The RF circuitry 105 receives and transmits RF signals, also referred to as electromagnetic signals. The RF circuitry 105 converts electrical signals to / from electromagnetic signals and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuitry 105 may include well-known circuits for performing these functions, including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and the like. The RF circuitry 105 may communicate by wireless communication with networks, such as the Internet, also referred to as the World Wide Web (WWW), intranets, and / or wireless networks, such as cellular telephone networks, wireless local area networks (LANs) and / or metropolitan area networks (MANs), and other devices.Wireless communication is a set of standards that includes Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPADA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11c, IEEE 802.11d, IEEE 802.11e, IEEE 802.11f, IEEE 802.11g, IEEE 802.11h, IEEE 802.11h, IEEE 802.11i, IEEE 802.11ih ... Optionally, the communication may use any of a number of communications standards, protocols, and technologies, including, but not limited to, IEEE 802.11g and / or IEEE 802.11n, Wi-MAX, protocols for email (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol.

[0199] In some embodiments, the simulation system 140 includes RF circuitry similar to the RF circuitry 105 for receiving data from the physical vehicle 101. Based on the telemetry information received from the physical vehicle 101, the simulation system 140 simulates the physical vehicle 101 as an avatar in the racing simulation 141. In some embodiments, as further described with reference to FIG. 5, the simulation system 140 receives inputs for controlling and simulating one or more virtual vehicles in the racing simulation. In some embodiments, the simulation system 140 calculates kinematic information of the virtual vehicle based on the received inputs and a current state of the virtual vehicle on the virtual racetrack in the racing simulation 141. For example, the current state may refer to the coordinates, position, speed, velocity, acceleration, orientation, etc. of the virtual vehicle being simulated on the virtual racetrack. To replicate a virtual race between a live participant, i.e., vehicle operator 114, and a virtual participant operating the vehicle operator's 114 virtual vehicle, the simulation system 140 transmits virtual vehicle kinematic information via RF circuitry to components within the physical vehicle 101, such as the display system 102 and simulation components 106. It should be understood that the display system 102 and other components shown in the physical vehicle 101 may be housed in other types of devices, depending on the type of context being simulated, as described elsewhere in this disclosure.

[0200] In some aspects, to enable two-way interactive racing in which interactions simulated in the racing simulation 141 can be replicated for the vehicle operator 114 driving the physical vehicle 101, the simulation system 140 determines whether an avatar of the physical vehicle 101 in the racing simulation 141 is in contact with an obstacle, such as a virtual vehicle, that is being simulated in the racing simulation 141. Upon determining the contact, the simulation system 140 calculates force information, audio information, or a combination thereof, associated with the contact. In some aspects, the simulation system 140 transmits the force information or audio information to the physical vehicle 101, where the force information or audio information is replicated in the physical vehicle 101 to enhance the virtual reality racing experience of the vehicle operator 114.

[0201] Returning to the physical vehicle 101, the physical vehicle 101 includes a display system 102 for generating a representation of the virtual vehicle based on information, such as virtual vehicle kinematics information, received from the simulation system 140 via the RF circuitry 105. In some aspects, the display system 102 is coupled to a simulation component 106 that generates a virtual representation of the virtual vehicle based on the virtual vehicle kinematics information received from the simulation system 140. In some aspects, the simulation component 106 generates the virtual representation based on the kinematics information and ocular measurements (e.g., spatial position) of the eyes 117 of the vehicle operator 114. To further enhance the realism of the virtual representation, i.e., the graphically generated vehicle, the simulation component 106 generates the virtual representation based on the kinematics information, the spatial position of the eyes 117 of the vehicle operator 114, the gaze direction of the eyes 117, and the focus of the eyes 117, according to some aspects. As described herein, the eye measurements may include the spatial position of the eye 117, the gaze direction of the eye 117, the focus of the eye 117, or a combination thereof for the left eye, right eye, or both eyes.

[0202] In some aspects, the physical vehicle 101 includes an eye position detector 110, such as a camera or a light (e.g., infrared) reflectance detector, to detect eye measurements of the eye 117 of the vehicle operator 114 (e.g., the spatial position, gaze direction, focus, or a combination thereof of the eye 117). In some aspects, the eye position detector 110 detects the spatial position of the head of the vehicle operator 114 to estimate the eye 117 measurements. For example, the eye position detector 110 may detect the helmet 116 or a visor of the helmet 116 to estimate the eye 117 measurements. Detecting the eye measurements and / or head position may also include detecting at least one of the position and orientation of the helmet 116 worn by the vehicle operator 114.

[0203] The eye position may be calculated directly (e.g., from fixed sensors such as roadside sensors) or based on a combination of in-vehicle sensors and the vehicle's position.

[0204] In some embodiments, the eye position detector 110 includes a camera capable of recording a real-time video sequence or capturing a series of images of the face of the vehicle operator 114. The eye position detector 110 may then track and detect measurements of the eye 117 by analyzing the real-time video sequence or series of images. In some embodiments, the eye position detector 110 implements one or more algorithms to track the movement or orientation of the head of the vehicle operator 114 to aid in tracking, detecting, and measuring the eye. As shown in FIG. 100, the eye position detector 110 may be coupled to the rearview mirror 122. However, the eye position detector 110 may be located elsewhere, for example, on the dashboard, inside or outside the physical vehicle 101, so long as the eye position detector 110 is implemented in close proximity to the physical vehicle. In some embodiments, to improve detection accuracy, the eye position detector 110 can be implemented in a helmet 116 or other head mounted display (HMD), such as a visor or goggles, worn by the vehicle operator 114 over the eyes 117.

[0205] In some embodiments, the eye position detector 110 implemented in the HMD further includes one or more adjustable focus lenses, one or more mechanically actuated displays, and the eyes are constantly monitored based on where the user looks in the virtual scene, allowing the re-enacted scene to be depicted with a moving eye-tracking technique that actually corrects for common refractive errors in the VR world. An advantage of the above technique is that the vehicle operator 114 does not need to wear contact lenses or corrective glasses while wearing the HMD with the embedded eye position detector 110.

[0206] In some embodiments, the simulation component 106 generates a virtual representation of the virtual vehicle based on a 3D model of the virtual vehicle. For example, the virtual representation may represent a view of the 3D model as seen from the eye 117. In some embodiments, by generating the virtual representation as seen from the eye 117, the measurements of which are detected or estimated by the eye position detector 110, the virtual representation can be simulated with accurate dimensions and scale to increase the realism of racing the virtual vehicle for the vehicle operator 114. In some embodiments, the 3D model may be pre-stored on the simulation component 106 or received from the simulation system 140.

[0207] In some aspects, a rendering component 107 in the display system 102 displays the generated virtual representation on one or more displays of the physical vehicle 101. As described above, the virtual representation may be generated by the simulation component 106 in some aspects and by the simulation system 140 in some aspects. The one or more displays may include windows of the physical vehicle 101, such as the windshield 120 or side windows 126A-B, or mirrors of the physical vehicle 101, such as the rearview mirror 122 or side mirrors 128A-B. In some aspects, the one or more displays may be components within the helmet 116. The helmet 116 may include a helmet, visor, glasses, or goggle system worn by the vehicle operator 114.

[0208] In some embodiments, the display(s) (e.g., windshield 120) may be a transparent organic light emitting diode (T-OLED) display that passes light through a T-OLED to display a field of view to the vehicle operator 114. In these embodiments, the rendering component 107 renders a virtual representation of the virtual vehicle as a layer of pixels on the display(s). The T-OLED display may enable the vehicle operator 114 to see both a simulated virtual vehicle and the physical, unsimulated world within his or her field of view.

[0209] In other embodiments, one or more displays (e.g., windshield 120) may be non-transparent liquid crystal displays (LCDs). In these embodiments, unlike T-OLED displays, LCDs do not allow light through which the vehicle operator 114 can see the physical, non-simulated world in his or her field of view. Instead, in these embodiments, a camera (e.g., a pinhole camera) coupled to the physical vehicle 101 facing outwardly relative to the LCD can record a live video feed representing a physical-world view of the physical, non-simulated world as it would be seen by the eye 117 of the vehicle operator 114 if the LCD were transparent (e.g., a T-OLED display). The rendering component 107 can then display the live video feed inside the LCD to display the view to the vehicle operator 114. Additionally, the rendering component 107 can overlay a generated virtual representation on the live video feed displayed by the LCD to allow the vehicle operator 114 to see the simulated virtual vehicle.

[0210] In some aspects, the non-transparent LCD is not capable of displaying images or live video feeds in 3D by itself. Thus, the camera used to record the live video feed may include one or more cameras that are part of a stereoscopic camera system that records the physical world in color and in 3D. In some aspects, to further enhance the 3D effect of the displayed live video feed, the non-transparent LCD is a multi-view autostereoscopic 3D display, i.e., an automultiscopic display, that allows the vehicle operator 114 to see the displayed 3D video feed from different angles as the vehicle operator 114 moves his / her head and, therefore, his / her eyes 117.

[0211] In high-speed racing events, the head of the vehicle operator 114 moves very little. Therefore, in some embodiments, to reduce some components and complexity in simulating a virtual vehicle with the physical vehicle 101, the eye position detector 110 for tracking the position of the head or eye 117 of the vehicle operator 114 may be omitted from the physical vehicle 101. In some embodiments, the eye position detector 110 for tracking the position of the head or eye 117 of the vehicle operator 114 is implemented to allow the vehicle operator 114 to move his or her head freely. In embodiments where the display is a non-transparent display, the angle of one or more cameras, for example, cameras in a stereo camera system, may be adjusted to correspond to the tracked position of the head or eye 117 of the vehicle operator 114.

[0212] In some aspects, to enable two-way interactive racing in which interactions simulated on a virtual racetrack in the racing simulation 141 can be replicated for the vehicle operator 114 driving the physical vehicle 101, the simulation component 106 determines the proximity of the virtual vehicle to the physical vehicle 101 based on the virtual vehicle kinematics information received from the simulation system 140. In some aspects, upon determining contact between the virtual vehicle and the physical vehicle 101 based on the determined proximity, the simulation system 140 calculates force information, audio information, or a combination thereof associated with the contact. The simulation component 106 may then send the force information to the force controller 112 and / or the audio information to the speaker 132. In some aspects, the audio information may be played by the speaker 132 to mimic the sounds of an engine, brakes, tires, and a collision between the physical vehicle 101 and the virtual vehicle being simulated by the simulation system 140. In some aspects, the audio information may include a volume calculated based on a calculated distance between the physical vehicle 101 and the virtual vehicle on the simulated racetrack and may take into account the head orientation and ear direction of the vehicle operator 114. In some aspects, the speaker 132 may include an audio device (e.g., a speaker or speaker system) equipped on the physical vehicle 101 or an audio device (e.g., headphones or earphones) worn by the vehicle operator 114. In aspects where the speaker 132 is worn by the vehicle operator 114, the speaker 132 may be implemented within a head mounted display, such as the helmet 116.

[0213] For sounds reproduced for observers not in the physical vehicle (e.g., at the racetrack or audience members watching at home), speakers can be placed around the track or through the headgear or audience members or camera viewpoints. In the virtual world, microphone locations can be set just like camera locations. Similarly, sound origin locations can be set. In some aspects, the further the virtual car is from the microphones, the less noise it makes.

[0214] In some embodiments, the force controller 112 controls one or more actuators based on the force information to mimic simulated contact between the physical vehicle 101 and the virtual vehicle on the virtual racetrack simulated by the simulation system 140. For example, the force controller 112 may control one or more force actuators incorporated into the seat and head brace 130 to mimic how the vehicle operator 114 would feel during a collision, e.g., a head bump sensation. Similarly, in some embodiments, the force controller 112 may communicate via wired or wireless communication with the haptic suit 118 worn by the vehicle operator 114 to mimic the sensation of the vehicle operator 114 contacting a real physical vehicle. The force controller 112 may also control one or more force actuators incorporated into the physical vehicle 101 to mimic the physical vehicle 101 contacting a virtual object. In some embodiments, the force controller 112 controls one or more mechanical systems that affect the actual functionality of the physical vehicle 101. For example, the force controller 112 may control one or more brakes 134, steering column, or power, among other mechanical and / or electrical systems, of the physical vehicle 101 to mimic the effects of contact between the physical vehicle 101 and a virtual vehicle or object, such as a virtual vehicle, if the virtual object were actually a physical object at the same race track as the physical vehicle 101. As mentioned above, in some aspects the virtual object being simulated may be a real-world physical vehicle at a race track in a different physical location. Thus, the present disclosure also enables two vehicle operators running at different race tracks to feel as if they are racing at the same race track, since they can simulate each other's physical vehicle as a virtual vehicle.

[0215] In some embodiments involving electric vehicles, mimicking an impact may include controlling power generation for the axel, and for each particular wheel of a car with potentially four electric motors (one per wheel). This may advantageously allow a crash to be simulated by a small spike down in the wheel motors. This can be controlled based on the duration of the impact or other factors.

[0216] 5, some or all of the functionality of the simulation component 106 described above may be performed remotely, for example, by the simulation system 140. In these embodiments, the display system 102 receives the virtual representation generated by the simulation system 140. Relatedly, in these embodiments, the force controller 112 and the speaker 132 may receive force information and audio information, respectively, calculated by the simulation system 140.

[0217] 2A-C are diagrams illustrating how multiple virtual vehicles may be displayed on one or more displays, according to some embodiments. For ease of explanation, FIGs. 2A-C are described with reference to elements of FIG. 1, such as display system 102 and vehicle operator 114. According to some embodiments, the virtual vehicles may be displayed according to an augmented reality aspect or a fully rendered aspect, each of which is further described below.

[0218] In an augmented reality embodiment, the views 202A and 204A of Figure 2A may be augmented with virtual vehicles 228 and 230 being output on displays 220 and 222, respectively, of Figure 2C to allow the vehicle operator 114 to see the portions of the virtual vehicles 212 and 214 shown in the respective views 202B and 204B of Figure 2B. In some embodiments, the displays 220 and 222 may correspond to the windshield 120 and rearview mirror 122 described with reference to Figure 1.

[0219] 2A, view 202A shows the field of view of vehicle operator 114, where a physical vehicle 206A is seen at the physical racetrack via displays 220 and 222. Similarly, view 204A shows empty space, as there are no real vehicles at the physical racetrack as seen by vehicle operator 114 via display 222.

[0220] In some aspects, to simulate one or more virtual vehicles for display on displays 220 and 222, display system 102 identifies one or more locations 224 and 226 on displays 220 and 222, respectively. In some aspects, location 224 may correspond to a physical location 208A in the field of view of vehicle operator 114. For example, location 224 and display 220 may correspond to a portion of a physical racetrack or a portion of a simulated building or landmark in a virtual racetrack in racing simulation 141 of simulation system 140 that is visible to vehicle operator 114 (shown as location 208A in view 202A). Similarly, location 26 may correspond to a different portion of a physical racetrack or a portion of a building or landmark that is visible to vehicle operator 114 (shown as location 210A in view 202A).

[0221] In some aspects, the simulation component 106 may generate the first virtual vehicle 228 as a first representation and the second virtual vehicle 230 as a second representation, as described with reference to FIG. 1. In some aspects, the simulation component 106 may generate the first virtual vehicle 228 based on the position 224, the measurements of the eye 117 (described in connection with the eye position detector 110), and the kinematics of the first virtual competitor vehicle. As shown on the display 220, the rendering component 107 displays the first virtual vehicle 228 aligned with the position 224. Similarly, the rendering component 107 may display the second virtual vehicle 230 aligned with the position 226 on the display 222.

[0222] As a result, the vehicle operator 114 sees physical objects and virtual objects as shown in views 202B and 204B. For example, like view 202A, view 202B shows that the vehicle operator 114 still sees the physical vehicle 206B. However, view 202B shows that the vehicle operator 114 sees a virtual object, such as virtual vehicle 212, displayed as virtual vehicle 228 on the display 220. Similarly, view 204B shows that the vehicle operator 114 sees a virtual object, such as virtual vehicle 214, displayed as virtual vehicle 230 on the display 222. Further aspects are described with reference to FIG.

[0223] In a full rendering embodiment, the displays 220 and 222 can be configured to render both physical and virtual objects for display so that the vehicle operator 114 can see the virtual vehicle along with the physical vehicle. In this embodiment, the rendering component 107 can render and display physical objects such as a road and a physical vehicle 206A as shown in views 202A and 204A of FIG. 2A. As described above in connection with the augmented reality embodiment, the simulation component 106 can generate a first virtual vehicle 228 as a first representation and a second virtual vehicle 230 as a second representation for display. In a full rendering embodiment, the displays 220 and 222 can be configured to display the virtual vehicles 228 and 230 along with the physical objects, respectively, as shown in views 202B and 204B visible to the vehicle operator 114. For example, view 202B shows a road, a physical vehicle 206B, and a virtual vehicle 212 being rendered and displayed. Similarly, view 204B shows a virtual vehicle 214 being rendered and displayed. In some embodiments, in full rendering embodiments, an outward-facing camera (relative to displays 220 and / or 222) can capture a live video feed of the surrounding environment. In these embodiments, displays 220 and 222 can be configured to display physical objects by displaying each frame of the live video feed. Additionally, displays 220 and 222 can be configured to display virtual objects by overlaying a virtual vehicle on each displayed frame.

[0224] 3A-D are diagrams illustrating how a visible portion 322 of a virtual vehicle and a visible portion 324 of a virtual vehicle are displayed on a display 320, according to some embodiments. For ease of explanation, the diagram 300 is described with reference to elements of FIG. 1 (e.g., display system 102, vehicle operator 114, simulation system 140). FIG. 3A illustrates an exemplary real-world view 302A as seen by a vehicle operator 114 via a conventional display. FIG. 3B illustrates an exemplary virtual rendering 332 of the real-world view 302A, including virtual vehicles 343 and 344. FIG. 3C illustrates an example of a display 320 for displaying the visible portions 322 and 324 of the virtual vehicles 343 and 344, respectively. The display 320 may correspond to a display implemented in a visor, helmet (e.g., helmet 116), or other head gear worn by an operator (e.g., vehicle operator 114) seated in and driving a physical vehicle (e.g., physical vehicle 101). FIG. 3D illustrates an example augmented view 302B as seen by the vehicle operator 114 via the display 320.

[0225] As shown in FIG. 3A, the real world view 302A shows the field of view of the vehicle operator 114 when the virtual vehicle is not displayed, i.e., through a conventional display. As shown in the real world view 302A, the vehicle operator 114 can see other physical vehicles, such as the physical vehicle 310A at the physical race track, and physical objects within the physical vehicle 101 through the display 320. For example, such physical objects may include the rearview mirror 304A, the vehicle frame 306A, the windshield wipers 308A, the dashboard 312A, etc. In some aspects, the physical objects may include the hands and arms of the vehicle operator 114. In addition, the vehicle operator 114 may also see a shadow of the physical vehicle 310A as shown in the real world view 302A. In some aspects, the vehicle operator 114 can see the physical vehicles and physical objects through the display 320, since the display 320 may be a transparent or semi-transparent display.

[0226] 1, kinematic information (e.g., position information) of the physical vehicle 101 and the position of the operator's viewpoint may be transmitted to a simulation system 140 configured to provide visible portions of the virtual vehicles 313 and 314. In some aspects, based on the kinematic information and the position of the operator's viewpoint, the simulation system 140 may compute a virtual world that includes representations of the virtual vehicles and the physical vehicle 101 racing against each other on a virtual race track that corresponds to the physical race track visible to the operator 114. In some aspects, the simulation system 140 may compute representations of various physical objects within the virtual world.

[0227] In some aspects, the vehicle operator 114 may wear gloves incorporating one or more sensors (e.g., accelerometers, position sensors, etc.) that transmit position-related measurements to the simulation system 140 such that the simulation system 140 can track and calculate a representation of the hand or arm of the vehicle operator 114. Based on the sensor measurements, e.g., position and acceleration information, the simulation system 140 may calculate a corresponding representation of the arm or hand (not shown) in the virtual world.

[0228] In some aspects, one or more cameras may be mounted on a physical vehicle being operated by vehicle operator 114 such that simulation system 140 can track and calculate representations of the hands or arms of vehicle operator 114. The one or more cameras may track the position of the arms and hands based on markers incorporated into or displayed on gloves or a tracksuit worn by vehicle operator 114. For example, the markers may include a particular color, pattern, material, etc. In these aspects, the one or more cameras may transmit the captured information to simulation system 140, which calculates a corresponding representation of the arm or hand (not shown) in the virtual world.

[0229] 3B, the simulation system 140 may compute a virtual rendering 332 of the real-world view 302A within the virtual world. In the virtual rendering 332, the simulation system 140 may compute a representation 340 of the physical vehicle 310A and virtual vehicles 343 and 344. In addition, the simulation system 140 may also compute representations 334, 336, 338, and 342 of the corresponding physical objects, namely, the rearview mirror 304A, the vehicle frame 306A, the windshield wipers 308A, and the dashboard 312A. As shown in the virtual rendering 332, the simulation system 140 may exclude computing representations of physical objects that do not obstruct the view of the virtual vehicles 343 and 344. For example, the speedometer and steering wheel visible to the vehicle operator 114 in the real-world view 302A may not be computed by the simulation system 140 in the virtual rendering 332. In some aspects, as shown in virtual rendering 332, simulation system 140 can calculate shadows for physical vehicle 340 and virtual vehicles 343 and 344.

[0230] In some aspects, the simulation system 140 can calculate the portions 322 and 324 of the virtual vehicle to be displayed on the display 320 of FIG. 3C to enable the vehicle operator 114 to compete with the virtual driver in the real world. In some aspects, the visible portion of the virtual vehicle from the vehicle operator's 114 viewpoint position is the portion of the virtual vehicle that is not obstructed by objects in the virtual world from a virtual location in the virtual world that corresponds to the viewpoint position of the physical vehicle 101. In some aspects, the simulation system can convert the viewpoint position of the vehicle operator 114 to virtual coordinates in the virtual world. For example, from the viewpoint of the physical vehicle operator, the corresponding virtual location in the virtual world should be inside the representation of the physical vehicle 101 in the virtual world. From a corresponding virtual position of the viewpoint of the physical operator 114 in the virtual world, the view of the virtual vehicle may be obstructed by simulated physical vehicles (e.g., representations of the vehicle frame 306A and windshield wipers 308A), other simulated physical vehicles (e.g., representations of the physical vehicle 310A), shadows, simulated trees and other stationary objects, the simulated race track (e.g., when the virtual vehicle is in a dip and partially obstructed by the track itself, etc.). The visible portion of the virtual vehicle is then an unobstructed view of the virtual vehicle. Further aspects are described with reference to Figures 6 and 9.

[0231] For example, as shown in virtual rendering 332 of FIGURE 3B, the simulated view of virtual vehicle 343 shows portions of virtual vehicle 343 being occluded by representation 336 of vehicle frame 306A. Similarly, in virtual rendering 332, the simulated view of virtual vehicle 343 shows portions of virtual vehicle 344 being occluded by representation 340 of physical vehicle 310A and representation 338 of windshield wiper 308A.

[0232] In some aspects, simulation system 140 may calculate visible portion 322 of virtual vehicle to be the portion of virtual vehicle 343 in virtual rendering 332 that is not obscured in the virtual world by representation 336 of car frame. Similarly, simulation system 140 may calculate visible portion 324 of virtual vehicle to be the portion of virtual vehicle 344 in virtual rendering 332 that is not obscured by representation 340 of physical vehicle 310A and representation 338 of windshield wiper 308A, respectively. In some aspects, information related to these calculated visible portions 322 and 324 may be transmitted to components within physical vehicle 101 and displayed by display 320.

[0233] In some aspects, as shown in Figure 3C, components within the physical vehicle 101, such as the display system 102, can display visible portions 322 and 324 of virtual vehicles 343 and 344, respectively, on the display 320. In some aspects, the visible portions 322 and 324 can include shadows of the virtual vehicles 343 and 344, which are calculated and shown in the virtual rendering 332 of Figure 3B. In some aspects, by augmenting the real-world view 302A with the visible portions 322 and 324 displayed on the display 320, the display system 102 allows the vehicle operator 114 to see both the real-world physical vehicle and the virtual vehicle.

[0234] In some aspects, the extended view 302B of FIG. 3D shows the field of view of the vehicle operator 114 at the time the display 320 displayed the visible portions 322 and 324 of the virtual vehicle (e.g., as rendered by the display system 102). For example, similar to the view 302A, the vehicle operator 114 still sees various physical objects at the real-world racetrack through the display 320. For example, as shown in the extended view 302B, the vehicle operator 114 still sees the rearview mirror 304B, the vehicle frame 306B, the windshield wipers 308B, the physical vehicle 310B, and the dashboard 312B. In addition, the vehicle operator 114 also sees the displayed virtual vehicles 313 and 314. In some aspects, as described above, the virtual vehicles 313 and 314 seen by the vehicle operator 114 correspond to the visible portions 322 and 324, respectively, of the virtual vehicle displayed on the display 320. In some embodiments, the techniques described with reference to Figures 2A-C may be combined with the techniques described with reference to Figures 3A-D.

[0235] 4A-D are diagrams illustrating how a visible portion 422 of a virtual vehicle may be displayed on a display 420, according to some embodiments. FIG. 4A illustrates an example real-world view 402A that may be visible to a spectator or imaged by a video camera. FIG. 4B illustrates an example virtual rendering 430 of the real-world view 402A that includes a virtual vehicle 434. FIG. 4C illustrates an example display 420 for displaying the visible portion 422 of the virtual vehicle 434. The display 420 may correspond to a visor, helmet (e.g., helmet 116), or other head gear worn by a spectator (e.g., an audience member) present at a physical racetrack. FIG. 4D illustrates an example augmented view 402B that may be visible to an audience member via the display 420 or imaged by a video camera.

[0236] As shown in FIG. 4A, a real-world view 402A shows a view of a spectator when no virtual vehicle is displayed, i.e., through a conventional display. As shown in the real-world view 402A, the spectator sees other spectators 404A and various physical objects in the real world through the display 420. For example, such physical objects may include a fence 406A, a public address (PA) horn speaker 408A, and physical vehicles 410A and 412A, and shadows 411A and 413A of the respective physical vehicles 410A and 412A. In some aspects, the spectator sees the physical vehicles and physical objects through the display 420, since the display 420 may be a transparent or semi-transparent display.

[0237] In some aspects, in addition to fixed objects such as fences and walls, the above-mentioned physical objects that may block a spectator's view of the racetrack may include non-stationary objects whose positions may change over time. For example, such non-stationary objects may include an audience member's head when the audience member is standing or an audience member's body. In some aspects, to allow the simulation system 140 to accurately calculate a representation of both fixed and non-stationary objects, the spectator's headset may include a camera that faces the racetrack and captures a portion of the racetrack. To determine whether one or more physical objects block a spectator's view of the racetrack, the camera may detect lines and boundaries of the racetrack or other markers disposed on the racetrack. For example, the camera may detect missing or gaps in edges, lines, or markers of the racetrack. In some aspects, information regarding the gaps or missing portions may be transmitted to the simulation system 140. In some aspects, the simulation system 140 may determine which portions of the virtual vehicle are blocked by the physical objects by determining an overlapping portion of the virtual vehicle with one or more gaps. Because discontinuities indicate that the spectator's view is obstructed by a physical object, the simulation system 140 can set the alpha values ​​of the pixels in the overlapping areas to "zero percent" (in RGBA), making these overlapping areas transparent.

[0238] In some aspects, information related to the spectator's viewpoint may be transmitted to a simulation system (e.g., simulation system 140 or simulation system 540 of FIG. 5) configured to provide the spectator with a visible portion 422 of the virtual vehicle. For example, such information may include the location of the spectator's viewpoint. As described with reference to FIG. 6, the simulation system may compute a virtual world that includes representations of virtual and physical vehicles racing against each other on a virtual racetrack. In some aspects, the simulation system 140 may compute representations of various physical objects (e.g., PA speaker 408A) within the virtual world.

[0239] As shown in FIG. 4B, the simulation system may compute a virtual rendering 430 of the real-world view 402A within the virtual world. In the virtual rendering 430, the simulation system may compute a virtual vehicle 434 and representations 440 and 432 of the physical vehicles 410A and 412A. In some aspects, the virtual vehicle 434 computed by the simulation system may include a computed shadow 435. Similarly, the representations 440 and 432 of the physical vehicles 410A and 412A may also be computed to include respective shadows 441 and 433. In some aspects, the simulation system computes shadows 441 and 433 in the virtual world to correspond to respective shadows 411A and 413A as seen by an onlooker in the real-world view 402A. In addition, the simulation system may also compute representations 436 and 438 of the corresponding physical objects, a fence 406A and a PA speaker 408A. As shown in virtual rendering 430, the simulation system may exclude calculation of representations of physical objects that do not obstruct the virtual vehicle's view 422. For example, the audience (e.g., spectators 404A) in the real-world view 402A may not be calculated by the simulation system in virtual rendering 430.

[0240] In some aspects, the simulation system can calculate the portion 422 of the virtual vehicle to be displayed on the display 420 of FIG. 4C to allow a spectator to see a race between the physical vehicles 410A and 412A and a virtual vehicle, such as the virtual vehicle 434, that is simulated in the virtual world. In some aspects, the visible portion of the virtual vehicle from the spectator's viewpoint position is the portion of the virtual vehicle that is not obstructed by objects in the virtual world from a virtual location in the virtual world corresponding to the spectator's viewpoint position. In some aspects, the simulation system can convert the spectator's viewpoint position to virtual coordinates in the virtual world. As seen from the corresponding virtual location of the spectator's viewpoint in the virtual world, the view of the virtual vehicle may be obstructed by the simulated physical vehicle (e.g., the representation of the physical vehicles 410A and 412A) as well as other simulated objects (e.g., the fence 406A and the horn speaker 408A), simulated trees, the simulated racetrack (e.g., when the virtual vehicle is in a hollow and partially obstructed by the track itself, etc.). The visible portion of the virtual vehicle is then an unobstructed view of the virtual vehicle. Further embodiments are described with reference to FIGS.

[0241] For example, the simulated view of the virtual vehicle 434 as shown in the virtual rendering 432 of FIG. 4B shows that portions of the virtual vehicle 434 are occluded by the representation 438 of the PA speaker 408A and the representation 432 of the physical vehicle 412A.

[0242] In some aspects, the simulation system may calculate the visible portion 422 of the virtual vehicle to be that portion of the virtual vehicle 434 that is not obscured by the representation 438 of the PA horn speaker in the virtual rendering 430 and that is not obscured by the representation 432 of the physical vehicle in the virtual world. In some aspects, information related to the calculated visible portion 422 may be transmitted to an onlooker and displayed by the display 420.

[0243] In some aspects, as shown in FIG. 4C, a component worn by a spectator, such as the display system 592 of FIG. 5, can display a visible portion 422 of a virtual vehicle 434 on a display 420. As shown in FIG. 4C, the visible portion 422 can include a frame and detail portion 424 of the car and a shadow 426 of the virtual vehicle 434. In some aspects, the visible portion 422 can include a shadow 428 that is calculated in the virtual rendering 430 and projected onto the virtual vehicle 434. For example, the shadow 428 can be a shadow cast by the representation 432 of the physical vehicle 412A that is calculated in the virtual rendering 430. In some aspects, by augmenting the real world view 402A with the visible portion 422 displayed on the display 420, the display system (e.g., the display system 592) allows the spectator to see both the real world physical vehicle and the virtual vehicle.

[0244] In some aspects, the extended view 402B of FIG. 4D shows a spectator's field of view as the display 420 displays the visible portion 422 of the virtual vehicle (e.g., as rendered by the display system 592). For example, similar to the view 402A, the spectator still sees various physical objects at the real-world racetrack through the display 420. For example, as shown in the extended view 402B, the spectator still sees other spectators 404B, fences 406B, public address (PA) horn speakers 408B, and physical vehicles 410B and 412B, and their respective shadows 411B and 413B. In addition, the spectator also sees the virtual vehicle 414 and the shadow 415 of the displayed virtual vehicle 414. In some aspects, as described above, the virtual vehicle 414 seen by the spectator may correspond to the visible portion 422 of the virtual vehicle displayed on the display 420. In the example of the augmented view 402B, the virtual vehicle 414 as seen by the spectator is occluded by the PA horn speaker 408B and the physical vehicle 412B. In some aspects, the displayed virtual vehicle 414 may overlap a portion of the shadow 413A as seen by the spectator in the real-world view 402A. As a result, a portion of the shadow 413A may be obscured by the visible portion 422 of the virtual vehicle, as shown by the shadow 413B in the augmented view 402B. In some aspects, the visible portion 422 of the virtual vehicle may include the shadow 428 of the physical vehicle 412A. In these aspects, the spectator may be able to see the shadow 416 being projected onto the virtual vehicle 414, as shown in the augmented view 402B.

[0245] In some aspects, the augmented view 402B may be provided to a spectator via a non-transparent display based on the full rendering technique described with reference to FIGS. 2A-C. In these aspects, a camera coupled to the spectator's headset may capture one or more video frames of the spectator's field of view as seen in the real-world view 402A. The simulation system may similarly compute the virtual vehicle 434 in the virtual rendering 430. However, in these aspects, instead of displaying only the visible portion 422 of the virtual vehicle on the display 420, the display implemented in the spectator's headset may be configured to output one or more captured video frames with the visible portion 422 of the virtual vehicle overlaid. In some aspects, physical objects (e.g., fences, beams, infrastructure, etc.) may obstruct the spectator's view of the racetrack. Because the augmented view 402B may be a full rendering view, the display may overlay both the visible vehicle and a representation of the physical vehicle (computed by the simulation system) on top of the captured video frames such that large physical objects do not obstruct the spectator's view of the race for an extended period of time.

[0246] In some aspects, as further described with reference to FIGS. 5, 6 and 9, the real world view 402A may correspond to a live video feed captured by a camera (e.g., video camera 580 of FIG. 5) installed at a physical racetrack. In these aspects, information related to the camera's viewpoint may be transmitted to the simulation system. For example, such information may include one or more of the camera's position, orientation, tilt, or rotation degree. Based on the camera's viewpoint information, the simulation system may calculate the virtual world to include a virtual rendering 430 including the virtual vehicle, a representation of the physical vehicle, and a representation of the physical objects, as described with reference to FIG. 4B. In some aspects, based on the virtual rendering 430, the simulation system may calculate a visible portion 422 of the virtual vehicle that should be visible at the camera's viewpoint. As described with reference to FIG. 4C, the visible portion 422 of the virtual vehicle may include the car's frame and details 424, the virtual vehicle's shadow 426, or a shadow 428 being cast by other objects in the virtual rendering 430. In some aspects, the visible portion 422 of the virtual vehicle is shown in the augmented view 402B and may be overlaid on a live video feed of the real world view 402A to display a race between the physical vehicle and the virtual vehicle, as described above with reference to FIG. 4D.

[0247] 5 is a system 500 for simulating a virtual race between a physical vehicle 510 and a simulated entity 530, according to some embodiments. In some embodiments, a network 502 communicatively couples various components, namely, the physical vehicle 510, the simulated entity 530, the simulation system 540, the viewing device 560, the content distribution system 570, the camera 580, and the spectators 590. In some embodiments, the camera 580 can be coupled to the content distribution system 570. As shown in the system 500, the network 502 can be a conduit for data flow between the various components. The network 502 can be a wired and / or wireless network, including any combination of a local area network (LAN), a wide area network (WAN), the Internet, and the like.

[0248] In some aspects, simulation system 540 includes a number of engines 552 that act on model 542 and information received from physical vehicles 510 and simulated entities 530 to simulate a virtual race, represented as racing simulation 550. Additionally, simulation system 540 includes RF circuitry 548, which may include components similar to RF circuitry 105 described with reference to FIG. 1 for communicating data (e.g., graphics data, kinematics information, force information, audio information, etc.) with physical vehicles 510, spectators 590, and simulated entities 530.

[0249] In some aspects, the model 542 includes a vehicle model 544 and a racetrack model 546. The vehicle model 544 may include 3D models of animate or inanimate objects in the virtual environment of the racing simulation 550. For example, the vehicle model 544 may include a 3D model of the physical vehicle 510 and a 3D model of a virtual vehicle corresponding to the simulated entity 530. The racetrack model 546 may include a 2D or 3D model of the physical racetrack on which the physical vehicle 510 is operating. In some aspects, the 2D or 3D model may include information related to the terrain, boundaries, or topological features, etc. The racetrack model 546 may include the racetrack and associated features (e.g., terrain, material type, length, etc.), grandstand seating, etc.

[0250] In some embodiments, to generate and maintain the racing simulation 550, the engine 552 includes a physics engine 554, a graphics engine 556, and an audio engine 558. The physics engine 554 may include algorithms for realistically mimicking the laws of physics within the racing simulation 550. In particular, the physics engine 554 includes algorithms that control how components, such as simulated physical vehicles and simulated virtual vehicles, interact with each other and with the virtual racetrack in the racing simulation 550. In some embodiments, the physics engine 554 generates and maintains the racing simulation 550 based on kinematic information received from the physical vehicles 510 and inputs received from the simulated entities 530, as described elsewhere herein. As further described with reference to FIG. 6 and FIG. 9, according to some embodiments, the kinematic information may include position information of the physical vehicles 510. For example, the physics engine 554 may generate an avatar of the physical vehicles 510 in the racing simulation 550 based on a corresponding model in the vehicle model 544, and the position of the avatar in the racing simulation 550 may be calculated based on the received kinematic information. In addition, physics engine 554 may also generate kinematics information for a virtual vehicle corresponding to simulated entity 530 based on input received from simulated entity 530. Using the generated kinematics information and vehicle model 544, physics engine 554 may simulate the virtual vehicle on a virtual racetrack in racing simulation 550.

[0251] In some aspects, to enable the physical vehicle 510 to simulate one or more virtual vehicles on the display 512, the simulation system 540 sends virtual vehicle kinematics information to the simulation component 522 or the display system 514.

[0252] 1, physics engine 554 (within simulation system 540) further calculates force information based on the simulated interactions between the physical and virtual vehicles in racing simulation 550. In some aspects, physics engine 554 calculates force information for physical vehicle 510 and force information for simulated entity 530. Simulation system 540 may then transmit the calculated force information via RF circuitry 548 to physical vehicle 510, simulated entity 530, or both.

[0253] In some aspects, to enhance the realism of the race experienced between the physical vehicles 510 and the simulated entities 530, an audio engine 558 (in the simulation system) includes algorithms for calculating sounds in the racing simulation 550. The audio engine 558 may include sound files related to engines, tires, explosions, as well as collisions between the vehicles. In some aspects, the audio engine 558 calculates the volume of the generated sounds based on the distance between the vehicles calculated by the physics engine 554 to generate the racing simulation 550. The audio engine 558 may then transmit the audio information to the physical vehicles 510, the simulated entities 530, or both.

[0254] In some aspects, the graphics engine 556 generates 3D animated graphics for the racing simulation 550. For example, the graphics engine 556 may use dedicated hardware to render vehicles (e.g., avatars for the physical vehicles 510 and virtual vehicles corresponding to the simulated entities 530) based on the vehicle model 544 and calculations from the physics engine 554. Additionally, the graphics engine 556 may render virtual race tracks in the racing simulation 550 based on the race track model 546. In some aspects, the graphics information (e.g., vehicles and race tracks) generated by the graphics engine 556 may be transmitted via the RF circuitry 548 to the physical vehicles 510, the simulated entities 530, or a combination thereof.

[0255] The graphics engine 556 may utilize techniques such as rasterization and ray tracing to generate 3D animated graphics. In some aspects, the graphics engine 556 comprises a computer software application programmed and compiled to run on one or more processors of the simulation system 540. In other aspects, the graphics engine 556 may be built with a graphics application programming interface (API) such as Direct3D or Open GL.

[0256] In some aspects, the physical vehicle 510 corresponds to the physical vehicle 101 described with reference to FIG. 1. To simulate a virtual vehicle corresponding to the simulated entity 530 in the physical vehicle 510, the physical vehicle 510 includes one or more of the following components: a display 512, a display system 514, a telemetry system 520 (which may include sensors 516), a force controller 518, and a simulation component 522. These components may correspond to similarly named components described with reference to FIG. 1. In general, the telemetry system 520 may be a two-way telemetry system that receives and transmits data. For example, the telemetry system 520 may transmit data monitored by the sensors 516, e.g., kinematic information of the physical vehicle 510, to the simulation system 540 via the network 502. In some aspects, the data includes a position of the physical vehicle 510 on a physical racetrack. In some aspects, the data captured by the sensors 516 may include a position of an operator's viewpoint of the physical vehicle 510, as further described with reference to FIG. 6 and FIG. 9.

[0257] In some aspects, the telemetry system 520 can receive kinematic information of the virtual vehicle from the simulation system 540 as described above. The telemetry system 520 can forward the received kinematic information to the display system 514. Based on the received kinematic information, the display system 514 can generate a virtual representation of the virtual vehicle that is processed by the display system 514 for display on the display 512. By simulating the virtual vehicle within the operator's field of view of the physical vehicle 510, the system 500 allows the operator to feel as if the simulated vehicle is physically close to the operator. As described above, the processing for generating the virtual representation of the virtual vehicle can be performed remotely, for example, offloaded to the simulation system 540. In these aspects, the virtual representation can be generated by the simulation system 540 and transmitted to the display system 514. The display system 514 can then display the virtual representation on the display 512 to simulate the virtual vehicle within the operator's field of view.

[0258] In some embodiments, to enhance the realism of the virtual representation, as further described with reference to Figures 6 and 9, the generated virtual representation may include portions of the virtual vehicle that are visible from the operator's viewpoint position. In some embodiments, the simulation system 540 may generate the visible portions. In some embodiments, a simulation component 522 on board the physical vehicle 510 may generate the visible portions.

[0259] In some aspects, to further enhance the realism of the race experienced by the operator of physical vehicle 510, force controller 518 receives force information to control one or more mechanical elements within physical vehicle 510. In some aspects, the force information calculated by physics engine 554 is received from simulation system 540, as described above. In other aspects, force calculations may be performed on-board physical vehicle 510.

[0260] In some aspects, the spectators 590 may correspond to audience members watching a race between the physical vehicle 510 and the simulated entity 530. To simulate a virtual vehicle for display to the spectators 590, the spectators 590 may wear or operate one or more devices implementing one or more of the following components: a display 594, a display system 592, a telemetry system 596 (which may include sensors 598), and simulation components 599. These components may correspond to similarly named components described with reference to the physical vehicle 510. In general, the telemetry system 596 transmits limited kinematic information of the spectators 590 detected by the sensors 598. For example, the limited kinematic information may include a position (e.g., a GPS position) of the spectators 590. Because the spectators 590 are likely to be associated with limited movement, according to some aspects, other types of kinematic information associated with the movement may not be captured by the sensors 598. In some aspects, similar to sensor 516, sensor 598 may include a camera for capturing the position of the viewpoint of spectator 590. In some aspects, display 594 may be implemented in one or more devices worn by spectator 590. For example, display 594 may be implemented in a headset (e.g., a helmet) or visor worn by spectator 590. In some aspects, spectator 590 may wear or operate a device that implements simulation component 599. Similar to the functionality of simulation component 522, simulation component 599 may process some of the calculations performed by simulation system 540.

[0261] In some aspects, the simulation system 540 communicates with a content delivery system 570 to display races between live and simulated participants, e.g., physical and virtual vehicles, to an audience via viewing devices 560. For example, the virtual vehicles may be overlaid onto actual video footage as viewed from a video camera 580 at the physical racetrack, and the combined video footage may be shown on one or more viewing devices 560 such that the audience sees the physical and virtual vehicles at the same racetrack competing.

[0262] In some aspects, content distribution system 570 includes video server 572 for broadcasting video content over a cable or television network and web server 574 for transmitting video content on demand or in live streaming over network 502, such as the Internet. As described above, video server 572 can broadcast video content captured by video camera 580. In some aspects, multiple video cameras may be present at a physical racetrack to record live video footage of the race from different viewpoints. In these aspects, video server 572 can select live video footage captured by one video camera (e.g., video camera 580) from among the multiple video cameras. In some aspects, each video camera includes a unique viewpoint, and each viewpoint is used to determine the visible portion of the virtual vehicle to combine with the live image feed from each video camera.

[0263] Although video server 572 and web server 574 are shown as being implemented by content distribution system 570, one or more of these servers (e.g., video server 572 and web server 574) may be implemented by separate entities or by simulation system 540.

[0264] In some embodiments, the viewing devices 560 include various electronic devices with displays for presenting video data. For example, audience members attending a live event / competition may watch the competition on viewing devices 560 including television (TV) screens, jumbotrons, etc. In another example, at-home audience members may operate viewing devices 560 such as TVs, laptops, tablets, smartphones, desktop computers, etc., among other mobile devices. Additionally, in some embodiments, both at-home and at-live competition audience members may wear viewing devices 560 such as HMDs or goggles that will display the combined scene (e.g., including live and virtual participants) as seen by that audience member based on location-based information of where that audience member is located, head and eye spatial and directional measurements, or from a remote server that streams display information to the audience member's HMD or portable computer / mobile device, where the camera and audience member positions are registered, that accurately recreates the scene using processors built into the HMD, on portable computers or mobile devices carried by the audience members, and location-based information of where that audience member is located.

[0265] In some embodiments, simulated entity 530 includes a simulation device 532 coupled to a display 534 and input controllers 536 for controlling a virtual vehicle in a virtual race. In some embodiments, simulation device 532 includes a force controller 538 and a display system 539. Simulation device 532 may be a general purpose computer or a special purpose computer such as a video game console.

[0266] In some embodiments, the input controller 536 may include a keyboard, a video game controller, a joystick, a steering wheel and pedals, a force pad, a treadmill, a steering wheel, among other types of input devices for controlling a virtual vehicle in a virtual race.

[0267] In some aspects, the simulation device 532 receives input from the input controller 536 and transmits the received input to the simulation system 540. In some aspects, the simulation system 540 simulates the virtual vehicle at the virtual racetrack based on the input. The simulation system 540 may then transmit display information corresponding to the portion of the virtual vehicle at the virtual racetrack. A display system 539 in the simulation device 532 may receive the display information and render the virtual race on the display 534 as computer generated imagery (CGI). In some aspects, the display system 539 projects the virtual race on the display 534.

[0268] As shown in system 500, display 534 may include a television screen, monitor, projector, or other device for displaying graphics data. In some aspects, display 534 may include a head mounted display (HMD) worn by a user of input controller 536, as described with reference to the display in FIG. 1. For example, an HMD may include a visor, a headset (e.g., a helmet), glasses, goggles, or other device worn in front of the user's eyes.

[0269] In other embodiments, the HMD implements retinal projection techniques to simulate one or more virtual vehicles. For example, the HMD may include a virtual retinal display (VRD) that projects images to the left and right eyes of the vehicle operator 114 to create a three-dimensional (3D) image of one or more virtual vehicles within the field of view of the vehicle operator 114.

[0270] In some embodiments, the force controller 538 receives force information from the simulation system 540. The force information may be associated with interactions, such as collisions, simulated by the simulation system 540 between the virtual vehicle and a physical vehicle at the virtual racetrack. To enhance the virtual racing experience for the simulated entity, the force controller 538 may provide feedback to the input controller 536, for example, by vibrating the input controller 536. In some embodiments, a user manipulating the input controller 536 may wear a haptic suit that includes one or more actuators controlled by the force controller 538 to mimic the physical sensations that would be felt by the user in an actual collision.

[0271] FIG. 6 is a flow diagram illustrating a method 600 for displaying a virtual vehicle on a display, according to some aspects. In some aspects, the method 600 includes steps performed by a physical vehicle 602, a simulation system 604, and a spectator 606. For example, the steps performed by the physical vehicle 602 may be performed by components within a physical vehicle, such as the physical vehicle 101 of FIG. 1 or the physical vehicle 510 of FIG. 5. For example, the steps performed by the simulation system 604 may be performed by the simulation system 140 of FIG. 1 or the simulation system 540 of FIG. 5. For example, the steps performed by the spectator 606 may be performed by a device (e.g., the components shown in spectator 590 of FIG. 5) worn by an audience member watching a race between the physical vehicle 602 and a virtual vehicle at a physical racetrack.

[0272] At step 610, the physical vehicle 602 identifies a position of the physical vehicle 602. In some aspects, the physical vehicle 602 may identify the position of the physical vehicle 602 by detecting kinematic information of the physical vehicle 602 via one or more sensors (e.g., sensor 108 of FIG. 1 or sensor 516 of FIG. 5) mounted on the physical vehicle 602. In some aspects, the position of the physical vehicle 602 includes position information for each of two portions of the physical vehicle 602. For example, the position information for the first portion of the physical vehicle 602 may be detected by a GPS sensor disposed on the first portion. In some aspects, the position of the physical vehicle 602 includes a position of one portion of the physical vehicle 602 and an orientation of the physical vehicle 602. In some aspects, the orientation of the physical vehicle 602 may include gyroscope data detected by a sensor (e.g., a gyroscope) mounted on the physical vehicle 602.

[0273] In step 611 , the physical vehicle 602 provides the position of the physical vehicle 602 to the simulation system 604 .

[0274] At step 620, the simulation system 604 receives inputs to control the virtual vehicle. In some embodiments, the inputs may be received from an input controller 536 as described with reference to FIG.

[0275] At step 621, the simulation system 604 calculates a virtual world for simulating a race between the virtual vehicle and the physical vehicle at the virtual racetrack based on the input from the physical vehicle 602 and the input controlling the virtual vehicle. In some aspects, the virtual world can be implemented in a racing simulation (e.g., racing simulation 141 of FIG. 1 or racing simulation 550 of FIG. 5). In some aspects, the input from the physical vehicle 602 can include a position of the physical vehicle 602 provided at step 611. In some aspects, to generate the racing simulation, the simulation system 604 can calculate a representation of a physical vehicle to be added to the virtual racetrack in the racing simulation based in part on the position provided at step 611. In some aspects, to generate the racing simulation, the simulation system 604 can calculate a virtual vehicle to be added to the virtual racetrack in the racing simulation based on the input at step 620. In some aspects, the simulation system 604 can use the input to update kinematic information associated with the virtual vehicle in the virtual world. In some aspects, the kinematic information includes one or more vectors of motion, one or more scalars of motion, a position vector, a GPS position, a velocity, an acceleration, an orientation, or a combination thereof of the virtual vehicle. According to some aspects, based on the updated kinematic information, the simulation system 604 can update the simulation of the virtual vehicle in the virtual world.

[0276] At step 612, the physical vehicle 602 identifies a position of an operator's viewpoint of the physical vehicle 602. In some aspects, the position of the operator's viewpoint may be determined relative to the operator's head detected by a sensor (e.g., a camera) in the physical vehicle 602. For example, the position of the viewpoint may include detecting a spatial position of the operator's head. For example, in some aspects, the position of the operator's viewpoint may be determined by at least one of tracking the head position, identifying a vector from a point on the head to a fixed point on the physical vehicle, identifying a vector from a point on the headgear to a fixed point on the physical vehicle, identifying a vector from a point on the head to a fixed point in the venue, or identifying a vector from a point on the headgear to a fixed point in the venue. In some aspects, the venue may include a physical racetrack, grandstands, or other infrastructure at a physical racetrack.

[0277] In some aspects, the location of the operator's eye gaze can be determined relative to the operator's eyes detected by a sensor (e.g., a camera) in the physical vehicle 602. For example, the location of the eye gaze can include detecting the spatial location of the user's eyes, the gaze direction of the user's eyes, or the focus of the user's eyes when the user is the operator. For example, in some aspects, the location of the operator's eye gaze can be determined by at least one of measuring the eye gaze point, tracking eye movement, identifying a vector from one or both eyes to a fixed point on the physical vehicle, identifying a vector from a point on the eyewear (e.g., a visor) to a fixed point in the venue, identifying a vector from one or both eyes to a fixed point on the racetrack, or identifying a vector from one or both eyes to a fixed point in the venue. In some aspects, the venue may include a physical racetrack, grandstands, or other infrastructure at the physical racetrack. In some aspects, the location of the operator's eye gaze can be identified by measuring the reflection or refraction of light from the eyes.

[0278] At step 613, the physical vehicle 602 provides the position of the operator's viewpoint to the simulation system 604. In some aspects, the physical vehicle 602 wirelessly transmits the position of the viewpoint to the simulation system 604. In some aspects, the transmission can be performed by a telemetry system coupled to the physical vehicle 602 (e.g., telemetry system 104 of FIG. 1 ).

[0279] At step 622, the simulation system 604 calculates a first portion of the virtual vehicle that is visible from the operator's viewpoint position. In some embodiments, as further described with reference to FIG. 9, the simulation system 604 calculates the first portion by determining which portions of the virtual vehicle are not obscured by a representation of a physical vehicle (relative to the virtual position in the racing simulation generated at step 621). In particular, the simulation system 604 may determine the virtual position to correspond to the viewpoint position provided by the physical vehicle 602 as described in step 613. In some embodiments, the simulation system 604 calculates the first portion by determining which portions of the virtual vehicle are not obscured by representations of physical objects in the racing simulation. In some embodiments, as described with reference to FIGS. 3A-D, portions of the virtual vehicle in the racing simulation may be occluded by representations of other physical or virtual vehicles relative to the virtual position of the operator's viewpoint. For example, a representation of a physical object may obscure a portion of the virtual vehicle from the virtual position of the viewpoint if the representation of the physical object is positioned on a straight line between the virtual position of the viewpoint and the occluded portion of the representation of the physical object. In some aspects, as described above, the first portion calculated by the simulation system 604 can include unobstructed portions of the virtual vehicle. In some aspects, as described above, the first portion can exclude occluded portions of the virtual vehicle.

[0280] In some aspects, the first portion of the virtual vehicle can include one or more virtual shadows that are generated within the virtual world. In some aspects, the one or more virtual shadows can include a virtual shadow of the virtual vehicle, a virtual shadow projected onto the virtual vehicle, or both a virtual shadow of the virtual vehicle and a virtual shadow projected onto the virtual vehicle. For example, the virtual shadow projected onto the virtual vehicle can include a virtual shadow of another virtual vehicle, a virtual shadow of a representation of a physical vehicle in the virtual world, or a virtual shadow of another virtual object that is generated in the virtual world.

[0281] In some aspects, the first portion of the virtual vehicle may include a virtual representation generated by the simulation system 604 (e.g., graphics engine 556). In some aspects, the virtual representation includes a set of graphic elements. In some aspects, the virtual representation may be generated by the simulation system 604 based on a digital 3D model of the virtual vehicle stored in a database of models (e.g., vehicle model 544).

[0282] At step 623, the simulation system 604 outputs the first portion calculated at step 622 to the physical vehicle 602. In some aspects, the simulation system 604 wirelessly transmits the first portion to the physical vehicle 602 (e.g., via RF circuitry 548 of FIG. 5).

[0283] At step 614, the physical vehicle 602 provides the first portion to a display system (e.g., display system 102 of FIG. 1 or display system 514 of FIG. 5). In some aspects, the first portion received from the simulation system 604 may include kinematic information calculated by the simulation system 604. In some aspects, the first portion received from the simulation system 604 may include graphics information. In some aspects, a telemetry system coupled to the physical vehicle 602 (e.g., telemetry system 104 of FIG. 1) may receive information related to the first portion of the virtual vehicle.

[0284] At step 615, the physical vehicle 602 displays a first portion of the virtual vehicle on a display (e.g., display 512 of FIG. 5) proximate to the physical vehicle 602. In some embodiments, a display system (e.g., rendering component 107 of FIG. 1) renders the first portion of the virtual vehicle on the display. In some embodiments, a rendering component in the display system (e.g., rendering component 107 of FIG. 1) converts the first portion of the virtual vehicle into a virtual representation for display on the display. In some embodiments, the virtual representation includes a set of graphic elements. In some embodiments, the display system displays a series of representations of the virtual vehicle over a period of time (each representation including the visible portion of the virtual vehicle output at step 623) by repeating one or more steps of method 600 (e.g., steps 610-615 and 620-623) to simulate the trajectory of the virtual vehicle around a racetrack within the operator's field of view.

[0285] In some aspects, the display system includes a simulation component (e.g., simulation component 106) that generates the virtual representation. In some aspects, the virtual representation is generated based on a digital 3D model of the virtual vehicle. In some aspects, the digital 3D model is stored in a memory of the display system. The digital 3D model may be received, for example, from a simulation system.

[0286] In some aspects, the display includes one or more windows of the physical vehicle 602. In some aspects, the display may include one or more windows or mirrors of the physical vehicle 602, such as any of the displays of the windshield 120, the rearview mirror 122, the rear window 124, the side windows 126A and 126B, and the side mirrors 128A and 128B, as described with reference to FIGURE 1. In some aspects, the display may be implemented within a visor or headset worn by the operator physical vehicle 602 (e.g., helmet 116 of FIGURE 1).

[0287] In some embodiments, simulation system 604 performs steps similar to steps 622 and 623 to allow other spectators, such as spectator 606, to view the virtual vehicle from other perspectives. In some embodiments, spectator 606 may be an audience member at a live racing event watching a race between physical vehicle 602 at a physical racetrack and a virtual vehicle that is not physically present at the physical racetrack.

[0288] Although illustrated together in one system, in some embodiments, one of the first visible portion and the second visible portion is calculated and output without calculating and outputting the other portion.

[0289] At step 630, the display system of the spectator 606 receives a selection for the second viewpoint. In some aspects, the selection may be the viewpoint of the spectator 606. For example, the spectator 606 may be an audience member present at the physical racetrack and viewing the physical vehicle 602 at the racetrack. In some aspects, the selection may be the viewpoint of a video camera present at the physical racetrack and imaging the portion of the racetrack on which the physical vehicle 602 is racing. When the physical vehicle 602 is traveling across the portion of the racetrack captured by the video camera, the video camera may image the physical vehicle 602 on the video feed. When the physical vehicle is not traveling across the portion of the racetrack captured by the camera, the camera may still capture the portion of the racetrack. In some aspects, the selection for the second viewpoint may default to the viewpoint of the spectator 606 or the video camera.

[0290] In step 631, the display system of the spectator 606 identifies a location of a second viewpoint. In some aspects where the second viewpoint is the viewpoint of the spectator 606, the location of the second viewpoint may be determined relative to the head of the spectator 606 detected by a sensor (e.g., sensor 598) proximate to the spectator 606. For example, in some aspects, the location of the second viewpoint may be determined by at least one of tracking the head position of the spectator 600, identifying a vector from a point on the head to a fixed point in the venue, identifying a vector from a point on the headgear to a fixed point in the venue. In some aspects, the venue may include a physical racetrack, grandstands, or other infrastructure at a physical racetrack.

[0291] In some embodiments, the second viewpoint is the viewpoint of the spectator 606. In some embodiments, the location of the second viewpoint can be determined relative to the eye of the spectator 606 detected by a sensor (e.g., sensor 598) proximate to the spectator 606. For example, the location of the second viewpoint can include detecting the spatial location of the user's eye, the gaze direction of the user's eye, or the focus of the user's eye when the user is the spectator 606. For example, in some embodiments, the location of the second viewpoint can be determined by at least one of measuring the gaze point of the eye, tracking eye movement, identifying a vector from a point on the eyewear (e.g., a visor) to a fixed point in the venue, or identifying a vector from one or both eyes to a fixed point in the venue. In some embodiments, the venue may include a physical racetrack, grandstands, or other infrastructure at the physical racetrack. In some embodiments, the location of the second viewpoint can be identified by measuring the reflection or refraction of light from the eye of the spectator 606.

[0292] In step 632 , the display system of the spectator 606 provides a second viewpoint location to the simulation system 604 .

[0293] At step 624, the simulation system 604 calculates a second portion of the virtual vehicle that is visible from the second viewpoint location. In some embodiments, as further described with reference to FIG. 9, the simulation system 604 calculates the second portion by determining what portions of the virtual vehicle are not obscured by the representation of the physical vehicle relative to the virtual location in the racing simulation generated at step 621. In particular, the simulation system 604 may determine the virtual location to correspond to the second viewpoint location provided by the display system of the spectator 606 as described at step 632.

[0294] At step 625, the simulation system 604 outputs the second portion calculated at step 624 to a display system of the spectator 606. In some embodiments, the simulation system 604 wirelessly transmits the second portion to the display system of the spectator 606 (e.g., via RF circuitry 548 of FIG. 5).

[0295] At step 633, a wireless interface proximate to the spectator 606 provides the second portion to a display system of the spectator 606. In some aspects, the second portion received from the simulation system 604 may include kinematic information calculated by the simulation system 604. In some aspects, the first portion received from the simulation system 604 may include graphical information. In some aspects where the display system of the spectator 606 includes a wireless interface, the display system of the spectator 606 may receive the second portion directly.

[0296] At step 634, the display system of the spectator 606 renders the second portion of the virtual vehicle on a display proximate to the spectator 606. In some aspects, the display system of the spectator 606 renders the second portion of the virtual vehicle on a display proximate to the spectator 606. In some aspects, the display proximate to the spectator 606 can be implemented in a visor or helmet worn by the spectator 606.

[0297] In some aspects, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device with a display, the one or more programs including instructions for performing any of the steps described above with reference to FIG. 6. In some aspects, the non-transitory computer-readable storage medium includes computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps described with reference to FIG. 6. In some aspects, a system includes at least one of the aforementioned non-transitory computer-readable storage mediums and one or more processors configured to execute instructions of the non-transitory computer-readable storage medium(s). In some aspects, a device includes one or more processors configured to perform any of the steps described above with reference to FIG. 6.

[0298] 7 is a flow diagram illustrating a method 700 for providing a physical vehicle operator with two-way interaction between a virtual vehicle and a physical vehicle, according to some aspects. Method 700 may be implemented by components within a physical vehicle, such as, for example, physical vehicle 101 of FIG. 1. In some aspects, method 700 enhances method 600 to provide haptic and audio feedback in addition to the visual feedback displayed and described with reference to method 600.

[0299] In step 702, a telemetry system (e.g., telemetry system 104) monitors kinematic information of a physical vehicle (e.g., physical vehicle 101). In some aspects, the telemetry system includes one or more sensors that detect the kinematic information. For example, the one or more sensors may include a GPS sensor, an accelerometer, a speedometer, an orientation sensor, a gyroscope, among other sensor types.

[0300] At step 704, the telemetry system transmits the kinematic information to a simulation system (e.g., simulation system 140). In some embodiments, the kinematic information is transmitted via an RF circuit (e.g., RF circuit 105).

[0301] In some aspects, the simulation system simulates a virtual race between a virtual vehicle and a physical vehicle in a virtual world based on the telemetered kinematic information. In some aspects, the virtual world includes a virtual racetrack where the virtual vehicle and a representation of the physical vehicle are simulated at the virtual racetrack. In some aspects, the simulation system calculates a distance between the virtual vehicle and the physical vehicle at the virtual racetrack. Based on the calculated distance, the simulation system determines whether contact (e.g., a collision) exists between the virtual vehicle and the physical vehicle at the virtual racetrack. The simulation system then calculates force information corresponding to the determined contact.

[0302] In some aspects, the simulation system calculates audio information based on the calculated distance and whether contact exists. In some aspects, the audio information includes one or more of engine, brake, tire, explosion, or explosion sounds, and one or more volume levels. For example, the simulation system may calculate a volume of the one or more sounds to be inversely proportional to the calculated distance between the virtual vehicle and the physical vehicle on the virtual racetrack.

[0303] At step 706, a force controller (e.g., force controller 112) receives the force information from a simulation system. For example, a display system (e.g., display system 102) receives the force information and forwards it to the force controller. In some aspects, some or all of the functionality of calculating the force information may be performed in a simulation component (e.g., simulation component 106) in the physical vehicle. In these aspects, the simulation component receives kinematic information of the virtual vehicle or other virtual object, as described in connection with step 816 of FIG. 8. The simulation component may then perform force calculations to generate the force information.

[0304] At step 708, the display component receives the audio information from the simulation system. In some aspects, some or all of the functionality of computing the audio information may be performed in the simulation component in the physical vehicle. In these aspects, the simulation component receives kinematics information of the virtual vehicle or other virtual object, as described in connection with step 816 of FIG. 8. The simulation component may then perform the audio calculations to generate the audio information.

[0305] At step 710, the force controller controls one or more mechanical elements implemented in the physical vehicle based on the received force information. In some aspects, the force controller sends commands to one or more force actuators (e.g., examples of mechanical elements) to mimic the physical sensations that would be felt by an operator of the physical vehicle if actual physical contact occurred between the physical vehicle and another vehicle displayed as a virtual representation of the virtual vehicle. In some aspects, the one or more force actuators may be implemented in a seat and head brace (e.g., seat and head brace 130) or in a haptic suit worn by the operator (e.g., haptic suit 118).

[0306] In some embodiments, the mechanical elements may include parts that affect the functionality of the physical vehicle. For example, the mechanical elements may include a steering wheel column, brakes, airbags, etc. Based on the received force information, the force controller may, for example, lock the brakes, deploy airbags, vibrate the steering wheel column, create a banging force on a section of the vehicle, slow the car down by reducing power, or control other mechanical and / or electrical elements in the physical vehicle.

[0307] At step 712, the display system may control one or more speakers of the physical vehicle (e.g., speaker 132) to output audio information. In some aspects, the display system may control one or more speakers in a helmet worn by an operator (e.g., helmet 116) to output audio information.

[0308] In some aspects, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device with a display, the one or more programs including instructions for implementing any of the steps described above with reference to FIG. 7. In some aspects, the non-transitory computer-readable storage medium includes computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps described above with reference to FIG. 7. In some aspects, a system includes at least one of the aforementioned non-transitory computer-readable storage mediums and one or more processors configured to execute instructions of the non-transitory computer-readable storage medium(s). In some aspects, a device includes one or more processors configured to perform any of the steps described above with reference to FIG. 7.

[0309] Figure 8 is a flow diagram illustrating a method 800 for simulating a race between a virtual vehicle and a physical vehicle to provide two-way interaction, according to some embodiments. The method 800 may be implemented by a simulation system, such as, for example, the simulation system 140 described with reference to Figure 1 or the simulation system 540 described with reference to Figure 5. As described with reference to Figures 1 and 5, the simulation system simulates a virtual race between a virtual vehicle and a physical vehicle at a virtual race track in a simulated virtual world, where the virtual race track corresponds to a physical race track.

[0310] At step 802, the simulation system receives input from a controller (e.g., input controller 536) to control a virtual vehicle on a virtual racetrack. In some embodiments, the controller can be a keyboard, a mouse, a video game controller, a joystick, a steering wheel and pedals, gestures on a touch screen, or a combination thereof, among other types of input devices.

[0311] In step 804, the simulation system receives kinematics information for a physical vehicle (e.g., physical vehicle 101 of FIG. 1). In some aspects, the kinematics information is received from a physical vehicle as described with reference to FIG.

[0312] At step 806, the simulation system simulates a virtual race between the virtual vehicle and the physical vehicle at the virtual racetrack. In some aspects, the simulation system simulates the virtual race according to one or more of steps 808-812. At step 808, the simulation system determines kinematic information of the virtual vehicle based on the input received at step 802. For example, the input may include an amount of force applied to a video game controller that is converted by the simulation system into an amount of acceleration. At step 810, the simulation system determines an interaction between the virtual vehicle and the physical vehicle at the virtual racetrack by comparing the kinematic information between the virtual vehicle and the physical vehicle. In some aspects, the simulation system determines a distance between the simulated virtual vehicle and the physical vehicle at the virtual racetrack to determine whether contact (e.g., a collision) occurs. At step 812, the simulation system generates force information based on the interaction determined at step 810. At step 813, the simulation system generates audio information based on the interaction determined at step 810.

[0313] At step 816, the simulation system sends the virtual vehicle kinematics information to the physical vehicle. In some aspects, the physical vehicle uses the virtual vehicle kinematics information to generate and display the virtual vehicle on a display of the physical vehicle.

[0314] At step 818, the simulation system sends the force information to the physical vehicle as described with reference to step 706 of Figure 7. In some embodiments, the physical vehicle controls one or more mechanical or electrical elements of the physical vehicle based on the force information to mimic the physical sensations that would be felt by an operator of the physical vehicle in an actual collision.

[0315] At step 820, the simulation system sends the generated audio information to the physical vehicle as described with reference to step 708 of Figure 7. In some aspects, the physical vehicle controls one or more speakers of the physical vehicle based on the audio information to mimic the auditory experience that would be felt by an operator of the physical vehicle if the virtual vehicle were physically present on a physical racetrack. For example, the one or more speakers may include speakers of the vehicle or speakers implemented within a headset worn by the operator.

[0316] At step 814, the simulation system renders the virtual race on a simulation display (eg, display 534 of FIG. 5).

[0317] In some aspects, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for performing any of the operations described above with reference to FIG. 8. In some aspects, the non-transitory computer-readable storage medium includes computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the operations described above with reference to FIG. 8. In some aspects, a system includes at least one of the aforementioned non-transitory computer-readable storage mediums and one or more processors configured to execute instructions of the non-transitory computer-readable storage medium(s). In some aspects, a device includes one or more processors configured to perform any of the operations described above with reference to FIG. 8.

[0318] 9 is a flow diagram illustrating a method 900 performed by a simulation system to facilitate display of a virtual vehicle, according to some embodiments. Method 900 may be performed, for example, by simulation system 140 described with reference to FIG 1 or simulation system 540 described with reference to FIG 5. In some embodiments, one or more steps of method 900 may correspond to one or more steps performed by simulation system 604 described with reference to FIG 6.

[0319] At step 902, the simulation system receives input to control the virtual vehicle. For example, the input may be received from the input controller 536 as described with reference to Figure 5. In some aspects, step 902 corresponds to step 620 of Figure 6.

[0320] At step 904, the simulation system receives a position of the physical vehicle. In some aspects, the physical vehicle may provide the position of the physical vehicle, for example, as described with reference to step 611 of FIG.

[0321] At step 906, the simulation system calculates a virtual world for simulating a race between the virtual vehicle and the physical vehicle at the virtual racetrack. In some aspects, the virtual world may be a racing simulation stored in racing simulation 550 described in FIG. 5. In some aspects, step 906 corresponds to step 621 of FIG. 6. In some aspects, to calculate the virtual world, method 900 performs steps 908-912.

[0322] At step 908, the simulation system simulates the virtual vehicle on a virtual race track in the virtual world based on the input received at step 902. In some aspects, the input controls kinematic characteristics that define how the virtual vehicle moves on the virtual race track in the virtual world.

[0323] At step 910, the simulation system calculates a representation of the physical vehicle in the virtual world based on the parts of the physical vehicle received at step 904. In some aspects, the simulation system can simulate a race between a physical vehicle and a virtual vehicle by adding the representation of the physical vehicle to the virtual world. In some aspects, to calculate the representation of the physical vehicle, the simulation system transforms physical coordinates associated with the parts of the physical vehicle into virtual coordinates in the virtual world.

[0324] At step 912, the simulation system calculates a representation of a plurality of objects in the virtual world. In some aspects, the representation of the object (from the plurality of representations) corresponds to a physical object present at a physical race track modeled in the virtual world. For example, a virtual race track in the virtual world may be simulated based on a physical race track that may include physical objects such as trees, banners, pit stops, etc. In some aspects, the representation of the object (among the plurality of representations) corresponds to a virtual object present at the virtual race track simulated in the virtual world but not at the physical race track. For example, the virtual object may include, but is not limited to, a simulated obstacle, smoke, a wall, an explosion, or debris resulting from a collision between a virtual vehicle simulated in the virtual world and a physical vehicle.

[0325] In some aspects, the virtual world simulated by the simulation system may include computed representations of a plurality of physics objects, hi some aspects, the simulation system may compute the representations of the plurality of physics objects by accessing a database of representations.

[0326] At step 914, the simulation system receives a position of a viewpoint at the racetrack. In some aspects, this position may be received from a physical vehicle, as described with reference to step 613 of FIG. 6. In these aspects, this position represents a viewpoint position of an operator of the physical vehicle. In some aspects, this position may be received from a spectator, as described with reference to step 632 of FIG. 6. In these aspects, this position represents a viewpoint position selected by the spectator.

[0327] At step 916, the simulation system calculates the portion of the virtual vehicle visible from the viewpoint location received at step 914. In some embodiments, step 916 corresponds to steps 622 or 621 described with reference to FIG. 6 based on the source of the viewpoint location received at step 914. In some embodiments, calculating the portion includes calculating a field of view from the virtual position of the viewpoint. In these embodiments, the calculated portion may be within the calculated field of view. In some embodiments, to calculate the portion of the virtual vehicle visible from the viewpoint, method 900 includes steps 918-926.

[0328] At step 918, the simulation system calculates a virtual position of the viewpoint within the virtual world based on the viewpoint position received at step 914. In some aspects, to calculate the virtual position, the simulation system transforms the physical coordinates of the viewpoint position into virtual coordinates within the virtual world.

[0329] At step 920, the simulation system determines whether representations of one or more objects exist in the virtual world between the virtual location and the virtual vehicle. In some aspects, the representations of the one or more physics objects are selected from among the representations of the plurality of physics objects calculated at step 912. As described above with reference to step 912, the representations of the one or more objects may include virtual representations of physics objects present at the physical racetrack. In some aspects, the representations of the one or more objects may include virtual objects simulated in the virtual world that do not exist at the physical racetrack. At step 922, if representations of the one or more objects exist, the method 900 proceeds to step 926. Otherwise, the method 900 proceeds to step 924.

[0330] In step 924, the simulation system extracts the portions of the virtual vehicle as seen from the virtual position that are not obscured by the representation of the physical vehicle in the virtual world.

[0331] At step 926 , the simulation system extracts from the virtual location the portions of the virtual vehicle that are not occluded by the representation of the physical vehicle and the representation of the one or more objects determined at step 920 .

[0332] At step 928, the simulation system provides a portion of the virtual vehicle visible from a virtual position of view that includes one or more of the extracted portions. In some aspects, the output portion includes only the extracted portions. In some aspects, the simulation system may calculate a field of view from a virtual position of view, as described above. In these aspects, the portion calculated by the simulation system may include a non-excluded portion that represents a portion of the portion that is visible in the calculated field of view, and an excluded portion that represents a portion of the portion that is excluded (not visible) in the calculated field of view. In some aspects where the field of view is calculated by a simulation system, the simulation system may calculate the portion to include only the non-excluded portion that represents a portion of the portion that is visible in the calculated field of view.

[0333] In some aspects, the simulation system provides the portions of the virtual vehicle to the source of the viewpoint position as described with reference to step 914. For example, step 928 may correspond to step 623 or 625 as described with reference to Figure 6 depending on the source of the viewpoint position as described with reference to step 914. In this example, if the viewpoint position is received from a physical vehicle, the simulation system may provide the portions of the virtual vehicle to the physical vehicle as described with reference to step 623 of Figure 6.

[0334] In some embodiments, the virtual world may be a racing simulation stored in racing simulation 550 depicted in Figure 5. In some embodiments, step 906 corresponds to step 621 of Figure 6. In some embodiments, to compute the virtual world, method 600 performs steps 908-912.

[0335] In some aspects, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for performing any of the operations described above with reference to FIG. 9. In some aspects, the non-transitory computer-readable storage medium includes computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the operations described above with reference to FIG. 9. In some aspects, a system includes at least one of the aforementioned non-transitory computer-readable storage mediums and one or more processors configured to execute instructions of the non-transitory computer-readable storage medium(s). In some aspects, a device includes one or more processors configured to perform any of the operations described above with reference to FIG. 9.

[0336] 10 is a flow diagram illustrating a method 1000 for enabling the display of a virtual vehicle, according to some aspects. Method 1000 may be implemented by components within a physical vehicle, such as, for example, physical vehicle 101 of FIG. 1 or physical vehicle 510 of FIG. 5. In some aspects, by simulating a virtual vehicle within an operator's field of view of a physical vehicle on a racetrack, method 1000 enhances the realism of the interaction between the physical vehicle and the virtual vehicle experienced by the operator with respect to the virtual vehicle.

[0337] In step 1002, a sensor in the physical vehicle (e.g., eye position detector 110) detects eye measurements of an operator (e.g., vehicle operator 114) of the physical system (e.g., physical vehicle 101). In some aspects, the sensor estimates the eye measurements of the eyes based on detection of the operator's head or a device worn on the operator's head (e.g., helmet 116, a visor covering eyes 117, or a head mounted display (HMD)). For example, the sensor may estimate the eye measurements of the operator's eyes based on detection of the position and / or orientation of a device worn on the operator's head.

[0338] In step 1004, a display system (e.g., rendering component 107) in the physical vehicle identifies a location of a physical object within the operator's field of view (e.g., location 208A in FIG. 2). In some aspects, this location corresponds to a location on a display (e.g., display 220 in FIG. 2) proximate to the physical system.

[0339] At step 1006, the display system receives kinematics information for a virtual vehicle that represents a competitor vehicle that is not physically on a racetrack. Additionally, the display system may receive information from a simulation system (e.g., simulation system 140) associated with virtual objects in racing simulation 550 or racing simulation 141 that are not physically on a racetrack. In some aspects, the kinematics information may include GPS coordinates, spatial position, orientation, velocity, acceleration, or combinations thereof associated with the virtual vehicle, as described with reference to FIG. 1. In some aspects, the kinematics information may be received from a simulation system (e.g., simulation system 140) that simulates a race between a physical vehicle and a virtual vehicle on a simulated racetrack.

[0340] At step 1008, the display system generates a representation of the virtual vehicle based on the positions of the physical objects identified at step 1004, the eye measurements detected at step 1002, and the kinematic information received at step 1006. In some embodiments, the display system includes a simulation component (e.g., simulation component 106) that generates this representation. Additionally, in embodiments in which the display system receives information about other virtual objects as described at step 1006, the display system generates a graphic representation of those virtual objects as well. For example, the virtual objects may include walls, debris from the virtual car, or objects of a virtual racetrack being simulated in racing simulation 550. In some embodiments, the virtual representation is generated based on a digital 3D model of the virtual vehicle. In some embodiments, the digital 3D model is stored in a memory of the display system. The digital 3D model may be received, for example, from a simulation system.

[0341] At step 1010, a display system (e.g., rendering component 107) displays a representation of the virtual vehicle on a display aligned with the physical object represented at the location identified in step 1004. In some aspects, a rendering component in the display system (e.g., rendering component 107 of FIG. 1 ) converts this representation into a set of graphical elements for display on the display. In some aspects, the display system displays a series of representations of the virtual vehicle over a period of time by repeating one or more steps of method 1000 (e.g., steps 1002-1010) to simulate the trajectory of the virtual vehicle on a racetrack within the operator's field of view.

[0342] In some aspects, the representation may be generated remotely, for example, by a simulation system. In these aspects, the display system may receive information related to the representation as generated by the simulation system. Further, the rendering component may convert this received information into a set of graphical elements for display on a display.

[0343] In some aspects, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device with a display, the one or more programs including instructions for performing any of the steps described above with reference to FIG. 10. In some aspects, the non-transitory computer-readable storage medium includes computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps described above with reference to FIG. 10. In some aspects, a system includes at least one of the aforementioned non-transitory computer-readable storage mediums and one or more processors configured to execute instructions of the non-transitory computer-readable storage medium(s). In some aspects, a device includes one or more processors configured to perform any of the steps described above with reference to FIG. 10.

[0344] An example computer according to one embodiment is shown in Figure 11. The computer 1100 may be a component of a system for simulating a virtual vehicle on a display according to the systems and methods described above, such as a device within the physical vehicle 101 or simulation system 140 described with reference to Figure 1, or may comprise an entire system in itself. In some embodiments, the computer 1100 is configured to execute a method for enhancing virtual racing between physical and virtual vehicles, such as each of methods 600, 700, 800, 900, and 1000 of Figures 6, 7, 8, 9, and 10, respectively.

[0345] The computer 1100 may be a host computer connected to a network. The computer 1100 may be a client computer or a server. As shown in FIG. 11, the computer 1100 may be any suitable type of microprocessor-based device, such as a personal computer, a workstation, a server, a video game console, or a handheld computing device such as a phone or tablet. The computer may include, for example, one or more of a processor 1110, an input device 1120, an output device 1130, a storage device 1140, and a communication device 1160. The input device 1120 and the output device 1130 may generally correspond to those described above and may be connectable to or integrated with the computer.

[0346] Input device(s) 1120 may be any suitable device for providing input, such as a touch screen, monitor, keyboard, mouse, voice recognition device, etc. Output device(s) 1130 may be any suitable device for providing output, such as a touch screen, monitor, printer, disk drive, or speaker.

[0347] The storage device 1140 may be any suitable device providing storage, such as an electrical, magnetic, or optical memory, including RAM, cache, hard drive, CD-ROM drive, tape drive, or removable storage disk. The communication device 1160 may include any suitable device capable of sending and receiving signals over a network, such as a network interface chip or card. The components of the computer may be connected in any suitable manner, such as via a physical bus or wirelessly. The storage device 1140 may be a non-transitory computer-readable storage medium containing one or more programs that, when executed by one or more processes, such as the processor 1110, cause the one or more processors to perform the methods described herein, such as each of methods 600, 700, 800, 900, and 1000 of FIGS. 6, 7, 8, 9, and 10, respectively.

[0348] The software 1150 may be stored on the memory device 1140 and executed by the processor 1110 and may include, for example, programming embodying functions of the present disclosure (embodied in the systems, computers, servers, and / or devices described above). In some aspects, the software 1150 may be implemented and executed on a combination of servers, such as an application server and a database server.

[0349] The software 1150, or portions thereof, may also be stored and / or transported within any computer-readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above, that can fetch and execute instructions associated with the software from the instruction execution system, apparatus, or device. In the context of this disclosure, a computer-readable storage medium may be any medium, such as storage device 1140, that can contain or store programming for use by or in connection with an instruction execution system, apparatus, or device.

[0350] The software 1150 may also be propagated in any carrier medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above, that can fetch and execute instructions associated with the software from the instruction execution system, apparatus, or device. In the context of this disclosure, a carrier medium may be any medium that can convey, propagate, or carry programming for use by or in connection with an instruction execution system, apparatus, or device. Carrier-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, or infrared wired or wireless propagation media.

[0351] The computer 1100 may be connected to a network, which may be any suitable type of interconnected communication system. The network may implement any suitable communication protocol and may be protected by any suitable security protocol. The network may include any suitable configuration of network links capable of effecting transmission and reception of network signals, such as wireless network connections, T1 or T3 lines, cable networks, DSL, or telephone lines.

[0352] The computer 1100 may implement any operating system suitable for operating on a network. The software 1150 may be written in any suitable programming language, such as C, C++, Java, or Python. In various embodiments, application software embodying functionality of the present disclosure may be deployed in various configurations, such as, for example, in a client / server configuration or via a web browser as a web-based application or web service.

[0353] The foregoing description sets forth exemplary methods, parameters, and the like. However, it should be understood that such description is not intended as a limitation on the scope of the present disclosure, but is provided as a description of exemplary embodiments. The exemplary embodiments described above are not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. Each embodiment has been selected and described in order to best explain the principles of the disclosed techniques and their practical application. These embodiments enable those skilled in the art to optimally utilize these techniques and various correspondences with various modifications as suited to the particular use contemplated.

[0354] Although the present disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will become apparent to those skilled in the art. Such changes and modifications should be understood to be included within the scope of the present disclosure and examples as defined by the claims. In the foregoing description of the present disclosure and embodiments, reference is made to the accompanying drawings, which show, by way of example, certain embodiments that can be implemented. It should be understood that other embodiments and examples can be implemented and changes can be made without departing from the scope of the present disclosure.

[0355] In the above description, terms such as first, second, etc. are used to describe various elements, but these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first virtual vehicle can be called the second virtual vehicle, and similarly, the second virtual vehicle can be called the first touch, without departing from the scope of the various above-mentioned aspects.

[0356] In addition, it is to be understood that the singular forms "a," "an," and "the" used in the preceding description are intended to include the plural unless the context clearly indicates otherwise. It is also to be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It is further to be understood that the terms "includes," "including," "comprises," and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, components, and / or units, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, units, and / or groups thereof.

[0357] The term "if" may be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" may be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]," depending on the context.

[0358] In some aspects, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device with a display, the one or more programs including instructions for performing any of the steps described or claimed herein. The present disclosure also relates to a device for performing the operations of the present disclosure. The device may be specially constructed for the required purposes or may include a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory computer-readable storage medium such as, but not limited to, any type of floppy disk, optical disk, CD-ROM, disk including magneto-optical disk, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical card, application specific integrated circuit (ASIC), or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus. Furthermore, the computer referred to in this disclosure may include a single processor or may be an architecture using a multiple processor design to obtain higher computing power.

[0359] The methods, devices, and systems described herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct a more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will appear from the description below. In addition, the disclosure is not described with reference to any particular programming language. It will be understood that a variety of programming languages ​​can be used to implement the teachings of the disclosure as described herein.

Claims

1. identifying a location of a first physics object and a location of a second physics object, the first physics object being a physical vehicle; identifying respective locations of a plurality of viewpoints on the racetrack; providing a position of the first physics object, a position of the second physics object, and the respective positions of the plurality of viewpoints on the racetrack to a simulation system; computing, by the simulation system, a virtual world including virtual objects at virtual locations; calculating, by the simulation system, virtual positions of each of the plurality of viewpoints within the virtual world based on the respective positions of the plurality of viewpoints at the racetrack; calculating, by the simulation system, a representation of the first physics object and a representation of the second physics object in the virtual world, the calculation including a position of the representation of the first physics object and a position of the representation of the second physics object in the virtual world, the position of the representation of the first physics object and the position of the representation of the second physics object being between the virtual position of the virtual object in the virtual world and at least one of the viewpoints; calculating, with the simulation system, a portion of the virtual object in the virtual world that is visible from the virtual location corresponding to the at least one viewpoint, wherein the portion of the virtual object in the virtual world that is visible from the virtual location corresponding to the at least one viewpoint includes a portion of the virtual object that is not obscured, as viewed from the virtual location of the at least one viewpoint, by a representation of the first physical object and a representation of the second physical object; outputting, by the simulation system, the portion of the virtual object that is visible from the virtual position of the at least one viewpoint; providing to a display system the portion of the virtual object that is visible from the virtual position of the at least one viewpoint; generating, in the display system, a representation of the portion of the virtual object that is visible from the virtual position of the at least one viewpoint; and determining, by the simulation system, contact between the virtual object and a representation of the first physical object in the virtual world; 16. A method for displaying and interacting with a virtual object, comprising:

2. Controlling a mechanical or electrical component based on the determined contact.

2. The method of claim 1, further comprising:

3. The method of claim 2 , wherein the mechanical or electrical components are components of the physical vehicle.

4. the component is a speaker, generating, by the simulation system, audio information based on the determined contact; providing the audio information to the speaker; and playing said audio information using said speaker.

4. The method of claim 3, further comprising:

5. The method of claim 2 , wherein the mechanical or electrical component is a counter.

6. generating, by the simulation system, force information based on the determined contact; providing the force information to a force controller of the physical vehicle; and controlling, by the force controller, the mechanical or electrical components of the physical vehicle based on the force information.

3. The method of claim 2, further comprising:

7. receiving information associated with an input of at least one of the physical vehicle operator or the virtual vehicle operator, and in response to receiving the information associated with the input, the force information is provided to the force controller; 7. The method of claim 6, further comprising:

8. controlling the mechanical or electrical components, controlling the brakes or steering column of the physical vehicle 3. The method of claim 2, comprising:

9. controlling the mechanical or electrical components, controlling the power provided to motors that control the wheels or axles of said physical vehicle; 3. The method of claim 2, comprising:

10. The method of claim 1 , wherein the virtual object comprises a virtual vehicle.

11. adjusting a performance parameter of the virtual vehicle based on the determined contact.

2. The method of claim 1, further comprising:

12. The method of claim 1 , wherein the respective locations of the plurality of viewpoints at the racetrack include locations of viewpoints of an operator of the physical vehicle.

13. 2. The method of claim 1, wherein the respective locations of the plurality of viewpoints at the racetrack include respective locations of viewpoints of audience members present at the racetrack and viewing the first physics object and the second physics object.

14. The method of claim 1 , wherein the respective positions of the plurality of viewpoints at the racetrack include respective positions of viewpoints of a camera present at the racetrack and imaging the physical vehicle at the racetrack.

15. A system for displaying a virtual object, comprising: a first sensor for detecting a position of a first viewpoint on the racetrack; a second sensor for detecting a position of a second viewpoint at the racetrack; 1. A simulation system comprising: receiving a position of the first viewpoint from the first sensor and a position of the second viewpoint from the second sensor; receiving a position of a first physics object, the first physics object being a physical vehicle, at the racetrack and a position of a second physics object at the racetrack; Computing a virtual world including a virtual object at a virtual location; calculating a virtual position of each of the viewpoints within the virtual world based on the position of each of the viewpoints at the racetrack; Computing a representation of the first physical object and a representation of the second physical object in the virtual world; calculating a portion of the virtual object in the virtual world that is visible from the virtual location corresponding to at least one of the first and second viewpoints, wherein the portion of the virtual object in the virtual world that is visible from the virtual location corresponding to the at least one viewpoint includes a portion of the virtual object that is not obscured by a representation of the first physical object and a representation of the second physical object, as viewed from the virtual location of the at least one viewpoint; outputting the portion of the virtual object that is visible from the virtual position of the at least one viewpoint; providing to a display system the portion of the virtual object that is visible from the virtual position of the at least one viewpoint; Determining contact between the virtual object and a representation of the first physical object in the virtual world a simulation system, wherein the computation of the representations of both physical objects includes a position of the representation of the first physics object and a position of the representation of the second physics object in the virtual world, the position of the representation of the first physics object and the position of the representation of the second physics object being between the virtual position of the virtual object in the virtual world and at least one of the first and second viewpoints; 1. A display system comprising: receiving from the simulation system the portion of the virtual object that is visible from at least one of the first and second viewpoints; generating a representation of the portion of the virtual object as seen from the virtual position of at least one of the first and second viewpoints; Display system.

16. The system of claim 15 , wherein the simulation system controls a mechanical or electrical component based on the determined contact.

17. The system of claim 16 , wherein the mechanical or electrical components are components of the physical vehicle.

18. The simulation system comprises: generating force information based on the determined contact; and providing the force information to a force controller of the physical vehicle; 17. The system of claim 16.

19. The system of claim 15 , wherein the virtual object comprises a virtual vehicle.

20. 16. The system of claim 15, wherein the respective locations of the first viewpoint and the second viewpoint at the racetrack comprise respective locations of viewpoints of audience members present at the racetrack and viewing the first physics object and the second physics object.

Images