Image correction system and method

Abstract

An image correction system may include a head mounted display, a server, and a local correction device. The head mounted display includes a display configured to display one or more corrected images and one or more sensors configured to generate initial tracking information and updated tracking information, where the initial tracking information is associated with a first point in time and the updated tracking information is associated with a second point in time. And the updated tracking information is associated with a second time point after the first time point. The local correction device is configured to: receive tracking information from the head mounted display; transmitting the initial tracking information to a server; receiving one or more rendered images from a server; generating one or more corrected images based on the one or more rendered images and the updated tracking information; and transmitting the one or more corrected images to the head mounted display.

Description

[0001] Cross Reference to Related Applications This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 538,980 (entitled “IMAGE CORRECTION SYSTEM AND METHOD,” filed September 18, 2023), the disclosure of which is incorporated by reference herein in its entirety. BACKGROUND

[0002] This section is intended to introduce the reader to various aspects of art that can be related to various aspects of the present disclosure and is not intended to limit the scope of the

[0003] The subject matter disclosed herein relates to amusement park attractions, and more specifically, to providing an augmented or virtual experience in an amusement park attraction.

[0004] An amusement park or theme park can include various entertainment attractions that are useful in providing enjoyment to customers of the amusement park. For example, an attraction can include a ride attraction (e.g., a closed-loop track, a dark ride, a thrill ride, or other similar ride), and the attraction can be part of a themed environment that can be traditionally established using equipment, furnishings, architectural layouts, props, decorations, displayed media, and the like. These themed environments can also incorporate virtual reality (VR) or augmented reality (AR) systems. These AR or VR systems can include a head-mounted display (HMD). SUMMARY

[0005] In the following detailed description, certain embodiments are outlined with reference to the drawings. These embodiments are not intended to limit the scope of the disclosure, but rather these embodiments are intended to provide a brief, general description of the various forms in which the subject matter disclosed can be practiced. As such, the disclosure can encompass a variety of forms that can be similar or different from the examples described in the specific contexts.

[0006] The present embodiments relate to a remote streaming client lens distortion system for an amusement park ride.

[0007] According to an embodiment, an image correction system includes a head-mounted display, a server, and a local correction device. The head-mounted display includes a display configured to display one or more corrected images and one or more sensors configured to generate tracking information over a period of time, the tracking information including initial tracking information and updated tracking information, wherein the initial tracking information is associated with a first point in time and the updated tracking information is associated with a second point in time after the first point in time. The server is configured to generate one or more rendered images based at least in part on the initial tracking information. The local correction device is configured to receive the tracking information from the head-mounted display, transmit the initial tracking information to the server, receive the one or more rendered images from the server, generate one or more corrected images based on the one or more rendered images and the updated tracking information, and transmit the one or more corrected images to the head-mounted display.

[0008] In an embodiment, an image correction method includes the steps of generating, at a server, one or more rendered images of an augmented reality / virtual reality (AR / VR) environment, wherein the one or more rendered images include at least one frame rendered based on tracking information, receiving, at a local correction device, the one or more rendered images, applying a first stage correction to the one or more rendered images, wherein the first stage correction is based on updated tracking information, applying a second stage correction to the one or more rendered images to generate one or more corrected images, wherein the second stage correction is a distortion correction associated with a head-mounted display, and transmitting the one or more corrected images to the head-mounted display.

[0009] In an embodiment, a non-transitory computer-readable medium includes processor-executable code that, when executed by a processor, causes the processor to receive one or more rendered images of an augmented reality / virtual reality (AR / VR) environment, the one or more rendered images including at least one frame rendered based on tracking information, apply a first stage correction to the one or more rendered images, the first stage correction based on updated tracking information, apply a second stage correction to the one or more rendered images to generate one or more corrected images, the second stage correction being a distortion correction associated with a head-mounted display, and transmit the one or more corrected images to the head-mounted display. BRIEF DESCRIPTION OF DRAWINGS

[0010] These and other features, aspects, and advantages of the present disclosure will become better understood when read with respect to the following detailed description, taken in conjunction with the accompanying drawings, wherein like reference numerals refer to like portions in the several views, and wherein: Figure 1 is a schematic illustration of an embodiment of an image correction system for an amusement park ride according to the present technology; Figure 2 is a flowchart of communication between devices of an image correction system according to the present technology; Figure 3 is a flowchart of an image correction method according to the present technology; Figure 4 is a schematic diagram of communication between hardware components of an embodiment of the present technology; Figure 2 Figure 5 is a schematic diagram of generating and correcting an image of an image correction system according to the present technology; Figure 6 is a schematic diagram of communication between hardware components of an embodiment of the present technology; Figure 2 Figure 7 is a schematic diagram of communication between hardware components of an embodiment of the present technology; Figure 2 Figure 8 is a block diagram of an image correction system according to the present technology, including a head-mounted display, a local correction device, and a server. DETAILED DESCRIPTION

[0011] One or more specific embodiments will be described below. To provide a context for the various embodiments, a brief, general description of a suitable environment in which the embodiments can be implemented will be described now. In the following description, specific details are set forth to provide a thorough understanding of the embodiments. However, one skilled in the relevant art will recognize that the embodiments can be practiced without resorting to the details specifically set forth herein. In this description and in the following claims, the singular forms "a," "an" and "the" include plural referents unless the context clearly dictates otherwise. As used herein, the term "one or more" with reference to a feature installed or used in a system means that one or more instances of such feature can be installed or used. As used herein, the term "plurality" with reference to a feature installed or used in a system means that two or more instances of such feature can be installed or used. As used herein, the term "comprises" means includes, is inclusive, and is not exclusive. As used herein, the term "based on" means at least partially based on.

[0012] When introducing elements of various embodiments of the present disclosure, the articles "a," "an," and "the" are intended to mean that there are one or more of the elements. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there can be additional elements other than the listed elements.

[0013] ​​​Amusement parks can include augmented reality (AR), virtual reality (VR), and / or mixed reality (a combination of AR and VR) systems (e.g., AR / VR systems) to enhance the customer experience of an amusement park attraction by providing AR / VR experiences (e.g., AR experiences, VR experiences, or both) to customers. For example, an AR / VR system can include a head-mounted display (e.g., electronic goggles or a display, glasses) that can be worn by a customer to enable the customer to view virtual or augmented reality features. In particular, the head-mounted display can be used to enhance the customer experience by superimposing virtual features onto the real-world environment of the amusement park, by providing adjustable virtual environments to provide different experiences at the attraction, and so on.

[0014] Providing immersive AR / VR experiences can be challenging. For example, superimposed AR images are more realistic when the coordinates of these images relative to the environment remain fixed, even when a customer turns around or moves. Thus, position offsets in the customer can be provided as input to the system, and the display characteristics of the AR / VR images can be at least partially updated based on these position offsets. However, any lag or delay between changes in the customer's position and the computation and transmission in the display environment can compromise the level of immersion and believability that the AR / VR system is able to provide. Moreover, excessive delays in such systems can cause motion sickness or other forms of discomfort to the customer. Updates to the AR / VR environment, while often computationally intensive, should therefore be completed quickly and displayed smoothly to ensure comfort and enjoyment for the customer.

[0015] The disclosed embodiments provide an image correction system that can be implemented as part of or in conjunction with an AR / VR experience of an amusement park attraction. The image correction system can include a distributed image correction technique in which a separate or remote server generates AR / VR images and sends these AR / VR images to a local correction device. The local correction device receives updated tracking or position information for a customer of a head-mounted device and applies a final correction to the AR / VR images before sending the correct AR / VR images to the head-mounted device. In one embodiment, customer tracking information is sent to the server to generate the AR / VR images. The local correction device that receives these images can then use updated or subsequent tracking information to apply a final correction or multiple final corrections. Thus, the AR / VR images are corrected using different and updated tracking information rather than older tracking information that was used to generate the AR / VR images. This allows the final correction to be more accurate and realistic, and reduces the perception of lag.

[0016] Further, the image correction system benefits from the modularity of its subsystems. Modular corrections provide the benefit of allowing for technological upgrades or changes without altering the overall correction logic. In one embodiment, correction factors specific to a particular hardware element, such as a head-mounted display, can be localized to the appropriate local correction device as a separate algorithm or modular correction. Thus, upgrading a head-mounted display to a different version or manufacturer can cause the upgraded correction factors to be distributed only to the relevant local correction devices associated with the upgraded head-mounted display. In this way, new devices can be mixed with older versions while maintaining appropriate correction factors on the relevant locations and without the need to change global operating parameters. In one example, the correction factors are lens distortion corrections associated with a particular lens arrangement of a head-mounted display. The delegation of rendering tasks between the head-mounted device, the local correction device, and the server allows for changes in the rendering process at one component to affect the ability of another component. Additionally, the combination of multiple sensors allows for more accurate and more fault-tolerant generation of tracking information.

[0017] While certain embodiments of the present disclosure are discussed in the context of head-mounted devices worn by a patron, it should be understood that the disclosed image correction techniques can also be used in conjunction with projection mapping or other image display techniques associated with tracking movement. The method or methods of image correction used in the image correction system can depend on the nature of the display technology being implemented.

[0018] It should also be understood that the images to be generated, corrected, and displayed can comprise data that is continuously transmitted (i.e., streamed) between the devices comprising the image correction system. Data transmission is discussed herein in the context of individual images, but these images can be continuously transmitted as video data or other sized or formed data (e.g., data sets, data packets, etc.), and the transmitted data can represent only a subset of the data for the overall ride experience. The images represented by the data can also be referred to as frames in the context of AR / VR graphics generation, and these terms can be used interchangeably herein.

[0019] Figure 1 is a representation of an image correction system 12 for an amusement ride 14. The image correction system includes a local correction device 31, a head-mounted display 26, and a server 30. The local correction device 31 can be communicatively coupled to the head-mounted display 26 via a wired (e.g., HDMI, USB, etc.) or wireless connection, and to the server 30 by a wired or wireless connection.

[0020] The amusement ride 14 includes a ride vehicle 18 that travels along a ride path 20, causing the ride vehicle 18 to move along the ride path and, in some cases, according to a particular motion pattern caused by vehicle motion or vehicle effects. As Figure 1As shown in FIG. 1, patrons 16 are positioned or seated within ride vehicles 18, and each patron 16 has an associated head-mounted display 26 that displays images and, in embodiments, can provide accompanying audio content to enhance the attraction experience. The head-mounted display can be configured for one or both of AR or VR content. In an AR implementation, at least at certain times of the ride, the patrons 16 can be able to view, through the screens of the head-mounted displays 26, physical structures in the real-world environment 24, as well as virtual features 28 that are displayed. Additionally or alternatively, the head-mounted display can be used in a fully immersive VR configuration in which the patrons 16 view the displayed images on the head-mounted displays 26 in an immersive manner and are unable to view the real-world environment through the lenses of the head-mounted displays 26. However, it will be appreciated that the displayed images can include images of physical structures that are captured by cameras and, in turn, provided as displayed images on the head-mounted displays 26. In an embodiment, different virtual features 28 can be presented to each of the patrons 16 so that each of the patrons 16 has a different experience on the ride attraction 14.

[0021] To provide an improved immersive experience via the head-mounted displays 26, media content, such as still images and / or streaming video, can be rendered based on data indicative of the position and orientation of a particular patron 16. Moreover, these rendered images can be corrected or adjusted based on real-time changes to the patron's position. Thus, as the patron moves or reacts to the real-world environment, the media content sent from the server 30 can be corrected using a local correction device 31 that incorporates patron position information as an input factor for correction.

[0022] As Figure 2 As shown in FIG. 1, patrons 16 are positioned or seated within ride vehicles 18, and each patron 16 has an associated head-mounted display 26 that displays images and, in embodiments, can provide accompanying audio content to enhance the attraction experience. The head-mounted display can be configured for one or both of AR or VR content. In an AR implementation, at least at certain times of the ride, the patrons 16 can be able to view, through the screens of the head-mounted displays 26, physical structures in the real-world environment 24, as well as virtual features 28 that are displayed. Additionally or alternatively, the head-mounted display can be used in a fully immersive VR configuration in which the patrons 16 view the displayed images on the head-mounted displays 26 in an immersive manner and are unable to view the real-world environment through the lenses of the head-mounted displays 26. However, it will be appreciated that the displayed images can include images of physical structures that are captured by cameras and, in turn, provided as displayed images on the head-mounted displays 26. In an embodiment, different virtual features 28 can be presented to each of the patrons 16 so that each of the patrons 16 has a different experience on the ride attraction 14.

[0023] In certain AR / VR streaming arrangements, sensors can transmit tracking information to a server via a streaming client, images can be rendered and corrected at the server based on the tracking information, and the images can be streamed to a client connected with a head-mounted display. In these arrangements, the server generates rendered images based only on initial tracking information, which limits the effectiveness of post-processing if the patron has moved since the tracking information was collected. This disparity in image correction and subsequent minor movements of the patron's head or orientation can contribute to a laggy feel, which reduces the immersion of the experience.

[0024] Figure 3 is a flowchart of a method 100 for correcting images in an AR / VR system using a local correction device, and is discussed with reference to the features of Figures 1-2 The process begins with generating initial tracking information indicating a position and / or orientation of a patron or a position and / or orientation of a head-mounted device 26 (block 102). For example, the tracking information can be generated using one or more sensors on-board the head-mounted display 26. Additionally or alternatively, the tracking information can be generated by environmental sensors of the attraction, such as sensors of the ride vehicle 18. The initial tracking information is then sent to the local correction device 31 (block 104), and subsequently sent to the server 30 (block 106). When the server 30 receives the initial tracking information, the server 30 generates rendered images based at least in part on the initial tracking information (block 108), and sends the generated images 40 to the local correction device 31 (block 110). At a point in time after the initial tracking information 41 is generated, updated tracking information of the head-mounted display 26 is generated and sent to the local correction device 31 (block 112). The local correction device 31 uses the updated tracking information and the received images to apply one or more corrections to the received images (block 120). These corrected images are then sent to the head-mounted display 26 and displayed to the patron (block 122).

[0025] Figure 4Communication pathways of the image correction system 12 are shown between the display device (e.g., head-mounted display 26), the local correction device 31, and the server 30. In the illustrated example, the guest tracking information 41 is generated directly from on-board sensors of the head-mounted display 26 and / or communicated from those sensors through the head-mounted display 26. However, in embodiments, the tracking information 41 can be communicated to the local correction device 31 from other sensors. The local correction device 31 can relay the tracking information 41 to the server 30, which in turn uses the tracking information to generate one or more rendered images 43 that are sent to the local correction device 31. The local correction device 31 uses updated tracking information 41 from a subsequent point in time to apply a local correction to generate one or more corrected rendered images 44 that are transmitted to the head-mounted display 26.

[0026] In embodiments, the communication between the local correction device 31 and the server is wired or wireless. The local correction device 31 can be located in the local range of the attraction or ride vehicle 18.

[0027] The local correction device 31 can be configured to apply a correction algorithm to compensate for artifacts generated as a byproduct of the earlier correction. For example, a spatial or temporal warp correction shifts displayed objects within an image based on the tracking information 41 supplied by the sensors. This shift can cause occluded portions of the environment to appear as part of the corrected image. These previously occluded elements of the image can appear as “blank” areas in the corrected image, thereby compromising the believability of the virtual environment and thus the immersion for the guest. An algorithm implemented by the local correction device 31 as a correction can identify areas that are artifacts of the previous correction and use patterns from surrounding areas of the image to “fill in” the blank areas. This correction can be accomplished using artificial intelligence (AI) algorithms or other means. In another embodiment, this correction can be accomplished by the head-mounted display 26 before the image is displayed.

[0028] In addition to the correction of the rendered image 43 and the storage of the tracking information 41, the local correction device 31 can undertake other tasks in the rendering pipeline. These capabilities can include overlay generation and updating associated with a guest-specific profile. For example, a guest profile associated with a particular guest 16 can include information about their score, health points, and ammo that is generated during the game activity at the attraction and provided by the server 30. In an embodiment, the local correction device 31 can provide a hub for storing and updating the values of the guest's score, health points, and ammo, as well as creating an overlay that displays that information on the generated image, such that the overlay is provided onto the corrected rendered image 44. The overlay can be added to or overlaid on the corrected rendered image 44 after the local correction device 31 applies the correction to the rendered image 43 received from the server. In another embodiment, this overlay is rendered as part of the rendered image 43 by the server 30.

[0029] In some embodiments, the head-mounted display 26 is configured to be worn on the head of the guest 16 and includes one or more sensors including a hand tracking camera for generating tracking information based at least in part on the position of the guest's hand. The image correction system can use the tracking information in combination with the image displayed by the head-mounted display to determine that the guest 16 is interacting with the virtual environment. This interaction can affect subsequent changes in the virtual environment (e.g., animation, special effects, movement). For example, the tracking information can include an indication that the guest is pointing up and to the left. The image correction system, with reference to the image displayed to the head-mounted display 26, determines that the guest 16 is pointing at a star object at the upper left of the virtual environment. The image correction system then generates an image showing the star exploding and displays the image to the guest, allowing the guest to see the virtual implications of their hand movement.

[0030] In an embodiment, at least one of the one or more sensors of the head-mounted display 26 includes an inertial measurement unit (IMU). The IMU is configured to generate tracking information 41 based at least in part on the position of the guest's head. The tracking information associated with the guest's head (i.e., motion data or orientation data) includes at least three values that correspond to three axes on which the guest's head can move. This tracking information is sent to the local correction device, where the tracking information is sent to the server or stored in memory.

[0031] In an embodiment, the server 30 is responsible for rendering any imagery of the guest's hand and their interaction with the virtual environment. The tracking information from the hand tracking camera embedded in or connected to the head-mounted display 26 is sent to the server 30 via the local correction device 31. The server 30 receives the tracking information 41 and generates a rendered image 43, which includes using the tracking information from the hand tracking camera.

[0032] Similarly, the server 30 can be responsible for generating certain other elements of the virtual environment with which the customers 16 interact. These elements can include objects within the virtual environment to be displayed by at least one of the head-mounted displays 26 served via the local correction devices 31. For example, as illustrated in FIG. 3, a virtual object 28 can be associated with a particular location in the attraction for all customers 16 in the ride vehicle 18. The server generates the object 28 as part of the rendered image 43 for each of the local correction devices 31 and head-mounted displays 26. This implementation uses the assumption that customers 16 experiencing the same AR / VR amusement park ride will interact with similar virtual environments, and those similar virtual environments often share common virtual objects. Figure 1

[0033] This delegation of the rendering task (i.e., multi-layer rendering) results in improved modularity for maintenance functions of the amusement park ride system. If a change to one or more rendering aspects of the AR / VR amusement park is performed, a separate maintenance function is available (based on whether that portion of the rendering process is occurring at the server level or at the local correction device). For example, to change the starting ammo value for a ride profile, an operator can change the maintenance function of the local correction device without changing the functions associated with the server 31. Similarly, to change the color of a virtual object, if the server is responsible for generating that object, an operator can use the maintenance function of the server without affecting the functions of the local correction device. Changes to the rendering system (where portions of the rendering system are separated as described) have less footprint, resulting in a decreased likelihood of changes to one portion of the system causing errors at other portions of the system.

[0034] Likewise, the correction logic used to adjust the image after generation is modularized such that changes can be made to one stage or aspect of the correction without changing other stages or aspects. For example, if a change in distortion is implemented to accommodate a larger head-mounted device, the system can remain functional without changing other late- latch warping effects.

[0035] Figure 5 is a schematic illustration of an image correction pathway 50 for generating and correcting an image using tracking information and a local correction device 31. The server 30 receives tracking information 45 and generates a frame 47 based at least in part on the tracking information 45. The local correction device 31 receives the generated frame 47 and applies two corrections (where at least one of the corrections is a lens distortion correction) to the generated frame to create a warped frame 48, and subsequently a distorted frame 49. Once the image correction pathway 50 has generated a frame and applied corrections, the frame is ready to be transmitted to the head-mounted display 26. ​

[0036] The latency of AR / VR image generation and correction methods (i.e., the "rendering pipeline") often depends on the effectiveness of latency combat, post-processing techniques. Post-processing corrections can be introduced after initial steps in the rendering pipeline (e.g., vertex processing, rasterization, fragment processing, etc.). These post-processing effects include "late- latch-in" effects such as asynchronous re-projection (i.e., "spatial warping" and "temporal warping"). These late-latch-in effects use a combination of previously rendered images and newer motion data from tracking sensors to warp the previous frame into a prediction of how the frame to be rendered can look. Post-processing techniques also include distortion, where the image is distorted to fit the distortion of the display environment. Post-processing techniques are most effective when they use the latest possible motion input from the head-mounted display to alter the image. Thus, it is beneficial to defer post-processing at least until updated tracking or position information becomes available. With the foregoing in mind, Figure 5 The system of FIG. 1 allows the local correction device 31 to implement post-processing corrections with updated tracking information 46.

[0037] In an embodiment, the local correction device can be configured to send only some elements of the initial tracking data it receives to the server and store other elements in memory. For example, if the server 30 uses only the initial camera data for image generation, but the initial tracking data includes camera data and inertial measurement unit (IMU) data, the local correction device can be configured to send only the camera data to the server. The IMU data can be stored by the local correction device in memory and can be used as input to the image correction logic at a later time.

[0038] Figure 6 is a schematic illustration of the communication pathway of the image correction system 12, where the local correction device 31 selects tracking information to send to the server 30. The sensors, including the head-mounted display 26, generate two sets of tracking information (tracking information 1 60 and tracking information 2 62) and send both sets of tracking information to the local correction device 31. The local correction device 31 is configured to determine that only tracking information 60 should be sent to the server 30. Thus, the tracking information 60 is sent to the server 30, while the tracking information 62 is stored in memory.

[0039] The local correction device 31 can be configured to process the tracking information it receives from the head-mounted display 26 and send that processed information to the server. The processing of data by the local correction device 31 can benefit the operation of the local correction device 31, the server 30, or the image correction system 12. For example, the set of processed tracking information can be smaller than the set of pre-processed tracking information. Reducing the size of the data to be sent can result in faster transmission of data over the streaming connection, resulting in less cumulative latency experienced by the customer.

[0040] Figure 7 The image correction system 12 is shown where only processed data 71 is sent from the local correction device 31 to the server 30. The local correction device 31 receives the set of tracking information 70 from the head-mounted display 26, processes the set of tracking information, and sends the processed set of tracking information 71 to the server 30. The server 30 is configured to use the processed tracking information 71 to generate the rendered image 43 and send it to the local correction device 31. The local correction device 31 then uses the correction logic to generate the corrected rendered image 44 and send it to the head-mounted display.

[0041] Figure 8 is a block diagram of an image correction system 85 according to the present embodiment, which incorporates the head-mounted display 26, the local correction device 31, one or more auxiliary or environmental sensors 99, and the server 30.

[0042] The head-mounted display 31 can include one or more processors 90 and one or more memory devices 91. In some embodiments, the processor(s) 90 and memory device(s) 91 can be external to the head-mounted display 26. The one or more processors 90 can execute software programs and / or instructions to display images to a display 92. Further, the processor(s) 90 can include multiple microprocessors, one or more "general-purpose" microprocessors, one or more special-purpose microprocessors and / or one or more application-specific integrated circuits (ASICS) and / or one or more reduced instruction set (RISC) processors. The memory device(s) 91 can include one or more storage devices and can store machine-readable and / or processor-executable instructions (e.g., firmware or software), such as instructions related to adjusting the display of virtual objects, for execution by the processor(s) 90. As such, the memory device(s) 91 can store, for example, control software, lookup tables, configuration data, and so on, to facilitate adjusting the display of virtual objects. In some embodiments, the processor(s) 90 and memory device(s) 91 can be external to the head-mounted display 26. The memory device(s) 91 can include tangible, non-transitory, machine-readable media, such as volatile memory (e.g., random access memory (RAM)) and / or non-volatile memory (e.g., read only memory (ROM), flash memory, hard drives, and / or any other suitable optical, magnetic, or solid-state storage media). Additionally, the head-mounted display 26 can include one or more sensors 93 (e.g., hand tracking sensors, eye tracking sensors, inertial measurement units, microphones, and the like).

[0043] The local correction device 31 receives tracking information from the head mounted display 26 and a rendered image 43 from the server 30. The local correction device 31 includes processor(s) 95, memory device(s) 96, input / output (I / O) ports 97, and communication circuitry 98. The I / O ports 97 can receive tracking information from the head mounted display 26 or generated images from the server 30. The processor(s) 95 can execute software programs to reduce the size of the tracking information or augment the tracking information. The processor(s) 95 can also execute software programs and / or instructions to adjust images, such as virtual features, displayed on the head mounted display 26. The memory device(s) 96 can include one or more storage devices and can store machine-readable and / or processor-executable instructions (e.g., firmware or software), such as instructions related to adjusting images based on tracking information, for execution by the processor(s) 95. The communication circuitry 98 facilitates wireless (e.g., Ethernet, WAN, and the like) and / or wired (HDMI, USB, and the like) communication with the head mounted display 26 and the server 30.

[0044] The server 30 can include processor(s) 101, memory device(s) 103, and communication circuitry 105. The processor(s) 101 can execute software programs to generate a rendered image based at least in part on received tracking information. The memory device(s) 103 can include one or more storage devices and can store machine-readable and / or processor-executable instructions (e.g., firmware or software), such as instructions related to AR / VR ride logic and updates, for execution by the processor(s) 101. The communication circuitry 105 facilitates a wireless streaming connection (or in embodiments, a wired connection) with the local correction device 31.

[0045] While only certain features of the application have been illustrated and described herein, many modifications and changes will occur to those skilled in the art. It is, therefore, to be understood that this application is intended to cover all such modifications and changes as fall within the true spirit of the application.

[0046] The technology presented and claimed herein is directed to solving the problems described in the background section of this patent. Any statements as to the scope of the technology or what can be claimed are intended to be construed broadly in accordance with the statute and are not to be limited by the details given in the foregoing description or by any limitations known in the prior art. Furthermore, any statements to the effect that the technology "enables" or "solves" any problem known in the art are not intended to be limiting of the scope of the technology in any way.

Claims

1. An image correction system, comprising: a head-mounted display, including: a display configured to display one or more corrected images; and one or more sensors configured to generate tracking information over a period of time, the tracking information including initial tracking information and updated tracking information, wherein the initial tracking information is associated with a first point in time and the updated tracking information is associated with a second point in time after the first point in time; a server configured to generate one or more rendered images based at least in part on the initial tracking information; and a local correction device configured to: receive the tracking information from the head-mounted display; transmit the initial tracking information to the server; receive the one or more rendered images from the server; generate one or more corrected images based on the one or more rendered images and the updated tracking information; and transmit the one or more corrected images to the head-mounted display.

2. The image correction system of claim 1, wherein, the local correction device is configured to generate the one or more corrected images by correcting for lens distortion associated with the head-mounted display.

3. The image correction system of claim 2, wherein, the local correction device is configured to generate the one or more corrected images by applying one or more of temporal warping, spatial warping, lens distortion factor correction, or late latch correction.

4. The image correction system of claim 3, wherein, the correction for the lens distortion factor occurs after the one or more of temporal warping, spatial warping, or late latch correction is applied.

5. The image correction system of claim 1, wherein, the local correction device and the head-mounted display are coupled to a ride vehicle.

6. The image correction system of claim 1, wherein, the local correction device is attached to or embedded in a housing of the head-mounted display.

7. The image correction system of claim 1, wherein, the local correction device is wirelessly coupled to the server.

8. The image correction system of claim 7, wherein, the local correction device is coupled to the head-mounted display via a wired connection.

9. The image correction system of claim 1, wherein, the local correction device transmits one or more corrected images to no more than one head-mounted display.

10. The image correction system of claim 1, wherein, the one or more sensors include at least a camera and an inertial measurement unit (IMU) that generate tracking information.

11. The image correction system of claim 10, wherein, the tracking information generated by the camera is passed to the local correction device and subsequently to the server, and the tracking information generated by the IMU is passed to the local correction device and is not passed to the server.

12. An image correction method, comprising: generating, at a server, one or more rendered images of an augmented reality / virtual reality (AR / VR) environment, wherein the one or more rendered images include at least one frame rendered based on tracking information; receiving, at a local correction device, the one or more rendered images; applying a first stage correction to the one or more rendered images, wherein the first stage correction is based on updated tracking information; applying a second stage correction to the one or more rendered images to generate one or more corrected images, wherein the second stage correction is a distortion correction associated with a head-mounted display; and transmitting the one or more corrected images to the head-mounted display.

13. The image correction method according to claim 12, wherein The head-mounted display includes a projector, and the distortion correction is a function of a distortion of a display surface on which a projected image will be displayed.

14. The image correction method according to claim 12, wherein The head-mounted display includes a lens, and the distortion correction is a function of a distortion of the lens.

15. The image correction method according to claim 12, wherein The local correction device or the head-mounted display is configured to apply a third stage correction to the one or more rendered images.

16. The image correction method according to claim 12, wherein The tracking information is generated at the head-mounted display and communicated to the server via the local correction device.

17. The image correction method of claim 12, wherein, The updated tracking information is generated at the head-mounted display and communicated to the local correction device without being communicated to the server.

18. A non-transitory computer readable medium comprising processor executable code that, when executed by a processor, causes the processor to: receiving one or more rendered images of an augmented reality / virtual reality (AR / VR) environment, wherein, The one or more rendered images include at least one frame rendered based on tracking information; apply a first stage correction to the one or more rendered images, wherein the first stage correction is based on updated tracking information; apply a second stage correction to the one or more rendered images to generate one or more corrected images, wherein the second stage correction is a distortion correction associated with a head-mounted display; and transmit the one or more corrected images to the head-mounted display.

19. The non-transitory computer-readable medium of claim 18, wherein, The processor executable code, when executed by the processor, causes the processor to: receive the tracking information from the head-mounted display; and transmit the tracking information to a server configured to generate the one or more rendered images of the AR / VR environment.

20. The non-transitory computer readable medium of claim 18, wherein, The distortion correction is one or more of a temporal warping, a spatial warping, a lens distortion factor correction, or a late latch correction.

Images