Image correction system and method
Patent Information
- Application Number
- JP2026516606
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-09-12
- Filing Date
- 2024-09-16
- Publication Date
- 2026-10-14
Smart Images
Figure 2026534142000001_ABST
Abstract
Description
TECHNICAL FIELD
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 538,980, filed on September 18, 2023 and entitled "IMAGE CORRECTION SYSTEM AND METHOD", the disclosure of which is hereby incorporated by reference in its entirety into the present specification.
BACKGROUND ART
[0002] This section is intended to introduce the reader to various aspects of technology that may be related to various aspects of the present disclosure. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, these statements are not to be taken as admissions of prior art, and are to be read in light of the above perspective.
[0003] The subject matter disclosed in the present specification relates to amusement park attractions, and specifically relates to providing augmented or virtual experiences in amusement park attractions.
[0004] Amusement parks or theme parks may include various entertainment attractions that help provide enjoyment to park guests. For example, attractions may include ride attractions (e.g., closed-loop tracks, dark rides, thrill rides, or other similar rides), and attractions may be part of a themed environment that is traditionally established using facilities, furnishings, building layouts, props, decorations, display media, and the like. These themed environments may also incorporate virtual reality (VR) or augmented reality (AR) systems. These AR or VR systems may include a head-mounted display (HMD).
SUMMARY OF THE INVENTION
[0005] The following summarizes several embodiments that fall within the same scope as the subject matter of the original claims. These embodiments are not intended to limit the scope of the disclosure, but rather to outline some of the disclosed embodiments. In practice, the disclosure may include a variety of forms that are similar to or different from the embodiments shown below.
[0006] This embodiment relates to a remote streaming client lens distortion system for amusement park rides.
[0007] According to one embodiment, the image correction system includes a head-mounted display, a server, and a local correction device. The head-mounted display includes a display configured to display one or more corrected images, and one or more sensors configured to generate tracking information over a period of time, the tracking information including initial tracking information and latest tracking information, the initial tracking information relating to a first time point in time, and the latest tracking information relating to a second time point in time after the first time point. The server is configured to generate one or more rendered images based at least in part on the initial tracking information. The local correction device is configured to receive tracking information from the head-mounted display, transmit the initial tracking information to the server, receive one or more rendered images from the server, generate one or more corrected images based on the one or more rendered images and the latest tracking information, and transmit one or more corrected images to the head-mounted display.
[0008] In one embodiment, the image correction method includes the steps of: generating one or more rendered images of an augmented reality / virtual reality (AR / VR) environment in a server, including at least one frame rendered based on tracking information; receiving one or more rendered images in a local correction device; applying a first-stage correction based on the latest tracking information to one or more rendered images; generating one or more corrected images by applying a second-stage correction, which is distortion correction related to a head-mounted display, to one or more rendered images; and transmitting one or more corrected images to a head-mounted display.
[0009] In one embodiment, a non-temporary computer-readable medium includes processor-executable code, which, when executed by the processor, causes the processor to: generate one or more rendered images of an augmented reality / virtual reality (AR / VR) environment, each including at least one frame rendered based on tracking information; apply a first-stage correction to the one or more rendered images based on the most recent tracking information; apply a second-stage correction, which is distortion correction related to a head-mounted display, to the one or more rendered images to generate one or more corrected images; and transmit the one or more corrected images to a head-mounted display.
[0010] A better understanding of these and other features, aspects and advantages of this disclosure will be gained by reading the following detailed description while referring to the attached drawings, which indicate the same parts throughout. [Brief explanation of the drawing]
[0011] [Figure 1] This is a schematic diagram of an embodiment of an image correction system for amusement park rides using this technology. [Figure 2] This is a flowchart illustrating the communication between devices in an image correction system using this technology. [Figure 3] This is a flowchart of the image correction method using this technology. [Figure 4] This is a schematic diagram of the communication between hardware components in the embodiment shown in Figure 2, which utilizes this technology. [Figure 5] This is a schematic diagram showing how the image correction system using this technology generates and corrects images. [Figure 6] This is a schematic diagram illustrating the tracking information set in the embodiment shown in Figure 2, based on this technology. [Figure 7] This is a schematic diagram illustrating the tracking information for the embodiment shown in Figure 2, based on this technology. [Figure 8] This is a block diagram of an image correction system, including a head-mounted display, a local correction device, and a server, based on this technology. [Modes for carrying out the invention]
[0012] The following describes one or more specific embodiments. For the sake of brevity, this specification does not describe all features of these embodiments. It should be understood that the development of any such implementation found in any engineering or design project will require numerous implementation-specific decisions to achieve the developer's specific objectives, such as compliance with system-related and business-related constraints, which may vary by implementation. Furthermore, while such development efforts may be complex and time-consuming, they should be understood by those skilled in the art who benefit from this disclosure as routine design, fabrication, and manufacturing activities.
[0013] When describing elements of various embodiments of this disclosure, the articles “a,” “an,” “the,” and “said” mean that there are one, two, or more of these elements. The terms “comprising,” “including,” and “having” are intended to be comprehensive and mean that there may be further elements other than those listed.
[0014] Amusement parks may include augmented reality (AR) systems, virtual reality (VR) systems, and / or mixed reality (AR and VR combination) systems (e.g., AR / VR systems) to enhance the guest experience of amusement park attractions by providing guests with augmented reality (AR) / virtual reality (VR) experiences (e.g., AR experiences, VR experiences, or both). For example, an AR / VR system may include a head-mounted display (e.g., electronic goggles or displays, glasses) that guests can wear to view virtual or augmented reality features. Specifically, the guest experience can be enhanced by using a head-mounted display to overlay virtual features onto the real-world environment of the amusement park and to provide a virtual environment that can be adjusted to offer different experiences at attractions.
[0015] Providing an immersive AR / VR experience can be challenging. For example, if overlaid AR images remain fixed relative to the environment's coordinates, these images will appear more realistic even if the guest turns around or moves. Therefore, changes in the guest's position can be provided as input to the system, and the display characteristics of the AR / VR image can be updated, at least partially, based on these positional changes. However, lag or delay between changes in the guest's position and the calculated and transmitted changes in the display environment can impair the level of immersion and realism that the AR / VR system can provide. Furthermore, excessive delays in such systems may induce motion sickness or other forms of discomfort in the guest. Therefore, updates to the AR / VR environment, while often computationally intensive, should be completed quickly and displayed smoothly to ensure guest comfort and enjoyment.
[0016] Embodiments of this disclosure provide an image correction system that can be implemented as part of or in combination with an AR / VR experience in an amusement park attraction. The image correction system may include a distributed image correction technique in which an independent or remote server generates AR / VR images and transmits these AR / VR images to a local correction device. The local correction device receives the latest tracking or location information of a guest on a head-mounted device, applies a final correction to the AR / VR image, and then transmits the correct AR / VR image to the head-mounted device. In one embodiment, guest tracking information is transmitted to a server to generate an AR / VR image. The local correction device, having received these images, can then apply one or more final corrections using the latest or subsequent tracking information. Thus, the AR / VR image is corrected using different, more recent tracking information rather than the slightly older tracking information used to generate the AR / VR image. As a result, the final correction is more accurate and realistic, and the lag interval is reduced.
[0017] Furthermore, image correction systems benefit from the modularization of their subsystems. Modular correction offers the advantage of enabling technical upgrades or changes without altering the entire correction logic. In one embodiment, correction coefficients specific to a particular hardware element, such as a head-mounted display, can be localized to an appropriate local correction device as a separate algorithm or modular correction. Thus, when upgrading a head-mounted display to a different version or manufacturer's model, the upgraded correction coefficients are distributed only to the local correction device associated with the upgraded head-mounted display. In this way, new devices can be mixed with older versions without changing the global operating parameters while maintaining the correction coefficients where appropriate. In one example, the correction coefficient is lens distortion correction related to a specific lens arrangement of the head-mounted display. By delegating rendering tasks between the head-mounted device, local correction device, and server, changes to the rendering process can be made in one component without affecting the capabilities of other components. Also, combining multiple sensors enables the generation of more accurate and fault-tolerant tracking information.
[0018] While some embodiments of this disclosure are described in the context of a head-mounted device worn by a guest, it should be understood that the image correction techniques of this disclosure can also be used in conjunction with projection mapping or other image display techniques related to tracked motion. The one or more image correction methods used in the image correction system may depend on the nature of the display technique being implemented.
[0019] Furthermore, it should be understood that the generated, corrected, and displayed images may include data that is transmitted continuously (i.e., streamed) between devices, including an image correction system. While data transmission is described herein in the context of a single image, these images can be transmitted continuously as video data or data of other sizes or forms (e.g., datasets and packets), and the transmitted data may represent only a subset of the data used for the entire ride experience. In the context of AR / VR graphics generation, the image represented by data may also be called a frame, and these terms may be used synonymously herein.
[0020] Figure 1 shows a representation of an image correction system 12 for an amusement park ride 14. The image correction system includes a local correction device 31, a head-mounted display 26, and a server 30. The local correction device 31 can be communicatively coupled to the head-mounted display 26 via a wired connection (e.g., HDMI®, USB, etc.) or a wireless connection, and can also be communicatively coupled to the server 30 via a wired connection or a wireless connection.
[0021] Amusement park ride 14 includes a ride vehicle 18 that moves along a ride path 20, and the ride path 20 causes the ride vehicle 18 to move along the ride path and, in some cases, according to specific motion patterns generated by the movement of the vehicle or vehicle effects. As shown in Figure 1, a guest 16 is positioned or seated within the ride vehicle 18, and each guest 16 has an associated head-mounted display 26, which displays images to enhance the attraction experience, and in embodiments may provide accompanying audio content. The head-mounted display may be configured for one or both of AR or VR content. In an AR implementation, at least at some points of the ride, the guest 16 can see both the physical structures of the real-world environment 24 and the displayed virtual features 28 through the display screen of the head-mounted display 26. Additionally or alternatively, the head-mounted display may be used in a fully immersive VR configuration where the guest 16 cannot see the real-world environment through the lenses of the head-mounted display 26 while viewing the displayed images on the head-mounted display 26 immersively. However, it should be understood that the displayed images may include images in which physical structures captured by a camera are provided as displayed images on the head-mounted display 26. In certain embodiments, different virtual features 28 may be presented to each guest 16 such that each guest 16 has a different experience on the ride 14.
[0022] To provide an improved immersive experience via the head-mounted display 26, media content such as still images and / or streaming videos may be rendered based on data indicating the position and orientation of a particular guest 16. Further, these rendered images may be modified or adjusted based on real-time changes in the guest position. Accordingly, when the guest is moving or reacting in the real-world environment, the media content transmitted from the server 30 can be corrected using a local correction device 31 that captures the guest's position information as an input factor for correction.
[0023] As shown in Figure 2, the image correction system 12 includes a local correction device 31, a head-mounted display 26, and a server 30. The local correction device 31 is communicatively coupled to the head-mounted display 26 via a wired connection (HDMI (registered trademark), USB, etc.) or a wireless connection, and is also communicatively coupled to the server 30 via a wired or wireless connection. The local correction device 31 can transmit tracking information 40 to the server 30. The server 30 uses the tracking information 40 to generate one or more rendered images, and transmits these to the local correction device 31. The local correction device 31 uses embedded correction logic 33 and tracking information 41 to correct the one or more rendered images, and transmits the corrected images 41 to the head-mounted display 26.
[0024] In some AR / VR streaming configurations, a sensor may transmit tracking information to a server via a streaming client, the server renders and corrects images based on this tracking information, and these images can be streamed to the client connected to the head-mounted display. In these configurations, the server generates rendered images based only on the initial tracking information, so the effectiveness of post-processing is limited if the guest moves after the tracking information is collected. This difference in image correction and subsequent slight movements of the guest's head or orientation may contribute to a sense of lag that reduces the immersive feel of the experience.
[0025] Figure 3 is a flowchart of method 100 for correcting images of an AR / VR system using a local correction device, which will be explained with reference to the features in Figures 1 and 2. The process begins by generating initial tracking information (block 102) indicating the guest's position and / or orientation, or the position and / or orientation of the head-mounted device 26. For example, the tracking information can be generated using one or more sensors mounted on the head-mounted display 26. In addition to or instead of this, the tracking information can also be generated by environmental sensors of the attraction, such as sensors on the ride vehicle 18. Next, the initial tracking information is sent to the local correction device 31 (block 104), and then to the server 30 (block 106). Upon receiving the initial tracking information, the server 30 generates a rendered image based at least partially on the initial tracking information (block 108), and sends the generated image 40 to the local correction device 31 (block 110). After the generation of the initial tracking information 41, the latest tracking information for the head-mounted display 26 is generated and sent to the local correction device 31 (block 112). The local correction device 31 uses the latest tracking information and the received image to apply one or more corrections to the received image (block 120). These corrected images are then transmitted to the head-mounted display 26 and displayed to the guest (block 122).
[0026] Figure 4 shows the communication path of the image correction system 12 between the display device (e.g., head-mounted display 26), the local correction device 31, and the server 30. In the illustrated example, guest tracking information 41 is generated directly from the head-mounted display 26 using onboard sensors and / or passed from these sensors through the head-mounted display 26. However, in embodiments, tracking information 41 can also be transmitted to the local correction device 31 from other sensors. The local correction device 31 transmits the tracking information 41 to the server 30, which uses the tracking information to generate one or more rendered images 43, which are then transmitted to the local correction device 31. The local correction device 31 applies local corrections using the most recent tracking information 41 from a subsequent point in time to generate one or more corrected rendered images 44, which are then transmitted to the head-mounted display 26.
[0027] In this embodiment, communication between the local correction device 31 and the server is either wired or wireless. The local correction device 31 can be locally located relative to the attraction or ride vehicle 18.
[0028] The local correction device 31 can be configured to apply a correction algorithm to correct artifacts generated as a byproduct of previous corrections. For example, spacewarp correction or timewarp correction displaces displayed objects in the image based on tracking information 41 supplied by sensors. This displacement may reveal hidden parts of the environment as part of the corrected image. These previously hidden image elements appear as "blank" areas in the corrected image, which can impair the authenticity of the virtual environment and, consequently, the guest's immersion. The algorithm implemented as a correction by the local correction device 31 can identify areas that are artifacts from previous corrections and "fill" in the blank areas using patterns from the surrounding areas of the image. This correction can be achieved using an artificial intelligence (AI) algorithm or other means. In another embodiment, the head-mounted display 26 can display the image after this correction is complete.
[0029] In addition to correcting the rendered image 43 and storing tracking information 41, the local corrector 31 can also perform other responsibilities within the rendering pipeline. These capabilities may include generating and updating overlays related to guest-specific profiles. For example, a guest profile associated with a particular guest 16 may include information about the guest's score, health points, and ammunition, which are generated during the attraction's gameplay and provided by the server 30. In one embodiment, the local corrector 31 may store and update the guest's score, health points, and ammunition values, and provide a hub that creates an overlay to display this information on the generated image, so that the overlay is provided on the corrected rendered image 44. This overlay can be added to or superimposed on the corrected rendered image 44 after the local corrector 31 has applied corrections to the rendered image 43 received from the server. In another embodiment, this overlay is rendered by the server 30 as part of the rendered image 43.
[0030] In some embodiments, a head-mounted display 26 is configured to be worn on the head of a guest 16 and includes one or more sensors, one or more of which include a hand-tracking camera used to generate tracking information based at least partially on the position of the guest's hands. An image correction system can use a combination of the tracking information and the image displayed by the head-mounted display to determine that the guest 16 is interacting with the virtual environment. This interaction can influence subsequent changes in the virtual environment (e.g., animation, special effects, movement). For example, the tracking information may include an indication that the guest is pointing to the upper left. The image correction system, referring to the image displayed on the head-mounted display 26, determines that the guest 16 is pointing to a star object in the upper left of the virtual environment. As a result, the image correction system generates and displays an image showing the star exploding, allowing the guest to confirm the virtual meaning of their hand movements.
[0031] In one embodiment, at least one of the one or more sensors of the head-mounted display 26 includes an inertial measurement unit (IMU). The IMU is configured to generate tracking information 41 based at least partially on the position of the guest's head. The tracking information related to the guest's head (i.e., motion data or orientation data) includes at least three values corresponding to three axes on which the guest's head can move. This tracking information is transmitted to a local compensator, which then transmits this information to a server or stores it in memory.
[0032] In one embodiment, server 30 is involved in rendering images of either the guest's hands or the guest's interaction with the virtual environment. Tracking information from a hand tracking camera embedded in or connected to a head-mounted display 26 is transmitted to server 30 via a local correction device 31. Server 30 receives the tracking information 41 and generates a rendered image 43, including using the tracking information from the hand tracking camera.
[0033] Similarly, the server 30 may also be involved in generating several other elements of the virtual environment with which the guests 16 interact. These elements may include objects within the virtual environment that should be displayed on at least one of the head-mounted displays 26 serviced via the local compensator 31. For example, as shown in Figure 1, a virtual object 28 may be associated with a specific location within an attraction for all guests 16 inside the ride vehicle 18. The server generates the object 28 as part of all rendering images 43 for all local compensators 31 and head-mounted displays 26. This implementation uses the assumption that guests 16 experiencing the same AR / VR amusement park ride interact with similar virtual environments, and these similar virtual environments often share common virtual objects.
[0034] This delegation of rendering tasks (i.e., multi-layer rendering) improves the modularity of the maintenance functions for the amusement park ride system. When one or more rendering modes of the AR / VR amusement park are changed, separate maintenance functions are available depending on whether that part of the rendering process is performed at the server level or in the local compensator. For example, to change the starting ammunition value of a ride profile, the operator can change the maintenance function of the local compensator without changing the functions related to server 31. Similarly, to change the color of a virtual object when the server is involved in its generation, the operator can use the server's maintenance function without affecting the functions of the local compensator. Changes to the rendering system with such isolated components have a small footprint and reduce the likelihood that changes to one part of the system will cause problems in other parts of the system.
[0035] Similarly, the correction logic used to adjust the image after generation is also modularized, allowing changes to be made to a single correction stage or mode without altering other parts. For example, when making distortion changes to accommodate larger head-mounted devices, the system can maintain functionality without altering other late-latch warping effects.
[0036] Figure 5 is a schematic diagram of the image correction path 50 for generating and correcting images using tracking information and a local correction device 31. The server 30 receives tracking information 45 and generates a frame 47 based at least partially on the tracking information 45. The local correction device 31 receives the generated frame 47 and applies two corrections to this generated frame, at least one of which is lens distortion correction, to create a distorted frame 48, and then creates a distorted frame 49. Once the image correction path 50 has generated the frame and applied the corrections, the frame is ready to be transmitted to the head-mounted display 26.
[0037] In many cases, the latency of AR / VR image generation and correction methods (i.e., "render pipelines") depends on the effectiveness of post-processing techniques to address latency. After the initial steps of the render pipeline (e.g., vertex processing, rasterization, fragmentation, etc.), post-processing corrections can be introduced. These post-processing effects include "ray latch" effects such as asynchronous reprojection (i.e., "space warp" and "time warp"). These ray latch effects warp previous frames using a combination of previously rendered images and new motion data from tracking sensors to predict how the frame to be rendered will look. Post-processing techniques also include distortion, which distorts the image to fit the distortions of the display environment. Post-processing techniques are most effective when modifying the image using the newest possible motion input from the head-mounted display. Therefore, it is beneficial to delay post-processing until at least the most recent tracking or positional information is available. Based on these considerations, the system shown in Figure 5 enables the local correction device 31 to perform post-processing corrections using the latest tracking information 46.
[0038] In one embodiment, the local correction device can be configured to send only some of the elements of the received initial tracking data to the server and store the other elements in memory. For example, if the server 30 uses only the initial camera data for image generation, but the initial tracking data includes both camera data and inertial measurement unit (IMU) data, the local correction device can be configured to select and send only the camera data to the server. The IMU data can be stored in memory by the local correction device and used as input for image correction logic at a later time.
[0039] Figure 6 is a schematic diagram of the communication path of the image correction system 12, in which the local correction device 31 selects the tracking information to be sent to the server 30. The sensors, including the head-mounted display 26, generate two sets of tracking information, tracking information 1 60 and tracking information 2 62, and send both sets to the local correction device 31. The local correction device 31 is configured to determine that only tracking information 60 should be sent to the server 30. Therefore, tracking information 60 is sent to the server 30, and tracking information 62 is stored in memory.
[0040] The local correction device 31 can be configured to process tracking information received from the head-mounted display 26 and transmit this processed information to the server. The data processing by the local correction device 31 can be beneficial to the operation of the local correction device 31, the server 30, or the image correction system 12. For example, the processed tracking information set can be smaller than the unprocessed tracking information set. Reducing the size of the data to be transmitted can speed up data transmission over the streaming connection and reduce the cumulative latency experienced by the guest.
[0041] Figure 7 shows an image correction system 12 in which only processed data 71 is sent from the local correction device 31 to the server 30. The local correction device 31 receives a tracking information set 70 from the head-mounted display 26, processes the tracking information set, and sends the processed tracking information set 71 to the server 30. The server 30 is configured to generate a rendered image 43 using the processed tracking information 71 and send it to the local correction device 31. The local correction device 31 then generates a corrected rendered image 44 using correction logic and sends it to the head-mounted display.
[0042] Figure 8 is a block diagram of an image correction system 85 according to this embodiment, which incorporates a head-mounted display 26, a local correction device 31, one or more auxiliary or environmental sensors 99, and a server 30.
[0043] The head-mounted display 31 may include one or more processors 90 and one or more memory devices 91. In some embodiments, the (one or more) processors 90 and the (one or more) memory devices 91 may be located outside the head-mounted display 26. The one or more processors 90 may execute software programs and / or instructions for displaying images on the display 92. Furthermore, the (one or more) processors 90 may include multiple microprocessors, one or more "general-purpose" microprocessors, one or more dedicated microprocessors, and / or one or more application-specific integrated circuits (ASICS), and / or one or more reduced instruction set (RISC) processors. The (one or more) memory devices 91 may include one or more storage devices that can store machine-readable instructions and / or processor-executable instructions (e.g., firmware or software) for the (one or more) processors 90 to execute, such as instructions related to coordinating the display of virtual objects. Therefore, the (single or duplicate) memory device 91 can store, for example, control software for facilitating adjustment of the display of virtual objects, lookup tables, and configuration data. In some embodiments, the (single or duplicate) processor 90 and the (single or duplicate) memory device 91 may be located outside the head-mounted display 26. The (single or duplicate) memory device 91 may include volatile memory (e.g., random access memory (RAM)) and / or non-volatile memory (e.g., read-only memory (ROM)), flash memory, hard drives, and / or any other suitable tangible non-temporary machine-readable media such as optical, magnetic, or solid-state storage media. The head-mounted display 26 may also include one or more sensors 93 (e.g., hand tracking sensors, eye tracking sensors, inertial measurement units, and microphones).
[0044] The local correction device 31 receives tracking information from the head-mounted display 26 and rendered images 43 from the server 30. The local correction device 31 includes a (one or multiple) processor 95, a (one or multiple) memory device 96, an input / output (I / O) port 97, and a communication circuit 98. The I / O port 97 can receive tracking information from the head-mounted display 26 or generated images from the server 30. The processor 95 can execute software programs to reduce the size of the tracking information or to enhance the tracking information. The (one or multiple) processor 95 can also execute software programs and / or instructions to adjust images such as virtual features displayed on the head-mounted display 26. The memory device 96 may include one or more storage devices and can store machine-readable instructions and / or processor-executable instructions (e.g., firmware or software) for the (one or multiple) processor 95 to execute, such as instructions relating to adjusting images based on tracking information. The communication circuit 98 facilitates wireless communication (Ethernet, WAN, etc.) and / or wired communication (HDMI®, USB, etc.) between the head-mounted display 26 and the server 30.
[0045] The server 30 may include one or more processors 101, one or more memory devices 103, and a communication circuit 105. The (one or more) processors 101 may execute software programs for generating rendering images based at least partially on the received tracking information. The (one or more) memory devices 103 may include one or more storage devices and may store machine-readable instructions and / or processor-executable instructions (e.g., firmware or software) for the (one or more) processors 101 to execute, such as instructions related to AR / VR amusement park ride logic and updates. The communication circuit 105 facilitates a wireless streaming connection, or in an embodiment, a wired connection, with the local correction device 31.
[0046] While only some features of the present invention have been illustrated and described herein, many modifications and changes will come to mind for those skilled in the art. Therefore, it should be understood that the appended claims are intended to cover all such modifications and changes that are part of the actual spirit of the invention.
[0047] The claimed technologies described herein refer to and apply to tangible objects and specific examples of a practical nature that are not abstract, intangible, or purely theoretical, but which certainly improve the art. Furthermore, if any of the claims appended to the end of this specification contain one or more elements designated as "...means for (performing) (a function)" or "...steps for (performing) (a function)," such elements should be interpreted in accordance with 112(f) of the United States Patent Act. On the other hand, any claim containing elements designated in any other form should not be interpreted in accordance with 112(f) of the United States Patent Act. [Explanation of Symbols]
[0048] 12 Image Correction Systems 14 Amusement park rides 16 Guests 18 Vehicles 20 Transportation routes 24 Real-world environment 26 Head-mounted displays 28 Virtual Features 30 servers 31 Local Correction Device
Claims
1. An image correction system, It is a head-mounted display, A display configured to display one or more corrected images, and One or more sensors configured to generate tracking information over a certain period of time, including initial tracking information associated with a first point in time and the most recent tracking information associated with a second point in time after the first point in time. Including a head-mounted display, A server configured to generate one or more rendering images based at least partially on the initial tracking information, It is a local correction device, The tracking information is received from the head-mounted display. The initial tracking information is transmitted to the server. The server receives one or more rendering images, One or more corrected images are generated based on the one or more rendered images and the latest tracking information. The one or more correction images described above are transmitted to the head-mounted display. A local correction device configured as follows, An image correction system equipped with the following features.
2. The image correction system according to claim 1, wherein the local correction device is configured to generate one or more corrected images by correcting lens distortion associated with the head-mounted display.
3. The image correction system according to claim 2, wherein the local correction device is configured to generate one or more corrected images by applying one or more of the following: time warp correction, space warp correction, lens distortion coefficient correction, or ray latch correction.
4. The image correction system according to claim 3, wherein the lens distortion coefficient correction is performed after applying one or more of the time warp correction, space warp correction, or ray latch correction.
5. The image correction system according to claim 1, wherein the local correction device and the head-mounted display are coupled to a vehicle.
6. The image correction system according to claim 1, wherein the local correction device is attached to or embedded in the housing of the head-mounted display.
7. The image correction system according to claim 1, wherein the local correction device is wirelessly connected to the server.
8. The image correction system according to claim 7, wherein the local correction device is coupled to the head-mounted display via a wired connection.
9. The image correction system according to claim 1, wherein the local correction device transmits the one or more correction images to one or fewer head-mounted displays.
10. The image correction system according to claim 1, wherein the one or more sensors include at least a camera and an inertial measurement unit (IMU) that generate tracking information.
11. The image correction system according to claim 10, wherein the tracking information generated by the camera is transmitted to the local correction device and then to the server, and the tracking information generated by the IMU is transmitted to the local correction device and not to the server.
12. An image correction method, On the server, one or more rendered images of an augmented reality / virtual reality (AR / VR) environment are generated, including at least one frame rendered based on tracking information. In the local correction device, receiving one or more rendering images, Applying a first-stage correction based on the latest tracking information to the one or more rendering images mentioned above, In order to generate one or more corrected images, a second-stage correction, which is distortion correction associated with the head-mounted display, is applied to the one or more rendered images. The one or more correction images described above are transmitted to the head-mounted display. Image correction methods, including those mentioned above.
13. The image correction method according to claim 12, wherein the head-mounted display includes a projector, and the distortion correction is a function of the distortion of the display surface on which the projected image is displayed.
14. The image correction method according to claim 12, wherein the head-mounted display includes a lens, and the distortion correction is a function of the distortion of the lens.
15. The image correction method according to claim 12, wherein the local correction device or the head-mounted display is configured to apply a third-stage correction to the one or more rendered images.
16. The image correction method according to claim 12, wherein the tracking information is generated in the head-mounted display and transmitted to the server via the local correction device.
17. The image correction method according to claim 12, wherein the latest tracking information is generated in the head-mounted display and transmitted to the local correction device but not to the server.
18. A non-temporary computer-readable medium containing processor-executable code, wherein the processor-executable code, when executed by the processor, Receiving one or more rendered images of an augmented reality / virtual reality (AR / VR) environment, including at least one frame rendered based on tracking information, Applying a first-stage correction based on the latest tracking information to the one or more rendering images mentioned above, In order to generate one or more corrected images, a second-stage correction, which is distortion correction associated with the head-mounted display, is applied to the one or more rendered images. The one or more correction images described above are transmitted to the head-mounted display. A non-temporary computer-readable medium that causes the processor to perform the above-mentioned action.
19. The aforementioned processor-executable code, when executed by the processor, Receiving the tracking information from the head-mounted display, The tracking information is transmitted to a server configured to generate one or more rendering images of the AR / VR environment. A non-temporary computer-readable medium according to claim 18, wherein the processor performs the operation.
20. The non-temporal computer-readable medium according to claim 18, wherein the distortion correction is one or more of time warp correction, space warp correction, lens distortion coefficient correction, or ray latch correction.