Systems and methods for mediated augmented physical interaction

Abstract

To provide an augmented reality (AR) experience as part of an attraction, such as a thrill ride, in an amusement park or theme park.SOLUTION: A system includes an image generation system for: generating streaming media of a real-world environment based on image data captured via a camera of a mobile device; generating augmentations superimposed on the streaming media of the real-world environment based on data relating to user interaction with the mobile device; and transmitting the streaming media of the real-world environment along with the augmentations to be displayed on a display of the mobile device. The system also includes a physical interaction system for: determining physical effects to be implemented for a physical object in the real-world environment based on the data relating to the user interaction with the mobile device; and transmitting a signal to the physical object to implement the determined physical effects.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to and the benefit of U.S. Provisional Patent Application No. 62 / 780,669, entitled "SYSTEMS AND METHODS FOR MEDIATED AUGMENTED PHYSICAL INTERACTION," filed December 17, 2018, which is hereby incorporated by reference in its entirety for all purposes.

[0002] (Technical field) The subject matter disclosed herein relates to amusement park attractions and, more particularly, to facilitating guest interaction with physical objects at an amusement park via mobile devices. [Background technology]

[0003] Since the early 20th century, the popularity of amusement parks (or theme parks) has increased significantly. This has resulted in an increased demand for amusement parks and an associated increase in competition. It is therefore desirable to add more entertainment value and a greater number of attractions to amusement parks. Adding larger attractions, such as rides and shows, generally provides an amusement park with the capacity to handle more visitors. However, such attractions tend to draw more visitors and become the hub of visitor traffic. Furthermore, simply adding traditional rides without adding additional layers of interest may not be sufficient to capture enough visitor attention to address visitor traffic issues or provide an advantage over competitors. It has therefore been recognized that systems and methods are desirable that facilitate the distribution of visitor traffic and / or increase the level of entertainment value.

[0004] This section is intended to introduce the reader to various aspects of art that may be related to various aspects of the inventive technique described and / or claimed herein. The discussion herein is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. As such, it should be understood that these statements are to be read in this light, and not as admissions of prior art. Summary of the Invention [Problem to be solved by the invention]

[0005] Certain embodiments within the scope of the present invention are summarized below. These embodiments are not intended to limit the scope of the disclosure, but rather, the embodiments are intended only to provide a brief summary of certain disclosed embodiments. Indeed, the disclosure may include a variety of forms that may be similar to or different from the embodiments set forth below. [Means for solving the problem]

[0006] In one embodiment, the amusement park system includes an image generation system configured to be communicatively coupled to the mobile device. The image generation system is configured to generate streaming media of the real-world environment based at least in part on image data captured via a camera of the mobile device. The image generation system is also configured to generate one or more augmentations superimposed on the streaming media of the real-world environment based at least in part on data associated with user interactions with the mobile device. The image generation system is further configured to transmit the streaming media of the real-world environment along with the one or more augmentations displayed on a display of the mobile device. The amusement park system also includes a physical interaction system configured to be communicatively coupled to the mobile device. The physical interaction system is configured to determine one or more physical effects to be implemented on physical objects in the real-world environment based at least in part on data associated with user interactions with the mobile device. The physical interaction system is also configured to transmit signals to the physical objects to implement the determined one or more physical effects.

[0007] In one embodiment, the method includes receiving a real-time video data stream from a mobile device via an image generation system. The method also includes generating, via the image generation system, a visualization of a real-world environment of the amusement park based at least in part on the received real-time video data stream. The method further includes overlaying an augmented reality (AR) image on the visualization of the real-world environment based at least in part on data associated with a user interaction with the mobile device via the image generation system. The method further includes determining, via a physical interaction system, one or more physical effects to be implemented on a physical object in the real-world environment based at least in part on data associated with a user interaction with the mobile device. The method also includes transmitting, via the image generation system, the overlaid AR image with the visualization of the real-world environment to the mobile device. The method further includes transmitting a signal via the physical interaction system to the physical object to implement the determined one or more physical effects.

[0008] These and other features, aspects, and advantages of the present disclosure will become better understood when read in conjunction with the accompanying drawings, in which like characters represent like parts throughout the drawings, in which: [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an amusement park including one or more attractions, according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram of a system for providing an AR experience to the amusement park shown in FIG. 1 according to an embodiment of the present disclosure. [Diagram 3] FIG. 2 illustrates a mobile device, an auxiliary device, and a physical object, according to an embodiment of the disclosure. [Figure 4]1 illustrates an example of a user accessing a hidden cave through an unlocked amusement park door based on the user physically interacting with a given number of physical objects using a mobile device (and / or an auxiliary device) in accordance with an embodiment of the present disclosure. [Diagram 5] 1 illustrates an example of a puzzle that can be solved based on user interaction with a mobile device (and / or an auxiliary device) according to an embodiment of the present disclosure. [Figure 6] FIG. 2 illustrates a method of operating the system described herein, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] One or more specific embodiments of the present disclosure are described herein. In order to provide a concise description of these embodiments, not all features of an actual implementation will be described herein. It will be appreciated that, as with any engineering or design project, the development of any such actual implementation will require numerous implementation-specific decisions to be made to achieve the developer's particular goals, which may vary from implementation to implementation, including compliance with system and business related constraints. It will be further appreciated that such a development effort may be complex and time consuming, but will be a routine exercise in design, fabrication, and manufacture for those of ordinary skill in the art having the benefit of this disclosure.

[0011] When introducing elements of various embodiments of the invention, the articles "a," "an," and "the" are intended to mean that there are one or more of the element. The terms "comprising," "including," and "having" are inclusive and mean that there may be additional elements other than the listed elements. Further, it is to be understood that references to "one embodiment" or "embodiments" of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the listed features.

[0012] The present embodiments relate to systems and methods for providing an augmented reality (AR) experience as part of an attraction, such as a thrill ride, at an amusement or theme park. In certain embodiments, patrons of the amusement park may use mobile devices and other auxiliary devices that facilitate both the manipulation of physical objects located within the amusement park as well as the AR experience. Specifically, in certain embodiments, the mobile device may be configured to capture real-time video data (e.g., video captured during live use and transmitted substantially in real time) of a real-world environment (e.g., aspects of a physical amusement park). Furthermore, in certain embodiments, the image generation system may receive the real-time video data (e.g., live video transmitted substantially in real time) from the mobile device and render the video stream of the real-world environment together with various AR images to the mobile device that are displayed on the display of the mobile device.

[0013] For example, in certain embodiments, the image generation system can render AR images on the mobile device based at least in part on the location and / or orientation of the mobile device, the proximity of the mobile device, detected manipulations of the mobile device (e.g., movements of the mobile device, patron interactions via a user interface of the mobile device, etc.) and manipulations of the auxiliary device by the patron, local settings entered into the mobile device by the patron, global settings of the amusement park (e.g., entered by an owner of the amusement park), previous interactions performed via the mobile device, and so forth, relative to certain physical objects located in the real-world environment of the amusement park. In certain embodiments, the mobile device, the auxiliary device, the physical objects of the amusement park, and so forth can be monitored by a monitoring system, which can communicate data (e.g., identification, location, orientation, movement, speed, acceleration, and so forth) regarding the mobile device, the auxiliary device, the physical objects of the amusement park, and so forth to the image generation system, such that the image generation system can generate AR images based at least in part on the data regarding the mobile device, the auxiliary device, the physical objects of the amusement park, and so forth.

[0014] Additionally, in certain embodiments, the monitoring system can be configured to communicate with a physical interaction system configured to trigger physical effects (e.g., motion effects, sound effects, haptic feedback effects, color effects, scent effects, and the like) of physical objects located within the real-world environment of the amusement park based at least in part on data regarding the mobile device, the auxiliary device, the physical objects of the amusement park, and the like. In certain embodiments, physical effects of physical objects located within the real-world environment of the amusement park can be triggered based at least in part on data received from the mobile device, the auxiliary device, the physical objects, and the like.

[0015] In this manner, the mobile devices, auxiliary devices, image generation systems, monitoring systems, and physical interaction systems may be used to create and display AR experiences, as well as to facilitate patrons' ability to trigger physical effects of physical objects located within the real-world environment of the amusement park, thereby enhancing patrons' overall experience within the amusement park. However, it should be appreciated that the techniques described herein are not limited to amusement park attraction applications, but may be extended to any of a variety of applications, such as, for example, medical applications (e.g., image-guided surgery, non-invasive imaging analysis), engineering design applications (e.g., engineering model development), manufacturing, construction, and maintenance applications (e.g., product manufacturing, new building construction, automobile repair), academic and / or vocational training applications, exercise applications (e.g., bodybuilding and weight loss models), television (TV) applications (e.g., weather and news), and the like.

[0016] With the above in mind, it may be useful to describe embodiments of amusement parks, such as the example amusement park 10 shown in Figure 1. As shown, the amusement park 10 may include a thrill ride 12, amusement park facilities 14 (e.g., restaurants, souvenir shops, and the like), and additional amusement park attractions 16. In certain embodiments, the thrill ride 12 may include a roller coaster or other similar thrill ride, and thus may further include a closed-loop track or system of closed-loop track 18 (e.g., several miles of track 18). The track 18 may provide infrastructure over which passenger vehicles 20 may traverse.

[0017] As the passenger ride vehicle 20 traverses the track 18, the passengers 22 may be provided with a moving tour of the scene (e.g., the facility 14, the additional amusement attractions 16, and the like) in the area around or near the thrill ride 12. For example, this may include the environment surrounding the thrill ride 12 (e.g., the building that fully or partially houses the thrill ride 12). The passengers 22 may find the thrill ride 12 to be an extremely enjoyable experience, and in certain embodiments, it may be useful to enhance the experience of the passengers 22 as they ride the thrill ride 12, for example, by enhancing the thrill element 12 of the thrill ride. In particular, rather than only physically viewing the facility 14 (e.g., restaurants, souvenir shops, and the like), the additional amusement attractions 16 (e.g., themed attractions associated with the amusement park 10), or other patrons 24 within the amusement park 10, it may be useful to provide the passengers 22 with an AR experience as the ride vehicle 20 traverses the track 18.

[0018] Similarly, while amusement park patrons 24 may find the facilities 14 of the amusement park 10, the additional amusement attractions 16, and the like to be enjoyable in and of themselves, in certain embodiments it may be useful to enhance the experience of patrons 24 as they move through the facilities 14 of the amusement park 10, the additional amusement attractions 16, and the like. In particular, it may be useful to provide AR experiences to patrons 24 with respect to the facilities 14 of the amusement park 10, the additional amusement attractions 16, and the like.

[0019] 1 , in certain embodiments, passengers 22 and other patrons 24 of amusement park 10 may use mobile devices 26 that facilitate interaction with physical objects 28 located in a real-world environment surrounding thrill rides 12, facilities 14, additional amusement attractions 16, and the like of amusement park 10. Specifically, as described in more detail herein, in certain embodiments, mobile devices 26 may be configured to provide AR views of the environment surrounding thrill rides 12, facilities 14, additional amusement attractions 16, and the like, and to enable passengers 22 and other patrons 24 of amusement park 10 to trigger physical effects (e.g., movement effects, sound effects, haptic feedback effects, color effects, scent effects, and the like) of physical objects 28 through manipulation of mobile devices 26. Furthermore, as described in more detail herein, in certain embodiments, the mobile devices 26 may be used in conjunction with auxiliary devices 30 to enable passengers 22 and other patrons 24 of the amusement park 10 to trigger physical effects of physical objects 28. In certain embodiments, the auxiliary devices 30 may be objects related to the theme of the amusement park, such as toy guns, flags, sticks, and the like.

[0020] FIG. 2 is a schematic diagram of a system 32 for providing an AR experience for the amusement park 10 shown in FIG. 1 according to an embodiment of the present disclosure. As shown in FIG. 2, in a particular embodiment, the system 32 can include one or more mobile devices 26 that can be used by the passengers 22 and other patrons 24 of the amusement park 10. The mobile devices 26 can be communicatively coupled to an image generation system 34 (e.g., within the amusement park 10) via a wireless network 36 (e.g., a wireless local area network (WLAN)), a wireless wide area network (WWAN), a near field communication (NFC) network, or other suitable wireless network). The mobile devices 26 can be used to create an AR experience for the real-world environment of the amusement park 10. Specifically, as described in more detail herein, the AR experience viewable via the mobile devices can be a real-time video that includes a real-world image 38 electronically merged with one or more AR images 40 (e.g., virtual augmentation). The term "real-time" means that the image is acquired and / or provided in a time frame that is substantially close to the actual viewing time.

[0021] In certain embodiments, the mobile device 26 may be a mobile phone (e.g., a smartphone), a tablet computer, a wearable device (e.g., glasses, goggles, watches, etc.), or any other suitable device that may be carried by the passenger 22 or other patrons 24 throughout the amusement park 10 and that may display AR experiences for the passenger 22 or other patrons 24. In certain embodiments, the mobile device 26 may include a number of orientation and position sensors 42 (e.g., accelerometers, magnetometers, gyroscopes, motion tracking sensors such as global positioning system (GPS) receivers, electromagnetic and solid state motion tracking sensors, etc.), which may be used to track the position, location, orientation, movement, etc. of the mobile device 26. Similarly, in certain embodiments, characteristics (e.g., geometric aspects or markings) of the mobile device 26 may be monitored by a monitoring system 44 (e.g., in certain embodiments, one or more cameras) to determine the position, location, orientation, movement, etc. of the mobile device 26. Further, in certain embodiments, passengers 22 and other passengers 24 may be monitored by a monitoring system 44 (e.g., cameras) to identify the position, location, orientation, movement, and the like, of passengers 22 and other passengers 24.

[0022] Further, in certain embodiments, the mobile device 26 may include one or more cameras 46 configured to capture images and videos (e.g., real-world images) and display the captured images and videos (and AR images 40) via one or more displays 48 of the mobile device 26. In certain embodiments, to support the creation of AR experiences, the mobile device 26 may include processing circuitry, such as a processor 50 and a memory 52. ​​The processor 50 may be operatively coupled to the memory 52 to execute instructions for at least partially implementing the techniques of the present disclosure (e.g., in cooperation with the image generation system 34) to generate real-world images 38 merged with the AR images 40 to enhance the experience of the passengers 22 and other patrons 24. These instructions may be encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as the memory 52 and / or other storage devices. In certain embodiments, the processor 50 may be a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), or any other similar processor configuration.

[0023] In certain embodiments, the one or more displays 48 may each include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, or other similar display useful for displaying the real world images 38 and the AR images 40 to the passengers 22 and other patrons 24. In certain embodiments, the one or more displays 48 may each include, for example, an opaque or see-through LCD or an opaque or see-through OLED display useful for allowing the passengers 22 and other patrons 24 to view the real world images 38 and the AR images 40 appearing on the display 48 while maintaining the ability to view the actual physical real-world environment of the amusement park 10 through the respective display 48.

[0024] One or more cameras 46 of the mobile device 26 may be used to capture real-time video data (e.g., live video) of the real-world environment of the amusement park 10. In certain embodiments, the mobile device 26 may transmit (e.g., wirelessly via one or more communication interfaces 54 included in the mobile device 26) the real-time video data captured via the one or more cameras 46 to the image generation system 34 for processing. However, in other embodiments, the real-time video data captured via the one or more cameras 46 may be processed on the mobile device 26 via the processor 50. Indeed, in certain embodiments, the image generation system 34 may be integrated with the mobile device 26. Furthermore, in certain embodiments, the mobile device 26 may also transmit orientation data, position data, motion tracking data, and the like, acquired and / or derived based at least in part on data acquired via the orientation and position sensor 42 of the mobile device 26.

[0025] In certain embodiments, the image generation system 34, which may also include processing circuitry such as a processor 56 (e.g., a general-purpose processor or other processor) and a memory 58, as described in more detail herein, may process real-time video data (e.g., live video) and orientation data, location data, and motion tracking data received from the mobile device 26 and / or the monitoring system 44 via one or more communication interfaces 60 of the image generation system 34. Furthermore, in certain embodiments, in addition to the AR image 40 (e.g., AR augmentation), the image generation system 34 may also trigger one or more motion effects, sound effects, haptic feedback effects, color effects, scent effects, and the like implemented via internal components of the mobile device 26, which may occur simultaneously with the appearance of the AR image 40 on the mobile device 26. Instructions for performing these functions may be encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as the memory 58 and / or other storage device. In certain embodiments, the processor 56 may be a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), or any other similar processor configuration.

[0026] In certain embodiments, the image generation system 34 can use orientation, position, and motion tracking data received from the mobile device 26 and / or the monitoring system 44 to generate a frame of reference for registering real-time video data to the generated real-world images 38 and AR images 40. In certain embodiments, the image generation system 34 can constantly (e.g., in real-time) update the rendering of the real-world images 38 to reflect changes in the orientation, position, and / or motion of the mobile device 26. To aid in the visibility of the rendered images, in certain embodiments, the image generation system 34 can render images (e.g., the real-world images 38 and the AR images 40) at a real-time rate of about 20 frames per second (FPS) or more, about 30 FPS or more, about 40 FPS or more, about 50 FPS or more, about 60 FPS or more, about 90 FPS or more, or about 120 FPS or more.

[0027] In certain embodiments, as described in more detail herein, the image generation system 34 may also generate and render one or more AR images 40 superimposed on the real-world image 38 to create a complete AR experience for the passenger 22 or other patron 24. For example, in certain embodiments, the image generation system 34 may utilize one or more of the discussed video merging and / or optical merging techniques to superimpose the AR image 40 onto the real-world image 38 such that the passenger 22 and other patron 24 perceive the real-world physical environment of the amusement park 10 (e.g., provided as video data rendered via the respective displays 48) as well as the AR image 40 (e.g., virtual augmentation) via one or more displays 48 of the mobile device 26. Specifically, in certain embodiments, the image generation system 34 may render a view of the AR image 40 that is temporally and spatially equivalent to the real-world image 38, such that the real-world image 38 is seen as a background upon which the AR image 40 is overlaid.

[0028] In certain embodiments, the image generation system 34 may also generate one or more brightness, lighting, or shading models, and / or other photorealistic rendering models to generate real-world images 38 and AR images 40 that are adjusted to accurately reflect the contrast and brightness of the real-world physical environment (e.g., sunny day, partly cloudy day, cloudy day, evening, night) in which the real-world images 38 and AR images 40 are rendered. For example, in certain embodiments, to enhance the photorealism of the real-world images 38 and AR images 40, the image generation system 34 may receive weather-related data from one or more weather forecasting and / or prediction systems (e.g., global forecast systems, Doppler radar, and the like) and use the weather-related data or other similar data to adjust the contrast, brightness, and / or other lighting effects of the real-world images 38 and / or the AR images 40. In other embodiments, image generation system 34 may adjust the contrast, brightness, and / or other lighting effects of real world image 38 and / or AR image 40 based at least in part on illumination detected from one or more light sensors included in mobile device 26 or based at least in part on real-time video data captured by camera 46 of mobile device 26. Additionally, as described above, image generation system 34 may constantly update (e.g., in real-time) the rendering of AR image 40 to reflect changes in orientation, position, and / or movement of mobile device 26.

[0029] Further, in certain embodiments, as described in more detail herein, the image generation system 34 may generate and render one or more AR images 40 based at least in part on data associated with the manipulation of the mobile device 26, which may be detected by the mobile device 26. For example, in certain embodiments, the one or more displays 48 of the mobile device 26 may include a touch screen display (or other user interface) through which the passenger 22 or other patron 24 may attempt to manipulate (e.g., via finger swipes, finger pinches, finger taps, and the like) objects displayed on the one or more displays 48, whether the objects displayed on the one or more displays 48 are real-world objects as part of the real-world image 38 or AR-generated objects of the AR image 40 depicted via the one or more displays 48. The manipulation detected by the mobile device 26 may then be communicated to the image generation system 34 via one or more communication interfaces 54 of the mobile device 26, and the image generation system 34 may generate an AR image 40 that is displayed on one or more displays 48 of the mobile device 26 based at least in part on the detected manipulation.

[0030] Further, in certain embodiments, as described in more detail herein, the image generation system 34 can generate and render one or more AR images 40 based on the orientation, position, and motion tracking data of the mobile device 26, which may be detected by the orientation and position sensors 42 of the mobile device 26. The orientation, position, and motion tracking data of the mobile device 26 can be communicated to the image generation system 34 via one or more communication interfaces 54 of the mobile device 26, and the image generation system 34 can generate the AR images 40 to be displayed on one or more displays 48 of the mobile device 26 based at least in part on the orientation, position, and motion tracking data of the mobile device 26.

[0031] Similarly, in certain embodiments, as described in more detail herein, the image generation system 34 can generate and render one or more AR images 40 based on the orientation data, location data, and motion tracking data of the mobile device 26, which may be detected by the monitoring system 44. For example, in certain embodiments, the monitoring system 44 can include one or more tracking devices 62 (e.g., in certain embodiments, one or more cameras) configured to detect the orientation data, location data, and motion tracking data of the mobile device 26. The monitoring system 44 can then communicate the orientation data, location data, and motion tracking data of the mobile device 26 to the image generation system 34 via one or more communication interfaces 64 of the monitoring system 44, and the image generation system 34 can generate the AR image 40 to be displayed on one or more displays 48 of the mobile device 26 based at least in part on the orientation data, location data, and motion tracking data of the mobile device 26. Further, in certain embodiments, the monitoring system 44 may include processing circuitry, such as a processor 66 (e.g., a general-purpose processor or other processor) and memory 68, which may process orientation data, location data, and motion tracking data of the mobile device 26 detected by one or more tracking devices 62, for example, to convert the orientation data, location data, and motion tracking data of the mobile device 26 into a format suitable for the image generation system 34. Instructions for performing these functions may be encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as the memory 68 and / or other storage device. In certain embodiments, the processor 66 may be a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), or any other similar processor configuration.

[0032] Further, in certain embodiments, as described in more detail herein, the image generation system 34 can generate and render one or more AR images 40 based at least in part on data related to the auxiliary device 30, which can be used in cooperation with the mobile device 26. In certain embodiments, the auxiliary device 30 can be an object related to an amusement park theme, such as a toy gun, a flag, a stick, and the like. In certain embodiments, the auxiliary device 30 can be a passive object (i.e., does not include sensors, control circuits, and the like), whose orientation, location, and movement can be tracked, for example, by the monitoring system 44. In such embodiments, passive features (e.g., geometric aspects or markings, radio frequency identification (RFID) tags, and the like) or more active features (e.g., wireless transmitters, such as ultra-wideband (UWB) wireless transmitters, and the like) can be monitored by the monitoring system 44 to determine the orientation, location, and movement, and the like, of the auxiliary device 30. Additionally, in certain embodiments, the auxiliary device 30 may include multiple orientation and position sensors 70 (e.g., accelerometers, magnetometers, gyroscopes, Global Positioning System (GPS) receivers, electromagnetic and solid-state motion tracking sensors, and the like) that can be used by the monitoring system 44 to track the orientation, position, movement, and the like, of the auxiliary device 30.

[0033] Thus, in certain embodiments, as described in more detail herein, the image generation system 34 can generate and render one or more AR images 40 based at least in part on data regarding the operation of the auxiliary device 30, which may be detected by the auxiliary device 30. For example, in certain embodiments, the orientation, position, movement, and so forth, detected by the orientation and position sensor 70 of the auxiliary device 30, may be communicated to the image generation system 34 via one or more communication interfaces 72 of the auxiliary device 30, and the image generation system 34 can generate the AR image 40 to be displayed on the one or more displays 48 of the mobile device 26 based at least in part on the orientation data, position data, and motion tracking data of the auxiliary device 30.

[0034] Similarly, in certain embodiments, as described in more detail herein, the image generation system 34 can generate and render one or more AR images 40 based at least in part on the orientation data, position data, and motion tracking data of the auxiliary device 30, which may be detected by the monitoring system 44. For example, in certain embodiments, the one or more tracking devices 62 of the monitoring system 44 can detect the orientation data, position data, and motion tracking data of the auxiliary device 30. The monitoring system 44 can then communicate the orientation data, position data, and motion tracking data of the auxiliary device 30 to the image generation system 34 via one or more communication interfaces 64 of the monitoring system 44, and the image generation system 34 can generate the AR image 40 displayed on the one or more displays 48 of the mobile device 26 based at least in part on the orientation data, position data, and motion tracking data of the auxiliary device 30. Further, in certain embodiments, the processing circuitry of the monitoring system 44 may process the orientation data, position data, and motion tracking data of the auxiliary device 30 detected by one or more tracking devices 62, for example, to convert the orientation data, position data, and motion tracking data of the auxiliary device 30 into a format suitable for the image generation system 34.

[0035] Similarly, in certain embodiments, as described in more detail herein, the image generation system 34 may generate and render one or more AR images 40 based at least in part on orientation data, location data, and motion tracking data of the passengers 22 and other patrons 24, which may be detected by the monitoring system 44. For example, in certain embodiments, the one or more tracking devices 62 of the monitoring system 44 may detect orientation data, location data, and motion tracking data of the passengers 22 and other patrons 24 (including, e.g., detecting the bodies of the passengers 22 and other patrons 24, detecting hand gesture patterns of the passengers and other patrons 24, and so forth). The monitoring system 44 may then communicate the orientation, location, and motion tracking data of the passengers 22 and other guests 24 to the image generation system 34 via one or more communication interfaces 64 of the monitoring system 44, and the image generation system 34 may generate the AR image 40 displayed on the one or more displays 48 of the mobile device 26 based at least in part on the orientation, location, and motion tracking data of the passengers 22 and other guests 24. Further, in certain embodiments, the processing circuitry of the monitoring system 44 may process the orientation, location, and motion tracking data of the passengers 22 and other guests 24 detected by the one or more tracking devices 62, e.g., convert the orientation, location, and motion tracking data of the passengers 22 and other guests 24 into a format suitable for the image generation system 34.

[0036] Similarly, in certain embodiments, as described in more detail herein, the image generation system 34 may generate and render one or more AR images 40 based at least in part on the thrill rides 12, the facilities 14, the additional amusement attractions 16, and / or other physical objects 28 of the amusement park 10 that may be detected by the monitoring system 44. For example, in certain embodiments, the one or more tracking devices 62 of the monitoring system 44 may detect three-dimensional props positioned around the amusement park 10, two-dimensional graphics within the environment of the amusement park 10, and other physical features associated with the thrill rides 12, the facilities 14, the additional amusement attractions 16, and / or other physical objects 28 of the amusement park 10. The monitoring system 44 can then communicate the orientation data, location data, and motion tracking data associated with the physical features of the thrill rides 12, the facilities 14, the additional amusement attractions 16, and / or other physical objects 28 of the amusement park 10 to the image generation system 34 via one or more communications interfaces 64 of the monitoring system 44, and the image generation system 34 can generate AR images 40 to be displayed on one or more displays 48 of the mobile device 26 based at least in part on the orientation data, location data, and motion tracking data associated with the physical features of the thrill rides 12, the facilities 14, the additional amusement attractions 16, and / or other physical objects 28 of the amusement park 10.

[0037] Additionally, in certain embodiments, the processing circuitry of the monitoring system 44 may process the orientation data, position data, and motion tracking data of the physical features of the thrill rides 12, the facilities 14, the additional amusement attractions 16, and / or other physical objects 28 of the amusement park 10, for example, to convert the orientation data, position data, and motion tracking data of the physical features of the thrill rides 12, the facilities 14, the additional amusement attractions 16, and / or other physical features of the amusement park 10 into a format suitable for the image generation system 34. In this manner, the image generation system 34 may use these detected physical features of the thrill rides 12, the facilities 14, the additional amusement attractions 16, and / or other physical objects 28 of the amusement park 10 as AR markers (i.e., reference points such as position indexes, and the like) to determine the one or more AR images 40 to be generated. In this manner, these physical feature markers assist in the initial placement and orientation of the image generation system 34 in the three-dimensional world space, which helps to improve the AR visualization and engage the passengers 22 and other users 24 in the three-dimensional world space.

[0038] Further, in certain embodiments, as described in more detail herein, the image generation system 34 may generate and render one or more AR images 40 based at least in part on data regarding particular physical objects 28 of the amusement park 10. For example, in certain embodiments, the physical objects 28 may include processing circuitry, such as a processor 74 and a memory 76. The processor 74 may be operatively coupled to the memory 76 to execute instructions for determining and / or setting particular operational states of the particular physical objects 28. These instructions may be encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as the memory 76 and / or other storage devices. In certain embodiments, the processor 74 may be a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), or any other similar processor configuration. In particular embodiments, the processor 74 of a particular physical object 28 may be configured to determine whether a particular physical effect has been activated for the physical object 28 and communicate this determination to the image generation system 34 via one or more communication interfaces 78 of the physical object 28, and the image generation system 34 may generate the AR image 40 that is displayed on one or more displays 48 of the mobile device 26 based at least in part on the determination.

[0039] In addition to facilitating the generation of AR experiences via mobile devices 26, in certain embodiments, the systems and methods described herein enable the generation of physical effects (e.g., motion effects, sound effects, haptic feedback effects, color effects, scent effects, and the like) for particular physical objects 28 of amusement park 10. Specifically, physical interaction system 80 may be used to generate physical effects for particular physical objects 28 of amusement park 10. In certain embodiments, physical interaction system 80, which may include processing circuitry such as processor 82 (e.g., a general purpose processor or other processor) and memory 84, as described in more detail herein, may process data from mobile devices 26, physical objects 28, auxiliary devices 30, image generation system 34, and monitoring system 44 to determine the type of physical effect to be generated for particular physical objects 28. In particular, it will be understood that all of the different types of data described as being communicated between the mobile device 26, the physical objects 28, the auxiliary device 30, the image generation system 34, and the monitoring system 44 may also be communicated between the components of the amusement park 10 and one or more communication interfaces 86 of the physical interaction system 80, which may determine, based at least in part on this data, particular physical effects to be generated for particular physical objects 28, and may communicate instructions to those physical objects 28 to generate the physical effects. These instructions may be encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as the memory 84 and / or other storage devices. In certain embodiments, the processor 82 may be a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), or any other similar processor configuration.

[0040] From the perspective of the functionality of the system 32 shown in FIG. 2, FIG. 3 illustrates the mobile device 26, the auxiliary device 30, and the physical object 28, according to an embodiment of the present disclosure. The real-world environment of the amusement park 10 illustrated in FIG. 3 (e.g., including the depicted physical object 28) can be displayed on a display 48 of the mobile device 26. Specifically, as described in more detail herein, the physical object 28 and the remainder of the real-world environment of the amusement park 10 can be captured by one or more cameras 46 of the mobile device 26, transmitted as a real-world image 38 to the image generation system 34 (e.g., via communications interfaces 54, 60), combined with the AR image 40 by the image generation system 34, and sent back to the mobile device 26 (e.g., via communications interfaces 54, 60) for display via the display 48 of the mobile device 26.

[0041] Additionally, as described in more detail herein, one or more user interactions with the mobile device 26 may be detected and used to generate the AR imagery 40 and implement certain physical effects (e.g., certain physical objects 28) in the amusement park real-world environment 10. In particular embodiments, the one or more user interactions with the mobile device 26 may include a pattern of movement with respect to the mobile device 26. For example, in particular embodiments, as shown by path 88, the pattern of movement may be captured via a display 48 (e.g., a touchscreen display) of the mobile device 26, such as via finger swipes, finger pinches, finger taps, and the like, of the display 48. Alternatively or in addition, in certain embodiments, as shown by path 90, the pattern of movement can be the orientation, location, and / or movement of the mobile device 26 detected by the tracking device 62 of the monitoring system 44, the orientation and location sensor 42 of the mobile device 26, or the tracking device 62 of the monitoring system 44 and the orientation and location sensor 42 of the mobile device 26 operating in conjunction with one another (e.g., one tracking the orientation, location, movement, etc. of the mobile device 26 and the other calibrating or otherwise verifying the orientation, location, movement, etc. of the mobile device 26). Alternatively or in addition, in certain embodiments, as shown by path 92, the pattern of movement can be the orientation, location, and / or movement of the auxiliary device 30 detected by the tracking device 62 of the monitoring system 44.

[0042] When the patterns of movement 88, 90, 92 are detected, for example, by the mobile device 26 and / or the monitoring system 44, the patterns of movement 88, 90, 92 can be transmitted to the image generation system 34 (e.g., via the communications interfaces 54, 60, 64), and the image generation system 34 can generate one or more augmentations (i.e., AR images 40) to be superimposed on the real world image 38 (e.g., streamed to the mobile device 26) based at least in part on the patterns of movement 88, 90, 92. In particular, in certain embodiments, the image generation system 34 can include a pattern recognition software module as part of instructions encoded in the memory 58 of the image generation system 34 for execution by the processor 56 of the image generation system 34, which is configured to analyze data regarding the patterns of movement 88, 90, 92 to determine whether the patterns of movement 88, 90, 92 correspond to known patterns. For example, in certain embodiments, the image generation system 34 may perform a comparison of the pattern of movements 88, 90, 92 with a number of known movement patterns, e.g., stored in the memory 58 of the image generation system 34, to determine one or more augmentations (i.e., AR image 40) to be superimposed on the real world image 38. In certain embodiments, if the pattern of movements 88, 90, 92 corresponds to a particular stored movement pattern, the image generation system 34 may superimpose a particular visualization onto the real world image 38. As one non-limiting example, in certain embodiments, if the pattern of movements 88, 90, 92 is identified as corresponding to a stored movement pattern corresponding to a particular magic spell, a visual effect 94 (e.g., related to the particular magic spell) may be generated as part of the AR image 40 to be superimposed on the real world image 38. For example, in certain embodiments, the visual effect 94 may represent electric sparks in the case of an electricity-based magic spell, flames in the case of a fire-based magic spell, and so forth.

[0043] Similarly, when a pattern of movements 88, 90, 92 is detected, for example, by mobile device 26 and / or monitoring system 44, the pattern of movements 88, 90, 92 may be transmitted to physical interaction system 80 (e.g., via communications interfaces 54, 64, 86), and physical interaction system 80 may determine one or more physical objects 28 (i.e., which may be part of real-world image 38) based at least in part on the patterns of movements 88, 90, 92. In particular, in certain embodiments, physical interaction system 80 may include a pattern recognition software module as part of instructions encoded in memory 84 of physical interaction system 80 for execution by processor 82 of physical interaction system 80, that is configured to analyze data regarding the patterns of movements 88, 90, 92 to determine whether the patterns of movements 88, 90, 92 correspond to known patterns. For example, in certain embodiments, the physical interaction system 80 may perform a comparison of the pattern of movements 88, 90, 92 with a plurality of known movement patterns, e.g., stored in the physical interaction memory 84, such that the system 80 may determine one or more physical effects to be implemented with respect to one or more physical objects. In certain embodiments, if the pattern of movements 88, 90, 92 corresponds to a particular stored movement pattern, the physical interaction system 80 may implement a particular physical effect with respect to one or more physical objects 28. As one non-limiting example, in certain embodiments, if the pattern of movements 88, 90, 92 is identified as corresponding to a stored movement pattern corresponding to a particular magic spell, a particular physical effect with respect to one or more physical objects 28 (e.g., associated with a particular magic spell) may be determined by the physical interaction system 80, and the physical interaction system 80 may communicate with the particular physical objects 28 (e.g., via the communication interfaces 78, 86) and implement a physical effect 96, e.g., via a physical movement mechanism 98 associated with the physical objects 28.For example, in certain embodiments, the physical effect 96 may be electric sparks emanating from the physical object 28 generated by a power source in the case of an electricity-based magical spell, flames emanating from the physical object 28 generated by an ignition system in the case of a fire-based spell, movement of a portion of the physical object 28 (as shown in FIG. 3 ) in the case of levitation, and so forth.

[0044] In this manner, the embodiments described herein enable both an AR experience via the display 48 of the mobile device 26, and a perceived physical interaction with the physical objects 28 located around the real-world environment of the amusement park 10. In certain embodiments, the physical interaction with the physical objects 28 located around the real-world environment of the amusement park 10 may not involve movement of a particular portion of the physical object 28, but rather may include a more complex physical interaction with the amusement park itself. For example, in certain embodiments, if a particular user of the mobile device 26 (and / or the auxiliary device 30, in certain embodiments) performs a certain number of interactions with other physical objects 28 located around the real-world environment of the amusement park 10 in a particular visit to the amusement park 10, the system 32 may determine that the user is able to physically interact with a particular door that may provide entrance to, for example, a hidden cavern similarly located around the amusement park 10.

[0045] FIG. 4 illustrates an example of a user accessing a hidden cave 100 through an unlocked door 102 of an amusement park 10 based on the user physically interacting with a given number of physical objects 28 using a mobile device 26 (and / or an auxiliary device 30), according to an embodiment of the present disclosure. In certain embodiments, the ability to unlock the door 102 may be the only first requirement, and the second requirement is that the user perform another interaction with the mobile device 26 (and / or an auxiliary device 30, in certain embodiments). For example, in certain embodiments, the user may be required to perform a specific door-opening spell via the mobile device 26 (and / or an auxiliary device 30, in certain embodiments), e.g., by triggering a specific pattern of movements 88, 90, 92, such as those described with respect to FIG. 3. When the user performs the specific pattern of movements 88, 90, 92, the physical interaction system 80 may cause the door 102 to open (e.g., as a physical effect 96), e.g., by actuating a locking mechanism 98 (e.g., a magnetic lock, in certain embodiments). Additionally, in certain embodiments, a particular visual effect 94 may be displayed on the display 48 of the mobile device 26 by the image generation system 34 in conjunction with the opening of the door 102.

[0046] Further, in certain embodiments, the AR experience and physical interaction with the physical objects 28 of the amusement park 10 enabled by the embodiments described herein may allow a user to solve a particular puzzle presented via a particular physical object 28. FIG. 5 illustrates an example of a puzzle 104 that may be solved based on user interaction with the mobile device 26 (and / or the auxiliary device 30), in accordance with embodiments of the present disclosure. In the illustrated embodiment, the plurality of physical objects 28 may be physically interacted with via user interaction with the mobile device 26 (and / or the auxiliary device 30, in certain embodiments). In certain embodiments, once the plurality of physical objects 28 are physically moved to the correct positions via user interaction with the mobile device 26 (and / or the auxiliary device 30, in certain embodiments), a particular visual effect 94 may be displayed on the display 48 of the mobile device 26 by the image generation system 34. For example, in the embodiment shown in FIG. 5 , user interaction with the mobile device 26 (and / or the auxiliary device 30 in certain embodiments) may cause physical objects 28 to be placed within respective compartments 106 of the base 108, as indicated by arrows 110, thereby solving the puzzle, and in certain embodiments may cause the image generation system 34 to display visual effects 94 on the display 48 of the mobile device 26.

[0047] 6 illustrates a method 112 of operating the system 32 described herein, according to an embodiment of the present disclosure. For example, in certain embodiments, the method 112 includes generating, via the image generation system 34, streaming media of the real-world environment of the amusement park 10 based at least in part on image data (e.g., real-world image 38) captured via one or more cameras 46 of the mobile device 26 (block 114). Further, in certain embodiments, the method 112 includes generating, via the image generation system 34, one or more augmentations (e.g., one or more AR images 40) superimposed on the streaming media of the real-world environment of the amusement park 10 based at least in part on data regarding user interactions with the mobile device 26 (block 116). Further, in certain embodiments, the method 112 includes transmitting, via the image generation system 34, the streaming media of the real-world environment of the amusement park 10 along with the one or more augmentations displayed on the display 48 of the mobile device 46 (block 118). Further, in certain embodiments, method 112 includes determining, via physical interaction system 80, one or more physical effects 96 to be implemented on physical objects 28 in the real-world environment of amusement park 10 based at least in part on the data regarding the user interaction with mobile device 26 (block 120). Further, in certain embodiments, method 112 includes transmitting, via physical interaction system 80, a signal to physical objects 28 to implement the determined one or more physical effects 96.

[0048] While only certain features of the present disclosure have been illustrated and described herein, numerous modifications and changes will occur to those skilled in the art. It is therefore to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the present disclosure. It is further to be understood that certain elements of the disclosed embodiments can be combined with or substituted for one another. [Explanation of symbols]

[0049] 26 Mobile Devices 30 Auxiliary Devices 28 Physical Objects 36 Wireless Network 34 Image Generation System 38 Real World Images 40 AR images 44 Monitoring System 80 Physical Interaction System

Claims

1. 1. A system comprising:

1. An image generation system configured to be communicatively coupled to a mobile device, comprising: generating streaming media of a real-world environment based at least in part on image data captured via a camera of the mobile device; generating one or more augmentations superimposed on the streaming media of the real-world environment based at least in part on data regarding user interactions with the mobile device; transmitting streaming media of the real-world environment along with the one or more augmentations displayed on a display of the mobile device; An image generating system configured as above; A physical interaction system configured to be communicatively coupled to the mobile device, determining one or more physical effects to be implemented on physical objects in the real-world environment based at least in part on data regarding the user interactions with the mobile device; transmitting a signal to the physical object to implement the determined one or more physical effects; A physical interaction system configured as follows: A system comprising:

2. The system of claim 1 , wherein the data regarding user interactions includes movement patterns.

3. 3. The system of claim 2, wherein the image generation system includes a pattern recognition module configured to analyze data regarding the user interaction to determine whether the movement pattern corresponds to a known pattern, and the image generation system is configured to generate the one or more augmentations based at least in part on whether the movement pattern corresponds to the known pattern.

4. 3. The system of claim 2, wherein the physical interaction system includes a pattern recognition module configured to analyze data regarding the user interaction to determine whether the movement pattern corresponds to a known pattern, and the physical interaction system is configured to determine the one or more physical effects to be implemented based at least in part on whether the movement pattern corresponds to the known pattern.

5. The system of claim 2 , wherein the patterns of movement include patterns of movement of the mobile device detected by a monitoring system that monitors the mobile device.

6. The system of claim 2 , wherein the movement pattern comprises input received via a user interface of the mobile device.

7. The system of claim 1 , wherein the user interaction comprises a movement of the mobile device.

8. 8. The system of claim 7, wherein the image generation system is configured to generate streaming media of the real-world environment based at least in part on an orientation of the mobile device, a position of the mobile device, a movement of the mobile device, or a combination thereof.

9. The system of claim 8 , wherein the image generation system is configured to receive an orientation of the mobile device, a location of the mobile device, a movement of the mobile device, or a combination thereof from the mobile device.

10. 10. The system of claim 8, comprising a monitoring system configured to monitor physical attributes of the mobile device to determine an orientation of the mobile device, a location of the mobile device, a movement of the mobile device, or a combination thereof.

11. The system of claim 1 , wherein the user interaction comprises input received through a user interface of the mobile device.

12. 2. The system of claim 1, wherein the image generation system is configured to generate the one or more augmentations superimposed on the streaming media of the real-world environment based at least in part on data regarding user interactions with an auxiliary device used in cooperation with the mobile device.

13. 2. The system of claim 1, wherein the physical interaction system is configured to determine the one or more physical effects to be implemented on the physical objects in the real-world environment based at least in part on data regarding user interactions with an auxiliary device used in cooperation with the mobile device.

14. 2. The system of claim 1 , wherein the image generation system is configured to generate the one or more augmentations superimposed on the streaming media of the real-world environment based at least in part on data regarding one or more physical objects located in the real-world environment.

15. 1. A method comprising: receiving a real-time video data stream from a mobile device via an image generation system; generating, via the image generation system, a visualization of a real-world environment based at least in part on the received real-time video data stream; via the image generation system, overlaying an augmented reality (AR) image onto the visualization of the real-world environment based at least in part on data regarding a user interaction with the mobile device; determining, via a physical interaction system based at least in part on data regarding the user interaction with the mobile device, one or more physical effects to be implemented on physical objects in the real-world environment; transmitting the overlaid AR image together with the visualization of the real-world environment to the mobile device via the image generation system; transmitting, via the physical interaction system, a signal to the physical object to implement the determined one or more physical effects; A method comprising:

16. The method of claim 15 , wherein the data regarding user interactions includes movement patterns.

17. 17. The method of claim 16, comprising overlaying the AR imagery onto a visualization of the real-world environment via the image generation system based at least in part on whether the pattern of movement corresponds to a known pattern.

18. 17. The method of claim 16, comprising determining, via the physical interaction system, the one or more physical effects to be implemented on the physical objects in the real-world environment based at least in part on whether the pattern of movement corresponds to a known pattern.

19. The method of claim 16 , wherein the patterns of movement include patterns of movement of the mobile device detected by a monitoring system monitoring the mobile device.

20. The method of claim 16 , wherein the movement pattern comprises input received via a user interface of the mobile device.

21. 16. The method of claim 15, comprising overlaying the AR imagery on the visualization of the real-world environment based at least in part on an orientation of the mobile device, a position of the mobile device, a movement of the mobile device, or a combination thereof, via the image generation system.

22. 22. The method of claim 21, comprising using one or more sensors of the mobile device to detect an orientation of the mobile device, a location of the mobile device, a movement of the mobile device, or a combination thereof.

23. 22. The method of claim 21, comprising using a monitoring system that monitors the mobile device to detect an orientation of the mobile device, a location of the mobile device, a movement of the mobile device, or a combination thereof.

Images