Systems and methods for mediating enhanced physical interactions

Abstract

A system (32) includes an image generation system (34) for generating a streaming media of a real-world environment based at least in part on image data (38) captured via a camera (46) of a mobile device (26), generating one or more augmentations (40) superimposed on the streaming media of the real-world environment based at least in part on data related to user interactions with the mobile device (26), and transmitting the streaming media of the real-world environment along with the one or more augmentations (40) to be displayed on a display (48) of the mobile device (26). The system (32) also includes a physical interaction system (80) for determining one or more physical effects (96) to be implemented for a physical object (28) in the real-world environment based at least in part on data related to user interactions with the mobile device (26), and transmitting a signal to the physical object (28) to implement the determined one or more physical effects (96).

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Application No. 62 / 780,669, filed December 17, 2018, entitled “Systems and Methods for Mediated Augmented Physical Interaction,” which is hereby incorporated by reference in its entirety for all purposes. Background Art

[0003] The subject matter disclosed herein relates to amusement park attractions, and more particularly, to facilitating guest interaction with physical objects at an amusement park via mobile devices.

[0004] Since the early 20th century, amusement parks (or theme parks) have seen a significant increase in popularity. Consequently, there has been an increased demand for amusement parks and a consequent increase in competition. Consequently, there has been a desire to add more entertaining and numerous attractions to amusement parks. The addition of large-scale attractions, such as rides and shows, typically provides amusement parks with additional capacity to handle the increased number of guests. However, such attractions tend to attract more visitors and become a focal point for guest traffic. Furthermore, adding traditional rides without an added layer of intrigue may not be sufficient to generate sufficient guest interest to address guest traffic issues or provide an advantage over competitors. Therefore, it is now recognized that systems and methods are desirable that facilitate the distribution of guest traffic and / or provide an increased level of entertainment value.

[0005] This section is intended to introduce the reader to various aspects of the art that may be related to various aspects of the present technology described and / or claimed herein. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Therefore, it should be understood that these statements are to be read in this light and not as admissions of prior art. Summary of the Invention

[0006] Certain embodiments commensurate in scope with the present disclosure are outlined herein. These embodiments are not intended to limit the scope of the present disclosure, but rather are intended only to provide a brief overview of possible forms of the present embodiment. Indeed, the present embodiment may encompass various forms that may be similar or different from the embodiments set forth herein.

[0007] In an embodiment, an amusement park system includes an image generation system configured to be communicatively coupled to a mobile device. The image generation system is configured to generate a streaming media of a real-world environment based, at least in part, on image data captured via a camera of the mobile device. The image generation system is further configured to generate one or more enhancements superimposed on the streaming media of the real-world environment based, at least in part, on data related to user interactions with the mobile device. The image generation system is further configured to transmit the streaming media of the real-world environment along with the one or more enhancements to be displayed on a display of the mobile device. The amusement park system also includes a physical interaction system configured to be communicatively coupled to the mobile device. The physical interaction system is configured to determine, based, at least in part, on the data related to user interactions with the mobile device, one or more physical effects to be implemented for physical objects in the real-world environment. The physical interaction system is further configured to transmit a signal to the physical object to implement the determined one or more physical effects.

[0008] In an embodiment, a method includes receiving a real-time video data stream from a mobile device via an image generation system. The method also includes generating, via the image generation system, a visualization of a real-world environment of an amusement park based at least in part on the received real-time video data stream. The method also includes overlaying, via the image generation system, an augmented reality (AR) image onto the visualization of the real-world environment based at least in part on data related to user interactions with the mobile device. Additionally, the method includes determining, via a physical interaction system, one or more physical effects to be implemented for physical objects in the real-world environment based at least in part on data related to user interactions with the mobile device. The method also includes transmitting, via the image generation system, the overlaid AR image along with the visualization of the real-world environment to the mobile device. The method also includes transmitting, via the physical interaction system, a signal to the physical object to implement the determined one or more physical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] These and other features, aspects, and advantages of the present disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings, in which like characters represent like parts throughout the several views, and in which:

[0010] Figure 1 An amusement park including one or more attractions according to an embodiment of the present disclosure is illustrated;

[0011] Figure 2 According to an embodiment of the present disclosure, Figure 1 Schematic diagram of a system for providing AR experience in an amusement park as shown in FIG;

[0012] Figure 3 illustrates a mobile device, an auxiliary device, and a physical object according to an embodiment of the present disclosure;

[0013] Figure 4 illustrates an example of a user accessing a hidden cave via a door to an amusement park, which door has been unlocked based on the user physically interacting with a given number of physical objects using a mobile device (and / or auxiliary device) in accordance with an embodiment of the present disclosure;

[0014] Figure 5 illustrates examples of puzzles that may be solved based on user interaction with a mobile device (and / or secondary device) according to embodiments of the present disclosure; and

[0015] Figure 6 A method of operating the system described herein according to an embodiment of the present disclosure is illustrated. DETAILED DESCRIPTION

[0016] One or more specific embodiments of the present disclosure will be described herein. In order to provide a concise description of these embodiments, all features of the actual implementation may not be described in the specification. It should be understood that in the development of any such actual implementation, as in any engineering or design project, many implementation-specific decisions must be made to achieve the developer's specific goals, such as meeting system-related and business-related constraints, which may vary depending on the implementation. In addition, it should be understood that such development work may be complex and time-consuming, but will still be a routine task of design, production, and manufacturing for those of ordinary skill in the art who benefit from the present disclosure.

[0017] When introducing elements of various embodiments of the present disclosure, the articles "a," "an," and "the" are intended to mean that there are one or more of the elements. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements other than the listed elements. Furthermore, it should be understood that references to "one embodiment" or "an embodiment" of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features.

[0018] The present embodiments relate to systems and methods for providing an augmented reality (AR) experience as part of an attraction (such as a thrill ride) in an amusement park or theme park. In some embodiments, patrons of the amusement park can use mobile devices and other auxiliary devices that facilitate the AR experience and facilitate the manipulation of physical objects set up within the amusement park. Specifically, in some embodiments, the mobile device can be configured to capture real-time video data (e.g., video captured during on-site use and transmitted substantially in real time) of a real-world environment (e.g., aspects of a physical amusement park). Additionally, in some embodiments, an image generation system can receive real-time video data (e.g., live video transmitted substantially in real time) from the mobile device and can render the video stream of the real-world environment along with various AR images to the mobile device for display on the mobile device's display.

[0019] For example, in some embodiments, the image generation system may render AR images to the mobile device based at least in part on, for example, in some embodiments, the position and / or orientation of the mobile device, the proximity of the mobile device to specific physical objects disposed in the real-world environment of the amusement park, detected manipulation of the mobile device (e.g., movement of the mobile device, interaction of a patron via a user interface of the mobile device, etc.) and manipulation of an auxiliary device by the patron, local settings input into the mobile device by the patron, global settings for the amusement park (e.g., as input by an owner of the amusement park), prior interactions performed via the mobile device, etc. In some embodiments, the mobile device, the auxiliary device, the physical objects of the amusement park, etc. may be monitored by a monitoring system, which may transmit data (e.g., identification, position, orientation, movement, velocity, acceleration, etc.) related to the mobile device, the auxiliary device, the physical objects of the amusement park, etc. to the image generation system, so that the image generation system may generate AR images based at least in part on the data related to the mobile device, the auxiliary device, the physical objects of the amusement park, etc.

[0020] Additionally, in some embodiments, the monitoring system may be configured to communicate with a physical interaction system configured to cause physical effects (e.g., movement effects, sound effects, haptic feedback effects, color effects, scent effects, etc.) on physical objects disposed within the real-world environment of the amusement park based at least in part on data related to the mobile device, the auxiliary device, the physical objects of the amusement park, etc. In some embodiments, the physical effects on the physical objects disposed within the real-world environment of the amusement park may be caused based at least in part on data received from the mobile device, the auxiliary device, the physical objects, etc.

[0021] In this manner, by using mobile devices, auxiliary devices, image generation systems, monitoring systems, and physical interaction systems to create and display AR experiences and facilitate the ability of patrons to cause physical effects of physical objects set within the real-world environment of the amusement park, the overall experience of patrons within the amusement park can be enhanced. However, it should be understood that the technology described herein may not be limited to amusement park attraction applications, but may also be extended to any of a variety of applications, such as, for example, medical applications (e.g., image-guided surgery, non-invasive imaging analysis), engineering design applications (e.g., engineering model development), manufacturing, construction, and maintenance applications (e.g., product manufacturing, new building construction, auto repair), academic and / or vocational training applications, exercise applications (e.g., fitness and weight loss models), television (TV) applications (e.g., weather and news), etc.

[0022] In view of the foregoing, describing an amusement park (such as Figure 1 1 may be useful. As shown, the amusement park 10 may include a thrill ride 12, amusement park facilities 14 (e.g., restaurants, souvenir shops, etc.), and additional attractions 16. In some embodiments, the thrill ride 12 may include a roller coaster or other similar thrill ride and, therefore, may further include a closed-loop track or closed-loop track system 18 (e.g., several miles of track 18). The track 18 may be provided as an infrastructure over which passenger ride vehicles 20 may traverse.

[0023] As a passenger rides the vehicle 20 across the track 18, the ride passengers 22 may be provided with a moving tour of the scenery (e.g., the ride 14, the additional attractions 16, etc.) throughout or near the thrill ride 12. For example, this may include the environment surrounding the thrill ride 12 (e.g., the building that fully or partially houses the thrill ride 12). While the ride passengers 22 may find the thrill ride 12 to be a very enjoyable experience, in certain embodiments, it may be useful to enhance the experience of the ride passengers 22 while riding the thrill ride 12 by, for example, enhancing the thrill factor of the thrill ride 12. Specifically, instead of having only a physical view of the ride 14 (e.g., a restaurant, a souvenir shop, etc.), the additional attractions 16 (e.g., attractions themed relative to the amusement park 10), or other patrons 24 within the amusement park 10, it may be useful to provide the ride passengers 22 with an AR experience as the ride vehicle 20 traverses the track 18.

[0024] Similarly, while amusement park patrons 24 may find the rides 14, additional attractions 16, etc., of the amusement park 10 to be entertaining on their own, in certain embodiments, it may be useful to enhance the experience of the patrons 24 as they walk through the rides 14, additional attractions 16, etc. of the amusement park 10. In particular, it may also be useful to provide the patrons 24 with AR experiences regarding the rides 14, additional attractions 16, etc. of the amusement park 10.

[0025] For example, Figure 1 As illustrated in FIG, in some embodiments, ride passengers 22 and other patrons 24 of the amusement park 10 can use a mobile device 26 that facilitates interaction with physical objects 28 disposed in the real-world environment surrounding the thrill ride 12, facilities 14, additional attractions 16, and the like of the amusement park 10. Specifically, as described in greater detail herein, in some embodiments, the mobile device 26 can be configured to provide an augmented reality view of the environment surrounding the thrill ride 12, facilities 14, additional attractions 16, and the like, and also enable the ride passengers 22 and other patrons 24 of the amusement park 10 to cause physical effects (e.g., movement effects, sound effects, haptic feedback effects, color effects, scent effects, etc.) of the physical objects 28 via manipulation of the mobile device 26. Furthermore, as also described in greater detail herein, in some embodiments, the mobile device 26 can be used in conjunction with an auxiliary device 30 to enable the ride passengers 22 and other patrons 24 of the amusement park 10 to cause physical effects of the physical objects 28. In some embodiments, the auxiliary device 30 can be an object related to the theme of the amusement park, such as a toy gun, a flag, a wand, and the like.

[0026] Figure 2 According to an embodiment of the present disclosure, Figure 1 FIG. 1 is a schematic diagram of a system 32 for providing an AR experience in an amusement park 10. Figure 2 As illustrated in FIG, in some embodiments, a system 32 may include one or more mobile devices 26 that can be used by ride passengers 22 and other patrons 24 of the amusement park 10. The mobile devices 26 may be communicatively coupled to an image generation system 34 (e.g., within the amusement park 10) via a wireless network 36 (e.g., a wireless local area network (WLAN), a wireless wide area network (WWAN), a near field communication (NFC) network, or any other suitable wireless network). The mobile devices 26 may be used to create an AR experience for the real-world environment of the amusement park 10. Specifically, as described in more detail herein, the AR experience viewable via the mobile device may be a real-time video that includes a real-world image 38 electronically fused with one or more AR images 40 (e.g., virtual augmentations). The term "real-time" indicates that the images are obtained and / or provided in a time frame that is substantially close to the actual viewing time.

[0027] In some embodiments, the mobile device 26 may be a mobile phone (e.g., a smartphone), a tablet computer, a wearable device (e.g., such as glasses, goggles, a watch, etc.), or any other suitable device that can be carried by a ride passenger 22 or other patrons 24 throughout the amusement park 10 and that can display an AR experience for the ride passenger 22 or other patrons 24. In some embodiments, the mobile device 26 may include a plurality of orientation and position sensors 42 (e.g., accelerometers, magnetometers, gyroscopes, global positioning system (GPS) receivers, motion tracking sensors (such as electromagnetic and solid-state motion tracking sensors), etc.), which can be used to track the position, orientation, orientation, motion, etc. of the mobile device 26. Similarly, in some embodiments, features (e.g., geometry or markings) of the mobile device 26 may be monitored by a monitoring system 44 (e.g., one or more cameras, in some embodiments) to determine the position, orientation, orientation, motion, etc. of the mobile device 26. Furthermore, in some embodiments, the ride passengers 22 and other patrons 24 may be monitored by the monitoring system 44 (e.g., cameras) to identify the position, orientation, orientation, motion, etc. of the ride passengers 22 and other patrons 24.

[0028] Additionally, in some embodiments, mobile device 26 may include one or more cameras 46 configured to capture images and video (e.g., real-world images) and to display the captured images and video (as well as AR images 40) via one or more displays 48 of mobile device 26. In some embodiments, to support the creation of AR experiences, mobile device 26 may include processing circuitry, such as a processor 50 and memory 52. ​​Processor 50 may be operatively coupled to memory 52 to execute instructions for at least partially performing (e.g., in conjunction with image generation system 34) the presently disclosed techniques for generating real-world images 38 fused with AR images 40 to enhance the experience of passengers 22 and other patrons 24. These instructions may be encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as memory 52 and / or other storage devices. In some embodiments, processor 50 may be a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), or some other similar processor configuration.

[0029] In some embodiments, one or more displays 48 may each include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, or other similar display useful in displaying the real-world images 38 and the AR images 40 to the ride passengers 22 and other patrons 24. In some embodiments, one or more displays 48 may each include an opaque or see-through LCD or an opaque or see-through OLED display, which may be useful in allowing, for example, the ride passengers 22 and other patrons 24 to view the real-world images 38 and the AR images 40 appearing on the displays 48, while retaining the ability to see the actual and physical real-world environment of the amusement park 10 through the corresponding displays 48.

[0030] The one or more cameras 46 of the mobile device 26 can be used to capture real-time video data (e.g., live video) of the real-world environment of the amusement park 10. In some embodiments, the mobile device 26 can then transmit the real-time video data captured via the one or more cameras 46 to the image generation system 34 (e.g., wirelessly via one or more communication interfaces 54 included in the mobile device 26) for processing. However, in other embodiments, the real-time video data captured via the one or more cameras 46 can be processed on the mobile device 26 via the processor 50. Indeed, in some embodiments, the image generation system 34 can be integrated with the mobile device 26. Additionally, in some embodiments, the mobile device 26 can also transmit orientation data, position data, motion tracking data, etc., obtained and / or derived at least in part based on data obtained via the orientation and position sensor 42 of the mobile device 26.

[0031] In certain embodiments, as described in greater detail herein, image generation system 34 (which may also include processing circuitry such as processor 56 (e.g., a general-purpose processor or other processor) and memory 58) may process real-time video data (e.g., live video) and orientation data, location data, and motion tracking data received from mobile device 26 and / or from monitoring system 44 via one or more communication interfaces 60 of image generation system 34. Furthermore, in certain embodiments, in addition to AR image 40 (e.g., AR augmentation), image generation system 34 may also trigger one or more motion effects, sound effects, haptic feedback effects, color effects, fragrance effects, etc., to be implemented via internal components of mobile device 26, which may coincide with the appearance of AR image 40 on mobile device 26. Instructions for performing these functions may be encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as memory 58 and / or other storage device. In certain embodiments, processor 56 may be a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), or some other similar processor configuration.

[0032] In certain embodiments, the image generation system 34 can use the orientation data, position data, and motion tracking data received from the mobile device 26 and / or from the monitoring system 44 to generate a reference frame to register the real-time video data with the generated real-world image 38 and AR image 40. In certain embodiments, the image generation system 34 can continuously update (e.g., in real time) the rendering of the real-world image 38 to reflect changes in orientation, position, and / or motion relative to the mobile device 26. To aid in the viewability of the rendered image, in certain embodiments, the image generation system 34 can render the image (e.g., the real-world image 38 and the AR image 40) at a real-time rate of greater than or equal to approximately 20 frames per second (FPS), greater than or equal to approximately 30 FPS, greater than or equal to approximately 40 FPS, greater than or equal to approximately 50 FPS, greater than or equal to approximately 60 FPS, greater than or equal to approximately 90 FPS, or greater than or equal to approximately 120 FPS.

[0033] In some embodiments, as described in greater detail herein, the image generation system 34 can also generate and render one or more AR images 40 that are superimposed on the real-world image 38 to create a complete AR experience for the ride passengers 22 or other patrons 24. For example, in some embodiments, the image generation system 34 can utilize one or more of the video fusion and / or optical fusion techniques discussed above to superimpose the AR images 40 on the real-world image 38, so that the ride passengers 22 and other patrons 24 perceive the real-world physical environment of the amusement park 10 (e.g., provided as rendered video data via the respective displays 48) along with the AR images 40 via one or more displays 48 of the mobile device 26 (e.g., virtually augmented). Specifically, in some embodiments, the image generation system 34 can render the view of the AR images 40 that is temporally and spatially proportional to the real-world image 38, such that the real-world image 38 appears as a background overlaid with the AR images 40.

[0034] In some embodiments, the image generation system 34 may also generate one or more brightness, lighting, or shading models and / or other photorealistic rendering models to generate the real-world image 38 and the AR image 40 that are adjusted to accurately reflect the contrast and brightness of the real-world physical environment (e.g., sunny, partly cloudy, cloudy, evening, night) when rendering the real-world image 38 and the AR image 40. For example, in some embodiments, to increase the realism of the real-world image 38 and the AR image 40, the image generation system 34 may receive weather-related data from one or more weather forecast and / or prediction systems (e.g., global forecast systems, Doppler radar, etc.) and use the weather-related data or other similar data to adjust the contrast, brightness, and / or other lighting effects of the real-world image 38 and / or the AR image 40. In other embodiments, the image generation system 34 may adjust the contrast, brightness, and / or other lighting effects of the real-world image 38 and / or the AR image 40 based at least in part on lighting detected from one or more light sensors included in the mobile device 26 or based at least in part on real-time video data captured by the camera 46 of the mobile device 26. Furthermore, as previously described, the image generation system 34 may continually update (eg, in real-time) the rendering of the AR image 40 to reflect changes in orientation, position, and / or motion relative to the mobile device 26 .

[0035] Additionally, in some embodiments, as described in greater detail herein, the image generation system 34 can generate and render one or more AR images 40 based at least in part on data related to manipulations of the mobile device 26 that can be detected by the mobile device 26. For example, in some embodiments, the one or more displays 48 of the mobile device 26 can include a touchscreen display (or other user interface) through which the ride passenger 22 or other patron 24 can attempt to manipulate (e.g., via a finger swipe, finger pinch, finger tap, etc.) an object displayed on the one or more displays 48, whether the object is a real-world object that is part of the real-world image 38 or an AR-generated object of the AR image 40 depicted via the one or more displays 48. The manipulations detected by the mobile device 26 can then be communicated to the image generation system 34 via the one or more communication interfaces 54 of the mobile device 26, and the image generation system 34 can generate the AR image 40 to be displayed on the one or more displays 48 of the mobile device 26 based at least in part on the detected manipulations.

[0036] Furthermore, in certain embodiments, as described in greater detail herein, the image generation system 34 can generate and render one or more AR images 40 based at least in part on orientation data, position data, and motion tracking data of the mobile device 26 that can be detected by the orientation and position sensor 42 of the mobile device 26. The orientation data, position data, and motion tracking data of the mobile device 26 can be transmitted to the image generation system 34 via one or more communication interfaces 54 of the mobile device 26, and the image generation system 34 can generate the AR images 40 to be displayed on one or more displays 48 of the mobile device 26 based at least in part on the orientation data, position data, and motion tracking data of the mobile device 26.

[0037] Similarly, in some embodiments, as described in greater detail herein, the image generation system 34 can generate and render one or more AR images 40 based at least in part on the orientation data, position data, and motion tracking data of the mobile device 26 that can be detected by the monitoring system 44. For example, in some embodiments, the monitoring system 44 can include one or more tracking devices 62 (e.g., in some embodiments, one or more cameras) that are configured to detect the orientation data, position data, and motion tracking data of the mobile device 26. The monitoring system 44 can then transmit the orientation data, position data, and motion tracking data of the mobile device 26 to the image generation system 34 via one or more communication interfaces 64 of the monitoring system 44, and the image generation system 34 can generate the AR images 40 to be displayed on one or more displays 48 of the mobile device 26 based at least in part on the orientation data, position data, and motion tracking data of the mobile device 26. Furthermore, in some embodiments, the monitoring system 44 may include processing circuitry, such as a processor 66 (e.g., a general-purpose processor or other processor) and memory 68, and may process the orientation data, position data, and motion tracking data of the mobile device 26 detected by one or more tracking devices 62, for example, to convert the orientation data, position data, and motion tracking data of the mobile device 26 into a form suitable for the image generation system 34. Instructions for performing these functions may be encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as memory 68 and / or other storage device. In some embodiments, the processor 66 may be a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), or some other similar processor configuration.

[0038] Additionally, in some embodiments, as described in greater detail herein, the image generation system 34 can generate and render one or more AR images 40 based at least in part on data associated with an auxiliary device 30, which can be used in conjunction with the mobile device 26. In some embodiments, the auxiliary device 30 can be an amusement park-themed object, such as a toy gun, a flag, a wand, or the like. In some embodiments, the auxiliary device 30 can be a passive object (i.e., it does not include sensors, control circuitry, etc.), whose orientation, position, and motion can be tracked, for example, by the monitoring system 44. In such embodiments, certain features 69 of the auxiliary device 30, such as passive features (e.g., geometric shapes or markings, radio frequency identification (RFID) tags, etc.) or more active features (e.g., wireless transmitters, such as ultra-wideband (UWB) radio transmitters, etc.), can be monitored by the monitoring system 44 to determine, among other things, the orientation, position, and motion of the auxiliary device 30. Additionally, in some embodiments, the auxiliary device 30 may include a plurality of orientation and position sensors 70 (e.g., accelerometers, magnetometers, gyroscopes, global positioning system (GPS) receivers, motion tracking sensors (such as electromagnetic and solid-state motion tracking sensors), etc.), which may be used by the monitoring system 44 to track, among other things, the orientation, position, and motion of the auxiliary device 30 .

[0039] Thus, in some embodiments, as described in greater detail herein, the image generation system 34 can generate and render one or more AR images 40 based at least in part on data related to manipulation of the auxiliary device 30 that can be detected by the auxiliary device 30. For example, in some embodiments, the orientation, position, and motion, etc., detected by the orientation and position sensors 70 of the auxiliary device 30 can be transmitted to the image generation system 34 via one or more communication interfaces 72 of the auxiliary device 30, and the image generation system 34 can generate the AR images 40 to be displayed on one or more displays 48 of the mobile device 26 based at least in part on the orientation data, position data, and motion tracking data of the auxiliary device 30.

[0040] Similarly, in some embodiments, as described in more detail herein, the image generation system 34 can generate and render one or more AR images 40 based at least in part on the orientation data, position data, and motion tracking data of the auxiliary device 30 that can be detected by the monitoring system 44. For example, in some embodiments, the one or more tracking devices 62 of the monitoring system 44 can detect the orientation data, position data, and motion tracking data of the auxiliary device 30. The monitoring system 44 can then transmit the orientation data, position data, and motion tracking data of the auxiliary device 30 to the image generation system 34 via one or more communication interfaces 64 of the monitoring system 44, and the image generation system 34 can generate the AR images 40 to be displayed on one or more displays 48 of the mobile device 26 based at least in part on the orientation data, position data, and motion tracking data of the auxiliary device 30. Furthermore, in some embodiments, the processing circuitry of the monitoring system 44 can process the orientation data, position data, and motion tracking data of the auxiliary device 30 detected by the one or more tracking devices 62 to, for example, convert the orientation data, position data, and motion tracking data of the auxiliary device 30 into a form suitable for the image generation system 34.

[0041] Similarly, in some embodiments, as described in greater detail herein, the image generation system 34 can generate and render one or more AR images 40 based at least in part on the orientation data, position data, and motion tracking data of the passengers 22 and other patrons 24 that can be detected by the monitoring system 44. For example, in some embodiments, one or more tracking devices 62 of the monitoring system 44 can detect the orientation data, position data, and motion tracking data of the passengers 22 and other patrons 24 (e.g., including detecting the bodies of the passengers 22 and other patrons 24, detecting patterns of gestures by the passengers and other patrons 24, etc.). The monitoring system 44 can then transmit the orientation data, position data, and motion tracking data of the passengers 22 and other patrons 24 to the image generation system 34 via one or more communication interfaces 64 of the monitoring system 44, and the image generation system 34 can generate the AR images 40 to be displayed on the one or more displays 48 of the mobile device 26 based at least in part on the orientation data, position data, and motion tracking data of the passengers 22 and other patrons 24. In addition, in some embodiments, the processing circuitry of the monitoring system 44 can process the orientation data, position data, and motion tracking data of the riding passengers 22 and other patrons 24 detected by one or more tracking devices 62 to, for example, convert the orientation data, position data, and motion tracking data of the riding passengers 22 and other patrons 24 into a form suitable for the image generation system 34.

[0042] Similarly, in some embodiments, as described in more detail herein, the image generation system 34 may generate and render one or more AR images 40 based at least in part on orientation data, position data, and motion tracking data of the thrill rides 12, facilities 14, additional attractions 16, and / or other physical objects 28 of the amusement park 10 that may be detected by the monitoring system 44. For example, in some embodiments, one or more tracking devices 62 of the monitoring system 44 may detect three-dimensional props located throughout the amusement park 10, two-dimensional graphics in the environment of the amusement park 10, and other physical features associated with the thrill rides 12, facilities 14, additional attractions 16, and / or other physical objects 28 of the amusement park 10. The monitoring system 44 may then transmit the orientation data, position data, and motion tracking data related to the physical features of the thrill rides 12, facilities 14, additional attractions 16, and / or other physical objects 28 of the amusement park 10 to the image generation system 34 via one or more communication interfaces 64 of the monitoring system 44, and the image generation system 34 may generate an AR image 40 to be displayed on one or more displays 48 of the mobile device 26 based at least in part on the orientation data, position data, and motion tracking data related to the physical features of the thrill rides 12, facilities 14, additional attractions 16, and / or other physical objects 28 of the amusement park 10.

[0043] Furthermore, in certain embodiments, the processing circuitry of the monitoring system 44 can process the orientation data, position data, and motion tracking data of the physical features of the thrill rides 12, facilities 14, additional attractions 16, and / or other physical objects 28 of the amusement park 10 to, for example, convert the orientation data, position data, and motion tracking data of the physical features of the thrill rides 12, facilities 14, additional attractions 16, and / or other physical objects 28 of the amusement park 10 into a form suitable for the image generation system 34. Thus, the image generation system 34 can use these detected physical features of the thrill rides 12, facilities 14, additional attractions 16, and / or other physical objects 28 of the amusement park 10 as AR markers (i.e., reference points, such as for location indexing, etc.) for determining one or more AR images 40 to generate. Thus, these physical feature markers aid in the initial placement and orientation of the three-dimensional world space for the image generation system 34, which facilitates both improved AR visualization and tying the ride passengers 22 and other patrons 24 to the three-dimensional world space.

[0044] Furthermore, in certain embodiments, as described in greater detail herein, the image generation system 34 can generate and render one or more AR images 40 based, at least in part, on data associated with certain physical objects 28 of the amusement park 10. For example, in certain embodiments, the physical objects 28 can include processing circuitry, such as a processor 74 and memory 76. The processor 74 can be operatively coupled to the memory 76 to execute instructions for determining and / or setting certain operational states of the particular physical objects 28. These instructions can be encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as the memory 76 and / or other storage device. In certain embodiments, the processor 74 can be a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), or some other similar processor configuration. In certain embodiments, the processor 74 of a particular physical object 28 can be configured to determine whether a particular physical effect has been activated for the physical object 28 and communicate this determination to the image generation system 34 via one or more communication interfaces 78 of the physical object 28. The image generation system 34 can generate the AR image 40 for display on one or more displays 48 of the mobile device 26 based, at least in part, on this determination.

[0045] As described in more detail herein, in addition to facilitating the generation of AR experiences via mobile device 26, in certain embodiments, the systems and methods described herein enable the generation of physical effects (e.g., motion effects, sound effects, haptic feedback effects, color effects, scent effects, etc.) for certain physical objects 28 of amusement park 10. Specifically, a physical interaction system 80 can be used to generate physical effects for certain physical objects 28 of amusement park 10. In certain embodiments, as described in more detail herein, physical interaction system 80 (which can include processing circuitry such as processor 82 (e.g., a general-purpose processor or other processor) and memory 84) can process data from mobile device 26, physical objects 28, auxiliary device 30, image generation system 34, and monitoring system 44 to determine the type of physical effects that should be generated for certain physical objects 28. In particular, it will be understood that all of the different types of data described as being transmitted between the mobile device 26, physical objects 28, auxiliary devices 30, image generation system 34, and monitoring system 44 can also be transmitted to / from those components of the amusement park 10 and one or more communication interfaces 86 of the physical interaction system 80, and that the physical interaction system 80 can determine, based at least in part on this data, certain physical effects that should be generated for certain physical objects 28 and can transmit instructions to those physical objects 28 to generate the physical effects. These instructions can be encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as the memory 84 and / or other storage device. In some embodiments, the processor 82 can be a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), or some other similar processor configuration.

[0046] To depict Figure 2 The functionality of the system 32 shown in FIG. Figure 3 Illustrated are a mobile device 26 , an auxiliary device 30 , and a physical object 28 , in accordance with an embodiment of the present disclosure. Figure 3 , (e.g., including the depicted physical objects 28) can be displayed on the display 48 of the mobile device 26. Specifically, as described in more detail herein, the physical objects 28 and the rest of the real-world environment of the amusement park 10 can be captured by one or more cameras 46 of the mobile device 26, transmitted as real-world images 38 to the image generation system 34 (e.g., via the communication interfaces 54, 60), combined with the AR images 40 by the image generation system 34, and transmitted back to the mobile device 26 for display via the display 48 of the mobile device 26 (e.g., via the communication interfaces 54, 60).

[0047] Furthermore, as described in greater detail herein, one or more user interactions with the mobile device 26 may be detected and used both to generate the AR image 40 and to enable certain physical effects in the real-world environment of the amusement park 10 (e.g., with respect to certain physical objects 28). In certain embodiments, the one or more user interactions with the mobile device 26 may include a moving pattern associated with the mobile device 26. For example, in certain embodiments, as illustrated by path 88, the moving pattern may be captured via the display 48 (e.g., a touch screen display) of the mobile device 26, such as via a finger swipe, finger pinch, finger tap, etc., of the display 48. Alternatively or additionally, in some embodiments, as illustrated by path 90, the pattern of movement may be the orientation, position, and / or motion of the mobile device 26 as detected by the tracking device 62 of the monitoring system 44, the orientation and position sensor 42 of the mobile device 26, or the tracking device 62 of the monitoring system 44 and the orientation and position sensor 42 of the mobile device 26 operating in conjunction with each other (e.g., one tracks the orientation, position, motion, etc. of the mobile device 26 and the other calibrates or otherwise confirms the orientation, position, motion, etc. of the mobile device 26). Alternatively or additionally, in some embodiments, as illustrated by path 92, the pattern of movement may be the orientation, position, and / or motion of the auxiliary device 30 as detected by the tracking device 62 of the monitoring system 44.

[0048] Once the moving patterns 88, 90, 92 have been detected, for example, by the mobile device 26 and / or the monitoring system 44, the moving patterns 88, 90, 92 may be transmitted to the image generation system 34 (e.g., via the communication interfaces 54, 60, 64), and the image generation system 34 may generate one or more augmentations (i.e., AR images 40) based at least in part on the moving patterns 88, 90, 92 to be superimposed on the real-world image 38 (e.g., streamed to the mobile device 26). In particular, in some embodiments, the image generation system 34 may include a pattern recognition software module as part of instructions encoded in the memory 58 of the image generation system 34, which is executed by the processor 56 of the image generation system 34, the processor 56 being configured to analyze data associated with the moving patterns 88, 90, 92 to determine whether the moving patterns 88, 90, 92 correspond to a known pattern. For example, in some embodiments, the image generation system 34 may perform a comparison of the moving patterns 88, 90, 92 with a plurality of known moving patterns, such as those stored in the memory 58 of the image generation system 34, to determine one or more augmentations (i.e., the AR image 40) to be superimposed on the real-world image 38. In some embodiments, if the moving patterns 88, 90, 92 correspond to a particular stored moving pattern, the image generation system 34 may cause certain visualizations to be superimposed on the real-world image 38. As a non-limiting example only, in some embodiments, if the moving patterns 88, 90, 92 are identified as corresponding to a stored moving pattern associated with a particular magic spell, a visual effect 94 (e.g., associated with the particular magic spell) may be generated as part of the AR image 40 to be superimposed on the real-world image 38. For example, in some embodiments, the visual effect 94 may represent sparks in the case of an electricity-based magic spell, flames in the case of a fire-based magic spell, and so on.

[0049] Similarly, once the moving patterns 88, 90, 92 have been detected, for example, by the mobile device 26 and / or the monitoring system 44, the moving patterns 88, 90, 92 may be transmitted to the physical interaction system 80 (e.g., via the communication interfaces 54, 64, 86), and the physical interaction system 80 may determine one or more physical effects to be implemented for one or more physical objects 28 (i.e., which may be part of the real-world image 38) based at least in part on the moving patterns 88, 90, 92. In particular, in some embodiments, the physical interaction system 80 may include a pattern recognition software module as part of instructions encoded in the memory 84 of the physical interaction system 80, which is executed by the processor 82 of the physical interaction system 80, the processor 82 being configured to analyze data associated with the moving patterns 88, 90, 92 to determine whether the moving patterns 88, 90, 92 correspond to a known pattern. For example, in some embodiments, the physical interaction system 80 may perform a comparison of the patterns of movement 88, 90, 92 with a plurality of known patterns of movement, e.g., stored in the memory 84 of the physical interaction system 80, to determine one or more physical effects to be implemented for one or more physical objects 28. In some embodiments, if the patterns of movement 88, 90, 92 correspond to a particular stored pattern of movement, the physical interaction system 80 may cause the certain physical effects to be implemented for the one or more physical objects 28. As just one non-limiting example, in some embodiments, similarly, if the patterns of movement 88, 90, 92 are identified as corresponding to a stored pattern of movement associated with a particular magic spell, the physical interaction system 80 may determine a physical effect 96 (e.g., associated with the particular magic spell) to be implemented for the one or more physical objects 28, and the physical interaction system 80 may communicate with the particular physical object 28 (e.g., via the communication interfaces 78, 86) to cause the physical effect 96 to be implemented, e.g., via a physical actuation mechanism 98 associated with the physical object 28. For example, in some embodiments, physical effect 96 may be an electric spark emanating from physical object 28 as generated by a power source in the case of an electricity-based magic spell, a flame emanating from physical object 28 as generated by an ignition system in the case of a fire-based spell, movement of a portion of physical object 28 as in the case of a levitation spell, or a movement of a portion of physical object 28 as in the case of a levitation spell. Figure 3 as shown in the figure) etc.

[0050] Thus, the embodiments described herein enable both an AR experience via the display 48 of the mobile device 26 and perceived physical interactions with physical objects 28 disposed around the real-world environment of the amusement park 10. In certain embodiments, the physical interactions with the physical objects 28 disposed around the real-world environment of the amusement park 10 may include not only the movement of certain portions of the physical objects 28, but also more complex physical interactions with the amusement park itself. For example, in certain embodiments, if a particular user of the mobile device 26 (and / or the auxiliary device 30, in certain embodiments) performs a certain number of interactions with other physical objects 28 disposed around the real-world environment of the amusement park 10 during a particular visit to the amusement park 10, the system 32 may determine that the user may be able to physically interact with certain doors, which may, for example, provide entrances to hidden caves also disposed around the amusement park 10.

[0051] Figure 4 An example of a user accessing a hidden cave 100 via a door 102 of an amusement park 10 according to an embodiment of the present disclosure is illustrated, where the door 102 of the amusement park 10 has been unlocked based on the user physically interacting with a given number of physical objects 28 using a mobile device 26 (and / or an auxiliary device 30). In some embodiments, the ability to unlock the door 102 may be only a first requirement, where the second requirement is for the user to perform another interaction with the mobile device 26 (and / or an auxiliary device 30). For example, in some embodiments, the user may need to access the hidden cave 100 via the mobile device 26 (and / or an auxiliary device 30) by, for example, causing a physical object 28 as described herein. Figure 3 102 to perform a specific door-opening spell. Once the user performs the specific movement patterns 88, 90, 92, the physical interaction system 80 can cause the door 102 to open (e.g., as a physical effect 96), for example, by causing actuation of a locking mechanism 98 (e.g., a magnetic lock in certain embodiments). Additionally, in certain embodiments, the image generation system 34 can cause certain visual effects 94 to be displayed on the display 48 of the mobile device 26 in conjunction with the opening of the door 102.

[0052] Furthermore, in certain embodiments, the AR experience enabled by the embodiments described herein and physical interactions with the physical objects 28 of the amusement park 10 may enable a user to solve certain puzzles presented via certain physical objects 28 . Figure 5An example of a puzzle 104 that can be solved based on a user's interaction with a mobile device 26 (and / or an auxiliary device 30) according to an embodiment of the present disclosure is illustrated. In the illustrated embodiment, a plurality of physical objects 28 can be physically interacted with via user interaction with the mobile device 26 (and / or an auxiliary device 30, in some embodiments). In some embodiments, once the plurality of physical objects 28 have been physically moved into the correct positions via user interaction with the mobile device 26 (and / or an auxiliary device 30, in some embodiments), certain visual effects 94 can be caused to be displayed on the display 48 of the mobile device 26 by the image generation system 34. For example, in Figure 5 In the embodiment illustrated in , user interaction with the mobile device 26 (and / or auxiliary device 30 in some embodiments) can cause the physical object 28 to be placed within the corresponding compartment 106 of the base 108, as illustrated by arrow 110, thereby solving the puzzle 104 and, in some embodiments, causing the visual effect 94 to be displayed by the image generation system 34 on the display 48 of the mobile device 26.

[0053] Figure 6 A method 112 for operating the system 32 described herein, according to embodiments of the present disclosure, is illustrated. For example, in some embodiments, the method 112 includes generating, via the image generation system 34, a streaming media of the real-world environment of the amusement park 10 based, at least in part, on image data (e.g., real-world images 38) captured via one or more cameras 46 of the mobile device 26 (block 114). Furthermore, in some embodiments, the method 112 includes generating, via the image generation system 34, one or more augmentations (e.g., one or more AR images 40) superimposed on the streaming media of the real-world environment of the amusement park 10 based, at least in part, on data related to user interactions with the mobile device 26 (block 116). Furthermore, in some embodiments, the method 112 includes transmitting, via the image generation system 34, the streaming media of the real-world environment of the amusement park 10 along with the one or more augmentations to be displayed on the display 48 of the mobile device 46 (block 118). Furthermore, in some embodiments, the method 112 includes determining, via the physical interaction system 80, one or more physical effects 96 to be implemented for physical objects 28 in the real-world environment of the amusement park 10 based, at least in part, on data related to user interactions with the mobile device 26 (block 120). Additionally, in certain embodiments, the method 112 includes transmitting a signal to the physical object 28 via the physical interaction system 80 to implement the determined one or more physical effects 96 .

[0054] Although only certain features of the present embodiment have been illustrated and described herein, many modifications and variations will occur to those skilled in the art. Therefore, it is to be understood that the appended claims are intended to cover all such modifications and variations as fall within the true spirit of the present disclosure. Furthermore, it should be understood that certain elements of the disclosed embodiments may be combined or interchanged with one another.

Claims

1. A system for a tourist attraction, comprising: An image generation system configured to be communicatively coupled to a mobile device and configured to: generating a streaming media of a real-world environment based at least in part on image data captured via a camera of the mobile device; generating one or more augmentations superimposed on the streaming media of the real-world environment based at least in part on data related to user interactions with the mobile device; as well as transmitting the streaming media of the real-world environment along with the one or more augmentations to be displayed on a display of the mobile device; as well as a physical interaction system configured to be communicatively coupled to the mobile device and configured to: determining one or more physical effects to be implemented for physical objects in the real-world environment based at least in part on the data related to the user interaction with the mobile device; as well as transmitting a signal to the physical object to achieve the determined one or more physical effects; The data related to the user interaction with the mobile device includes a pattern of movement associated with the mobile device. 2 . The system of claim 1 , wherein the data related to the user interaction comprises a pattern of movement.

3. The system of claim 2 , wherein the image generation system includes a pattern recognition module configured to analyze the data related to the user interaction to determine whether the pattern of movement corresponds to a known pattern, and the image generation system is configured to generate the one or more enhancements based at least in part on whether the pattern of movement corresponds to the known pattern.

4. A system according to claim 2, wherein the physical interaction system includes a pattern recognition module, which is configured to analyze the data related to the user interaction to determine whether the pattern of movement corresponds to a known pattern, and the physical interaction system is configured to determine the one or more physical effects to be achieved based at least in part on whether the pattern of movement corresponds to the known pattern. 5 . The system of claim 2 , wherein the pattern of movement comprises a pattern of movement of the mobile device detected by a monitoring system that monitors the mobile device. The system of claim 2 , wherein the pattern of movement comprises input received via a user interface of the mobile device. The system of claim 1 , wherein the user interaction comprises movement of the mobile device.

8. The system of claim 7, wherein the image generation system is configured to generate the streaming media of the real-world environment based at least in part on an orientation of the mobile device, a position of the mobile device, a motion of the mobile device, or a combination thereof. 9 . The system of claim 8 , wherein the image generation system is configured to receive the orientation of the mobile device, the position of the mobile device, the motion of the mobile device, or a combination thereof from the mobile device.

10. The system of claim 8, comprising a monitoring system configured to monitor physical properties of the mobile device to determine the orientation of the mobile device, the position of the mobile device, the motion of the mobile device, or a combination thereof.

11. The system of claim 1, wherein the user interaction comprises input received via a user interface of the mobile device.

12. The system of claim 1 , wherein the image generation system is configured to generate the one or more augmentations superimposed on the streaming media of the real-world environment based at least in part on data related to user interactions with an auxiliary device used in conjunction with the mobile device.

13. The system of claim 1 , wherein the physical interaction system is configured to determine the one or more physical effects to be implemented for the physical objects in the real-world environment based at least in part on data related to user interactions with an auxiliary device used in conjunction with the mobile device.

14. The system of claim 1 , wherein the image generation system is configured to generate the one or more augmentations superimposed on the streaming media of the real-world environment based at least in part on data related to one or more physical objects disposed in the real-world environment.

15. A method for use in a tourist attraction, comprising: receiving a real-time video data stream from a mobile device via an image generation system; generating, via the image generation system, a visualization of a real-world environment based at least in part on the received real-time video data stream; overlaying, via the image generation system, an augmented reality (AR) image onto the visualization of the real-world environment based at least in part on data related to user interactions with the mobile device; determining, via a physical interaction system, one or more physical effects to be implemented for physical objects in the real-world environment based at least in part on the data related to the user interaction with the mobile device; transmitting, via the image generation system, the overlaid AR image along with the visualization of the real-world environment to the mobile device; as well as transmitting a signal to the physical object via the physical interaction system to achieve the determined one or more physical effects; The data related to the user interaction with the mobile device includes a pattern of movement associated with the mobile device. The method of claim 15 , wherein the data related to the user interaction comprises a pattern of movement. 17 . The method of claim 16 , comprising overlaying, via the image generation system, the AR image onto the visualization of the real-world environment based at least in part on whether the pattern of movement corresponds to a known pattern.

18. The method of claim 16, comprising determining, via the physical interaction system, the one or more physical effects to be implemented for the physical object in the real-world environment based at least in part on whether the pattern of movement corresponds to a known pattern.

19. The method of claim 16, wherein the pattern of movement comprises a pattern of movement of the mobile device detected by a monitoring system that monitors the mobile device.

20. The method of claim 16, wherein the pattern of movement comprises input received via a user interface of the mobile device.

21. The method of claim 15, comprising overlaying, via the image generation system, the AR image onto the visualization of the real-world environment based at least in part on an orientation of the mobile device, a position of the mobile device, a motion of the mobile device, or a combination thereof.

22. The method of claim 21, comprising detecting the orientation of the mobile device, the position of the mobile device, the motion of the mobile device, or a combination thereof using one or more sensors of the mobile device.

23. The method of claim 21, comprising detecting the orientation of the mobile device, the position of the mobile device, the motion of the mobile device, or a combination thereof using a monitoring system that monitors the mobile device.

Images