Artificial Intelligence (AI) Assisted Dynamic Vehicle Profile Head Tracking System and Method

Abstract

The ride system can include a ride vehicle supporting a passenger and an attention tracker that determines a current attention direction of the passenger. The ride system can also include a control system for maintaining an environment for the passenger. Maintaining the environment can include determining a set of content to incorporate into the environment based at least in part on the current attention direction.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 333,382, entitled "ARTIFICIAL INTELLIGENCE (AI)-ASSISTED AND DYNAMIC RIDE PROFILE HEAD TRACKING SYSTEMS AND METHODS," filed April 21, 2022, which is incorporated by reference in its entirety.

[0002] The present disclosure relates generally to amusement rides / attractions, and more particularly to systems and methods for tracking, predicting, and / or utilizing head and / or eye movements in amusement rides / attractions. [Background technology]

[0003] A variety of entertainment rides have been created to provide passengers with unique motion and visual experiences. For example, themed rides can be implemented that include single or multi-passenger vehicles that travel along fixed or variable paths. The ride vehicles themselves can include features (e.g., various buttons and knobs) that allow passengers to have varying levels of control over the ride vehicle and / or the surrounding environment. However, traditional controls typically relegated to the passengers of the ride vehicle are limited when the ride vehicle follows a predetermined fixed path.

[0004] Additionally, fixed or variable path ride vehicles and simulated ride vehicles may render or actuate digital and / or physical content, respectively, to enhance the passenger(s) experience. In some amusement park rides, the vehicle's motion and the rendering / actuation of such content may be constrained to pre-programmed profiles (e.g., animations) embedded in the vehicle's programmable logic controller (PLC) or the like. However, it is now recognized that these program profiles are essentially static and therefore are not updated or modified based on passenger interaction with the vehicle and / or actual physics models. As a result, passengers in the ride may perceive the ride as staged or unrealistic, which may limit the passenger's engagement and enjoyment.

[0005] It is also recognized that as a passenger experiences a ride / attraction, where the passenger focuses their attention (i.e., where the passenger looks) may change throughout the ride / attraction. Thus, it is now recognized that if a ride utilizes pre-programmed profiles to determine and generate a passenger's experience independent of the direction the passenger focuses their attention, this may limit the ability of the ride to immerse the passenger in an experience that feels true to the actual physical model.

[0006] The Background section above is intended to introduce the reader to various aspects of the art that may be related to various aspects of the present technology described and / or claimed below. This discussion is believed to be helpful in providing the reader with background to facilitate a better understanding of the various aspects of the present disclosure. As such, it should be understood that these statements are to be read in this light, and not as admissions of prior art. Summary of the Invention [Means for solving the problem]

[0007]

[0013] The following summarizes certain embodiments common in scope to the originally claimed subject matter. These embodiments are not intended to limit the scope of the claimed subject matter, but rather to outline possible forms of the subject matter. Indeed, the subject matter may include a variety of forms that may be similar to or different from the embodiments set forth below.

[0008] In some embodiments, a ride system may include a ride vehicle supporting a passenger and an attention tracker that determines a current attention direction of the passenger. The ride system may also include a control system for maintaining an environment for the passenger. Maintaining the environment may include determining a set of content to incorporate into the environment based at least in part on the current attention direction.

[0009] In an embodiment, a method can include determining a first attention direction of a first passenger at a first time during a first ride session of a ride system and training an artificial intelligence (AI) algorithm to predict a future attention direction of a second passenger based on the first attention direction of the first passenger. The method can also include determining a second attention direction of a second passenger at a second time during a second ride session of the ride system and estimating, via the AI ​​algorithm, a future attention direction of the second passenger based on the second attention direction of the second passenger. The method can also include determining whether to limit rendering of content of a content set generated for the second passenger based on the estimated future attention direction.

[0010] In some embodiments, the method may include receiving input data, such as an attention direction of a passenger of the ride vehicle, associated with one or more input devices of the ride vehicle. The method may include generating a virtual environment relevant to the passenger based at least in part on the input data. Content of the virtual environment may be determined based on the attention direction. The method may further include rendering a first portion of the virtual environment and displaying a second portion of the rendered first portion of the virtual environment based on a viewpoint of the passenger relative to an axis of the ride vehicle.

[0011] These and other features, aspects, and advantages of the present disclosure will be better understood from the following detailed description when read in conjunction with the accompanying drawings, in which like parts are designated with like numerals throughout. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a perspective view of a ride system including a ride vehicle according to an embodiment of the present disclosure. [Diagram 2] 2 is a hybrid schematic block diagram of the ride system of FIG. 1 with the ride vehicle coupled to a control system in accordance with an embodiment of the present disclosure. [Diagram 3] 3 is a schematic diagram illustrating the flow of information within the control system of FIG. 2 associated with two ride vehicles in accordance with an embodiment of the present disclosure. [Figure 4] 2 is a top view of the ride vehicle of FIG. 1 including one or more attention trackers to determine a passenger's attention direction in accordance with an embodiment of the present disclosure. [Diagram 5] FIG. 2 is a perspective view of the ride vehicle of FIG. 1 surrounded by physical and / or virtual content in accordance with an embodiment of the present disclosure; [Figure 6] 1 is a flowchart of an example process for generating and displaying content based on a passenger's attention direction and / or field of view, according to an embodiment of the present disclosure. [Figure 7]1 is a flowchart of an example process for utilizing artificial intelligence algorithms to predict a passenger's attention direction and / or field of view and adjust content rendering based thereon, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] One or more specific embodiments of the present disclosure will be described below. In order to concisely describe these embodiments, not all features of an actual implementation may be described herein. It should be understood that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developer's particular goals, such as compliance with system-related and business-related constraints that may vary from implementation to implementation. Moreover, such development efforts may be complex and time-consuming, but should be understood to be a routine undertaking of design, fabrication, and manufacture for those skilled in the art having the benefit of this disclosure. Furthermore, to the extent that specific terms, such as parallel and perpendicular, are used in the present invention, it should be understood that these terms allow for deviations from strict mathematical definitions, such as deviations related to manufacturing imperfections and related tolerances.

[0014] When introducing elements of various embodiments of the disclosure, the articles "a," "an," and "the" are intended to mean the presence of one or more of the element. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements other than the listed elements. It should also be understood that references to "one embodiment" or "an embodiment" of the disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also contain the recited features.

[0015] It should be appreciated that a variety of entertainment vehicles have been created to provide passengers with unique motion and visual experiences. For example, entertainment vehicles can be implemented that include single or multi-passenger vehicles that travel along fixed or variable paths. The ride vehicles themselves can include pre-programmed profiles and / or features (e.g., various buttons and knobs) that allow passengers to have different levels of control over the ride vehicle and / or the surrounding environment. However, even if some degree of control is available to the passengers, the pre-programmed profiles may appear substantially static. As a result, passengers in the ride may experience the ride as staged or unrealistic, thereby limiting the passengers' immersion and enjoyment. Thus, to enhance passenger immersion and enjoyment, dynamic ride profiles based on a combination of sensed parameters, physics models, game feedback, and passenger interaction can be utilized to render content and adjust the motion of the ride vehicle. The dynamic ride profiles thus enable the ride to provide realistic simulation movements and digitally rendered content that enhances passenger immersion and enjoyment.

[0016] The present embodiments generally relate to amusement park style rides / attractions that utilize head and / or eye movement tracking and / or prediction to increase content rendering efficiency and / or modify / change the dynamic ride profile of the ride / attraction. Generally, as a passenger experiences a ride / attraction, where the passenger focuses their attention (i.e., where the passenger looks) may change throughout the ride / attraction. Thus, in some embodiments, passenger interactions and influences on the dynamic ride profile go beyond typical passenger controls such as buttons, levers and / or interactive handheld devices, but may also include the passenger's attention direction and / or field of view(s). For example, the motion of the ride vehicle and / or content (e.g., digital content and / or physical content) depicted in or around the ride vehicle may be generated or actuated based on where the passenger's attention is focused. In some embodiments, the ride vehicle, its surroundings, and / or passenger-carried devices (e.g., headsets, glasses, handheld devices, etc.) may include one or more head-tracking and / or eye-tracking sensors (e.g., attention trackers) to estimate a passenger's attention direction and / or field of view. The attention trackers may be coupled to a dynamic control system to generate a dynamic ride profile.

[0017] In some embodiments, content of the dynamic ride profile may be adjusted to correct for warps, obscurations, and / or other point-of-view distortions based on the passenger's direction of attention so that the content appears realistic / immersive to the passenger. Additionally or alternatively, the passenger's direction of attention may be used to modify what content (e.g., subject matter or its placement) is generated / displayed to the passenger. Furthermore, the motion of the ride vehicle may be adjusted based on (e.g., directly as a result of) the passenger's direction of attention or indirectly (e.g., based on modified content based on the passenger's direction of attention).

[0018] Additionally or alternatively, in some embodiments, the passenger's attention direction may be used to adjust the rendering of digital content and / or the operation of physical content so that processing time and / or operation energy is not wasted on content that is not likely to be observed (e.g., outside the passenger's field of view). For example, if the passenger's attention direction is in a first direction, processing power / bandwidth and / or energy may be saved by ignoring (i.e., not rendering) during rendering content associated with a ride profile (dynamic, static, or other ride profile) located in a second direction that is not likely to be seen within the passenger's field of view. Furthermore, in some embodiments, content adjustments may be independent of a dynamic ride profile (which may also be determined based on the passenger's attention direction) or may be determined in conjunction with such a dynamic ride profile. For example, adjustments to content rendering may be performed regardless of what content the ride profile includes and whether the subject matter or location of the content (e.g., in the dynamic ride profile, etc.) was determined by the passenger's attention direction.

[0019] Additionally, in some embodiments, artificial intelligence (AI) algorithms, such as machine learning algorithms, deep learning algorithms, artificial neural networks (ANNs), etc., may be used to predict in real-time where a passenger's attention direction is likely to be based on historical data and / or current head and / or eye position data. For example, head tracking and / or eye tracking may be performed over the entire training period and / or service life of the ride system to train an AI algorithm to predict where a passenger is likely to be looking at any point in time (e.g., the current time or real-time time) or as an average during operation of the ride system. Additionally, the predicted attention direction may be used to enhance and / or speed up processing that relies on the passenger's attention direction, such as generating dynamic ride profiles and / or adjusting content rendering. For example, the ride system may pre-process future content based on the predicted attention direction, thereby reducing or eliminating latency associated with processing. Additionally or alternatively, processing associated with content that is likely not visible to the passenger in a particular area relative to the passenger's location may be reduced based on the passenger's predicted attention direction and / or field of view.

[0020] With this in mind, FIG. 1 is a perspective view of an embodiment of a ride system 10. The ride system 10 can include one or more ride vehicles 12 that hold one or more passengers 14. In some embodiments, multiple ride vehicles 12 can be coupled together (e.g., by link 16). In some scenarios, the ride vehicles 12 can move along a ride path 18 during operation of the ride system 10. The ride path 18 can be any surface along which the ride vehicles 12 move. For example, the ride path 18 can be defined by a track, a gimbal system, an enclosed or predefined area, etc. The ride path 18 can or can not determine the path along which the ride vehicles 12 travel. In some embodiments, the ride path 18 can control the movement (e.g., direction, speed, and / or orientation) of the ride vehicles 12 as they progress along the ride path 18, similar to a train on a railroad track. In another embodiment, another system can control the path that the ride vehicles 12 take during operation of the ride system 10. For example, the ride path 18 can be an open surface that allows the passengers 14 to control certain aspects of the movement of the ride vehicles 12 through an interface system of the ride vehicles 12. Additionally, in some embodiments, the ride vehicles 12 can articulate on one or more axes while remaining stationary relative to a geographic location. Thus, the ride path 18 is virtual, and the ride vehicles 12 can articulate to simulate movement within or on the ride path 18. It should be understood that the ride system 10 can include any suitable number of ride vehicles 12, and each ride vehicle 12 can accommodate any suitable number of passengers 14.

[0021] It should be understood that the embodiment of ride system 10 depicted in FIG. 1 is a simplified representation intended to provide background and facilitate discussion of the technology disclosed herein. Other embodiments of ride system 10, including ride vehicles 12 and ride path 18, may include similar and / or different elements or configurations. For example, while the illustrated embodiment shows ride vehicles 12 moving along a ride path 18 located below ride vehicles 12, other embodiments of ride system 10 may include ride vehicles 12 suspended from a ride path 18 located above ride vehicles 12.

[0022] 2 is a hybrid schematic block diagram representation of a ride system 10 including ride vehicles 12 coupled to a control system 20. Each ride vehicle 12 may include a number of output devices 22, a number of input devices 24, one or more sensors 26, and / or one or more controllers. For example, the controllers may include, but are not limited to, one or more motion controllers (e.g., a speed controller 28 and a rotation controller 30), and / or a main controller 32, such as a programmable logic controller (PLC). It is understood that each controller may include separate or shared processor circuitry 34 and memory 36. Additionally, the sensors 26 may include position sensors (e.g., proximity sensors, radio frequency identification (RFID) sensors, cameras, light detection and ranging (LIDAR) sensors), speedometers, accelerometers, gyroscopes, revolutions per minute (RPM) sensors, voltage / current sensors, or other suitable sensors capable of measuring parameters of the vehicle 12, the passenger 14 (e.g., head or eye movement), and / or the ride system 10. Additionally, sensors may be located within the ride vehicle 12 or at locations external to the ride vehicle 12 , such as on or along the ride path 18 .

[0023] In some embodiments, the output devices 22 may include any suitable number of displays (e.g., displays mounted inside the vehicle, head mounted displays), speakers, haptic feedback devices (e.g., rumble / vibration feedback devices, acoustic or ultrasonic haptic devices), physical effects devices (e.g., devices generating hot or cold air jets, devices generating mist jets). Additionally, in some embodiments, the output devices 22 may fully or partially surround the passenger(s) to provide a highly immersive experience. Additionally or alternatively, the output devices 22 may be located external to the ride vehicle 12. It should be understood that each ride vehicle 12 may include other suitable output devices 22 or other combinations of output devices 22 in accordance with the present disclosure.

[0024] Further, the input devices 24 may include buttons (e.g., ignition button), steering devices (e.g., steering wheel, joystick), control pedals (e.g., brake pedal, accelerator pedal, clutch pedal, etc.), knobs, levers (e.g., gear shift, brake lever, etc.), or other physical media. Additionally or alternatively, the input devices 24 may include head and / or eye tracking systems that monitor the passenger's head and / or eye position to determine attention direction and / or field of view. It should be understood that each ride vehicle 12 may include other input devices 24 or other combinations of input devices 24 in accordance with the present disclosure. In some embodiments, each passenger 14 may have a respective set of input devices 24, while in other embodiments, each passenger 14 may have a complementary set of input devices 24 (e.g., used collaboratively) or shared input devices 24.

[0025] The ride system 10 may also include a control system 20 that controls the movement of the ride vehicle 12 in accordance with a dynamic ride profile as described in further detail below. In particular, the illustrated control system 20 includes a dynamic ride profile server 38, a game server 40, and may be communicatively coupled to the controller 32 of the ride vehicle 12 (e.g., via a network 42). It should be appreciated that the network 42 may provide communications between the ride vehicle 12 and the control system 20 utilizing any suitable wired or wireless connection.

[0026] As used herein, and described in more detail below, game server 40 generally refers to a computing device or group of computing devices (e.g., physical computing devices or virtual computing nodes) responsible for managing the video "game" aspects of vehicle system 10. Game server 40 may thus be programmed to generate a virtual environment (e.g., a virtual 3D space) within which virtual vehicles are designed to travel. Furthermore, as used herein, a "virtual vehicle" refers to a video game entity or element of the virtual environment having certain attributes (e.g., speed, position, health / damage, fuel, appearance) that are maintained by game server 40. For example, a virtual vehicle may be associated with a physical vehicle vehicle 12. In some embodiments, there may also be additional virtual vehicles (e.g., non-playable characters / vehicles) within the virtual environment.

[0027] In some embodiments, passengers 14 may be presented with an augmented reality environment or a fully virtual environment 44 including digital and / or physical content inside or outside of ride vehicle 12. For example, in some embodiments, ride system 10 may be a racing simulator, and thus game server 40 may generate and maintain a virtual environment representing the nature of the race track traversed by virtual vehicles, the relative speeds and positions of the virtual vehicles, interactions between virtual vehicles, and attributes associated with the virtual vehicles (e.g., performance upgrades, health, bonuses, scores, etc.). Game content may be generated and / or modified based on a pre-designed program (e.g., a general "game") and input from input devices 24 and / or sensors 26. Additionally, output devices 22 may output game content (e.g., video content, audio content) that is delivered to ride vehicle 12 to effect at least a portion of environment 44 presented to passengers 14. For example, in one embodiment, video content presented to a particular passenger 14 by a display device of the ride vehicle 12 includes content corresponding to the perspective vie of the particular passenger 14 that is generated within a virtual environment hosted by the game server 40. It is to be understood that the environment 44 can be a virtual environment (e.g., displayed entirely via digital media), a physical environment (e.g., a physical and / or mechanical surrounding environment), or a combination thereof (e.g., a virtually augmented physical environment).

[0028] As used herein, and described in more detail below, the dynamic ride profile server 38 generally refers to one or a group of computing devices (e.g., physical computing devices or virtual computing nodes) responsible for making decisions about how the physical ride vehicles 12 should move based on a number of different input data and one or more physics models. As described above, the input data may include information received from the game server 40 indicating or describing what is happening to each corresponding virtual vehicle in the virtual environment, such as how the virtual vehicles respond to textures or interactions in the game. For example, the decision about how the ride vehicles 12 should move may take into account the slope of the race track, environmental hazards (e.g., rain, puddles, ice), and interactions between the virtual vehicles. Additionally, in some embodiments, the dynamic ride profile server 38 receives input data from the input devices 24 and / or various sensors 26 of the ride system 10. As described below, the dynamic ride profile server 38 provides this received data as input to one or more physics models that describe how the physical ride vehicles 12 should move to respond to what is happening in the environment 44 being presented to the passengers 14. In this manner, the dynamic vehicle profile server 38 generates a dynamic vehicle profile that instructs each ride vehicle 12 how to behave consistent with what is being presented to the passengers 14 by the game server 40 .

[0029] In some embodiments, the dynamic vehicle profile server 38 and the game server 40 may be hosted by different physical computing devices or may exist as virtual server instances hosted by a common physical computing device. In general, it should be understood that the one or more computing devices hosting the dynamic vehicle profile server 38 and the game server 40 may include any suitable memory 36 (e.g., a non-transitory computer-readable medium) capable of storing instructions and data, and any suitable processing circuitry 34 capable of executing stored instructions to provide the functionality described herein.

[0030] It can be appreciated that in some embodiments, the ride path 18 can be loosely defined by a set of physical and virtual boundaries that allow greater freedom of movement of the ride vehicle 12 than a conventional track. Thus, in addition to causing effects within the environment 44 presented to the passengers 14, the input devices 24 can also trigger real-world effects, such as altering the motion (e.g., position, velocity, or direction) of the vehicle 12 within a predetermined set of limits. For example, the dynamic ride profile server 38 can provide control signals to one or more motion controllers (e.g., speed controller 28 and / or rotation controller 30) to modify the vehicle's yaw 46, pitch angle 48, ride path position (e.g., displacement 50 along the ride path 18, lateral displacement 52 relative to the boundaries of the ride path 18), velocity (e.g., rate of change of displacement 50 and / or lateral displacement 52), and / or rotation rate (e.g., rate of change of yaw 46 and / or lateral displacement 52), or any other suitable parameter of the ride vehicle 12 according to a physics-based dynamic ride profile that takes into account passenger inputs. That is, embodiments of the dynamic ride profile server 38 may provide control signals to modify one or more aspects of the position and / or orientation of the ride vehicle 12 along the ride path 18 along one or more axes (e.g., along six degrees of freedom). This may result in the ride vehicle 12 moving in a manner generally consistent with that presented in the environment 44 to provide an immersive experience for the passengers 14.

[0031] 3 is a schematic diagram illustrating the flow of information within the control system 20 of FIG. 2 associated with two ride vehicles 12 (e.g., ride vehicle 12A and ride vehicle 12B). In general, the dynamic vehicle profile server 38 and / or the game server 40 may be provided with input data 54 (e.g., from input devices 24) and sensor data 56 (e.g., from sensors 26). Additionally, game data 58 generated by the game server 40 may be provided to the dynamic vehicle profile server 38 to generate different dynamic vehicle profiles 60A and 60B (cumulatively 60) for the different ride vehicles 12A and 12B, respectively. As discussed above, the dynamic vehicle profile server 38 may apply one or more physics models 61 (e.g., stored in memory 36) in determining the dynamic vehicle profiles 60. Such physics models 61 may define how a virtual vehicle (corresponding to the ride vehicle 12) moves within the environment 44, such as moving along a smooth or layered path, moving along a bumpy or turbulent path, sliding or drifting, or transitioning between different mediums (e.g., moving between air and water). The physics models 61 may also include models that describe how two or more virtual vehicles interact and affect each other (e.g., via drafting, collisions, missile attacks) within the environment 44 presented to the passengers 14.

[0032] As described herein, the control system 20 generates content that depicts the environment 44 and determines suitable ride vehicle 12 motions that create the sensation of the ride experience being as if the vehicle is actually moving through the environment 44. For example, the output devices 22 provide audio / visual information (e.g., video content, sound effects, music, virtual reality (VR) content, augmented reality (AR) content) related to the video game aspects of the ride system 10, and the ride vehicle 12 moves accordingly. The control system may also receive input from the input devices 24 and / or sensors 26 and responsively update the content presented to the passengers 14. In other words, the content and motions may be based on input data and / or sensor data that are indicative of the nature of the ride vehicle 12 and / or the actions or behaviors of the passengers.

[0033] In particular, the attention direction 62 of a passenger 14, as shown in Figure 4, may be utilized to alter or enhance a dynamic ride profile 60 of the ride vehicle 12 or a particular passenger's experience within the ride vehicle 12. For example, one or more attention trackers 64 may be located within the ride vehicle 12 (e.g., mounted within or on the ride vehicle, on a headrest of the ride vehicle, etc.), located external to the ride vehicle 12 (e.g., along or to the side of the ride path 18), and / or worn by the passenger 14 (e.g., via a headset 66). The attention tracker 64 may utilize any suitable type of head and / or eye movement tracking (e.g., via a camera, gyroscope, accelerometer, etc.) to identify the attention direction 62 of the passenger(s) 14 and / or the passenger's field of view 68. Additionally, in some embodiments, a headset 66 (e.g., an AR headset, a VR headset, 3D glasses, etc.) may be utilized as both an input device 24 (e.g., attention tracker 64) and an output device 22 (e.g., providing at least a portion of the content of environment 44). Additionally, some attention trackers 64 may include multi-directional trackers 64-1 and / or actuating trackers 64-2, for example, for use in a mobile and / or multiple ride vehicle 12. It should be appreciated that attention direction 62 and / or field of view 68 may be relative to any suitable axis, such as a direction of movement 70 (e.g., virtual or physical movement) along the ride path 18, a longitudinal, latitudinal or vertical axis of the ride vehicle 12, as set by the passenger 14 or pre-set by the ride system 10.

[0034] In some embodiments, the attention direction 62 and / or field of view 68 of the passenger 14 may be tracked (e.g., via one or more attention trackers 64) throughout a ride session (e.g., the period during which the passenger 14 is engaged with the ride system 10). Additionally, tracking may be performed continuously throughout the ride session, at predetermined locations along the ride path 18, at predetermined locations or events within the virtual surface of the environment 44, and / or at periodic points during the ride session. Additionally or alternatively, tracking may be performed based on the position and / or orientation of the ride vehicle 12. For example, the attention direction 62 and / or field of view 68 may be derived based on a nominal direction of attention 70 of the ride vehicle 12 and / or an orientation of the ride vehicle 12 relative to a direction of travel 70. Additionally, the nominal direction of attention 62 and / or field of view 68 may be based on the number and / or placement of the passengers 14 within the ride vehicle 12. For example, a ride vehicle 12 with two passengers 14 positioned on the left side of the ride vehicle 12 may have a direction of attention 62 and / or field of view 68 that is biased to the left more than a ride vehicle 12 with passengers 14 positioned side-by-side.

[0035] As shown in FIG. 5 , potential content (e.g., Content A, Content B, Content C, and Content D) associated with the environment 44 can surround the passengers 14 (e.g., passengers 14A and 14B). It should be understood that the content of the environment 44 can be fully physical, fully virtual, or a virtually augmented physical environment 44. As described herein, the attention direction 62 and / or field of view 68 of the passengers 14 can be utilized in determining what content should be generated (e.g., what subject matter is generated by the game server 40 and / or the location of such subject matter within the environment 44), where and how content should be rendered on the output device 22 (e.g., based on the passengers' 14's viewpoint within the environment 44), and / or what content should be rendered (e.g., to reduce rendering by portions of the environment 44 that do not need to be seen). Additionally, the attention direction 62 and / or field of view 68 as used herein can be considered individually or as a composite direction / field of view of multiple passengers 14, such as an intersection, addition, average, or other combination of multiple attention directions 62 and / or field of view 68. For example, the field of view 68 used in determining what content should be generated can be a composite field of view based on the field of view 68A of the first passenger 14A and the field of view 68B of the second passenger 14B. Additionally or alternatively, the attention direction 62 and / or field of view 68 can also include a hypothetical or otherwise assumed passenger orientation, such as when the ride vehicle 12 is empty and / or for testing purposes. In some embodiments, the virtual passenger can have a variable or set attention direction 62 and / or field of view 68 that can be based on or match the direction of travel 70 of the ride vehicle 12.

[0036] What content is generated (e.g., by the game server 40) can depend on the attention direction 62 and / or field of view 68. For example, if the passenger's attention direction 62 remains directed toward the sky, an airplane or bird can be generated to provide excitement or realism in the sky that would not have been generated if the passenger had momentarily turned their gaze toward the sky. As a further example, in a thriller or horror amusement ride or the like, a skeleton or ghost (e.g., content A in FIG. 5) can be generated (or activated in the case of the physical / augmented environment 44) at the edge of the passenger's field of view (e.g., at the edge of the field of view 68A of passenger 14A in FIG. 5) to enhance the thrill by providing stimulation from a direction that the passenger 14 does not expect. Thus, a passenger 14 can experience the same ride system 10 multiple times and have multiple different experiences based on the attention direction 62 and / or field of view 68. In other words, depending on the attention direction 62 and / or field of view 68, different content can be incorporated into the passenger experience (e.g., incorporated into a game) that would not have been available or generated with a different attention direction 62 and / or field of view 68.

[0037] Additionally or alternatively, the passenger's attention direction 62 and / or field of view 68 may determine where / how digital content is rendered (e.g., via AR, VR, or a display screen) relative to the passenger's viewpoint. For example, in FIG. 5, content B is generally left-center relative to the viewpoint of passenger 14A and generally centered relative to the viewpoint of passenger 14B. Furthermore, content C may appear farther away than content A to passenger 14A. Additionally or alternatively, such content placement may be used to correct for distortion (e.g., lens distortion from a headset or display), ambiguity (e.g., portions of the ride vehicle 12 or other passengers 14 obscuring the content), and / or other viewpoint distortions so that the content appears realistic to the passenger 14.

[0038] 6 is a flow chart of an example process 71 for generating and displaying content based on the attention direction 62 and / or field of view 68 of the passenger 14. In some embodiments, the sensor data 56 (including the attention direction 62 and / or field of view 68) and the input data 54 may be received (e.g., via the game server 40 and / or the dynamic vehicle profile server 38) (process block 72). Game data may be generated based on the attention direction 62 and / or field of view 68 (process block 74). For example, the attention direction 62 and / or field of view 68 may be utilized to determine what will occur in the game and therefore what the environment 44 will include. A dynamic vehicle profile 60 may be generated based on the game data (process block 76) and content may be rendered (process block 78). During rendering, the content may be corrected for the passenger's viewpoint based on the attention direction 62 and / or field of view 68 (process block 80). Although rendering content based on the dynamic vehicle profile 60 is described herein, it should be understood that the dynamic vehicle profile server 38, controller 32, or independent processor 34 may also be used to render the graphical content and / or send control signals to the physical content. Thus, in some embodiments, the rendered content may be sent to the ride vehicle 12 simultaneously with the dynamic vehicle profile 60 (e.g., to control the movement of the ride vehicle). Finally, the rendered content may be displayed via one or more output devices 22 (process block 82).

[0039] Referring again to FIG. 5 , the attention direction 62 and / or field of view 68 may also be used to determine what content should be rendered that is related to the generated game data and environment 44. In the example of FIG. 5 , content D is not visible to either passenger 14A / 14B because it is located relatively far from the attention direction 62A / 62B and field of view 68A / 68B of the passengers 14A / 14B, respectively. Thus, content D may not be rendered. In other words, in response to determining that the passengers 14 will not see the particular content, the rendering of the particular content may be limited (e.g., not rendered or only partially rendered). Partial rendering may include rendering at a lower resolution and / or with more detailed aspects removed from the environment 44 (e.g., removing clouds from the sky, reducing the number of trees in a forest, etc.).

[0040] However, in some scenarios, it may be difficult to render previously unseen content in response to a change in passenger attention direction 62 and / or field of view 68. For example, if passenger 14B suddenly turns around, there may or may not be time to fully render content D before passenger 14 notices artifacts or missing content. Thus, artificial intelligence (AI) algorithms may be utilized to predict passenger 14's attention direction 62 and / or field of view 68, such that content may be rendered before the passenger sees it, while allowing content to be unrendered or only partially rendered while not being viewed (e.g., to conserve processing bandwidth and / or power). For example, if the AI ​​algorithm determines that passenger 14 has a tendency to look back at a particular point in the ride, content D may be rendered and ready during that point in the ride.

[0041] In some embodiments, the AI ​​algorithm can be a machine learning algorithm, a deep learning algorithm, an artificial neural network (ANN), or other suitable type of AI algorithm. Furthermore, the AI ​​algorithm can be part of the control system 20 or can be implemented independently. In some scenarios, training data for the AI ​​algorithm can include attention tracking data from one or more attention trackers 64 acquired (e.g., periodically or continuously) throughout the ride session. Furthermore, in some scenarios, training data can be acquired during a learning period, rendering all content or a preselected portion thereof while the AI ​​algorithm learns head and / or eye movement frequency and / or behavior characteristics. In some scenarios, the preselected portion of the content can be based on the ride vehicle's direction of travel 70 or the virtual passenger's attention direction 62 and / or field of view 68, which can be based on a pre-set algorithm. Once trained, the AI ​​algorithms can output a predicted attention direction 62 and / or field of view 68 for each passenger 14 based on their current attention direction 62 and / or field of view 68, which can be used (e.g., by the AI ​​algorithms, the dynamic ride profile server 38, the game server 40, the controller 32, and / or other processors 34) to determine what content should be rendered. Additionally or alternatively, the AI ​​algorithms can also predict or conceptually take into account composite attention directions 62 and / or fields of view 68 associated with or based on multiple passengers.

[0042] In some embodiments, the AI ​​algorithm can predict the passenger's attention direction 62 and / or field of view 68 for a preset future period (e.g., 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds, and until the end of the ride session, etc.). In other words, the AI ​​algorithm can predict that the passenger's attention direction 62 and / or field of view 68 will remain within a certain range for the near future (e.g., the next 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds, and until the end of the ride session, etc.) and adjust the rendering of the content based on the predicted range. For example, if the passenger's 14A / 14B attention direction 62A / 62B and / or field of view 68A / 68B are predicted to vary between content A, content B, and content C over a given future period, but not including content D, content D can be rendered only partially or not at all at present and / or during such given period. It should be appreciated that this immediate future period can be a sliding window ahead of the present time and / or the time at which the current attention direction 62 and / or field of view 68 is measured / calculated.

[0043] Additionally or alternatively, AI algorithms may be used to adjust the generation of game data 58 when some areas of the content are not expected to be viewed (e.g., based on predicted attention direction 62 and / or field of view 68). For example, more game data content may be generated in frequently viewed content areas, and game data content may be relatively sparse in less viewed content areas. Such reallocation of game data may allow for more interaction or entertainment for passengers 14 without increasing processing bandwidth. Furthermore, the predicted attention direction 62 and / or field of view 68 may be a proxy for the current attention direction 62 and / or field of view 68 (e.g., determined based on one or more attention trackers 64) in any of the embodiments described herein.

[0044] Additionally or alternatively, the learning period may include activation / deactivation of content rendering controlled by the AI ​​algorithm, where the AI ​​algorithm may learn to directly control what content is rendered and what content is left unrendered or partially rendered (e.g., rendered at a lower resolution or with less content), regardless of whether it outputs the predicted attention direction 62 and / or field of view 68. In other words, the predicted attention direction 62 and / or field of view 68 may be used conceptually (e.g., to generate changes to the dynamic ride profile 38) without direct calculation. Furthermore, the AI ​​algorithm may also utilize input data 54 and / or sensor data 56 from the input device 24 along with attention tracking data from one or more attention trackers 64 to update the dynamic ride profile 38 and / or generate the predicted attention direction 62 and / or field of view 68. It should be appreciated that in some embodiments, the AI ​​algorithm may be constantly or periodically updated based on the attention tracking data as the ride system 10 is utilized by an increasing number of passengers 14.

[0045] FIG. 7 is a flow chart of an example process 84 for utilizing an AI algorithm to predict a passenger's 14 attention direction 62 and / or field of view 68 and adjust content rendering based thereon. The AI ​​algorithm may receive training data including attention tracking data of the passenger 14 experiencing the ride system 10 (process block 86). The AI ​​algorithm may then be trained based on the training data (process block 88). The trained AI algorithm may receive current attention tracking data associated with the passenger 14 (e.g., input data 54 from the attention tracker 64) (process block 90). It should be appreciated that the current attention tracking data may include the passenger's 14 current attention direction 62 and / or current field of view 68, and / or a cumulative history of the passenger's 14 attention direction 62 and / or field of view 68 during the current ride session. Additionally, the AI ​​algorithm may predict a future attention direction 62 or field of view 68 based on the current attention tracking data (process block 92). An AI algorithm or other controller / processor (e.g., dynamic ride profile server 38, game server 40, controller 32, and / or other processor 34) may adjust content rendering for passenger 14 based on the determined future attention direction or field of view (process block 94). It should be appreciated that the AI ​​algorithm may adjust content rendering (e.g., determine what content should be rendered, not rendered, and / or partially rendered) directly based on the current attention tracking data, regardless of whether the AI ​​algorithm calculates a value indicative of predicted attention direction 62 and / or field of view 68. Also, in some embodiments, the AI ​​algorithm may be retrained or updated based on the current attention tracking data (process block 96).

[0046] Although only certain features have been illustrated and described herein, many modifications and changes will occur to those skilled in the art. It is therefore to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the present disclosure. Furthermore, although the above-described flow charts are shown in a given order, in some embodiments, process blocks may be rearranged, modified, eliminated, and / or performed simultaneously. Additionally, the referenced flow charts are shown as exemplary tools, and additional decision and processing blocks may be added depending on the implementation.

[0047] The technology presented and claimed herein refers to and applies to tangible objects and specific examples of a practical nature that positively improve the art and are therefore not abstract, intangible, or purely theoretical. Moreover, if any claim appended at the end of this specification contains one or more elements designated as "means for [performing] ... [function]" or "step for [performing] ... [function]," such elements are to be construed pursuant to 35 U.S.C. 112(f). On the other hand, for any claim containing an element designated in any other manner, such elements are not to be construed pursuant to 35 U.S.C. 112(f). [Explanation of symbols]

[0048] 12 Vehicles 14 passengers 18 Vehicle Routes 62 Caution direction 64 Attention Tracker 64-1 Multi-directional tracker 64-2 Operational Tracker 66 Headset 68 field of view 70 Movement direction

Claims

1. It is a vehicle system, A vehicle configured to support passengers, An attention tracker configured to determine the current direction of the passenger's attention, A control system configured to maintain the passenger environment, Maintaining the environment includes determining changes to the content set of the environment based at least partially on the current direction of attention, The changes to the content set include adding or removing objects or effects from the content set. A vehicle system characterized by the following.

2. Maintaining the aforementioned environment is at least partially based on the current direction of attention, Render the first content from the content set to be incorporated into the environment, Partially render or not render the second content of the content set to be incorporated into the environment. The vehicle system according to claim 1, comprising the determination thereof.

3. The first content to be incorporated into the environment is located within the passenger's field of view corresponding to the current direction of attention, and the second content to be incorporated into the environment is located outside the passenger's field of view. The vehicle system according to claim 2.

4. The system comprises an artificial intelligence (AI) algorithm configured to determine a predicted direction of attention based at least partially on the current direction of attention, and maintaining the environment is based at least partially on the predicted direction of attention. The first content is rendered, The aforementioned second content is partially rendered or not rendered. The vehicle system according to claim 2, including the determination thereof.

5. The aforementioned predicted directions of attention include a range of directions related to some point in the near future, The vehicle system according to claim 4.

6. The control system includes a dynamic vehicle profile server configured to determine a dynamic vehicle profile based at least partially on the passenger's current attention direction, wherein the dynamic vehicle profile includes first information relating to controlling the movement of the vehicle, second information relating to at least a portion of the environment to be displayed to the passenger, or the first information and the second information. The vehicle system according to claim 1.

7. The environment includes a physical medium arranged along the vehicle's route. The vehicle system according to claim 1.

8. The current direction of attention represents at least a portion of the passenger's field of vision, or at least a portion of the combined field of vision of the passenger and the second passenger in the vehicle. The vehicle system according to claim 1.

9. A headset comprising one or more display screens, wherein the environment includes a virtual environment to be displayed via the one or more display screens, The vehicle system according to claim 1.

10. To determine the first attention direction of the first passenger at a first point in time during the first ride session of the vehicle system, Training an artificial intelligence (AI) algorithm to predict the future attention direction of a second passenger based at least partially on the first attention direction of the first passenger, Determining the second attention direction of the second passenger at a second point in time during a second ride session of the vehicle system, The AI ​​algorithm is used to estimate the future attention direction of the second passenger based at least partially on the second attention direction of the second passenger, Based at least in part on the estimated future attention directions, determine whether the rendering of the first content from the content set generated for the second passenger should be restricted, A method characterized by including the following.

11. This includes generating the content set based at least partially on the second attention direction of the second passenger, The method according to claim 10.

12. The content set includes a virtual environment to be displayed at least partially to the second passenger, The method according to claim 11.

13. This includes retraining the AI ​​algorithm based at least partially on the second attention direction of the second passenger, The method according to claim 10.

14. The estimated future attention directions include a range of attention directions related to a future time window beyond the second point in time. The method according to claim 10.

15. If the first content should be incorporated outside the range of the estimated future attention direction within the environment of the vehicle system, the rendering of the first content is restricted. The method according to claim 14.

16. Receiving input data related to one or more input devices of a vehicle, including the direction of attention of a passenger in the vehicle, A virtual environment related to the passenger is generated based at least partially on the input data, wherein the content of the virtual environment is determined at least partially on the attention direction. Rendering the first part of the virtual environment, Displaying the second portion of the rendered first portion of the virtual environment based at least partially on the passenger's viewpoint relative to the axis of the vehicle, A method characterized by including the following.

17. This includes estimating a predicted attention direction based at least partially on the aforementioned attention direction via an artificial intelligence algorithm, wherein the predicted attention direction includes a range of attention directions related to a future time window. The method according to claim 16.

18. The process includes determining the first portion of the virtual environment to be rendered, at least partially based on the predicted attention direction, wherein the first portion of the virtual environment includes the first content of the content of the virtual environment that lies within the attention direction range of the predicted attention direction. The method according to claim 17.

19. Displaying the second portion of the rendered first portion of the virtual environment includes determining viewpoint correction based at least in part on the passenger's viewpoint, the passenger's viewpoint being determined at least in part on the direction of attention. The method according to claim 16.

20. The aforementioned passengers include people or virtual passengers, The method according to claim 16.

21. A vehicle system, A vehicle configured to support passengers throughout the duration of their journey, One or more sensors configured to generate sensor data indicating the passenger's current direction of attention, A control system, Based on the aforementioned sensor data, the current direction of the passenger's attention is determined. To estimate a passenger's future attention direction for at least a portion of the journey, based at least partially on their current attention direction, a machine learning algorithm is used. Maintaining the passenger environment, and maintaining the environment, includes determining, at least in part, changes to the content set of the environment based on the future direction of attention, and such changes to the content set include adding or removing objects or effects from the content set, by a control system. A boarding system that includes this.

22. The vehicle system according to claim 21, wherein the one or more sensors include one or more cameras, one or more accelerometers, or both.

23. The vehicle system according to claim 21, wherein the future direction of attention includes a range of directions associated with a portion of time in the near future during the ride, and the current direction of attention represents at least a portion of the current field of view of the passenger, or at least a portion of the current combined field of view of the passenger and a second passenger of the vehicle.

24. The vehicle system according to claim 21, wherein the environment includes a physical medium arranged along the vehicle's route, a virtual environment displayed via one or more display screens, or both the physical medium and the virtual environment.

25. Maintaining the environment is Rendering the first portion of the content set to be incorporated into the environment, Displaying the rendered first portion of the content set, The vehicle system according to claim 21, including

26. The vehicle system according to claim 25, wherein maintaining the environment includes not displaying the second portion of the content set.

27. ​​The vehicle system according to claim 26, wherein the first portion of the content set is within the passenger's field of vision corresponding to the future direction of attention, and the second portion of the content set is outside the passenger's field of vision.

28. The vehicle system according to claim 21, wherein the future direction of attention includes a range of directions associated with a portion of time in the near future during the ride, and the current direction of attention represents at least a portion of the current field of view of the passenger, or at least a portion of the current combined field of view of the passenger and a second passenger of the vehicle.

29. The vehicle system according to claim 21, wherein the control system is configured to train the machine learning algorithm based on the predetermined attention directions of one or more past passengers during one or more past ride periods.

30. The ride system according to claim 29, wherein the control system is configured to retrain the machine learning algorithm based on the passenger's current attention direction during the ride.

31. A vehicle system, A vehicle configured to support passengers throughout the duration of their journey, One or more sensors configured to generate sensor data indicating the passenger's current direction of attention, A control system, Based on the aforementioned sensor data, the current direction of the passenger's attention is determined. Based at least partially on the current attention direction, a machine learning algorithm is used to estimate the passenger's future attention direction for at least a portion of the ride period. Based on the aforementioned future direction of attention, select the first portion of the digital environment for rendering. Based on the aforementioned future direction of attention, select a second portion of the digital environment for display, or Both A control system configured to maintain the passenger's digital environment, A vehicle system that includes [this].

32. The control system includes one or more digital displays, Selecting a first portion of the digital content from the aforementioned digital environment for rendering, Selecting a second portion from the first portion of the digital content for display, To enable the display of the second portion of the digital content as the digital environment via the one or more digital displays, The vehicle system according to claim 31, configured to maintain the digital environment as a result.

33. The vehicle system according to claim 32, wherein the control system is configured to maintain the digital environment by selecting a third portion of the first portion of the digital content, the second portion of the first portion of the digital content being within the passenger's field of view corresponding to the future direction of attention, and the third portion of the first portion of the digital content being outside the passenger's field of view corresponding to the future direction of attention.

34. The boarding system according to claim 31, wherein the future direction of attention includes a range of directions associated with a portion of the near future time during the boarding period, and the current direction of attention represents at least a portion of the current field of view of the passenger, or a portion of the current combined field of view of the passenger and a second passenger of the vehicle.

35. The vehicle system according to claim 34, wherein the control system is configured to train the machine learning algorithm based on the predetermined attention directions of one or more past passengers during one or more past ride times.

36. The vehicle system according to claim 31, wherein the passengers include people or virtual passengers.

37. A method, To generate sensor data indicating the current attention direction of a passenger in a vehicle system via one or more sensors, The control circuit of the vehicle system determines the passenger's current direction of attention based on the sensor data, The control circuit utilizes a machine learning algorithm to estimate the passenger's future attention direction for at least a portion of the ride, based at least partially on the current attention direction. Maintaining the digital environment of the vehicle system via the control circuit, and maintaining the digital environment means Based on the aforementioned future direction of attention, render the first part of the digital environment, Based on the aforementioned future direction of attention, display the second part of the digital environment, or Including both, Methods that include...

38. The method according to claim 37, wherein the machine learning algorithm is trained on training data including multiple attention directions of multiple past passengers across multiple past ride sessions of the ride system.

39. The method according to claim 38, comprising retraining the machine learning algorithm based on the passenger's current attention direction during the ride.

Images