Virtual reality motion simulator system

Abstract

Method for operating a motion simulation system comprising receiving first data indicative of motion from a motion base 102 comprising a moveable platform 104, determining one or more first values bas

Description

Field of invention

[01] The present invention relates to motion simulator systems, in particular, motion simulator systems incorporating virtual reality display devices. Background

[02] It is known to provide motion simulator systems comprising a motion base. Such motion bases comprise one or more actuators supporting a movable platform, the number and type of actuators depending on the desired number of degrees of freedom of the motion platform, the type and extent of motion required, whether redundant load paths are required, etc. In such systems, the motion platform supports a human user, and visual feedback is typically provided to the user via one or more display screens connected to, and in a fixed position relative to, the platform. Such systems are used in vehicle training simulators (for example aircraft simulators), entertainment rides (such as “4D cinema theatres” and “Dark Rides”), and other contexts.

[03] Virtual reality (VR) headsets are also known, in which a wearable head mounted display (HMD) displays a viewpoint of a virtual environment. The HMD is coupled to or includes one or more motion sensors (typically inertial measurement units, accelerometers, gyroscopes, etc.), which detect motion of the wearer’s head. The viewpoint of the virtual environment displayed by the wearable display is then adjusted in accordance with the detected motion, such that the user is able to look around the virtual environment by moving their head.

[04] HMDs employing inertial measurement units, accelerometers, gyroscopes, etc. as motion sensors are not suited for use with motion simulator systems. An HMD worn by a user on the platform of a motion base experiences motion due to both the motion of the user (e.g., moving their head) and additionally motion of the platform. The HMD thus outputs an adjusted viewpoint that does not correspond to the viewpoint that the user is expecting.

[05] Prior art solutions to this problem employ optical sensors mounted to the platform, in which the position of the HMD in the frame of reference of the platform is tracked using the optical sensors. The optically tracked position of the HMD is then used to determine which motion experienced by the HMD is due to a user’s movement of their head, as opposed to due to movement of the platform. However, such optical tracking systems are complex and expensive. Moreover, optical tracking systems are often sensitive to vibration, and are ill suited to applications where significant vibration is provided as part of the simulation (for example in rotary wing aircraft training simulators).

[06] It is an aim of the present invention to address one or more of the disadvantages associated with the prior art. Summary of invention

[07] In order to mitigate at least some of the issues above, in one aspect of the present invention there is provided a computer-implemented method for operating a motion simulation system. The method comprises: receiving first data from a motion base comprising a moveable platform, the first data indicative of motion of the moveable platform (e.g., linear position / velocity / acceleration, angular velocity / acceleration, attitude, orientation, etc.); determining one or more first values based on the received first data, the one or more first values indicative of an estimated motion of a head mounted display (or other wearable display device) worn by a user on the movable platform; receiving second data from one or more sensors of the head mounted display, the second data indicative of a detected motion of the head mounted display (e.g., linear position / velocity / acceleration, angular velocity / acceleration, attitude, orientation, etc.); determining one or more second values based on the received second data; comparing the one or more first values to the one or more second values; based on the comparison, identifying a movement of the head mounted display relative to the movable platform; determining a field of view of a virtual environment based at least in part on the identified movement of the head mounted display relative to the platform; and causing the field of view to be displayed via one or more display screens of the head mounted display.

[08] Advantageously, the present invention effectively adjusts / compensates the output of the one or more sensors of the head mounted display so as to isolate motion of the head mounted display in the movable platform’s frame of reference. This reduces / prevents undesirable visual artefacts and effects (caused by the one or more sensors detecting motion due to the motion base simulating a particular force) from being presented to the user via the one or more display screens. The user is thus presented with a smoother visual representation of the simulated environment, that more accurately reflects the user’s expected visual feedback in the simulation. This additionally assists in reducing the likelihood that the user experiences motion sickness.

[09] Moreover, the compensation can be beneficially performed using the output of the motion base and of the head mounted display themselves, without the requirements for external optical tracking systems. Not only does this avoid the use of components with high sensitivity / low robustness to vibration, but it allows the invention to provide improved, lower levels of latency. Low levels of latency are further advantageous for reducing the likelihood that the user experiences motion sickness.

[10] The invention thus provides a motion simulation technique providing improved visual output, lower component cost, and with reduced maintenance requirements.

[11] Optionally, the one or more first values comprise a first position (e.g., relative to a position of the platform) indicated by the first data; and the one or more second values comprise a second position (of the head mounted display) indicated by the second data. Alternatively or in addition, the one or more first values comprise, or are calculated based on, one or more of a first linear velocity, a first linear acceleration, a first attitude, a first angular velocity and / or and a first angular acceleration indicated in the first data; and the one or more second values comprise, or are calculated based on, one or more of a second linear velocity, a second linear acceleration, a second attitude, a second angular velocity and / or and a second angular acceleration indicated in the second data.

[12] Optionally, comparing the first value to the second value comprises subtracting the first value from the second value, or subtracting the second value from the first value.

[13] Optionally the first data is indicative of motion that would be experienced at a first reference point relative to the moveable platform due to motion of the moveable platform, and the second data is indicative of motion experienced at a second reference point relative to the head mounted display due to motion of the head mounted display. In this case, the method comprises, prior to receiving the first data and the second data: calibrating the motion base and / or the one or more sensors, such that, at a first time, the first reference point and the second reference point are coincident. In an example, the first reference point is offset from a surface of the movable platform, and determined by a location of the head mounted display relative to the movable platform at the first time; and the second reference point corresponds to the location of the head mounted display at the first time. Advantageously, this allows for the compensation module to simply compare (e.g., subtract) respective values from data streams coming from the motion base and the one or more sensors - this reduction in computational complexity further reduces latency, as well as placing reduced demand for processing resource on the system.

[14] In an embodiment, the method further comprises: prior to receiving the first data: determining a force or change in force to be simulated; instructing the motion base to move the moveable platform so as to simulate the force or change in force; wherein obtaining the second position of the moveable platform comprises receiving, from the motion base, an indication that the moveable platform has moved to the second position. Optionally determining the force or change in force to be simulated is based at least in part on an input received at a user input device of the motion base.

[15] In another aspect of the invention, there is provided a computer readable medium (for example a non-transitory computer readable medium) comprising instructions that, when executed by one or more processors (for example a plurality of processors distributed across a plurality of computing devices), cause the processors to perform the method above.

[16] In another aspect of the present invention there is provided a computing device (for example a host device configured to run a simulation of a vehicle in a virtual environment, provide instructions to a motion base, and provide image data to a head mounted display) comprising one or more processors, the processors configured to perform the method above.

[17] In another aspect of the present invention there is provided a head mounted display system configured to communicate with a motion base having a movable platform, the head mounted display system comprising: one or more display screens; one or more sensors configured to detect movement of the head mounted display; one or more processors configured to perform the method above.

[18] In another aspect of the present invention there is provided a motion simulator system configured to communicate with a head mounted display having one or more sensors and one or more display screens, the motion simulation system comprising: a motion base comprising a movable platform and one or more actuators configured to move the platform; one or more processors configured to perform the method above. The motion simulation system optionally further comprises the head mounted display having one or more sensors and one or more display screens.

[19] The motion simulation system is optionally: a vehicle simulator; a fixed wing aircraft flight simulator, a rotary wing aircraft flight simulator; or an entertainment ride. The motion simulator system may alternatively be used for any other application in which a human user is sitting / standing or otherwise positioned on a motion base and using a head mounted display.

[20] Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and / or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and / or features of any embodiment can be combined in any way and / or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and / or incorporate any feature of any other claim although not originally claimed in that manner. Brief description of the drawings

[21] Embodiments of the present invention will now be described, by way of example only, with reference to the following figures in which:

[22] Figure 1 shows a schematic representation of a motion simulation system in accordance with an embodiment of the present invention.

[23] Figure 2 shows a flow chart of a method in accordance with an embodiment of the present invention.

[24] Figure 3 shows a schematic system diagram of the motion simulation system in accordance with an embodiment of the present invention.

[25] Figure 4A shows a schematic representation of the motion simulation system of figure 1 in a first state and a corresponding field of view presented to a user.

[26] Figure 4B shows a schematic representation of the motion simulation system of figure 1 in a second state and a corresponding field of view presented to a user.

[27] Figure 4C shows a schematic representation of the motion simulation system of figure 1 in a third state and a corresponding field of view presented to a user. Detailed description

[28] Figure 1 shows a schematic representation of a motion simulation system 100 comprising a motion base 102 and a head mounted display 120. The motion simulation system 100 is configured to provide a user 101 with visual and motion simulation of an object in a virtual environment. In some examples, the object being simulated is a vehicle, and the motion simulation system 100 simulates the user riding in, and optionally controlling the vehicle. The vehicle being modelled may be real-world vehicle, such as a fixed wing or rotary wing aircraft, or any other vehicle, and the simulation system can be used, for example, as a driver / pilot / operator training simulator. Alternatively, the motion simulation system 100 may be used in an entertainment experience context, wherein the vehicle being simulated does not necessarily have a real-world analogue. Similarly, the motion simulation system 100 may be used for any training, research, entertainment or other application in which a human user is positioned on a motion base 102 while viewing a virtual environment on an HMD 120.

[29] The motion base 102 comprises a moveable platform 104 and one or more actuators 106a, 106b, 106c. In use, the platform 104 supports a human user 101 and, depending on the application of the motion simulation system 100, will generally include seating for one or more users.

[30] The one or more actuators 106a, 106b, 106c are coupled to the moveable platform and preferably also to a support 108 or ground. Actuation of the actuators 106a, 106b, 106c causes the platform 104 to move relative to the support 108. As shown, the actuators 106a, 106b, 106c are linear actuators (for example linear electric or hydraulic actuators) coupled to the platform 104 (and the support 108 or ground) via universal joints 107a, 107b, 107c, 107d, 107e. However, the actuators 106a, 106b, 106c may additionally or alternatively include other types of actuators (for example rotational / crankshaft actuators configured to produce vibration).

[31] Figure 1 illustrates three actuators 106a, 106b, 106c, however it will be appreciated that more or fewer actuators may be provided, for example depending on the number degrees of freedom (DOF) that the platform is required to move in. In one example, the motion base 102 is a “hexapod” or Stewart platform, comprising 6 linear actuators configured to move the platform 104 in six degrees of freedom. Being able to move the platform 104 in six degrees of freedom allows the motion base 102 to simulate roll, pitch, heave, sway, surge and yaw motions.

[32] The motion simulation system 100 comprises a control module 110. The control module 110 is configured to control operation of the actuators 106a, 106b, 106c in order to move the platform 104. The control module 110 receives commands to cause the motion base 102 to simulate a certain motion / force, and optionally implements one or more cueing and / or washout filters as known in the art to calculate one or more setpoints for the motion base 102, and cause the actuators 106a, 106b, 106c to move the platform 104 such that the setpoints are achieved.

[33] Optionally, the motion base 102 also includes one or more user input devices 109. The user input devices 109 provide the user with a means to interact with the simulation being provided, and may be different depending on the simulation being provided. For example, in some embodiments, the motion simulation system 100 can be used as a rotary wing aircraft training simulator - in this case the user input devices 109 can replicate the controls typically found in a rotary wing aircraft (for example, a cyclic control, collective pitch control, throttle, and anti-torque pedals), and user interaction with the user input devices 109 is used to control a simulated rotary wing aircraft in a virtual environment. Similarly, the user input devices 109 may replicate the control found in other vehicles when simulating such vehicles using the motion simulation system 100. It will be appreciated that in some embodiments, no input devices are provided (for example when the motion simulation system 100 is used in the context of an entertainment ride).

[34] The motion simulation system 100 further comprises a head mounted display (HMD) 120, comprising one or more display screens 122 (for example two display screens providing a stereoscopic output) and one or more sensors 124. In some examples, the HMD 120 is or comprises a virtual reality (VR), mixed reality (MR), augmented reality (AR) and / or extended reality (XR) headset / glasses. The HMD 120 is configured to be worn by a user when on the platform 104. Though a head mounted display is described herein, it will be appreciated that alternative user-wearable display devices may be used. The HMD 120 is configured to display a field of view of a virtual environment (or a mixture of real and virtual environments) to a user wearing the HMD 120. The virtual environment depends on the end application of the motion simulation system 100 - for example, the virtual environment may be a virtual aircraft cockpit when using the motion simulation system 100 as a flight training simulator. The field of view simulates a view of the virtual environment from a certain point within in the virtual environment looking in a certain direction. The sensors 124 are configured to detect motion of the HMD 120. In some examples, the sensors 124 include an inertial measurement unit (IMU), one or more gyroscopes, one or more accelerometers and / or one or more position sensors. In a preferred example, the one or more sensors collectively provide information describing the position, linear velocity, linear acceleration, attitude (i.e., pitch roll and yaw), angular velocity and angular acceleration of the HMD 120.

[35] The motion simulation system 100 comprises a simulation module 130. The simulation module 130 is configured to generate / maintain a computer simulation of an object (e.g., vehicle) within the virtual environment. The simulation module 130 models motion of the object (for example according to a predetermined routine / path through the virtual environment, or alternatively in response to user input), including the forces that would be experienced by the object / by a user on the object, and provides instructions to the control module 110 to cause the motion base 102 to simulate the required forces.

[36] The simulation model 130 also determines content within a user’s field of view of the virtual environment as the simulated object moves through the virtual environment, based on both the simulated motion of the object through the virtual environment, and motion of the HMD detected by the one or more sensors 124 as discussed in more detail below.

[37] The motion simulation system 100 also includes a compensation module 132 configured to receive signals from the control module 110 indicative of the motion of the platform 104, and signals from the one or more sensors indicative of the motion of the HMD. Operation of the compensation module 132 is discussed in more detail below.

[38] The control module 110, the simulation module 130 and the one or more sensors 124 are each communicatively coupled to the compensation module 132, for example via a wired or wireless data connection. Optionally, the control module 110 is also communicatively coupled to the simulation module 130. The one or more user input devices 109 are communicatively coupled to the control module 110, the simulation module 130, or both. It is noted that each of the control module 110, the simulation module 130 and the compensation module 132 can be implemented as dedicated hardware modules or as software modules implemented by a suitable computing device. In some embodiments, the control module 110, the simulation module 130 and the compensation module 132 are implemented in separate computing devices. Alternatively, they may be combined in one or more computing devices. For example, the control module 110 can be provided in a computing device integrated into the motion base 102, and the simulation module 130 and the compensation module 132 can be provided in a separate host computing unit communicatively coupled to the motion base 102. In further examples the simulation moule 130 and / or the compensation module 132 are implemented together at the HMD 120 or with the control module 110 at the motion base 102.

[39] The operation of the motion simulation system 100 will now be described with reference to the method 200 of figure 2 and the schematic architecture shown in figure 3. The method 200 may be performed by one or more processors comprised in, or executing, the control module 110, simulation module 130 and compensation module 132. In an embodiment, a computer readable medium (in some examples a non-transitory medium) is provided, the medium comprising instructions that, when executed by one or more processors, cause the processors to perform the method 200. As shown in figure 3, the simulation module 130 and the compensation module are software modules executing on a host computing device 320. As noted above, the compensation module 132 may alternatively be implemented / executed at the HMD 120 or the control module 110.

[40] The method 200 is performed while the HMD 120 is being worn by a user situated on the platform 104. The method 200 optionally begins at optional step S202, in which an output of the one or more sensors 124 describing the position, orientation, and / or motion of the HMD 120, and an output of the motion base 102 (for example an output of the controller 110) describing the position, orientation and / or motion of the platform 104, are calibrated. In this embodiment, the motion base 102 (e.g., the controller 110) is configured to output data that describes or is indicative of motion that would be experienced (e.g., by a fixed, rigid body on the platform 104) due to movement of the platform 104 at a first predetermined reference point (or “referential”). The one or more sensors 124 are configured to output data that describes or is indicative of the position, orientation and / or motion experienced by the HMD 120 relative to a second predetermined reference point (or “referential”). In this embodiment, the respective outputs of the sensors 124 and motion base 102 are calibrated such that the first and second reference points are coincident in space at a first point in time (e.g., at a time before the simulation begins). For example, the second point of reference may be chosen to correspond to a position of the HMD 120, and the first reference point may be chosen based on the height of the user, such that the first reference point is at the same position and orientation as the second reference point at a time before the simulation begins. The first reference point in effects acts as an estimated position / orientation of the HMD 120 relative to the platform 104. Advantageously, this calibration allows for the direct comparison of data output by the one or more sensors 124 and the motion base 102, reducing processing resource demand and reducing latency as discussed in more detail below.

[41] At step S204, the one or more displays 122 display a first field of view of the virtual environment to the wearer of the HMD 120. In the example shown in figure 3, the first field of view is determined by the simulation module 130, which outputs image data 304 corresponding to the first field of view to the HMD 120 for display to the user.

[42] The method then proceeds to step S206, in which the position of the platform 104 is updated, and first data describing or indicative of the position and / or motion of the platform 104 is received by the compensation module. In use, the position of the platform 104 changes to simulate different forces / changes in force experienced by the user according to the simulation. As the simulation progresses, the compensation module 132 (and optionally the simulation module 130) is preferably periodically updated with the current position, linear velocity, linear acceleration, attitude, angular velocity and / or angular acceleration of the platform 104 (for example as would be experienced at the first reference point).

[43] For example, if the simulation module 130 determines that a certain force should be simulated for the user when the platform is in a first position, either based on a predetermined routine (for example when the motion simulation system 100 is used as an entertainment ride) or based on a user input 312 signal from the one or more user input devices 109 (for example when the motion simulation system 100 is used as a flight simulator), the simulation module 130 sends a request 302 to the control module 110 to cause the motion base 102 to simulate said force / change in force. After applying any appropriate cueing and / or washout filters, the control module 110 then causes the actuators 106a, 106b, 106c to move the platform 104 (for example providing vibration, and / or motion in roll, pitch, yaw, heave, sway and / or surge degrees of freedom) in such a way as to simulate the force for the user. This results in the platform 104 moving to, or moving via, a new position. Data 306 describing the new position and / or motion of the platform 104 is then sent to the compensation module 132 by the control module 110. The first data 306 preferably comprise the current position, linear velocity, linear acceleration, attitude, angular velocity and / or angular acceleration of the platform 104 as described above, but may alternatively comprise one or more new setpoints of the platform 104.

[44] It will be appreciated that in some instances, the position of the platform 104 may not change at step S206, for example if the simulation module 130 instructs the control module 110 to maintain a steady state (for example when simulating an aircraft cruising). In this case, the first data reflects that there has been no motion / change in position of the platform 104.

[45] At optional step S208, a change in estimated position of the HMD 120can be determined. Such a step is beneficial in the event that the initial calibration at step S202 is not performed. In this case, the compensation module 132 optionally estimates a change in position of the HMD 120 due to motion of the platform 104. When estimating the position of the HMD 120, it is assumed that the user’s head, and hence the HMD 120 is in a fixed position in the frame of refence of the platform 104 (put differently, it is assumed that the HMD 120 remains at the same referential origin in the frame of reference of the platform 104). Alternatively, the position of the HMD 120 may be determined by one of more external sensors (not shown). It is noted that the change in estimated position of the HMD 120 is based only on any changes due to movement of the platform 104 (since the estimate is based on the assumption that the HMD 120 has remained stationary in the frame of reference of the platform 104). Changes in HMD 120 position due to a user additionally moving their head are not accounted for in the estimate.

[46] At step S209 one or more first values are determined, for example by the compensation module 132, based on the received first data. Where calibration at step S202 has been performed, the first values are optionally the values of the current position, linear velocity, linear acceleration, attitude, angular velocity and / or angular acceleration comprised in the first data 306 (in which case the determination may simply be reading, identifying or otherwise retrieving the respective values from the first data). In other examples, the first values may be other quantities calculated using the first data 306. Where calibration at step S202 has not been performed, determining the first values may comprise calculating an estimated position, linear velocity, linear acceleration, attitude, angular velocity and / or angular acceleration of the HMD 120 based on the first data 306 and on the estimated change in HMD position determined at optional steps S208.

[47] The first values may thus be considered to correspond to (or be indicative of) an expected output of the one or more sensors 124 in response to motion of the platform 104 causing motion of the HMD 120. For example, the first values may represent an estimated / expected position, linear velocity, linear acceleration angular velocity, angular acceleration and or attitude of the HMD 120.

[48] At step S210, the compensation module 132 receives second data 308 from the one or more sensors 124 indicative of the actual current position / motion of the HMD 120. Preferably, the second data are received substantially contemporaneously with the first data at step S206 to reduce lag in the simulation. In a preferred embodiment, the second data 308 are an actual output of the one or more sensors 124. For example, the second data 308 may comprise, or be indicative of, a measured / detected position, linear velocity, linear acceleration, angular velocity, angular acceleration and / or attitude of the HMD 120. Optionally, the second data 308 includes a position of the HMD 120 as measured by and transmitted by one or more external sensors 350, or relative to one or more external references 352 (for example one or more optical sensors positioned external to the motion base 102 and which are not moved by the platform 104 as the platform moves).

[49] Based on the received second data 308, the compensation module 132 determines one or more second values at step S211. In some examples, the one or more second values are or comprise the measured / detected position, linear velocity, linear acceleration, angular velocity, angular acceleration and / or attitude of the HMD 120 as comprised in the second data 308 (in which case the determination may simply be reading, identifying or otherwise retrieving the respective values from the first data). Alternatively, the second values may be other quantities calculated using the second data 308.

[50] At step S212, the one or more second values are compared to the one or more first values calculated at step S209. In one example, the comparison is / includes subtraction of the first value from the second value, or vice versa. Advantageously, by performing this comparison, a component of the movement (or the measured effect of that movement) experienced by the HMD 120 that is due to the user moving their head is isolated from the component of the movement experienced by the HMD 120 that is due to motion of the platform 104. The result of the comparison, indicative of the movement of the HMD 120 relative to the platform 104, is then fed back to the simulation module 130.

[51] In a preferred example in which calibration has been performed at step S202, the first values comprise the position, linear velocity, linear acceleration, angular velocity, angular acceleration and attitude experienced at the first reference point as indicated in the first data 306 received from the motion base 102. Similarly, in this example the second values comprise the position, linear velocity, linear acceleration, angular velocity, angular acceleration and attitude of the HMD 120 as measured by the one or more sensors 124. By virtue of the calibration at step S202, the second values are provided / determined in the same frame of reference as the first values. In this example, the compensation model 132 is configured to compare the values by subtracting each first value from the respective second value (for example subtracting the linear velocity reported by the motion base 192 from the linear velocity reported by the one or more sensors 124, and so on). The respective positions x, linear velocities x, linear accelerations x, angular velocities co, and angular accelerations co are vector quantities. A corrected vector value for each of the above vector parameters is calculated by subtracting HMD vector components (of the second data 308, from the one or more sensors 124) from motion base vector components (of the first data 306, from the control module 110) as follows: ■^■corrected HMD sensor ^-Motion base data ^corrected %HMD sensor — ^Motion base data •^corrected -^HMD sensor ^-Motion base data ^corrected ^HMD sensor ^Motion base data ^corrected ^HMD sensor ^-Motion base data Equation 1 Equation 2 Equation 3 Equation 4 Equation 5

[52] The respective estimated and detected attitudes A are expressed as quaternion values of the form A(a, b, c, d) = a + bi + cj dk where i, j, and k, are mutually perpendicular square roots of -1. In this example, the corrected attitude Acon-ected is calculated by multiplying the conjugate of the estimated attitude AMotion base data with the detected attitude Ahmd sensor as follows: ^■corrected = conj(A Motion base data) * HMD sensor Equation 6

[53] At step S216, the simulation module 130 determines how the HMD 120 has moved relative to the platform 104 based on the output 310 of the compensation module 132 (for example based on the results of subtractions described above). Alternatively, the compensation module 132 determines how the HMD 120 has moved relative to the platform 104 and sends this as an output 310 to the simulation module 130. The simulation module 130 then determines an updated field of view of the user in the virtual environment based on the movement of the HMD 120 relative to the platform 104 as a proxy for the user’s head moving within the simulated environment. Optionally content in the field of view is also updated, for example based on events occurring within the virtual environment (for instance characteristics shown by a simulated cockpit instrument panel may change, the view of a simulated landscape outside a window of the simulated vehicle may change, etc.). The simulation module 130 generates image data corresponding to the updated field of view, and provides this to the display 122. An image corresponding to the updated field of view is then displayed to the user via the display 122 at step S218.

[54] By compensating the output of the one or more sensors 124 in the manner described above, the invention advantageously identifies and isolates the motion of the HMD 120 in the platform’s 104 frame of reference. This allows the correct field of view to be displayed to the user, effectively filtering out extraneous effects due to motion of the platform 104. Thus disturbances / undesirable visual effects and artefacts that do not correspond to the user’s expected feedback from the simulation are beneficially avoided.

[55] Moreover, the compensation is performed using the output of the motion base 102 and of the HMD 120 themselves, providing lower levels of latency than known techniques requiring feedback from additional optical sensors. Low levels of latency are advantageous for reducing the likelihood that the user experiences motion sickness.

[56] Steps of the method 200 are preferably repeated while the simulation is ongoing, iteratively updating the field of view displayed on the one or more displays 122 as the platform 104 moves, as the user’s head moves, and as the simulation progresses (e.g., as the simulated vehicle moves through the virtual environment). In one embodiment, the HMD 120 displays a series of frames of image data sequentially via the one or more display screens 122, and the field of view is updated each time a new frame of image data is to be displayed. That is, steps S202 to S218 are performed every time a new frame of image data is to be displayed - put differently the frequency with which steps S204-S218 are performed preferably at least matches the refresh rate of the one or more displays 122. Advantageously, the control module 110 and the one or more sensors 124 are configured to output data (describing the movement of the platform 104, and the movement of the HMD 120, respectively) to the compensation module 132 at a rate that is sufficiently high enough to provide low latency and high frame rates. In some examples, data received from the controller 110 and from the one or more sensors is substantially temporally synchronised, beneficially yet further improving the accuracy of the updated field of view presented to the user. Alternatively, or in addition, steps S204 to S218 are performed responsive to the compensation module 132 receiving a signal from the controller 110 indicating that the platform 104 has moved / is moving, and / or responsive to the compensation module 132 receiving a signal from the one or more sensors 124 indicating that the HMD 120 has moved / is moving.

[57] An example implementation of method 200 above is illustrated in relation to figures 4A to 4C, in which the motion simulation system 100 is used as a rotary wing aircraft flight simulator. Figure 4A shows an initial state of the motion simulator system 100, and a schematic of an image of a corresponding field of view 404a. Figures 4B and 4C show alternative states of the motion simulator system 100 at a point in time after the state shown in figure 4A, again with respective schematic images of corresponding fields of view 404b, 404c.

[58] In this example, the virtual environment includes the interior of a rotary wing aircraft cockpit within a larger environment through which the simulated aircraft travels. The initial position 400a of the HMD 120 and the initial position 402a of the platform 104 are shown in figure 4A. An image corresponding to a first field of view 404a of the virtual environment is presented to the user via the HMD 120 (step S204). The motion simulation system 100 has been calibrated (step S202).

[59] In the case of figure 4B, the user makes an input via the user input devices 109 corresponding to an instruction for the simulated aircraft to accelerate. In response, the simulation module 130 instructs the control module to simulate the acceleration of the aircraft. The actuators 106a, 106b, 106b change the position of the platform 104 to a new position 402b to tilt the user backwards to simulate the increase in forward acceleration of the aircraft. Accordingly, the HMD 120 moves to new position 400b. In response, the compensation module 132 receives data from both the control module 110 and the one or more sensors relating to the motion of the platform 104 and the HMD 120 respectively (steps S206, S210) and the first and second values are determined steps S209, S211 - in this case the determination simply involves reading / identifying / retrieving the respective positions, linear velocities, linear accelerations, angular velocities, angular accelerations, and attitudes from the first and second data). The first values are then subtracted from the second values (step S212). Based on the subtraction, movement of the HMD 120 relative to the platform 104 is identified and provided to the simulation module 130. In the example of figure 4B, the user’s head has not moved substantially relative to the platform 104; accordingly, the compensation module 132 identifies that the HMD 120 has not moved substantially relative to the platform 104.

[60] In contrast, in the case of figure 4C, the user has not made an input via the user input devices 109, and as a result the position 402c of the platform 104 is substantially the same as its initial position 402a. However, the user has tilted their head to look “up” in the simulation, and as a result the position of the HMD 120 has changed to position 400c. In this case, after following steps S206, S209, S210, S211, and S212, the compensation module 132 effectively identifies how the HMD 120 has moved relative to the platform 104.

[61] The simulation module 130 then determines an updated field of view of the interior of the cockpit based on the movement of the user’s head relative to the platform and other factors (step S216). The content in the virtual environment within the updated field of view may include an updated speed display on an instrument panel to reflect the acceleration. The virtual environment outside the simulated cockpit is further moved relative to the cockpit interior in the updated field of view, to simulate motion of the aircraft through the virtual environment. The updated field of view, including the updated content in the virtual environment within the updated field of view, is then displayed to the user via the display 122 (step S218).

[62] In the case of figure 4B, the updated field of view 404b displayed to the user reflects the increase of speed of the simulated aircraft, (for example virtual landmarks outside the simulated cockpit are shown at a different position relative to the virtual cockpit). However, the horizon is displayed at substantially the same position as in the initial field of view 404a in figure 4A, since the user’s head has not moved relative to the platform 104.

[63] In the case of figure 4C, the updated field of view 404c displayed to the user includes the horizon at a lower position than in the initial field of view 404a in figure 4A, to reflect the user moving their head upwards.

[64] It will be appreciated that in other examples both movements of the platform 104 and movements of the HMD 120 relative to the platform can occur, and are dealt with according to the method 200 above.

[65] Embodiments of the present invention thus provide a robust, cost effective and low latency means for ensuring the field of view presented to a user of a head mounted display accurately corresponds to motion of a user’s head, when operated on a motion base providing simulated forces to the user. 5

[66] The above embodiments are provided as examples only. The scope of the invention is defined by the appended independent claims. The invention covers all modifications, variations and equivalents as fall within the scope of the appended independent claims.

Claims

1. A computer-implemented method for operating a motion simulation system, the method comprising:receiving first data from a motion base comprising a moveable platform, the first data indicative of motion of the moveable platform;determining one or more first values based on the received first data, the one or more first values indicative of an estimated motion of a head mounted display worn by a user on the movable platform;receiving second data from one or more sensors of the head mounted display, the second data indicative of a detected motion of the head mounted display;determining one or more second values based on the received second data;comparing the one or more first values to the one or more second values;based on the comparison, identifying a movement of the head mounted display relative to the movable platform;determining a field of view of a virtual environment based at least in part on the identified movement of the head mounted display relative to the platform; andcausing the field of view to be displayed via one or more display screens of the head mounted display.

2. The method of claim 1, wherein:the one or more first values comprise a first position indicated by the first data; andthe one or more second values comprise a second position indicated by the second data.

3. The method of claim 1, wherein:the one or more first values comprise, or are calculated based on, one or more of a first linear velocity, a first linear acceleration, a first attitude, a first angular velocity and / or and a first angular acceleration indicated in the first data; andthe one or more second values comprise, or are calculated based on, one or more of a second linear velocity, a second linear acceleration, a second attitude, a second angular velocity and / or and a second angular acceleration indicated in the second data.

4. The method of any preceding claim, wherein:the first data is indicative of motion that would be experienced at a first reference point relative to the moveable platform due to motion of the moveable platform;the second data is indicative of motion experienced at a second reference point relative to the head mounted display due to motion of the head mounted display; andthe method comprises, prior to receiving the first data and the second data:calibrating the motion base and / or the one or more sensors, such that, at a first time, the first reference point and the second reference point are coincident.

5. The method of claim 4, wherein:the first reference point is offset from a surface of the movable platform, and determined by a location of the head mounted display relative to the movable platform at the first time; andthe second reference point corresponds to the location of the head mounted display at the first time.

6. The method of any preceding claim, wherein comparing the first value to the second value comprises subtracting the first value from the second value, or subtracting the second value from the first value.

7. The method of any preceding claim, further comprising:prior to receiving the first data:determining a force or a change in force to be simulated; andinstructing the motion base to move the moveable platform so as to simulate the force or change in force;wherein the first data is based on the moveable platform having moved so as to simulate the force or change in force.

8. The method of claim 7, wherein determining the force or change in force to be simulatedis based at least in part on an input received at a user input device of the motion base.

9. A computer readable medium comprising instructions that, when executed by one or more processors, cause the processors to perform the method of any of claims 1 to 8.

10. A computing device comprising one or more processors, the processors configured to perform the method of any of claims 1 to 8.

11. A head mounted display system configured to communicate with a motion base having a movable platform, the head mounted display system comprising:one or more display screens;one or more sensors configured to detect movement of the head mounted display; andone or more processors configured to perform the method of any of claims 1 to 6.

12. A motion simulator system configured to communicate with a head mounted display having one or more sensors and one or more display screens, the motion simulation system comprising:a motion base comprising a movable platform and one or more actuators configured to 5 move the platform; andone or more processors configured to perform the method of any of claims 1 to 8.

13. The motion simulation system of claim 12, further comprising the head mounted display having one or more sensors and one or more display screens.1014. The motion simulation system of claim 12 or 13, wherein the motion simulator system is:a vehicle simulator;a fixed wing aircraft flight simulator;15 a rotary wing aircraft flight simulator; oran entertainment ride.

Images