Head tracking system

Abstract

A head tracking system (HTS) configured to determine the position and orientation of a user's head such that a display image is configured to be aligned with and presented relative to an external scene, the HTS comprising: one or more optical sensors that generate optical measurement values; several inertial sensors that generate inertial measurement values; and a processor configured to determine the position and orientation of the head from one or both of the one or more optical sensors and the several inertial sensors, wherein the processor is configured to determine an optical tracking state from the optical measurement values and an inertial tracking state from the inertial measurement values; flag / assert that the operation of the HTS is normal when both the optical tracking state and the inertial tracking state are valid; flag / assert a fault of the HTS when both the optical tracking state and the inertial tracking state are invalid; and compare the optical measurement values and the inertial measurement values to determine whether the HTS can operate safely based on a predetermined tracking performance required by the system when the optical tracking state and the inertial tracking state are valid.

Description

Technical Field

[0001] The present invention relates to a head tracking system.

Background Art

[0002] A head tracking system is used to determine where and how a user's head is oriented and positioned based on a platform and / or a head-mounted sensor and / or an emitter and / or a reference. Data on the orientation and position of the head can be used to align an image presented in a head-mounted display with the real world and / or a virtual world as a reference. Also, data on the orientation and / or position of the head can be utilized by other platform systems to control sensors and / or other devices. Head tracking is generally based on the position (XYZ) and orientation (yaw, pitch, and roll) of the user's head.

[0003] A head tracking system uses a single head tracking technique and sometimes multiple head tracking techniques that operate together to improve data on the orientation and / or position of the head by using a combination of optical measurements and inertial measurements. When both are used, the system is called hybrid tracking. The ability to access optical and inertial measurements provides an element of redundancy that can be used to improve the integrity of the head tracker output.

[0004] In certain situations where one or the other set of orientation and / or position measurements is not received or gives incorrect information, using the orientation and / or position of the head can lead to dangerous situations. Further, when relying only on inertial measurements, i.e., in a scenario often called coasting, errors tend to accumulate. When relying only on optical measurements, the head tracking system becomes unavailable when there is no optical solution that can be achieved.

[0005] There is a need for a head - tracking system that overcomes at least some of the unavailable head - tracking data or the risk of increased error, i.e., that overcomes at least some of the current problems associated with known head - tracking systems. SUMMARY OF THE INVENTION

[0006] According to one aspect of the present invention, there is provided a head - tracking system (HTS) (100, 200) configured to determine the position and orientation of a user's head such that a displayed image is aligned with and presented relative to an external scene. The HTS includes one or more optical sensors (112) that generate optical measurements, several inertial sensors (104) that generate inertial measurements, and a processor (114) configured to determine the position and orientation of the head from one or both of the one or more optical sensors and the several inertial sensors. The processor is configured to determine an optical tracking state from the optical measurements, determine an inertial tracking state from the inertial measurements, flag / assert that the operation of the HTS is normal when both the optical tracking state and the inertial tracking state are valid, assert / flag a failure of the HTS when both the optical tracking state and the inertial tracking state are invalid, and compare the optical measurements with the inertial measurements to determine whether the HTS can operate safely based on a predetermined tracking performance required by the system when the optical tracking state and the inertial tracking state are valid.

[0007] In one aspect, comparing the optical measurements with the inertial measurements comprises determining data associated with the inertial measurements to identify whether one or more of the inertial sensors have failed.

[0008] In one aspect, the several inertial sensors comprise two or more inertial sensors, and the processor is further configured to receive inertial measurements from two or more of the inertial sensors and use the inertial measurements to identify whether some of the inertial sensors have failed.

[0009] In one aspect, some inertial sensors comprise two or more inertial sensors, and the processor is further configured to determine respective inertial measurement values in parallel for each inertial sensor, and the respective inertial measurement values are compared in a coasting mode to determine whether the combined inertial measurement values cause a failure.

[0010] In one aspect, if the inertial measurement values cause a failure, based on the optical measurement values, a calculated head position and head orientation are generated.

[0011] In one aspect, if the inertial measurement values do not cause a failure, the inertial measurement values are combined with the optical measurement values to generate a calculated head position and head orientation based on the optical measurement values and the inertial measurement values.

[0012] In one aspect, some inertial sensors (104) comprise three or more gyroscopes located on a head-mounted assembly associated with the HTS. Further, some inertial sensors may comprise four or more gyroscopes, and two gyroscopes sense the same parameters to provide redundancy.

[0013] In one aspect, one or more optical sensors (112) comprise optical sensors (112) in the vicinity of the HTS configured to receive optical measurement values from one or more light elements (110) located on a head-mounted assembly associated with the HTS. In an alternative aspect, the optical sensors may be mounted on the head-mounted assembly, and the light elements are mounted in the vicinity (e.g., mounted on the cockpit surface).

[0014] In one aspect, one or more measurement values are received from sensors associated with a vehicle in which the HTS is used.

[0015] In one aspect, the vehicle control system is notified that both the optical tracking state and the inertial tracking state are invalid.

[0016] In one aspect, it is incorporated into a head-mounted assembly such as a helmet.

[0017] In one aspect, a displayed image aligned with the external scene is presented to the user based on the position and orientation of the head provided by the HTS of any preceding claim.

[0018] In one aspect, an optical display is provided for displaying an image aligned with the external scene to a user wearing the head-mounted assembly, and at least the presentation of the virtual image is based on an optical tracking state and an inertial tracking state.

[0019] In one aspect, the optical display comprises at least one of a visor projection display system and a waveguide-based display arranged to present a virtual image such that the virtual image appears overlaid with the real-world image through a visor or waveguide forming part of the head-mounted assembly.

[0020] According to a second aspect, there is provided a method of determining the position and orientation of a user's head configured to be aligned with and presented with a displayed image aligned with the external scene, the method comprising determining an optical tracking state (306) from optical measurements and an inertial tracking state from inertial measurements, flagging / asserting the operation of the HTS as normal when both the optical tracking state and the inertial tracking state are valid (316), flagging / asserting a failure of the HTS when both the optical tracking state and the inertial tracking state are invalid (312), and comparing the optical measurements with the inertial measurements to determine whether the HTS can operate safely based on a predetermined tracking performance required by the system when one of the optical tracking state and the inertial tracking state is valid.

[0021] In one aspect, comparing the optical measurements with the inertial measurements comprises determining data associated with the inertial measurements to identify whether any sensor has failed.

[0022] In one aspect, a plurality of inertial measurement values are received, inertial measurement values from two or more inertial sensors are determined, and the inertial measurement values are used to identify whether any of the inertial sensors has failed.

[0023] In one aspect, a plurality of inertial measurement values are received and the plurality of inertial measurement values are compared in a coasting mode to determine whether the combined inertial measurement values cause a failure.

[0024] In one aspect, when the inertial measurement values cause a failure, a calculated head orientation is generated based on optical measurement values.

[0025] In one aspect, when the inertial measurement values do not cause a failure, the inertial measurement values are combined with the optical measurement values to generate a head orientation based on the optical measurement values and the inertial measurement values.

[0026] In one aspect, the inertial measurement values are received from two or more gyroscopes located on a head-mounted assembly associated with the HTS.

[0027] In one aspect, the optical measurement values are received from an optical sensor in the vicinity of the HTS configured to receive the optical measurement values from one or more light elements located on a head-mounted assembly associated with the HTS.

[0028] In one aspect, one or more measurement values are received from sensors associated with a vehicle in which the HTS is used.

[0029] In one aspect, the vehicle control system is notified that both the optical tracking state and the inertial tracking state are invalid.

[0030] Next, embodiments of the present invention will be described by way of example only with reference to the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031]

Figure 1

Figure 2

Figure 3

Figure 4

DETAILED DESCRIPTION OF THE INVENTION

[0032] The present invention relates to a head tracking system (HTS). In particular, the present invention relates to the testability and integrity of inertial tracking systems. Hybrid tracking is used to describe the operation of an HTS that utilizes both optical and inertial tracking together. Coasting is used to describe operation with inertial-only tracking. The degradation of the performance of the coasting orientation due to drift over time / temperature and other environmental factors of current inertial measurement units (IMUs) means that the typically available time for coasting (in a very dynamic environment) can be limited to a very short time period. This is because the accuracy of the coasting orientation data may exceed the specified accuracy required for a particular application under a particular combination of dynamic conditions. The present invention addresses determining inertial tracking performance during hybrid tracking, during coasting, and during the transition from coasting back to hybrid tracking. IMU measurements are also compared with optical tracking data (when available) to ensure that the system operates in a state of minimum error, thereby ensuring improved integrity operation of the HTS.

[0033] Hybrid tracking uses both a primary tracking system and a head-mounted IMU to perform orientation tracking when a primary tracking source is available. Generally, the primary tracking system is an optical system, but it may also be a magnetic or other tracking technology that can provide consistent absolute measurements of the position and orientation of the head over dynamic operating conditions.

[0034] The degradation of the orientation performance of head tracking during coasting is a limiting factor, forcing the need to maintain good optical coverage over a large head motion box within the cockpit or other space for head tracking. This forces the possibility of having more sensors, emitters, and / or references within the cockpit (or other space for head tracking), as well as more trackers, optical sensors, emitters, and / or references on the head assembly, which adds undesirable mass and complexity.

[0035] Instead, the present invention can help improve the available time and, along with continuous monitoring of performance, can help extend the coasting period and reduce the degradation of orientation tracking performance.

[0036] Referring to FIG. 1, a head tracker system (HTS) 100 integrated with a helmet 106 is shown.

[0037] HTS 100 includes several inertial sensors mounted on the outer shell of helmet 106 and an optical head tracker system.

[0038] Each inertial sensor is in the form of an IMU 104 that includes three gyros (gyroscopes) that are nominally orthogonal to each other. At least one gyro in one IMU is configured to measure a first parameter, and at least one gyro in at least one other IMU is configured to measure the same or an equivalent parameter. Thus, comparisons can be made between the gyros / IMUs.

[0039] In other embodiments, the number of IMUs 104 and the number of gyros per IMU can vary. If there are three orthogonal gyros (e.g., aligned along the X, Y, and Z axes), the position and orientation of the head can be tracked even in the presence of a drifting error. If there is some redundancy between all the gyros across all the IMUs (e.g., there are four gyros and two of them are aligned with one of the X, Y, or Z axes), comparisons can be made to help determine the drifting error.

[0040] The optical section of the head tracker system 100 includes an array of LEDs 110 integrated within or attached to the outer shell of the helmet 106 and an array of one or more optical sensors 112, such as cameras, each of the optical sensors 112 being attached at a fixed position in the vicinity of the helmet, e.g., in an aircraft cockpit or other vehicle, and being adapted to have visibility of at least some of the LEDs 110 in any possible orientation of the helmet 102. The present invention is not limited to LEDs and, instead, any other suitable light element or marker can be used.

[0041] In an alternative embodiment, the optical sensor may be attached to the head-mounted assembly or helmet and the light element or marker is attached in the vicinity (e.g., attached to the cockpit surface).

[0042] The head tracker system controller 114 is linked to each of the optical sensors 112 to receive image data resulting from the detection of light emitted by the LEDs 110 within the field of view of the camera, and is also linked to the LEDs 110 of the helmet 102 so as to be able to control the illumination of the LEDs 110. The controller 114 is arranged, for example, to trigger the periodic illumination of the LEDs 110 in a predetermined pattern and to receive the resulting image data from the optical sensors 112. The controller 114 includes a digital processor programmed to illuminate the LEDs 110 and interpret the resulting image data from the optical sensors 112 to determine the orientation and / or position of the helmet with respect to a reference frame associated with the aircraft in this example.

[0043] The head tracker system controller 114 also receives, via the interface 118, rate data output from the helmet-mounted gyro (IMU) 104 and rate data from an aircraft inertial navigation system (not shown). These data are combined with the optical measurements to generate the orientation and / or position of the head. The head tracker system controller 114 is also configured to output a predicted measure of the orientation of the head.

[0044] The processing module or processor 116 receives data on the orientation and / or position of the head output from 114 and data on the orientation and position of the aircraft from an aircraft inertial navigation system (not shown), and generates a world-stabilised image for presentation to the user via a display (not shown) integrated within the head-mounted display assembly 102.

[0045] Also, the data on the orientation and / or position of the head output from 114 can also be output to the aircraft system via the interface 118.

[0046] Figure 2 shows a block diagram of a hybrid HTS200 for determining the orientation and position of the head with respect to a predetermined location such as an aircraft cockpit. The HTS200 receives sensor data from two or more helmet-mounted IMUs 202 (each IMU includes three or more gyros), an optical sensor HTS204, and an aircraft navigation system 206. The HTS includes an optical tracking system that acquires optical measurements and outputs an optical tracking state, and an inertial tracking system that acquires inertial measurements and outputs an inertial tracking state. When an optical tracking solution is available, the optical tracking state is valid. When an optical tracking solution is not available, the optical tracking state is invalid. Similarly, when an inertial tracking solution is available, the inertial tracking state is valid. When an inertial tracking solution is not available, the inertial tracking state is invalid.

[0047] Each of the two or more IMUs 202 includes a plurality of or several (e.g., three) small gyroscopes (gyros), each associated with one of three nominally orthogonal gyro axes. The axes enable determination of yaw, pitch, and roll. Each gyro can sense the rate of change of the orientation of the helmet-mounted gyro 104 system in inertial space resolved along its respective gyro axis. The system can determine the rate of change of the helmet orientation sensed by the individual gyros about their respective gyro axes. These are called inertial measurements and are collected at 208.

[0048] The system provides for measuring the same or equivalent parameters such that at least two gyros or at least two IMUs can perform a comparison.

[0049] The optical HTS204 includes an LED 110 on the helmet and an optical sensor 112 in the vicinity of the helmet as described in FIG. 1. The data collected by the optical sensor enables identification of the exact position and orientation of the helmet. These are called optical measurements and are collected at 210.

[0050] The aircraft navigation system 206 provides multiple types of information regarding the aircraft and the environment supplied by any associated sensors used by the aircraft to provide information. This is called aircraft data and is collected at 212.

[0051] A combination of IMU measurements, optical measurements, and optionally aircraft measurements is used to determine the exact position and orientation of the head display assembly in space 214. The position and orientation are used to determine how to present an image to the user and / or provided for use by other external systems (e.g., steering of aircraft sensors). The image is a combination of a real-world image and a virtual image such as a symbol or the like. The real-world image is obtained from the scene the user sees. The virtual image is based on sensor data related to the operation and operating conditions of the vehicle. This can be combined and presented to the user. The presentation of certain virtual images (such as compass bearings or other symbols fixed to external objects) is based on the position and orientation of the user provided by the HTS200.

[0052] As described above, the HTS200 is a hybrid system and ideally combines optical measurements and inertial measurements. Depending on the head pose, the optical measurements are sometimes not available and the system continues by using only inertial measurements (also called coasting). During inertial-only tracking, the system relies on the correct operation of the inertial measurement unit (IMU). Further, during inertial-only tracking, due to the inherent characteristics of the IMU and external factors, the error in the orientation calculation slowly increases over time.

[0053] As a result, there may be a risk of danger based on coasting errors. It is necessary to determine in real time that the IMU is functioning correctly and the accumulation of errors over time.

[0054] During the coursing operation, the orientation errors generated using the measurements from each IMU increase over time as all IMUs drift. This variation between the separate orientations calculated from each IMU is used to confirm that the coursing error is too large to provide the user with sufficient spatial stabilization display. This variation can be used as a performance index to selectively turn off spatial stabilization symbols according to the accuracy requirements of individual symbol functions and their associated accuracy requirements.

[0055] Having multiple IMUs can be helpful, first, by averaging out some of the individual IMU errors using data from all IMUs. Further, if the IMUs are oriented along different axes within the helmet, this makes it possible to understand and somewhat correct for any IMU accuracy variation between the axes.

[0056] Data merging can be achieved in several ways as follows.

[0057] In the first method, the average of the IMU measurements from the gyroscopes is determined, and the average IMU measurements are treated as if obtained from one IMU. This improves performance but leaves other problems.

[0058] In the second method, each IMU is processed in parallel to generate results for two orientations. This makes it possible to directly compare the results during coursing to confirm with a reasonably high confidence that the IMU is providing valid results. The two final orientation results are merged to provide an improved orientation output. This also brings about performance improvement.

[0059] The third method combines the first two methods. The main orientation calculation is performed as in the first method, and a secondary monitoring system checks the individual outputs of the gyroscopes.

[0060] The fourth method compares IMU measurements with optical measurements and will be described in more detail below.

[0061] Next, the above method will be described in detail with reference to FIG. 3. HTS200 includes the following process 300 to enhance safety and reduce a part of the errors inherent in HTS. HTS uses three orthogonally mounted gyros included in at least one IMU104. Other numbers of IMUs and gyros per IMU may be used.

[0062] In block 302, the system processes and compares IMU measurements. This outputs an average IMU rate and a fault notification (IMU fault 1) if any of the IMUs are not operating or are operating inaccurately. The IMU measurements are passed to the tracking state block 304.

[0063] HTS goes through continuous measurement cycles. For each cycle, the optical section of HTS acquires a set of optical measurements, and the inertial section acquires a set of IMU measurements (e.g., comprising three data sets for each of the three orthogonally mounted gyros).

[0064] The tracking state block 304 determines the current tracking states of both the optical tracker and the inertial tracker. If the tracking state block determines that an optical solution can be obtained 306, in other words, if it determines that the optical tracking state is valid, in block 310, a comparison between the IMU measurements and the optical measurements is performed. The comparison is based on the average IMU rate and the optical tracking. If there is a significant error in the comparison, an IMU fault (IMU fault 2) is generated. As shown in block 312, if there is an IMU fault (IMU fault 1 from 302 or IMU fault 2 from 310), there is an overall IMU fault and reliance can only be placed on optical tracking. The optical measurements with or without the IMU measurements are processed in block 314 to generate a hybrid calculated orientation output 316.

[0065] In some cases, the estimated IMU error 318 is determined and the estimation 320 of the gyroscopic coursing error is performed. If coursing is valid 322, the gyroscopic coursing process 324 is performed to generate the orientation output 326 calculated by gyroscopic coursing.

[0066] When both optical tracking and gyroscopic coursing are operating, the HTS has IMU measurements and optical measurements to identify orientation and position and, if necessary, makes comparisons to ensure that the system does not degrade in any way that leads to invalid tracking and risks to the operator and others. The comparison of the states of optical tracking and inertial tracking can result in indicating the safety level of the HTS, and the safety level of the HTS is compared with a predetermined known safety level to confirm that the HTS is operating properly and safely. The required safety level is based on the application or the predetermined tracking performance required by the HTS.

[0067] In the absence of optical tracking, the HTS can rely only on IMU measurements and the system is coursing. During coursing, only gyro data is used.

[0068] As described above, coursing causes error accumulation. This error accumulation is a combination of the estimated gyro bias error, the random drift and noise of the gyro, and external factors such as gyro gain, alignment error, and the quality and latency of aircraft data.

[0069] To reduce the coursing error, the present invention provides a process of comparing IMU measurements from each IMU, as shown in FIGS. 4a and 4b. It should be noted that although only two measurements are shown as being compared, the process may use other numbers of IMU rates.

[0070] Figures 4a and 4b depict a more detailed implementation of the process for comparing measurements from each IMU in block 302 of FIG. 3, and process 310 for comparing IMU measurements with optical measurements of FIG. 3.

[0071] In FIG. 4, a first IMU rate 402 and a second IMU rate 404 are received along with respective calibrations 406, 408 and respective bias estimates 410, 412. These values are passed to respective gyro correction modules 414A and 414B. The outputs from gyro correction modules 414A and 414B are compared 416. The compared error is compared with a limit 420. If the error is greater than the limit, the system asserts an IMU fault 1 422. The difference between the error and the limit can be used to provide a figure of merit (FOM) for the tracking system or each tracking system. The figure of merit provides a more granular indication of the effectiveness or ineffectiveness of the tracking system. This can then be used to determine the available operations for the user based on the FOM.

[0072] The outputs from gyro correction modules 414A and 414B are also averaged 418 to determine an average IMU rate 419.

[0073] In the tracking status 424 module, the effectiveness of optical tracking is checked. If optical tracking is considered valid, the system may proceed to compare optical tracking with IMU tracking. Such a comparison may indicate an IMU fault.

[0074] Referring to FIG. 4b, there is a comparison between the optical measurement value and the inertial measurement value. The compared error is compared with the limit 442. If the error is greater than the limit, the system asserts an IMU failure 2 at 444. The optical measurement value 426 and the orientation of the platform 428 are used to convert the measurement value into the inertial space at 430, and the rate is calculated at 432. This is then compared with the average IMU rate 436 determined above at 434, whereby an error 438 is generated. This error is then filtered at 440 and compared with the error limit at 442. If the error is above the threshold or limit, the system asserts an IMU failure 2 at 444. The IMU measurement data can continue to be checked against the optical measurement value, but is not used for position and orientation calculations unless it is confirmed to have recovered to a valid state.

[0075] The ideal situation is that both the optical measurement value and the inertial measurement value are available to the HTS. As described above, this is not always the case. The present invention provides means for comparing measurement values in order to optimize the reliability in inertial tracking and optical tracking.

[0076] When both inertial tracking and optical tracking are identified as operable, the HTS is operating safely and the system is flagged as fully operable. When one or the other of inertial tracking and optical tracking is identified as inoperable, the system is flagged as potentially faulty and monitoring is enhanced until the system is flagged as fully operable or faulty. When both inertial tracking and optical tracking are identified as inoperable, the system is flagged as faulty and corrective action is required.

[0077] There are several additional processes proposed by the present invention to further improve the reliability of the HTS.

[0078] A further set of comparisons is based on continuously monitoring optical and IMU measurements when either optical tracking or inertial tracking is resumed. When resuming the optical system, this requires a direct measure of the IMU error over the course of the coasting period. By monitoring the resumption of control by either tracking system, a greater confidence in returning to normal operation is better understood.

[0079] The values of the optical and inertial measurements are used to determine a training data set for machine learning analysis of real-time data. This includes data from the above-mentioned resumption of tracking. A deep neural network (DNN) is trained using the optical and inertial measurements to identify conditions leading to loss in either or both of inertial and optical tracking. Then, by feeding real-time data through the DNN, an early warning is triggered that one or both of the tracking systems are likely to fail. This effect is to initiate remedial measures as early as possible.

[0080] Another extension is to add local IMU temperature readings, which are useful for error correction and identification.

[0081] The accurate tracking provided here enables the system to present symbols to the user at the correct location and time within their field of view. In particular, it can enable the symbols to appear aligned or fixed to real-world objects beyond the helmet. For example, a virtual "N" may float above the north azimuth, or a virtual waypoint marker may appear on the flight path, or a virtual ring may surround and thereby highlight an instrument panel area. If the exact head position and / or orientation cannot be determined, the presentation of symbols assumed to be fixed to real-world objects may be misaligned and thus risk leading the user into errors and accidents.

[0082] By comparing optical tracking and inertial tracking, the system can selectively control the presentation of information according to the accuracy required for that specific symbol. As described above, different types of symbols are enabled or not enabled depending on the comparison value. If the symbol does not need to be aligned or fixed to an external real-world object, it may continue to be presented regardless of whether accurate head tracking is possible. For example, a digital odometer can always be presented in the upper left corner of the user's field of view regardless of where the user's head is facing.

[0083] The symbol can include at least one or more of the following. · Signs and symbols · Data from sensors · Processed data from sensors · Combinations of sensor data · Military symbols · Vehicle-related symbols · Scene-related symbols · Location and positioning symbols · Map symbols · Speed and velocity symbols

[0084] It will be understood that the above description of the present invention has many variations and alternative forms that will be apparent to those skilled in the art.

Claims

**Claim 1** A head tracking system (HTS) configured to determine the position and orientation of a user's head such that a display image is aligned with and presented relative to an external scene, the system comprising: one or more optical sensors that generate optical measurements; several inertial sensors that generate inertial measurements; a processor configured to determine the position and orientation of the head from one or both of the one or more optical sensors and the several inertial sensors; wherein the processor is configured to: determine an optical tracking state from the optical measurements and an inertial tracking state from the inertial measurements; flag / assert that the operation of the HTS is normal when both the optical tracking state and the inertial tracking state are valid; flag / assert a failure of the HTS when both the optical tracking state and the inertial tracking state are invalid; compare the optical measurements with the inertial measurements to determine whether the HTS can operate safely based on a predetermined tracking performance required by the system when the optical tracking state and the inertial tracking state are valid; A head tracking system (HTS) configured to perform the above operations. **Claim 2** The head tracking system (HTS) of claim 1, wherein comparing the optical measurements with the inertial measurements comprises determining data associated with the inertial measurements to identify whether one or more of the inertial sensors have failed. **Claim 3** The head tracking system (HTS) of claim 2, wherein the several inertial sensors comprise two or more inertial sensors, and the processor is further configured to receive inertial measurements from two or more of the inertial sensors and use the inertial measurements to identify whether one or more of the inertial sensors have failed. **Claim 4** The head tracking system (HTS) of claim 2 or 3, wherein the several inertial sensors comprise two or more inertial sensors, and the processor is further configured to determine respective inertial measurements in parallel for each inertial sensor, and the respective inertial measurements are compared in a coasting mode to determine whether the combined inertial measurements cause a failure. **Claim 5** The head tracking system (HTS) according to any one of claims 2 to 4, which generates the calculated head position and head orientation based on the optical measurement values when the inertial measurement values cause a failure.

6. The head tracking system (HTS) according to any one of claims 2 to 4, which combines the inertial measurement values with the optical measurement values to generate the calculated head position and head orientation based on the optical measurement values and the inertial measurement values when the inertial measurement values do not cause a failure.

7. The head tracking system according to any one of claims 1 to 6, wherein the several inertial sensors include three or more gyroscopes located on a head-mounted assembly associated with the HTS.

8. The head tracking system according to any one of claims 1 to 7, wherein the one or more optical sensors (112) include optical sensors (112) in the vicinity of the HTS configured to receive optical measurement values from one or more optical elements (110) located on a head-mounted assembly associated with the HTS.

9. The head tracking system according to any one of claims 1 to 8, which is configured to receive one or more measurement values from sensors associated with a vehicle in which the HTS is used.

10. The head tracking system according to any one of claims 1 to 9, which is configured to notify a vehicle control system that both the optical tracking state and the inertial tracking state are invalid.

11. The head tracking system according to any one of claims 1 to 10, which is incorporated in a head-mounted assembly such as a helmet.

12. A head-mounted assembly configured to present a display image aligned with the external scene to a user based on the head position and head orientation provided by the HTS according to any one of claims 1 to 11.

13. The head-mounted assembly according to claim 12, further including an optical display for displaying an image aligned with the external scene to a user wearing the head-mounted assembly, wherein at least the presentation of the virtual image is based on the optical tracking state and the inertial tracking state.

14. The head-mounted assembly according to claim 13, wherein the optical display comprises at least one of a visor projection display system and a waveguide-based display arranged to present the virtual image such that the virtual image is overlaid with the real-world image through a visor or a waveguide forming part of the head-mounted assembly.

15. A method for determining the position and orientation of a user's head configured to be aligned and presented such that a displayed image is aligned with an external scene, comprising: determining an optical tracking state (306) from the optical measurement values and determining an inertial tracking state from the inertial measurement values; flagging / asserting that the operation of the HTS is normal when both the optical tracking state and the inertial tracking state are valid (316); flagging / asserting a failure of the HTS when both the optical tracking state and the inertial tracking state are invalid (312); comparing the optical measurement values and the inertial measurement values to determine whether the HTS can operate safely based on a predetermined tracking performance required by the system when one of the optical tracking state and the inertial tracking state is valid; A method comprising.

16. The method according to claim 15, wherein comparing the optical measurement values and the inertial measurement values comprises determining data associated with the inertial measurement values to identify whether any sensor has failed.

17. The method according to claim 16, further comprising receiving a plurality of inertial measurement values, determining the inertial measurement values from two or more inertial sensors, and using the inertial measurement values to identify whether any of the inertial sensors has failed.

18. The method according to claim 16 or 17, further comprising receiving a plurality of inertial measurement values and comparing the plurality of inertial measurement values in a coasting mode to determine whether the combined inertial measurement values cause a failure.

19. The method according to any one of claims 16 to 18, wherein when the inertial measurement values cause a failure, a calculated head orientation is generated based on the optical measurement values.

20. The method according to any one of claims 16 to 18, wherein when the inertial measurement value does not cause a failure, the inertial measurement value is combined with the optical measurement value in order to generate the orientation of the head based on the optical measurement value and the inertial measurement value.

21. The method according to any one of claims 15 to 20, wherein the inertial measurement value is received from three or more gyroscopes located on a head-mounted assembly associated with the HTS.

22. The method according to any one of claims 15 to 21, wherein the optical measurement value is received from an optical sensor in the vicinity of the HTS, configured to receive the optical measurement value from one or more light elements located on a head-mounted assembly associated with the HTS.

23. The method according to any one of claims 15 to 22, further comprising receiving one or more measurement values from a sensor associated with a vehicle in which the HTS is used.

24. The method according to any one of claims 15 to 23, further comprising notifying a vehicle control system that both the optical tracking state and the inertial tracking state are invalid.

Images