Head tracking system

Abstract

A head tracking system (HTS) (100, 200) and associated methods and head-mounted assemblies configured to determine the position and / or orientation of a user's head, comprising: one or more optical sensors (112) that generate optical measurements; one or more inertial sensors (104) that generate inertial measurements; and a processor configured to operate in a coasting mode using inertial measurements from the one or more inertial sensors to determine the location and orientation of the user's head when the optical sensors are unable to provide optical measurements for determining the location and orientation of the user's head. The processor is further configured to determine a figure of merit (FOM) (346) of coasting error based on one or more combinations of an estimated inertial sensor bias error, a random drift and noise of the inertial sensor, and a gain error and an alignment error of the inertial sensor; compare the FOM with a threshold (348) to determine whether the inertial measurements provide a valid head location and / or orientation or an invalid head location and / or orientation; and determine a subsequent operation from a plurality of possible subsequent operations of the HTS based on whether the head location and / or orientation is valid or invalid.

Description

Technical Field

[0001] The present invention relates to a head tracking system.

Background Art

[0002] A head tracking system is used to determine where and how a user's head is oriented and positioned based on a platform and / or a head-mounted sensor and / or an emitter and / or a reference. Data on the orientation and position of the head can be used to align an image presented within a head-mounted display with the real world and / or a virtual world as a reference. Also, data on the orientation and / or position of the head can be utilized by other platform systems to control sensors and / or other devices. Head tracking is generally based on the position (XYZ) and orientation (yaw, pitch, and roll) of a user's head.

[0003] A head tracking system uses a single head tracking technique and sometimes multiple head tracking techniques that operate together to improve data on the orientation and / or position of the head by using a combination of optical measurements and inertial measurements. When both are used, the system is called hybrid tracking. Access to optical measurements and inertial measurements provides an element of redundancy that can be used to improve the integrity of the head tracker output.

[0004] In certain situations where one or the other set of orientation and / or position measurements is not received or gives incorrect information, using the orientation and / or position of the head can lead to dangerous situations. Further, when relying only on inertial measurements (which may be called coasting), errors tend to accumulate. When relying only on optical measurements, the head tracking system becomes unusable when there is no optical solution that can be achieved.

[0005] There is a need for a head tracking system that overcomes at least a portion of the impossible head tracking data or the risk of increased error, i.e., overcomes at least a portion of the current problems with known head tracking systems. SUMMARY OF THE INVENTION

[0006] According to one aspect of the present invention, there is provided a head tracking system (HTS) configured to determine the position and / or orientation of a user's head. The HTS includes one or more optical sensors that generate optical measurements, one or more inertial sensors that generate inertial measurements, and a processor configured to operate in a coasting mode using inertial measurements from the one or more inertial sensors to determine the location and orientation of the user's head when the optical sensors are unable to provide optical measurements for determining the location and orientation of the user's head. The processor is further configured to determine a figure of merit (FOM) for coasting error based on one or more combinations of an estimated inertial sensor bias error, a random drift and noise of the inertial sensors, and a gain error and an alignment error of the inertial sensors, compare the FOM to a threshold to determine whether the inertial measurements provide a valid head location and / or orientation or an invalid head location and / or orientation, and determine a subsequent operation from a plurality of possible subsequent operations of the HTS based on whether the head location and / or orientation is valid or invalid.

[0007] In one aspect, when the coasting error is less than the threshold, the head location and / or orientation is valid.

[0008] In one aspect, when the coasting error is greater than or equal to the threshold, the head location and / or orientation is invalid.

[0009] In one aspect, the HTS further includes an action control module for determining which subsequent operations are available.

[0010] In one aspect, possible subsequent operations include normal operations when the location and / or orientation of the head is valid, or the operation does not require the user's position or orientation.

[0011] In one aspect, possible subsequent operations include limited operations when the location and / or orientation of the head is invalid, and the operation does not require the user's position or orientation.

[0012] In one aspect, possible subsequent operations include declaring tracking invalid when the location and / or orientation of the head is invalid, and the operation requires the user's position or orientation.

[0013] In one aspect, the processor is configured to determine the position and / or orientation of the head from one or both of one or more optical sensors and one or more inertial sensors, determine the position and / or orientation of the head from one or more inertial sensors when the optical measurements are invalid, and determine the position and / or orientation of the head from one or more optical sensors when the inertial measurements are invalid.

[0014] In one aspect, the rate of change of the FOM is determined to identify trends and risk potential.

[0015] In one aspect, one or more inertial sensors include three or more gyroscopes located on a head-mounted assembly associated with the HTS.

[0016] In one aspect, one or more optical sensors include a camera in the vicinity of the HTS configured to receive optical measurements from one or more light elements located on a head-mounted assembly associated with the HTS.

[0017] In one aspect, an interface (118) is provided for receiving the orientation of the aircraft, the position of the aircraft, and the latency associated with the data of the orientation and position of the aircraft, which are used when calculating the FOM.

[0018] In one aspect, the head tracking system is incorporated into a head-mounted display assembly such as a helmet.

[0019] According to a further aspect of the present invention, a head-mounted display assembly is provided that is configured to present an image aligned with the external scene to a user presenting a real image and a virtual image, based on the position and orientation of the head provided by an HTS of another aspect.

[0020] In one aspect, an optical display is provided for displaying a real image and a virtual image to a user wearing the head-mounted assembly, and at least the presentation of the image is based on the FOM of the inertial measurement values.

[0021] In one aspect, the optical display comprises at least one of a visor projection display system and a waveguide-based display arranged to present a virtual image such that the virtual image appears superimposed through the visor of the head-mounted display.

[0022] According to a further aspect of the present invention, there is provided a method of determining the location and orientation of a user's head at which a real image and a virtual image are presented, using a head tracking system (HTS) configured to generate optical measurement values and generate inertial measurement values, the method comprising: determining a figure of merit (FOM) of a coasting error based on one or a combination of a bias error of an estimated inertial sensor, a random drift and noise of the inertial sensor, and a gain error and an alignment error of the inertial sensor when there are no optical measurement values for determining the location and position of the head using the inertial measurement values; comparing the FOM with a threshold (348) to determine whether the inertial measurement values provide a valid head location and / or head orientation or an invalid head location and / or head orientation; and determining a subsequent operation from a plurality of possible subsequent operations of the HTS based on whether the head location and / or head orientation is valid or invalid.

[0023] In one aspect, determining the location and / or orientation of the head is valid when the FOM is less than the threshold (406).

[0024] In one aspect, determining the location and / or orientation of the head is invalid when the FOM is greater than or equal to the threshold (408).

[0025] In one aspect, via an action control module (410), it is determined which subsequent operations are available.

[0026] In one aspect, when the head location and / or head orientation is valid, it is determined that the possible subsequent operation is a normal operation (414), or the operation does not require the user's position or location.

[0027] In one aspect, when the head location and / or head orientation is invalid, it is determined that the possible subsequent operation is a restricted operation (416), and the operation does not require the user's position or location.

[0028] In one aspect, when the head location and / or the head orientation is invalid, it is determined (418) that a possible subsequent action declares tracking invalid, and the action requires the user's position or location.

[0029] In one aspect, the rate of change of the FOM (412) is determined to identify trends and risk potential.

[0030] In one embodiment, to enable the vehicle control system to reset head tracking, the vehicle control system is notified when the head location and / or the head orientation is invalid.

[0031] Next, embodiments of the present invention will be described merely by way of example with reference to the drawings.

Brief Description of the Drawings

[0032]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Modes for Carrying Out the Invention

[0033] The present invention relates to a helmet tracking system (HTS). A hybrid helmet tracking system that uses an inertial management unit (IMU) attached to a helmet can be used to perform orientation tracking when a primary tracking source is not available. This is the case when the optical system cannot provide an optical solution in the case of an optical / inertial tracking system. The present disclosure describes a method that can estimate the error accumulated during tracking using only the IMU. For example, if the error exceeds a certain value, this can be used to signal that the tracking output is invalid.

[0034] Deterioration of orientation and accuracy caused by an inertial measurement unit (IMU) that drifts over time / temperature and other environmental factors limits the available time for coasting, which is a limiting factor in tracking using only the IMU. Coasting refers to using an angular velocity derived from inertia during a short time period when optical measurements are not available.

[0035] The present invention provides an estimated value of the angular error during coasting. This estimated value can be used internally by the helmet tracking system, an external system, or a user to determine whether the can tracking data is accurate enough to be used.

[0036] Referring to FIG. 1, a helmet tracker system (HTS) 100 integrated with a helmet 106 is shown. The HTS 100 includes several inertial sensors mounted on the outer shell of the helmet 106 and an optical helmet tracker system.

[0037] Each inertial sensor is in the form of an IMU 104 that includes three gyroscopes (gyroscopes) that are nominally orthogonal to each other. At least one gyro in one IMU is configured to measure a first parameter, and at least one gyro in at least one other IMU is configured to measure the same or an equivalent parameter. Thus, comparisons can be made between gyros / IMUs.

[0038] In other embodiments, the number of IMUs 104 and the number of gyros per IMU can vary. If there are three orthogonal gyros (e.g., aligned along the X, Y, and Z axes), the position and orientation of the head can be tracked even with a course error. If there is some redundancy among all the gyros (e.g., there are four gyros and two of them are aligned with one of the X, Y, or Z axes), comparisons can be made to help determine the course error.

[0039] The optical helmet tracker system includes an array of LEDs 110 integrated within or attached to the outer shell of the helmet 106 and an array of one or more cameras 112, each of the cameras 112 being attached at a fixed position in the vicinity of the helmet, e.g., in an aircraft cockpit or within another vehicle, and being adapted to have a line of sight to at least some of the LEDs 110 in any possible orientation of the helmet 102. The present invention is not limited to LEDs, and instead, any other suitable light element or marker can be used.

[0040] Furthermore, instead of an optical tracker, any tracking system that provides an absolute measurement of the orientation compared to an inertial system may be used.

[0041] The optical helmet tracker controller 114 is linked to each of the cameras 112 to receive image data resulting from the detection of light emitted by the LEDs 110 within the field of view (FOV) of the cameras, and is also linked to the LEDs 110 of the helmet 102 so as to be able to control the illumination of the LEDs 110. The controller 114 is arranged, for example, to trigger the periodic illumination of the LEDs 110 in a predetermined pattern to receive the resulting image data from the cameras 112. The controller 114 includes a digital processor that is programmed to illuminate the LEDs 110 and interpret the resulting image data from the cameras 112 to determine the orientation and position of the helmet with respect to a reference frame associated with a world stabilised environment and, in this example, a platform stabilised environment such as an aircraft.

[0042] A further processing module 116 is provided to receive orientation and position data from the controller 114 and from the helmet-mounted gyro (IMU) 104 and to transmit the world stabilised and platform stabilised image data received from an image generation system (not shown) installed on the platform, via the interface 118, to a helmet-mounted display (not shown) integrated within the helmet 102. The helmet-mounted display system may be a visor projection display system or a waveguide-based display, and in either case is arranged to present a virtual image to the pilot such that the virtual image appears to overlap the pilot's field of view through the visor of the helmet 102 of the external scene.

[0043] Figure 2 shows a block diagram of a hybrid HTS200 for determining the orientation of a helmet with respect to a predetermined reference point such as an aircraft cockpit. The HTS200 receives sensor data from two or more helmet-mounted IMUs 202 (each IMU comprising three or more gyros), an optical HTS204, and an aircraft navigation system 206.

[0044] Each of the two or more IMUs 202 includes three small gyroscopes (gyros), each associated with one of three nominally orthogonal gyro axes. The axes enable determination of yaw, pitch, and roll. Each gyro can sense the rate of change of the orientation of the system of helmet-mounted gyro 104 in inertial space, decomposed along its respective gyro axis. The system can determine the rate of change of the orientation of the helmet sensed by the individual gyros about their respective gyro axes. These are called inertial measurements and are collected at 208.

[0045] The system provides for measuring the same or equivalent parameters so that at least two gyros or at least two IMUs can perform a comparison.

[0046] The optical HTS 204 includes an LED 110 on the helmet and a camera 112 near the helmet, as described in FIG. 1. The data collected by the camera enables identification of the exact position and orientation of the helmet from the perspective of XYZ positioning. These are called optical measurements and are collected at 210.

[0047] The aircraft or platform navigation system 206 provides multiple types of information regarding the aircraft and the environment supplied by any associated sensors used by the aircraft to provide information. This is called aircraft data and is collected at 212.

[0048] A combination of IMU measurement values, optical measurement values, and optionally aircraft measurement values is used to determine the exact position and orientation of the helmet in space 214. The position and orientation are used to determine how to present an image to the pilot. The image is a combination of a real-world image and a virtual image such as a symbol or the like. The real-world image is obtained from the scene the pilot sees. The virtual image is based on sensor data related to the operation and operating conditions of the platform or vehicle. This can be combined and presented to the pilot. The presentation of a particular virtual image (such as the compass bearing or other symbol fixed to an external object) is based on the position and orientation of the user or operator provided by the HTS200.

[0049] As described above, the HTS200 is a hybrid system and ideally combines optical measurement values and inertial measurement values. Depending on the position and location of the head, the optical measurement values may sometimes be unavailable and the system continues by using only inertial measurement values. During inertial-only tracking, the system relies on the correct operation of the inertial measurement unit (IMU). Further, during inertial-only tracking, due to the inherent characteristics of the IMU and external factors, errors slowly increase over time.

[0050] As a result, there may be a safety risk in relying on inertial-only tracking data. It is necessary to check in real time that the IMU is functioning correctly and for the accumulation of errors over time.

[0051] Next, referring to FIG. 3, the coasting error tracking model 300 will be described. Note that the tracker uses three orthogonally mounted gyros included in the IMU, which may be referred to as a single gyro or rate in some cases.

[0052] The HTS200 undergoes continuous measurement cycles. For each cycle, the optical section of the HTS acquires a set of measurement values, and the inertial section of the HTS acquires a set of IMU measurement values (consisting of three data sets for each of the three orthogonal gyros). In practice, note that the optical section may operate on the substrate of the inertial section, resulting in continuous short gyro coursing.

[0053] If an optical solution cannot be calculated, i.e., if insufficient data has been collected to generate a solution, the HTS can use only the IMU measurement values. This results in a state called gyro coursing.

[0054] If an optical solution can be obtained, the tracker updates the estimated value of the bias of gyro block A. A certain amount of valid history data is required to correctly calculate the angular velocity from the angular output data. The angle 324 of the platform (in the inertial space) is required to compare the gyro rate (in the inertial space) with the optical tracker (platform space). The angle measurements of the aircraft are typically acquired at different times, and this difference is compensated, for example, by using time stamps and suitable time alignment. Under dynamic conditions, if there is a time error in the measurement time stamp of the aircraft, an error will occur in the estimated bias. Block A estimates this error using the platform orientation data 324 and the calculated rate 326 when there is optical tracking.

[0055] A delay 330 is added to the rate 326 of the platform orientation data by an amount equal to the estimated platform measurement time stamp error 328. The error 328 is the estimated worst-case error of the platform orientation measurement time tag.

[0056] The difference 332 between the rate 326 and the delayed rate 330 is determined.

[0057] Based on the filter used in the core tracker algorithm that filters the gyro bias estimate value, filter 334 is applied. Typically, the filter within the core tracker removes noise. The output of filter 334 is an estimate of the IMU bias error and is stored in temporary storage 336. When there is no optical solution and the coursing is operating, the estimated IMU bias error is not updated.

[0058] In block A, an estimate of the gyro coursing error is determined. As described above, during coursing, only gyro data is used. This is effectively integrated to generate an orientation. The error accumulation during coursing is one or a combination of the estimated gyro bias error, the random drift and noise of the gyro, and the gyro gain error and alignment error.

[0059] The impact of the error caused by the gyro bias error is estimated by using the estimated error 336 of the gyro bias and multiplying this by the time 318 that the system is gyro coursing to generate a value 332.

[0060] The error 320 due to the random drift and noise of the gyro is determined by using a polynomial 316 that represents the expected orientation drift as a function of time and giving the time that the system is gyro coursing to the polynomial. The coefficients of the polynomial are calculated offline based on the measurements of a sample set of gyros combined with the error model of the core tracker system.

[0061] Errors due to gyro gain and alignment (including intrinsic and extrinsic) errors are calculated by using the head rotation rate 344, which is calculated by the core tracker system and generated from corrected gyro measurements corrected for bias, gain, and alignment. Thereafter, an error is applied 304 based on the gain error 306. The gain error 306 is applied in a specific direction (+ or -) for each of the three gyros. This gain error represents the estimated equivalent gain error calculated offline that represents the error in calibrating the gyro gain and the alignment error.

[0062] The rate is integrated 308. The angular difference between the orientations calculated by the core tracker is calculated 312. The difference 310 between the orientations 308 is the estimated gyroscopic drift error due to gyro gain error and alignment error. If the estimated equivalent IMU gain error 306 is zero, the difference 310 is zero.

[0063] The above steps 304, 308, and 310 are repeated 314 for four combinations of sign directions. Note that there are three gyros, each of which can have a + or - error, for a total of eight, but two sets of four are the same except for the sign, for a total of four combinations. The algorithm uses the square of the error value, and thus the sign is removed (this is not shown in FIG. 3).

[0064] The difference 310 due to gyro gain error and alignment error, the error 320 due to gyro random drift and noise, and the error 332 due to the estimated gyro bias error are added to estimate the total gyroscopic drift error 338 and generate an estimated gyroscopic drift performance index (FOM) 346. In parallel, the total gyroscopic drift error 338 is compared with the error limit 348, and if it is less, a flag is generated 350 indicating that the gyroscopic drift is valid.

[0065] Note that there are many measurements taken to accumulate data for determining various types of errors. These include, but are not limited to, one or more of the following. · Time from the last optical measurement · Angle difference of the head due to loss of optical tracking · Angle difference of the platform due to loss of optical tracking · Number of IMUs · Performance of the IMU, e.g., noise and stability · Gyroscopic scaling error · Misalignment of the gyro (with respect to the optical axis) · Non - orthogonality of the gyro · Gyroscopic scaling temperature coefficient · Temperature variations · Bias filter constant · Unknown aircraft latency · Maximum aircraft rotation speed · Earth rotation speed (if not compensated) · Aircraft latitude and longitude

[0066] The FOM is determined and used by the HTS to control future actions, as described below with reference to Figure 4.

[0067] The HTS may also include a subsystem 400 that monitors the FOM and is used to determine which actions are acceptable and which are not based on the FOM. The FOM 402 is determined as described above.

[0068] The FOM value is also evaluated to determine the rate of change 412 of the FOM. This is also passed to the action control module 410.

[0069] The action control module is used to evaluate the FOM and, in some situations, the rate of change 412 of the FOM, each of which is detailed below.

[0070] The action control module compares the FOM with a plurality of acceptable levels, each associated with an action or task that can be performed by the user, and generates a plurality of validity flags. For example, there are tasks that can continue normal operation 414 without requiring the details of the exact position or orientation.

[0071] The FOM can be derived from one or more error estimate values associated with the tracking system. These errors are based on sensor bias errors, drift errors, noise errors, sensor gain errors, alignment errors, and aircraft-related errors associated with latency, acceleration, and any other errors that can affect inertial tracking.

[0072] There are other tasks that strongly depend on correct orientation data. The action control module determines which subsequent actions are available from a plurality of possible subsequent actions. Possible subsequent actions include any actions that can be performed by the operator or automatically by the vehicle system. Possible subsequent actions are accessible from a lookup table, database, any other suitable device or method.

[0073] The action control module also receives the rate of change 412 of the FOM over time. This is processed to determine the trend of any change in the FOM. In response to a particular change, the action controller module can take similar steps regarding subsequent or future actions of the HTS. For example, by using the current value of the FOM and the rate of change of the FOM, the HTS can notify an external system that the FOM is about to reach a critical value and provide an estimated time until gyroscopic coasting becomes invalid. If the rate of change is very small, the HTS is likely to be operating stably (regardless of whether it is a valid or invalid operation). This allows the external system or user to act before an invalid FOM is declared.

[0074] By combining the actual FOM and the rate of change of the FOM, the risk associated with errors in providing inaccurate information to the user, for example via symbols, is significantly reduced. Further, each can contribute to performance improvement independently.

[0075] Referring to FIG. 5, an example 500 thereof is shown. The actual FOM is determined 502 in the same manner as the rate of change of the FOM 504. Each combined value is compared to a threshold 506. This generates a plurality of validity flags 508, which are passed to an action module 510 and a plurality of actions 512 are initiated. Actions generally affect how data is used by an external system, are compared to a threshold, and / or the action module can be part of either an external system or an internal element of the HTS.

[0076] In all cases, the action controller module determines, based on the actual value of the FOM, which of a plurality of tasks are safe or not. Depending on the level of risk, some tasks continue, some are restricted, or some are declared deterministically invalid and the system must attempt to recover the tracking in an appropriate way.

[0077] The FOM can be generated such that the system can selectively control the presentation of information according to the accuracy required for that particular symbol. As described above, different types of symbols are enabled or not enabled depending on the value of the FOM. The FOM can include different levels, for example 1, 2, 3. For each level, the associated symbol is hidden from view due to the possibility that the FOM falls below a predetermined threshold for that particular symbol. If the FOM threshold falls below the required level for a particular symbol, the required safety threshold is not met and using that particular symbol can lead to risks.

[0078] The accurate tracking provided here enables the system to present symbols to the user at the correct locations and times within their field of view. In particular, it can enable the symbols to appear aligned or fixed to real-world objects beyond the helmet. For example, a virtual "N" may float above the north direction, or a virtual waypoint marker may appear on the flight path, or a virtual ring may surround and thereby highlight the instrument panel area. If the exact head position and / or orientation cannot be determined, the presentation of symbols that are assumed to be fixed to real-world objects may be misaligned and thus risk leading the user into errors and accidents.

[0079] If the symbols do not need to be aligned or fixed to external real-world objects, they may continue to be presented regardless of whether accurate head tracking is possible. For example, a digital odometer may always be presented in the upper left of the user's field of view regardless of where the user's head is facing.

[0080] The symbols can include at least one or more of the following. · Signs and symbols · Data from sensors · Processed data from sensors · Combinations of sensor data · Military symbols · Platform-related symbols · Scene-related symbols · Location and positioning symbols · Map symbols · Speed and velocity symbols

[0081] The present invention is as described above, but it will be understood that many variations and alternative forms will be apparent to those skilled in the art. The optical measurements may be replaced by any measurements that can be processed to provide absolute measurements of position and orientation.

[0082] It will be understood that the above invention may be modified to include equivalent features different from those described, but still fall within the scope of the claims.

Claims

1. A head tracking system (HTS) configured to determine the position and / or orientation of a user's head, one or more optical sensors that generate optical measurements, one or more inertial sensors that generate inertial measurements, a processor configured to operate in a coasting mode using the inertial measurements from the one or more inertial sensors to determine the location and orientation of the user's head when the optical sensors are unable to provide optical measurements for determining the location and orientation of the user's head, comprising, wherein the processor determines a figure of merit (FOM) for coasting error based on one or more combinations of an estimated inertial sensor bias error, random drift and noise of the inertial sensors, and gain error and alignment error of the inertial sensors, compares the FOM to a threshold to determine whether the inertial measurements provide a valid head location and / or orientation or an invalid head location and / or orientation, and determines a subsequent operation from a plurality of possible subsequent operations of the HTS based on whether the head location and / or orientation is valid or invalid, A head tracking system, further configured to perform.

2. The head tracking system according to claim 1, wherein when the coasting error is less than the threshold, the head location and / or orientation is valid.

3. The head tracking system according to claim 1, wherein when the coasting error is greater than or equal to the threshold, the head location and / or orientation is invalid.

4. The head tracking system according to claim 1, 2, or 3, wherein the HTS further includes an action control module for determining which subsequent operations are available.

5. The head tracking system according to claim 4, wherein the possible subsequent operations include normal operation when the head location and / or orientation is valid.

6. The head tracking system according to claim 4, wherein the possible subsequent operations include limited operation when the head location and / or orientation is invalid, and the operation does not require the user's position or orientation.

7. The possible subsequent operations include declaring tracking invalid when the location and / or orientation of the head is invalid, and the operations require the position or orientation of the user. The head tracking system according to claim 4.

8. The processor determines the position and / or orientation of the head from one or both of the one or more optical sensors and the one or more inertial sensors; when the optical measurements are invalid, determines the position and / or orientation of the head from one or more inertial sensors; when the inertial measurements are invalid, determines the position and / or orientation of the head from one or more optical sensors; The head tracking system according to any one of claims 1 to 7, which is configured to perform.

9. The head tracking system according to any one of claims 1 to 8, which is further configured to determine a rate of change of FOM to identify trends and risk potential.

10. The head tracking system according to any one of claims 1 to 9, wherein the one or more inertial sensors include three or more gyroscopes located on a head-mounted assembly associated with the HTS.

11. The head tracking system according to any one of claims 1 to 10, wherein the one or more optical sensors include a camera in the vicinity of the HTS configured to receive optical measurements from one or more light elements located on a head-mounted assembly associated with the HTS.

12. The head tracking system according to any one of claims 1 to 11, further comprising an interface for receiving data related to the orientation of the aircraft, the position of the aircraft, and the latency associated with the orientation and position of the aircraft used in calculating the FOM.

13. The head tracking system according to any one of claims 1 to 12, incorporated in a head-mounted display assembly such as a helmet.

14. A head-mounted display assembly configured to present an image aligned with the external scene to a user presenting a real image and a virtual image, based on the position of the head and the orientation of the head provided by the HTS according to any one of claims 1 to 13.

15. The head-mounted assembly according to claim 14, further comprising an optical display for displaying the real image and the virtual image to a user wearing the head-mounted assembly, wherein at least the presentation of the image is based on the FOM of the inertial measurement values.

16. The head-mounted assembly according to claim 14 or 15, wherein the optical display comprises at least one of a visor projection display system and a waveguide-based display arranged to present the virtual image such that the virtual image is seen overlapping through the visor of the head-mounted display.

17. A method of determining the location and orientation of a user's head at which a real image and a virtual image are presented, using a head tracking system (HTS) configured to generate optical measurement values and generate inertial measurement values, when there are no optical measurement values for determining the location and position of the head using the inertial measurement values, determining a figure of merit (FOM) of a costing error based on one or a combination of a bias error of an estimated inertial sensor, a random drift and noise of the inertial sensor, and a gain error and an alignment error of the inertial sensor; comparing the FOM with a threshold value to determine whether the inertial measurement values provide a valid head location and / or head orientation or an invalid head location and / or head orientation; determining a subsequent operation from a plurality of possible subsequent operations of the HTS based on whether the head location and / or the head orientation is valid or invalid; A method comprising the steps of:

18. The method according to claim 17, wherein determining the head location and / or the head orientation is valid when the FOM is less than the threshold value, and determining the head location and / or the head orientation is invalid when the FOM is greater than or equal to the threshold value.

19. further comprising determining, via an action control module, which subsequent operations are available, when the head location and / or the head orientation is valid, determining that the possible subsequent operation is a normal operation, or the operation does not require the position or the location of the user, When the location and / or orientation of the head is invalid, determining that the possible subsequent operation is a restricted operation, and the operation does not require the user's position or location. When the location and / or orientation of the head is invalid, determining that the possible subsequent operation declares tracking invalid, and the operation requires the user's position or location. The method according to claim 18, further comprising one or more of the above.

20. The method according to any one of claims 17 to 19, further comprising determining a rate of change of FOM to identify trends and risk potential.

21. The method according to any one of claims 17 to 20, further comprising notifying the vehicle control system when the FOM is invalid to enable the vehicle control system to reset head tracking.

Images