Position tracking system and method for head-mounted display systems

Abstract

The present invention provides a system and method for tracking the position of one or more head-mounted display (HMD) system components of an HMD system. [Solution] The HMD component may hold multiple angle-sensing detectors or other types of detectors. The HMD system may act to detect damaged position tracking samples, allowing such samples to be ignored, thereby improving the position tracking process. The control circuit causes a light source to emit light according to a specified pattern and receives sensor data from multiple detectors. The control circuit may process the sensor data using, for example, machine learning or other techniques to track the position of one or more HMD components.

Claims

1. A first head-mounted display system component configured to be held or worn by the user; Multiple angle-sensing detectors are held by the first head-mounted display system component; during operation, each of the multiple angle-sensing detectors captures sensor data indicating the arrival angle of light emitted from one or more light sources; A second head-mounted display system component including one or more light sources; and To cause light to be emitted from the one or more light sources; Receiving sensor data from one or more of the aforementioned angle-sensing detectors; Processing the received sensor data, including the identification of one or more corrupted sensor data samples, each of which includes a sensor data sample from one of the angle-sensing detectors, which is identified as being highly likely not to represent light directly received by one of the angle-sensing detectors from the one or more light sources. Tracking the position of the second head-mounted display system component based at least in part on the processing of the received sensor data. Control circuits that act for A head-mounted display system equipped with [specific features / features].

2. The head-mounted display system according to claim 1, wherein the control circuit ignores the corrupted sensor data samples in order to track the position of the second head-mounted display system component.

3. The head-mounted display system according to claim 1, wherein the identification of the one or more damaged sensor data samples is at least partially based on a known geometry of at least one of the first head-mounted display system component and the second head-mounted display system component.

4. The identification of the one or more damaged sensor data samples is: The past position or orientation of at least one of the first head-mounted display system component and the second head-mounted display system component; The current position or orientation of at least one of the first head-mounted display system component and the second head-mounted display system component; or The predicted future position or orientation of at least one of the first head-mounted display system component and the second head-mounted display system component. A head-mounted display system according to claim 1, based at least in part on one or more of the above.

5. The identification of the one or more damaged sensor data samples includes determining which of the multiple angle-sensing detectors is most likely not receiving light directly from the one or more of the multiple light sources, using at least one projection model of the first head-mounted display system component or the second head-mounted display system component.

6. The head-mounted display system according to claim 5, wherein the use of at least one projection model includes comparing a received sensor data sample with the at least one projection model, and identifying any received sensor data sample that does not match the at least one projection model as a corrupted sensor data sample.

7. Identifying a received sensor data sample that does not match the at least one projection model as a corrupted sensor data sample includes identifying a received sensor data sample that does not match the at least one projection model within a defined threshold as a corrupted sensor data sample, according to claim 6.

8. The head-mounted display system according to claim 1, further comprising disabling one or more of the angle-sensing detectors associated with the corrupted sensor data samples over a number of samples or period of time.

9. The head-mounted display system according to claim 8, wherein the number of samples or the period of time is selectively varied based on the movement of at least one of the first head-mounted display system component or the second head-mounted display system component.

10. The head-mounted display system according to claim 1, wherein the second head-mounted display system component includes a wearable head-mounted display device or handheld controller on the user's head.

11. The head-mounted display system according to claim 1, wherein each of the plurality of angle-sensing detectors includes one of a photodiode detector or a position-sensing detector.

12. The head-mounted display system according to claim 1, wherein each of the plurality of angle-sensing detectors includes a photodiode detector having at least four cells.

13. The head-mounted display system according to claim 1, wherein the first head-mounted display system component includes one of a head-mounted display device, a controller, or a base station, and the second head-mounted display system component includes another of a head-mounted display device, a controller, or a base station.

14. The head-mounted display system according to claim 1, wherein at least one of the first head-mounted display system component or the second head-mounted display system component includes a component fixed in a location close to the environment in which the head-mounted display system is operated.

15. The head-mounted display system according to claim 1, further comprising a scattered light detector that captures scattered light detector data indicating whether light received by one or more of the plurality of angle-sensing detectors may have been reflected or scattered before arriving at one or more of the plurality of angle-sensing detectors, wherein the scattered light detector data is used for the identification of the one or more corrupted sensor data samples.

16. The head-mounted display system according to claim 15, wherein the control circuit processes the scattered light detector data to identify the one or more corrupted sensor data samples and ignores the corrupted sensor data samples during the tracking of the position of the second head-mounted display system component.

17. The head-mounted display system according to claim 1, further comprising a plurality of scattered light detectors, each of which captures scattered light detector data indicating whether light received by one or more of the plurality of angle-sensing detectors was reflected or scattered before arriving at one or more of the plurality of angle-sensing detectors, wherein the scattered light detector data is used for the identification of the one or more corrupted sensor data samples.

18. The head-mounted display system according to claim 1, wherein the control circuit provides the received sensor data as input to a trained machine learning model in order to process the received sensor data.

19. The head-mounted display system according to claim 1, wherein during operation, the control circuit causes the one or more light sources to emit light using multiplexing.

20. The head-mounted display system according to claim 19, wherein the multiplexing includes at least one of time multiplexing, wavelength multiplexing, frequency multiplexing, or polarization multiplexing.

21. The head-mounted display system according to any one of claims 1 to 20, wherein during operation, the control circuit causes the one or more light sources to emit light using at least one of wavelength division multiplexing (WDMA), time division multiplexing (TDMA), code division multiplexing (CDMA), or orthogonal frequency division multiplexing (OFDMA).

22. A first head-mounted display system component configured to be held or worn by the user; An optical detector held by the first head-mounted display system component; during operation, the optical detector captures sensor data indicating light emitted from one or more light sources; A second head-mounted display system component including one or more light sources; and Receiving optical detector data from the aforementioned optical detector; Processing the received optical detector data; and Adaptively adjusting the brightness of at least one of the one or more light sources based at least partially on the processed optical detector data. Control circuits that act for A head-mounted display system equipped with [specific features / features].

23. The head-mounted display system according to claim 22, wherein the control circuit adaptively adjusts the brightness of at least one of the one or more light sources based on the dynamic range of the optical detector.

24. The head-mounted display system according to claim 22, wherein the control circuit adaptively adjusts the brightness of at least one of the one or more light sources and adjusts the pulse width of the signal supplied to at least one of the one or more light sources.

25. The head-mounted display system according to claim 22, wherein the control circuit disables the one or more light sources and receives optical detector data from the optical detector while the one or more light sources are disabled.

26. The head-mounted display system according to claim 22, wherein the control circuit adaptively adjusts the brightness of at least one of the one or more light sources at a rate that varies at least in part based on the parameters of at least one component of the head-mounted display system.

27. The head-mounted display system according to claim 26, wherein the parameter includes the movement of at least one component of the head-mounted display system.

28. The head-mounted display system according to claim 22, wherein the control circuit adaptively adjusts the brightness of at least one of the one or more light sources at a rate that varies at least in part based on the amount of ambient light in the environment in which the head-mounted display system is operated.

29. The head-mounted display system according to claim 22, wherein the control circuit processes the received optical detector data to track the position of at least one of the first head-mounted display system component or the second head-mounted display system component.

30. The head-mounted display system according to claim 22, wherein the optical detector includes an angle-sensing detector that acts to detect the arrival angle of the light detected by the angle-sensing detector.

31. The head-mounted display system according to claim 22, wherein the first head-mounted display system component includes a wearable head-mounted display device or handheld controller on the user's head.

32. The head-mounted display system according to claim 22, wherein the optical detector includes one of a photodiode detector or a position-sensing detector.

33. The head-mounted display system according to claim 22, wherein the optical detector comprises a photodiode detector having at least four cells.

34. The head-mounted display system according to claim 22, wherein the first head-mounted display system component includes one of a head-mounted display device, a controller, or a base station, and the second head-mounted display system component includes another of a head-mounted display device, a controller, or a base station.

35. The head-mounted display system according to any one of claims 22 to 34, wherein the control circuit provides the received optical detector data as input to a trained machine learning model in order to process the received optical detector data.

36. A first head-mounted display system component that can be worn by the user; An angle-sensing detector held by the first head-mounted display system component, during operation, captures sensor data indicating the arrival angle of light emitted from one or more light sources, wherein the angle-sensing detector includes a plurality of sensor cells, and during operation, each sensor cell outputs a sensor cell sample representing the intensity of light incident on the sensor cell; A second head-mounted display system component including one or more light sources; and To cause the one or more light sources to emit light during the irradiation period; Receiving sensor data from the angle sensing detector, including sequentially capturing sensor cell samples from the plurality of sensor cells during the irradiation period; Processing the received sensor data, including applying calibration data to the sensor cell sample to account for the non-uniform brightness of the one or more light sources during the sequential capture of the sensor cell sample; and Tracking the position of the second head-mounted display system component based at least in part on the processing of the received sensor data. Control circuits that act for A head-mounted display system equipped with [specific features / features].

37. The head-mounted display system according to claim 36, wherein the calibration data represents the characteristic gradient of the brightness of the one or more light sources during the irradiation period.

38. The head-mounted display system according to claim 36 or 37, wherein during operation, the control circuit iteratively determines updated calibration data and uses the updated calibration data to track the position of the second head-mounted display system component.

39. To determine the updated calibration data, the control circuit: Disable the one or more light sources over the calibration period; During the calibration period, sensor cell samples are sequentially captured from the plurality of sensor cells; Interpolate the captured sensor cell sample; and Based on the interpolation of the captured sensor cell sample, the updated calibration data is determined. The head-mounted display system according to claim 38.

40. A method for calibrating a head-mounted display system component, wherein the head-mounted display system component includes an angle-sensing detector that captures sensor data indicating the angle of arrival of light emitted from one or more light sources during operation, wherein the angle-sensing detector includes a plurality of sensor cells, each sensor cell outputting a sensor cell sample representing the intensity of light incident on the sensor cell, and the method is: A step of causing the one or more light sources to emit light over an irradiation period; A step of receiving sensor data from the angle sensing detector during the irradiation period, which includes sequentially capturing sensor cell samples from the plurality of sensor cells; A step of processing the received sensor cell sample to generate calibration data that takes into account the non-uniform brightness of the one or more light sources during the irradiation period; and Steps to store the calibration data in a non-temporary processor-readable storage medium for later use in tracking at least one component of the head-mounted display system. A method for providing this.

41. The method according to claim 40, wherein the calibration data represents the characteristic gradient of the luminance on the one or more light sources during the irradiation period.

42. Processing the received sensor cell sample: Interpolating the captured sensor cell sample; and The calibration data is determined based on the interpolation of the captured sensor cell sample. The method according to claim 40 or 41, including the method described in claim 40 or 41.

43. A first head-mounted display system component that can be worn by the user; Multiple optical detectors held by the first head-mounted display system component; A second head-mounted display system component including multiple light sources; and To cause one or more of the aforementioned multiple light sources to emit light; Receiving sensor data from one or more of the aforementioned optical detectors; Tracking the position of the second head-mounted display system component based at least partially on the received sensor data; Processing the received sensor data, including determining whether to disable either the optical detector or the light source based on the determined deactivation criteria; and During the tracking of the position of the second head-mounted display system component, disable each of the optical detectors or light sources that meet the disabling criteria for the duration of their respective disabling periods. Control circuits that act for A head-mounted display system equipped with [specific features / features].

44. The head-mounted display system according to claim 43, wherein the invalidation criterion is at least partially based on the determined relative position between the first head-mounted display system component and the second head-mounted display system component.

45. The head-mounted display system according to claim 43, wherein each of the disabled periods is at least partially based on the movement of at least one of the first head-mounted display system component and the second head-mounted display system component.

46. The head-mounted display system according to claim 43, wherein, for each of the plurality of light sources of the second head-mounted display system component, the deactivation criterion provides a determination that it is unlikely that light emitted by the light source will be received by any of the optical detectors of the first head-mounted display system component.

47. The head-mounted display system according to claim 43, wherein, for each of the plurality of optical detectors of the first head-mounted display system component, the deactivation criterion provides a determination that it is unlikely that light emitted by any of the plurality of light sources of the second head-mounted display system component will be received by the optical detector.

48. The respective disabled period for each of the disabled light sources or optical detectors is: The relative position between the first head-mounted display system component and the second head-mounted display system component; or Relative movement between the first head-mounted display system component and the second head-mounted display system component The head-mounted display system according to claim 43, determined at least in part on at least one of the following.

49. The head-mounted display system according to claim 43, wherein the respective disabled period for each of the disabled light sources is determined at least partially based on the position of the light source on the second head-mounted display system component.

50. The head-mounted display system according to claim 43, wherein the respective disabled period for each of the disabled optical detectors is determined at least partially based on the position of the optical detector on the first head-mounted display system component.

51. The head-mounted display system according to claim 43, wherein the deactivation criterion provides a determination of whether it is likely that light emitted by each light source in a first subset of the plurality of light sources is received by at least one of the optical detectors of the first head-mounted display system component, and the deactivation criterion acts to deactivate a second subset of light sources in the first subset of light sources for each deactivation period.

52. The head-mounted display system according to claim 51, wherein the light sources in the second subset of disabled light sources are selected to provide light sources in the first subset of light sources that are not disabled and are relatively far apart from each other, and provide a relatively large angle of separation with respect to the optical detectors that detect the light emitted by the light sources.

53. The head-mounted display system according to any one of claims 43 to 52, wherein the invalidation criterion is at least in part based on sensor data previously received from one or more of the plurality of optical detectors.

54. A head-mounted display system, A first head-mounted display system component that can be worn by the user; An inertial measuring unit held by the first head-mounted display system component; Multiple angle-sensing optical detectors held by the first head-mounted display system component; A camera held by the first head-mounted display system component; and Receiving inertial sensor data from the aforementioned inertial measurement unit. Receiving optical sensor data from one or more of the aforementioned angle-sensing optical detectors; To receive image sensor data from the camera; Processing the received inertial sensor data, optical sensor data, and image sensor data; and Tracking the position of at least one head-mounted display system component of the head-mounted display system based at least in part on the processing of the received inertial sensor data, optical sensor data, and image sensor data. Control circuits that act for A head-mounted display system equipped with [specific features / features].

55. The head-mounted display system according to claim 54, further comprising a second head-mounted display system component including one or more light sources, wherein the control circuit causes the one or more light sources to emit light detected by one or more of the plurality of angle-sensing optical detectors during operation.

56. The head-mounted display system according to claim 55, wherein the second head-mounted display system component includes a head-mounted display device, a controller, or a base station.

57. The head-mounted display system according to claim 54, wherein the first head-mounted display system component includes a head-mounted display device that can be worn on the user's head.

58. The head-mounted display system according to claim 54, wherein each of the plurality of angle-sensing optical detectors includes one of a photodiode detector or a position-sensing detector.

59. The head-mounted display system according to claim 54, wherein each of the plurality of angle-sensing optical detectors includes a photodiode detector having at least four cells.

60. The head-mounted display system according to any one of claims 54 to 59, wherein the control circuit provides the inertial sensor data, optical sensor data, and image sensor data as input to a trained machine learning model in order to process the received inertial sensor data, optical sensor data, and image sensor data.

Images