Active reduction of fan noise in head-mounted displays

Abstract

Described herein are active noise reduction (ANR) techniques for reducing noise generated by a fan of a head-mounted display (HMD). An example process may include receiving data indicative of noise being generated by the fan, determining one or more audio parameter values ​​based at least in part on the data and using a model, and outputting sound having one or more audio characteristics based at least in part on the one or more audio parameter values ​​via one or more extra-aural speakers of the HMD to reduce the noise generated by the fan at ears of a user of the HMD.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Patent Application No. 17 / 941,363, filed September 9, 2022. Application No. 17 / 941,363 is incorporated herein by reference in its entirety. [Background technology]

[0002] Head-mounted displays (HMDs), such as virtual reality (VR) headsets, are used both inside and outside the video game industry. Some users run their headsets to their maximum performance, such as by throttling the central processing unit (CPU) and graphics processing unit (GPU), to achieve a high-fidelity experience. When electronic components are running at or near maximum performance, they tend to heat up. Many VR headsets include fans to cool the electronic components when they get too hot, thereby preventing them from overheating.

[0003] In VR headsets with out-of-ear speakers, noise generated by the headset's fan can be annoying to the headset user. Furthermore, fan noise can interfere with the user's ability to hear audio content being output through the out-of-ear speakers, such as the sound of a video game being played by the user. For example, when the headset's fan is running, the fan may generate a humming, buzzing, or whirring sound that is audible to the headset user through the headset's out-of-ear speakers.

[0004] Technical solutions are provided herein to improve and enhance these and other systems. [Brief explanation of the drawings]

[0005] DETAILED DESCRIPTION OF THE INVENTION The detailed description is described with reference to the accompanying drawings. In the drawings, the leftmost digit(s) of a reference number identifies the figure in which the reference number first appears. Use of the same reference number in different figures indicates similar or identical components or features.

[0006] [Figure 1] 1 illustrates an example HMD system that can implement the techniques disclosed herein to actively reduce noise generated by the HMD's fan, according to embodiments disclosed herein. [Figure 2] FIG. 1 is a flow diagram of an exemplary process for actively reducing noise generated by a fan of an HMD, according to embodiments disclosed herein. [Figure 3] 10A-10C illustrate an example technique for using microphones of an HMD to actively reduce noise-generating fans of the HMD for at least one of the microphones. [Figure 4] FIG. 1 is a flow diagram of an exemplary process for actively reducing noise generated by a fan of an HMD in an energy-efficient manner, according to embodiments disclosed herein. [Figure 5] 1 illustrates example components of an HMD system capable of implementing the techniques disclosed herein, according to embodiments disclosed herein. DETAILED DESCRIPTION OF THE INVENTION

[0007] When a user of an HMD desires a high-fidelity experience, the user may configure the HMD to run certain electronic components (e.g., CPU, GPU, etc.) at their maximum performance, or the HMD may be a high-performance HMD that always defaults to running electronic components at their maximum performance. Such an HMD may enable a user to play “intense” video games with high-fidelity graphics, low latency, and / or other optimized performance parameters. As mentioned above, when the electronic components of an HMD run at or near their maximum performance, they tend to generate heat, which causes the electronic components to become hot. A temperature sensor in the HMD may sense the temperature of an electronic component that is overheating, which may activate and switch on or otherwise operate the HMD's fan, so that the fan begins to cool the overheated electronic component. The fan may prevent the electronic component from overheating and shutting down and / or causing user discomfort. However, the fan also generates noise as a byproduct. In the case of an HMD's out-of-ear speakers, this fan noise can be annoying to the user of the HMD and can disrupt the user's experience, such as by making it difficult for the user to hear desired sounds being output by the out-of-ear speakers (e.g., sounds of a video game being played by the user through the HMD).

[0008] Described herein are, among other things, active noise reduction (ANR) techniques, as well as devices and systems for implementing techniques for reducing noise generated by a fan of an HMD at the ear location of a user of the HMD. While "ANR" is the term used throughout this disclosure, it should be understood that alternative terminology, such as "active noise control (ANC)," "noise cancellation (NC)," "noise control," and / or "noise reduction," may be used interchangeably with the ANR terminology used herein. Furthermore, it should be understood that the ANR techniques described herein are at least partially "active" (e.g., using a power source for noise reduction), as opposed to fully passive noise reduction techniques that use noise-isolating materials (e.g., insulation) rather than a power source. Nevertheless, passive noise reduction techniques may be used in combination with the ANR techniques described herein.

[0009] When implemented on an HMD with one or more extra-aural speakers, the ANR techniques described herein improve the audio experience for a user of the HMD by reducing fan noise at the user's ears. To illustrate, an HMD may be worn by a user for the purpose of immersing the user in a VR or augmented reality (AR) environment. An application (e.g., a video game) may be executed on the HMD or on a host computer of an HMD system that includes the HMD. As it executes, the application generates pixel data and audio data for output on the HMD. The pixel data is used to display the video content of the running application as a series of frames on the HMD, and the audio data is used to output sound corresponding to the audio content of the running application via the HMD's speakers.

[0010] Speakers in HMDs include "out-of-ear" speakers (sometimes called "open-ear" speakers), meaning that the speakers do not cover the user's ears. Instead, out-of-ear speakers leave the user's ears uncovered, for example, by spacing the speakers away from the ears at a distance that allows the user to hear both sounds corresponding to the audio content of a running application and sounds around the user. VR headset users tend to enjoy out-of-ear speakers to better experience VR game sounds reaching the ears from different directions. This effect is often lost when speakers are implemented as on-ear speakers or earphones. Out-of-ear speakers in HMDs also allow for the preservation of some of the depth cues / signals that the human ear is designed to pick up, thereby providing a more immersive audio experience than can be provided by on-ear speakers or earphones. However, the use of out-of-ear speakers also means that the user can still hear sounds around them, including noise from the HMD's fan whenever the fan is operating to cool the HMD's electronic components. In particular, the techniques, devices, and systems described herein are tailored to reduce the specific noise generated by the fan of an HMD without reducing other noises around the user. In other words, the techniques described herein allow for preserving ambient noise at the user's ears while isolating and reducing the specific noise being generated by the fan of an HMD.

[0011] In an exemplary ANR process, a processor of an HMD system may receive data indicative of noise being generated by a fan of the HMD. The data received by the processor may be any suitable type of data that can be used as a proxy for determining characteristics of the fan noise at the ear location of a user of the HMD, as described in more detail below. Based at least in part on the received data and using a model, the processor may determine one or more audio parameter values, such as a frequency parameter value, a phase parameter value, and / or an amplitude parameter value. The determined audio parameter values ​​may then be used to output sound having one or more audio characteristics (e.g., frequency, phase, and / or amplitude, etc.) via an extra-aural speaker of the HMD to reduce the noise generated by the fan at the ear location of the user of the HMD.

[0012] The ANR techniques described herein provide a more enjoyable, higher-fidelity audio experience for users of HMDs with out-of-ear speakers by reducing, if not eliminating, fan noise at the user's ears, which may allow users to throttle the performance of their HMD components to their maximum performance without hearing the noise generated by the fans used to cool the HMD's electronic components (i.e., the fans are barely audible, if not audible, to the HMD user).

[0013] Also disclosed herein are techniques for actively reducing fan noise in an energy-efficient manner, which conserves power resources in an already power-strapped HMD and / or HMD system. For example, the ANR algorithm described herein may be executed as needed, such as in response to determining that the fan is drawing power from the HMD's power source (e.g., a battery) and / or in response to determining that the fan is generating noise at a level that warrants noise reduction. In other words, there may be a level of fan noise that is acceptable to most HMD users, such as a low level of fan noise. Thus, as long as the fan is operating at or below a level that is acceptable to most users, which may be a configurable level, power resources can be conserved even when the fan is running. In another example, power conservation techniques may include determining whether and when to "open" an HMD's microphone to perform ANR with improved accuracy and reserving the use of the HMD's microphone for use in ANR for those cases where the benefits of more accurate noise reduction outweigh the power consumption costs resulting from using the microphone for ANR.

[0014] Also disclosed herein are systems including an HMD configured to implement the techniques and processes disclosed herein, and a non-transitory computer-readable medium storing computer-executable instructions for implementing the techniques and processes disclosed herein. While the techniques and systems disclosed herein are often discussed in the context of video game applications, particularly VR game applications, by way of example, it should be understood that the techniques and systems described herein may provide advantages to other applications, including, but not limited to, non-VR applications (e.g., AR applications) and / or non-gaming applications such as industrial machinery applications, defense applications, robotics applications, etc.

[0015] 1 illustrates an example HMD system 100 capable of implementing the techniques disclosed herein to actively reduce noise generated by a fan of an HMD 102, according to embodiments disclosed herein. In some examples, the HMD system 100 may include a standalone HMD 102 (sometimes referred to as an “all-in-one” HMD 102) that includes all of the components described herein and is operable without or with minimal assistance from a separate computer. In other examples, the HMD system 100 is a distributed system that includes the HMD 102 and at least one additional computer that is separate from but communicatively coupled to the HMD 102. For example, the HMD system 100 may include a host computer 104 communicatively coupled to the HMD 102. In some examples, the host computer 104 may be co-located in the same environment as the HMD 102, such as in the household of a user 106 wearing the HMD 102. The host computer 104 and the HMD 102 may be communicatively coupled together via wireless and / or wired connections. For example, the devices 102 / 104 may exchange data using Wi-Fi, Bluetooth, radio frequency (RF), and / or any other suitable wireless protocol. Additionally or alternatively, the devices 102 / 104 may include one or more physical ports to facilitate a wired connection (e.g., a tether, cable, etc.) for data transfer therebetween. In some examples, the HMD system 100 may include a remote system 108 in addition to or instead of the host computer 104 located within the environment of the HMD 102. The remote system 108 may be communicatively coupled to the host computer 104 and / or the HMD 102 via a wide area network 110, such as the Internet. Thus, the remote system 108 may represent one or more server computers located at one or more remote geographic locations relative to the geographic location of the HMD 102.In other examples, the network 110 may represent a local area network (LAN), and the remote system 108 is considered to be remote from the HMD 102, although the remote system 108 may be located, for example, within the same building as the HMD 102.

[0016] The HMD 102 and the host computer 104 and / or the remote system 108 collectively represent a distributed system for executing an application 112 (e.g., a video game) to render associated video content (e.g., a series of images) on the display of the HMD 102 and / or to output sounds corresponding to the running application's audio content via one or more extra-aural speakers 114 of the HMD 102. By being communicatively coupled together, the HMD 102 and the host computer 104 and / or the remote system 108 may be configured to work together in a cooperative manner to output such video and / or audio content via the HMD 102. Accordingly, at least some of the components, programs, and / or data described herein, such as the processor 116, the application 112, etc., may reside on the host computer 104 and / or the remote system 108. Alternatively, as mentioned above, the components, programs, and / or data may reside entirely on the HMD 102, such as within a standalone HMD 102. The host computer 104 and / or the remote system 108 may be implemented as any type and / or number of computing devices, including, but not limited to, a personal computer (PC), a laptop computer, a desktop computer, a portable digital assistant (PDA), a mobile phone, a tablet computer, a set-top box, a game console, a server computer, a wearable computer (e.g., a smart watch, etc.), or any other electronic device capable of transmitting and receiving data.

[0017] In some examples, the HMD 102 may represent a VR headset for use in a VR system, such as for use with a VR gaming system. However, the HMD 102 may additionally or alternatively be implemented as an AR headset for use in AR applications or a headset usable for non-gaming-related VR and / or AR applications (e.g., industrial applications). In AR, the user 106 sees virtual objects superimposed on the real-world environment, whereas in VR, the user 106 typically does not see the real-world environment but is fully immersed in the virtual environment as perceived through the display panel and optical elements (e.g., lenses) of the HMD 102. It should be understood that in some VR systems, a pass-through image of the user's 106 real-world environment may be displayed in conjunction with the virtual image to create an augmented VR environment within the VR system, whereby the VR environment is augmented with the real-world image (e.g., overlaid on the virtual world). While the examples described herein primarily relate to a VR-based HMD 102, it should be understood that the HMD 102 is not limited to implementation in VR applications.

[0018] 1 depicts the HMD 102 as including various components, but as noted, at least some of the components, programs, and / or data may reside elsewhere (e.g., on the host computer 104 and / or the remote system 108). The components may include, without limitation, an in-ear speaker 114, one or more fans 118, a driver circuit 120, one or more sensors 122, such as one or more temperature sensors 124, one or more microphones 126, one or more power supplies 128, one or more processors 116, such as one or more CPUs 130 and / or one or more GPUs 132, a memory 134 that stores, among other things, an application 112, an ANR component 136, and / or a model 138.

[0019] The fan 118 may generate noise 140 during operation of the fan 118. This noise 140 may be heard by the user 106 through the "out-of-ear" speakers 114 of the HMD 102 while the application 112 is running (e.g., while a video game is being played). In the example disclosed herein, the HMD 102 includes a pair of out-of-ear speakers 114, including a first (e.g., left) out-of-ear speaker 114(1) and a second (e.g., right) out-of-ear speaker 114(2). For example, the first (e.g., left) out-of-ear speaker 114(1) may be located on the left side of the HMD 102, and the second (e.g., right) out-of-ear speaker 114(2) may be located on the right side of the HMD 102 (e.g., from the perspective of the user 106 wearing the HMD 102). Both out-of-ear speakers 114(1) and 114(2) are shown in FIG. 3 , whereas FIG. 1 illustrates a close-up view of the second (e.g., right) out-of-ear speaker 114(2) of HMD 102. As the name implies, the “out-of-ear” speakers 114 do not cover the ears of the user 106. Instead, the out-of-ear speakers 114 may be spaced a respective distance from the ears of the user 106 and remain uncovered. As mentioned, this allows the user 106 to hear both sounds corresponding to the audio content of the running application 112 and sounds around the user 106, and the user 106 may be able to adjust the distance the out-of-ear speakers 114 are spaced from their ears for a desired audio experience. However, without the ANR techniques described herein, the use of the extra-aural speakers 114 means that the user 106 may hear noise 140 from the fan 118 whenever the fan 118 is running to cool the electronic components of the HMD 102. As used herein, it should be understood that an "extra-aural" speaker refers to a speaker (or an arrangement of speakers) that does not cover the ear of the user / listener. Thus, the extra-aural speakers may be spaced (e.g., offset) away from the ear by a distance such that the speaker is close but does not touch or cover the ear.As another example, the extra-ear speakers may be disposed within (or secured to) the headband of the HMD 102 and / or may be disposed within or on (e.g., inside) the main housing of the HMD 102. Thus, it should be understood that, as used herein, "extra-ear" speakers are not limited to speakers that are in close proximity to the ear. Such extra-ear speakers may be positioned at any suitable distance from and / or in any suitable position relative to the ear, so long as the ear is not covered by the speaker and so long as the speaker does not plug into the ear canal.

[0020] The ANR component 136 may represent computer-executable instructions that, when executed by the processor 116, cause the performance of the operations and techniques described herein, such as the execution of an ANR algorithm to reduce fan noise 140 at the ear of the user 106. For example, the ANR algorithm may cause the one or more extra-aural speakers 114 to output sound 142 having one or more audio characteristics (e.g., frequency, phase, amplitude, etc.) to reduce the noise 140 generated by the fan 118 at the ear of the user 106 of the HMD 102.

[0021] For example, when the ANR component 136 is executed by the processor 116, the processor 116 may receive data indicative of the noise 140 being generated by the fan 118. The data received by the processor 116 may be any suitable type of data usable as a proxy for determining characteristics of the fan noise 140 at the ear location of the user 106 of the HMD 102. In some examples, the data received by the processor 116 includes audio data representing the noise 140 captured by the microphone 126 of the HMD 102. In some examples, the data received by the processor 116 may additionally or alternatively include non-audio data, such as temperature data indicative of a temperature sensed by the temperature sensor 124 of the HMD 102, drive data indicative of the level at which the fan 118 is being driven via the drive circuitry 120 of the HMD 102, utilization data indicative of utilization of the CPU 130 and / or GPU 132, and / or any other suitable type of data, such as vibration data indicative of vibrations generated by the fan as detected by an accelerometer, operational data indicative of the operating speed of the fan 118 as determined by a tachometer, etc. These are just example types of data that may be indicative of the noise 140 being produced by the fan 118 .

[0022] For example, when an electronic component (e.g., CPU 130) starts to heat up and temperature sensor 124 senses a high temperature of the electronic component, this may trigger fan 118 to start operating to cool the electronic component. Thus, the high temperature sensed by temperature sensor 124 may indicate, for example, that fan 118 is operating and therefore making noise 140. As another example, if fan 118 is being driven at a certain level, and drive data indicative of this drive level is received by processor 116, such drive data may indicate that fan 118 is operating and therefore making noise 140. As yet another example, if utilization of CPU 130 and / or GPU 132 suddenly increases, this may indicate that CPU 130 and / or GPU 132 are running at or near maximum performance, which may indicate that CPU 130 and / or GPU 132 are overheating or about to overheat, which may in turn indicate that fan 118 is operating to cool the electronic component and therefore making noise 140.

[0023] The model 138 stored in memory 134 may be referenced and used at runtime to determine, based on the received data, one or more audio parameter values ​​that may be used to output a noise-reduced sound 142 having one or more audio characteristics (e.g., frequency, phase, amplitude, etc.) to reduce the noise 140 generated by the fan 118 at the ear location of the user 106 of the HMD 102. Such a model 138 may represent any suitable type of model, such as a mathematical model, a statistical model, a weight model, a trained machine learning model, etc. Thus, the model 138 may be generated in various ways depending on the type of model 138.

[0024] Any given fan 118 can produce a tolerable range of noise depending on the level at which the fan 118 is driven, as well as due to inherent differences from one fan 118 to another. With this understanding, testing can be performed by operating various fans 118 throughout their respective operating ranges (e.g., from minimum input current to maximum input current), capturing the corresponding fan noise 140 using microphones positioned at locations representative of where the user's 106's ears would be while wearing the HMD 102, and recording the audio characteristics of the fan noise 140. This heuristic testing can be performed iteratively to calibrate for a range of noises generated by the fan 118, as well as for a range of head sizes and shapes (as head sizes and shapes vary across a user population), a range of different locations of the fan 118 relative to the microphone, and / or a range of test environments with different levels of ambient / background noise. During the course of such testing, the recorded audio characteristics of the fan noise 140 can be plotted as a function of the level (e.g., input current) at which the fan 118 is driven. For example, the X-axis of the graph may represent the drive level (e.g., number of milliamps (mA) of input current) of the fan 118, ranging from zero (e.g., the fan 118 is off) to N (e.g., the fan 118 is driven at its maximum level). Meanwhile, one or more audio characteristics of the fan noise 140 may be plotted on the Y-axis of the graph, where the audio characteristics include, but are not limited to, frequency, amplitude, phase, etc. In some examples, average values ​​and / or other statistics may be calculated as part of this testing process, such as the average phase of the fan noise 140 recorded where the user's 106's ear is located relative to the fan 118. In some examples, the final audio parameter values ​​derived from the testing may compensate for the latency of the microphone used to mimic the user's 106's ear during the testing, thereby capturing the audio characteristics of the fan noise 140 as it would sound to a human ear (as opposed to the noise 140 being "heard" by the microphone).Additionally, application 112 may be executed by HMD system 100 under test while test data is collected and recorded, including, for example, the temperature data, drive data, utilization data, etc., mentioned above. This collected data may also be plotted as a function of drive level of fan 118. A model 138 may be generated based on such testing, and model 138 may be used to determine, based on one or more types of input data, audio parameter values ​​(e.g., frequency parameter values, phase parameter values, amplitude parameter values, tone parameter values, pitch parameter values, signal-to-noise ratio (SNR) parameter values, etc.) that may be used to generate noise-reducing sound 142 to reduce fan noise 140 at the ear location of user 106.

[0025] In examples where the model 138 is a trained machine learning model, test data collected during the testing described above (e.g., by running the fan 118 of the HMD 102 over its / their operating range and recording various types of test data as the fan 118 is running) may be used as training data to train the machine learning model 138. Additionally, or alternatively, data collected by the HMD 102, when used by the end user 106, may be used as training data to train the machine learning model 138. Such a trained machine learning model 138 may be configured to output a classification of the noise-reduced signal (e.g., in terms of its audio parameter values, such as frequency, amplitude, phase, etc.) for use in generating the noise-reduced sound 142. The trained machine learning model 138 used for the reduction of the fan noise 140 may represent a single model or an ensemble of base-level machine learning models and may be implemented as any type of machine learning model. For example, machine learning models suitable for use with the techniques and systems described herein include neural networks (e.g., deep neural networks (DNNs), recurrent neural networks (RNNs), etc.), tree-based models, support vector machines (SVMs), kernel methods, random forests, splines (e.g., multivariate adaptive regression splines), hidden Markov models (HMMs), Kalman filters (or extended Kalman filters), Bayesian networks (or Bayesian belief networks), multilayer perceptrons (MLPs), expectation maximization, genetic algorithms, linear regression algorithms, nonlinear regression algorithms, logistic regression-based classification models, or ensembles thereof. An "ensemble" can include a collection of machine learning models whose outputs (predictions) are combined, such as by using a weighted average or voting. Individual machine learning models in an ensemble can differ in their expertise, and an ensemble can operate as a committee of individual machine learning models that are collectively "smarter" than the individual machine learning models in the ensemble.Furthermore, the machine learning model may be trained using any suitable learning technique, such as supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, etc.

[0026] Once the model 138 is generated and accessible to the HMD system 100 (e.g., stored in its memory 134), the model 138 can be used during run time to reduce the fan noise 140. That is, the audio parameter values ​​(determined based on the received data and using the model 138) can be used to generate a noise-reducing signal that causes a noise-reducing sound 142 to be output through one or more extra-aural speakers 114 of the HMD 102, which, if not eliminated, reduces the fan noise 140 at the ears of the user 106. For example, in the example HMD 102 described herein, the first (e.g., left) extra-aural speaker 114(1) may output a first sound 142 having one or more first audio characteristics (e.g., frequency, amplitude, phase, etc.) based at least in part on the determined audio parameter values, and the second (e.g., right) extra-aural speaker 114(2) may output a second sound 142 having one or more second audio characteristics (e.g., frequency, amplitude, phase, etc.) based at least in part on the determined audio parameter values. In this manner, the first sound 142 may reduce the noise 140 generated by the fan 118 at the location of the first (e.g., left) ear of the user 106, and the second sound 142 may reduce the noise 140 generated by the fan 118 at the location of the second (e.g., right) ear of the user 106. The noise reducing sound 142 may have the same amplitude as the amplitude of the fan noise 140, but may have an inverted phase (e.g., due to a phase shift) and / or a nulling frequency to reduce the unwanted fan noise 140. In some examples, the noise reducing sound 142 may be represented by a sound wave having the same or directly proportional amplitude and opposite polarity (e.g., a reverse polarity waveform) as the sound wave of the fan noise 140. The noise reducing sound 142 described herein may be represented by a sound wave having any suitable audio characteristics that disruptively interfere with the sound wave of the fan noise 140 at the location of the user's 106's ear to reduce or attenuate the noise 140 generated by the fan 118 in the vicinity of the user's ear.

[0027] In some examples, the first sound 142 output through the first (e.g., left) extra-aural speaker 114(1) has different audio characteristics (e.g., different phase, different frequency, different amplitude, etc.) than the audio characteristics of the second sound 142 output through the second (e.g., right) extra-aural speaker 114(2). This may be because, for example, due to asymmetry of the HMD 102 (e.g., the fan 118 may be located on one side (e.g., the left or right) of the HMD 102, the tolerances of the extra-aural speakers 114(1) and 114(2) may be different, etc.), the fan noise 140 may have different audio characteristics at the left and right ears of the user 106, thereby creating a noise 140 at the left ear that is different from the noise 140 at the right ear (i.e., a noise 140 with different audio characteristics). In other examples, each extra-aural speaker 114 may output the same noise-reduced sound 142 with the same audio characteristics.

[0028] It should be understood that the noise-reducing sound 142 configured to reduce the fan noise 140 may be output by the extra-aural speaker 114 while the speaker 114 is also outputting sound corresponding to the audio content of the running application 112. For example, the application 112 may represent a video game, and the user 106 may be playing the video game using the HMD 102, such that the noise-reducing sound 142 may be output simultaneously with the sound of the video game. In particular, the noise-reducing sound 142 does not refer to reducing sound corresponding to the audio content of the running application 112. Rather, the noise-reducing sound 142 refers to reducing the particular noise 140 being generated by the fan 118 of the HMD 102.

[0029] The processes described herein are illustrated as a collection of blocks in a logic flow graph, which represent a sequence of operations that can be implemented in hardware, software, firmware, or a combination thereof (i.e., logic). In the software context, the blocks represent computer-executable instructions, which, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as limiting, and any some of the described blocks can be combined in any order and / or in parallel to implement a process.

[0030] 2 is a flow diagram of an example process 200 for actively reducing noise 140 generated by a fan 118 of an HMD 102, according to an embodiment disclosed herein. For discussion purposes, the process 200 is described with reference to the previous figure. Furthermore, the process 200 may be implemented by the HMD system 100 described herein.

[0031] At 202, the processor 116 of the HMD system 100 may receive data indicative of noise 140 being generated by the fan 118 of the HMD 102 of the HMD system 100. In some examples, the data received by the processor 116 at block 202 includes audio data representing the fan noise 140 captured by the microphone 126 of the HMD 102. In such examples, the audio data received at block 202 may be received from the microphone 126 and / or from other components that process the sound captured by the microphone 126. However, it should be understood that the data received at block 202 may include non-audio data in combination with the audio data received from the microphone 126 or without such microphone-generated audio data. In examples where the data received at block 202 includes non-audio data, the non-audio data may be received from one or more other components of the HMD 102 (or HMD system 100) other than the microphone 126. For example, in block 202, temperature data may be received from the temperature sensor 124 of the HMD 102, drive data may be received from the drive circuit 120 of the HMD 102, and / or utilization data may be received from the processor 116.

[0032] There may be costs and benefits to using the microphone 126 of the HMD 102 for ANR. Therefore, whether or not to use the microphone 126 of the HMD 102 for ANR may depend on whether the costs of using the microphone 126 for ANR outweigh the benefits of doing so. For example, using audio data representing the fan noise 140 captured by the microphone 126 may enable the noise 140 to be reduced with improved accuracy compared to not using the microphone 126 for ANR. For example, using the microphone 126 to actively reduce the fan noise 140 may achieve measurably better noise reduction than performing ANR without using the microphone 126 (e.g., using non-audio data received from one or more other components). However, it also requires power to operate the microphone 126 for ANR. Thus, in some scenarios, for example, if using the microphone 126 for ANR results in a slight (e.g., 10% or less) improvement in noise reduction performance compared to using non-audio data received from another component for ANR, the extra power to operate the microphone 126 for ANR may not be worthwhile. Therefore, it should be understood that in some examples, the data received in block 202 may exclude audio data. In these examples, the data received in block 202 may include, without limitation, the previously mentioned temperature data indicating the temperature sensed by the temperature sensor 124 of the HMD 102, drive data indicating the level at which the fan 118 is being driven via the drive circuit 120 of the HMD 102, and / or utilization data indicating utilization of the CPU 130 and / or GPU 132. These are merely example types of data that may be indicative of the noise 140 being generated by the fan 118, and other types of data indicative of fan noise 140 may be received in block 202.

[0033] At 204, the processor 116 may determine one or more audio parameter values ​​based at least in part on the data received at block 202 and using the model 138. For example, the processor 116 may determine audio parameter values ​​such as, but not limited to, a frequency parameter value, a phase parameter value, an amplitude parameter value, etc. The model 138 used by the processor 116 to make this determination at block 204 may include, but is not limited to, a mathematical model, a statistical model, a weighted model, a trained machine learning model, etc. These types of models 138 may be generated based on previously collected data associated with operating the fan 118 of the HMD 102 while the HMD system 100 is executing the application 112, as described above. For example, an HMD 102 with a fan 118 can be used as a test device during heuristic testing to determine data (e.g., audio data, temperature data, drive data, utilization data, etc.) indicative of the noise 140 being generated by the fan 118 over its operating range (or a portion thereof), and the resulting mapping between the input data and the audio characteristics of the fan noise 140 can be reflected in the model 138, such that the processor 116 can determine audio parameter values ​​for the noise-reduced signal from the new data received in block 202 and using the model 138. As another example, the machine learning model 138 can be trained using previously collected data associated with the operation of the fan 118 of the HMD 102 while the HMD system 100 is executing the application 112, and the trained machine learning model 138 can be used in block 204 to output audio parameter values ​​as a classification of the noise-reduced signal (e.g., a classification in terms of audio parameter values ​​of the noise-reduced signal, such as frequency, amplitude, phase, etc.).

[0034] At 206, a sound 142 having one or more audio characteristics based at least in part on the audio parameter values ​​determined in block 204 may be output via the extra-aural speaker 114 of the HMD 102 to reduce the noise 140 generated by the fan 118 at the ear location of the user 106 of the HMD 102. In an implementation of an HMD 102 having multiple extra-aural speakers 114, such as a first (e.g., left) extra-aural speaker 114(1) and a second (e.g., right) extra-aural speaker 114(2), outputting in block 206 may include outputting a first sound 142 having one or more first audio characteristics for reducing the noise 140 generated by the fan 118 at the location of the first (e.g., left) ear of the user 106 via the first (e.g., left) extra-aural speaker 114(1) in block 208, and outputting a second sound 142 having one or more second audio characteristics for reducing the noise 140 generated by the fan 118 at the location of the second (e.g., right) ear of the user 106 via the second (e.g., right) extra-aural speaker 114(2) in block 210. In some examples, the audio characteristics of the sound 142 output in block 206 may include, without limitation, the frequency of the sound 142 (or sound wave), the amplitude of the sound 142 (or sound wave), or the phase of the sound 142 (or sound wave), among other possible audio characteristics. In some examples, a first audio characteristic of a first sound 142 output by a first (e.g., left) extra-aural speaker 114(1) may differ from a second audio characteristic of a second sound 142 output by a second (e.g., right) extra-aural speaker 114(2). In some examples, the noise-reducing sound 142 output in block 206 may be represented by sound waves having any suitable audio characteristics that destructively interfere with the sound waves of the fan noise 140 at the location of the user's 106's ear to reduce or attenuate the noise 140 generated by the fan 118 near the user's ear, as described herein.

[0035] 3 is a diagram illustrating an example technique for using the microphone 126 of the HMD 102 to actively reduce noise generated by the fan 118 (i.e., a source of unwanted noise 140) of the HMD 102. FIG. 3 depicts a schematic representation of the HMD 102 as viewed by a user 106 wearing the HMD 102 on their face (i.e., looking at the front of the HMD 102). Accordingly, the HMD 102 is shown as having a pair of extra-aural speakers 114, including a first (e.g., left) extra-aural speaker 114(1) and a second (e.g., right) extra-aural speaker 114(2). The HMD 102 is also shown as having a pair of microphones 126, including a first (e.g., left) microphone 126(1) and a second (e.g., right) microphone 126(2). The first extra-aural speaker 114(1) and the first microphone 126(1) may be located on the left side of the HMD 102, and the second extra-aural speaker 114(2) and the second microphone 126(2) may be located on the right side of the HMD 102. For example, the first microphone 126(1) may be positioned at or near the left cheekbone of the user 106 when the user 106 is wearing the HMD 102, and the second microphone 126(2) may be positioned at or near the right cheekbone of the user 106 when the user 106 is wearing the HMD 102. It should be understood that these are merely example locations for the microphones 126(1) and 126(2), and that the microphones 126 may be located elsewhere on the HMD 102. It should also be understood that additional microphones 126 may be included in HMD 102 for a microphone array having several microphones 126 that is larger than two microphones 126(1) and 126(2). The pair of microphones 126(1) and 126(2) shown in FIG. 3 may be used as a dual microphone array or a linear array of microphones 126 configured to capture audio data around them. In some examples, each microphone 126 is an omnidirectional microphone, meaning that the microphone 126 responds equally to sound coming from any direction.

[0036] While one use of the microphone 126 of the HMD 102 can be to capture the sound of the user 106's voice (e.g., to enable the user 106 to issue voice commands and / or speak with other users wearing other HMDs, etc.), the microphone 126 of the HMD 102 can also be configured to generate audio data representing the noise 140 being generated by the fan 118 of the HMD 102, for use of the audio data in ANR, as described herein. In the example of FIG. 3 , first audio data representing the noise 140 captured by the first microphone 126(1) can be generated when the fan 118 generates the noise 140. Additionally, second audio data representing the noise 140 captured by the second microphone 126(2) can be generated when the fan 118 generates the noise 140. The first audio data (associated with the first microphone 126(1)) and the second audio data (associated with the second microphone 126(2)) may exhibit a phase difference during the time interval in which the microphones 126(1) and 126(2) receive or capture the fan noise 140. This phase difference can be used to find the direction of the noise source, which in this case is the fan 118. Thus, the microphones 126(1) and 126(2) of the HMD 102 can be used as a phased array to localize the source of the unwanted fan noise 140.

[0037] 3 illustrates a first instance 300(1) of noise 140 generated by fan 118 as detected by first microphone 126(1) and a second instance 300(2) of noise 140 generated by fan 118 as detected by second microphone 126(2). Fan noise 140 may result in different audio data generated by each microphone 126 due to the direction and / or location of fan 118 relative to each microphone 126. In some examples, the direction and / or location of fan 118 relative to each microphone 126 can be described in terms of a relative azimuth angle 302 and a relative elevation angle 304 from either microphone 126(1) or 126(2) or from another location (e.g., from the midpoint between microphones 126(1) and 126(2)). As described herein, the relative orientation and / or location of fan 118 from each of microphones 126 of HMD 102 may define audio parameter values ​​of the noise-reduced signal. That is, model 138 may include or otherwise reflect the orientation and / or relative location of fan 118 of HMD 102 with respect to microphones 126, as well as the audio parameter values ​​(e.g., frequency parameter values, phase parameter values, and / or amplitude parameter values, etc.) determined in block 204 of process 200—which may be used to output noise-reduced sound 142 via extra-aural speakers 114(1) and 114(2)—which may vary depending on the orientation and / or location of fan 118 with respect to first microphone 126(1) and / or second microphone 126(2). Thus, in some examples, the orientation of fan 118 may be factored into the determination of the audio parameter values ​​in block 204 of process 200.

[0038] It should also be understood that the HMD 102 may include multiple fans 118, and in such examples, the respective directions for each individual fan of the multiple fans 118 may be factored into the ANR techniques described herein. In such examples, each fan 118 may be treated as an independent source of unwanted noise, and the ANR techniques described herein may be implemented independently with respect to each individual fan of the fans 118. Alternatively, the model 138 may be used to output a noise-reducing sound 142 when a coordinated effort to reduce multiple different noises 140 is being generated by multiple different fans 118 of the HMD 102. In some examples, the multiple fans 118 may be located on the head strap 107 of the HMD 102, such as at the rear of the head strap 107.

[0039] 3 illustrates a third instance 302(1) of noise 140 generated by fan 118 when sounding at the location of a first (e.g., left) ear of user 106 near first (e.g., left) extra-aural speaker 114(1) and a fourth instance 302(2) of noise 140 generated by fan 118 when sounding at the location of a second (e.g., right) ear of user 106 near second (e.g., right) extra-aural speaker 114(2). In particular, third instance 302(1) of fan noise 140 is different from first instance 300(1) of fan noise 140. This may be due to first microphone 126(1) being located differently from the first (e.g., left) ear of user 106. Similarly, the fourth instance 302(1) of fan noise 140 differs from the second instance 300(2) of fan noise 140, which may be due to the second microphone 126(2) being at a different location than the second (e.g., right) ear of the user 106. As described above, unit-by-unit tests can be performed to characterize how the fan noise 140 sounds at both ear locations based on audio data captured by the microphone 126, which may not be co-located with the ears, and the results of such tests can be used to generate the model 138. At runtime, the model 138 can be used to infer audio parameter values ​​for a noise reduction signal based on the audio data captured by the microphone 126 to reduce the undesirable fan noise 140 at the user's ear locations.

[0040] In some examples, the HMD 102 may include a microphone 126(3), such as microphone 126(3), disposed inside the HMD 102 (e.g., within the main housing of the HMD 102). Accordingly, FIG. 3 illustrates another instance 300(3) of noise 140 generated by fan 118 being detected by microphone 126(3). In some examples, the HMD 102 includes microphone 126(3) instead of including microphones 126(1) and 126(2). In other examples, the HMD 102 includes a third microphone 126(3) in addition to microphones 126(1) and 126(2). The microphone 126(3) may be strategically positioned within the HMD 102 to capture the fan noise 140. In some examples, microphone 126(3) is dedicated to capturing fan noise 140, as opposed to having dual functionality as a microphone that can be used to capture user voice and fan noise 140, as described above with respect to microphones 126(1) and 126(2). In some examples, microphone 126(3) is positioned in close proximity to (e.g., within a threshold distance from) fan 118 (e.g., a few millimeters, centimeters, etc. from fan 118) to optimize the capture of fan noise 140 by microphone 126(3). In either case, at runtime, model 138 can be used to infer audio parameter values ​​for a noise reduction signal based on the audio data captured by microphone 126(3) to reduce undesirable fan noise 140 at the user's ear locations.

[0041] In some examples, the ANR techniques described herein may utilize any suitable feedback and / or feedforward noise reduction techniques. Feedback noise reduction techniques may refer, for example, to techniques using one or more feedback sensors (e.g., the microphone 126 of the HMD 102, one or more additional microphones of the HMD 102 positioned at or near the extra-aural speaker 114, etc.) to provide feedback for determining the effectiveness of the ANR algorithm. Feedforward noise reduction techniques may refer, for example, to adaptive or dynamic equalization techniques. In some examples, audio data representing the fan noise 140 captured by the microphone 126 of the HMD 102 may be decomposed into frequency space, such as by computing a Fourier transform, and the resulting power spectrum may be analyzed to determine dominant frequency bands. Using a Taylor series expansion, the dominant frequency bands may be simplified into sine and cosine terms, each having a phase, frequency, and amplitude. A least-squares equation, or any other suitable optimization equation, can then be used to determine variables that optimize the phase, frequency, and / or amplitude (i.e., audio characteristics) of the noise-reduced sound 142. It should be understood that this is one example of a feed-forward ANR technique, and that any other suitable ANR technique and / or mechanism can be utilized for ANR, such as a feed-forward filter (e.g., an equalizer, such as an adaptive, dynamic, or feedback equalizer, including a finite impulse response (FIR) filter, an infinite impulse response (IIR) filter, etc.), a feedback filter, a fixed filter, a programmable filter, a programmable filter controller, an amplifier, a velocity-noise converter, and / or any other suitable technique and / or mechanism. The feed-forward ANR technique may enable dynamic adjustment of audio parameter values ​​of the noise-reduced signal in real time based on input data (e.g., data received in block 202 of process 200), such that the noise-reduced sound 142 output via the extra-aural speaker 114 converges to audio characteristics that optimize the noise reduction performance of the HMD system 100.

[0042] 4 is a flow diagram of an example process 400 for actively reducing noise 140 generated by a fan 118 of an HMD 102 in an energy-efficient manner, according to an embodiment disclosed herein. For discussion purposes, the process 400 will be described with reference to the previous figure. Furthermore, the process 400 may be implemented by the HMD system 100 described herein.

[0043] At 402, the processor 116 of the HMD system 100 may determine whether the fan 118 of the HMD 102 of the HMD system 100 is on or off. As used herein, the fan 118 being “on” means that the fan 118 is running and therefore making noise 140. In some examples, the determination at block 402 may be made by determining whether the fan 118 is drawing power from the power supply 128 (e.g., a battery) of the HMD 102. If the fan 118 is not on at block 402 (e.g., the fan 118 is not drawing power from the power supply 128 of the HMD 102), the process 400 may follow the “No” route from block 402 and repeat the determination at block 402 (e.g., by continuously monitoring the on / off status of the fan 118). In other words, if the fan 118 is not running, the processor 116 may determine to refrain from executing the ANR algorithm because it requires resources, including power resources, to perform the ANR techniques described herein in order to conserve energy in the HMD system 100 (e.g., to conserve battery power). Thus, if the fan 118 is not running, energy can be conserved by refraining from performing ANR and by continuing to monitor the on / off status of the fan 118. In response to a determination that the fan 118 is on (e.g., in response to a determination that the fan 118 is drawing power from the power supply 128 of the HMD 102), the process 400 may follow the "yes" route from block 402 to block 404.

[0044] At 404, the processor 116 may determine whether the fan 118 is being driven at a level that meets a threshold level. As used herein, “meeting” a threshold may mean that a value is equal to or greater than the threshold, or that a value is strictly above the threshold. Furthermore, the drive level and threshold evaluated in block 404 may be in terms of any suitable metric, such as an index level, a percentage of a maximum drive level, a number of mA of input current, and / or any other suitable metric. For example, the threshold evaluated in block 404 may be set to 10% of the maximum drive level of the fan 118. If the threshold is not met, meaning that the fan 118 is being driven at a level less than (or equal to) 10% of the maximum drive level of the fan 118 in this example, process 400 may follow a “no” route from block 404, repeating blocks 402 and 404, such as by continuously monitoring the on / off status and drive level of the fan 118. For example, when the fan 118 is driven at a relatively low drive level, the noise 140 generated by the fan 118 may be tolerable to the average HMD user, and in this scenario, the additional power required to run an ANR algorithm to reduce the fan noise 140 may not be worthwhile. Thus, in block 404, as long as the fan 118 is off or driven at a level that does not meet the threshold, the processor 116 may refrain from running the ANR algorithm. As a result, the noise 140 generated by the fan 118 may not be actively reduced to conserve energy in the HMD system 100. In contrast, if the processor 116 determines in block 404 that the fan 118 is driven at a level that meets the threshold, the process 400 may follow the “Yes” route from block 404 to block 406 and begin executing the ANR algorithm. In other words, in following the “Yes” route from block 404, the processor 116 may decide to run the ANR algorithm to reduce the fan noise 400 at the expense of utilizing some resources, such as power resources.It should be appreciated that this (i.e., following the "Yes" route from block 404) may also trigger an example ANR algorithm, process 200 of Figure 2. Process 400 after block 404, as illustrated in Figure 4, is an example ANR algorithm that is more detailed than the ANR algorithm of process 200 of Figure 2.

[0045] In some examples, before proceeding to block 406, the user 106 may be asked for approval to run the ANR algorithm. That is, the processor 116 may cause the output device (e.g., display, speaker 114, etc.) of the HMD 102 to output a prompt asking the user 106 to approve running the ANR algorithm before the ANR algorithm is executed. The prompt may be output in any suitable format (e.g., a pop-up notification on the display of the HMD 102, an audible prompt output via the speaker 114, etc.) and may present any suitable type of message (e.g., "Do you want to actively reduce fan noise in your headset? Yes or No?"). If the user 106 approves running the ANR algorithm (e.g., by providing user input indicating selection of the "yes" option in the prompt), the process 400 may proceed to block 406 and begin running the ANR algorithm. Otherwise, if the user 106 does not provide their approval, the process 400 follows the prompt, and the process 400 may not run the ANR algorithm beyond block 406.

[0046] At 406, before receiving input data to be used in the ANR algorithm, the processor 116 may determine whether to use or refrain from using the microphone 126 of the HMD 102 for ANR. In other words, the processor 116 may determine whether to use the power supply 128 of the HMD 102 to power the microphone 126 for the purpose of using the microphone 126 in the ANR algorithm. Because “opening” the microphone 126 and capturing audio data that can be used in the ANR techniques described herein requires extra power, this determination at block 406 may be yet another power conservation decision that may be based on any suitable data or information. For example, if the remaining / stored / available power of the power supply 128 fails to meet (e.g., is less than (or equal to)) a threshold power level, the determination at block 406 may be a decision to refrain from using the microphone 126 for ANR in order to conserve the remaining / stored / available power so that the power can be used for something other than powering the microphone 126 for ANR. As another example, the user 106 may specify a user preference in the settings of the HMD system 100 regarding whether the user 106 wants to use the microphone 126 for ANR, and the determination in block 406 may be based at least in part on the user preference in the settings of the HMD system 100. As yet another example, the level at which the fan 118 is driven may dictate whether to use the microphone 126 for ANR. For example, when the fan 118 is driven at a relatively low level (e.g., if the fan drive level fails to satisfy a second threshold greater than the threshold evaluated in block 404), the determination in block 406 may be to refrain from using the microphone 126 for ANR; conversely, if the second threshold is met, the determination in block 406 may be to utilize the microphone 126 for ANR.As yet another example, the predicted ANR performance gain may be evaluated against the remaining power level available from the power source 128 of the HMD 102 to determine whether to use the microphone 126 for ANR in block 406. For example, if one or more criteria, such as the amount of remaining power meets a threshold and the predicted ANR performance gain of using the microphone 126 for ANR, are met, the determination in block 406 may be to utilize the microphone 126 for ANR; conversely, if one or more of the criteria are not met, the determination in block 406 may be to refrain from using the microphone 126 for ANR. In any case, if the processor 116 determines to refrain from using the power source 128 of the HMD 102 to power the microphone 126 of the HMD 102 for the purposes of using the microphone 126 in an ANR algorithm, the process 400 may follow the “No” route from block 406 to block 408.

[0047] At 408, the processor 116 may receive non-audio data indicative of the noise 140 being generated by the fan 118 of the HMD 102. This non-audio data may be any suitable type of data as described herein, such as, but not limited to, temperature data received from the temperature sensor 124 of the HMD 102, drive data received from the drive circuitry 120 of the HMD 102, and / or utilization data related to utilization of the CPU 130, the GPU 132, and / or any other suitable non-audio data. In these examples, the temperature data may indicate a temperature sensed by the temperature sensor 124 of the HMD 102, the drive data may indicate the level at which the fan 118 is being driven via the drive circuitry 120 of the HMD 102, and / or the utilization data may indicate utilization of the CPU 130, the GPU 132, and / or another electronic component. These are merely example types of non-audio data that may be received at block 408 and that may be indicative of the noise 140 being generated by the fan 118. It should also be appreciated that multiple different types of non-audio data may be received at block 408 .

[0048] At 410, processor 116 may determine one or more audio parameter values ​​for the noise-reduced signal based at least in part on the non-audio data received at block 408 and using model 138. Examples of audio parameter values ​​determined at block 410 are described elsewhere herein and may include one or more of the audio parameter values ​​described with reference to block 204 of process 200.

[0049] At 412, a sound 142 having one or more audio characteristics based at least in part on the audio parameter values ​​determined at block 410 may be output via the extra-aural speaker 114 of the HMD 102 to reduce the noise 140 generated by the fan 118 at the ear of the user 106 of the HMD 102. Additionally, the noise-reduced sound 142 output at block 412 may be a first sound of one or more different sounds simultaneously output via the extra-aural speaker 114. For example, the HMD system 100 may be running an application 112 (e.g., a video game being played by the user 106 using the HMD 102). Thus, the extra-aural speaker 114 may be outputting the first noise-reduced sound 142 at block 412 while also outputting one or more second sounds corresponding to audio content of the running application 112 (e.g., audio content of the video game). However, the user 106 does not hear the first noise-reduced sound 142 because the first noise-reduced sound 142 destructively interferes with the fan noise 140 at the ear of the user 106. Instead, the user 106 hears a sound corresponding to the audio content of the running application 112 (e.g., the audio content of a video game) along with any other ambient noises around the user 106, excluding the fan noise 140.

[0050] In an implementation of an HMD 102 having multiple extra-aural speakers 114, such as a first (e.g., left) extra-aural speaker 114(1) and a second (e.g., right) extra-aural speaker 114(2), the output in block 412 may include outputting, in block 414, a first sound 142 having one or more first audio characteristics for reducing the noise 140 generated by the fan 118 at the location of the first (e.g., left) ear of the user 106 via the first (e.g., left) extra-aural speaker 114(1), and outputting, in block 416, a second sound 142 having one or more second audio characteristics for reducing the noise 140 generated by the fan 118 at the location of the second (e.g., right) ear of the user 106 via the second (e.g., right) extra-aural speaker 114(2). In some examples, the audio characteristics of the sound 142 output in block 412 may include, without limitation, the frequency of the sound 142 (or sound wave), the amplitude of the sound 142 (or sound wave), and the phase of the sound 142 (or sound wave). In some examples, a first audio characteristic of the first sound 142 output by the first (e.g., left) extra-aural speaker 114(1) in block 414 may differ from a second audio characteristic of the second sound 142 output by the second (e.g., right) extra-aural speaker 114(2) in block 416. In some examples, the noise-reducing sound 142 output in block 412 may be represented by sound waves having any suitable audio characteristics that destructively interfere with the sound waves of the fan noise 140 at the location of the user's 106's ear to reduce or attenuate the noise 140 generated by the fan 118 in the vicinity of the user's ear.

[0051] In some examples, the HMD 102 may include multiple extra-aural speakers 114 located at different locations relative to the ears of the user 106. For example, the multiple extra-aural speakers 114 may surround the right ear of the user 106 and / or the left ear of the user 106. Thus, the HMD 102 may be configured to output sound from any individual extra-aural speaker 114 at any suitable time, such as via an extra-aural speaker 114 positioned behind the user's right ear and / or via an extra-aural speaker 114 positioned in front of the user's left ear. For example, a speaker array may be implemented in or on the headband of the HMD 102, and some of the extra-aural speakers 114 may be positioned behind the ears, some of the extra-aural speakers 114 may be positioned in front of the ears, and some of the extra-aural speakers 114 may be aligned laterally with the ears. In some examples, the extra-aural speaker 114 may be disposed within the HMD 102 (e.g., inside the main housing of the HMD 102). In these or other similar implementations of the HMD 102, the processor 116 may determine, in block 412, which of the multiple extra-aural speakers 114 to output the noise-reduced sound 142 from. For example, the processor 116 may select a first extra-aural speaker 114 positioned in front of the left ear of the user 106 and a second extra-aural speaker 114 positioned behind the right ear of the user 106, and the processor 116 may output the noise-reduced sound 142 via the selected extra-aural speaker 114 in block 412. Thus, the location relative to the ear for emitting the noise-reduced sound 142 from the selected extra-aural speaker 114 may be dynamically determined in the ANR techniques described herein.

[0052] Returning to decision block 406, if the processor 116 determines that the power supply 128 of the HMD 102 is to be used to power the microphone 126 of the HMD 102 for the purpose of using the microphone 126 in the ANR algorithm, the process 400 may follow the "Yes" route from block 406 to block 418.

[0053] At 418, the microphones 126 of the HMD 102 may be powered for use in the ANR algorithm. This may be referred to as “opening” the microphones 126 to capture sounds in the environment. In some examples, a subset of the microphones 126 may be selected for use in the ANR algorithm, and at block 418, the selected subset of microphones 126 may be powered exclusively. For example, at block 418, the processor 116 may select a first microphone 126(1) and power the first microphone 126(1) without powering the second microphone 126(2), or vice versa. This is merely one example of how a subset of microphones 126 may be selected and powered. In other examples, at block 418, the entire set of microphones 126 of the HMD 102 may be powered.

[0054] At 420, the processor 116 may receive audio data representing fan noise 140 captured by the powered microphone 126 of the HMD 102 because the fan 118 is operating at or above the threshold drive level (e.g., following the “Yes” route from block 404) and therefore making the noise 140. The fan noise 140 is also audible in the vicinity of the powered microphone 126, which enables the microphone 126 to capture the fan noise 140 and generate audio data based on the captured fan noise 140. As indicated by the dashed arrow from block 420 to block 408, the process 400 may also include receiving non-audio data at block 408 in addition to receiving audio data at block 420.

[0055] At 424, processor 116 may determine one or more audio parameter values, such as those described elsewhere herein, including those described with reference to block 204 of process 200, based at least in part on the audio data received at block 420 (and potentially based on non-audio data received at block 408, as indicated by the dashed line in FIG. 4 from block 408 to block 424), and using model 138. In some examples, processor 116 may use first microphone 126(1) and second microphone 126(2) to determine audio characteristics (e.g., phase, amplitude, etc.) of a waveform corresponding to noise 140 generated by fan 118, and audio parameter values ​​may be determined at block 424 based at least in part on the audio characteristics of the waveform. For example, multiple data points of the audio characteristics can be determined over time and used to reconstruct the complex amplitude of a waveform corresponding to the fan noise 140, which provides the processor 116 with spatial information (e.g., directionality) associated with the source of the noise 140 and allows the processor 116 to infer how the fan noise will sound at the ear. Following block 424, the noise-canceling sound 142 can be output via the extra-aural speakers 114 of the HMD 102 in block 412, as described above.

[0056] 5 illustrates example components of an HMD system 100 capable of implementing the techniques disclosed herein, according to embodiments disclosed herein. As mentioned above, the HMD system 100 may be a distributed system including the HMD 102 and one or more additional computers 500 communicatively coupled to the HMD 102. In FIG. 5, the additional computers 500 may represent the host computer 104 and / or the remote system 108 of FIG. 1. Alternatively, the HMD system 100 may be a standalone HMD 102.

[0057] The HMD 102 may be implemented as a device to be worn by the user 106 (e.g., on the head of the user 106). In some embodiments, the HMD 102 may be head-mountable, such as by allowing the user 106 to secure the HMD 102 on their head using a fastening mechanism (e.g., an adjustable band) sized to fit around the head of the user 106. In some embodiments, the HMD 102 comprises a VR or AR headset that includes a near-eye or near-to-eye display. Accordingly, the terms “wearable device,” “wearable electronic device,” “VR headset,” “AR headset,” and “head-mounted display (HMD)” may be used interchangeably herein to refer to the device 102. However, it should be understood that these types of devices are merely examples of the HMD 102, and that the HMD 102 may be implemented in a variety of other form factors.

[0058] In the illustrated implementation, the HMD system 100 includes one or more processors 116 and memory 134 (e.g., computer-readable medium 134) introduced in FIG. 1 . In some implementations, the processor 116 may include a CPU 130, a GPU 132, both the CPU 130 and the GPU 132, a microprocessor, a digital signal processor, or other processing devices or components known in the art. Alternatively, or in addition, the functionality described herein may be implemented at least in part by one or more hardware logic components. For example, without limitation, illustrative types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), etc. Additionally, each processor 116 may have its own local memory, which may also store program modules, program data, and / or one or more operating systems.

[0059] Memory 134 may include volatile and nonvolatile memory, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Such memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), flash memory or other memory technology, compact disc ROM (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, redundant array of independent disks (RAID) storage systems, or any other medium that can be used to store desired information and that can be accessed by a computing device. Memory 134 may be implemented as a computer-readable storage medium (“CRSM”), which may be any available physical medium that is accessible by processor 116 to execute instructions stored on memory 134. In one basic implementation, CRSM may include RAM and flash memory. In other implementations, the CRSM may include, but is not limited to, a ROM, an EEPROM, or any other non-transitory and / or tangible medium that can be used to store desired information and that can be accessed by the processor 116.

[0060] In general, HMD system 100 may include logic (e.g., software, hardware, and / or firmware) configured to implement the techniques, functions, and / or operations described herein. Computer-readable medium 134 is shown as including various modules, such as instructions, data stores, etc., that may be configured to execute on processor 116 to perform the techniques, functions, and / or operations described herein. While several exemplary functional modules are shown as stored on computer-readable medium 134 and executable on processor 116, the same functionality may alternatively be implemented in hardware, firmware, or as a system-on-chip (SOC) and / or other logic.

[0061] The operating system module 502 may be configured to manage hardware within and coupled to the HMD 102 and / or HMD system 100 for the benefit of other modules. Additionally, in some cases, the HMD system 100 may include one or more applications 112 stored in the memory 134 or otherwise accessible to the HMD system 100. In this implementation, the application 112 includes a gaming application 4110. However, the HMD system 100 may include any number or type of applications and is not limited to the specific example shown here. The ANR component 136 may be configured to perform the ANR techniques described herein to reduce fan noise 140 at the location of the user's 106's ear. The memory 134 may further store the model 138 described herein when usable to implement the ANR techniques described herein.

[0062] Generally, the HMD system 100 includes input devices 504 and output devices 506. The input devices 504 may include control buttons. In some implementations, one or more microphones 126 may function as the input devices 504 for receiving audio input, such as user voice input, and for capturing audio data representing fan noise 140 for use in the ANR techniques described herein. In some examples, the microphones 126 have a frequency response in the range of approximately 20 Hertz (Hz) to 24 Kilohertz (Hz) and a sensitivity of approximately −25 decibels full scale (dBFS) per Pascal (Pa) at 1 kHz. In some implementations, one or more cameras 508 or other types of sensors 122, such as the previously mentioned temperature sensor 124 or an inertial measurement unit (IMU) 510, may function as the input devices 504. For example, the IMU 510 may be configured to detect head movement of the user 106, which may be included for gesture input purposes. Sensors 122 may further include sensors used to generate movement, position, and orientation data, such as gyroscopes, accelerometers, magnetometers, color sensors, or other movement, position, and orientation sensors. Sensors 122 may also include sub-portions of sensors, such as a series of active or passive markers that can be viewed externally by a camera or color sensor to generate movement, position, and orientation data. For example, a VR headset may include multiple markers on its exterior, such as reflectors or lights (e.g., infrared or visible light), that, when viewed by an external camera or illuminated with light (e.g., infrared or visible light), may provide one or more reference points for interpretation by software to generate movement, position, and orientation data. Sensors 122 may include light sensors that are sensitive to light (e.g., infrared or visible light) projected or broadcast by a base station in the HMD 102's environment. The IMU 510 may be an electronic device that generates calibration data based on measurement signals received from accelerometers, gyroscopes, magnetometers, and / or other sensors suitable for detecting movement, correcting errors associated with the IMU 510, or some combination thereof.Based on the measurement signals, such motion-based sensors, such as the IMU 510, may generate calibration data indicating an estimated position of the HMD 102 relative to the initial position of the HMD 102. For example, multiple accelerometers may measure translational motion (forward / backward, up / down, left / right), and multiple gyroscopes may measure rotational motion (e.g., pitch, yaw, roll). The IMU 510 may, for example, rapidly sample the measurement signals and calculate an estimated position of the HMD 102 from the sampled data. For example, the IMU 510 may integrate measurement signals received from the accelerometers over time to estimate a velocity vector and integrate the velocity vector over time to determine an estimated position of a reference point on the HMD 102. A reference point is a point that can be used to represent the position of the HMD 102. While a reference point may generally be defined as a point in space, in various embodiments, the reference point is defined as a point within the HMD 102 (e.g., the center of the IMU 510). Alternatively, the IMU 510 provides the sampled measurement signals to an external console (or other computing device), which determines the calibration data.

[0063] The sensor 122 may operate at a relatively high frequency to provide sensor data at a high rate. For example, the sensor data may be generated at a rate of 1000 Hz (or one sensor reading every millisecond). In this manner, 1000 readings are generated per second. If the sensor generates this much data at this rate (or even faster), the data set used to predict movement becomes very large, even over a relatively short period of time, on the order of tens of milliseconds. As mentioned, in some embodiments, the sensor 122 may include an optical sensor sensitive to light emitted by a base station in the HMD 102's environment for the purpose of tracking the position and / or orientation, posture, etc., of the HMD 102 in three-dimensional (3D) space. Calculations of position and / or orientation may be based on timing characteristics of the light pulses and the presence or absence of light detected by the sensor 122.

[0064] In some embodiments, additional input devices 504 may be provided in the form of a keyboard, keypad, mouse, touchscreen, joystick, etc. In other embodiments, the HMD system 100 may omit a keyboard, keypad, or other similar form of mechanical input. In some examples, the input devices 504 may include controls such as basic volume control buttons for increasing / decreasing the volume, as well as power and reset buttons.

[0065] The output device 504 may include a display or display panel 512 (e.g., a stereo pair of display panels). The display panel 512 of the HMD system 100 may utilize any suitable type of display technology, such as an emissive display that utilizes light-emitting elements (e.g., light-emitting diodes (LEDs)) to emit light during the presentation of frames on the display panel 512. By way of example, the display panel 512 of the HMD system 100 may include a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, an inorganic light-emitting diode (ILED) display, or any other suitable type of display technology for HMD applications. The output device 506 may further include, without limitation, light elements (e.g., LEDs), a vibrator for generating haptic sensations, and the previously mentioned fan 118 and associated drive circuitry 120, as well as the extra-aural speaker 114. In some examples, the extra-aural speakers 114 are 37.5 millimeter (mm) extra-aural balanced mode radiators (BMR) with a frequency response in the range of approximately 40 Hz and 24 kHz, an impedance of 6 Ohms, and a sound pressure level (SPL) of 98.96 dBSPL at 1 centimeter (cm). In some examples, the extra-aural speakers 114 are implemented in the form of sound bars, speakers embedded in the main HMD body, and / or speakers embedded in the headband of the HMD 102.

[0066] The HMD system 100 may include a power source 128, such as one or more batteries. Additionally or alternatively, the HMD 102 may include a power cable port for connecting to an external power source via wired means, such as a cable.

[0067] The HMD system 100 may further include a communication interface 514, such as a wireless device coupled to an antenna, to facilitate wireless connection to a network. Such a wireless device may implement one or more of a variety of wireless technologies, such as Wi-Fi, Bluetooth, radio frequency (RF), etc. However, it should be understood that the HMD 102 may also include a physical port to facilitate a wired connection to a network, connected peripheral devices (including a computer 500, such as the host computer 104, which may be a PC, a game console, etc.), or a plug-in network device that communicates with other wireless networks.

[0068] The HMD 102 may further include an optical subsystem 516 that directs light from the electronic display panel 512 to the user's eyes using one or more optical elements. The optical subsystem 516 may include various types and combinations of different optical elements, including, but not limited to, apertures, lenses (e.g., Fresnel lenses, convex lenses, concave lenses, etc.), filters, etc. In some embodiments, one or more optical elements in the optical subsystem 516 may have one or more coatings, such as anti-reflective coatings. The expansion of image light by the optical subsystem 516 may allow the electronic display panel 512 to be physically smaller, lighter, and consume less power than larger displays. Additionally, the expansion of image light may increase the field of view (FOV) of the displayed content (e.g., images). For example, the FOV of the displayed content may be such that the displayed content is presented using nearly all (e.g., 120-150 degrees diagonal), or in some cases all, of the user's FOV. AR applications may have a narrower FOV (e.g., an FOV of approximately 40 degrees). The optical subsystem 516 may be designed to correct one or more optical errors, such as, but not limited to, barrel distortion, pincushion distortion, longitudinal chromatic aberration, lateral aberration, spherical aberration, coma, field curvature, astigmatism, etc. In some embodiments, the content provided to the electronic display panel 512 for display is pre-distorted, and the optical subsystem 516 corrects the distortion when it receives image light from the electronic display panel 512 generated based on the content.

[0069] The HMD system 100 may further include an eye tracking system 518. A camera 508 or other optical sensor inside the HMD 102 may capture image information of the user's eyes, and the eye tracking module 518 may use the captured information to determine the interpupillary distance, the 3D position of each eye relative to the HMD 102, including the magnitude of twist and rotation (i.e., roll, pitch, and yaw), and the gaze direction of each eye (e.g., for distortion adjustment purposes). In one example, infrared light is emitted within the HMD 102 and reflected from each eye. The reflected light is received or detected by a camera in the HMD 102 and analyzed to extract eye rotation from changes in the infrared light reflected by each eye. Many methods for tracking the eyes of the user 106 may be used by the eye tracking system 518. Thus, the eye tracking system 518 may track up to six degrees of freedom (i.e., 3D position, roll, pitch, and yaw) of each eye, and at least a subset of the tracked quantities may be combined from the two eyes of the user 106 to estimate a point of gaze (i.e., the 3D location or position within the virtual scene at which the user is viewing). For example, the eye tracking system 518 may integrate information from past measurements, measurements identifying the position of the user's 106's head, and 3D information describing the scene presented by the electronic display panel 512. Thus, information about the position and orientation of the user's 106's eyes may be used to determine a point of gaze within the virtual scene presented by the HMD 102 at which the user 106 is viewing.

[0070] The HMD system 100 may further include a head tracking system 520. The head tracking system 520 may utilize one or more of the sensors 122, as described above, to track head movement, including head rotation, of the user 106. For example, the head tracking system 520 may track up to six degrees of freedom (i.e., 3D position, roll, pitch, and yaw) of the HMD 102. These calculations may be performed for each frame of a series of frames, so that the application 112 (e.g., a video game) can determine how to render a scene in the next frame depending on the head position and orientation. In some embodiments, the head tracking system 520 is configured to predict the future position and / or orientation of the HMD 102 based on current and / or past data. This is because the application 112 is required to render frames before the user 106 actually sees the light (and therefore the image) on the display 512. Thus, the next frame can be rendered based on this future prediction of head position and / or orientation made at an earlier time, such as approximately 25-30 milliseconds (ms) before rendering the frame. In a distributed system in which the host computer 104 is communicatively (e.g., wirelessly) coupled to the HMD 102, the future prediction of head pose may be made 30 ms or more in advance of the light time for the frame to account for network latency, compression behavior, etc. The rotational data provided by the head tracking system 520 may be used to determine both the direction of rotation of the HMD 102 and the amount of rotation of the HMD 102 in any suitable units of measure. For example, the direction of rotation may be simply output in terms of positive or negative horizontal directions and positive or negative vertical directions corresponding to left, right, up, and down. The amount of rotation may be in units of degrees, radians, etc. Angular velocity may be calculated to determine the rate of rotation of the HMD 102.

[0071] The HMD system 100 may further include a controller tracking system 522. The controller tracking system 522 may utilize one or more of the sensors 122 to track the movement of the controller. For example, the controller tracking system 522 may track up to six degrees of freedom (i.e., 3D position, roll, pitch, and yaw) of the controller that the user 106 holds in his or her hand. These calculations may be performed for each frame of a series of frames, so that the application 112 (e.g., a video game) can determine how to render a virtual controller and / or virtual hands in a scene in the next frame depending on the position and orientation of the controller. In some embodiments, as described above with respect to the controller tracking system 522, the controller tracking system 522 is configured to predict the future position and / or orientation of the HMD 102 based on current and / or past data.

[0072] Although the present subject matter has been described in language specific to structural features, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features described. Rather, the specific features are disclosed as example forms of implementing the claims.

Claims

1. A head-mounted display (HMD) system, comprising: An HMD, With fans, Left external ear speaker, a right extra-aural speaker; and a processor; A memory storing computer-executable instructions that, when executed by the processor, cause the processor to: receiving data indicative of noise being produced by the fan; determining one or more audio parameter values ​​based at least in part on the data and using a model; outputting a first sound having one or more first audio characteristics via the left extra-aural speaker based at least in part on the one or more audio parameter values ​​so as to reduce the noise generated by the fan at a left ear of a user of the HMD; and outputting a second sound having one or more second audio characteristics through the right extra-aural speaker based at least in part on the one or more audio parameter values ​​so as to reduce the noise generated by the fan at the location of the user's right ear.

2. The HMD includes: The left microphone and a right microphone; The data is first audio data representing the noise captured by the left microphone; and second audio data representing the noise captured by the right microphone.

3. the processor comprises a central processing unit (CPU); The HMD includes: a temperature sensor for sensing the temperature of an electronic component of the HMD; a drive circuit for driving the fan; or a graphics processing unit (GPU), The data is temperature data indicative of the temperature sensed by the temperature sensor; drive data indicative of the level to which the fan is driven via the drive circuit; or The HMD system of claim 1 , further comprising: usage data indicating usage of the CPU or the GPU.

4. The instructions, when executed by the processor, cause the processor to: determining that the fan is operating at a level that meets a threshold level; The HMD system of claim 1 , further comprising: determining to run an active noise reduction algorithm in response to determining that the fan is driven at the level.

5. a video game is being played using the HMD; The instructions, when executed by the processor, cause the processor to: outputting a third sound through the left outer ear speaker while outputting the first sound, the third sound corresponding to audio content of the video game; 2. The HMD system of claim 1, further comprising: outputting a fourth sound through the right outer ear speaker while outputting the second sound, the fourth sound corresponding to the audio content.

6. 1. A method comprising: receiving, by a processor, data indicative of noise being generated by a fan of a head mounted display (HMD); determining, by the processor, one or more audio parameter values ​​based at least in part on the data and using a model; and outputting sound having one or more audio characteristics based at least in part on the one or more audio parameter values ​​via one or more extra-aural speakers of the HMD so as to reduce the noise generated by the fan at ear locations of a user of the HMD.

7. The method of claim 6 , wherein the data includes audio data representing the noise captured by one or more microphones of the HMD.

8. the one or more microphones comprise a first microphone and a second microphone; The method further includes determining, by the processor and using the first microphone and the second microphone, an audio characteristic of a waveform corresponding to the noise produced by the fan; The method of claim 7 , wherein the determining of the one or more audio parameter values ​​is based at least in part on the audio characteristics.

9. The data is temperature data indicative of the temperature of electronic components of the HMD sensed by a temperature sensor of the HMD; Drive data indicating the level to which the fan is driven via a drive circuit of the HMD; or The method of claim 6 , including utilization data indicative of at least one of a central processing unit (CPU) of the HMD or a graphics processing unit (GPU) utilization of the HMD.

10. 10. The method of claim 9, further comprising, prior to receiving the data, determining by the processor to refrain from using a power source of the HMD to power one or more microphones of the HMD for the purpose of using the one or more microphones in an active noise reduction algorithm.

11. determining, by the processor, that the fan is drawing power from a power supply of the HMD; The method of claim 6 , further comprising: determining, by the processor, to execute an active noise reduction algorithm in response to determining that the fan is drawing the power.

12. the one or more extra-aural speakers comprise a first extra-aural speaker and a second extra-aural speaker; The outputting of the sound having the one or more audio characteristics may include: outputting a first sound having one or more first audio characteristics via the first extra-aural speaker to reduce the noise generated by the fan at a first ear location of the user; and outputting a second sound having one or more second audio characteristics via the second extra-aural speaker so as to reduce the noise generated by the fan at a second ear of the user.

13. The method of claim 12 , wherein the one or more first audio characteristics of the first sound are different from the one or more second audio characteristics of the second sound.

14. A head-mounted display (HMD) system, comprising: An HMD, With fans, an HMD comprising one or more extra-aural speakers; a processor; A memory storing computer-executable instructions that, when executed by the processor, cause the processor to: receiving data indicative of noise being produced by the fan; determining one or more audio parameter values ​​based at least in part on the data and using a model; and outputting sound having one or more audio characteristics through the one or more extra-aural speakers based at least in part on the one or more audio parameter values ​​so as to reduce the noise generated by the fan at the location of the ear of a user of the HMD.

15. the one or more extra-aural speakers comprise a first extra-aural speaker and a second extra-aural speaker; outputting the sound having the one or more audio characteristics outputting a first sound having one or more first audio characteristics via the first extra-aural speaker to reduce the noise generated by the fan at a first ear location of the user; and outputting a second sound having one or more second audio characteristics via the second extra-aural speaker so as to reduce the noise generated by the fan at a second ear location of the user.

16. the first extra-aural speaker is located on the left side of the HMD; the first ear is the left ear of the user; the second extra-aural speaker is located on the right side of the HMD; The HMD system of claim 15 , wherein the second ear is the right ear of the user.

17. The HMD includes: a first microphone; a second microphone; The data is first audio data representing the noise captured by the first microphone; and second audio data representing the noise captured by the second microphone.

18. the processor comprises a central processing unit (CPU); The HMD includes: a temperature sensor for sensing the temperature of an electronic component of the HMD; a drive circuit for driving the fan; or a graphics processing unit (GPU), The data is temperature data indicative of the temperature sensed by the temperature sensor; drive data indicative of the level to which the fan is driven via the drive circuit; or The HMD system of claim 14 , further comprising: usage data indicative of utilization of the CPU or the GPU.

19. The instructions, when executed by the processor, cause the processor to: determining that the fan is operating at a level that meets a threshold level; The HMD system of claim 14 , further comprising: determining to run an active noise reduction algorithm in response to determining that the fan is driven at the level.

20. the sound is a first sound, The HMD system of claim 14, wherein the instructions, when executed by the processor, cause the processor to further output a second sound corresponding to audio content of a running application through the one or more extra-aural speakers while outputting the first sound.

Images