Head-mounted device for generating binaural audio

Abstract

A head-mounted device for generating binaural audio content includes a first rod coupled to a first microphone housing enclosing a first front microphone and a first rear microphone, respectively, that generate acoustic signals. The first microphone housing includes a first front port facing downward and a first rear port facing rearward. The apparatus includes a second rod coupled to a second microphone housing enclosing a second front microphone and a second rear microphone, respectively, that generate acoustic signals. The second microphone housing includes a second front port facing downward and a second rear port facing rearward. In one embodiment, an apparatus includes a binaural audio processor including a beamformer and a storage device. The beamformer generates a first beamformer signal based on acoustic signals from the first front microphone and the first rear microphone and a second beamformer signal based on acoustic signals from the second front microphone and the second rear microphone. A storage device stores the first and second beamformer signals as a two-channel file.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 202080024964.3. The filing date of the original application is March 12, 2020, the priority date is March 29, 2019, the date of entry into the Chinese National Phase is September 27, 2021, and the invention title is "Head-mounted device for generating binaural audio".

[0002] Priority Claims

[0003] This application claims the priority of U.S. Patent Application Serial No. 16 / 370,190, filed on March 29, 2019, which is hereby incorporated by reference in its entirety. Background Art

[0004] Currently, many consumer electronic devices are adapted to capture audio and / or visual content. For example, a user can use the built-in camera on a mobile device to quickly capture events or moments that occur in the user's life.

[0005] However, the time required to retrieve the mobile device may still be too long to capture some fleeting events or moments. The user may also feel as if they have to remove themselves from the presence and experience of the event or moment in order to retrieve their mobile device. In addition, when played back, the audio portion of the content recorded using a mobile device may not adequately reproduce the sounds perceived by the user. Brief Description of the Drawings

[0006] In the drawings, which are not necessarily to scale, the same reference numerals may describe similar components in different views. The same reference numerals with different letter suffixes may represent different instances of similar components. Some embodiments are shown in the drawings by way of example and not limitation, in which:

[0007] Figure 1 A perspective view of a head-mounted device for generating binaural audio according to an example embodiment is shown.

[0008] Figure 2 Shows a bottom view of the head-mounted device from Figure 1 according to an example embodiment.

[0009] Figures 3A - 3B Shows a detail of a portion of one microphone housing of the head-mounted device from Figure 1 according to an example embodiment.

[0010] Figure 4 Is an exemplary flowchart of a process for generating binaural audio using the head-mounted device from Figure 1 according to various aspects of the present disclosure.

[0011] Figure 5A block diagram of a system for generating binaural audio included in a head-mounted device from Figure 1 is shown.

[0012] Figure 6 A block diagram of a binaural audio processor included in a system in Figure 5 is shown.

[0013] Figure 7 is a block diagram showing a representative software architecture that can be used in conjunction with various hardware architectures described herein.

[0014] Figure 8 is a block diagram showing components of a machine capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and performing any one or more of the methods discussed herein.

[0015] Figure 9 is a high-level functional block diagram of an example head-mounted device communicatively coupled to a mobile device and a server system via various networks. DETAILED DESCRIPTION

[0016] The following description includes systems, methods, techniques, instruction sequences, and computer program products embodying illustrative embodiments of the present disclosure. In the following description, for purposes of explanation, numerous specific details are set forth to provide an understanding of various embodiments of the inventive subject matter. However, it will be apparent to one skilled in the art that embodiments of the inventive subject matter may be practiced without these specific details. Generally, well-known instruction instances, protocols, structures, and techniques need not be shown in detail.

[0017] To improve audio recordings captured by current electronic mobile devices, some embodiments of the present disclosure relate to a head-mounted device 100 capable of capturing audio content that, when played back, mimics the sounds perceived by a user of the head-mounted device 100. Specifically, the head-mounted device 100 can use microphones to record audio, which are arranged to create a three-dimensional (3D) sound sensation for a listener as if it were present during the audio recording. This is referred to as binaural audio. The playback of the captured audio content will have the effect of binaural audio, which has stereo separation and spectral content that mimics the human ear and head response. The design of the head-mounted device 100 makes particular use of the diffraction pattern of the human head, the placement of microphones on either side of the head, and beamforming techniques.

[0018] Figure 1 A perspective view of a head-mounted device 100 for generating binaural audio according to an example embodiment is shown. Figure 2 Shown is a bottom view of the head-mounted device 100 from Figure 1 in accordance with an example embodiment. InFigure 1 and Figure 2 In Figure 2 , the head - mounted device 100 is a pair of glasses. In some embodiments, the head - mounted device 100 can be sunglasses or goggles. Some embodiments can include one or more wearable devices, such as a pendant with an integrated camera that is integrated with, communicates with, or is coupled to the head - mounted device 100 or the client device. Any desired wearable device can be used in conjunction with the embodiments of the present disclosure, such as a watch, headphones, a wristband, earbuds, clothing (such as a hat or jacket with integrated electronics), a clip - on electronic device, or any other wearable device. It should be understood that although not shown, one or more parts of the system included in the head - mounted device can be included in a client device (e.g., Figure 8 the machine 800 in Figure 8 ) that can be used in conjunction with the head - mounted device 100. For example, one or more elements as shown in Figure 5 and Figure 6 can be included in the head - mounted device 100 and / or the client device.

[0019] As used herein, the term "client device" can refer to any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. The client device can be, but is not limited to, a mobile phone, a desktop computer, a laptop computer, a portable digital assistant (PDA), a smartphone, a tablet computer, a super - book, a netbook, a laptop computer, a multi - processor system, a microprocessor - based or programmable consumer electronics product, a gaming console, a set - top box, or any other communication device that a user can use to access the network.

[0020] In Figure 1 and Figure 2 In Figure 1 and Figure 2 , the head - mounted device 100 is a pair of glasses that includes a frame 103, which includes a rim (or frame) that is coupled to two rods (or temples) via hinges and end pieces, respectively. The rim of the frame 103 carries or holds a pair of lenses 104_1, 104_2. The frame 103 includes a first (e.g., right) side coupled to the first rod and a second (e.g., left) side coupled to the second rod. The first side is opposite the second side of the frame 103.

[0021] The device 100 further includes a camera module that includes camera lenses 102_1, 102_2 and at least one image sensor. The camera lens can be a perspective camera lens or a non - perspective camera lens. The non - perspective camera lens can be, for example, a fish - eye lens, a wide - angle lens, an omnidirectional lens, etc. The image sensor captures digital video through the camera lens. The image can also be a still image frame or a video that includes multiple still image frames. The camera module can be coupled to the frame 103. As Figure 1 and Figure 2As shown in the figure, the frame 103 is coupled to the camera lenses 102_1 and 102_2 such that the camera lenses face forward. The camera lenses 102_1 and 102_2 can be perpendicular to the lenses 104_1 and 104_2. The camera module can include dual front cameras separated by the width of the frame 103 or the width of the user's head of the device 100.

[0022] In Figure 1 and Figure 2 two rods (or stay wires) are respectively coupled to the microphone housings 101_1 and 101_2. The first rod and the second rod are coupled to opposite sides of the frame 103 of the head-mounted device 100. The first rod is coupled to the first microphone housing 101_1 and the second rod is coupled to the second microphone housing 101_2. The microphone housings 101_1 and 101_2 can be coupled to the rod between the position where they are coupled to the frame 103 and the stay wire foot sleeves. When the user wears the device 100, the microphone housings 101_1 and 101_2 can be located on either side of the user's stay wires.

[0023] As Figure 2 shown, the microphone housings 101_1 and 101_2 enclose a plurality of microphones 110_1 to 110_N (N>1). The microphones 110_1 to 110_N are air interface sound pickup devices that convert sound into electrical signals. More specifically, the microphones 110_1 to 110_N are transducers that convert sound pressure into electrical signals (e.g., acoustic signals). The microphones 110_1 to 110_N can be digital or analog microelectromechanical system (MEMS) microphones. The acoustic signals generated by the microphones 110_1 to 110_N can be pulse density modulation (PDM) signals.

[0024] In Figure 2 the first microphone housing 101_1 encloses the microphones 110_3 and 110_4, while the second microphone housing 101_2 encloses the microphones 110_1 and 110_2. In the first microphone housing 101_1, the first front microphone 110_3 and the first rear microphone 110_4 are spaced a predetermined distance d1 and form a first-order differential microphone array. In the second microphone housing 101_2, the second front microphone 110_1 and the second rear microphone 110_2 are also spaced a predetermined distance d2 and form a first-order differential microphone array. The predetermined distances d1 and d2 can be the same distance or different distances. The predetermined distances d1 and d2 can be set based on the Nyquist frequency. The content above the Nyquist frequency of the beamformer is irrecoverable, especially for speech. The Nyquist frequency is determined by the following equation:

[0025]

[0026] In this equation, c is the speed of sound, and d is the spacing between the microphones. Using this equation, in one embodiment, the predetermined distances d1 and d2 can be set to any d value that results in a frequency higher than 6 kHz (which is the cut-off frequency of wideband speech).

[0027] Although in Figure 1 system 100 includes four microphones 110_1 to 110_4, the number of microphones can vary. In some embodiments, the microphone housings 101_1, 101_2 can include at least two microphones and can form a microphone array. Each of the microphone housings 101_1, 101_2 can also include a battery.

[0028] A user naturally perceives audio through the two ears separated by the head, enabling the user to distinguish the direction from which the sound originates. Thus, by placing the microphone housings 101_1, 101_2 on the stem of the head-mounted device 100, the head-mounted device 100 can achieve capturing the sounds perceived by a user wearing the head-mounted device 100.

[0029] Referring to Figure 2 , each of the microphone housings 101_1, 101_2 includes a front port and a rear port. The front port of the first microphone housing 101_1 is coupled to the microphone 110_3 (e.g., the first front microphone) and the rear port of the first microphone housing 101_1 is coupled to the microphone 110_4 (e.g., the first rear microphone). The front port of the second microphone housing 101_2 is coupled to the microphone 110_1 (e.g., the second front microphone) and the rear port of the second microphone housing 101_2 is coupled to the microphone 110_2 (e.g., the second rear microphone). In one embodiment, the microphones 101_1 to 101_4 can be further moved towards the earcups on the stem of the device 100 (e.g., the back of the device 100) to emphasize the binaural effect captured by the microphones.

[0030] Figures 3A - 3B Shows details of a portion of one of the microphone housings of a head-mounted device according to an example embodiment. Specifically, Figure 1 shows details of the microphone 110_1 (e.g., the second front microphone) and the front port associated therewith. Although Figure 3A shows details of the microphone 110_1 coupled to the front port in the second microphone housing 101_2, it should be understood that the details of the microphone 110_3 (e.g., the first front microphone) coupled to the front port in the first microphone housing 101_1 are similar to Figure 3A the details in Figure 3A .

[0031] Figure 3AIt is a cross-sectional view of the front microphone 110_1 and the acoustic path 112. As shown, the acoustic path 112 passes through the gap between the block 114 (e.g., the second microphone housing 101_2) and the housing 113 of the second rod. The housing 113 of the rod can be made of metal. The front microphone 110_1 and the front port point (or face) downward. For example, when the user stands and wears the device 100, Figure 3A the front port in

[0032] Figure 3B shows details of another part of the head-mounted device according to an example embodiment. Specifically, Figure 1 it shows details of the microphone 110_2 (e.g., the second rear microphone) and the rear port associated therewith. Although Figure 3B it shows details of the microphone 110_2 coupled to the rear port in the second microphone housing 101_2, it should be understood that the details of the microphone 110_4 (e.g., the first rear microphone) coupled to the rear port in the first microphone housing 101_1 are similar to Figure 3B those in Figure 3B .

[0033] Figure 3B It is a cross-sectional view of the rear microphone 110_2 and the acoustic path 112. As shown, the acoustic path 112 passes through the gap between the block 114 (e.g., the second microphone housing 101_2) and the housing 113 of the second rod. Figure 3B The housing 113 of the rod in Figure 3B can also be made of metal. The rear microphone 110_2 and the rear port point (or face) backward. For example, when the user wears the device 100,

[0034] as Figures 3A - 3BAs shown, microphones 110_1 to 110_4 can be part of a microphone component stack that includes a flexible circuit board and a pressure-sensitive adhesive (PSA) stack that includes a waterproof membrane sandwiched between PSA layers. The waterproof membrane can protect microphones 110_1 to 110_4 from water ingress and air leakage. The PSA stack is then coupled to the microphone component housing. In some embodiments, the microphone component housing is the block 114 or housing 113 of the second bar.

[0035] Figure 4 is an exemplary flowchart of a process for generating binaural audio using a head-mounted device 100 in accordance with various aspects of the present disclosure. Although the flowchart may depict the operations as a sequential process, many of the operations can be performed in parallel or concurrently. Additionally, the order of the operations can be rearranged. The process terminates when its operations are complete. The process can correspond to a method, program, etc. The steps of the method can be performed in whole or in part, can be combined with some or all of the steps in other methods, and can be performed by any number of different systems (such as Figure 1 and / or Figure 1 and the systems described in Figure 8 . The process 400 can also be performed by a processor included in the head-mounted device 100 included in Figure 1 or by a processor included in a client device 800 included in Figure 8 .

[0036] The process 400 begins at operation 401, where microphones 110_1 to 110_4 generate acoustic signals. Microphones 110_1 to 110_4 can be MEMS microphones that convert sound pressure into an electrical signal (e.g., an acoustic signal). The first front microphone 110_3 and the first rear microphone 110_4 are enclosed in a first microphone 101_1 housing coupled to the first bar of the head-mounted device 100. The first front microphone 110_3 and the first rear microphone 110_4 form a first-order differential microphone array. The second front microphone 110_1 and the second rear microphone 110_2 are enclosed in a second microphone housing 101_2 coupled to the second bar of the head-mounted device 100. The second front microphone 110_1 and the second rear microphone 110_2 form a first-order differential microphone array. The first bar and the second bar are coupled to opposite sides of the frame 103 of the head-mounted device 100. The acoustic signal can be a pulse density modulation (PDM) signal.

[0037] At operation 402, audio codecs 501_1, 501_2 decode acoustic signals from microphones 110_1 to 110_4. Audio codec 501_2 decodes acoustic signals from the first front microphone 110_ and the first rear microphone 110_4 to generate a first decoded acoustic signal, and audio codec 501_1 decodes the second front microphone 110_1 and the second rear microphone 110_2 to generate a second decoded acoustic signal. The first and second decoded acoustic signals are pulse code modulation (PCM) signals. The first decoded acoustic signal is a PCM signal based on the acoustic signals from the first front microphone 110_3 and the first rear microphone 110_4. The second decoded acoustic signal is a PCM signal based on the acoustic signals from the second front microphone 110_1 and the second rear microphone 110_2.

[0038] At operation 403, TDM 502 processes the decoded acoustic signals from audio codecs 501_1, 501_2. TDM 502 processes the first and second decoded acoustic signals by time-division multiplexing the first and second decoded acoustic signals. The TDM-processed signal includes the first decoded acoustic signal and the second decoded acoustic signal.

[0039] At operation 404, beamformer 601 beamforms the TDM-processed signal. As further described below, beamformer 601 can be a fixed beamformer including a fixed beam pattern that is sub-cardioid or cardioid. Beamformer 601 beamforms the first decoded signal to generate a first beamformer signal and beamforms the second decoded signal to generate a second beamformer signal.

[0040] At operation 405, storage device 604 stores the beamformer signals as a dual-channel file. The dual-channel file can be a dual-channel PCM file or a dual-channel Advanced Audio Coding (AAC) / PCM file. Storage device 604 can be a flash memory device.

[0041] In one embodiment, noise suppressor 602 suppresses noise from the first beamformer signal and the second beamformer signal and generates a first noise-suppressed signal and a second noise-suppressed signal. Voice enhancer 603 can enhance the voice from the first noise-suppressed signal and the second noise-suppressed signal to generate a first clean signal and a second clean signal, respectively. In this embodiment, storage device 604 stores the first and second clean signals as a dual-channel PCM file.

[0042] Figure 5 Illustrated is a method for generating a signal including in accordance with one exemplary embodiment Figure 1Block diagram of a system 500 for binaural audio in a head-mounted device 100. In some embodiments, one or more portions of the system 500 may be included in the head-mounted device 100 or may be included in a client device (e.g., Figure 8 machine 800 in

[0043] The system 500 includes microphones 110_1 to 110_N, audio codecs 501_1, 501_2, a time division multiplexer (TDM) 502, and a binaural audio processor 503. A first front microphone 110_3 and a first rear microphone 110_4 enclosed in a first microphone housing 101_1 form a first-order differential microphone array. Similarly, a second front microphone 110_1 and a second rear microphone 110_2 enclosed in a second microphone housing 101_2 form another first-order differential microphone array. The microphones 110_1 to 110_4 may be analog or digital MEMS microphones. The acoustic signals generated by the microphones 110_1 to 110_4 may be pulse density modulation (PDM) signals.

[0044] The audio codec 501_1 decodes the acoustic signals from the first front microphone 110_3 and the first rear microphone 110_4 to generate a first decoded acoustic signal. The audio codec 501_2 decodes the acoustic signals from the second front microphone 110_1 and the second rear microphone 110_2 to generate a second decoded acoustic signal. The first and second decoded acoustic signals may be pulse code modulation (PCM) signals. In one embodiment, the audio codecs 501_1, 501_2 decode the acoustic signals as PDM signals from a single-bit PDM format to a multi-bit pulse code modulation (PCM) format. The audio codecs 501_1, 501_2 may include a PDM input with a filter that converts the PDM signal to the PCM format. In one embodiment, the audio codecs 501_1, 501_2 use a microcontroller with a synchronous serial interface to capture the PDM data stream from the microphones 110_1 to 110_4 and use a filter implemented in software to convert the PDM data stream to the PCM format.

[0045] The PCM signal may be interpreted by an interface of the binaural audio processor 503. In some embodiments, the binaural audio processor 503 is a system on chip (SoC). The SoC may include an interface, such as an I2S interface, to receive and interpret the PCM signal.

[0046] In one embodiment, the interface of the binaural audio processor 503 can only process one packet (e.g., 2-channel audio), and the time-division multiplexer (TDM) 502 in the system 500 receives and processes the first and second decoded acoustic signals (e.g., PCM signals) to generate a TDM-processed signal. The TDM 502 time-division multiplexes the first and second decoded acoustic signals to generate a TDM-processed signal. In one embodiment, the system 500 further oversamples the TDM-processed signal to allow for four microphone signals to be input. The system 500 may also include a switch for creating additional slots to allow for microphone signals.

[0047] In one embodiment, the microphones 110_1 to 110_4 are digital MEMS microphones. The acoustic signals generated by the digital MEMS microphones are relatively immune to noise, but signal integrity is still an issue due to distortion caused by parasitic capacitance, resistance, and inductance between the outputs of the microphones 110_1 to 110_4 and the SoC (such as the binaural audio processor 503). Impedance mismatches also create reflections that can distort the signal in applications where the distance between the digital microphones 110_1 to 110_4 and the SoC is long. In one embodiment, the microphones 110_1 to 110_4 are attached to a flexible circuit that is designed to maximize signal integrity and also minimize the trace length between components. In this embodiment, the flexible circuit is enclosed within the microphone housings 101_1, 101_2.

[0048] Figure 6 A block diagram of the binaural audio processor 503 included in the system 500 according to one example embodiment is shown. The binaural audio processor 503 includes a beamformer 601, a noise suppressor 602, a voice enhancer 603, and a storage device 604. Figure 5 In one embodiment, the binaural audio processor 503 includes an interface that receives the TDM-processed signal. As described above, the TDM-processed signal is generated based on the acoustic signals from the first front microphone 110_3, the first rear microphone 110_4, the second front microphone 110_1, and the second rear microphone 110_2. In one embodiment, the microphones 110_1 to 110_4 are inherently omnidirectional digital MEMS microphones.

[0049] The beamformer 601 with direction-steering characteristics is a differential beamformer that allows for a flat frequency response except at the Nyquist frequency. The beamformer 601 uses the transfer function of a first-order differential microphone array. For two microphones, the transfer function of the first-order differential microphone array is as follows:

[0050] Same as the original text without translation required for these tags

[0051]

[0052] In the above equations, theta θ is the angle, and beta β is 180 degrees. For a fixed-frequency (or frequency-independent) beam (such as a beamformer signal), the equation simplifies to E = A + Bcosθ. E in the simplified equation is the fixed-frequency output of the beamformer. In one embodiment, the beamformer 601 is a fixed beamformer including a fixed beam pattern that is sub-cardioid with A and B coefficients of 0.25 and 0.75, respectively. In one embodiment, the beamformer 601 is a fixed beamformer including a fixed beam pattern that is cardioid with A and B coefficients of 0.5 and 0.5, respectively.

[0053] In one embodiment, the beamformer 601 receives acoustic signals from the first front microphone 110_3, the first rear microphone 110_4, the second front microphone 110_1, and the second rear microphone 110_2. In one embodiment, the beamformer 601 receives signals processed by TDM. The beamformer 601 generates a first beamformer signal based on the acoustic signals from the first front microphone 110_3 and the first rear microphone 110_4, and generates a second beamformer signal based on the acoustic signals from the second front microphone 110_1 and the second rear microphone 110_2. The storage device 604 may store the first and second beamformer signals as a dual-channel file.

[0054] The noise suppressor 602 suppresses noise from the first beamformer signal and the second beamformer signal. The noise suppressor 602 is a dual-channel noise suppressor and generates a first noise-suppressed signal and a second noise-suppressed signal. In one embodiment, the noise suppressor 602 may implement a noise suppression algorithm.

[0055] The voice enhancer 603 enhances the voice from the first noise-suppressed signal and the second noise-suppressed signal to generate a first clean signal and a second clean signal. In one embodiment, the voice enhancer 603 may implement model-based voice enhancement. The voice enhancer 603 may search for multiple voice features in the first and second noise-suppressed signals. When the voice enhancer 603 identifies a portion in the first and second noise-suppressed signals that matches at least one of the voice features, the voice enhancer 603 enhances or emphasizes the identified portion. In one embodiment, the voice enhancer 603 may implement a voice enhancement algorithm.

[0056] The storage device 604 stores the first and second clean signals from the voice enhancer 603 as a dual-channel file. The dual-channel file may be a dual-channel PCM file (or dual-channel AAC / PCM file) representing the left and right channels. The storage device 604 may be a flash memory device.

[0057] Figure 7 is a block diagram showing an exemplary software architecture 706 that can be used in conjunction with the various hardware architectures described herein. Figure 7 is a non - limiting example of a software architecture, and it can be understood that many other architectures can be implemented to facilitate the functions described herein. The software architecture 706 can execute on hardware such as Figure 8 machine 8, which includes a processor 804, a memory 814, and I / O components 818. A representative hardware layer 752 is shown and can represent, for example, Figure 8 machine 800. The representative hardware layer 752 includes a processing unit 754 with associated executable instructions 704. The executable instructions 704 represent the executable instructions of the software architecture 706, including the implementation of the methods, components, etc. described herein. The hardware layer 752 also includes a memory or storage module memory / storage device 756, which also has executable instructions 704. The hardware layer 752 may also include other hardware 758.

[0058] As used herein, the term "component" may refer to a device, physical entity, or logic having boundaries defined by functions or subroutine calls, branch points, application programming interfaces (APIs), or other techniques that provide specific processing or control functions. Components can be combined with other components via their interfaces to perform machine processes. A component can be an encapsulated functional hardware unit designed to be used with other components and a part of a program that typically performs related functions.

[0059] Components can constitute software components (e.g., code embodied on a machine - readable medium) or hardware components. A "hardware component" is a tangible unit capable of performing certain operations and can be physically configured or arranged in some way. In various example embodiments, one or more computer systems (e.g., a stand - alone computer system, a client computer system, or a server computer system) or one or more hardware components of a computer system (e.g., a processor or a group of processors) can be configured by software (e.g., an application or a portion of an application) to operate as a hardware component that performs certain operations as described herein. A hardware component can also be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware component can include dedicated circuitry or logic that is permanently configured to perform certain operations.

[0060] A hardware component can be a dedicated processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). A hardware component can also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware component can include software executed by a general-purpose processor or other programmable processor. After being configured by such software, the hardware component becomes a particular machine (or a particular component of a machine) that is uniquely customized to perform the configured functions and is no longer a general-purpose processor. It should be understood that the decision to implement a hardware component mechanically in a dedicated and permanently configured circuit or in a temporarily configured circuit (e.g., configured by software) can be driven by cost and time considerations.

[0061] A processor can be or include any circuit or virtual circuit (a physical circuit simulated by logic executed on an actual processor) that manipulates data values in accordance with control signals (e.g., "commands", "opcodes", "machine code", etc.) and produces corresponding output signals applied to operate a machine. For example, a processor can be a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), or any combination thereof. A processor can further be a multi-core processor having two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously.

[0062] Accordingly, the phrase "hardware component" (or "hardware-implemented component") should be understood to include a tangible entity, i.e., an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate or perform certain operations as described herein. Considering embodiments in which a hardware component is temporarily configured (e.g., programmed), it is not necessary to configure or instantiate every hardware component at any one time. For example, in the case where a hardware component includes a general-purpose processor that is configured by software to become a dedicated processor, the general-purpose processor can be configured to be a corresponding different dedicated processor (e.g., including different hardware components) at different times. The software accordingly configures a particular one or more processors, e.g., to constitute a particular hardware component at one moment and a different hardware component at a different moment. A hardware component can provide information to and receive information from other hardware components. Accordingly, the described hardware components can be regarded as being communicatively coupled. In cases where multiple hardware components are present simultaneously, communication can be achieved through signal transmission between two hardware components or among more than two hardware components (e.g., via appropriate circuitry and buses). In embodiments in which multiple hardware components are configured or instantiated at different times, such communication between hardware components can be achieved, e.g., through the storage and retrieval of information in a memory structure accessible to the multiple hardware components.

[0063] For example, a hardware component can perform an operation and store the output of the operation in a storage device communicatively coupled thereto. Then, another hardware component can later access the storage device to obtain and process the stored output. The hardware component can also initiate communication with an input or output device and can operate on resources (e.g., a collection of information). The various operations of the example methods described herein can be performed, at least in part, by one or more processors temporarily configured (e.g., by software) or permanently configured to perform the associated operations. Whether temporarily or permanently configured, the processor can comprise a processor-implemented component for performing one or more operations or functions described herein. As used herein, a "processor-implemented component" refers to a hardware component implemented using one or more processors. Similarly, the methods described herein can be implemented, at least in part, by a processor, where a particular one or more processors are examples of hardware. For example, at least some of the operations of the method can be performed by one or more processors or processor-implemented components.

[0064] In addition, one or more processors can also operate to support the performance of associated operations in a "cloud computing" environment or as "software as a service" (SaaS). For example, at least some of the operations can be performed by a group of computers (as an example of a machine including processors), and these operations can be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application programming interfaces (APIs)). The performance of certain operations can be distributed among processors, not only residing within a single machine but also deployed across multiple machines. In some example embodiments, the processor or processor-implemented component can be located in a single geographical location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the processor or processor-implemented component can be distributed across multiple geographical locations.

[0065] In Figure 7 the exemplary architecture, the software architecture 706 can be conceptualized as a stack of layers, where each layer provides a specific function. For example, the software architecture 706 can include layers such as an operating system 702, libraries 720, applications 716, and a presentation layer 714. In operation, an application 716 or other component within a layer can call an application programming interface (API) API call 708 through the software stack and receive a message 712 in response to the API call 708. The layers shown are representative in nature, and not all software architectures have all layers. For example, some mobile or dedicated operating systems may not provide a framework / middleware 718, while other operating systems may provide such a layer. Other software architectures can include additional layers or different layers.

[0066] The operating system 702 can manage hardware resources and provide common services. The operating system 702 can include, for example, a kernel 722, services 724, and drivers 726. The kernel 722 can act as an abstraction layer between the hardware and other software layers. For example, the kernel 722 can be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, etc. The services 724 can provide other common services for other software layers. The drivers 726 are responsible for controlling the hardware or interfacing with the underlying hardware. For example, depending on the hardware configuration, the drivers 726 include a display driver, a camera driver, a driver, a flash drive, a serial communication driver (e.g., a Universal Serial Bus (USB) driver), a driver, an audio driver, a power management driver, and so on.

[0067] The library 720 provides a common infrastructure used by the application 916 or other components or layers. The library 720 provides functions that allow other software components to perform tasks in a way that is easier than directly interfacing with the underlying operating system 702 functions (e.g., the kernel 722, the services 724, or the drivers 726). The library 720 can include a system library 744 (e.g., a C standard library), which can provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. Additionally, the library 720 can include an API library 746, such as a media library (e.g., a library for supporting the rendering and manipulation of various media formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG), a graphics library (e.g., an OpenGL framework that can be used to render two-dimensional and three-dimensional graphics content on a display), a database library (e.g., SQLite that can provide various relational database functions), a web library (e.g., WebKit that can provide web browsing functions), etc. The library 720 can also include various other libraries 748 that provide many other APIs to the application 716 and other software components / modules.

[0068] The framework / middleware 718 (sometimes also referred to as middleware) provides a higher-level common infrastructure that can be used by the application 716 or other software components / modules. For example, the framework / middleware 718 can provide various Graphical User Interface (GUI) functions, advanced resource management, advanced location services, etc. The framework / middleware 718 can provide a wide range of other APIs that can be used by the application 716 or other software components / modules, some of which can be specific to a particular operating system 702 or platform.

[0069] The application 716 includes built-in applications 738 or third-party applications 940. Examples of representative built-in applications 738 may include, but are not limited to, a contacts application, a browser application, a book reader application, a location application, a media application, a messaging application, and / or a gaming application. Third-party applications 740 may include applications developed by entities other than the vendor of a particular platform using a software development kit (SDK), and may be mobile software that runs on a mobile operating system. Third-party applications 740 may invoke API calls 708 provided by the mobile operating system (such as operating system 702) to facilitate the functions described herein.

[0070] The application 716 may use built-in operating system features (e.g., kernel 722, services 724, or drivers 726), libraries 720, and frameworks / middleware 718 to create a user interface for interacting with a user of the system. Alternatively or additionally, in some systems, interaction with the user may occur through a presentation layer (such as presentation layer 714). In these systems, the application / component "logic" may be separated from the aspects of the application / component that interact with the user.

[0071] Figure 8 is a block diagram showing components (also referred to herein as "modules") of a machine 800 according to some exemplary embodiments, the machine 800 being capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and performing any one or more of the methods discussed herein. Specifically, Figure 8FIG. shows a graphical representation of a machine 800 in an example form of a computer system within which instructions 810 (e.g., software, program, application, applet, application software, or other executable code) can be executed to cause the machine 800 to perform any one or more of the methods discussed herein. Thus, the instructions 810 can be used to implement the modules or components described herein. The instructions 810 transform the general unprogrammed machine 800 into a particular machine 800 programmed to perform the described and illustrated functions in the described manner. In alternative embodiments, the machine 800 operates as a stand-alone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 800 can operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 800 can include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular telephone, a smartphone, a mobile device, a wearable device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of sequentially or otherwise executing the instructions 810 specifying the actions to be taken by the machine 800. Further, although only a single machine 800 is shown, the term "machine" shall also be taken to include a collection of machines that individually or jointly execute the instructions 1010 to implement any one or more of the methods discussed herein.

[0072] The machine 800 can include a processor 804, a memory / storage device 806, and I / O components 818 that can be configured to communicate with each other, such as via a bus 802. The memory / storage device 806 can include a memory 814, such as a main memory or other memory storage device, and a storage unit 816, both of which can be accessed by the processor 804, such as via the bus 802. The storage unit 816 and the memory 814 store the instructions 810 embodying any one or more of the methods or functions described herein. The instructions 810 can also reside, completely or partially, within the memory 814, within the storage unit 816, within at least one of the processors 804 (e.g., within a cache memory of the processor), or within any suitable combination thereof, during execution of the instructions on the machine 800. Thus, the memory 814, the storage unit 816, and the memory of the processor 804 are examples of machine-readable media.

[0073] As used herein, the terms "machine-readable medium", "computer-readable medium", etc. can refer to any component, device, or other tangible medium capable of storing instructions and data either temporarily or permanently. Examples of such media can include, but are not limited to, random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, caches, other types of storage (e.g., erasable programmable read-only memory (EEPROM)), or any suitable combination thereof. The term "machine-readable medium" should be considered to include a single medium or multiple media capable of storing instructions (e.g., a centralized or distributed database, or associated caches and servers). The term "machine-readable medium" can also be considered to include any medium or combination of media capable of storing instructions (e.g., code) executable by a machine such that the instructions, when executed by one or more processors of the machine, cause the machine to perform any one or more of the methods described herein. Thus, "machine-readable medium" can refer to a single storage device or apparatus, as well as a "cloud-based" storage system or storage network comprising multiple storage devices or apparatuses. The term "machine-readable medium" does not include the signal itself.

[0074] The I / O components 818 can include a variety of components to provide a user interface for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, etc. The specific I / O components 818 included in the user interface of a particular machine 800 will depend on the type of the machine. For example, a portable machine such as a mobile phone will likely include a touch input device or other such input mechanism, while a headless server machine will likely not include such a touch input device. It should be understood that the I / O components 818 can include Figure 8Many other components not shown. Merely for the sake of simplifying the following discussion, the I / O components 818 are grouped according to function, and the grouping is in no way restrictive. In various exemplary embodiments, the I / O components 818 may include an output component 826 and an input component 828. The output component 826 may include visual components (e.g., a display, such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), tactile components (e.g., a vibration motor, a resistive mechanism), other signal generators, and the like. The input component 828 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optoelectronic keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing instruments), tactile input components (e.g., physical buttons, a touch screen that provides the location or force of a touch or touch gesture, or other tactile input components), audio input components (e.g., a microphone), and the like. The input component 828 may also include one or more image capture devices, such as a digital camera for generating digital images or videos.

[0075] In further exemplary embodiments, the I / O components 818 may include a biometric component 830, a motion component 834, an environmental component 836, or a positioning component 838, as well as numerous other components. One or more such components (or portions thereof) may herein be collectively referred to as “sensor components” or “sensors” for collecting various data related to the machine 800, the environment of the machine 800, the user of the machine 800, or combinations thereof.

[0076] For example, the biometric component 830 may include components that detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body postures, or eye tracking), measure biometric signals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identify people (e.g., voice recognition, retina recognition, facial recognition, fingerprint recognition, or electroencephalogram-based recognition), etc. The motion component 834 may include an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a speed sensor component (e.g., a speedometer), a rotation sensor component (e.g., a gyroscope), etc. The environmental component 836 may include, for example, a lighting sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor for detecting the concentration of hazardous gases for safety or measuring pollutants in the atmosphere), or other components that may provide an indication, measurement, or signal corresponding to the surrounding physical environment. The location component 838 may include a position sensor component (e.g., a Global Positioning System (GPS) receiver component), an altitude sensor component (e.g., an altimeter or a barometer that can detect air pressure from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), etc. For example, the position sensor component may provide location information associated with the system 800, such as the GPS coordinates of the system 800 or information about the location where the system 1000 is currently located (e.g., the name of a restaurant or other business).

[0077] Various techniques can be used to implement communication. The I / O component 818 may include a communication component 840 that is operable to couple the machine 800 to the network 832 or the device 820 via the coupler 822 and the coupler 824, respectively. For example, the communication component 840 may include a network interface component or other suitable device to interface with the network 832. In a further example, the communication component 840 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, components (e.g., low power), components and other communication components that can provide communication via other modes. The device 820 may be another machine or any of a variety of peripheral devices (e.g., a peripheral device coupled via a Universal Serial Bus (USB)).

[0078] In addition, the communication component 840 can detect an identifier or include components operable to detect an identifier. For example, the communication component 840 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as universal product code (UPC) barcodes, multi-dimensional barcodes such as quick response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, hypercodes, UCC RSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying a tagged audio signal). In addition, various information can be derived via the communication component 840, such as a location via Internet protocol (IP) geolocation, a location via signal triangulation, a location via detecting an NFC beacon signal that can indicate a specific location, etc.

[0079] Figure 9 is a high-level functional block diagram of an example head-mounted device 100 communicatively coupled to a mobile device 800 and a server system 998 via various networks.

[0080] The device 100 includes a camera, such as at least one of a visible light camera 950, an infrared emitter 951, and an infrared camera 952. The camera can include a camera module having Figure 1 and 2 lenses 104_1, 104_2 therein.

[0081] The client device 800 can be capable of connecting to the device 100 using both a low-power wireless connection 925 and a high-speed wireless connection 937. The client device 800 is connected to the server system 998 and the network 995. The network 995 can include any combination of wired and wireless connections.

[0082] The device 100 further includes two image displays of the optical components 980A-B. The two image displays 980A-980B include one display associated with the left lateral side of the device 100 and one display associated with the right lateral side of the device 100. The device 100 also includes an image display driver 942, an image processor 912, a low-power circuit 920, and a high-speed circuit 930. The image displays of the optical components 980A-B are used to present images and videos to a user of the device 100, including images that can include a graphical user interface.

[0083] The image display driver 942 commands and controls the image displays of the optical assemblies 980A-B. The image display driver 942 may directly transfer image data to the image displays of the optical assemblies 980A-B for presentation, or may have to convert the image data into a signal or data format suitable for transfer to the image display device. For example, the image data may be video data formatted according to a compression format, such as H.264 (MPEG-4 Part 10), HEVC, Theora, Dirac, RealVideo RV40, VP8, VP9, etc., and the still image data may be formatted according to a compression format such as Portable Network Graphics (PNG), Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), or Exchangeable Image File Format (Exif).

[0084] As described above, the device 100 includes a frame 103 and rods (or foot wires) extending from the lateral sides of the frame 103. The device 100 further includes a user input device 991 (e.g., a touch sensor or button), which includes an input surface on the device 100. The user input device 991 (e.g., a touch sensor or button) receives input selections from the user to manipulate the graphical user interface of the presented image.

[0085] Figure 9 The components for the device 100 shown in are located on one or more circuit boards, such as a PCB or a flexible PCB, in the spectacle frame or the foot wires. Alternatively or additionally, the depicted components may be located in the body, frame, hinge, or center beam of the device 100. The left and right visible light cameras 950 may include digital camera elements, such as complementary metal oxide semiconductor (CMOS) image sensors, charge-coupled devices, lenses 104_1, 104_2, or any other corresponding visible light or light-capturing elements that can be used to capture data, including images of scenes with unknown objects.

[0086] The device 100 includes a memory 934 that stores instructions to perform a subset or all of the functions described herein for generating binaural audio content. The memory 934 may also include a storage device 604. Figure 4 The exemplary processes shown in the flowchart of can be implemented in the instructions stored in the memory 934.

[0087] As Figure 9As shown, high-speed circuit 930 includes a high-speed processor 932, a memory 934, and a high-speed wireless circuit 936. In this example, an image display driver 942 is coupled to the high-speed circuit 930 and is operated by the high-speed processor 932 to drive the left and right image displays of the optical components 980A-B. The high-speed processor 932 can be any processor capable of managing the high-speed communication and operation of any general computing system required by the device 100. The high-speed processor 932 includes the processing resources required to manage high-speed data transmission over a high-speed wireless connection 937 to a wireless local area network (WLAN) using the high-speed wireless circuit 936. In certain examples, the high-speed processor 932 executes an operating system, such as the LINUX operating system or other such operating systems of the device 100, and the operating system is stored in the memory 934 for execution. In addition to any other duties, the high-speed processor 932 that executes the software architecture of the device 100 is used to manage data transmission with the high-speed wireless circuit 936. In certain examples, the high-speed wireless circuit 936 is configured to implement the Institute of Electrical and Electronics Engineers (IEEE) 802.11 communication standard, also known herein as Wi-Fi. In other examples, other high-speed communication standards can be implemented by the high-speed wireless circuit 936.

[0088] The low-power wireless circuit 924 and the high-speed wireless circuit 936 of the device 100 may include a short-range transceiver (Bluetooth TM ) and a wireless wide area network, local area network, or wide area network transceiver (e.g., cellular or WiFi). The client device 800 (including transceivers that communicate via the low-power wireless connection 925 and the high-speed wireless connection 937) can be implemented using the details of the architecture of the device 100, as can other elements of the network 995.

[0089] The memory 934 includes any storage device capable of storing various data and applications, including camera data generated by the left and right visible light cameras 950, the infrared camera 952, and the image processor 912, and images generated by the image display driver 942 for display on the image displays of the optical components 980A-B. Although the memory 934 is shown as integrated with the high-speed circuit 930, in other examples, the memory 934 can be a separate, stand-alone element of the device 100. In certain such examples, electrical wiring can provide a connection from the image processor 912 or the low-power processor 922 to the memory 934 through a chip that includes the high-speed processor 932. In other examples, the high-speed processor 932 can manage the addressing of the memory 934 such that whenever a read or write operation involving the memory 934 is required, the low-power processor 922 will initiate the high-speed processor 932.

[0090] As Figure 9As shown, the processor 932 of the device 100 can be coupled to a camera (visible light camera 950; infrared emitter 951, or infrared camera 952), an image display driver 942, a user input device 991 (e.g., a touch sensor or a button), and a memory 934.

[0091] The device 100 is connected to a host computer. For example, the device 100 is paired with the client device 800 via a high-speed wireless connection 937 or connected to the server system 998 via a network 995. The server system 998 can be one or more computing devices that are part of a service or network computing system. For example, it includes a processor, a memory, and a network communication interface to communicate with the client device 800 and the device 100 via the network 995.

[0092] The client device 800 includes a processor and a network communication interface coupled to the processor. The network communication interface allows communication via the network 925 or 937. The client device 800 can further store at least a portion of the instructions for generating binaural audio content in the memory of the client device 800 to implement the functions described herein.

[0093] The output components of the device 100 include visual components, such as a display, such as a liquid crystal display (LCD), a plasma display panel (PDP), a light-emitting diode (LED) display, a projector, or a waveguide. The image display of the optical component is driven by the image display driver 942. The output components of the device 100 further include acoustic components (e.g., speakers), tactile components (e.g., vibration motors), other signal generators, etc. The input components (such as the user input device 991) of the device 100, the client device 800, and the server system 998 can include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing instruments), tactile input components (e.g., physical buttons, a touch screen that provides the position and force of a touch or touch gesture, or other tactile input components), audio input components (e.g., a microphone), etc.

[0094] The device 100 can optionally include additional peripheral device elements. Such peripheral device elements can include biometric sensors, additional sensors, or display elements integrated with the device 100. For example, the peripheral device elements can include any I / O components, including output components, motion components, position components, or any other such elements described herein.

[0095] For example, biometric components include those for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identifying people (e.g., voice recognition, retina recognition, facial recognition, fingerprint recognition, or electroencephalogram-based recognition), etc. Motion components include acceleration sensor components (e.g., accelerometers), gravity sensor components, rotational sensor components (e.g., gyroscopes), etc. Location components include location sensor components for generating position coordinates (e.g., global positioning system (GPS) receiver components), WiFi or Bluetooth TM transceivers, altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude can be derived), orientation sensor components (e.g., magnetometers), etc. Such location system coordinates can also be received from the client device 800 via the low-power wireless circuit 924 or the high-speed wireless circuit 936 through the wireless connections 925 and 937.

[0096] In cases where phrases similar to "at least one of A, B, or C", "at least one of A, B, and C", "one or more of A, B, or C", or "one or more of A, B, and C" are used, the phrase is intended to be interpreted as meaning that A can exist alone in an embodiment, B can exist alone in an embodiment, C can exist alone in an embodiment, or any combination of elements A, B, and C can exist in a single embodiment; for example, A and B, A and C, B and C, or A and B and C.

[0097] Changes and modifications can be made to the disclosed embodiments without departing from the scope of the present disclosure. These and other changes or modifications are intended to be included within the scope of the present disclosure as expressed in the following claims.

Claims

1. A head-mounted device, comprising: A frame that carries a pair of lenses and is coupled to a camera, wherein the camera lens of the camera faces forward; A first rod that is coupled to a first side of the frame and is coupled to a first microphone housing that encloses a first front microphone and a first rear microphone that generate acoustic signals, the first microphone housing including a first front port facing downward and a first rear port facing rearward, A second rod that is coupled to a second side of the frame and is coupled to a second microphone housing that encloses a second front microphone and a second rear microphone that generate acoustic signals, the second microphone housing including a second front port facing downward and a second rear port facing rearward; and A binaural audio processor that includes A beamformer to generate A first beamformer signal based on acoustic signals from the first front microphone and the first rear microphone, and A second beamformer signal based on acoustic signals from the second front microphone and the second rear microphone.

2. The head-mounted device according to claim 1, wherein The beamformer receives acoustic signals from the first front microphone, the first rear microphone, the second front microphone, and the second rear microphone.

3. The head-mounted device according to claim 2, wherein The wearable device is a pair of glasses, wherein the first side of the frame is opposite the second side of the frame, and wherein when the user wears the pair of glasses, the first front port and the second front port face downward toward the user's feet, and the first rear port and the second rear port face the back of the user's head.

4. The head-mounted device according to claim 3, wherein, The frame carries a pair of lenses and is coupled to a camera, wherein the camera lens of the camera faces forward.

5. The head-mounted device according to claim 1, wherein, The beamformer is a fixed beamformer, wherein the fixed beamformer includes a sub-cardioid or cardioid fixed beam pattern.

6. The head-mounted device according to claim 1, wherein, The first front microphone and the first rear microphone form a first-order differential microphone array, and the second front microphone and the second rear microphone form a first-order differential microphone array.

7. The head-mounted device according to claim 1, wherein, The first beamformer signal and the second beamformer signal are stored as a dual-channel file on a storage device, wherein the storage device includes a flash memory device.

8. The head-mounted device according to claim 1, wherein The binaural audio processor is a system-on-chip (SoC).

9. The head-mounted device according to claim 1, further comprising: A first audio codec that decodes acoustic signals from the first front microphone and the first rear microphone to generate a first decoded acoustic signal; A second audio codec that decodes acoustic signals from the second front microphone and the second rear microphone to generate a second decoded acoustic signal.

10. The head-mounted device according to claim 9, further comprising: A time-division multiplexer (TDM) that processes the first decoded acoustic signal and the second decoded acoustic signal and generates a TDM-processed signal.

11. The head-mounted device according to claim 10, wherein, The acoustic signals generated by the first front microphone and the second front microphone and the first rear microphone and the second rear microphone are pulse density modulation (PDM) signals.

12. The head-mounted device according to claim 11, wherein, The first decoded acoustic signal and the second decoded acoustic signal are Pulse Code Modulation (PCM) signals.

13. The head-mounted device according to claim 12, wherein, The binaural audio processor further comprises: a noise suppressor that suppresses noise from the first beamformer signal and the second beamformer signal and generates a first noise-suppressed signal and a second noise-suppressed signal.

14. The head-mounted device according to claim 13, wherein, The binaural audio processor further comprises: a voice enhancer that enhances the voice from the first noise-suppressed signal and the second noise-suppressed signal to generate a first clean signal and a second clean signal, wherein the storage device stores the first clean signal and the second clean signal.

15. A pair of glasses, comprising: a frame that carries a pair of lenses; a camera coupled to the frame, wherein a camera lens of the camera faces a forward direction; a first rod coupled to a first side of the frame and coupled to a first microphone housing that encloses a first front microphone and a first rear microphone that generate acoustic signals, the first microphone housing including a first front port facing a downward direction and a first rear port facing a backward direction, a second rod coupled to a second side of the frame and coupled to a second microphone housing that encloses a second front microphone and a second rear microphone that generate acoustic signals, the second microphone housing including a second front port facing the downward direction and a second rear port facing the backward direction; and a binaural audio processor that includes a beamformer that generates a first beamformer signal based on acoustic signals from the first front microphone and the first rear microphone, and a second beamformer signal based on acoustic signals from the second front microphone and the second rear microphone.

16. The pair of glasses according to claim 15, wherein, The binaural audio processor further comprises: a noise suppressor that suppresses noise from the first beamformer signal and the second beamformer signal and generates a first noise-suppressed signal and a second noise-suppressed signal.

17. The pair of glasses according to claim 15, wherein, The beamformer is a fixed beamformer, wherein the fixed beamformer includes a fixed beam pattern that is sub-cardioid or cardioid.

18. The pair of glasses according to claim 15, further comprising: a first audio codec that decodes acoustic signals from the first front microphone and the first rear microphone to generate a first decoded acoustic signal; a second audio codec that decodes acoustic signals from the second front microphone and the second rear microphone to generate a second decoded acoustic signal; and a Time Division Multiplexer (TDM) that processes the first decoded acoustic signal and the second decoded acoustic signal and generates a TDM-processed signal.

19. A pair of glasses according to claim 16, wherein, The binaural audio processor further comprises: a voice enhancer that enhances the voice from the first noise-suppressed signal and the second noise-suppressed signal to generate a first clean signal and a second clean signal, wherein the storage device stores the first clean signal and the second clean signal.

20. A method for generating binaural audio content using a head-mounted device, comprising: Generating acoustic signals by a first front microphone and a first rear microphone, wherein the first front microphone and the first rear microphone are enclosed in a first microphone housing, and the first microphone housing is coupled to a first rod of the head-mounted device; Generating acoustic signals by a second front microphone and a second rear microphone, wherein the second front microphone and the second rear microphone are enclosed in a second microphone housing, and the second microphone housing is coupled to a second rod of the head-mounted device, wherein the first rod and the second rod are coupled to opposite sides of a frame of the head-mounted device; Decoding the acoustic signals from the first front microphone and the first rear microphone by a first audio codec to generate a first decoded acoustic signal; Decoding the acoustic signals from the second front microphone and the second rear microphone by a second audio codec to generate a second decoded acoustic signal; Processing the first decoded acoustic signal and the second decoded acoustic signal by a time-division multiplexer (TDM) to generate a TDM-processed signal, wherein the TDM-processed signal includes the first decoded acoustic signal and the second decoded acoustic signal; Performing beamforming on the TDM-processed signal by a beamformer, wherein performing beamforming on the TDM-processed signal includes Performing beamforming on the first decoded acoustic signal to generate a first beamformer signal, and Performing beamforming on the second decoded acoustic signal to generate a second beamformer signal.

21. The method according to claim 20, wherein, The first decoded acoustic signal is a pulse code modulation (PCM) signal based on the acoustic signals from the first front microphone and the first rear microphone; and The second decoded acoustic signal is a PCM signal based on the acoustic signals from the second front microphone and the second rear microphone.

Images