Hybrid speakers and converters

Abstract

An audio converter system is provided. The system includes: an audio input configured to receive source audio; an audio output configured to be coupled to a hybrid speaker including at least two non-directional speakers and a directional speaker; and a processor configured to generate output audio of the hybrid speaker based on the source audio by: identifying a specific sound in the source audio; isolating the specific sound from the source audio; generating a directional speaker output of the directional speaker in the hybrid speaker based on the specific sound; and generating at least two channels of non-directional speaker output for the at least two non-directional speakers in the hybrid speaker.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a continuation-in of and claims the benefit of U.S. patent application No. 16 / 279,896, filed on February 19, 2019, entitled “HYBRID SPEAKER AND TRANSDUCER,” the entire contents and disclosure of which are hereby fully incorporated herein by reference in their entirety. Background of the Invention

[0003] 1. Field of the Invention

[0004] The present invention relates generally to audio processing and, more particularly, to generating audio signals for directional speakers.

[0005] 2. Related technical description

[0006] A loudspeaker (or speaker) is an electroacoustic transducer that converts an electrical audio signal into a corresponding sound. For example, in a traditional dynamic cone speaker, a coil of wire is suspended in a circular gap between the poles of a permanent magnet. When an alternating current audio signal is applied, the coil is forced to move back and forth rapidly due to Faraday's law of induction, which causes the diaphragm attached to the coil to move back and forth, pushing the air to disperse the sound waves around the diaphragm. Summary of the invention

[0007] One embodiment provides a system comprising: an audio input configured to receive source audio; an audio output configured to couple to a hybrid speaker including at least two non-directional speakers and a directional speaker; and a processor configured to generate output audio for the hybrid speaker based on the source audio by: identifying a specific sound in the source audio, isolating the specific sound from the source audio, generating a directional speaker output for a directional speaker in the hybrid speaker based on the specific sound, and generating at least two channels of non-directional speaker output for at least two non-directional speakers in the hybrid speaker.

[0008] Another embodiment provides a method comprising: identifying a specific sound in source audio using a processor; isolating the specific sound from the source audio; generating a directional speaker output for a directional speaker in a mixed speaker based on the specific sound; and generating at least two channels of non-directional speaker output for at least two non-directional speakers in the mixed speaker.

[0009] Another embodiment provides an audio speaker device, comprising: a speaker housing; a left non-directional speaker and a right non-directional speaker enclosed in the speaker housing; and a directional speaker, the directional speaker being located between the left non-directional speaker and the right non-directional speaker and enclosed in the speaker housing.

[0010] A better understanding of the features and advantages of the various embodiments of the present invention will be obtained by reference to the following detailed description and accompanying drawings, which set forth illustrative embodiments in which the principles of the various embodiments of the present invention are utilized. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other aspects, features and advantages of embodiments of the present invention will become more apparent from the following more particular description presented in conjunction with the following drawings, in which:

[0012] Figure 1 is a block diagram illustrating a system for driving a hybrid speaker according to some embodiments of the present invention;

[0013] Figure 2 is a block diagram illustrating a system for converting audio for mixed speakers according to some embodiments of the present invention;

[0014] Figure 3 is a flow chart illustrating a method for converting source video for mixed speakers according to some embodiments of the present invention; and

[0015] Figure 4A and Figure 4B is an illustration of a directional speaker with multiple panels according to some embodiments of the present invention. DETAILED DESCRIPTION

[0016] Directional speakers generally refer to devices that produce less sound field than traditional speakers. In some embodiments, directional speakers may include parametric array speakers that generate sound from ultrasound. Directional speaker devices achieve high directivity by modulating audible sound onto high-frequency ultrasound. Higher frequency sound waves have shorter wavelengths and therefore do not propagate as quickly, resulting in higher directivity than traditional non-directional speakers.

[0017] In some embodiments described herein, a directional speaker is combined with one or more traditional non-directional speakers to simulate immersive sound and / or selectively project selected sounds to specific users in a room. In some embodiments, a directional speaker and a non-directional speaker can be packaged in the same housing to provide a hybrid soundbar. In some embodiments, a converter is provided to convert a traditional audio file and / or stream into a signal for driving a hybrid speaker. The converter can be configured to identify and isolate a specific sound to be reproduced by a directional speaker and generate the output of the directional speaker and the non-directional speaker in the speaker assembly.

[0018] First reference Figure 1, a system for driving a hybrid speaker is shown. The system includes an audio source 110, an audio converter 120, and a hybrid speaker assembly 100 for generating the sound of a user 130.

[0019] Audio source 110 generally refers to a device that outputs sound to be played at one or more speakers. In some embodiments, audio source 110 may include a playback device, such as a television, a cable box, a media player, a hard drive, a media storage device, a home entertainment system, a computer, a game console, a mobile device, etc. Audio source 110 can output mono, stereo, and / or surround sound. In some embodiments, audio source 110 can output one or more non-directional audio channels. In some embodiments, audio source 110 can output any number of audio channels, such as two channels (e.g., left / right stereo), three channels (e.g., center, left, and right), six channels (e.g., 5.1 surround), eight channels (e.g., 7.1 surround), etc.

[0020] The audio converter 120 includes a device configured to convert an audio signal from an audio source 110 into a signal configured to drive the directional and non-directional speakers of the hybrid speaker assembly 100. In some embodiments, the audio converter 120 is configured to take one or more traditional audio channels and generate at least one output channel of the directional speaker 103. In some embodiments, the audio converter 120 includes a processor configured to: identify a specific sound in the source audio; isolate the specific sound from the source audio; generate a directional speaker output of a directional speaker in the hybrid speaker based on the specific sound; and generate at least two channels of non-directional speaker output for at least two non-directional speakers in the hybrid speaker. In some embodiments, the audio converter 120 may also include a sound database for identifying specific sounds in the source audio. In some embodiments, the sound database may include a cloud database shared between multiple audio converters 120 and accessible via a network adapter on the audio converter 120, the network adapter being configured to communicate over a network such as a home network and / or the Internet. In some embodiments, the audio converter 120 may be configured to run a machine learning algorithm to identify specific sounds based on a growing and evolving sound learning set stored in the sound database. References Figure 2 and Figure 3 Additional details of the audio converter 120 are provided.

[0021] The hybrid speaker assembly 100 generally refers to a speaker device including at least one directional speaker and one or more non-directional speakers. Figure 1In the illustrated embodiment, the hybrid speaker includes a left non-directional speaker and a right non-directional speaker enclosed in a speaker housing and a directional speaker located between the left non-directional speaker and the right non-directional speaker. In some embodiments, the hybrid speaker assembly 100 can include a form factor of a sound bar. In some embodiments, the hybrid speaker assembly 100 can include two or more separately movable housings.

[0022] The left speaker 101 and the right speaker 102 include non-directional speakers. A non-directional speaker generally refers to a traditional speaker with a large propagation range and low directivity. For example, the sound produced by a non-directional speaker can generally be heard similarly by a listener positioned in any direction relative to the speaker. In some embodiments, the left speaker 101 and the right speaker 102 may include a dynamic speaker including a diaphragm connected to a frame via a flexible suspension constraining a voice coil. In some embodiments, the left speaker 101 and the right speaker 102 may include speaker elements similar to speaker elements in a traditional sound bar. In some embodiments, the hybrid speaker assembly 100 may include additional non-directional speakers, such as a center speaker, a subwoofer, and the like.

[0023] Directional speakers 103 generally refer to speaker devices with narrow propagation range and high directivity. Typically, the sound projected by a directional speaker can only be heard in a specific direction relative to the speaker. In some embodiments, the directional speaker 103 can be configured to carry audible sound waves on inaudible high-frequency waveforms (e.g., ultrasound). In some embodiments, the directional speaker 103 can include a parametric array speaker that generates sound from ultrasound.

[0024] In some embodiments, the directional speaker 103 may include multiple fixed panels. For example, the directional speaker 103 may include two or more fixed parameter array panels that are angled relative to each other. Figure 4A An example of a two-panel directional speaker configuration is shown in Figure 4B An example of a three-panel directional speaker configuration is shown in . The panels of the directional speaker 103 can be controlled individually to affect the overall projection direction of the directional speaker 103. For example, the output of each panel can be modulated so that the projection direction of the directional speaker 103 sweeps through multiple angles in the projection area to selectively direct the sound to different intended directions. In some embodiments, the scanning can be performed at a speed that is recognizable by the human ear. In some embodiments, the speed scanning can be between 0.01 Hz and 10 Hz or faster.

[0025] The projection angle controlled by modulating the output of different panels can also be used to direct sound to two or more listeners sitting separately and / or create a simulated surround sound effect by projecting different sounds to the left and right ears of the listeners. In some embodiments, each panel of the directional speaker 103 can be driven by a separate audio stream output from the audio converter 120. In some embodiments, the hybrid speaker assembly 100 and / or the audio converter 120 can be configured to convert audio and projection direction data into the audio output of each panel of the directional speaker 103. For example, sound modulation can dynamically process the sound signal over time using phase shift and / or amplitude control to provide a sound effect by scanning the signal from left to right / right to left.

[0026] In some embodiments, the directional speaker 103 may include a mechanical rotor configured to physically rotate the directional speaker 103 to control the projection direction of the speaker. For example, the rotor can turn the parameter array to the user 130 to guide the sound to the user. In some embodiments, the rotor can sweep the projection direction of the directional speaker 103 across the projection area, and the directional speaker 103 can only produce the sound of the expected angle during the rotation, so that it can be perceived that a single directional speaker projects the sound in multiple discrete or continuous directions. For example, the directional speaker 103 can guide different sounds to different listeners sitting separately in the room by timing the output audio to the rotation. In another example, the directional speaker 103 can guide different sounds to the left and right ears of the user to simulate surround sound or immersive sound. In some embodiments, the directional speaker 103 is driven by a signal including audio and projection direction data associated with one or more time portions of the audio, and the rotor can be controlled by the projection direction data associated with the audio signal.

[0027] Although only one directional speaker 103 is shown in the hybrid speaker assembly 100, in some embodiments, the hybrid speaker assembly 100 may include multiple directional speakers 103 driven by different inputs. For example, the hybrid speaker may include two or more parameter arrays mounted on separate rotors and / or multiple sets of parameter array panels. Although the audio transducer 120 is shown as separate from the hybrid speaker assembly 100, in some embodiments, the audio transducer 120 can be integrated into the housing of the hybrid speaker assembly 100 as a physical device. In some embodiments, the audio transducer 120 can be local to the hybrid speaker assembly 100 and / or include a cloud-based and / or server-based transducer. In some embodiments, the audio transducer 120 can be integrated with an audio source 110 such as a home entertainment console, a game console, a computer system, etc. The audio source 110 can selectively engage the audio transducer 120 based on the audio format (e.g., traditional audio or hybrid speaker-ready audio format) and / or the speaker device (e.g., traditional speaker or hybrid speaker) coupled to the audio source 110.

[0028] pass Figure 1 In the system shown, a traditional audio source 110 can be converted into a signal for driving a hybrid speaker assembly 100 including a directional speaker 103 by an audio converter 120. For a single user 130, the directional speaker 103 can direct different sounds to the left and right sides of the user, and the sound is mixed with the output from the traditional speaker to simulate surround sound and / or immersive sound effects. In some embodiments, the directional speaker 103 can be configured to project sounds that only specific listeners can hear in the projection area, while the traditional speaker generates sounds that all listeners can hear. For example, in a multiplayer game, different specific sounds (e.g., voice instructions, status alarms, warnings) can be projected to different players, while ambient sounds or background music are played through non-directional speakers.

[0029] In some embodiments, the hybrid speaker assembly 100 can be used without the audio converter 120. In some embodiments, the audio source 110 can output audio configured for the hybrid speaker assembly 100 including directional and non-directional speaker signals, and the audio converter 120 can be omitted. For example, a hybrid speaker-compatible gaming console can output selected sound files (e.g., voice commands, status alerts, warnings) through directional speaker channels and output the remaining sound files (e.g., background music) in its audio through one or more non-directional speaker channels. In another example, pre-recorded audio (e.g., movies, music, etc.) can be recorded with at least one directional audio channel, which can be used directly by the hybrid speaker without conversion.

[0030] Next reference Figure 2 , a system for converting an audio source for use with a hybrid speaker is shown. The audio converter 200 is generally configured to obtain a signal from an audio source 210 and output the signal to a non-directional speaker 221 and a directional speaker 222 of a hybrid speaker assembly 220. In some embodiments, the audio source 210, the audio converter 200, and the hybrid speaker assembly 220 may include the audio source 210, the audio converter 200, and the hybrid speaker assembly 220. Figure 1 An audio source 110, an audio transducer 120, and a hybrid speaker assembly 100 are depicted.

[0031] The audio converter 200 includes an audio input 201 , a recognition module 202 , a sound database 203 , a sound filter 204 , a non-directional speaker output 205 , a projection direction module 206 and a directional speaker output 208 .

[0032] The audio input 201 is generally configured to receive an analog or digital audio signal from an audio source 210. The audio input 201 may include a wired and / or wireless connector, such as an audio connector, an audio / video connector, an HDMI connector, a USB connector, a Bluetooth transceiver, a wireless HDMI transceiver, etc. In some embodiments, the audio input 201 may also include other components for filtering, converting, and / or enhancing the input signal for the rest of the processing circuitry.

[0033] Identification module 202 is generally configured to identify a specific sound in input audio to be played by a directional speaker. Identification module 202 can be implemented on a processor, a control circuit, an application specific integrated circuit (ASIC) and / or one or more digital or analog audio filter components. In some embodiments, a specific sound is identified by comparing the source audio with the sound fragments and / or sound profiles in the sound database 203. For example, the sound database 203 may include sound fragments and / or sound profiles (e.g., spectral characteristics, frequency distribution, time frequency or amplitude variation, etc.) associated with various types of sounds suitable for playing via a directional speaker. Examples of this sound may include car sounds, bird sounds, ballistic throwing sounds, human voices, alarms, warning sounds, etc. In some embodiments, identification module 202 may include an audio event detection algorithm utilizing a convolutional neural network (CNN), a recurrent neural network (RNN) and / or a generative adversarial network (GAN), which identifies a specific sound using the sound in the sound database 203 as a learning set by machine learning. In some embodiments, the audience response may be used as feedback to the machine learning algorithm. In some embodiments, other types of feedback for machine learning may include camera input, face detection / tracking, and / or object detection / tracking. In some embodiments, user and environmental feedback may be used for direction and volume control. In some embodiments, specific sounds may be identified based on frequency, amplitude, and / or duration. For example, the sound recognition module 202 may be configured to detect a sound with high frequency and short duration as a specific sound for playback by a directional speaker. In some embodiments, a specific sound may be identified based on at least one of the sound frequency and amplitude difference between two or more channels of the source audio. For example, if the sound in the sound stream is larger between the left or right speaker channels (e.g., with an interaural intensity difference (IID) or a time delay (e.g., with an interaural time difference (ITD)), the recognition module may identify the sound associated with a significant IID or ITD to be played on a directional speaker. In some embodiments, the sound may be initially identified based on IID and / or ITD and added to the sound database 203. The recognition module 202 may then use the sound samples in the sound database 203 to identify similar sounds that may not have significant IID and / or ITD in the source audio.

[0034] In some embodiments, sound database 203 may include local and / or network-based databases storing audio segments and / or sound profiles. In some embodiments, sound database 203 may be maintained by a central service shared between a plurality of audio converters 200 associated with different users and different spaces. In some embodiments, sound database 203 may be updated and pruned by using the audio and feedback received at a plurality of audio converters by, for example, the machine learning algorithm of a generative adversarial network (GAN). In some embodiments, audio converter 200 may include a network interface (such as a network adapter, a wireless transceiver, etc.) for communicating with network-based sound database 203. In some embodiments, at least a portion of the functions of recognition module 202 and sound filter 204 may be performed by a remote server via a network interface.

[0035] The sound filter 204 is configured to isolate the specific sound identified by the recognition module 202 from other sounds in the audio source 210. In some embodiments, the sound filter 204 can use time filtering, amplitude filtering and / or frequency filtering such as high, low or bandpass filtering to isolate the sound. In some embodiments, the sound filter 204 can use the difference between the different channels of the audio source 210 to isolate the specific sound. For example, if a high ITD or IID is detected between the channels of the input audio stream, one channel can be used to isolate the specific sound in another channel by subtraction or masking. In some embodiments, filtering can only approximately isolate the specific sound; since the sound of the directional speaker will be mixed with the sound of the non-directional speaker during playback, incomplete isolation may not significantly affect the user experience. In some embodiments, the sound of the matching specific sound from the sound database 203 can be used to filter the sound and / or recreate the specific sound. The sound filter 204 then provides the specific sound isolated from the input audio to the projection direction module 206 and the directional speaker output 208. The sound filter 204 can also subtract specific sounds from one or more channels of the source audio and output the modified non-directional audio signal to the non-directional speaker 221 of the mixing speaker. In some embodiments, the source audio can be provided to the non-directional speaker 221 without modification.

[0036] The non-directional speaker output 205 is generally configured to couple to the hybrid speaker assembly 220 and provide an audio signal for playback at the non-directional speaker 221. In some embodiments, the non-directional speaker output 205 may include a wired and / or wireless connector, such as an audio connector, an audio / video connector, an HDMI connector, a USB connector, a Bluetooth transceiver, a wireless HDMI transceiver, etc. In some embodiments, the non-directional speaker output 205 may also include other components for providing signals to the speakers, such as amplifiers, digital-to-analog converters, etc. In some embodiments, the non-directional speaker output 205 may output multiple channels via a single-wire, multi-wire, or wireless connection, each channel corresponding to a non-directional speaker (e.g., a right speaker, a left speaker) of the hybrid speaker assembly 220.

[0037] The projection direction module 206 is configured to determine the projection direction associated with the output to the directional speaker 222. In some embodiments, the projection direction module 206 determines the projection direction based on one or more of the time delay or / or amplitude difference between the user position and two or more channels of the source audio. In some embodiments, the projection direction can be determined based on the user position detected by the user sensor 224 on the hybrid speaker assembly 220 and / or the audio converter 200, so that the directional speaker projects a specific sound toward one or more users. In some embodiments, the user position and / or the projection area can be determined by a speaker calibration process. In some embodiments, the projection direction module 206 can be determined based on the amplitude (e.g., loudness) difference of a specific sound in two or more channels (IID) of the source audio and / or the time delay difference of a specific sound between two or more channels of the source audio (ITD). For example, if the specific sound of the left channel is significantly larger, the projection direction of the directional speaker 222 can be tilted to the left. In another example, if the amplitude and timing of the specific sound in the two channels are substantially the same, the projection direction of the directional speaker 222 can point to the center. In some embodiments, the projection direction module 206 can determine the 2D soundscape to be output by the directional speaker 222, and determine the sound to be output at each angle when the directional speaker 222 sweeps the projection area of ​​the 2D soundscape. In some embodiments, the directional speaker 222 can be configured to sweep the soundscape space in a fixed pattern, and the projection direction module 206 is configured to time the audio signal output to the scanning position of the directional speaker 222 to output the 2D soundscape. In some embodiments, an optical sensor can be used to detect the user position and the projection direction can be determined using user detection / tracking information. In some embodiments, the user position can be determined by a calibration process and / or manually adjusted by the user.

[0038] The directional speaker output 208 is generally configured to couple to a hybrid speaker and provide a signal for playing at the directional speaker 222 of the hybrid speaker assembly 220. In some embodiments, the directional speaker output 208 may include a wired and / or wireless connector, such as an audio connector, an audio / video connector, an HDMI connector, a USB connector, a Bluetooth transceiver, a wireless HDMI transceiver, etc. In some embodiments, the directional speaker input 201 may also include other components for providing signals to the speaker, such as an amplifier, a digital-to-analog converter, etc. In some embodiments, the output signal may include an audio signal synchronized with the scanning of the directional speaker 222. In some embodiments, the output signal may include one or more projection angles, each projection angle being associated with a time portion of the audio stream. For example, the audio output may specify that the first 2 nanoseconds of audio are to be projected at 35 degrees and the next 3 nanoseconds of audio are to be projected at 50 degrees. In some embodiments, the output signal may include an audio stream and a directional speaker movement instruction timed to the audio stream. For example, the directional speaker movement instruction may specify a scanning speed, a scanning range, etc. In some embodiments, for a directional speaker 222 with a mechanical rotor, the audio converter may output a rotor control signal separately from the audio stream. In some embodiments, for a directional speaker 222 having multiple fixed panels, the output signal may include a separate audio channel to be output at each panel of the directional speaker 222, which is modulated to control the projection direction of the directional speaker 222. In some embodiments, the output signal may include a 2D soundscape for playback by the directional speaker 222, and the hybrid speaker assembly 220 may include a processor configured to control the projection direction and audio output of the directional speaker 222 to reproduce the 2D soundscape.

[0039] In some embodiments, the outputs from the non-directional speaker output 205 and the directional speaker output 208 are timed so that when the sound from each speaker device reaches the listener, the outputs from the non-directional speaker 221 and the directional speaker 222 are synchronized and mixed. In some embodiments, machine learning, such as GANs, can be applied to selectively mix and synchronize the outputs to the non-directional speaker 221 and the directional speaker 222 using the sound database 203.

[0040] In some embodiments, one or more functions of the projection direction module 206 described herein may be performed alternatively by the hybrid speaker assembly 220. For example, in an embodiment where the directional speaker 222 is configured to project only specific sounds based on the user's position, the directional speaker output 208 may output only specific sounds and the hybrid speaker assembly 220 may determine the projection direction based on the user's position detected by the user sensor 224. In this case, the user sensor 224 may not provide the user's position information to the audio transducer 200 and the projection direction module 206 may be omitted on the audio transducer 200.

[0041] The user sensor 224 is generally configured to detect the position of one or more listeners near the directional speaker. In some embodiments, the user sensor 224 may include an optical sensor, a microphone, an acoustic sensor, an ultrasonic sensor, a thermal sensor, etc. In some embodiments, the user position can be determined by detecting the sound reflection from the sound produced by the directional speaker. For example, the directional speaker 222 can scan the projection area with directional sound, and the user sensor 224 including a microphone can measure the timing of the sound reflection at each angle to determine the presence and / or position of the user.

[0042] In some embodiments, the connection between the audio converter 200 and the hybrid speaker assembly 220, although shown with multiple arrows, can include a single wire or wireless connection. In some embodiments, the audio converter 200 and the hybrid speaker assembly 220 can be an integrated device encapsulated in a single housing. In some embodiments, the audio converter 200 can be integrated with an audio source 210 that selectively engages the audio converter based on the audio format and / or speaker type.

[0043] Next reference Figure 3 , showing an example of a method for converting audio for mixed speakers. In some embodiments, Figure 3 The steps shown in can be performed by one or more of an audio converter, a hybrid speaker controller, a computer system, and a server. In some embodiments, Figure 3 One or more of the steps shown in may be replaced by reference Figure 1 The audio converter 120 and / or hybrid speaker assembly 100 described herein and / or referenced herein Figure 2 The audio converter 200 and / or the hybrid speaker assembly 220 described herein are implemented. Figure 3 The steps in may be performed by one or more processor-based devices including a control circuit and a memory storing a set of computer-executable instructions.

[0044] In step 301, the system identifies a specific sound in the audio provided from an audio source 310. A specific sound generally refers to a sound selected for playing through a directional speaker. In some embodiments, a specific sound is identified by comparing the source audio with the sound and / or sound profile in a sound database 312. In some embodiments, a specific sound is identified by using a sound database 312 as a learning set by a machine learning algorithm 311. In some embodiments, the machine learning algorithm 311 may include a generative adversarial network (GAN). In some embodiments, a specific sound is identified based on one or more of the frequency, pitch, and amplitude of the sound. In some embodiments, the sound database 312 may include sound clips and / or sound profiles (e.g., spectral characteristics, frequency distribution, time frequency, or amplitude variation, etc.) associated with various types of sounds suitable for playing via a directional speaker. Examples of such sounds may include car sounds, bird sounds, ballistic throwing sounds, human voices, alarms, warning sounds, etc. In some embodiments, sounds may be initially identified based on the frequency, duration, amplitude, and / or IID and / or ITD between two or more channels of the source audio, and the sounds may be added to the sound database 312. The machine learning algorithm 311 may use the sound to identify other similar sounds. In some embodiments, the sound database 312 may be prioritized and / or pruned by the machine learning algorithm. In some embodiments, step 301 may be performed by the machine learning algorithm described herein. Figure 2 The identification module 202 described above performs

[0045] In step 302, the system isolates the specific sound. Typically, in step 302, the specific sound identified in step 301 is isolated from other sounds in the audio received from the audio source 210. In some embodiments, step 302 can be performed based on time filtering, amplitude filtering, and / or frequency filtering such as high, low, or bandpass filtering. In some embodiments, the system can use different channels of the audio source 310 to isolate the specific sound. For example, if there is a large IID or ITD between the channels of the input audio stream, one channel can be used to isolate the specific sound in another channel by subtraction or masking. In some embodiments, the specific sound can be isolated by subtraction or masking using matching sounds in the sound database 312. In some embodiments, only an approximate isolation of the specific sound is performed in step 302. In some embodiments, the specific sound can be recreated using the sound in the sound database. For example, the playback speed, pitch, and / or amplitude of the sound in the sound database can be modified to approximate the specific sound identified in the audio source 310. In some embodiments, step 302 can be performed by reference to Figure 2 The sound filter 204 described here performs.

[0046] In step 303, the system generates an output for the directional speaker 321. In some embodiments, the output includes the specific sound isolated in step 302. In some embodiments, the directional speaker can be configured to perform periodic scans of the projection area, and the directional speaker audio output can be timed to the periodic scans. In some embodiments, the directional speaker output can also include projection direction data. In some embodiments, the projection direction data can be generated by a reference Figure 2 The projection direction module 206 described is determined. In some embodiments, the projection direction can be associated with the user position. In some embodiments, the projection direction data can include two or more discrete projection areas associated with two or more users, so that different sounds can be projected to different users in the same room using a directional speaker. In some embodiments, the projection direction data can associate the projection angle with the time portion of the directional speaker audio. In some embodiments, the directional speaker output can include a 2D soundscape to be produced by the directional speaker. In some embodiments, the directional speaker output can include a separate audio stream for each panel of a fixed multi-panel directional speaker. In some embodiments, the directional speaker output can include a rotation instruction for a rotor-controlled directional speaker.

[0047] In step 304, the system generates an output of a non-directional speaker 322. In some embodiments, the directional speaker output may include an audio source 310 minus a specific sound. In some embodiments, the directional speaker output may correspond to the unchanged source audio. In some embodiments, the non-directional speaker output may include two or more channels, each channel corresponding to a non-directional speaker device (e.g., left speaker, right speaker) in a hybrid speaker assembly.

[0048] In some embodiments, Figure 3 The steps shown in can be continuously repeated for the audio stream in near real time so that the converted audio and the video playback associated with the source audio are synchronized. In some embodiments, the system can pre-buffer the source audio to allow video / audio playback synchronization. In some embodiments, Figure 3 The steps shown in can be performed on pre-recorded audio files. In some embodiments, instead of outputting to speakers, the generated directional speaker output and non-directional speaker output can be stored as audio files for later playback.

[0049] Figure 4A and Figure 4B is an illustration of an example of a multi-panel directional speaker according to some embodiments. Figure 4A and Figure 4BA top view of a panel including directional speakers arranged to project sound at multiple angles across a projection area 420 through modulation. In some embodiments, panels 410a-e can each include a parametric array configured to project a narrow range of sound propagation through ultrasound.

[0050] exist Figure 4A , a two-panel configuration is shown. Panels 410a and 410b are positioned at an angle relative to each other and are located between a non-directional left speaker 430 and a right speaker 431. The angle between the two panels can be between 170 degrees and 90 degrees. The two panels 410a and 410b are configured to change the projection direction of sound within the projection area 420 by modulating the audio output of the two panels.

[0051] exist Figure 4B , a three-panel configuration is shown. Panels 410c, 410d, and 410e are positioned at an angle toward each other and are located between the non-directional left speaker 430 and the right speaker 431. In some embodiments, the center panel 410d is generally parallel to a line drawn between the non-directional left speaker 430 and the right speaker 431 of the speaker assembly. The angle between each of the side panels 410c and 410e and the center panel 410d can be between 170 degrees and 70 degrees. The three panels 410c to 410e are configured to change the projection direction of the sound within the projection area 420 by modulating the audio output of the three panels.

[0052] In some embodiments, the hybrid speaker may include directional speakers in other configurations. For example, the directional speaker may include four or more parameter panels. In another example, the directional speaker may include one or more curved parameter panels. In yet another example, the directional speaker may include a 2D matrix of panels (e.g., 2x2, 3x3, etc.) to project sound on two axes (e.g., left and right and up and down). In some embodiments, the hybrid speaker may include a directional speaker driven by a mechanical rotor that is configured to rotate and / or tilt one or more parameter arrays. Figure 4A and Figure 4B The distances, proportions, and relative positions of elements in the drawings are provided for reference only and are not necessarily drawn to scale.

[0053] In some embodiments, one or more of the above embodiments, methods, approaches and / or technologies can be implemented in one or more computer programs or software applications that can be executed by a processor-based device or system. For example, such a processor-based device or system may include a computer, an entertainment system, a game console, a workstation, a graphics workstation, a server, a client, a portable device, a tablet device, etc. Such a computer program can be used to perform various steps and / or features of the above methods and / or technologies. In other words, the computer program can be suitable for enabling or configuring a processor-based device or system to perform and implement the above functions. For example, such a computer program can be used to implement any embodiment of the above methods, steps, technologies or features. As another example, such a computer program can be used to implement any type of tool or similar utility program using any one or more of the above embodiments, methods, approaches and / or technologies. In some embodiments, program code macros, modules, loops, subroutines, calls, etc. within or without a computer program can be used to perform various steps and / or features of the above methods and / or technologies. In some embodiments, a computer program can be stored or embodied in one or more computer-readable storage or recording media, such as any one of one or more computer-readable storage or recording media described herein.

[0054] Thus, in some embodiments, the present invention provides a computer program product comprising: a medium for embodying a computer program for input to a computer; and a computer program embodied in the medium to cause a computer to perform or implement steps comprising: any one or more steps involved in any one or more of the embodiments, methods, approaches, and / or methods described herein. For example, in some embodiments, the present invention provides one or more non-transitory computer-readable storage media storing one or more computer programs adapted or configured to cause a processor-based device or system to perform steps comprising: rendering a computer-simulated scene for display to a user; detecting the start of a glance that causes a glance masking in an eye movement of a user viewing the computer-simulated scene; and reducing computing resources used to render frames of the computer-simulated scene during at least a portion of the duration of the glance.

[0055] While the invention disclosed herein has been described by way of specific embodiments and applications thereof, numerous modifications and variations may be made thereto by those skilled in the art without departing from the scope of the invention as set forth in the claims.

Claims

1. An audio converter system, comprising: an audio input configured to receive source audio; as well as A processor configured to generate output audio based on the source audio by performing the steps comprising: identifying a specific sound in the source audio, wherein the specific sound is identified by machine learning using sounds in a sound database as a learning set; Generate an output of a directional speaker based on the specific sound, wherein the output of the directional speaker includes projection direction data, and the projection direction data includes two or more discrete projection areas associated with two or more users, so that different sounds can be projected to the two or more users using the directional speaker.

2. The system of claim 1, wherein the machine learning is based on a generative adversarial network (GAN).

3. The system of claim 1, wherein the processor is further configured to perform the steps comprising: generating at least two channels of non-directional speaker output for at least two non-directional speakers; in, The at least two channels of non-directional speaker output are generated based on subtracting the particular sound from the source audio.

4. The system of any one of claims 1 to 3, wherein the processor is further configured to perform the steps comprising: The projection direction of the directional speaker is determined based on a user position, an amplitude difference between two or more channels of the source audio, and / or a time delay difference between two or more channels of the source audio.

5. The system of claim 4, wherein the projection direction of the directional speaker is output to the directional speaker to control a mechanical rotor configured to rotate the directional speaker.

6. A system as described in any one of claims 1 to 3, wherein the directional speaker includes a plurality of parameter array panels, and the directional speaker output includes the output of each of the plurality of parameter array panels.

7. A system as claimed in claim 6, wherein the projection direction of the directional speaker is controlled by modulating the output of the plurality of parameter array panels.

8. The system of any one of claims 1 to 3, further comprising: An audio output configured to be coupled to a hybrid speaker including a sound bar having the directional speaker positioned between a left non-directional speaker and a right non-directional speaker.

9. An audio conversion method, comprising: receiving source audio; identifying a specific sound in the source audio, wherein the specific sound is identified by machine learning using sounds in a sound database as a learning set; Generate an output of a directional speaker based on the specific sound, wherein the output of the directional speaker includes projection direction data, and the projection direction data includes two or more discrete projection areas associated with two or more users, so that different sounds can be projected to the two or more users using one directional speaker.

10. The method of claim 9, wherein the machine learning is based on a generative adversarial network (GAN).

11. The method of claim 9, further comprising: generating at least two channels of non-directional speaker output for at least two non-directional speakers; Wherein, the at least two channels output by the non-directional speaker are generated based on subtracting the specific sound from the source audio.

12. The method of any one of claims 9 to 11, further comprising: The projection direction of the directional speaker is determined based on a user position, an amplitude difference between two or more channels of the source audio, and / or a time delay difference between two or more channels of the source audio.

13. The method of claim 12, wherein the projection direction of the directional speaker is output to the directional speaker to control a mechanical rotor configured to rotate the directional speaker.

14. A method as claimed in any one of claims 9 to 11, wherein the directional speaker comprises a plurality of parameter array panels, and the directional speaker output comprises the output of each of the plurality of parameter array panels.

15. The method of claim 14, wherein the projection direction of the directional speaker is controlled by modulating the outputs of the plurality of parameter array panels.

16. The method of any one of claims 9 to 11, wherein the directional speaker is included in a hybrid speaker including a sound bar having the directional speaker located between a left non-directional speaker and a right non-directional speaker.

17. An audio system comprising: Speaker housing; a directional speaker enclosed in the speaker housing; as well as An audio converter is configured to receive source audio, identify a specific sound in the source audio, and generate an output of the directional speaker based on the specific sound, wherein the specific sound is identified by machine learning using sounds in a sound database as a learning set, wherein the output of the directional speaker includes projection direction data, the projection direction data includes two or more discrete projection areas associated with two or more users, so that different sounds can be projected to the two or more users using one directional speaker.

18. The system of claim 17, wherein the directional speaker comprises one or more parametric array panels.

19. The system of claim 17, further comprising: A mechanical rotor is used to control the projection direction of the directional speaker.

20. The system of any one of claims 17 to 19, wherein the directional speaker comprises a plurality of parameter array panels, and wherein the projection direction of the directional speaker is controlled by modulating the output of the plurality of parameter array panels.

21. The system of any one of claims 17 to 19, further comprising: A sensor is used to detect the position of a user, wherein the projection direction of the directional speaker is controlled based on the position of the user.

Images