Audio devices with ultrasound guided processing and related methods

Abstract

Audio devices with ultrasound guided processing and related methods are disclosed. An audio device includes an interface, a memory, one or more processors including a signal processor, one or more speakers, one or more microphones, and an ultrasound device. The audio device is configured to: output an ultrasonic signal via an ultrasonic device output; obtaining an input ultrasound signal via the ultrasound device; obtaining an audio input signal via one or more microphones; determining a direction of arrival of the audio input signal based on the audio input signal; determining position data based on the input ultrasound signal and the output ultrasound signal, the position data including first position data indicating a first position of the first moving object; the audio input signal is processed for providing an audio output signal based on whether the direction of arrival satisfies a position criterion.

Description

Technical Field

[0001] The present disclosure relates to the field of audio devices, and in particular to audio devices with ultrasound guided processing and related methods. Background Art

[0002] An audio device (e.g., a speakerphone) can improve the signal-to-noise ratio (SNR) by using multiple microphones and signal processing techniques. The direction of arrival of an audio source can be detected to determine where the audio source is located in the azimuth plane. A limitation of current direction of arrival (DOA) processing pipelines is that it is difficult to estimate the distance from the audio source to the microphone using only the audio signal. Summary of the invention

[0003] Another limitation of systems that rely on audio to detect and localize sound sources is that they require the user associated with the sound source to interact with the audio device to be detected. Therefore, silent speakers will not be detected. When cameras are available, some of this detection can be achieved by processing the camera stream, but processing high-resolution video (such as 4K) is expensive and not always feasible in resource-constrained applications.

[0004] Therefore, there is a need for audio devices and methods with ultrasound guided processing that can mitigate, alleviate or address existing shortcomings and can provide more accurate and / or targeted detection of one or more audio sources of interest in the environment in which the audio device is positioned.

[0005] An audio device is disclosed. The audio device includes an interface, a memory, one or more processors including a signal processor, one or more speakers, one or more microphones, and an ultrasonic device. The audio device is configured to output an ultrasonic signal via the ultrasonic device. The audio device is configured to obtain an input ultrasonic signal via the ultrasonic device. The audio device is configured to obtain an audio input signal via one or more microphones. The audio device is configured to determine a direction of arrival of the audio input signal based on the audio input signal. The audio device is configured to determine position data based on the input ultrasonic signal and the output ultrasonic signal, the position data including first position data indicating a first position of a first mobile object. The audio device is configured to determine whether the direction of arrival meets a position criterion based on the first position data. The audio device is configured to process the audio input signal based on whether the direction of arrival meets the position criterion for providing an audio output signal.

[0006] Furthermore, a conference system is disclosed, which comprises the audio device disclosed herein and one or more ultrasonic transducers disclosed herein.

[0007] A method of operating an audio device, such as the audio device disclosed herein, is disclosed. The method includes outputting an output ultrasonic signal via an ultrasonic device. The method includes obtaining an input ultrasonic signal via an ultrasonic device. The method includes obtaining an audio input signal via one or more microphones. The method includes determining a direction of arrival of the audio input signal based on the audio input signal. The method includes determining position data based on the input ultrasonic signal and the output ultrasonic signal, the position data including first position data indicating a first position of a first mobile object. The method includes determining whether the direction of arrival meets a position criterion based on the first position data. The method includes processing the audio input signal for providing an audio output signal based on whether the direction of arrival meets the position criterion.

[0008] The present disclosure provides an audio device capable of performing distance-aware processing of an input audio signal. It will be appreciated that the present disclosure allows for sighted audio processing of an audio source, and thereby allows for distance-dependent sighting of an audio source, for example, by using ultrasound-guided processing. An audio device may provide improved detection of one or more audio sources by using both an audio signal and an ultrasound signal. For example, an audio device may be able to use both an ultrasound signal and an audio signal to accurately determine the distance between an audio source and the audio device, thereby improving speech quality and intelligibility.

[0009] The disclosed conference system, audio device, and related method provide detection of one or more moving objects, thereby achieving a more natural conference experience. In other words, such motion detection can provide a more flexible interaction between a user and an audio device by allowing the user to interact with the audio device without having to speak (e.g., by raising and / or waving hands).

[0010] By knowing the approximate number of users utilizing audio devices and the corresponding movements, the present disclosure can allow for improved tuning of the processing pipeline, thereby improving detection of one or more audio sources of interest under various acoustic conditions. The present disclosure can provide improved voice quality of output audio signals (such as transmitted audio signals) output to the far end. In other words, the present disclosure can provide improved audio quality of speech at the far end (e.g., during a phone call or conference). BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features and advantages of the present disclosure will become apparent to those skilled in the art through the following detailed description of examples thereof with reference to the accompanying drawings, in which:

[0012] Figure 1 schematically illustrates an example audio device according to the present disclosure,

[0013] Figure 2 schematically illustrates an example scenario in which the disclosed technology is applied, and

[0014] FIG. 3A to FIG. 3B A flow chart of an example method according to the present disclosure is shown. DETAILED DESCRIPTION

[0015] Various examples and details are described below with reference to the accompanying drawings when relevant. It should be noted that the drawings may be drawn to scale or not to scale, and that elements of similar structure or function are represented by similar reference numerals throughout the drawings. It should also be noted that the drawings are intended only to facilitate the description of the examples. They are not intended to be an exhaustive description of the present disclosure or a limitation on the scope of the present disclosure. In addition, the illustrated examples do not need to have all the aspects or advantages shown. The aspects or advantages described in conjunction with a specific example are not necessarily limited to that example, and may be practiced in any other example even if not shown as such or if not so explicitly described.

[0016] The accompanying drawings are schematic and simplified for clarity, and they only show details that are helpful for understanding the present disclosure, while other details have been omitted. Throughout the specification, the same reference numerals are used for the same or corresponding parts.

[0017] A conference system is disclosed. The conference system includes an audio device as disclosed herein and one or more ultrasonic transducers. In one or more example conference systems, one or more ultrasonic transducers are positioned at different locations in a conference room. The conference system can be considered a hands-free telephone system. The conference system can be considered a hands-free telephone system for performing calls (such as audio and / or video calls). A conference system such as a hands-free telephone system can be considered a system with ultrasound guided processing.

[0018] An audio device is disclosed. The audio device as disclosed herein may include an interface, a memory, one or more processors including a signal processor, one or more speakers, one or more microphones, and an ultrasound device.

[0019] In one or more example audio devices, the audio device may include one or more interfaces, one or more processors and one or more memories. In addition, the audio device may include one or more microphones, such as a first microphone, optionally a second microphone, optionally a third microphone and optionally a fourth microphone. The audio device may include one or more speakers, such as an audio speaker, for example, one or more speakers.

[0020] An audio device may be considered a speakerphone. An audio device may be configured to obtain audio signals, output audio signals, and process audio signals. An audio device may be considered a conference audio system, such as a conference speakerphone, for example, configured to be used by one party (such as one or more users at the near end) to communicate with one or more other parties (such as one or more users at the far end). An audio device may be considered a smart audio device, such as a smart speakerphone. An audio device may be used for large conferences and / or meetings between two or more parties that are far away from each other. An audio device may be used by one or more users in the vicinity (also referred to as the near end) where the audio device is located.

[0021] The audio device can be a single audio device. The audio device can be considered as a plurality of interconnected audio devices, such as a conference system. The conference system can include one or more users. Note that the term speaker can be considered as a user of the audio device. The audio device can be configured to process one or more audio signals. Optionally, the audio device can be configured to output an audio signal. The audio device can be configured to obtain an audio signal, such as receiving an audio signal via an interface and / or one or more microphones.

[0022] In one or more example audio devices, the interface includes a wireless transceiver (also denoted as a radio transceiver) and an antenna for wireless transmission and reception of audio signals (such as wireless transmission of output signals and / or wireless reception of wireless input signals). The audio device can be configured to wirelessly communicate with one or more electronic devices (such as a hands-free phone, a smart phone, a tablet computer, a computer, and / or a smart watch). The audio device optionally includes an antenna for converting one or more wireless input audio signals into one or more antenna output signals. The conference system, audio device, and / or electronic device can be configured to wirelessly communicate via a wireless communication system (such as a short-range wireless communication system, such as Wi-Fi, Bluetooth, Zigbee, IEEE 802.11, IEEE 802.15, infrared, etc.).

[0023] The conferencing system and / or audio device can be configured for wireless communication via a wireless communication system (such as a 3GPP system), such as a 3GPP system supporting one or more of: New Radio NR, Narrowband IoT, NB-IoT and Long Term Evolution - Enhanced Machine Type Communication LTE-M, millimeter wave communication, such as millimeter wave communication in a licensed band, such as device-to-device millimeter wave communication in a licensed band.

[0024] In one or more example conference systems, audio devices and / or electronic devices, the interface of the audio device includes one or more of the following: a Bluetooth interface, a Bluetooth low energy interface, and a magnetic induction interface. For example, the interface of the electronic device and / or the interface of the audio device may include a Bluetooth antenna and / or a magnetic interference antenna.

[0025] In one or more example audio devices, the interface may include a connector for wired communication via a connector (such as by using a cable). The connector may connect one or more microphones to the audio device. The connector may connect the audio device to an electronic device, such as for a wired connection.

[0026] One or more interfaces may be or include a wireless interface (such as a transmitter and / or a receiver) and / or a wired interface (such as a connector for physical coupling). For example, an audio device may have an input interface configured to receive data (such as a microphone input signal). In one or more example audio devices, the audio device may be used in all form factors in all types of environments, such as for headphones and / or video conferencing equipment. For example, the audio device may not have specific microphone placement requirements. In one or more example audio devices, the audio device may include an external microphone, such as an external microphone located at multiple locations in the environment of the audio device.

[0027] An audio device includes an interface, a memory, one or more processors including a signal processor, one or more speakers, one or more microphones, and an ultrasound device. In one or more example audio devices, the ultrasound device includes one or more ultrasonic transducers. The ultrasound device can be considered as a device configured to transmit, generate and / or output ultrasonic signals. The ultrasound device can be configured to transmit, generate and / or output ultrasonic pulses and / or bursts.

[0028] In one or more example audio devices, the ultrasound device may be configured to obtain or receive an input ultrasound signal. The audio device is configured to: output an output ultrasound signal via the ultrasound device; obtain an input ultrasound signal via the ultrasound device; obtain an audio input signal via one or more microphones; determine a direction of arrival of the audio input signal based on the audio input signal; determine position data based on the input ultrasound signal and the output ultrasound signal, the position data including first position data indicating a first position of a first mobile object; and process the audio input signal for providing an audio output signal based on whether the direction of arrival satisfies a position criterion.

[0029] Alternatively, the ultrasound device may be considered as a speaker, wherein one or more microphones are used to obtain an ultrasound input signal. In other words, the ultrasound transducer may include one or more speakers and one or more microphones, wherein the one or more speakers and the one or more microphones act as an ultrasound transducer. It is understood that one or more speakers may act as an ultrasound transmitting or generating device, and one or more microphones may act as an ultrasound receiving device.

[0030] The audio device is configured to output an output ultrasonic signal via an ultrasonic device. In one or more example audio devices, the ultrasonic device is configured to generate and / or output an output ultrasonic signal, such as a first output ultrasonic signal, a second output ultrasonic signal, and / or a third output ultrasonic signal. The output ultrasonic signal can be viewed as a sound wave and / or an ultrasonic wave in an ultrasonic spectrum.

[0031] The audio device is configured to obtain an input ultrasonic signal via an ultrasonic device. In one or more example audio devices, the input ultrasonic signal may be considered as a reflected ultrasonic signal, such as an echo signal. In other words, the input ultrasonic signal may be considered as a reflected version of the output ultrasonic signal, such as reflected in the environment of the audio device. In one or more example audio devices, the input ultrasonic signal may be considered as one or more reflected ultrasonic signals, such as a first reflected ultrasonic signal, a second reflected ultrasonic signal, and / or a third reflected ultrasonic signal. For example, when encountering an obstacle, the output ultrasonic signal is reflected back to the audio device, such as reflected back to the ultrasonic device, to provide the input ultrasonic signal. The input ultrasonic signal may include a portion of the energy of the output ultrasonic signal that has been reflected in the environment of the audio device.

[0032] The audio device is configured to obtain an audio input signal via one or more microphones. In one or more example audio devices, the audio device is configured to obtain an audio input signal (e.g., including a first input signal, a second input signal, and / or a third input signal) via one or more microphones (e.g., a first microphone, a second microphone, and / or a third microphone). In one or more example audio devices, the audio device may be configured to obtain a first audio input signal using a first microphone, a second audio input signal using a second microphone, and / or a third audio input signal using a third microphone. The first audio input signal, the second audio input signal, and / or the third audio input signal may be combined and considered as an audio input signal.

[0033] In one or more example audio devices, the audio input signal indicates an audio signal generated by one or more users, people, or other sound sources in the environment of the audio device (such as a conference system). In one or more example audio devices, the audio input signal may indicate an audio signal generated by one or more users of the audio device when using the audio device. In other words, the audio input signal may indicate user speech. In one or more example audio devices, the audio input signal may include one or more of the following: user's speech, interfering speech, echo, and noise (such as ambient noise, continuous noise, intermittent noise, impulse noise, and / or low-frequency noise).

[0034] The audio device is configured to determine the direction of arrival DOA of the audio input signal based on the audio input signal. In one or more example audio devices, the audio device is configured to determine the first DOA of the first audio input signal based on the first audio input signal. In one or more example audio devices, the audio device is configured to determine the second DOA of the second audio input signal based on the second audio input signal. In one or more example audio devices, the audio device is configured to determine the third DOA of the third audio input signal based on the third audio input signal. In one or more example audio devices, the DOA of the audio input signal includes the first DOA, the second DOA and the third DOA. In one or more example audio devices, the first DOA, the second DOA and the third DOA can be combined and regarded as the DOA of the audio input signal. It can be understood that the audio device can be configured to determine one or more DOAs depending on the number of audio sources in the environment of the audio device. The DOA of the audio input signal can be determined in various ways. For example, the DOA can be determined using one or more of the following technologies: time difference of arrival TDOA, beamforming by analyzing the strength across beams, cross-correlation of audio signals, spectrum analysis by analyzing the frequency content of the audio input signal, and machine learning techniques.

[0035] The audio device is configured to determine position data based on the input ultrasonic signal and the output ultrasonic signal, the position data including first position data indicating a first position of a first moving object (e.g., a first non-static object). In one or more example audio devices, the position data includes a distance and / or direction of a viewing angle of the audio device. In one or more example audio devices, the first position data includes a distance between the first moving object and the audio device and / or a direction associated with the first moving object relative to the audio device.

[0036] It is understood that the audio device may be configured to receive a reflected version of the output ultrasonic signal, such as the input ultrasonic signal, when the output ultrasonic signal encounters the first mobile object. The audio device may be configured to determine the time interval between outputting the output ultrasonic signal and obtaining the input ultrasonic signal. For example, such a time interval may be converted into a corresponding distance, such as the distance between the audio device and the first mobile object. For example, determining the position data may include using time of flight ToF, measurement of ultrasonic signals and / or using Doppler shift detection to determine position data, such as the distance to the mobile object. The audio device may be configured to determine the angle interval between the angle associated with the transmission of the output ultrasonic signal (such as the departure angle) and the angle associated with the reception of the input ultrasonic signal (such as the arrival angle). For example, such an angle interval may be converted into a corresponding direction, such as a direction relative to the audio device associated with the first mobile object. For example, determining the position data may include using a phased array ultrasonic measurement of the ultrasonic signal to determine the position data, such as the direction of arrival or the angle of arrival of the reflected ultrasonic signal on the mobile object. Alternatively or additionally, the direction of arrival of the reflected ultrasonic signal on the mobile object may be determined based on the ultrasonic transducer output and the obtained ultrasonic signal. For example, the ultrasonic transducer may have a detection range capable of detecting a moving object. Therefore, the arrival direction of the reflected ultrasonic signal on the moving object can be regarded as the direction range.

[0037] In one or more example audio devices, the first moving object can be considered as a first moving person, such as a person who is not necessarily speaking. For example, the first moving person may be moving, such as raising his hand, but not speaking. In one or more example audio devices, the present disclosure allows detection of small movements from "stationary" objects, such as "stationary" people and / or users, but which will still make very small movements that can be detected. In one or more example audio devices, the first moving object can be considered as an obstacle to the propagation of the output ultrasonic signal. The first moving object can be considered as an object near the audio device or in the environment (such as near ultrasonic detection from the audio device).

[0038] The audio device is configured to determine whether the direction of arrival DOA meets the location criteria based on the first location data. In one or more example audio devices, the audio device is configured to determine whether the DOA meets the location criteria based on the first location data and / or the second location data. The location criteria can be determined based on the first location data (such as based on the first distance and / or the first direction). It can be understood that the location criteria can vary depending on the first location data. The location criteria can be based on the first position of the first mobile object. Determining whether the DOA meets the location criteria can include determining whether the distance of the audio source associated with the DOA from the audio device and / or the direction of the audio device meets the location criteria. In one or more example audio devices, the location criteria are dynamic depending on whether the DOA is associated with the first mobile object. The location criteria may include a distance threshold and / or an angle range threshold.

[0039] Thus, it is possible to distinguish between a moving object that emits sound and a stationary object that emits sound. For example, it may be desirable to aim at a person who is speaking and therefore also non-static as an audio source, but on the other hand, the noise from a static machine is attenuated. Therefore, the position criterion may vary in distance, for example, depending on the nature of the audio source. In one or more example audio devices, the position criterion is user-defined. For example, a user of an audio device may predetermine one or more conditions to be included in the position criterion. For example, a user may set a distance threshold and / or a direction threshold to be included in the position criterion. For example, a user may set a distance range threshold of 5 meters from an audio device. For this example, the audio device will attenuate an audio source located more than 5 meters from the audio device. Another example may be that a user sets a direction range threshold of 0° to 180° with respect to an audio device (such as a reference positioning with respect to an audio device). For this example, the audio device will attenuate an audio source located within a direction range of 180° to 360° with respect to the audio device. For example, if 0° to 180° is about the front hemisphere of the audio device, such as the "front" of the audio device, then 180° to 360° may be about the rear hemisphere of the audio device, such as the "rear" of the audio device.

[0040] The audio device may be configured to determine whether the first position data is associated or related to the DOA of the audio input signal, or whether it is associated or related to the audio source associated with the DOA. For example, when the first person emits a sound from a certain position about the audio device, the audio device may determine the DOA of the sound emitted by the first person, for example, by processing the audio input signal. The audio device may then determine the first position data of the first person, or at least the position data in the direction range around the first person, by using ultrasound. The audio device may then determine whether the DOA comes from a direction in which the first position data meets the position criteria. In other words, the audio device may determine whether the DOA comes from a direction in which the first position of the first mobile object meets the position criteria. For example, when the first person emitting the sound has a first position that meets the position criteria, the DOA may meet the position criteria. The audio device may be configured to match the DOA with the position data in order to determine the first position data associated with the DOA. In other words, the audio device may be configured to determine whether the DOA is associated with the first mobile object, and if so, the audio device may be configured to determine whether the first position data meets the position criteria.

[0041] The audio device is configured to process an audio input signal for providing an audio output signal based on whether the direction of arrival DOA meets the location criteria. In other words, the audio device is configured to process the audio input signal based on the result of the determination of whether the DOA meets the location criteria. To put it another way, the processing of the audio input signal may vary depending on whether the DOA meets the location criteria. In one or more example audio devices, processing the audio input signal includes determining the audio output signal based on the result of the determination of whether the DOA meets the location criteria (such as based on the result). In one or more example audio devices, the audio device is configured to process the audio input signal based on the result of such determination. In one or more example audio devices, the audio device is configured to output the audio output signal via one or more interfaces, for example, outputting the audio output signal to a remote audio device.

[0042] An example of applying the technology disclosed herein may be as follows:

[0043] The audio device is located in a conference room with multiple people in the conference room. The audio device outputs an ultrasound signal via the ultrasound device output, such as outputting one or more ultrasound pulses or bursts in the conference room.

[0044] The audio device obtains an input ultrasound signal via the ultrasound device, wherein the input ultrasound signal is a reflection of the output ultrasound signal.The audio device obtains an audio input signal via one or more microphones, such as obtaining an audio input signal indicative of speech from a first person in a conference room.

[0045] The audio device determines an arrival direction of an audio input signal, such as an arrival direction of a first person's voice, based on the audio input signal.

[0046] The audio device determines position data based on the input ultrasonic signal and the output ultrasonic signal, the position data including first position data indicating a first position of a first mobile object. For example, the audio device determines first position data indicating a first position of a first person as the first mobile object.

[0047] The audio device determines whether the direction of arrival meets the position criteria based on the first position data. For example, the audio device determines whether the direction of arrival of the first person's voice is within a first direction range about the audio device and is determined by the position criteria and is associated with the first person's first position. In other words, the audio device determines whether the first person's first position is within a certain direction range and / or distance range about the audio device determined by the position criteria. The audio device processes the audio input signal for providing an audio output signal based on whether the direction of arrival meets the position criteria. For example, when the first person's first position is within a certain direction range and / or distance range about the audio device, the audio device targets the first person, such as targeting an audio signal from the direction of the first person.

[0048] In one or more example audio devices, processing an audio input signal includes aiming at an audio source associated with a direction of arrival according to the direction of arrival meeting a position criterion. In one or more example audio devices, the audio device is configured to aim at an audio source associated with a DOA when the DOA meets the position criterion. In one or more example audio devices, the audio device is configured to aim at a portion of an audio input signal from a DOA. The audio device may be configured to aim at an audio source based on the distance of the audio source from the audio device. The audio device may be configured to aim at an audio source by controlling a beamformer of the audio device to perform beamforming in a direction range substantially in the direction of the DOA. Aiming at an audio source associated with a DOA may be considered as marking an audio source associated with a DOA. In other words, the audio device may be configured to filter out a portion of an audio input signal that is not from a DOA. Aiming at an audio source may be considered as isolating or extracting a specific sound or audio signal of interest from a mix or recording containing multiple audio sources. Aiming at an audio source may include using one or more of the following: noise reduction, source separation, audio enhancement, beamforming, spectral subtraction, adaptive filtering, audio transcription, and machine learning algorithms.

[0049] The term "targeting" can be thought of as directing or focusing attention on a specific audio source to selectively extract or manipulate it. This can involve various algorithms and signal processing techniques designed to isolate the desired audio source while minimizing the effects of other unwanted sources or background noise. This can allow the quality or intelligibility of the target audio source to be enhanced, or to be isolated for further analysis or processing.

[0050] In one or more example audio devices, upon determining that the direction of arrival meets the location criteria, the audio device is configured to forward the audio input signal to a signal processor, such as a digital signal processor DSP, to process the audio input signal.

[0051] In one or more example audio devices, processing an audio input signal includes attenuating an audio source associated with a direction of arrival according to the direction of arrival not meeting a position criterion. In one or more example audio devices, the audio device is configured to attenuate an audio source associated with a direction of arrival when the direction of arrival does not meet a position criterion. In one or more example audio devices, the audio device is configured to filter out a portion of an audio input signal from a DOA. In other words, the audio device may be configured to attenuate one or more audio sources associated with an audio input signal from a DOA. This may be, for example, noise in the environment of the audio device, rather than a user of the audio device or a voice from a person who does not use the audio device. Another example may be that an audio source associated with a DOA is too far from the audio device and / or is in a direction range where a user of no audio device is located with respect to the audio device. Therefore, the present disclosure allows attenuation of an audio source that does not meet a position criterion.

[0052] In one or more example audio devices, the audio output signals may be considered audio signals associated with one or more audio sources of interest, such as audio sources associated with DOAs that satisfy the location criteria.

[0053] In one or more example audio devices, the first position data includes a first distance and a first direction of the first mobile object. In one or more example audio devices, the first position data includes a first distance (such as a distance between the first mobile object and the audio device) and / or a first direction (such as a direction associated with the first mobile object relative to the audio device). The first direction can be regarded as a direction associated with the first mobile object relative to the ultrasonic device and / or the ultrasonic transducer performing the measurement. In other words, the first direction can be regarded as a direction associated with the first mobile object relative to the ultrasonic device, which has a width of emission and / or propagation of ultrasonic signals (such as output and / or input ultrasonic signals) corresponding to one or more ultrasonic transducers. In one or more example audio devices, the first distance can be regarded as the distance between the first mobile object and the ultrasonic device and / or ultrasonic transducer performing the measurement. In other words, the first distance can be regarded as a distance determined based on the output power of one or more ultrasonic transducers. In one or more example audio devices, the first distance can be an approximate value, such as a value within a distance range. The distance range can be regarded as a part of the environment in which the audio device is positioned, aiming at one or more audio sources within the part.

[0054] In one or more example audio devices, determining whether the direction of arrival meets the position criterion includes determining whether the direction of arrival is within a first direction range associated with the first direction and / or whether the first distance is within a distance range (such as a first distance range). In one or more example audio devices, the audio device is configured to determine that the direction of arrival meets the position criterion when the audio device determines that the direction of arrival is within a first direction range associated with the first direction and / or the first distance is within the distance range. The audio device can be configured to provide an input audio signal detected within the distance range to provide an output audio signal. In one or more example audio devices, the audio device is configured to determine that the direction of arrival does not meet the position criterion when the audio device determines that the direction of arrival is not within a first direction range associated with the first direction and / or the first distance is not within the distance range. The audio device can be configured to attenuate a portion of the input audio signal determined to be outside the distance range. In one or more example audio devices, the distance range can be associated with and / or include a distance threshold. The position criterion can be based on the distance range or include a distance range. It can be understood that the distance threshold can vary depending on the direction of the audio device. For example, the distance threshold can be larger or smaller in certain directions about the audio device. The distance range may include a set of distances from the audio device that are less than or equal to a distance threshold depending on the direction of the audio device. For example, the distance range may be viewed as a bubble distance, such as a distance from the audio device. The distance range may create a so-called pickup bubble around the audio device. The audio device may be configured to target audio sources inside the pickup bubble and attenuate audio sources outside the pickup bubble. The distance range may be associated with a circular shape, such as a circular bubble, such that the distance threshold is the boundary of the circular shape. The distance range may be associated with any other suitable shape. In one or more example audio devices, the distance range may be determined and / or changed based on the output power of one or more ultrasonic transducers.

[0055] In one or more example audio devices, the audio device is configured to determine a location criterion based on a first distance and / or a first direction. In other words, the audio device can be configured to determine a location criterion based on a first distance and / or a first direction, wherein the first distance and / or the first direction are used as input in determining the location criterion. It can be understood that determining the location criterion based on the first distance and / or the first direction is dynamic. In one or more example audio devices, the location criterion can be user-defined, such as depending on one or more locations of one or more users. For example, the location criterion can be based on one or more sources of interest associated with one or more users. The location criterion can be based on a mapping of one or more sources of interest, such as a mapping of one or more users of the audio device. For example, the audio device is configured to determine a dynamic distance range, such as a dynamic bubble, based on a first distance and / or a first direction of a first moving object.

[0056] In one or more example audio devices, the processing of the audio input signal is based on the first position data and / or the second position data. The first position data and / or the second position data can be used for mapping of an audio source of interest, such as mapping of one or more users of interest. In one or more example audio devices, the present audio device allows the user to map using an ultrasonic signal and process the audio input signal based on the mapping. For example, the audio device is configured to detect and / or map a user (such as a first mobile object and / or a second mobile object) using an ultrasonic signal (such as using an output ultrasonic signal and an input ultrasonic signal) to provide position data (such as the first position data and / or the second position data). For example, the audio device is configured to process the audio input signal based on such detection and / or mapping. The advantage of processing the audio input signal based on the first position data and / or the second position data is that the audio device can predict where the audio signal of interest may come from. For example, a user or person who raises his / her hand in a meeting can be detected by the audio device as a mobile object within a distance range of interest to the audio device, and thereby the audio device can aim the direction to this user before this person starts speaking.

[0057] In one or more example audio devices, processing the audio input signal includes aiming at a first mobile object and / or a second mobile object. In other words, the first mobile object and / or the second mobile object may be located within a distance range from the audio device. The audio device may be configured to aim at an audio source by controlling a beamformer of the audio device to perform beamforming within a range of directions generally in the direction of the first mobile object and / or the second mobile object. Aiming at the first mobile object and / or the second mobile object may be viewed as marking an audio source of interest in the direction of the first mobile object and / or the second mobile object. In other words, the audio device may be configured to filter out portions of the audio input signal that are not from the first mobile object and / or the second mobile object.

[0058] In one or more example audio devices, the position data includes second position data indicating a second position of a second mobile object, wherein the second position data includes a second distance and a second direction of the second mobile object (e.g., a second non-static object). In one or more example audio devices, the second position data includes a distance between the second mobile object and the audio device and / or a direction associated with the second mobile object relative to the audio device. In other words, the second position data includes a second distance and a second direction of the second mobile object (such as from the audio device). In one or more example audio devices, the second mobile object can be regarded as a second moving person. The second mobile object can be different from the first mobile object. The second mobile object can be, for example, a second moving person in the environment of the audio device. The description of the first mobile object can also be equivalently applied to the description of the second mobile object, but the second mobile object is different from the first mobile object. In other words, the audio device can be configured to determine whether the DOA matches the position data of each detected mobile object. When the audio device has matched the DOA with the detected mobile object in the environment of the audio device, the audio device is configured to determine whether the mobile object is within a certain distance and / or direction with respect to the audio device.

[0059] In one or more example audio devices, determining whether the arrival direction meets the location criteria includes determining whether the arrival direction is within a second direction range associated with the second direction and whether the second distance is within a distance range (such as a second distance range). In one or more example audio devices, the audio device is configured to determine that the arrival direction meets the location criteria when the audio device determines that the arrival direction is within the second direction range associated with the second direction and the second distance is within the distance range. In one or more example audio devices, the audio device is configured to determine that the arrival direction does not meet the location criteria when the audio device determines that the arrival direction is not within the second direction range associated with the second direction and / or the second distance is not within the distance range. In one or more example audio devices, the distance range can be associated with a distance threshold. The distance range can include multiple distances from the audio device that are less than or equal to the distance threshold. For example, the distance range can be considered as a bubble distance, such as a distance from the audio device. The distance range can be associated with a circular shape (such as a circular bubble) so that the distance threshold is the boundary of the circular shape. The distance range can be associated with any other suitable shape. In one or more example audio devices, the distance range can be determined based on the power of one or more ultrasonic transducers. Thus, for example, a mobile object emitting sound and a stationary object emitting sound can be distinguished, for example by having different distance ranges. For example, it may be desirable to target a person who is speaking and therefore also non-static as an audio source, but on the other hand to attenuate noise from a static machine. Therefore, the position criterion can vary in distance, for example, depending on the nature of the audio source. To put it another way, the audio device can be configured to determine whether the DOA matches the position data of each detected mobile object (such as a first mobile object and a second mobile object). When the audio device has matched the DOA with a detected mobile object in the environment of the audio device, the audio device is configured to determine whether the mobile object is within a certain distance and / or direction with respect to the audio device.

[0060] In one or more example audio devices, the distance range is less than or equal to 10 meters from the audio device. For example, the distance threshold is a distance of 10 meters from the audio device, such as within 10 meters of the audio device. The distance range can be less than or equal to 8 meters, such as less than or equal to 5 meters from the audio device, or less than or equal to 3 meters. Determining whether the direction of arrival meets the position standard can include determining whether the audio source associated with the direction of arrival is within a distance range of less than or equal to 10 meters from the audio device, such as less than or equal to 8 meters, less than or equal to 5 meters, or less than or equal to 3 meters. In other words, the position standard can be based on the distance range or include the distance range. For example, the position standard can include the distance range criterion.

[0061] The audio device may be configured to only target audio sources associated with DOAs within a distance range, such as within a distance range less than or equal to 10 meters (such as less than or equal to 8 meters, less than or equal to 5 meters, or less than or equal to 3 meters) from the audio device. Stated another way, the audio device may be configured to attenuate audio sources associated with DOAs outside the distance range (such as outside a distance range greater than 10 meters, such as greater than 8 meters, greater than 5 meters, or greater than 3 meters from the audio device).

[0062] In one or more example audio devices, determining the position data includes determining one or more amplitude changes based on the input ultrasonic signal and the output ultrasonic signal. In other words, the audio device can be configured to compare the input ultrasonic signal with the output ultrasonic signal to determine one or more amplitude changes. Determining one or more amplitude changes can include processing the input ultrasonic signal and the output ultrasonic signal. For example, determining one or more amplitude changes can include performing a Goertzel frequency analysis of the input ultrasonic signal and the output ultrasonic signal. For example, the Goertzel frequency analysis can include one or more of the following: in order to analyze a specific frequency, listing the additions generated for each frequency tone group frequency (e.g., frequency tone and side tone), adding frequencies within one or more frequency tone group frequencies (e.g., adding / subtracting the maximum Doppler shift), and ignoring one or more frequency tone group frequencies in the list. In one or more example audio devices, one or more amplitude changes can indicate the detection of a first moving object and / or a second moving object by an audio device (such as using an ultrasonic device).

[0063] In one or more example audio devices, the ultrasonic device includes one or more ultrasonic transducers, which include a first ultrasonic transducer and a second ultrasonic transducer. For example, the first ultrasonic transducer and / or the second ultrasonic transducer is a transducer such as a transducer array of one or more transducers. In one or more example audio devices, one or more ultrasonic transducers are positioned at different locations in a conference room and / or meeting room. In one or more example audio devices, one or more ultrasonic transducers are arranged around the audio device so as to be able to detect movement of substantially 360° around the audio device. In one or more example audio devices, the first ultrasonic transducer is configured to output an ultrasonic signal having a first frequency (such as a first frequency range), and the second ultrasonic transducer is configured to output an ultrasonic signal having a second frequency (such as a second frequency range). In one or more example audio devices, each transducer has a detection area. For example, the first ultrasonic transducer has a first detection area, and the first detection area is associated with a first frequency. For example, the second ultrasonic transducer has a second detection area, and the second detection area is associated with a second frequency.

[0064] In one or more example audio devices, the audio device is configured to determine one or more audio sources of interest based on a determination of whether the direction of arrival meets the position criteria. In one or more example audio devices, the audio device is configured to determine a first audio source of interest associated with a first mobile object when the DOA meets the position criteria. In one or more example audio devices, the audio device is configured to determine a second audio source of interest associated with a second mobile object when the DOA meets the position criteria. In one or more example audio devices, processing an audio input signal includes processing the audio input signal based on one or more audio sources of interest. In other words, the audio device can be configured to mark or map one or more sources of interest based on a determination of whether the direction of arrival meets the position criteria. For example, the audio device can be configured to aim at one or more sources of interest when it is determined that the direction of arrival meets the position criteria. The audio source of interest can be, for example, a user of the audio device, such as a participant in a meeting using the audio device. The audio source of interest can be, for example, considered as a potential audio source, such as a person who is near the audio device but has not yet spoken.

[0065] In one or more example audio devices, the ultrasonic device has a detection range that depends on the output power of the ultrasonic device, and wherein the position criterion is based on the detection range. In other words, the detection range of the ultrasonic device depends on the output power of the output ultrasonic signal. For example, the detection range can be regarded as a distance range from the audio device within which the audio device can detect movement. In one or more example audio devices, the detection range of the ultrasonic device depends on the output power of one or more ultrasonic transducers or is proportional to the output power of one or more ultrasonic transducers. The detection range of the ultrasonic device can also be associated with the detection width of one or more ultrasonic transducers, which corresponds to the emission and / or propagation of ultrasonic signals (such as output and / or input ultrasonic signals). It is understood that the audio device can be configured to adjust the detection range by changing the output power of the output ultrasonic signal. For example, the distance range can be determined based on the detection range of the ultrasonic device, such as based on the output power of the ultrasonic device. In one or more example audio devices, the audio device is configured to perform distance-dependent processing. In other words, a certain distance range or detection range can be assigned to each ultrasonic transducer. It is understood that each ultrasonic transducer can be assigned to a person, such as an audio source of interest. Thus, audio signals of persons located at different distances may be processed and have different distance thresholds depending on the position or direction of the audio source.

[0066] In one or more example audio devices, each of the one or more ultrasonic transducers has a dedicated detection range. In one or more example audio devices, the position criteria are based on the corresponding detection ranges of the one or more ultrasonic transducers. In one or more example audio devices, each of the one or more ultrasonic transducers has a dedicated detection cone. In one or more example audio devices, the dedicated cone detection is associated with a detection angle and / or distance. In other words, the dedicated detection range can be considered as a range in which the ultrasonic transducer is configured to output and / or obtain an ultrasonic signal.

[0067] In one or more example audio devices, processing an audio input signal includes controlling the beamforming of a signal processor based on whether the direction of arrival meets the position criteria. In one or more example audio devices, the audio device is configured to process the audio input signal by controlling the beamforming of the signal processor based on the first position data and / or the second position data. In other words, the audio device can be configured to control the beamforming of the signal processor based on the direction of arrival when the DOA meets the position criteria. For example, when the DOA meets the position criteria, the audio device can guide the beamformer toward the direction of arrival. The audio device can be configured to control the beamformer of the signal processor based on the result of the determination of whether the DOA meets the position criteria. In other words, the beamforming of the signal processor can vary depending on whether the DOA meets the position criteria. In one or more example audio devices, controlling the beamformer of the signal processor includes determining one or more beamforming coefficients to be applied to the audio output signal based on the result of the determination of whether the DOA meets the position criteria (such as according to the result).

[0068] In one or more example audio devices, determining the location data includes determining a first area indicating no movement. In one or more example audio devices, the audio device is configured to determine a first area in which there is no movement, such as a first area in the environment of the audio device. The first area can be considered as a part of a conference room and / or a section in which the audio device is located where the audio device cannot detect moving objects. In one or more example audio devices, an audio device such as an ultrasound device can distinguish stationary objects from users (e.g., sitting still but still making small movements) because of its high sensitivity to small movements. In one or more example audio devices, the location criteria is based on the first area. The audio device can be configured to attenuate audio sources associated with the first area. The first area can indicate an area in the environment surrounding the audio device where there are no moving objects. By discarding audio signals from the direction of the first area, the audio processing of the audio device can be simplified.

[0069] In one or more example audio devices, the present disclosure provides an audio device capable of recording non-verbal cues. For example, high movement from a user may indicate an intention to speak for both a video stream and an audio stream.

[0070] In one or more example audio devices, determining position data includes determining one or more environmental dimension parameters. In one or more example audio devices, the position criteria are based on one or more environmental dimension parameters. In one or more example audio devices, environmental dimension parameters can be regarded as parameters indicating the environment of the audio device (such as the surrounding environment of the audio device). For example, the environment where the audio device is located can include the room where the audio device is located, such as a large conference room, a conference room, a discussion room, a conference hall and an auditorium. For example, the environment where the audio device is located can include the interior of a building (such as an internal environment) and / or the exterior of a building (such as an external environment). In other words, the audio device can be located in an environment inside a building, in an environment outside a building and / or in a mixed environment including both an internal part and an external part. The environment where the audio device is located can include building elements such as walls, windows and / or doors, objects such as tables, chairs, lamps and / or carpets, and / or people such as humans located in the environment. The audio device can be configured to determine one or more characteristics of the environment of the audio device.

[0071] In one or more example audio devices, the environment dimension parameter may indicate a shape and / or size of a room (such as a conference room and / or a large meeting room) in which the audio device is located. In one or more example audio devices, the environment dimension parameter may indicate a room mapping.

[0072] A method for operating an audio device is disclosed. The audio device includes an interface, a memory, one or more processors including a signal processor, one or more speakers, one or more microphones, and an ultrasonic device. The method includes outputting an ultrasonic signal via the ultrasonic device. The method includes obtaining an input ultrasonic signal via the ultrasonic device. The method includes obtaining an audio input signal via one or more microphones. The method includes determining a direction of arrival of the audio input signal based on the audio input signal. The method includes determining position data based on the input ultrasonic signal and the output ultrasonic signal, the position data including first position data indicating a first position of a first mobile object. The method includes determining whether the direction of arrival meets a position criterion based on the first position data. The method includes processing the audio input signal for providing an audio output signal based on whether the direction of arrival meets the position criterion.

[0073] It should be understood that descriptions of features regarding an audio device and / or conferencing system also apply to corresponding features in one or more systems and / or one or more methods of operating an audio device as disclosed herein, and vice versa.

[0074] It should be understood that the words based on as disclosed herein may be viewed or understood as: "based on", "as input to", and / or "depending on". For example, determining the direction of arrival of an audio input signal based on an audio input signal may be understood as determining the direction of arrival of the audio input signal based on the audio input signal, using the audio input signal as input to determine the direction of arrival of the audio input signal, and / or determining the direction of arrival of the audio input signal depending on the audio input signal.

[0075] Figure 1 An audio device, such as an audio device 10 according to the present disclosure (such as Figure 2 The audio device 10 disclosed in ). The audio device 10 includes one or more processors 10C, one or more speakers 10D, one or more microphones 10E and an ultrasonic device 10F. Figure 1 , the ultrasound device 10F is shown as part of the one or more speakers 10D. It will be appreciated that in some examples, the ultrasound device 10F may be a separate device from the audio device 10. The one or more processors 10C include a signal processor, such as the signal processor 64.

[0076] The audio device 10 is configured to output an output ultrasonic signal 52, such as a first output ultrasonic signal 52A and / or a second output ultrasonic signal 52B, via the ultrasonic device 10F. The audio device 10 is configured to obtain an input ultrasonic signal 54, such as a first input ultrasonic signal 54A and / or a second input ultrasonic signal 54B, via the ultrasonic device 10F.

[0077] In one or more example audio devices, the ultrasonic device 10F includes one or more ultrasonic transducers 10G, which include a first ultrasonic transducer 10GA and a second ultrasonic transducer 10GB. In one or more example audio devices, the first ultrasonic transducer 10GA is configured to output an ultrasonic signal having a first frequency, and the second ultrasonic transducer 10GB is configured to output an ultrasonic signal having a second frequency.

[0078] The audio device 10 is configured to obtain an audio input signal 56 (such as a first audio input signal 56A and / or a second audio input signal 56B) via one or more microphones 10E (such as a first microphone and / or a second microphone).

[0079] The audio device 10 is configured to determine a direction of arrival DOA of the audio input signal 56 based on the audio input signal 56 (e.g., using one or more processors 10C). In one or more example audio devices, the one or more processors 10C include a DOA determination module 58, which is configured to determine the DOA 58A, DOA 58B of the audio input signal 56 based on the audio input signal 56. The DOA determination module 58 can be configured to provide the DOA 58A, DOA 58B to the position determination module 60. In one or more example audio devices, the DOA 58A is a first DOA, e.g., associated with the first audio input signal 56A. In one or more example audio devices, the DOA 58B is a second DOA, e.g., associated with the second audio input signal 56B.

[0080] The audio device 10 is configured to determine position data (e.g., using one or more processors 10C) based on the input ultrasonic signal 54 and the output ultrasonic signal 52, the position data including first position data indicating a first position of a first mobile object. In one or more example audio devices, one or more processors 10C include a position determination module 60, which is configured to determine position data 60A based on the input ultrasonic signal 54 and the output ultrasonic signal 52. The position data 60A may include first position data 60AA indicating a first position of the first mobile object. For example, the first position data 60AA includes a first distance and a first direction of the first mobile object. The position data 60A may include second position data 60AB indicating a second position of the second mobile object. For example, the second position data 60AB includes a second distance and a second direction of the second mobile object. In one or more example audio devices, the first audio input signal 56A is associated with the first mobile object. In one or more example audio devices, the second audio input signal 56B is associated with the second mobile object. The position determination module 60 may be configured to provide the position data 60A to the matching module 62.

[0081] In one or more example audio devices, the audio device 10 is configured to determine one or more amplitude changes based on the input ultrasonic signal 54 and the output ultrasonic signal 52, such as using one or more processors 10C (e.g., a position determination module 60). For example, the one or more amplitude changes may indicate an object moving in the environment in which the audio device is located, such as a first moving object and / or a second moving object. For example, the position determination module 60 is configured to determine one or more amplitude changes associated with the first moving object based on the first input ultrasonic signal 54A and the first output ultrasonic signal 52A. For example, the audio device 10 can detect the first moving object by outputting the first output ultrasonic signal 52A and obtaining the first input ultrasonic signal 54A, wherein the first output ultrasonic signal 54A is a reflected version of the first output ultrasonic signal 52A when encountering the first moving object. For example, the position determination module 60 is configured to determine one or more amplitude changes associated with the second moving object based on the second input ultrasonic signal 54B and the second output ultrasonic signal 52B. For example, the audio device 10 may detect the second moving object by outputting a second output ultrasonic signal 52B and obtaining a second input ultrasonic signal 54B, wherein the second output ultrasonic signal 54B is a reflected version of the second output ultrasonic signal 52B when encountering the second moving object.

[0082] In one or more example audio devices, the audio device 10 is configured to determine a first area 60D indicating no movement, such as using one or more processors 10C (e.g., a location determination module 60). In one or more example audio devices, the audio device is configured to determine the first area 60D indicating no movement using the location determination module 60. In one or more example audio devices, the location determination module 60 is configured to determine one or more environmental dimension parameters 60C. The audio device is configured to determine the one or more environmental dimension parameters 60C using the location determination module 60. The location determination module 60 may be configured to provide the location data 60A, the first area 60D, and / or the one or more environmental dimension parameters 60C to a matching module 62.

[0083] The audio device 10 is configured to determine whether the direction of arrival meets the position criteria based on the first position data, such as using one or more processors 10C. In one or more example audio devices, the one or more processors 10C include a matching module 62, which is configured to determine whether DOA 58A meets the position criteria based on the first position data 60AA. In one or more example audio devices, the matching module 62 is configured to determine whether DOA 58B meets the position criteria based on the second position data 60AB. The matching module 62 can be configured to provide matching parameters 62A to the signal processor 64, and the matching parameters 62A indicate whether DOA 58A meets the position criteria. For example, the matching parameters 62A can indicate that DOA 58A and DOA 58B meet the position criteria. For example, the matching parameters 62A can indicate that DOA 58A and DOA 58B do not meet the position criteria. In one or more example audio devices, the matching module 62 is configured to determine the position criteria based on the first position data 60AA and / or the second position data 60AB. In one or more example audio devices, the matching module 62 is configured to determine the location criteria based on one or more of: a first area 60D, one or more environmental dimension parameters 60C, first location data 60AA, and second location data 60AB.

[0084] In one or more example audio devices, the audio device 10 is configured to determine, such as using one or more processors 10C (e.g., matching module 62), whether DOA 58A is within a first direction range associated with a first direction and whether the first distance is within a distance range. In one or more example audio devices, the matching module 62 is configured to determine whether DOA 58B is within a second direction range associated with a second direction and whether the second distance is within a distance range. In one or more example audio devices, the distance range is less than or equal to 10 meters from the audio device.

[0085] In one or more example audio devices, the ultrasound device 10F has a detection range 10FA, such as a distance range, that depends on the output power of the ultrasound device 10F. In one or more example audio devices, the matching module 62 is configured to determine the location criteria based on the detection range 10FA.

[0086] In one or more example audio devices, each of the one or more ultrasonic transducers 10G has a dedicated detection range 10FB. In one or more example audio devices, the matching module 62 is configured to determine the position criteria based on the respective detection ranges 10FB of the one or more ultrasonic transducers 10G.

[0087] In one or more example devices, the audio device 10 is configured to determine one or more audio sources of interest based on a determination of whether the DOAs 58A, 58B meet the location criteria, such as using one or more processors 10C (e.g., matching module 62). For example, DOA 58A is associated with an audio source of a first mobile object. For example, DOA 58B is associated with an audio source of a second mobile object.

[0088] The audio device 10 is configured to process the audio input signal 56 based on whether the direction of arrival meets the position criteria for providing the audio output signal 68A, such as using one or more processors 10C. In one or more example audio devices, the one or more processors 10C include a signal processor 64 that is configured to process the audio input signal 56 based on whether the DOA 58A, DOA 58B meets the position criteria for providing the audio output signal 68A.

[0089] In one or more example audio devices, the audio device is configured to process the audio input signal 56 based on one or more audio sources of interest, such as using one or more processors 10C (e.g., signal processor 64). For example, the signal processor 64 is configured to process the first audio input signal 56A based on the audio source associated with the DOA 58A. For example, the signal processor 64 is configured to process the first audio input signal 56B based on the audio source associated with the DOA 58B.

[0090] In one or more example audio devices, the audio device is configured to process an audio input signal 56 (such as a first audio input signal 56A and / or a second audio input signal 56B) based on the first position data 60AA and / or the second position data 60AB, such as using one or more processors 10C (e.g., a signal processor 64).

[0091] In one or more example audio devices, the matching module 62 is configured to determine that the DOA 58A meets the position criteria. The signal processor 64 can be configured to receive matching parameters 62A including a first matching parameter 62AA from the matching module 62, and the first matching parameter 62AA indicates that the DOA 58A meets the position criteria. The signal processor 64 can be configured to process the audio input signal 56 to provide an audio output signal 68A. In one or more example audio devices, the signal processor 64 is configured to aim at the first mobile object. For example, the signal processor 64 is configured to process an audio source associated with the DOA 58A, such as an audio source associated with the first mobile object. For example, the signal processor 64 is configured to process one or more audio sources associated with the input audio signal 56 that meets the position criteria.

[0092] For example, the signal processor 64 is configured to process the first audio input signal 56A based on an audio source associated with the DOA 58 A. For example, the signal processor 64 is configured to process the first audio input signal 56B based on an audio source associated with the DOA 58B.

[0093] In one or more example audio devices, the signal processor 64 is configured to control beamforming of the signal processor 64 based on whether the DOA 58A satisfies the location criteria. In other words, the signal processor 64 can process the first audio input signal 56A when the DOA 58A satisfies the location criteria. The audio output signal can be considered an audio signal of interest, such as an audio signal associated with a DOA 58A that satisfies the location criteria.

[0094] In one or more example audio devices, the matching module 62 is configured to determine that the DOA 58B does not meet the position criteria. The signal processor 64 can be configured to receive matching parameters 62A including a second matching parameter 62AB from the matching module 62, and the second matching parameter 62AB indicates that the DOA 58B does not meet the position criteria. The signal processor 64 can be configured to process the second audio input signal 56B to attenuate an audio source associated with the DOA 58B (such as an audio source associated with a second mobile object). For example, the signal processor 64 is configured to attenuate one or more audio sources associated with the input audio signal 56 that does not meet the position criteria. In one or more example audio devices, the signal processor 64 can avoid aiming at an audio source associated with the second mobile object. The audio device 10 can be configured to output the audio output signal 68A to a far-end audio device, such as using one or more processors 10C, for example, using the signal processor 64.

[0095] Figure 2 An example scenario for applying the technology disclosed herein is schematically illustrated. Figure 2 An audio device is disclosed, such as an audio device 10 according to the present disclosure (such as Figure 1 ). The audio device 10 includes one or more processors 10C, one or more speakers 10D, one or more microphones 10E (such as a first microphone 10E_1 and / or a second microphone 10E_2), and an ultrasound device 10G including one or more ultrasound transducers 10G. Figure 2 A conference room 300 is shown including three participants or persons (a first participant 1A_1 , a second participant 1B_2 and a third participant 1A_3 ) in which the audio device 10 is used.

[0096] The audio device 10 is configured to output an ultrasonic signal via an ultrasonic device 10F. The audio device 10 is configured to obtain an input ultrasonic signal, such as a first input ultrasonic signal and / or a second input ultrasonic signal, via the ultrasonic device 10F. In one or more example audio devices, the ultrasonic device 10F includes one or more ultrasonic transducers 10G, which include a first ultrasonic transducer and a second ultrasonic transducer. In one or more example audio devices, the first ultrasonic transducer is configured to output an ultrasonic signal having a first frequency, and the second ultrasonic transducer is configured to output an ultrasonic signal having a second frequency. In one or more example audio devices, the ultrasonic device 10F includes an ultrasonic transducer array.

[0097] The audio device 10 is configured to obtain audio input signals, such as a first audio input signal, a second audio input signal, and / or a third audio input signal, via one or more microphones 10E (such as a first microphone 10E_1, a second microphone 10E_2, and / or a third microphone 10E_3). For example, the first audio input signal, the second audio input signal, and / or the third audio input signal may be combined and regarded as a first audio input signal.

[0098] In one or more example audio devices, the audio input signal may indicate an audio signal generated by one or more users of the audio device 10 in the conference room 300, such as the first participant 1A_1, the second participant 1B_2, and / or the third participant 1A_3. In other words, the first participant 1A_1, the second participant 1B_2, and / or the third participant 1A_3 may be considered as a speaker. The first participant 1A_1, the second participant 1B_2, and / or the third participant 1A_3 may take turns speaking in the conference room 300 or may speak at the same time. In one or more example audio devices, the audio input signal may indicate an audio signal generated by one or more users of the audio device 10 when using the audio device 10 in the conference room 300. In one or more example audio devices, the first audio input signal indicates an audio signal generated by the first participant 1A_1. In one or more example audio devices, the second audio input signal indicates an audio signal generated by the second participant 1A_2. In one or more example audio devices, the third audio input signal indicates an audio signal generated by the third participant 1A_3. Alternatively, the audio input signal may be indicative of an audio signal generated by a user located outside of the conference room (such as not using the audio device 10 ).

[0099] The conference room 300 can be considered as a room where a conference is being held between two or more parties, where the two or more parties are located at a long distance from each other. In other words, the audio device 10 can be considered as a conference speakerphone, for example, configured to be used by one party (such as one or more users and / or participants at the near end) to communicate with one or more other parties (such as one or more users and / or participants at the far end). The audio device 10 can be used for large conferences and / or meetings between two or more parties at a long distance from each other. The conference room 300 is a room where the audio device 10 is located, such as at the near end. The audio device 10 can be used by one or more users (such as the first participant 1A_1, the second participant 1A_2, and / or the third participant 1A_3) near the audio device 10 (also referred to as the near end and / or conference room 300).

[0100] In one or more example audio devices, the audio device 10 is configured to use one or more processors 10C and determine a DOA associated with an audio input signal based on the audio input signal. In one or more example audio devices, the audio device 10 is configured to determine a first DOA, a second DOA, and / or a third DOA associated with a first audio input signal, a second audio input signal, and / or a third audio input signal, respectively.

[0101] In one or more example audio devices, the audio device 10 is configured to use one or more processors 10C and determine position data based on an input ultrasonic signal and an output ultrasonic signal. In one or more example audio devices, an ultrasonic device allows the use of the audio device 10 to detect the movement of a user. In one or more example audio devices, the audio device 10 is configured to use one or more processors 10C and determine position data including first position data based on a first input ultrasonic signal and a first output ultrasonic signal. The position data may include first position data indicating a first position of a first mobile object (such as a first participant 1A_1). For example, the first position data includes a first distance and a first direction of the first participant 1A_1. In one or more example audio devices, the audio device 10 is configured to use one or more processors 10C and determine position data including second position data based on a second input ultrasonic signal and a second output ultrasonic signal. The position data may include second position data indicating a second position of a second mobile object (such as a second participant 1A_2). For example, the second position data includes a second distance and a second direction of the second participant 1A_2. In one or more example audio devices, the audio device 10 is configured to use one or more processors 10C and determine position data including third position data based on the third input ultrasonic signal and the third output ultrasonic signal. The position data may include third position data indicating a third position of a third mobile object (such as a third participant 1A_3). For example, the third position data includes a third distance and a third direction of the third participant 1A_3.

[0102] In one or more example audio devices, the audio device 10 is configured to determine one or more amplitude changes based on an input ultrasonic signal and an output ultrasonic signal and using one or more processors 10C. For example, one or more amplitude changes may indicate the movement of a participant located in a conference room. For example, the audio device 10 is configured to determine one or more amplitude changes associated with the first participant 1A_1 based on a first input ultrasonic signal and a first output ultrasonic signal and using one or more processors 10C. For example, the audio device 10 can detect the first participant 1A_1 by outputting a first output ultrasonic signal and obtaining a first input ultrasonic signal, wherein the first output ultrasonic signal is a reflected version of the first output ultrasonic signal when encountering the first participant 1A_1. For example, the audio device 10 is configured to determine one or more amplitude changes associated with the second participant 1A_2 based on a second input ultrasonic signal and a second output ultrasonic signal and using one or more processors 10C. For example, the audio device 10 can detect the second participant 1A_2 by outputting a second output ultrasonic signal and obtaining a second input ultrasonic signal, wherein the second output ultrasonic signal is a reflected version of the second output ultrasonic signal when encountering the second participant 1A_2. For example, the audio device 10 is configured to determine one or more amplitude changes associated with the third participant 1A_3 based on the third input ultrasonic signal and the third output ultrasonic signal and using one or more processors 10C. For example, the audio device 10 can detect the third participant 1A_3 by outputting a third output ultrasonic signal and obtaining a third input ultrasonic signal, wherein the third output ultrasonic signal is a reflected version of the third output ultrasonic signal when encountering the third participant 1A_3.

[0103] In one or more example audio devices, the audio device 10 is configured to use one or more processors 10C to determine a first area indicating no movement. In one or more example audio devices, the audio device 10 is configured to use one or more processors 10C to determine one or more environmental dimension parameters. Environmental dimension parameters can be regarded as the environment of the audio device, such as the surrounding environment of the audio device. For example, the environment in which the audio device is located may include the room in which the audio device is located, such as conference room 300. In one or more example audio devices, environmental dimension parameters can indicate the shape and / or size of conference room 300. In one or more example audio devices, environmental dimension parameters can indicate room mapping.

[0104] In one or more example audio devices, the audio device 10 is configured to determine whether the DOA of the input audio signal meets the location criteria based on the first location data. For example, the audio device 10 is configured to determine whether the first DOA of the first input audio signal meets the location criteria based on the first location data. For example, the audio device 10 is configured to determine whether the second DOA of the second input audio signal meets the location criteria based on the first location data. For example, the audio device 10 is configured to determine whether the third DOA of the third input audio signal meets the location criteria based on the third location data. In one or more example audio devices, the audio device 10 is configured to determine the location criteria based on one or more of the following: a first area, one or more environmental dimension parameters, a first location data, a second location data, and a third location data.

[0105] In one or more example audio devices, the audio device 10 is configured to determine whether the first DOA is within a first direction range associated with the first direction and whether the first distance is within the distance range 304. In one or more example audio devices, the audio device 10 is configured to determine whether the second DOA is within a second direction range associated with the second direction and whether the second distance is within the distance range 304. In one or more example audio devices, the audio device 10 is configured to determine whether the third DOA is within a third direction range associated with the third direction and whether the third distance is within the distance range 304. In one or more example audio devices, the distance range 304 can be less than or equal to 10 meters from the audio device 10. In one or more example audio devices, the distance range can be associated with a distance threshold. For example, the distance range 304 can be considered as a bubble distance, such as a distance from the audio device. The distance range 304 can be associated with a circular shape (such as a circular bubble) so that the distance threshold is the boundary of the circular shape. The distance range 304 can be associated with any other suitable shape.

[0106] In one or more example audio devices, the ultrasound device 10F has a detection range, such as a distance range 304, that depends on the output power of the ultrasound device 10F. In one or more example audio devices, the audio device 10 is configured to determine a location criterion based on the detection range 10. In one or more example audio devices, each of the one or more ultrasonic transducers 10G has a dedicated detection range, such as a first dedicated detection range 302A, a second dedicated detection range 302B, a third dedicated detection range 302C, and a fourth dedicated detection range 302D. For example, a dedicated detection range can be considered a dedicated detection code. A dedicated cone detection can be associated with a width for achieving propagation of an ultrasonic signal. In one or more example audio devices, the first dedicated detection range 302A is associated with the first participant 1A_2. In one or more example audio devices, the second dedicated detection range 302B is associated with the second participant 1A_2. In one or more example audio devices, the third dedicated detection range 302C is associated with the third participant 1A_3. In one or more example audio devices, the fourth dedicated detection range 302D is not associated with any other participant because the ultrasound device has not yet detected it. In other words, the fourth output ultrasonic signal may not have encountered a fourth moving object, such as a fourth participant. In one or more example audio devices, the audio device 10 is configured to determine the location criteria based on the corresponding detection ranges of the one or more ultrasonic transducers 10G, such as the first dedicated detection range 302A, the second dedicated detection range 302B, and / or the third dedicated detection range 302C. In one or more example audio devices, the distance range 304 may be determined based on the output power of the ultrasonic device 10F and / or the one or more ultrasonic transducers 10G.

[0107] In one or more example devices, the audio device 10 is configured to determine one or more audio sources of interest based on a determination of whether a DOA (such as a first DOA, a second DOA, and / or a third DOA) satisfies a location criterion. For example, the first DOA is associated with an audio source of a first participant 1A_1. For example, the second DOA is associated with an audio source of a second participant 1A_2. For example, the third DOA is associated with an audio source of a third participant 1A_3.

[0108] In one or more example audio devices, the audio device 10 is configured to process an audio input signal for providing an audio output signal based on whether a DOA (such as a first DOA, a second DOA, and / or a third DOA) satisfies a position criterion.

[0109] In one or more example audio devices, the audio device 10 is configured to process an audio input signal based on one or more audio sources of interest. For example, the audio device is configured to process a first audio input signal based on an audio source associated with a first DOA (such as associated with the first participant 1A_1). For example, the audio device is configured to process a second audio input signal based on an audio source associated with a second DOA (such as associated with the second participant 1A_2). For example, the audio device is configured to process a third audio input signal based on an audio source associated with a third DOA (such as associated with the third participant 1A_3).

[0110] In one or more example audio devices, the audio device 10 is configured to process an audio input signal, such as the first audio input signal, the second audio input signal 56B, and / or the third audio input signal, based on the first position data, the second position data, and / or the third position data.

[0111] In one or more example audio devices, the audio device 10 is configured to determine that the second DOA and the third DOA meet the location criteria. For example, the audio device 10 is configured to determine that the first DOA is within a first direction range associated with the first direction and the first distance is within a distance range 304, such as a direction and / or distance in the field of view of the audio device. For example, the audio device 10 is configured to determine that the third DOA is within a third direction range associated with the third direction and the third distance is within the distance range 304, such as a direction and / or distance in the field of view of the audio device. In other words, the second participant 1A_2 and the third participant 1A_2 can be located within the second dedicated detection range 302B and the third dedicated detection range 302C, respectively.

[0112] The audio device 10 is configured to process the second audio input signal and the third audio input signal upon determining that the second DOA and the third DOA meet the position criteria for providing the audio output signal.

[0113] In one or more example audio devices, the audio device 10 is configured to target the second participant 1A_2 and the third participant 1A_3. For example, the audio device 10 is configured to forward the second audio input signal and the third audio input signal to the signal processor for processing.

[0114] For example, the audio device 10 is configured to process one or more audio sources associated with the input audio signal that meets the location criteria. For example, the audio device 10 is configured to process the second audio input signal based on the audio source associated with the second DOA. For example, the audio device 10 is configured to process the third audio input signal based on the audio source associated with the third DOA. In other words, the audio device 10 is configured to, such as aim at one or more portions of the audio input signal (such as the second audio input signal and the third audio input signal) from the direction of arrival (such as from the second DOA and the third DOA).

[0115] In one or more example audio devices, the audio device 10 is configured to control the beamforming of the signal processor based on whether the second DOA and the third DOA meet the location criteria. In other words, when the second DOA and the third DOA meet the location criteria, the audio device 10 can adapt the second audio input signal and the third audio input signal. The audio output signal can be regarded as an audio signal of interest, such as an audio signal associated with the second DOA and the third DOA that meet the location criteria.

[0116] In one or more example audio devices, the audio device 10 is configured to determine that the first DOA does not meet the location criteria. In one or more example audio devices, the audio device 10 is configured to attenuate an audio source associated with the first DOA (such as an audio source associated with the first participant 1A_1). The audio device 10 is configured to attenuate a first audio input signal from the audio input signal when determining that the first DOA does not meet the location criteria.

[0117] For example, the audio device 10 is configured to attenuate one or more audio sources associated with an input audio signal that does not meet the location criteria. In one or more example audio devices, the audio device 10 is configured to filter out portions of an audio input signal (such as a first audio input signal) that do not come from a direction of arrival (such as from a first DOA).

[0118] The audio device 10 may be configured to perform FIG. 3A to FIG. 3B Any method disclosed in.

[0119] The processor 10C is optionally configured to execute FIG. 3A to FIG. 3BThe operations of the audio device 10 may be embodied in the form of an executable logic routine (e.g., a line of code, a software program, etc.) that is stored on a non-transitory computer-readable medium (e.g., the memory 10A) and executed by the processor 10C.

[0120] Furthermore, the operation of the audio device 10 may be considered as a method that the audio device 10 is configured to perform. Also, although the functions and operations described may be implemented in software, such functions may also be performed via dedicated hardware or firmware, or some combination of hardware, firmware and / or software.

[0121] The memory 10A may be one or more of the following: a buffer, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory, a random access memory (RAM), and any other suitable device. In a typical arrangement, the memory 10A may include a non-volatile memory for long-term data storage and a volatile memory used as a system memory for the processor 10C. The memory 10A may exchange data with the processor 10C on a data bus. Control lines and an address bus between the memory 10B and the processor 10C may also exist ( Figure 1 Memory 10A is considered a non-transitory computer-readable medium.

[0122] The memory 10A may be configured to store information in a portion of the memory, such as input ultrasound signals, audio input signals, distance to arrival, position data as disclosed herein.

[0123] System 2 may be configured to perform FIG. 3A to FIG. 3B Any method disclosed in .

[0124] Furthermore, the operations of system 2 may be considered as methods that system 2 is configured to perform. Also, although the functions and operations described may be implemented in software, such functions may also be performed via dedicated hardware or firmware, or some combination of hardware, firmware, and / or software.

[0125] FIG. 3A to FIG. 3B A flow chart of an example method, such as method 100, is shown. A method 100 of operating an audio device, such as the audio device disclosed herein (eg, Figure 1 and Figure 2 audio device 10).

[0126] The method 100 comprises outputting S102 an output ultrasonic signal via an ultrasonic device. The method 100 comprises obtaining S104 an input ultrasonic signal via the ultrasonic device. The method 100 comprises obtaining S106 an audio input signal via one or more microphones. The method 100 comprises determining S108 a direction of arrival of the audio input signal based on the audio input signal. The method 100 comprises determining S110 position data based on the input ultrasonic signal and the output ultrasonic signal, the position data comprising first position data indicating a first position of a first mobile object. The method 100 comprises determining S112 whether the direction of arrival satisfies a position criterion based on the first position data. The method 100 comprises processing S114 the audio input signal based on whether the direction of arrival satisfies the position criterion for providing an audio output signal.

[0127] In one or more example methods, processing S114 the audio input signal includes, based on the direction of arrival satisfying a location criterion, the method 100 includes aiming S114A an audio source associated with the direction of arrival.

[0128] In one or more example methods, processing S114 the audio input signal includes, based on the direction of arrival not satisfying the location criterion, the method includes attenuating S114B an audio source associated with the direction of arrival.

[0129] In one or more example methods, the first position data includes a first distance and a first direction of a first moving object.

[0130] In one or more example methods, determining S112 whether the arrival direction satisfies the location criteria includes determining S112A whether the arrival direction is within a first direction range associated with the first direction and whether the first distance is within a distance range. In one or more example methods, based on determining S112A that the arrival direction is within the first direction range associated with the first direction and the first distance is within the distance range, method 100 includes aiming S114A an audio source associated with the arrival direction. In one or more example methods, based on determining S112A that the arrival direction is not within the first direction range associated with the first direction and the first distance is not within the distance range, method 100 includes attenuating S114B an audio source associated with the arrival direction.

[0131] In one or more example methods, the position data includes second position data indicating a second position of the second mobile object. In one or more example methods, the second position data includes a second distance and a second direction of the second mobile object.

[0132] In one or more example methods, determining S112 whether the arrival direction satisfies the location criteria includes determining S112B whether the arrival direction is within a second direction range associated with the second direction and whether the second distance is within a distance range. In one or more example methods, based on determining S112B that the arrival direction is within the second direction range associated with the second direction and the second distance is within the distance range, method 100 includes aiming S114A an audio source associated with the arrival direction. In one or more example methods, based on determining S112B that the arrival direction is not within the second direction range associated with the second direction and the second distance is not within the distance range, method 100 includes attenuating S114B an audio source associated with the arrival direction.

[0133] In one or more example methods, the distance range is less than or equal to 10 meters from the audio device.

[0134] In one or more example methods, determining S110 position data includes determining S110A one or more amplitude changes based on the input ultrasound signal and the output ultrasound signal.

[0135] In one or more example methods, the ultrasound device includes one or more ultrasonic transducers, the one or more ultrasonic transducers including a first ultrasonic transducer and a second ultrasonic transducer. In one or more example methods, the method includes outputting S102A an ultrasonic signal having a first frequency via the first ultrasonic transducer, and outputting S102B an ultrasonic signal having a second frequency via the second ultrasonic transducer.

[0136] In one or more example methods, the method comprises determining S113 one or more audio sources of interest based on the determination of whether the direction of arrival meets the location criteria. In one or more example methods, processing S114 the audio input signal comprises processing S114C the audio input signal based on the one or more audio sources of interest.

[0137] In one or more example methods, the method includes determining S111 a location criterion based on a first distance and a first direction.

[0138] In one or more example methods, processing of the audio input signal is based on the first position data and / or the second position data.

[0139] In one or more example methods, processing S114 the audio input signal includes aiming S114D the first moving object and / or the second moving object.

[0140] In one or more example methods, processing S114 the audio input signal includes controlling S114E beamforming of a signal processor based on whether the direction of arrival meets a position criterion.

[0141] In one or more example methods, determining S110 location data includes determining a first area indicating S110B that there is no movement. In one or more example methods, the location criterion is based on the first area.

[0142] In one or more example methods, determining S110 location data includes determining S110C one or more environmental dimension parameters. In one or more example methods, the location criteria is based on the one or more environmental dimension parameters.

[0143] Examples of audio devices, systems, and methods according to the present disclosure are set forth in the following items:

[0144] Item 1. An audio device comprising an interface, a memory, one or more processors including a signal processor, one or more speakers, one or more microphones, and an ultrasound device, wherein the audio device is configured to:

[0145] outputting an ultrasonic signal via an ultrasonic device;

[0146] obtaining an input ultrasound signal via an ultrasound device;

[0147] obtaining an audio input signal via one or more microphones;

[0148] determining a direction of arrival of the audio input signal based on the audio input signal;

[0149] determining position data based on the input ultrasound signal and the output ultrasound signal, the position data including first position data indicating a first position of a first mobile object;

[0150] determining whether the direction of arrival meets the location criteria based on the first location data; and

[0151] The audio input signal is processed for providing an audio output signal based on whether the direction of arrival meets a position criterion.

[0152] Item 2. An audio device according to Item 1, wherein processing the audio input signal includes: aiming at an audio source associated with the arrival direction based on the arrival direction satisfying a position criterion.

[0153] Item 3. An audio device according to Item 1, wherein processing the audio input signal includes: attenuating an audio source associated with the arrival direction based on the arrival direction not satisfying the position criterion.

[0154] Item 4. The audio device of any one of the preceding items, wherein the first position data comprises a first distance and a first direction of the first moving object.

[0155] Item 5. An audio device according to Item 4, wherein determining whether the arrival direction meets the location criteria includes determining whether the arrival direction is within a first direction range associated with the first direction and whether the first distance is within the distance range.

[0156] Item 6. An audio device according to any one of the preceding items, wherein the position data includes second position data indicating a second position of the second moving object, wherein the second position data includes a second distance and a second direction of the second moving object.

[0157] Item 7. An audio device according to Item 6, wherein determining whether the arrival direction meets the location criteria includes determining whether the arrival direction is within a second direction range associated with the second direction and whether the second distance is within the distance range.

[0158] Item 8. An audio device according to any one of Items 5 to 7, wherein the distance range is less than or equal to 10 meters from the audio device.

[0159] Item 9. The audio device of any of the preceding items, wherein determining the position data comprises determining one or more amplitude changes based on the input ultrasonic signal and the output ultrasonic signal.

[0160] Item 10. An audio device according to any one of the preceding items, wherein the ultrasonic device includes one or more ultrasonic transducers, the one or more ultrasonic transducers include a first ultrasonic transducer and a second ultrasonic transducer, and wherein the first ultrasonic transducer is configured to output an ultrasonic signal having a first frequency, and the second ultrasonic transducer is configured to output an ultrasonic signal having a second frequency.

[0161] Item 11. An audio device according to any of the preceding items, wherein the audio device is configured to determine one or more audio sources of interest based on a determination of whether the arrival direction satisfies a location criterion, and wherein processing the audio input signal includes processing the audio input signal based on the one or more audio sources of interest.

[0162] Item 12. An audio device according to any one of Items 4 to 11, wherein the audio device is configured to determine the location criterion based on a first distance and a first direction.

[0163] Item 13. An audio device according to any of the preceding items or any of items 6 to 12, wherein the processing of the audio input signal is based on the first position data and / or the second position data.

[0164] Item 14. The audio device of Item 13, wherein processing the audio input signal comprises aiming at the first moving object and / or the second moving object.

[0165] Item 15. An audio device as described in any of the preceding items, wherein the ultrasound device has a detection range that depends on the output power of the ultrasound device, and wherein the position criterion is based on the detection range.

[0166] Item 16. An audio device as described in any one of Items 10 to 14, wherein each of the one or more ultrasonic transducers has a dedicated detection range, and wherein the position criterion is based on the respective detection ranges of the one or more ultrasonic transducers.

[0167] Item 17. An audio device according to any of the preceding items, wherein processing the audio input signal includes controlling beamforming of a signal processor based on whether the direction of arrival meets a position criterion.

[0168] Item 18. An audio device as described in any of the preceding items, wherein determining the position data includes determining a first area indicating no movement, and wherein the position criteria is based on the first area.

[0169] Item 19. An audio device according to any of the preceding items, wherein determining the location data includes determining one or more environmental dimension parameters, and wherein the location criteria is based on the one or more environmental dimension parameters.

[0170] Item 20. A conferencing system comprising an audio device according to any one of Items 1 to 17 and one or more ultrasonic transducers.

[0171] Item 21. A method of operating an audio device, the audio device comprising an interface, a memory, one or more processors including a signal processor, one or more speakers, one or more microphones, and an ultrasound device, the method comprising:

[0172] outputting an ultrasonic signal via an ultrasonic device;

[0173] obtaining an input ultrasound signal via an ultrasound device;

[0174] obtaining an audio input signal via one or more microphones;

[0175] determining a direction of arrival of the audio input signal based on the audio input signal;

[0176] determining position data based on the input ultrasound signal and the output ultrasound signal, the position data including first position data indicating a first position of a first mobile object;

[0177] determining whether the direction of arrival meets the location criteria based on the first location data; and

[0178] The audio input signal is processed for providing an audio output signal based on whether the direction of arrival meets a position criterion.

[0179] Item 22. A method for operating an audio device according to Item 21, wherein processing the audio input signal includes: aiming at an audio source associated with the arrival direction based on the arrival direction satisfying a position criterion.

[0180] Item 23. A method for operating an audio device according to Item 22, wherein processing the audio input signal includes: aiming at an audio source associated with the arrival direction based on the arrival direction satisfying a position criterion.

[0181] Item 24. A method of operating an audio device according to any one of Items 21 to 23, wherein the first position data comprises a first distance and a first direction of the first moving object.

[0182] Item 25. A method of operating an audio device according to Item 24, wherein determining whether the arrival direction meets the location criteria includes determining whether the arrival direction is within a first direction range associated with the first direction and whether the first distance is within the distance range.

[0183] Item 26. A method of operating an audio device according to any one of Items 21 to 25, wherein the position data includes second position data indicating a second position of a second moving object, wherein the second position data includes a second distance and a second direction of the second moving object.

[0184] Item 27. A method of operating an audio device according to Item 26, wherein determining whether the arrival direction meets the location criteria includes determining whether the arrival direction is within a second direction range associated with the second direction and whether the second distance is within the distance range.

[0185] Item 28. A method for operating an audio device according to any one of Items 25 to 27, wherein the distance range is less than or equal to 10 meters from the audio device.

[0186] Item 29. A method of operating an audio device according to any one of Items 21 to 28, wherein determining the position data includes determining one or more amplitude changes based on the input ultrasonic signal and the output ultrasonic signal.

[0187] Item 30. A method for operating an audio device according to any one of Items 21 to 29, wherein the ultrasonic device includes one or more ultrasonic transducers, the one or more ultrasonic transducers include a first ultrasonic transducer and a second ultrasonic transducer, wherein the method includes outputting an ultrasonic signal having a first frequency via the first ultrasonic transducer, and outputting an ultrasonic signal having a second frequency via the second ultrasonic transducer.

[0188] Item 31. A method for operating an audio device according to any one of Items 21 to 30, wherein the method includes determining one or more audio sources of interest based on a determination of whether a direction of arrival satisfies a location criterion, and wherein processing an audio input signal includes processing the audio input signal based on the one or more audio sources of interest.

[0189] Item 32. A method of operating an audio device according to any one of Items 24 to 31, wherein the method includes determining a position criterion based on a first distance and a first direction.

[0190] Item 33. A method of operating an audio device according to any one of items 21 to 32 or any one of items 26 to 32, wherein processing of the audio input signal is based on the first position data and / or the second position data.

[0191] Item 34. A method of operating an audio device according to Item 33, wherein processing the audio input signal includes aiming at the first moving object and / or the second moving object.

[0192] Item 35. A method of operating an audio device according to any one of Items 21 to 34, wherein processing the audio input signal includes controlling the beamforming of the signal processor based on whether the arrival direction meets the position criterion.

[0193] Item 36. A method of operating an audio device according to any one of Items 21 to 35, wherein determining the position data includes determining a first area indicating no movement, and wherein the position criterion is based on the first area.

[0194] Item 37. A method of operating an audio device according to any one of Items 21 to 36, wherein determining the location data includes determining one or more environmental dimension parameters, and wherein the location criteria is based on the one or more environmental dimension parameters.

[0195] The use of the terms "first", "second", "third", and "fourth", "primary", "secondary", "tertiary", etc. does not imply any specific order, but is included to identify individual elements. In addition, the use of the terms "first", "second", "third", and "fourth", "primary", "secondary", "tertiary", etc. does not imply any order or importance, but rather the terms "first", "second", "third", and "fourth", "primary", "secondary", "tertiary", etc. are used to distinguish one element from another. Note that the words "first", "second", "third", and "fourth", "primary", "secondary", "tertiary", etc. used here and elsewhere are used only for labeling purposes and are not intended to represent any particular spatial or temporal order. In addition, the labeling of the first element does not imply the presence of the second element, and vice versa.

[0196] It is understood that the accompanying drawings include some circuits or operations shown in solid lines and some circuits, components, features or operations shown in dotted lines. The circuits or operations included in the solid lines are circuits, components, features or operations included in the most extensive examples. The circuits, components, features or operations included in the dotted lines are examples that can be included in the circuits, components, features or operations of the solid line examples, or are parts of the circuits, components, features or operations of the solid line examples, or are other circuits, components, features or operations that can be adopted in addition to the circuits, components, features or operations of the solid line examples. It should be understood that these operations do not need to be performed in the order presented. In addition, it should be understood that not all operations need to be performed. The example operations can be performed in any order and in any combination. It should be understood that these operations do not need to be performed in the order presented. The circuits, components, features or operations included in the dotted lines can be considered optional.

[0197] Other operations not described herein may be incorporated into the example operations. For example, one or more additional operations may be performed before, after, simultaneously with, or between any described operations.

[0198] Certain features discussed above as separate implementations may also be implemented in combination as a single implementation. Conversely, features described as a single implementation may also be implemented in multiple implementations, either individually or in any suitable subcombination. Furthermore, although features may be described above as functioning in certain combinations, in some cases one or more features from a claimed combination may be deleted from that combination, and the combination may be claimed as any subcombination or variation of any subcombination.

[0199] It should be noted that the word "comprising" does not necessarily exclude the presence of other elements or steps than those listed.

[0200] It should be noted that the word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.

[0201] It should also be noted that any reference signs do not limit the scope of the claims, that examples may be implemented at least partially by means of both hardware and software, and that several "tools", "units" or "means" may be represented by the same piece of hardware.

[0202] Although features have been shown and described, it should be understood that they are not intended to limit the claimed disclosure, and it will be apparent to those skilled in the art that various changes and modifications may be made without departing from the scope of the claimed invention. Therefore, this specification and drawings should be considered illustrative rather than restrictive. The claimed disclosure is intended to cover all alternatives, modifications and equivalents.

[0203] Reference numerals list

[0204] 2Conference System

[0205] 10 Audio Devices

[0206] 10A Memory

[0207] 10B One or more interfaces

[0208] 10C One or more processors

[0209] 10D One or more speakers

[0210] 10E One or more microphones

[0211] 10E_1 First Microphone

[0212] 10E_2 Second microphone

[0213] 10F Ultrasound Equipment

[0214] 10FA detection range

[0215] 10FB dedicated detection range

[0216] 10G one or more ultrasound transducers

[0217] 10GA first ultrasonic transducer

[0218] 10GB Second Ultrasonic Transducer

[0219] 52 Output ultrasonic signal

[0220] 52A First output ultrasonic signal

[0221] 52B Second output ultrasonic signal

[0222] 54 Input ultrasonic signal

[0223] 54A First input ultrasonic signal

[0224] 54B Second input ultrasonic signal

[0225] 56 audio input signal

[0226] 56A First audio input signal

[0227] 56B Second audio input signal

[0228] 58 Arrival direction determination module

[0229] 58A First Arrival Direction

[0230] 58B Second arrival direction

[0231] 60 Position determination module

[0232] 60A position data

[0233] 60AA first position data

[0234] 60AB second position data

[0235] 60C One or more environmental parameters

[0236] 60D First Area

[0237] 62 matching modules

[0238] 62A matching parameters

[0239] 62AA first match parameter

[0240] 62AB second matching parameters

[0241] 64 signal processors

[0242] 68A audio output signal

[0243] Meeting Room 300

[0244] 302A First dedicated detection range

[0245] 302B Second dedicated detection range

[0246] 302C Third dedicated detection range

[0247] 302D Fourth Special Detection Range

[0248] 304 Distance Range

[0249] 1A_1 First Participant

[0250] 1A_2 Second participant

[0251] 1A_3 Third Participant

[0252] 100 ways to operate audio equipment

[0253] S102 outputs the ultrasonic signal

[0254] S104 obtains input ultrasonic signal

[0255] S106 obtains audio input signal

[0256] S108 Determine the arrival direction

[0257] S110 Determine location data

[0258] S111 Determine location criteria

[0259] S112 Determine whether the arrival direction meets the location standard

[0260] S113 Determine one or more audio sources of interest

[0261] S114 processes the audio input signal.

Claims

1. An audio device comprising an interface, a memory, one or more processors including a signal processor, one or more speakers, one or more microphones, and an ultrasound device, wherein: The audio device is configured as: outputting an ultrasonic signal via the ultrasonic device; obtaining an input ultrasonic signal via the ultrasonic device; obtaining an audio input signal via the one or more microphones; determining a direction of arrival of the audio input signal based on the audio input signal; determining position data based on the input ultrasonic signal and the output ultrasonic signal, the position data comprising first position data indicating a first position of a first mobile object; determining whether the direction of arrival meets a location criterion based on the first location data; as well as The audio input signal is processed for providing an audio output signal based on whether the direction of arrival meets the position criterion.

2. The audio device according to claim 1, wherein Processing the audio input signal includes targeting an audio source associated with the direction of arrival based on the direction of arrival satisfying the location criterion.

3. An audio device according to any one of the preceding claims, wherein The first position data includes a first distance and a first direction of the first moving object.

4. The audio device according to claim 3, wherein: Determining whether the direction of arrival satisfies the location criteria includes determining whether the direction of arrival is within a first direction range associated with the first direction and whether the first distance is within a distance range.

5. An audio device according to any one of the preceding claims, wherein The position data includes second position data indicating a second position of a second moving object, wherein the second position data includes a second distance and a second direction of the second moving object.

6. The audio device according to claim 5, wherein: Determining whether the direction of arrival satisfies the location criteria includes determining whether the direction of arrival is within a second direction range associated with the second direction and whether the second distance is within a distance range.

7. An audio device according to any one of the preceding claims, wherein: The ultrasonic device includes one or more ultrasonic transducers, the ultrasonic transducers include a first ultrasonic transducer and a second ultrasonic transducer, and wherein the first ultrasonic transducer is configured to output an ultrasonic signal having a first frequency and the second ultrasonic transducer is configured to output an ultrasonic signal having a second frequency.

8. An audio device according to any one of the preceding claims, wherein: The audio device is configured to determine one or more audio sources of interest based on a determination of whether the direction of arrival satisfies the location criteria, and wherein processing the audio input signal comprises processing the audio input signal based on the one or more audio sources of interest.

9. The audio device according to any one of claims 3 to 8, wherein: The audio device is configured to determine the location criterion based on a first distance and a first direction.

10. An audio device according to any one of the preceding claims or any one of claims 5 to 9, wherein: The processing of the audio input signal is based on the first position data and / or the second position data.

11. An audio device according to any preceding claim, wherein: The ultrasound device has a detection range that depends on an output power of the ultrasound device, and wherein the position criterion is based on the detection range.

12. The audio device according to any one of claims 7 to 11, wherein: The ultrasonic transducer or each of the plurality of ultrasonic transducers has a dedicated detection range, and wherein the position criterion is based on the respective detection range of the ultrasonic transducer or the plurality of ultrasonic transducers.

13. An audio device according to any preceding claim, wherein: Determining the location data includes determining a first area indicating no movement, and wherein the location criteria is based on the first area.

14. An audio device according to any preceding claim, wherein: Determining the location data comprises determining one or more environmental dimension parameters, and wherein the location criteria is based on the one or more environmental dimension parameters.

15. A conference system comprising the audio device according to any one of claims 1 to 14 and one or more ultrasonic transducers.

16. A method of operating an audio device, the audio device comprising an interface, a memory, one or more processors including a signal processor, one or more speakers, one or more microphones, and an ultrasound device, the method comprising: outputting an ultrasonic signal via the ultrasonic device; obtaining an input ultrasonic signal via the ultrasonic device; obtaining, using the audio device, an audio input signal via the one or more microphones; determining a direction of arrival of the audio input signal based on the audio input signal; determining position data based on the input ultrasonic signal and the output ultrasonic signal, the position data comprising first position data indicating a first position of a first mobile object; determining whether the direction of arrival meets a location criterion based on the first location data; as well as The audio input signal is processed for providing an audio output signal based on whether the direction of arrival meets the position criterion.

Images