Cancellation of ultrasonic signals

Abstract

Example embodiments describe an apparatus, method, and computer program relating to cancellation of an ultrasonic signal. The apparatus may include means for providing first data derived from a signal received by a microphone of the user device and means for providing second data representative of mechanical vibrations within a gyroscope of the user device. The apparatus may further include means for detecting, based on the first data and the second data, that the signal received by the microphone includes an ultrasonic signal, means for processing the third data to generate fourth data representing an estimate of one or more audible sounds in the first data, and means for transmitting the fourth data to the microphone. Wherein the third data is the second data or is derived from the second data, and means for subtracting the fourth data from the first data to provide fifth data.

Description

Technical Field

[0001] Example embodiments relate to cancellation of ultrasonic signals.For example, embodiments may relate to apparatus, methods, and computer programs for canceling ultrasonic signals that may be expected to cause an unwanted event or output at a user device. Background Art

[0002] An apparatus, such as a user device, may include one or more microphones for picking up audio signals, which may then be used for various purposes.

[0003] For example, a hearing aid may amplify a received audio signal for output via a speaker to help a hearing-impaired wearer better hear external sounds. For example, an earbud may process a received audio signal for active noise reduction (ANC) and / or pass it to a speaker in a so-called transparent operating mode. For example, a digital assistant may respond to certain utterances in a received audio signal to perform certain functions, such as playing a music track, obtaining information from a remote web service, and / or initiating a call or payment to a remote service.

[0004] Receiving ultrasonic signals by the microphone or microphones of such a device may be problematic. Summary of the invention

[0005] The protection scope of various embodiments of the present invention is defined by the independent claims. Embodiments and features described in this specification that do not belong to the scope of the independent claims (if any) should be interpreted as examples that help understand the various embodiments of the present invention.

[0006] According to one aspect, a device is described, comprising: a component for providing first data, the first data being derived from a signal received by a microphone of a user device; a component for providing second data, the second data representing mechanical vibrations within a gyroscope of the user device; a component for detecting, based on the first data and the second data, that the signal received by the microphone includes an ultrasonic signal; a component for processing third data to generate fourth data, the fourth data representing an estimate of one or more audible sounds in the first data, wherein the third data is the second data or is derived from the second data; and a component for subtracting the fourth data from the first data to provide fifth data.

[0007] The processing component may include a filter configured to receive the third data and generate fourth data based at least in part on filter coefficients of the filter.

[0008] The filter may be an adaptive filter in which current values ​​of filter coefficients are modified based at least in part on the fifth data to provide updated filter coefficients.

[0009] The current value of the filter coefficient may be modified based at least in part on multiplying the fifth data by the third data and adding to the current filter coefficient.

[0010] The current values ​​of the filter coefficients may also be modified based on the adaptation rate parameter.

[0011] In some examples, a first instance of third data may be associated with a first time window for use in generating a first set of updated filter coefficients, one or more consecutive instances of the third data may be associated with consecutive time windows for use in generating one or more consecutive sets of updated filter coefficients, and wherein the device may be configured to iteratively modify the filter coefficients using the one or more consecutive instances of the third data until a predetermined condition is met.

[0012] The predetermined condition may be met when the predetermined mean square error value is lower than a predetermined threshold.

[0013] The second data may represent two or more sets of gyroscope data, the two or more gyroscope data representing mechanical vibrations for respective axes of the gyroscope, and wherein the apparatus may further comprise means for pre-processing the second data to provide the third data.

[0014] The pre-processing component may be configured to select one of the two or more gyroscope data sets to provide third data based on the selected set representing a maximum value of the mechanical vibration.

[0015] The pre-processing component may be configured to provide the third data as a weighted sum of two or more gyroscope data sets.

[0016] The detecting component may be configured to detect that the signal received by the microphone includes the ultrasonic signal for a time period T, and wherein the subtracting component is configured to subtract the fourth data from the first data only for a portion of the time period T.

[0017] The subtracting component may be configured to: for a plurality of spaced-apart sub-periods within the time period T, subtract the fourth data from the first data.

[0018] The first data may be provided at an output of one or more non-linear components that process a signal received by the microphone.

[0019] The detecting component may be configured to detect that the signal received by the microphone includes an ultrasonic signal based at least in part on identifying non-zero values ​​of the first data and the second data or the third data for one or more corresponding instances or time periods.

[0020] The detection component may be configured to perform amplitude envelope correlation using corresponding waveforms represented by the first data and the second data or the third data to generate a first parameter indicating similarity between the corresponding waveforms, and wherein the detection may be based at least in part on the first parameter.

[0021] The detection component may be configured to perform a spectral analysis of frequency domain representations of respective waveforms represented by the first data and the second data or the third data to generate a second parameter indicative of similarity between the frequency domain representations, and wherein the detection may be based at least in part on the second parameter.

[0022] The detecting may be based at least in part on the first parameter and the second parameter satisfying respective predetermined conditions.

[0023] The detection component may include one or more machine learning models trained using training data, the training data including a predetermined set of first parameters and second parameters, the predetermined set being known to be generated in response to an ultrasonic signal being sent to a user device, wherein the detection may be based on the output of the one or more machine learning models.

[0024] According to a second aspect, a method is described, comprising: providing first data derived from a signal received by a microphone of a user device; providing second data representing mechanical vibrations within a gyroscope of the user device; based on the first data and the second data, detecting that the signal received by the microphone includes an ultrasonic signal; processing third data to generate fourth data, the fourth data representing an estimate of one or more audible sounds in the first data, wherein the third data is the second data or is derived from the second data; and subtracting the fourth data from the first data to provide fifth data.

[0025] The process may include a filter configured to receive the third data and generate fourth data based at least in part on filter coefficients of the filter.

[0026] The filter may be an adaptive filter in which current values ​​of filter coefficients are modified based at least in part on the fifth data to provide updated filter coefficients.

[0027] The current value of the filter coefficient may be modified based at least in part on: multiplying the fifth data with the third data and adding to the current filter coefficient.

[0028] The current values ​​of the filter coefficients may also be modified based on the adaptation rate parameter.

[0029] In some examples, a first instance of third data may be associated with a first time window for use in generating a first set of updated filter coefficients, one or more consecutive instances of the third data may be associated with consecutive time windows for use in generating one or more consecutive sets of updated filter coefficients, and wherein the filter coefficients may be iteratively modified using the one or more consecutive instances of the third data until a predetermined condition is met.

[0030] The predetermined condition may be met when the predetermined mean square error value is lower than a predetermined threshold.

[0031] The second data may represent two or more sets of gyroscope data, the two or more gyroscope data representing mechanical vibrations for respective axes of the gyroscope, and wherein the method may further comprise pre-processing the second data to provide the third data.

[0032] The pre-processing may select one of the two or more gyroscope data sets to provide the third data based on the selected set representing maximum values ​​of the mechanical vibrations.

[0033] The pre-processing may provide the third data as a weighted sum of two or more gyroscope data sets.

[0034] The detecting may include detecting that a signal received by the microphone includes an ultrasonic signal for a time period T, and wherein the subtracting may include subtracting the fourth data from the first data only for a portion of the time period T.

[0035] The subtraction may subtract the fourth data from the first data for a plurality of spaced-apart sub-periods within the time period T.

[0036] The first data may be provided at an output of one or more non-linear components, which process the signal received by the microphone.

[0037] The detecting can include detecting that the signal received by the microphone includes an ultrasonic signal based at least in part on identifying non-zero values ​​of the first data and the second or third data for one or more corresponding instances or time periods.

[0038] The detecting may include performing an amplitude envelope correlation using corresponding waveforms represented by the first data and the second data or the third data for generating a first parameter indicative of similarity between the corresponding waveforms, and wherein the detecting may be based at least in part on the first parameter.

[0039] The detecting may comprise performing a spectral analysis of frequency domain representations of respective waveforms represented by the first data and the second data or the third data for generating a second parameter indicative of similarity between the frequency domain representations, and wherein the detecting may be based at least in part on the second parameter.

[0040] The detecting may be based at least in part on the first parameter and the second parameter satisfying respective predetermined conditions.

[0041] The detection may include one or more machine learning models trained using training data, the training data including a predetermined set of first parameters and second parameters, the predetermined sets known to be generated in response to an ultrasonic signal being sent to a user device, wherein the detection may be based on an output of the one or more machine learning models.

[0042] According to a third aspect, a computer program is described, comprising instructions for causing an apparatus to perform at least the following: providing first data derived from a signal received by a microphone of a user device; providing second data representing mechanical vibrations within a gyroscope of the user device; based on the first data and the second data, detecting that the signal received by the microphone includes an ultrasonic signal; processing the third data to generate fourth data representing an estimate of one or more audible sounds in the first data, wherein the third data is the second data or is derived from the second data; and subtracting the fourth data from the first data to provide fifth data.

[0043] The third aspect may further include any features of the second aspect.

[0044] According to a fourth aspect, the present specification describes a computer-readable medium (such as a non-volatile computer-readable medium) comprising program instructions stored thereon for performing at least the following items: providing first data derived from a signal received by a microphone of a user device; providing second data representing mechanical vibrations within a gyroscope of the user device; based on the first data and the second data, detecting that the signal received by the microphone includes an ultrasonic signal; processing third data to generate fourth data representing an estimate of one or more audible sounds in the first data, wherein the third data is the second data or is derived from the second data; and subtracting the fourth data from the first data to provide fifth data.

[0045] The fourth aspect may further include any features of the second aspect.

[0046] According to a fifth aspect, the present specification describes a device comprising: at least one processor; and at least one memory, the at least one memory comprising computer program code, which, when executed by the at least one processor, causes the device to: provide first data derived from a signal received by a microphone of a user device; provide second data representing mechanical vibrations within a gyroscope of the user device; based on the first data and the second data, detect that the signal received by the microphone includes an ultrasonic signal; process third data to generate fourth data representing an estimate of one or more audible sounds in the first data, wherein the third data is the second data or is derived from the second data; and subtract the fourth data from the first data to provide fifth data.

[0047] The fifth aspect may further include any features of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a perspective view of a first example user device, which is a digital assistant, that may be used in an example embodiment;

[0049] Figure 2 is a perspective view of a second example user device, which is a set of headphones that may be used in an example embodiment;

[0050] Figure 3 yes Figure 2 A schematic diagram of components of a left-hand earphone of an earphone set shown in , which may be used to understand example embodiments;

[0051] Figure 4 is a schematic diagram of components of an apparatus according to one or more example embodiments;

[0052] Figure 5 is a schematic diagram of an example gyroscope that can be used as Figure 4 A component in a component of;

[0053] Figure 6 is a graph showing a waveform representing gyroscope data from a gyroscope, which waveform may be indicative of an ultrasonic attack;

[0054] Figure 7 is a flowchart illustrating operations that may be performed according to one or more example embodiments;

[0055] Figure 8 is a schematic diagram of components of another apparatus according to one or more example embodiments;

[0056] Fig. 9 is a block diagram illustrating an example method used by a controller component for detecting an ultrasonic attack according to one or more example embodiments;

[0057] Fig.10 is a block diagram illustrating another example method used by a controller component for detecting an ultrasonic attack according to one or more example embodiments;

[0058] Fig.11 is a block diagram illustrating another example method used by a controller component for detecting an ultrasonic attack according to one or more example embodiments;

[0059] Fig.12 is a schematic block diagram of an apparatus according to one or more example embodiments; and

[0060] Fig.13 is a non-transitory medium for storing computer-readable instructions that, when executed or processed by one or more processors of a device, may perform operations according to one or more example embodiments. DETAILED DESCRIPTION

[0061] Example embodiments relate to cancellation of ultrasonic signals.

[0062] For example, example embodiments may relate to apparatus, methods, and computer programs for the elimination of ultrasonic signals that may be expected to cause unwanted events or unwanted outputs by a user device. This may be referred to as an "ultrasonic attack."

[0063] The apparatus itself may include a user device, and the method operations described herein may be performed by the user device. However, in some cases, the apparatus may be an entity different from the user device, such as a smart phone, a tablet computer, or other computer system that communicates with the user device, and the apparatus performs the method operations described herein.

[0064] A user device may be a device that includes one or more microphones and can be used by (e.g., interacts with and / or is carried by) one or more users. Examples may include a smartphone, a tablet computer, a wearable computer, a digital assistant, a hearing aid, and a headset (or a pair of headsets). The term headset also includes earbuds and headphones.

[0065] As is known, some headphones include at least one microphone that can be used as part of an active noise cancellation (ANC) system. Sometimes, headphones contain two or more microphones, such as an external microphone and an internal microphone, to provide advanced methods of ANC. Sometimes, headphones may be operated in a so-called transparent mode, in which at least some of the external sounds received by the microphone are passed to the headphone's speakers so that the user can hear part of their environment.

[0066] An ultrasonic attack may involve some external entity sending unwanted content encoded or modulated onto an ultrasonic carrier signal (above 20kHz).

[0067] The unwanted content may be for malicious purposes, such as to annoy the user, play unsolicited advertisements, control certain aspects of a voice response system (e.g., a smart speaker or digital assistant), and / or initiate communications with one or more external systems. For example, the unwanted content may include speech that contains one or more commands that will cause a digital assistant to make a digital payment.

[0068] Due to inherent nonlinearities in one or more components of the user device, the ultrasonic signal received by the microphone of the user device may be shifted down in the frequency domain to the audio range (typically between 20 Hz and 20 kHz). Here, the "derived signal" produced by the nonlinearity can be manipulated or otherwise processed like an audio signal. For example, the derived signal can be manipulated or processed by one or more audio processing functional components, such as an amplifier, an ANC processor, a digital assistant processor (e.g., a voice processing function), a system on a chip that provides such functions, etc.

[0069] Nonlinearities are often present in the preamplifier and / or analog-to-digital converter (ADC) located between the microphone and the audio processing functions.

[0070] Figure 1 An example user device in the form of a digital assistant 100 is shown.

[0071] The digital assistant 100 may include a body 102, a display screen 104 (which may or may not be touch-sensitive), a microphone 106, and first and second speakers 108A and 108B. The digital assistant 100 may include processing circuitry or functionality for receiving and processing signals received by the microphone 106.

[0072] For example, the digital assistant 100 may include processing functionality, such as speech processing functionality, for detecting one or more spoken utterances in an audio signal. The one or more spoken utterances may be processed to perform one or more actions. Some actions may be offline actions that do not require communication with one or more remote devices over a network. An example of an offline action may be notifying a user of the current date and / or time based on a verbal request. Some actions may be online actions that require communication with one or more remote devices over a network in order to receive the required data. For example, a spoken utterance may request an image, video, or music from a remote device or server for playback by the digital assistant 100. Image and video data may be output using the display screen 104, and audio data may be output using the first speaker 108A and the second speaker 108B.

[0073] The digital assistant 100 may include one or more communication modules for processing online actions. Example communication modules may operate according to a radio access network (RAN) standard such as 3G, 4G, 5G, or a future generation standard. Alternatively, or in addition, there may be a communication module for WiFi communication or similar communication. Alternatively, or in addition, there may be a communication module for short-range communication using, for example, Bluetooth or Zigbee.

[0074] Figure 2 Another example user device is shown in the form of a pair of headphones 200 .

[0075] A pair of headphones 200 includes a left-hand earphone 202 and a right-hand earphone 204. The left-hand earphone 202 may include one or more external microphones 206 on a first side and a speaker 208 on a second side that is partially located within a channel 209. In some cases, a built-in microphone (not shown) may also be provided within the channel 209 for enhanced ANC processing.

[0076] The same components may be provided on the right earphone 204, although this is not necessarily the case.

[0077] At least one of the earphones 200, in this case the left-hand earphone 202, may include processing circuitry for receiving and processing signals received by the external microphone 206. For example, the left-hand earphone 202 may include processing functionality for manipulating or processing data derived from the received signals, such as by performing ANC, where a cancellation signal may be output to the speaker 208, or by providing a filtered or amplified version of the received signal to the speaker.

[0078] The processing functions may be performed locally, such as by one or more controllers on the left hand earphone 202 and / or at least some processing functions may be performed by an external device, such as the smartphone 220. The smartphone 220 may communicate with the earphone 200 using any of the communication modules and standards described above.

[0079] Figure 3 2 is a schematic diagram of the functional components of the left-hand earphone 202, which may be helpful in understanding the exemplary embodiment. The same components and operations may be provided and performed by the right-hand earphone 204.

[0080] The components of the left hand earphone 202 are shown in dashed boxes. The left hand earphone 202 may include an external microphone 206, a preamplifier 304, an ADC 306, a processor 308, a digital-to-analog converter (DAC) 310, and a speaker 312. The processor 308 may include the processing circuitry described above or provide such functionality.

[0081] Reference numeral 300 shows a frequency domain representation of an ultrasonic signal sent to the left-hand earphone 202. The ultrasonic signal is represented by a frequency band 301 centered around a frequency 302 outside the audible range (above 20kHz). This ultrasonic signal will not be heard by the user of the earphone pair 200 and does not serve any intended purpose to the user; it may be a form of ultrasonic attack as described above. As shown in the output of the ADC 306, inherent nonlinearities in the one or more preamplifiers 304 and the ADC may shift the components of the frequency band 301 down into the audible range, below 20kHz. This derived version of the ultrasonic signal may be considered by the processor 308 as a legitimate audio input if within the low-pass filter response 324 before the processor 308. This audio input may be used for the execution of ANC, filtering and / or amplification to the speaker 312.

[0082] exist Figure 1 In the case of the digital assistant 100, the same issues may apply, whereby the ultrasonic attack may be reduced to the audible range, below 20 kHz, due to inherent nonlinearities in one or more components of the digital assistant. Thus, the processing functionality of the digital assistant 100 configured to receive and process signals in the audible range (e.g., spoken utterances and actions based on those spoken utterances) may manipulate the signals or data derived from the ultrasonic attack to perform certain offline and / or online actions described above.

[0083] Example embodiments are intended to prevent or at least mitigate the risk of interference with one or more microphones such as Figure 1 The microphone 106 of the digital assistant 100 or Figure 2 An event associated with the reception of an ultrasonic signal by the external microphone 206 of the left-hand earphone 202.

[0084] Figure 4 is a schematic diagram of an apparatus 400 according to some example embodiments.

[0085] The device 400 may be similar to Figure 3 304, ADC 306, processor 308, DAC 310, and speaker 312.

[0086] Although not shown, one or more other processing functions may be provided at the output of the processor 308 in addition to or as an alternative to the DAC 310 and the speaker 312. For example, speech processing functionality may be provided to perform offline or online actions in response to detecting one or more spoken utterances.

[0087] The device 400 may also include a gyroscope 402, which may be provided as part of the device's inertial measurement unit (IMU) 404. As is well known, many user devices, including smartphones and tablet computers, include an IMU for measuring parameters such as force, angular rate, and orientation of the smartphone. Other components that may be found in an IMU include an accelerometer and a magnetometer.

[0088] The gyroscope 402 may generate gyroscope data representative of mechanical oscillations within the or each gyroscope, which mechanical oscillations are found to be at least partially representative of sensing of ultrasonic signals in the environment.

[0089] The controller 406 may be configured to receive first data derived from a signal received by the microphone 206, e.g., after being manipulated or processed by the preamplifier 204 and the ADC 306 and / or any other component that introduces nonlinearity. The controller 406 may also be configured to receive second data, i.e., gyroscope data, representing mechanical oscillations within the gyroscope 402. The controller 406 may be configured to detect whether the signal received by the microphone 206 includes an ultrasonic signal based at least in part on the first data and the second data (which may correspond to a common time instance or time period). If so, the signal is considered to represent an ultrasonic attack, and appropriate mitigation action(s) may be performed by the processor 308.

[0090] For example, controller 406 may output a control signal indicating attack / no attack detection to processor 308, which takes appropriate mitigation action(s) in response to attack detection.

[0091] In some example embodiments, the mitigation action may include a cancellation action to subtract at least some, if not all, data corresponding to the ultrasonic attack from the first data.

[0092] In some example embodiments, the second data may be provided from the gyroscope 402 to the processor 308 .

[0093] Alternatively, third data derived from the second data may be provided to the processor 308. For example, the third data may include a weighted sum of a subset or separate subsets of the second data, which represents the mechanical oscillation of the gyroscope on its respective axis. For this purpose, an optional pre-processor module (not shown) may be provided as part of the apparatus 400.

[0094] In some embodiments, controller 406 may include part of processor 308 or may be a separate entity including one or more controllers and / or one or more processors, or their equivalents.

[0095] The gyroscope 402 may be a MEMS gyroscope.

[0096] Figure 5 is a schematic diagram of a gyroscope 402, which includes a resonant mass 502, a sensing frame 504, a plurality of drive springs 506 between the resonant mass and the sensing frame, and a plurality of sensing springs 508 connected to external wires 509. Connected to the sensing frame 504 are a plurality of Coriolis sensing fingers 510 inserted between terminals 512. MEM gyroscopes can have different forms, and Figure 5 The schematic diagram is an example.

[0097] In general, a gyroscope is a device that can measure the azimuth or angular rate of rotation of an object. The principle relies on the way the resonant mass 502 is suspended so that the Coriolis force can be sensed when the resonant mass rotates relative to inertial space.

[0098] Therefore, a gyroscope such as Figure 5 The gyroscope 402 shown in FIG. 4 typically includes movable components in at least two orthogonal directions to generate Coriolis forces.

[0099] In the drive direction (in this case the y-axis), the drive spring 506 uses a sinusoidal voltage to cause the resonant mass 502 to oscillate at its natural frequency. The Coriolis sensing finger 510 moves due to the lateral Coriolis force / motion. In the sensing direction (in this case the x-axis), acceleration causes a change in capacitance, which can be sensed at terminal 512. This acceleration is analogous to the angular rate of rotation of the resonant mass 502. The structure of the gyroscope is typically a single degree of freedom system with a high damping ratio ξ. Damping can be ignored at low frequencies, and the gyroscope can produce linear results. As the frequency increases, damping becomes dominant and oscillations occur with the characteristics of forced vibrations. Since sound waves are pressure waves, gyroscopes have been found to respond to high frequency (ultrasonic) signals but not to audio signals; as a result, gyroscopes (such as Figure 5 A gyroscope 402 shown in FIG. 4 may be used to detect ultrasonic signals using gyroscope data.

[0100] Reference numeral 520 indicates three axes (x, y, z) for which gyroscope data may be generated in this case. The y-axis is the drive direction, the x-axis is the sense direction, and the z-axis is the rate input.

[0101] Figure 6600 is a diagram showing three waveforms 602, 603, 604 (indicated as gx, gz, gy) representing gyroscope or second data described herein, which may be caused by the exposure of the gyroscope 402 to ultrasonic signals in the environment for the above three axes (x, y, z). It is assumed that the body (e.g., device 400) to which the gyroscope 402 (or IMU 404) is mounted is static. One or more of the three waveforms 602, 603, 604 may be sampled over time to generate the gyroscope or second data.

[0102] As will be seen, the three waveforms 602, 603, 604 represent mechanical vibrations sensed on all three axes (x, y, z), and most of the vibrations or spikes are present in the third waveform 604 associated with the y-axis or drive direction. Which of the three waveforms 602, 603, 604 indicates the most oscillations depends on many factors, including the orientation of the gyroscope 402 on the device 400 and / or the relative direction of the ultrasonic signal reaching the gyroscope. For this reason, the gyroscope data or second data provided to the controller 406 may include only one, two, or all three sampled waveforms 602, 603, 604, where two or three waveforms (if used) may be averaged or weighted and used.

[0103] For example, in some embodiments, each of the three waveforms 602, 603, 604 may be monitored to determine which waveform has the most oscillations within a predetermined time frame; that waveform may be sampled and provided to the controller 406 as gyroscope or second data.

[0104] Alternatively, a weighted sum of the three waveforms 602, 603, 604 may be provided to the controller as the gyroscope or second data.

[0105] Back to Figure 4 , the controller 406 may receive as input first data at the output of the ADC 306 (derived from the signal received by the microphone 206, if any) and second data from the gyroscope 402. The controller 406 is configured to determine whether the signal received by the microphone includes an ultrasonic signal (attack) based on the first and second data for the same time instance or time period. Various ways of how this may be performed will be described later.

[0106] If the controller 406 determines that the signal received by the microphone 206 includes an ultrasonic signal (attack), an “attack” or “enable” signal may be sent to the processor 308 .

[0107] Processor 308 may responsively perform one or more mitigation actions, such as cancellation actions, to subtract at least some, if not all, data corresponding to the ultrasonic attack from the first data.

[0108] Figure 7 is a flowchart indicating processing operations that may be performed by an apparatus according to one or more example embodiments.

[0109] The processing operations may be performed by hardware, software, firmware, or a combination thereof. The order shown does not necessarily indicate the order of processing. The apparatus may include or at least include one or more components for performing the operations, wherein the components may include one or more processors or controllers that, when executing computer-readable instructions, may cause the operations to be performed.

[0110] A first operation 701 may include providing first data derived from a signal received by a microphone of a user device.

[0111] The second operation 702 may include providing second data representing mechanical vibrations within a gyroscope of the user device.

[0112] The third operation 703 may include detecting, based on the first data and the second data, that a signal received by the microphone includes an ultrasonic wave signal.

[0113] The fourth operation 704 may include processing the third data to generate fourth data representing an estimate of one or more audible sounds in the first data, wherein the third data is or is derived from the second data.

[0114] The fifth operation 705 may include subtracting the fourth data from the first data to provide fifth data.

[0115] The term "providing" may mean receiving, generating and / or storing.

[0116] The third operation 703 may use the first data and the second data corresponding to the same time instance or time period.

[0117] The fourth operation 704 and the fifth operation 705 may be performed in response to the third step of detecting that the signal received by the microphone includes the ultrasonic wave signal.

[0118] The second data of the second operation 702 may include one second data set representing mechanical vibration for a single axis, or may include a weighted sum of two or more second data sets representing multiple vibrations for two or more axes of the gyroscope.

[0119] Thus, the fifth data may represent a noise-canceled version of the first data, wherein at least some audible sounds (eg speech) are removed prior to output via, eg, a speaker and / or prior to input to another processing function, which may include a speech processing function.

[0120] In this case, the voice processing function is therefore less likely to interpret the fifth data as comprising unwanted voice (eg, a command), and the consequences of an ultrasonic attack can therefore be avoided.

[0121] Figure 8 is a schematic diagram of another apparatus 800 according to some example embodiments.

[0122] The device 800 is similar to Figure 4 As shown in and referenced Figure 4 An apparatus 400 is described.

[0123] The device 800 may include a microphone 802, a preamplifier 804, an ADC 806, a processor 808, a controller 810, and a gyroscope 812. The gyroscope 812 may be provided as part of an IMU 814 of the device 800.

[0124] for Figure 4 In the example of , microphone 802 can pick up a signal through preamplifier 804 and ADC 806. First data can be provided at the output of ADC 807 on signal line 822, which can represent unwanted sounds, such as malicious voices generated by downward displacement as described above. Gyroscope 812 can generate gyroscope data or second data representing mechanical vibrations within the gyroscope or each gyroscope. Second data is provided on signal line 824 and can represent mechanical vibrations for only one axis (e.g., the y-axis), or can include two or more (e.g., three) gyroscope data sets representing mechanical vibrations for two or more axes (e.g., x, z, y axes).

[0125] The first data and the second data may be input to the controller 810 , which may be configured to detect whether a signal received by the microphone 806 includes an ultrasonic signal based at least in part on the first data and the second data (eg, for the same time instance or time window).

[0126] If so, the controller 810 may output a control signal indicating attack / no attack detection, for example, to the processor 808 .

[0127] According to some example embodiments, in response to the control signal indicating attack detection, the processor 808 may be configured to perform a cancellation action to subtract at least some, if not all, data corresponding to the ultrasonic attack from the first data.

[0128] According to some example embodiments, the processor 808 may include a filter 818 and a subtraction element 820, either of which may be implemented in hardware, software, firmware, or a combination thereof. Equivalent circuitry may be provided as an alternative to perform the same or substantially the same functions.

[0129] The processor 808 may receive third data labeled g via signal line 824, which may include the second data, or alternatively, the apparatus 800 may further include a pre-processor 804 located between the gyroscope 812 and the processor for providing the third data. If the second data includes two or more gyroscope data sets, such as gx, gy, gz representing mechanical vibrations for three axes, the pre-processor 804 may be configured to provide the third data for input to the processor 808 using signal line 826. For example, the pre-processor 804 may select one of the three second data sets as the third data. The selected set may include a set representing the maximum value of the mechanical vibration, such as Figure 6 Alternatively, the pre-processor 804 may generate a weighted sum of two or more second data sets to provide third data, for example, such as:

[0130] g=a.gx+b.gy+c.gz(1)

[0131] Where a, b and c are weights.

[0132] In this sense, the third data is derived from the second data.

[0133] The filter 818 may be configured to process the third data to generate fourth data representing an estimate of one or more audible sounds in the first data. For example, the one or more audible sounds may represent speech, such as one or more malicious words in the ultrasonic signal, and thus may be represented in the first data.

[0134] The filter 818 may include an adaptive filter, such as a finite impulse response (FIR) filter, configured to generate fourth data, such as:

[0135]

[0136] in is the fourth data, indicating one or more audible sounds, g n-i refers to a current instance or window of third data, such as a sample of geographic data, including the current sample and a predetermined set of previous samples, a i refers to the filter coefficients, and N refers to the filter length. The length of the current instance or window of the third data may include the same number of samples as the adaptive filter length N.

[0137] Fourth Data may be provided to the subtraction element 820 .

[0138] The subtraction element 820 may be configured to subtract the fourth data from the first data received on the signal line 822. For example, for the same time instance or time window, to generate the fifth data. This can be expressed as:

[0139]

[0140] Wherein err is fifth data, which may be referred to as an error signal or error data, y refers to first data, such as speech data derived from a microphone signal, and Refers to the fourth data.

[0141] Thus, the fifth data represents a modified version of the first data with at least some of the first data subtracted or eliminated. The fifth data may be provided to the speech processing function 816 on signal line 828.

[0142] The speech processing functionality 816 can be configured to receive the fifth data and detect and process one or more utterances, such as digital assistant commands, in a conventional manner.

[0143] Therefore, the speech processing functionality 816 is less likely to recognize and therefore process malicious speech in the first data as it receives and processes the fifth data.

[0144] In some example embodiments, filter 818 may include an adaptive filter, where the filter coefficients a i The current value of can be modified based at least in part on the current fifth data set to provide updated filter coefficients.

[0145] If from Figure 7 As can be seen, the current fifth data set can be fed back to the filter 818 via the signal line 812 to be used to modify the filter coefficient a i The current value of .

[0146] For example, the filter coefficient a i The current value of may be modified based at least in part on multiplying the fifth data with the third data, representing a current window of the third data, and adding to the current value of the filter coefficient.

[0147] For example, the current values ​​of the filter coefficients may also be modified based on an adaptation rate parameter μ, which may be a user-determined default value, typically on the order of about 0.01 to 0.1.

[0148] For example, the processor 808 may be configured to modify the current value of the filter coefficient to:

[0149]

[0150] in is the updated value of the filter coefficient, is the current value of the filter coefficient, μ is the adaptive rate parameter, err is the fifth data, and is the current window of the third data.

[0151] and The values ​​of can be defined accordingly as vectors (5) and (6):

[0152]

[0153] Thus, a first instance of the third data (comprising a current sample of the third data and one or more previous samples of the third data) is associated with a first time window for use in generating the first set of updated filter coefficients.

[0154] Thus, one or more consecutive instances of third data may be associated with one or more consecutive time windows (e.g., where the windows are iteratively shifted along one or a predetermined number of samples) for use in generating one or more consecutive updated sets of filter coefficients.

[0155] In this manner, filter 818 may iteratively update its filter coefficients as successive instances of the third data are received such that an optimal set of filter coefficients is reached in a convergent manner.

[0156] For example, the filter coefficients may be continuously updated until a predetermined condition is reached. For example, the filter coefficients may be iteratively updated, and the corresponding mean square error values ​​may be iteratively calculated. When the mean square error value is below a predetermined threshold, the updating of the filter coefficients may, for example, be stopped.

[0157] At this point, filter 818 may be able to use the fifth data to eliminate most, if not all, of the unwanted sound in the first data.

[0158] Therefore, the speech processing function 816, upon receiving the fifth data on the signal line 828, should not identify and process one or more unwanted utterances that may be present in the first data.

[0159] Even prior to the predetermined condition, while the filter coefficients are being updated, the speech processing functionality 816 is less likely to recognize and process one or more unwanted utterances.

[0160] In some example embodiments, the subtraction element 820 may be configured to subtract the fourth data from the first data within substantially the time period T. During this time period, the ultrasonic attack is detected by the controller 810 and signaled to the processor 808. The time period T may correspond to one or more speech words.

[0161] In some example embodiments, the subtraction element 820 may be configured to subtract the fourth data from the first data only during a portion of the time period T during which the ultrasonic attack is detected by the controller 810 and signaled to the processor 808. This may explain the fact that even partial elimination of the unwanted speech may be sufficient to prevent it from being recognized and processed by the speech processing function 816. In some example embodiments, the subtraction element 820 may be configured to subtract the fourth data from the first data at multiple spaced-apart time periods within the time period T. By reducing the number of subtractions by the subtraction element 820, processing resources may be used efficiently.

[0162] Thus, example embodiments provide apparatus, methods, and computer programs for mitigating ultrasonic attacks. Example embodiments avoid causing disruption to the user of the device that might otherwise involve, for example, disabling a microphone or some other part of the device in response to an ultrasonic attack. The user may not even be aware that a mitigation operation has been performed.

[0163] Detection of ultrasonic signals

[0164] Various methods of detecting ultrasonic signals will now be described by way of non-limiting examples, such as by Figure 4 and Figure 8 Controllers 406, 810 shown in .

[0165] refer to Fig. 9 , if the first data (indicated by the first waveform 902) and the second gyroscope data (indicated by the second waveform 904) represent non-zero values ​​within one or more corresponding time instances or time periods, the controller 406, 810 can, for example, detect that the signal received by the microphone 206, 802 is an ultrasonic signal.

[0166] Here, the first and second waveforms 902, 904 for the same time period are slightly different, but clearly represent non-zero values ​​for most of the time period. This indicates that, through the first data, some signal has been received by the microphone 206, 802, and through the second gyroscope data, mechanical oscillations within the gyroscope 402, 812 have been detected in the same time period. This may indicate the presence of an ultrasonic signal in the environment. On this basis, it can be determined that the signal received by the microphone 206, 802 is an ultrasonic signal and represents a potential form of ultrasonic attack.

[0167] This relatively simple detection example may be appropriate where the device 400, 800 is static and generally not in motion. Figure 1 The digital assistant 100 in is typically static and sits on a table or work surface.

[0168] For situations where the device 400, 800 may be in motion, such as a smartphone, tablet computer, hearing aid, and headset, such as Figure 2 The headset set 200 shown in FIG. 2 needs to distinguish gyroscope data that may also indicate the motion of the device when carried by the user. However, the same operation can be applied to static devices, such as Figure 1 The digital assistant shown in .

[0169] In this case, it may be appropriate to use one or more different detection methods to better distinguish between gyroscope data representing ultrasonic signals and gyroscope data generated by normal movement of the device when carried by the user. Using two or more different detection methods may give more accurate results.

[0170] refer to Fig.10 , the controller 406, 810 may similarly receive first data (represented by waveform 1002) and second gyroscope data (represented by waveform 1004), and may identify similarities between the first data and the second data.

[0171] For example, the controller 406, 810 may include an amplitude envelope correlation (AEC) module 1006. As the name implies, the AEC module 1006 may perform a correlation (e.g., a cross-correlation) using the first data and the second data, effectively determining a similarity between the time domain envelopes of the waveforms 1002, 1004 represented by the first data and the second data to generate a first parameter. The first parameter may be an indication of similarity, such as in the form of a ratio or a percentage. For example, a value closer to 1 (ratio) or 100% (percentage) may indicate a high similarity, while a value closer to zero (ratio) or 0% may indicate a low similarity.

[0172] For example, the controller 406, 810 may additionally or alternatively include a spectrum analysis module 1008. The spectrum analysis module 1008 may perform spectrum analysis on the frequency domain representations of the waveforms 1002, 1004 represented by the first data and the second data. The spectrum analysis module 1008 may generate a second parameter indicating similarity between the frequency domain representations, for example, in terms of energy level of the baseband and / or one or more harmonics. Similar to the AEC module 1006, the second parameter may be an indication of similarity, for example, in terms of a ratio or percentage. For example, a value closer to 1 (ratio) or 100% (percentage) may indicate a high similarity, while a value closer to zero (ratio) or 0% may indicate a low similarity.

[0173] One or both of the AEC module 1006 and the spectrum analysis module 1008 may be employed.

[0174] The first parameter and / or the second parameter generated accordingly by the AEC module 1006 and / or the spectrum analysis module 1008 may be provided to a decision logic module 1010 which is configured to determine, based on the value(s) of the parameter(s), whether the signal received by the microphone 206, 802 is an ultrasonic signal and therefore may represent a form of ultrasonic attack.

[0175] For example, the decision logic module 1010 may implement one or more rules. The one or more rules may be based on predetermined thresholds for the first parameter and / or the second parameter.

[0176] For example, the decision logic module 1010 can determine that the signal received by the microphone 206, 802 is an ultrasonic signal if the first parameter and / or the second parameter is above 0.65 or 65%. For example, the decision logic module 1010 can determine that in order to make such a determination, both the first parameter and the second parameter need to be above 0.65 or 65%. In other examples, the ratios or percentages for the first and second parameters can be different, such as 0.75 or 75% for the first parameter and 0.6 or 60% for the second parameter. These values ​​are examples only and can be changed as needed.

[0177] For example, the decision logic module 1010 may alternatively include one or more machine learning (ML) models. Any suitable form of ML model may be used, such as a generative adversarial network (GAN) or the like. Known deep learning methods for training and subsequent reasoning may be used. Embodiments are not limited to any particular form of model or neural network.

[0178] One or more ML models may be trained using training data that includes a predetermined set of first parameters and / or second parameters that are known to be generated in response to an ultrasonic signal emitted to the device 400, 800 or one or more different test devices. During the inference phase, the one or more ML models may receive the first parameters and / or second parameters as input from the corresponding AEC module 1006 and / or spectrum analysis module 1008. Based on how the one or more ML models have been trained, the output of the one or more ML models may be a detection classification (ultrasonic signal / attack or no attack).

[0179] In another example embodiment, consider Figure 4An adapted version of the apparatus 400 in which a second microphone is provided and which is connected to the processor 308 in the same manner, ie via one or more non-linear components.

[0180] The apparatus 400 may for example include Figure 2 The left hand earphone 202 is shown in FIG.

[0181] In addition to one or more external microphones 206, an internal microphone may also be provided within the channel 209 housing the speaker 208. This may be useful for enhanced ANC system processing and / or transparent mode operation by the processor 308.

[0182] In this case, another form of analysis may also involve comparing the respective energy levels of the signals from the external and internal microphones over the same time period. Due to the passive attenuation by the body of the left-hand earphone 202, the signal derived from the audio signal received at the internal microphone will inevitably have less energy than that at the external microphone.

[0183] This attenuation or ratio of corresponding energy levels is usually known or measurable. The 20dB figure is a real-life example for audio signals.

[0184] In the case of external and internal microphones receiving ultrasonic signals, the ratio of attenuation or corresponding energy levels will be much higher because high frequency signals are attenuated to a higher degree by objects (such as the subject of the left hand earphone 202).

[0185] Thus, a ratio of attenuation or corresponding energy levels above a threshold (alone or in combination with one or more of the other methods described above) may be used to determine whether a signal received by at least the external microphone 206 includes an ultrasonic signal.

[0186] refer to Fig.11 In another example embodiment, the controller 406, 810 may provide first data (represented by waveform 1102) derived from a signal received by the external microphone 206. The controller 406, 810 may also receive second gyroscope data (represented by waveform 1104) from the gyroscope 402, 812 during the same time period. The controller 406, 810 may also receive third data (represented by decaying waveform 1103) derived from the same signal received by the internal microphone during the same time period.

[0187] The first likelihood module 1102 may receive the first and second gyroscope data. The first likelihood module 1102 may include one or both of the ACL module 1006 and the spectrum analysis module 1008 to generate the first parameter and / or the second parameter, as described above with reference to Fig.10Said, for example, if both the first parameter and the second parameter are used, is a ratio or a percentage or an average of the ratios or percentages.

[0188] The first likelihood module 1102 may output a first likelihood value 1103 to the decision logic module 1108 based on the first parameter and / or the second parameter.

[0189] Optionally, a second likelihood module 1104 may be provided. The second likelihood module 1104 may receive the second gyroscope data and the third data. The second likelihood module 1104 may be configured the same as the first likelihood module 1102, but obviously operates on the third data representing an attenuated version of the first data.

[0190] The second likelihood module 1104 may output a second likelihood value 1105 to the decision logic module 1108 based on the first parameter and / or the second parameter.

[0191] The energy ratio module 1106 may receive the first data and the third data and may determine a third parameter 1107 indicative of an energy ratio between waveforms represented by the first data and the third data. This may be performed by comparing corresponding time instances within a time period.

[0192] The energy ratio module 1106 may output the third parameter 1107 to the decision logic module 1108 .

[0193] The decision logic module 1108 may implement one or more rules. The one or more rules may be based on predetermined thresholds for the first likelihood value 1103 , the second likelihood value 1105 (if used), and the third parameter 1107 .

[0194] For example, the decision logic module 1108 can determine that if the first likelihood value 1103 is above 0.65 or 65%, and the third parameter 1107 is a ratio above a threshold ratio (e.g., equivalent to 30 dB), then the signal received by the microphone 206 is an ultrasonic signal. For example, the decision logic module 1108 can determine that in addition to the third parameter 1107 condition, both the first likelihood value and the second likelihood value need to be above 0.65 or 65% to make such a determination. These values ​​are examples and can be changed as needed.

[0195] For example, the decision logic module 1108 may alternatively include one or more machine learning (ML) models. Any suitable form of ML model may be used, such as a generative adversarial network (GAN) or the like. Known deep learning methods for training and subsequent reasoning may be used. Embodiments are not limited to any particular form of model or neural network.

[0196] One or more ML models may be trained using training data including a predetermined set of first likelihood values ​​and / or second likelihood values ​​and a third parameter known to be generated in response to an ultrasonic signal emitted to the device 400, 800 or one or more different test devices. During the inference phase, the one or more ML models may receive the first likelihood value and / or the second likelihood value and the third parameter as input from the corresponding first likelihood module 1102, second likelihood module 1104, and energy ratio module 1106. Based on how the one or more ML models have been trained, the output of the one or more ML models may be a detection classification (ultrasonic signal / attack or no attack).

[0197] As described above, the apparatus 400, 800 may include a device configured to perform a reference Figure 7 Alternatively, operations 701-705 may be performed by a device other than the user device, the device including at least microphones 206, 802 and gyroscopes 402, 812. For example, operations 701-705 may be performed by a device other than the user device. Figure 1 The digital assistant 100 or hearing aid or Figure 2 The pair of earphones 200 shown are implemented separately from the smartphone.

[0198] When the apparatus 400, 800 includes left-hand and right-hand devices, each device includes one or more microphones and a gyroscope (such as Figure 2 , assuming that the right-hand earphone 204 includes the same functionality as the left-hand earphone 202), each of the left-hand and right-hand devices may be able to operate or be controlled independently. For example, if an ultrasonic attack is detected by two devices having the same characteristics, the same or similar mitigation operations described above may be used. On the other hand, if the ultrasonic attack comes from more angles on one side than on the other side, different mitigation operations may be used for different devices. Detecting this situation may use known microphone beamforming methods or the like (e.g., correlation and / or delay calculations). If the left-hand device detects an ultrasonic attack and the right-hand device does not, then mitigation operations may be used for the left-hand device and not necessarily (or in a reduced amount) for the right-hand device.

[0199] Thus, the user device may include a left-hand entity and a right-hand entity, each entity including one or more microphones, speakers, and gyroscopes, wherein first data and second data are provided for each of the left-hand entity and the right-hand entity, wherein the detection component is configured to detect that a signal received by a first one of the entities includes an ultrasonic signal, while a signal received by another second entity does not include an ultrasonic signal, or a signal received by the first one of the entities includes a greater amount of ultrasonic signals than the other second entity. The controller 406, 810 may be configured to control the first entity differently from the second entity to mitigate one or more events.

[0200] Example Device

[0201] Fig.12 A schematic diagram of an apparatus according to an embodiment is shown, which may include an apparatus as described herein. The apparatus may be configured to perform operations as described herein, such as operations described with reference to any previous process. The apparatus includes at least one processor 1200 and at least one memory 1210 directly or closely connected to the processor. The memory 1210 includes at least one random access memory (RAM) 1210a and at least one read-only memory (ROM) 1210b. Computer program code (software) 1220 is stored in ROM1210b. The apparatus may be connected to a transmitter (TX) and a receiver (RX). The apparatus may optionally be connected to a user interface (UI) for instructing the apparatus and / or for outputting data. At least one processor 1200, at least one memory 1210, and computer program code 1220 are arranged to cause the apparatus to at least perform at least one method according to any of the aforementioned processes by executing the computer program code.

[0202] Fig.13 A non-transient medium 1300 according to some embodiments is shown. The non-transient medium 1300 is a computer readable storage medium. It can be, for example, a CD, a DVD, a USB stick, a Blu-ray disc, etc. The non-transient medium 1300 stores computer program code that causes an apparatus to perform a method of any of the aforementioned processes by means of the apparatus executing the computer program code.

[0203] Any of the mentioned means and / or other features of the specifically mentioned means may be provided by means which are arranged so that they become configured to perform the desired operation only when enabled (e.g. turned on, etc.). In this case, they may not necessarily load appropriate software into active memory when not enabled (e.g. off state), but only load appropriate software when enabled (e.g. turned on state). The means may include hardware circuitry and / or firmware. The means may include software loaded onto a memory. Such software / computer program may be recorded on the same memory / processor / functional unit and / or one or more memories / processors / functional units.

[0204] In some examples, the specifically mentioned apparatus may be pre-programmed with appropriate software to perform the desired operations, and where the appropriate software may be enabled by a user downloading a "key", e.g., to unlock / enable the software and its associated functionality. Advantages associated with such examples may include reduced requirements for downloading data when the device requires additional functionality, which is useful in examples where the device is deemed to have sufficient capacity to store such pre-programmed software to implement functionality that may not be enabled by the user.

[0205] Any device / circuitry / component / processor mentioned may have other functions in addition to the functions described, and these functions may be performed by the same device / circuitry / component / processor. One or more disclosed aspects may cover the electronic distribution of the associated computer program and the computer program (which may be source / transmission encoded) recorded on an appropriate carrier (e.g., memory, signal).

[0206] Any "computer" described herein may include a collection of one or more separate processors / processing elements that may or may not be located on the same circuit board, or located in the same area / location of a circuit board, or even on the same device. In some examples, one or more of any of the processors mentioned may be distributed across multiple devices. The same or different processors / processing elements may perform one or more of the functions described herein.

[0207] The term "signal" may refer to one or more signals that are emitted as a series of electrical / optical signals emitted and / or received. The series of signals may include one, two, three, four or more individual signal components or different signals to constitute the signaling. Some or all of these individual signals may be emitted / received simultaneously, sequentially and / or in a manner that they overlap in time via wireless or wired communication.

[0208] With reference to any discussion of any mention of computers and / or processors and memory (including, for example, ROM, CD-ROM, etc.), these may include computer processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and / or other hardware components that have been programmed in a manner to perform the functions of the present invention.

[0209] The applicant hereby discloses separately each individual feature described herein and any combination of two or more such features, as long as these features or combinations can be implemented based on the entirety of this specification and according to the general knowledge of those skilled in the art, regardless of whether these features or combinations of features solve any problems disclosed herein, and do not limit the scope of the claims. The applicant indicates that the disclosed aspects / examples may include any such individual features or combinations of features. In view of the above description, it will be apparent to those skilled in the art that various modifications may be made within the scope of the present disclosure.

[0210] Although the basic novel features applied to the examples thereof have been shown and described and pointed out, it will be understood that various omissions, substitutions and changes in the form and details of the apparatus and methods may be made by those skilled in the art without departing from the scope of the present disclosure. For example, expressly, all combinations of those elements and / or method steps that perform substantially the same function in substantially the same manner to achieve the same results are within the scope of the present disclosure. In addition, it will be recognized that the structures and / or elements and / or method steps shown and / or described in connection with any disclosed form or example may be incorporated into any other disclosed or described or suggested form or example as a general matter of design choice. In addition, means-plus-function clauses in the claims are intended to cover structures described herein as performing the functions described, not only structural equivalents, but also equivalent structures.

Claims

1. A device for communication, the device comprising: means for providing first data derived from a signal received by a microphone of a user device; means for providing second data representing mechanical vibrations within a gyroscope of the user device; means for detecting, based on the first data and the second data, that the signal received by the microphone comprises an ultrasonic signal; means for processing third data to generate fourth data representing an estimate of one or more audible sounds in the first data, wherein the third data is or is derived from the second data; as well as Means for subtracting the fourth data from the first data to provide fifth data. 2 . The apparatus of claim 1 , wherein the processing component comprises a filter configured to receive the third data and generate the fourth data based at least in part on filter coefficients of the filter.

3. The apparatus of claim 2, wherein the filter is an adaptive filter in which current values ​​of the filter coefficients are modified based at least in part on the fifth data to provide updated filter coefficients.

4. The apparatus of claim 3, wherein the current value of the filter coefficient is modified based at least in part on: multiplying the fifth data with the third data and adding to the current filter coefficient.

5. The apparatus of claim 4, wherein the current value of the filter coefficient is further modified based on an adaptation rate parameter.

6. The device according to any one of claims 3 to 5, wherein: a first instance of the third data being associated with a first time window for use in generating the first set of updated filter coefficients, one or more consecutive instances of the third data are associated with consecutive time windows for use in generating one or more consecutive sets of updated filter coefficients, Wherein the apparatus is configured to iteratively modify the filter coefficients using the one or more consecutive instances of the third data until a predetermined condition is met. The apparatus according to claim 6 , wherein the predetermined condition is met when a predetermined mean square error value is lower than a predetermined threshold value.

8. An apparatus according to any preceding claim, wherein the second data represents two or more gyroscope data sets representing mechanical vibrations for respective axes of the gyroscope, and wherein the apparatus further comprises: Means for preprocessing said second data to provide said third data.

9. The apparatus of claim 8, wherein the pre-processing component is configured to select one of the two or more gyroscope data sets to provide the third data based on the selected set representing a maximum value of mechanical vibration.

10. The apparatus of claim 8, wherein the pre-processing component is configured to provide the third data as a weighted sum of the two or more gyroscope data sets.

11. An apparatus according to any preceding claim, wherein the detection component is configured to detect that the signal received by the microphone includes an ultrasonic signal for a time period T, and wherein the subtraction component is configured to subtract the fourth data from the first data only for a portion of the time period T. 12 . The apparatus according to claim 11 , wherein the subtracting component is configured to: subtract the fourth data from the first data for a plurality of spaced-apart sub-periods within the time period T.

13. An apparatus according to any preceding claim, wherein the first data is provided at the output of one or more non-linear components which process the signal received by the microphone.

14. An apparatus according to any preceding claim, wherein the detection component is configured to detect that the signal received by the microphone includes an ultrasonic signal based at least in part on identifying non-zero values ​​of the first data and the second data or the third data for one or more corresponding instances or time periods.

15. An apparatus according to any preceding claim, wherein the detection component is configured to: perform amplitude envelope correlation using corresponding waveforms represented by the first data and the second data or third data to generate a first parameter indicating similarity between the corresponding waveforms, and wherein the detection is based at least in part on the first parameter.

16. An apparatus according to any preceding claim, wherein the detection component is configured to: perform a spectral analysis of frequency domain representations of corresponding waveforms represented by the first data and the second data or third data to generate a second parameter indicating similarity between the frequency domain representations, and wherein the detection is at least partially based on the second parameter.

17. The apparatus of claim 15 and claim 16, wherein the detecting is based at least in part on the first parameter and the second parameter satisfying respective predetermined conditions.

18. An apparatus according to claim 15 or claim 16, wherein the detection component includes one or more machine learning models trained using training data, the training data including a predetermined set of first parameters and second parameters, the predetermined set being known to be generated in response to an ultrasonic signal being sent to the user device, wherein the detection is based on the output of the one or more machine learning models.

19. A method of communication, comprising: providing first data derived from a signal received by a microphone of a user device; providing second data representing mechanical vibrations within a gyroscope of the user device; Based on the first data and the second data, detecting that the signal received by the microphone includes an ultrasonic signal; processing third data to generate fourth data, the fourth data representing an estimate of one or more audible sounds in the first data, wherein the third data is or is derived from the second data; as well as The fourth data is subtracted from the first data to provide fifth data.

Images