Audio device with uncertainty quantification and related methods

Abstract

The present disclosure relates to audio devices with uncertainty quantification and related methods. An audio device includes a memory, an interface, and one or more processors, wherein the one or more processors are configured to: obtain audio data; process the audio data to provide an audio output; process the audio data to provide one or more audio parameters indicative of one or more characteristics of the audio data; map the one or more audio parameters to a first latent space of a first neural network to provide a mapping parameter indicative of whether the one or more audio parameters belong to a training manifold of the first latent space; determine, based on the mapping parameter, an uncertainty parameter indicative of an uncertainty of a quality of processing; and control, based on the uncertainty parameter, the processing of the audio data to provide the audio output.

Description

Technical Field

[0001] This disclosure relates to the field of audio devices and methods of operating audio devices, and more particularly to audio devices and related methods with quantified uncertainty. Background Technology

[0002] Signal processing technologies such as deep neural networks (DNNs), as well as other types of neural networks, are rapidly becoming integrated components in electronic devices such as audio equipment because they can solve problems that were previously unsolvable using traditional methods. With the continuous advancement and increasing use of neural networks, such as in audio signal processing, the reliability of their predictions is becoming increasingly important. Unfortunately, traditional neural networks do not provide deterministic reliable estimates and can suffer from overconfidence or underconfidence, resulting in poor calibration. Therefore, it can be difficult to assess the reliability of a neural network's output because several factors involved in the training of the neural network can lead to errors in its output. Summary of the Invention

[0003] Deploying DNNs in products can be difficult due to several factors: DNN inference models lack expressiveness and transparency, making it difficult to trust their results; they cannot distinguish between in-domain and out-of-domain samples and are sensitive to domain shifts (generalization problems); they cannot provide reliable uncertainty estimates for decision-making using deep neural networks and frequently result in overconfidence predictions.

[0004] Therefore, there is a need for audio devices and methods for operating audio devices with uncertainty quantization, which can mitigate, alleviate or resolve existing drawbacks and provide improved uncertainty quantization and thus improved signal processing.

[0005] An audio device is disclosed. The audio device can be configured to act as a receiver device and / or a transmitter device. The audio device includes memory, an interface, and one or more processors. Optionally, the audio device includes one or more audio speakers and one or more microphones. The one or more processors are configured to acquire audio data. In other words, the audio device is configured to use one or more processors to acquire audio data. The one or more processors are configured to process the audio data to provide audio output. In other words, the audio device is configured to use one or more processors to process the audio data to provide audio output. The one or more processors are configured to process the audio data to provide one or more audio parameters indicating one or more characteristics of the audio data. In other words, the audio device is configured to use one or more processors to process the audio data to provide one or more audio parameters indicating one or more characteristics of the audio data. The one or more processors are configured to map the one or more audio parameters to a first latent space of a first neural network to provide mapping parameters indicating whether the one or more audio parameters belong to a training manifold of the first latent space. In other words, the audio device is configured to use one or more processors to map one or more audio parameters to a first latent space of a first neural network to provide mapping parameters indicating whether the one or more audio parameters belong to a training manifold of the first latent space. The one or more processors are configured to determine an uncertainty parameter indicating an uncertainty in processing quality based on the mapping parameters. In other words, the audio device is configured to determine the uncertainty parameter indicating an uncertainty in processing quality based on the mapping parameters and using the one or more processors. The one or more processors are configured to control the processing of the audio data based on the uncertainty parameter to provide the audio output. In other words, the audio device is configured to use the one or more processors to control the processing of audio data based on the uncertainty parameter to provide audio output.

[0006] A method for operating an audio device is disclosed. The method includes acquiring audio data. The method includes processing the audio data to provide an audio output. The method includes processing the audio data to provide one or more audio parameters indicating one or more characteristics of the audio data.

[0007] The method includes mapping one or more audio parameters to a first latent space of a first neural network to provide mapping parameters indicating whether the one or more audio parameters belong to a training manifold of the first latent space. The method includes determining uncertainty parameters based on the mapping parameters, indicating uncertainty about processing quality. The method includes controlling the processing of audio data based on the uncertainty parameters to provide audio output.

[0008] This disclosure provides improved determination of uncertainty parameters for processing quality, such as those of a signal processor (e.g., a signal processor using a DNN). The disclosed audio devices and related methods provide improved signal processing for input signals including audio, such as improved processing of audio data. In other words, this disclosure provides improved control of audio data processing based on uncertainty parameters. It is understood that this disclosure provides notified control over the signal processing of audio data. This disclosure can improve the efficiency and effectiveness of signal processing, such as signal processing using a DNN. For example, this disclosure can prevent the unintentional filtering of audio data. This disclosure can allow estimation of uncertainties in processing quality, such as the uncertainty estimated by DNN signal processing. Furthermore, it is understood that this disclosure allows quantification of uncertainties in processing quality, such as those in DNN signal processing, which in turn allows for classification of predictions and more confident decisions regarding signal processing. By estimating the uncertainty of signal processing predictions by a signal processor, such as when using a DNN, action can be taken when the uncertainty parameter indicates high uncertainty in processing quality, for example, when the neural network uncertainty is high.

[0009] This disclosure provides improved feedback on the performance of signal processing for audio data, such as using DNNs. It is understood that this disclosure provides easier tuning and integration of neural networks such as DNNs into audio devices. An advantage of this disclosure is that it can provide feedback to machine learning, ML, engineers responsible for the training and / or optimization of specific products such as audio devices. For example, this can help in designing better products, such as when uncertainty parameters are documented as part of the data analysis process. It is understood that this can also help in understanding the range of values ​​required for optimal training of NNs such as DNNs. Attached Figure Description

[0010] The above and other features and advantages of this disclosure will become apparent to those skilled in the art from the following detailed description of examples of the disclosure with reference to the accompanying drawings, wherein:

[0011] Figure 1 An exemplary audio device according to this disclosure is illustrated schematically.

[0012] Figure 2 An exemplary audio device illustrating the application of the techniques disclosed herein is shown schematically.

[0013] Figure 3A and Figure 3B A flowchart of an exemplary method according to this disclosure is shown. Detailed Implementation

[0014] In the following description, various examples and details are illustrated with reference to the accompanying drawings, where applicable. It should be noted that the drawings may be drawn to scale or not, and in all drawings, elements with similar structures or functions are indicated by similar reference numerals. It should also be noted that the drawings are intended only to facilitate the description of examples. They are not intended as an exhaustive description of this disclosure or as a limitation on the scope of this disclosure. Furthermore, the examples shown do not need to possess all the aspects or advantages shown. Aspects or advantages described in connection with a particular example are not necessarily limited to that example and may be practiced in any other example even if not so shown or so explicitly described.

[0015] For clarity, the accompanying drawings are schematic and simplified, and they show only details that aid in understanding this disclosure, while other details are omitted. Throughout the drawings, the same reference numerals are used for the same or corresponding parts.

[0016] An audio device is disclosed. The audio device can be configured to act as a receiver device and / or a transmitter device. In other words, the audio device is configured to receive input signals, such as audio data, from an audio device configured to act as a transmitter device, or vice versa. The audio device disclosed herein may include one or more interfaces, one or more audio speakers, one or more microphones (e.g., including a first microphone), one or more processors, and one or more memories. The one or more interfaces may include one or more of the following: a wireless interface, a wireless transceiver, an antenna, an antenna interface, a microphone interface, and a speaker interface.

[0017] In addition, the audio device may include one or more microphones, such as a first microphone, optionally a second microphone, optionally a third microphone, and optionally a fourth microphone. The audio device may also include one or more audio speakers, such as audio receivers, for example, speakers.

[0018] Audio devices can be considered as devices configured to acquire audio data (such as input signals, e.g., audio signals), output audio signals, and process input signals (such as audio signals). Audio devices can be considered as or include headsets, speakerphones, and / or video all-in-ones. Audio devices can be considered, for example, as conferencing audio devices, configured for use by one party (such as one or more users at a near end) to communicate with one or more other parties (such as one or more users at a far end). Audio devices configured to act as receiver devices can also be configured to act as transmitter devices when transmitting outgoing signals back to a far end. Therefore, receiver audio devices and transmitter audio devices can switch between receiver audio devices and transmitter audio devices. Audio devices can be considered as smart audio devices. Audio devices can be used for conferences and / or meetings between two or more parties located far from each other. Audio devices can be used by one or more users in the vicinity of the location of the audio device, also known as near end. Audio devices can be configured, for example, to use audio speakers and output audio devices at the receiver end based on input signals. The output of an audio device can be considered as an audio output signal, which is the output of the audio speakers located near the audio device and the user of the audio device.

[0019] An audio device can be a single audio device. An audio device can also be considered as multiple interconnected audio devices, such as a system (e.g., an audio device system). This system may include one or more users.

[0020] In one or more exemplary audio devices, the interface includes a wireless transceiver (also referred to as a radio transceiver) and an antenna for wirelessly transmitting and receiving input signals such as audio signals, wirelessly transmitting output signals, and / or wirelessly receiving wireless input signals. The audio device can be configured to wirelessly communicate with one or more electronic devices such as another audio device, a smartphone, a tablet, a computer, and / or a smartwatch. The audio device may optionally include an antenna for converting one or more wireless input audio signals into an antenna output signal. The audio device system and / or the audio device can be configured for wireless communication via a wireless communication system, such as a short-range wireless communication system, such as Wi-Fi, Bluetooth, Zigbee, IEEE 802.11, IEEE 802.15, infrared, and / or similar systems.

[0021] Audio equipment systems and / or audio devices can be configured for wireless communication via wireless communication systems such as 3GPP systems that support one or more of the following 3GPP systems: New Radio, NR, Narrowband IoT, NB-IoT and LTE-Enhanced Machine Type Communications, LTE-M, millimeter wave communications, millimeter wave communications in licensed bands, and device-to-device millimeter wave communications in licensed bands.

[0022] In one or more exemplary audio device systems and / or audio devices, the interface of the audio device includes one or more of the following: a Bluetooth interface, a Bluetooth Low Energy interface, and a magnetic induction interface. For example, the interface of the audio device may include a Bluetooth antenna and / or a magnetic interference antenna.

[0023] In one or more exemplary audio devices, the interface may include a connector for wired communication, such as through a cable. A connector can connect one or more microphones to the audio device. A connector can connect the audio device to electronic equipment, such as for a wired connection. A connector can be considered an electrical connector, such as a physical connector used to connect an audio device to another device via a wire.

[0024] The one or more interfaces may be or include wireless interfaces, such as transmitters and / or receivers, and / or wired interfaces, such as connectors for physical coupling. For example, an audio device may have an input interface configured to receive data such as microphone input signals. In one or more exemplary audio devices, the audio device can be used in all types of environments and for all forms of application, such as for headsets and / or video conferencing equipment. For example, the audio device may not have specific microphone placement requirements. In one or more exemplary audio devices, the audio device may include an external microphone.

[0025] Audio devices are configured to acquire audio data, such as using one or more processors and / or via an interface. In one or more exemplary audio devices, the audio device may be configured to acquire input signals, such as audio signals, from a transmitter device. In one or more exemplary audio devices, the audio device is configured to acquire audio data from a remote location, such as a remote end or a user. For example, a processor may be configured to acquire audio data via one or more microphones of the audio device (such as microphones associated with and / or included in the audio device). Audio data may include and / or be based on one or more audio signals acquired by the audio device. In other words, the transmitter device may be considered as an audio device at a remote location. Audio data may be considered as data that includes audio. In one or more embodiments or examples, the audio signal undergoes signal processing at the transmitter device, such as encoding, compression, and / or enhancement. Audio data may indicate audio signals generated by a user at a remote location. In other words, audio data may indicate speech, such as speech from a remote transmitter device. Audio data may be based on and / or considered as an output signal of the transmitter device, such as the output signal of the transmitter device's signal processor. Acquiring audio data may include retrieving and / or receiving audio data.

[0026] Audio devices are configured to, for example, use one or more processors to process audio data to provide audio output. Processing audio data to provide audio output may include performing one or more audio processing steps on the audio data. For example, processing audio data to provide audio output may include, for example, using a signal processor to perform noise reduction on the audio data, such as background noise reduction, to provide a denoised audio output. Another example may include processing audio data to provide audio output, which may include, for example, using a signal processor to perform filtering on the audio data to provide a filtered audio output. Furthermore, processing audio data to provide audio output may include performing compression on the audio data.

[0027] An audio device is configured to process audio data to provide one or more audio parameters indicative of one or more characteristics of the audio data. In one or more exemplary audio devices, the audio device includes an uncertainty quantization determiner. The uncertainty quantization determiner can be configured to determine one or more audio parameters. These one or more audio parameters can be considered quality parameters of the audio data, such as those indicative of the audio quality of the audio data. The one or more audio parameters can, for example, indicate one or more characteristics of the audio device, such as one or more of the following: bit rate, sampling rate, dynamic range, frequency response, distortion, noise level, stereo imaging, compression artifacts, interference, anomalous noise, and speech artifacts. These one or more audio parameters can indicate characteristics including one or more of the following: signal-to-noise ratio, confidence probability plot, quality representation, or mean opinion score. A confidence probability plot (time-frequency, TF, plot) can indicate the confidence of a denoised signal, such as the reliability of a gain-time-frequency, TF, or mapping. The mean opinion score can be considered a predicted mean opinion score, such as a predicted mean opinion score quality prediction. For example, the mean opinion score can be determined based on an intrusive method, such as by comparing the audio data with a reference signal, such as a reference audio signal. Alternatively or additionally, the average opinion score can be determined based on non-intrusive methods, such as by performing blind predictions, such as using a pre-trained neural network specifically designed for MOS score and / or alternative score estimation.

[0028] In one or more exemplary audio devices, the one or more audio parameters include direct reverberation ratio (DRR), coherence spread ratio (CDR), spatial noise coherence, room impulse response, noise / speech / interference level / direction, and an audio copy of the audio data. The direct reverberation ratio (DRR), coherence spread ratio (CDR), spatial noise coherence, room impulse response, and noise / speech / interference level / direction may be associated with the room and / or location of the transmitter.

[0029] In one or more exemplary audio devices, processing audio data to provide one or more audio parameters includes encoding the audio data to provide one or more audio parameters using a first neural network as disclosed herein. In other words, an uncertainty quantization determiner can be configured to operate according to the first neural network. The uncertainty quantization determiner can be considered as a module or network configured to extract audio parameters, such as representative parameters, from the audio data. In different precise expressions, encoding audio data to provide one or more audio parameters can include encoding an audio signal included in the audio data using the first neural network to provide one or more audio parameters. The uncertainty quantization determiner can include a signal feature extractor. In one or more exemplary audio devices, providing one or more audio parameters includes extracting one or more audio features from the audio data using a signal feature extractor. For example, the audio device can be configured to extract one or more of the following from the audio data using a signal feature extractor: bit rate, sampling rate, dynamic range, frequency response, distortion, noise level, stereo imaging, compression artifacts, interference, anomalous noise, speech artifacts, signal-to-noise ratio, confidence probability map, quality representation, and average opinion score. In one or more exemplary audio devices, providing one or more audio parameters can include encoding one or more extracted features to provide one or more audio parameters. Feature extractors can be configured to extract features using ML-based methods and / or signal processing-based methods. For example, features can be extracted by a feature extractor by performing one or more operations, such as combining spectra, on a spectrum.

[0030] An audio device is configured to, for example, use one or more processors to map one or more audio parameters to a first latent space of a first neural network to provide mapping parameters that indicate whether one or more audio parameters belong to a training manifold of the first latent space.

[0031] The first latent space can be viewed as a mathematical representation of a multidimensional space, where data such as audio data can be represented. The first latent space can be viewed as a multidimensional space where highly complex data, such as audio data, can be represented in a low dimension, while capturing the underlying structure and patterns of the data. Locations in the latent space can be defined by latent variables. A first neural network can be configured to operate based on a first machine learning model. The first neural network can be trained on training audio samples to provide a latent space representation of the training audio samples. The dimension and other properties of the latent space can be defined during the training of the first machine learning model. The first neural network may include an autoencoder. The autoencoder may include an encoder and / or a decoder. The autoencoder can be configured to generate a first latent space representation of the data by training the first machine learning model based on training audio samples. In other words, the autoencoder can be configured to compress high-dimensional audio data, such as highly complex training audio samples, into a low-dimensional space while retaining as much of the original information of the audio data as possible. In other words, the autoencoder can be configured to compress input data, such as training audio samples, with a high-dimensional feature space into a low-dimensional first latent space while retaining as much of the original information of the audio data as possible, while discarding irrelevant information. For example, a first neural network has been trained based on the loss between the input to the encoder (such as audio data, e.g., training audio data) and the output at the decoder (such as audio output, e.g., training audio output). Training audio data, such as training audio samples, can be recorded and / or simulated. For example, training audio data can be simulated by convolving noise and pure audio data together. The encoder of an autoencoder can be trained to encode data such as audio data into a corresponding latent space representation. The decoder of the autoencoder can be trained to decode the latent space representation to attempt to predict the original encoded data. The first latent space representation can be viewed as a compressed representation of the input data, e.g., as audio training data, which captures representative features of the data in a more compact form. The first latent space can be viewed as a low-complexity encoding space. The training manifold of the first latent space can be viewed as the distribution of training data points in the first latent space used to train the first neural network. The training manifold can be based on training data used in training the first neural network, such as audio training data. The training manifold can be represented as one or more intervals defining one or more regions within the latent space. For example, each region of the training manifold can include multiple training data points encoded by the encoder. Each region of the training manifold can be associated with one or more audio parameters. In other words, each region of the training manifold can be associated with different audio parameters classified by the first machine learning model. As the first machine learning model is trained, the training manifold in the latent space can become more defined, and the regions defined by the training manifold can form clusters in the latent space of the training data points encoded by the encoder.A training manifold can be represented as one or more thresholds. It can be understood that during training, a first neural network can be configured to learn to map each data point in a high-dimensional input feature space to a corresponding point in a low-dimensional latent space. In other words, the first latent space includes a training manifold of first latent space parameters. The training manifold can be viewed as a representation of the set of data points in the low-dimensional latent space that has been matched with the set of data points in the high-dimensional input space. In one or more exemplary audio devices, the training manifold includes probability distributions. In other words, the training manifold is defined by probability distributions, such as Gaussian distributions. In one or more exemplary audio devices, mapping one or more audio parameters includes mapping one or more audio parameters to probability distributions to provide mapping parameters. In one or more exemplary audio devices, the training manifold includes multiple probability distributions. In other words, the training manifold can be defined by multiple probability distributions, such as multiple Gaussian distributions; for example, each training data point can be represented in the latent space via a Gaussian distribution. In one or more exemplary audio devices, mapping one or more audio parameters includes mapping one or more audio parameters to multiple probability distributions to provide mapping parameters.

[0032] In the example, the first neural network can be trained as a variational autoencoder. A variational autoencoder can be trained using the following loss function.

[0033]

[0034] First loss item l 重构 It is an estimate of the accuracy of the autoencoder in reconstructing the input. In other words, it is the ratio of the autoencoder's input x to its output x. The differences between them. In one or more exemplary audio devices, l can be... r Reconstruction is represented as the mean square error between the input and output of the autoencoder. The second loss term... ) is the Kullback-Leibler divergence between the latent variable z and a given probability distribution, which is chosen as a Gaussian distribution in this example. It can be understood that other probability distributions can be applied similarly. β can be a weighting factor between the first and second loss terms. The encoder of the autoencoder can be configured to output the mean and / or variance. The latent variable z can then be expressed as a function of the mean and / or variance of the encoder output. The latent variable z can be sampled from the variance and mean. The latent variable z can be represented as...

[0035]

[0036] Where μ x is the average value of the encoder output as a result of sample x, V is the change in the encoder output as a result of sample x, and ∈ can be considered as such The sampling noise is a Gaussian distribution. During training, one or more vectors comprising multiple means and / or variance values ​​can be created. The training manifold can then be defined based on these multiple means and / or variance values. The means and / or variance values ​​can be considered as latent space parameters. The training manifold can be defined based on one or more training vectors. The training manifold can be defined based on one or more mean vectors and / or one or more variance vectors, each mean vector comprising multiple means and each variance vector comprising multiple variance values. For example, the training manifold can be defined based on the maximum and / or minimum values ​​of the means and / or variance values. Alternatively or additionally, the training manifold can be defined based on the percentiles of the means and / or variance values.

[0037] In one or more exemplary audio devices, one or more audio parameters may include mean, variance, and / or latent variables.

[0038] Mapping one or more audio parameters to a first latent space can be viewed as mapping one or more audio parameters to a training manifold. In other words, by using one or more processors to map one or more audio parameters to a first latent space, an audio device can determine whether one or more audio parameters belong to the training manifold to provide mapping parameters. Mapping parameters can indicate whether one or more audio parameters are within the training conditions of the first latent space, such as within the training manifold. In other words, mapping one or more audio parameters to a first latent space can be viewed as mapping one or more audio parameters to a low-dimensional latent space. For example, whether one or more characteristics of the audio data are similar to the characteristics of the training audio data used to provide the first latent space.

[0039] In one or more exemplary audio devices, mapping one or more audio parameters to a first latent space of a first neural network includes determining the distance between the one or more audio parameters and a training manifold. For example, determining the distance between the one or more audio parameters and the training manifold may include determining the distance between the one or more audio parameters and the probability distribution of the training manifold. Determining the distance between the one or more audio parameters and the training manifold may include determining the variation of the one or more audio parameters relative to the training manifold. Determining the distance between the one or more audio parameters and the training manifold may include determining the mean of the one or more audio parameters relative to the training manifold. For example, when one or more audio parameters are outside the variance and / or mean of the probability distribution of the training manifold, the one or more audio parameters may be considered to be outside the training manifold.

[0040] Determining the distance between the one or more audio parameters and the training manifold may include determining the distance between the one or more audio parameters and the set of points in the low-dimensional latent space.

[0041] The basis described in this article can be viewed as a “function” and / or “used as input.” For example, a mapping parameter can be a function of one or more audio parameters, and / or one or more audio parameters can be used as input to determine the mapping parameter.

[0042] An audio device is configured to determine an uncertainty parameter based on mapping parameters, indicating an uncertainty in the quality of the processing. In other words, the audio device is configured to determine an uncertainty parameter that indicates the uncertainty in the audio device's estimation of the quality of processed audio data to provide an audio output. The uncertainty parameter can indicate an estimate of the quality of the audio data processing result, such as an estimate of the quality of the audio output. For example, the uncertainty parameter can indicate an estimate of the quality of the result of processing audio data by a digital signal processor as disclosed herein. In a different explicit expression, the uncertainty parameter can indicate the uncertainty of the prediction of the audio data processing by a digital signal processor as disclosed herein. For example, the uncertainty parameter can indicate the uncertainty of the prediction of the audio data processing by a digital signal processor using a second neural network disclosed herein. In other words, compared to the original audio signal on which the audio data is based, the uncertainty parameter can indicate an estimate of the reconstruction quality of the audio data processing result, such as an estimate of the reconstruction quality of the audio output. For example, when the audio data is based on an audio signal similar to the training audio signal, the uncertainty parameter can indicate that the quality of the audio data processing will result in a better reconstruction quality compared to the original audio signal on which the audio data is based. On the other hand, when audio data is based on an audio signal different from the training audio signal, the uncertainty parameter can indicate that the processing quality of the audio data will result in a poorer reconstruction quality compared to the original audio signal on which the audio data is based. The uncertainty parameter can be viewed as a prediction by the audio device of the processing quality of the audio data. For a given input, such as given audio data, the uncertainty parameter can be seen as an estimate of how confident the audio device is about the processing quality when using a digital signal processor such as a second neural network. For example, for a given input, such as given audio data, the uncertainty parameter can be seen as an estimate of the confidence of the second neural network in the processing quality. However, an uncertainty parameter indicating high uncertainty in processing quality may lead to good processing quality of the audio data, and vice versa, while an uncertainty parameter indicating low uncertainty in processing quality may lead to poor processing quality of the audio data.

[0043] For example, the uncertainty parameter can be a function of the mapping parameter and / or one or more audio parameters. In other words, the mapping parameter and / or one or more audio parameters can be used as input to determine the uncertainty parameter.

[0044] Audio devices are configured to control the processing of audio data based on uncertainty parameters to provide audio output. In other words, audio devices are configured to control the processing of audio data by one or more processors based on uncertainty parameters. For example, an audio device is configured to control the processing of audio data by a digital signal processor as disclosed herein (e.g., using a second neural network as disclosed herein) based on uncertainty parameters. It is understood that the processing of audio data can vary depending on the uncertainty parameters, such as adaptation. By controlling the processing of audio data based on uncertainty parameters, this disclosure can improve the quality of the processing and, consequently, the quality of the audio output. For example, this disclosure can improve the reconstruction quality of the audio data and, consequently, the audio quality of the audio output. For example, when signal processing is performed under conditions outside of training conditions, the performance of the signal processing can be reduced. This can result in lower speech quality, difficulty in hearing the other end, and an overall deterioration in the experience. When a certain level of uncertainty in the processing quality has been identified, the audio device can control processing such as signal path to rely less on the signal processing, for example, using a neural network to predict. Instead, the audio device can be configured to use a neural network to perform more conservative signal processing and, for example, provide feedback to the user, such as reporting data to the signal processing about the conditions causing the problem.

[0045] In one or more exemplary audio devices, one or more processors include a digital signal processor (DSP) that includes a second neural network. The DSP can be configured to operate according to the second neural network.

[0046] In other words, one or more processors, such as digital signal processors, may include a second machine learning engine configured to operate according to a second machine learning model. The second machine learning engine can be viewed as a predictor engine configured to operate according to a second prediction model. The second machine learning engine can be configured to process audio data using a second neural network based on uncertainty parameters. The second machine learning model may, for example, include or utilize neural networks, artificial intelligence, deep learning, and / or machine learning, such as a second neural network. The second machine learning model can be viewed as a model for signal processing. In one or more exemplary audio devices, the second neural network may use the same or similar training manifold as the first neural network. By using the same or similar training manifold, the first neural network can be used to determine uncertainty parameters for processing by the digital signal processor using the second neural network. In one or more exemplary audio devices, the second neural network may have already been trained on the same training data as the first neural network. By using the same training data, the first neural network can be used to determine uncertainty parameters for processing by the digital signal processor using the second neural network.

[0047] In one or more exemplary audio devices, a second machine learning model includes a model layer comprising an input layer for processing audio data such as an audio input signal, one or more intermediate layers, and an output layer. In other words, the input layer, one or more intermediate layers, and / or the output layer can be considered as layers of the second machine learning model, such as layers of a second neural network. The one or more intermediate layers can be considered as hidden layers (such as hidden features). These one or more intermediate layers may include a first intermediate layer. The model mentioned herein (such as the second machine learning model) can be considered as a model and / or scheme and / or mechanism and / or method configured to process audio data based on uncertain parameters (such as based on one or more processing schemes) and / or a prior model.

[0048] In one or more exemplary audio devices, the model mentioned herein may be stored on a non-transitory storage medium (e.g., on the memory of the audio device). The model may be stored on a non-transitory storage medium of an audio device configured to execute the model. In one or more exemplary audio devices, the model may include model data and / or computer-readable instructions (e.g., based on uncertain parameters and / or processing schemes disclosed herein). The model data and / or computer-readable instructions may be used by the audio device. The model (such as model data and / or computer-readable instructions) may be used by the audio device to process audio data, such as audio input signals.

[0049] In one or more exemplary audio devices, processing audio data to provide audio output includes feeding audio data as input to a second neural network, and processing the audio data using the second neural network based on uncertainty parameters to provide a first-level output. In other words, the processing of audio data using the second neural network can be controlled based on uncertainty parameters. It is understood that the processing of audio data using the second neural network can vary depending on the uncertainty parameters, such as adaptation.

[0050] For example, a digital signal processor (DSP) such as a second neural network might not be used to perform noise reduction when the uncertainty parameter indicates an estimate of poor processing quality, as it could potentially damage the audio data. Audio devices such as DSPs using second neural networks can damage audio data by unnecessarily introducing artifacts and other processing defects during processing. Audio devices, such as DSPs using second neural networks, can also damage the input signal by removing regions of audio data with high signal-to-noise ratios and / or high mean opinion scores and attempting to reproduce those regions.

[0051] For example, in audio devices such as headphones, a digital signal processor (DSP) can use a second neural network in the signal path to provide a mask to guide an adaptive beamformer and / or filter residual noise from the audio data. The second neural network can predict a mask to update the beam direction at each frame. When uncertainty parameters indicate an estimate of poor processing quality, such as high uncertainty in the prediction, the DSP, such as the second neural network, can be controlled to not update the covariance matrix, thus preventing beamformer oscillations and maintaining speech quality. Furthermore, the second neural network can predict a mask for residual noise in the audio data. When uncertainty parameters indicate an estimate of poor processing quality, such as high uncertainty in the prediction, the DSP, such as the second neural network, can be controlled to apply only partial noise reduction to preserve the user's speech. Partial noise reduction can be viewed as reducing noise from the audio data without completely removing it, for example, to avoid distortion of the audio data and / or introducing artifacts into the processed audio data, such as the audio output. For example, it can be determined that the DSP can perform noise reduction only on certain portions of the audio data and can perform noise reduction or regular noise reduction on other portions of the audio data.

[0052] For example, a digital signal processor (DSP) may include a noise reducer and / or an echo controller, such as a deep noise reduction DNS, configured to operate according to a neural network (NN) such as a DNN. For example, the processing of audio data controlled by a DSP based on uncertainty parameters may include masking regions of the audio data's time-frequency, TF, and spectral characteristics, and allowing the entire audio data (e.g., a speech signal including noise) to pass through regions with high signal-to-noise ratio and / or high average opinion score. For example, for audio data different from the training audio data, an audio device such as a DSP may not need to perform noise reduction because it may impair the audio data by removing excessive audio signal or by introducing processing artifacts.

[0053] In one or more exemplary audio devices, the audio output is based on a first-level output. In other words, a first-level output from a digital signal processor can be used as input to provide the audio output. The audio output can be based directly on the first-level output, mixed with one or more other outputs, and / or filtered.

[0054] In one or more exemplary audio devices, the second neural network includes a deep neural network (DNN). In other words, an audio device, such as a digital signal processor, can be configured to operate according to the DNN.

[0055] In one or more exemplary audio devices, the digital signal processor includes a controller configured to determine a controller output based on uncertainty parameters. In other words, the controller can be configured to receive uncertainty parameters and determine the controller output based on the uncertainty parameters.

[0056] In one or more exemplary audio devices, controlling the processing of audio data to provide audio output includes controlling the processing of audio data based on a controller output. In other words, the controller output can be used as input to process the audio data to provide audio output.

[0057] The controller can be configured to use uncertainty parameters to determine the controller output. Determining the controller output can include determining the processing scheme for processing audio data. For example, the controller can be configured to determine whether the audio data should be processed more or less by an audio device (such as a signal processor).

[0058] In one or more exemplary audio devices, one or more processors include a secondary processor that is different from a digital signal processor (DSP). The secondary processor can be considered an alternative processor, such as a secondary signal processor that differs from a DSP using a second neural network. For example, the secondary processor can be configured to perform signal processing without using a neural network, such as using conventional signal processing as an alternative. The secondary processor can be configured as a different signal processing path in the audio device, such as a secondary signal processing path. The DSP can be considered as a primary signal processing path. In one or more exemplary audio devices, the DSP and the secondary processor are included in the same processor. In one or more exemplary audio devices, the DSP and the secondary processor are included in different processors.

[0059] In one or more exemplary audio devices, processing audio data to provide audio output includes processing the audio data using a secondary processor to provide a secondary output that differs from the primary output, wherein the audio output is based on the secondary output. In one or more exemplary audio devices, the secondary output is determined based on the uncertainty parameter. In other words, the secondary output can be used as input to process the audio data to provide the audio output.

[0060] In one or more exemplary audio devices, one or more processors include a mixer. In one or more exemplary audio devices, processing audio data to provide audio output includes mixing primary outputs and secondary outputs to provide a mixed output. In one or more exemplary audio devices, controlling the processing of audio data may include mixing primary outputs and secondary outputs to provide a mixed output. For example, the mixer may be configured to combine a primary output with, for example, a secondary output with different weights to provide a mixed output. The mixer may be configured, for example, to add a primary output to, for example, a secondary output with different weights to provide a mixed output. A controller may be configured to control how the mixer should mix the primary and secondary outputs. For example, the mixer may be configured to partially apply a masking method.

[0061] In one or more exemplary audio devices, the audio output is based on the mixed output.

[0062] In one or more exemplary audio devices, the audio output is based partly on a primary output and partly on a secondary output. For example, a first portion of the audio output is based on a primary output, while a second portion of the audio output is based on a secondary output. In other words, a digital signal processor can be configured to process a sub-portion of the audio data to provide a primary output, and / or a secondary processor can be configured to process a sub-portion of the audio data to provide a secondary output.

[0063] In one or more exemplary audio devices, the controller can be configured to determine, based on uncertainty parameters, which portions of the audio data will be processed by the digital signal processor and which portions will be processed by the secondary processor. By mixing the primary and secondary outputs, the audio device can be configured to combine traditional noise reduction with neural network methods.

[0064] In one or more exemplary audio devices, one or more processors are configured to output audio output via an interface. In other words, the audio device may be configured to output audio output via a wired and / or wireless interface, such as to a remote end, and / or via one or more speakers (such as receivers) at a near end on the audio device itself.

[0065] In one or more exemplary audio devices, the one or more processors are configured to determine whether the uncertainty parameter satisfies a first criterion. The first criterion can be considered as a criterion indicating when the uncertainty parameter indicates low uncertainty in processing quality, such as low uncertainty in estimating the processing quality of given audio data. In other words, when the first criterion is satisfied, the uncertainty parameter indicates an estimate of processing quality that results in good reconstruction quality of the audio data. The first criterion may include a first threshold. In one or more exemplary audio devices, based on the uncertainty parameter satisfying the first criterion, the one or more processors are configured to process audio data according to a first signal processing scheme to provide audio output. It is understood that the first criterion is satisfied when the uncertainty parameter is higher than or equal to the first threshold. For example, the uncertainty parameter may satisfy the first criterion when one or more audio parameters belong to a training manifold. For example, the uncertainty parameter may satisfy the first criterion when one or more audio parameters are located at a distance from the training manifold, such as within a first distance. In one or more exemplary audio devices, the uncertainty parameter may satisfy the first criterion when one or more audio parameters are within a probability distribution of the training manifold, such as within a portion of the probability distribution. Processing audio data according to the first signal processing scheme to provide audio output may include using a digital signal processor (DSP) to process the audio data, such as using a second neural network, to provide audio output. In other words, the first signal processing scheme may include instructions that instruct the audio device to process the audio data using a DSP and a second neural network, such as processing with a DSP only, for example, processing with a DNN only. The first signal processing scheme can be considered normal operation.

[0066] In one or more exemplary audio devices, one or more processors are configured to process audio data according to a second signal processing scheme to provide audio output, based on an uncertainty parameter that does not meet a first criterion and / or an uncertainty parameter that meets a second criterion. In other words, the one or more processors are configured to determine whether the uncertainty parameter meets the second criterion.

[0067] The second criterion can be viewed as a criterion indicating when the uncertainty parameter indicates moderate uncertainty in processing quality. In other words, when the second criterion is met, the uncertainty parameter indicates moderate uncertainty in processing quality, such as indicating moderate reconstruction quality of audio data. The second criterion may include a second threshold.

[0068] It is understood that the second criterion is satisfied when the uncertainty parameter is higher than or equal to the second threshold. For example, the uncertainty parameter can satisfy the second criterion when one or more audio parameters are located at a certain distance from the training manifold, such as within the second distance. In one or more exemplary audio devices, the uncertainty parameter can satisfy the second criterion when one or more audio parameters are within a certain distance from the probability distribution of the training manifold.

[0069] It is understood that the first criterion is not met when the uncertainty parameter is below a first threshold. For example, the uncertainty parameter may not meet the first criterion when one or more audio parameters are located beyond a certain distance from the training manifold, such as beyond the first distance. In one or more exemplary audio devices, the uncertainty parameter may not meet the first criterion when one or more audio parameters are beyond a certain distance from the probability distribution of the training manifold. It is understood that when the uncertainty parameter does not meet the first criterion, the audio device is configured to avoid processing audio data according to the first signal processing scheme.

[0070] Processing audio data according to a second signal processing scheme to provide audio output may include using a combination of a digital signal processor (DSP), such as a second neural network, and a secondary processor to process the audio data to provide audio output. In other words, the second signal processing scheme may include instructions that instruct the audio device to use a combination of a DSP and a secondary processor to process the audio data, such as using both a DSP (e.g., a DNN) and a secondary processor. The second signal processing scheme can be considered a conservative operation.

[0071] In one or more exemplary audio devices, based on uncertainty parameters that do not meet a first criterion and uncertainty parameters that do not meet a second criterion, one or more processors are configured to process audio data according to a third signal processing scheme to provide audio output.

[0072] It is understood that the second criterion is not met when the uncertainty parameter is below the second threshold. For example, when one or more audio parameters are located beyond a certain distance from the training manifold, such as beyond the second distance, the uncertainty parameter may not meet the second criterion. In one or more exemplary audio devices, the uncertainty parameter may not meet the second criterion when one or more audio parameters are beyond a certain distance from the probability distribution of the training manifold. It is understood that when the uncertainty parameter does not meet the second criterion, the audio device is configured to avoid processing audio data according to the second signal processing scheme. When the first and second criteria are not met, the uncertainty parameter can indicate high uncertainty in processing quality. In other words, when the first and second criteria are not met, the uncertainty parameter indicates high uncertainty in processing quality, such as representing poor reconstruction quality of the audio data.

[0073] Processing audio data according to a third signal processing scheme to provide audio output may include using a secondary processor to process the audio data to provide audio output. In other words, a third signal processing scheme may include instructions to instruct the audio device to process the audio data using only a secondary processor. A third signal processing scheme can be considered a very conservative operation.

[0074] This disclosure discloses a method of operating an audio device, such as an audio device configured to act as a receiver device. The method includes, for example, acquiring audio data via an interface and / or using one or more processors of the audio device. The method includes, for example, processing the audio data using one or more processors of the audio device to provide audio output. The method includes, for example, processing the audio data using one or more processors of the audio device to provide one or more audio parameters indicative of one or more characteristics of the audio data. The method includes, for example, mapping one or more audio parameters to a first latent space of a first neural network using one or more processors of the audio device to provide mapping parameters indicative of whether one or more audio parameters belong to a training manifold of the first latent space. The method includes, for example, determining uncertainty parameters indicative of processing quality based on the mapping parameters using one or more processors of the audio device. The method includes, for example, controlling the processing of audio data to provide audio output based on the uncertainty parameters using one or more processors of the audio device.

[0075] It should be understood that descriptions of the characteristics of audio devices can also be applied to corresponding characteristics in methods of operating audio devices as disclosed herein, and vice versa.

[0076] Figure 1 An exemplary audio device, such as audio device 10 according to this disclosure, is schematically illustrated. Audio device 10 can be considered as an audio communication device. Audio device 10 can be considered as a communication device for performing calls, such as audio and / or video calls. Audio device 10 can be considered as an audio device having uncertainty quantization processing.

[0077] Audio device 10 can be configured to act as a receiver device and / or a transmitter device. In other words, audio device 10 can be configured to receive input signals from other audio devices configured to act as transmitter devices and / or to send output signals to other audio devices. Audio device 10 includes an interface and memory (not shown). Optionally, audio device 10 includes an audio speaker 10D and one or more microphones, such as a first microphone 10E1 and a second microphone 10E2. Optionally, audio device 10 includes one or more transceivers, such as a first wireless transceiver 10F and / or a second wireless transceiver 10G. Audio device 10 can be considered as an audio device configured to acquire audio signals, output audio signals, and process audio signals. Audio device 10 can be considered as a conference audio device, for example, configured for one party (such as one or more users at a near end) to communicate with one or more other parties (such as one or more users at a far end). Audio device 10 can be considered as a smart audio device. Audio device 10 can be used for communication, conferencing, and / or meetings between two or more parties located far apart from each other. Audio device 10 can be used by one or more users in the vicinity of where audio device 10 is located, also referred to as the near end. In this example, the receiver end can be considered the near end, while the transmitter end can be considered the far end.

[0078] Audio device 10 includes one or more processors 10C. The one or more processors 10C are configured to acquire audio data. Audio device 10 may be configured to acquire, for example, a first microphone input signal 50 from a first microphone 10E1, a second microphone input signal 72 from a second microphone 10E2, and / or a transceiver input signal 74 from a first transceiver, using one or more processors 10C and / or via an input interface 10B. The first microphone input signal 50, the second microphone input signal 72, and / or the transceiver input signal 74 may be considered as input signals. In one or more exemplary audio devices, the transceiver input signal 74 may be acquired via a first transceiver interface 16, for example, as a transceiver interface input signal, and forwarded to the input interface 10B as a transceiver interface output 76. In one or more exemplary audio devices, audio device 10 is configured to acquire input signals such as the transceiver input signal 74 from a remote end, such as a remote end or a user. It is understood that the input signal includes audio. In one or more embodiments or examples, the input signal, such as transceiver input signal 74, has undergone signal processing such as encoding, compression, and / or enhancement. Transceiver input signal 74 may indicate an audio signal generated by a user at a remote end. In other words, transceiver input signal 74 may indicate speech, such as speech from a remote transmitter device. When acquired from the first microphone 10E1 and / or the second microphone 10E2, the input signal may indicate audio from a near end, such as speech. Audio data may be based on the input signal, such as based on the first microphone input signal 50, the second microphone input signal 72, and / or the transceiver input signal 74. Input interface 10B may be configured to provide input interface output 52 based on the first microphone input signal 50, the second microphone input signal 72, and / or the transceiver input signal 74. Audio data may be based on input interface output 52.

[0079] Audio device 10 is configured to, for example, use one or more processors 10C to process audio data to provide audio output. In other words, audio device 10 is configured to process a first microphone input signal 50, a second microphone input signal 72, a transceiver input signal 74, and / or an input interface output 52 to provide audio output. Audio device 10 may include an output interface 10A configured to output audio output. For example, audio device 10 may be configured to output audio output as an audio speaker input 78 to an audio speaker 10D via output interface 10A. Audio speaker 10D may be configured to output audio output based on audio speaker input 78, for example, outputting audio output at a near end. For example, audio device 10 may be configured to output audio output as a second transceiver input 80 to a second wireless transceiver 10G via output interface 10A. The second wireless transceiver 10G may be configured to output audio output based on the second transceiver input 80, for example, outputting audio output to a far end. In one or more exemplary audio devices, the second transceiver input 80 may be output via the second transceiver interface 18, for example, as a second transceiver interface input signal, and forwarded as the second transceiver output 82 to the second wireless transceiver interface 10G. It is understood that the audio output may be based on and / or include the audio speaker input 78 and / or the second transceiver input 80.

[0080] Audio device 10 is configured to process audio data, such as using one or more processors 10C, to provide one or more audio parameters indicative of one or more characteristics of the audio data. In one or more exemplary audio devices, audio device 10 includes an uncertainty quantization determiner 12. The uncertainty quantization determiner 12 can be configured to determine or provide the one or more audio parameters. The one or more audio parameters can be considered as quality parameters of the audio data, such as indicative of the audio quality of the audio data. For example, the uncertainty quantization determiner 12 can include an encoder 12A configured to determine one or more audio parameters, for example, by encoding the audio data. The uncertainty quantization determiner 12 can be configured to determine one or more audio parameters, such as using the encoder 12A based on an input interface output 52. In one or more exemplary audio devices, processing audio data to provide one or more audio parameters includes encoding the audio data using a first neural network as disclosed herein to provide one or more audio parameters. In other words, the uncertainty quantization determiner 12 can be configured to operate according to the first neural network. For example, the encoder 12A can be configured to operate according to the first neural network. The uncertainty quantization determiner 12 can be considered as a module or network configured to extract audio parameters, such as quality parameters, from the audio data. Encoding audio data to provide one or more audio parameters, using different explicit expressions, may include encoding the audio signals included in the audio data using an encoder 12A with a first neural network to provide one or more audio parameters. An uncertainty quantization determiner may include, for example, a signal feature extractor (not shown) included in the encoder 12A. In one or more exemplary audio devices, providing one or more audio parameters includes extracting one or more audio features from the audio data using the signal feature extractor.

[0081] The audio device 10 is configured to, for example, map one or more audio parameters to a first latent space of a first neural network using one or more processors 10C to provide mapping parameters indicating whether one or more audio parameters belong to a training manifold of the first latent space. In one or more exemplary audio devices, the audio device 10, such as the uncertainty quantization determiner 12, includes an uncertainty parameter determiner 12B. The encoder 12A may be configured to output the one or more audio parameters 56 to the uncertainty parameter determiner 12B. The uncertainty parameter determiner 12B may be configured to map one or more audio parameters to a first latent space of a first neural network to provide mapping parameters indicating whether one or more audio parameters belong to a training manifold of the first latent space. The audio device 10 is configured to, for example, use one or more processors 10C to determine uncertainty parameters indicating uncertainty in processing quality. In one or more exemplary audio devices, the uncertainty quantization determiner 12 is configured to, for example, use the uncertainty parameter determiner 12B to determine uncertainty parameters. The uncertainty quantization determiner 12 may be configured to, for example, use the uncertainty parameter determiner 12B to determine uncertainty parameters based on the input interface output 52, the mapping parameters, and / or one or more audio parameters. The uncertainty quantization determiner 12 can be configured to output uncertainty parameters 58, for example, to signal processor 14, using uncertainty parameter determiner 12B.

[0082] Audio device 10 is configured to control the processing of audio data based on uncertain parameters to provide audio output. In one or more exemplary audio devices, audio device 10, such as one or more processors 10C, may include signal processor 14.

[0083] In one or more exemplary audio devices, one or more processors 10C, such as signal processor 14, include a digital signal processor 14B, which includes a second neural network.

[0084] Audio device 10 is configured to, for example, use signal processor 14 to process audio data to provide audio output. In other words, audio device 10 is configured to use signal processor 14 to process first microphone input signal 50, second microphone input signal 72, transceiver input signal 74 and / or input interface output 52 to provide audio output.

[0085] In one or more exemplary audio devices, processing audio data to provide audio output includes providing audio data as input to a second neural network, such as input 52 to a digital signal processor 14B, and processing the audio data using the second neural network based on uncertainty parameters to provide a primary output 66. For example, the digital signal processor 14B may be configured to process audio data to provide a primary output 66. Optionally, the digital signal processor 14B may be configured to process audio data to provide a feedback output 64. The digital signal processor 14B may be configured to output the feedback output 64 to a controller 14A. In other words, processing audio data to provide audio output includes providing audio data, such as input interface output 52, as input to the signal processor 14, and processing the audio data using the signal processor 14, such as using the digital signal processor 14B, for example, by operating according to a second neural network. In one or more exemplary audio devices, the signal processor 14 is configured to process audio data based on uncertainty parameters to provide audio output. In one or more exemplary audio devices, the signal processor 14 is configured to control the processing of audio data based on uncertainty parameters to provide audio output. The signal processor 14 can be configured to receive uncertainty parameters from the uncertainty quantization determiner 12, such as receiving uncertainty parameters from the uncertainty parameter determiner 12B.

[0086] In one or more exemplary audio devices, the second neural network includes a deep neural network (DNN). In other words, a signal processor 14, such as a digital signal processor 14B, may include a DNN. For example, a signal processor 14, such as a digital signal processor 14B, may be configured to operate according to a DNN.

[0087] In one or more exemplary audio devices, an audio device 10, such as one or more processors 10C, includes a controller 14A. In one or more exemplary audio devices, a signal processor 14, such as a digital signal processor 14B, includes a controller 14A. The controller 14A can be configured to determine controller outputs 60, 61, and 62 based on uncertainty parameters. In other words, the controller 14A can be configured to receive uncertainty parameters and determine controller outputs 60, 61, and 62 based on the uncertainty parameters.

[0088] In one or more exemplary audio devices, controlling the processing of audio data to provide audio output includes controlling the processing of audio data based on controller outputs, such as one or more of controller outputs 60, 61, and 62. In other words, controller outputs can be used as inputs to process audio data to provide audio output. Controller 14A can be configured to determine controller outputs 60, 61, and 62 using uncertainty parameters. Determining controller outputs 60, 61, and 62 can include determining a processing scheme for processing the audio data. For example, controller 14A can be configured to determine whether the audio data will be processed more or less by audio device 10 (such as by signal processor 14). In one or more exemplary audio devices, controller 14A can be configured to control the processing of digital signal processor 14B based on uncertainty parameters, such as output 58.

[0089] In one or more exemplary audio devices, audio device 10, such as one or more processors 10C, includes a secondary processor 14D that is different from the digital signal processor 14B. The secondary processor 14D can be considered as an alternative processor, such as a secondary signal processor different from the digital signal processor 14B that uses a second neural network. For example, the secondary processor 14D can be configured to perform signal processing without using a neural network, such as alternatively using conventional signal processing.

[0090] In one or more exemplary audio devices, processing audio data to provide audio output includes processing the audio data using a secondary processor to provide a secondary output 68 that differs from the primary output, wherein the audio output is based on the secondary output. In one or more exemplary audio devices, the secondary output is determined based on the uncertainty parameter. In other words, the secondary output can be used as input to process the audio data to provide the audio output.

[0091] In one or more exemplary audio devices, controller 14A may be configured to control the processing of digital signal processor 14B based on uncertain parameters, such as output 58. For example, controller 14A may be configured to control the processing of digital signal processor 14B based on uncertain parameters by providing controller output 62 to digital signal processor 14B.

[0092] In one or more exemplary audio devices, controller 14A may be configured to control the processing of secondary processor 14D based on uncertain parameters, such as output 58. For example, controller 14A may be configured to control the processing of secondary processor 14D based on uncertain parameters by providing controller output 61 to secondary processor 14D.

[0093] In one or more exemplary audio devices, controller 14A may be configured to control the processing of mixer 14C based on uncertain parameters, such as output 58. For example, controller 14A may be configured to control the processing of mixer 14C based on uncertain parameters by providing controller output 60 to mixer 14C.

[0094] In one or more exemplary audio devices, an audio device 10, such as one or more processors 10C, includes a mixer 14C. In other words, a signal processor 14 includes a mixer 14C. In one or more exemplary audio devices, the mixer 14C is configured to process audio data to provide an audio output, including mixing a primary output 66 and a secondary output 68 to provide a mixed output 70. In one or more exemplary audio devices, controlling the processing of the audio data may include mixing the primary output 66 and the secondary output 68 to provide a mixed output 70. For example, the mixer 14C may be configured to combine the primary output 66 with, for example, a secondary output 68 with different weights to provide a mixed output 70. The mixer 14C may, for example, be configured to add the primary output 66 to, for example, a secondary output 68 with different weights to provide a mixed output 70. A controller 14A may be configured to control how the mixer 14C should mix the primary output 66 and the secondary output 68. For example, the mixer 14C may be configured to partially apply a masking method.

[0095] In one or more exemplary audio devices, the audio output is based on mixout 70. For example, audio speaker input 78 and / or second transceiver input 80 may be based on mixout 70.

[0096] It is understood that primary output 66 and / or secondary output 68 can be output directly from signal processor 14, such as via output interface 10A directly to audio speaker 10D and / or second transceiver 10G. In other words, primary output 66 and / or secondary output 68 can be output directly from signal processor 14 without going through mixer 14C.

[0097] In one or more exemplary audio devices, the audio output is based partly on a first-level output 66 and partly on a second-level output 68. For example, a first portion of the audio output is based on the first-level output 66, while a second portion of the audio output is based on the second-level output 68. In other words, a digital signal processor 14B can be configured to process a sub-portion of the audio data to provide the first-level output 66, and / or a second-level processor 14D can be configured to process a sub-portion of the audio data to provide the second-level output 68.

[0098] In one or more exemplary audio devices, controller 14A can be configured to determine, based on uncertainty parameters, which portions of the audio data will be processed by digital signal processor 14B and which portions will be processed by secondary processor 14D. By mixing primary output 66 and secondary output 68, audio device 10 can be configured to combine conventional noise reduction with neural network methods.

[0099] In one or more exemplary audio devices, the one or more processors 10C are configured to determine whether the uncertainty parameter satisfies a first criterion. In other words, the uncertainty quantization determiner and / or signal processor 14 are configured to determine whether the uncertainty parameter satisfies the first criterion.

[0100] In other words, when the first criterion is met, the uncertainty parameter indicates low uncertainty in processing quality, such as indicating good reconstruction quality of audio data. The first criterion may include a first threshold. In one or more exemplary audio devices, one or more processors 10C are configured to process audio data according to a first signal processing scheme to provide audio output, based on the uncertainty parameter that satisfies the first criterion.

[0101] Processing audio data according to the first signal processing scheme to provide audio output may include using a digital signal processor 14B, such as using a second neural network, to process the audio data to provide audio output. In other words, the first signal processing scheme may include instructions to the audio device 10, such as the signal processor 14B, to process the audio data using a second neural network (such as using only the digital signal processor 14B, for example, processing using only a DNN). The first signal processing scheme can be considered as "normal" operation.

[0102] In one or more exemplary audio devices, one or more processors 10C are configured to process audio data according to a second signal processing scheme to provide audio output, based on an uncertainty parameter that does not meet a first criterion and / or an uncertainty parameter that meets a second criterion. In other words, an uncertainty quantization determiner and / or signal processor 14 are configured to determine whether the uncertainty parameter meets the second criterion. The second criterion can be considered as a criterion indicating when the uncertainty parameter represents moderate uncertainty in processing quality. In other words, when the second criterion is met, the uncertainty parameter indicates moderate uncertainty in processing quality, such as indicating moderate reconstruction quality of the audio data. Processing audio data according to the second signal processing scheme to provide audio output may include using a combination of a digital signal processor 14B, such as using a second neural network, and a secondary processor 14D to process the audio data to provide audio output. In other words, the second signal processing scheme may include instructions instructing the audio device 10 (such as signal processor 14) to process the audio data using a combination of a second neural network and a secondary processor 14D (e.g., using digital signal processor 14B, such as using a DNN, and processing by both secondary processor 14D). The second signal processing scheme can be considered as conservative operation.

[0103] In one or more exemplary audio devices, one or more processors 10C are configured to process audio data according to a third signal processing scheme to provide audio output, based on an uncertainty parameter that does not meet a first criterion and an uncertainty parameter that does not meet a second criterion. When the first and second criters are not met, the uncertainty parameter can indicate high uncertainty in processing quality. In other words, when the first and second criters are not met, the uncertainty parameter indicates high uncertainty in processing quality, such as indicating poor reconstruction quality of the audio data. Processing audio data according to the third signal processing scheme to provide audio output may include using a secondary processor 14D to process the audio data to provide audio output. In other words, the third signal processing scheme may include instructions instructing the audio device 10 (such as signal processor 14) to process the audio data only using the secondary processor 14D. The third signal processing scheme can be considered a very conservative operation.

[0104] For example, the audio speaker input 78 and / or the second transceiver input 80 may be based solely on the first-level output 66. For example, the audio speaker input 78 and / or the second transceiver input 80 may be based solely on the second-level output 68.

[0105] Figure 2 An exemplary audio device according to this disclosure, such as audio device 10, is illustrated schematically. Figure 1 The audio device 10 uses the techniques disclosed herein. Figure 2The diagram above illustrates an exemplary training phase of audio device 10, while Figure 2 The following diagram illustrates an exemplary operational phase of the audio device 10.

[0106] During the training phase, audio device 10 is configured to acquire audio data, such as training audio data 30A. Audio device 10 may include encoder 20. Encoder 20 may be configured to process training audio data 30A to provide one or more training audio parameters indicative of one or more characteristics of the training audio data. Encoder 20 may be included in an uncertainty quantization determinist, such as... Figure 1 Uncertainty quantization determinist 12. Audio device 10 may include a first latent space 22. The first latent space 22 may include a first latent space representation. The first latent space representation may be considered as a mathematical representation of data, such as data in a low-dimensional space of the underlying structure and patterns of captured data (such as audio data), such as audio data. Encoder 20 may include a first neural network. The encoder may be configured to operate according to the first neural network. The first neural network may be configured to operate according to a first machine learning model. It is understood that the first neural network has been trained based on training audio data 30A to provide the first latent space representation. For example, the first neural network has been trained based on the loss between the input to encoder 20, such as audio data, such as training audio data 30A, and the output at decoder 24. Audio device 10 such as the first neural network may include an autoencoder, such as including encoder 20 and / or decoder 24, which is configured to generate the first latent space representation by training a first machine learning model based on training audio data 30A to provide the first latent space 22. Audio device 10 may include decoder 24, which is configured to process training audio data 30A, such as using a first latent space 22, to provide training audio output 32. In other words, audio device 10, such as using an autoencoder, may be configured to compress high-dimensional data of the training audio data, such as highly complex training audio samples, into a low-dimensional space while retaining as much of the original information of the training audio data as possible. The first latent space can be viewed as a compression of the input data, such as the training audio data, which captures representative features of the data in a more compact form. The first latent space can be viewed as a low-complexity encoding space based on the training audio data. Audio device 10 may be configured to generate a training manifold 26 in the first latent space 22 during the training phase. The first latent space is illustrated in the figure above using an XY graph.

[0107] The training manifold 26 of the first latent space 22 can be viewed as a distribution of training data points in a high-dimensional input space, used to train the first neural network to learn the low-dimensional latent space. It can be understood that during training, the first neural network can be configured to learn to map each data point in the high-dimensional input space to a corresponding point in the low-dimensional latent space. As can be seen in the diagram, the first training data point 28A is shown in the diagram. The training manifold 26 can be viewed as the set of all these data points in the high-dimensional input space used to train the first neural network. In other words, the first latent space includes the training manifold of the first latent space parameters. The training manifold can be viewed as a set of data points in the low-dimensional latent space that has been matched with the set of data points in the high-dimensional input space. In one or more exemplary audio devices, the training manifold 26 includes a probability distribution. In other words, the training manifold 26 is defined by a probability distribution, such as a Gaussian distribution. For example, each region of the training manifold 26 can include multiple training data points encoded by the encoder 20. Each region of the training manifold 26 can be associated with one or more audio parameters. In other words, each region of the training manifold 26 can be associated with different audio parameters classified by the first machine learning model. As the first machine learning model is trained, the first latent space can become more clearly defined, and the regions defined in the training manifold 26 can form clusters of training data points encoded by the encoder. During training, one or more vectors comprising multiple mean and / or variance values ​​can be created. The training manifold 26 can then be defined based on the multiple mean and / or variance values. The training manifold 26 can be defined based on one or more trained vectors. The training manifold 26 can be defined based on one or more mean vectors and / or one or more variance vectors, each of which includes multiple means and each of which includes multiple variance values. For example, the training manifold 26 can be defined based on the maximum and / or minimum values ​​of the mean and / or variance values. Alternatively or additionally, the training manifold 26 can be defined based on the percentiles of the mean and / or variance values. Figure 2 The XY chart in the upper middle section and Figure 2 In the XY chart below, the X-axis can be considered as representing the mean, and the Y-axis represents the variance.

[0108] During operation, audio device 10 is configured to acquire audio data 30B. Audio device 10 may include encoder 20. Encoder 20 may be configured to process audio data 30B to provide one or more audio parameters indicative of one or more characteristics of audio data 30B. Encoder 20 may be included in an uncertainty quantization determiner, for example... Figure 1Uncertainty quantization determiner 12. During the operation phase, audio device 10 is configured to map one or more audio parameters to a first latent space 22, such as a first neural network, to provide mapping parameters indicating whether one or more audio parameters belong to a training manifold 26 of the first latent space. The first latent space is... Figure 2 The diagram below illustrates this. Audio device 10 is configured to determine uncertainty parameters that indicate uncertainty in processing quality based on mapping parameters. In one or more exemplary audio devices, training manifold 26 includes a probability distribution. In one or more exemplary audio devices, mapping one or more audio parameters includes mapping one or more audio parameters to a probability distribution to provide mapping parameters.

[0109] Figure 2 The figure below illustrates the first audio parameter 29A in the first latent space, such as the first coded input audio. It can be seen that the audio device 10 has determined that the first audio parameter 29A belongs to the training manifold 26.

[0110] Figure 2 The figure below illustrates the second audio parameter 29B in the first latent space, such as the second coded input audio. It can be seen that the audio device 10 has determined that the second audio parameter 29B is outside the training manifold 26.

[0111] In one or more exemplary audio devices, mapping one or more audio parameters to a first latent space of a first neural network includes determining the distance between the one or more audio parameters and a training manifold 26. In other words, the audio device 10 may be configured to determine a first distance between a first audio parameter 29A and the training manifold 26 and / or to determine a second distance between a second audio parameter 29B and the training manifold 26. For example, the audio device 10 may be configured to determine whether the first audio parameter 29A is within or outside the variance of the training manifold 26 and / or to determine whether the second audio parameter 29B is within or outside the variance of the training manifold 26.

[0112] It is understood that, for the first audio parameter 29A, the uncertainty parameter can satisfy the first criterion disclosed herein, because the first audio parameter 29A belongs to the training manifold 26. Therefore, the audio device 10 can process the audio data associated with the first audio parameter 29A according to the first signal processing scheme disclosed herein.

[0113] It is understood that, for the second audio parameter 29B, the uncertainty parameter may not satisfy the first criterion disclosed herein because the second audio parameter 29B does not belong to the training manifold 26. The audio device 10 can therefore process the audio data associated with the second audio parameter 29B according to either the second or third signal processing scheme disclosed herein.

[0114] Audio device 10 can be configured to perform Figures 3A to 3B Any of the methods disclosed herein.

[0115] Processor 10C is configured to execute Figures 3A to 3B Any operation disclosed in the form (such as any one or more of S012, S104, S105, S106, S106A, S106B, S106C, S106D, S106E, S106F, S106G, S108, S108A, S110, S110A, S110B, S112, S114, S114A, S116).

[0116] The operation of the audio device 10 can be implemented in the form of an executable logic routine (e.g., a line of code, a software program, etc.) stored on a non-transitory computer-readable medium (e.g., memory) and executed by one or more processors 10C.

[0117] Furthermore, the operation of the audio device 10 can be considered as a method configured to be performed by the audio device 10. Additionally, while the described functions and operations can be implemented in software, such functions can also be implemented via dedicated hardware or firmware, or some combination of hardware, firmware, and / or software.

[0118] The memory of an audio device can be one or more of a buffer, flash memory, hard disk drive, removable media, volatile memory, non-volatile memory, random access memory (RAM), or other suitable devices. In a typical arrangement, the memory may include non-volatile memory for long-term data storage and volatile memory used as system memory for the processor 10C. The memory can exchange data with the processor 10C via a data bus. Control lines and an address bus may also be present between the memory and the processor 10C. Figure 1 (Not shown in the image). The memory is considered a non-transitory computer-readable medium.

[0119] The memory can be configured to store information such as training audio data, audio data, latent space, training manifold, mapping parameters and / or uncertainty parameters, as disclosed herein, in a portion of the memory.

[0120] Figures 3A to 3B A flowchart of an exemplary method, such as method 100, is shown.

[0121] This disclosure discloses a method 100 for operating an audio device, such as an audio device configured to act as a receiver device. Method 100 includes, for example, acquiring audio data via an interface and / or using one or more processors of the audio device, S102.

[0122] Method 100 includes, for example, using one or more processors of an audio device to process the audio data of S106 to provide audio output.

[0123] Method 100 includes, for example, using one or more processors of an audio device to process audio data in S108 to provide one or more audio parameters indicating one or more characteristics of the audio data.

[0124] The method 100 includes, for example, using one or more processors of an audio device to map one or more audio parameters S110 to a first latent space of a first neural network to provide mapping parameters that indicate whether one or more audio parameters belong to a training manifold of the first latent space.

[0125] The method 100 includes one or more processors, such as those using the audio device, determining, based on the mapping parameters, an uncertainty parameter that indicates the uncertainty of the processing quality in S112.

[0126] The method 100 includes controlling the processing of audio data S114 based on uncertain parameters such as using one or more processors of an audio device to provide audio output.

[0127] In one or more exemplary methods, processing audio data in S108 to provide one or more audio parameters includes, for example, using one or more processors of an audio device to encode the S108A audio data using a first neural network to provide one or more audio parameters.

[0128] In one or more exemplary methods, processing the audio data S106 to provide an audio output includes, for example, providing the audio data S106A as input to a second neural network using one or more processors of an audio device, and processing the audio data S106B to provide a first-level output using the second neural network based on uncertainty parameters, such as using one or more processors of an audio device, and wherein the audio output is based on the first-level output.

[0129] In one or more exemplary methods, controlling the processing of audio data S114 to provide audio output includes, for example, using one or more processors of an audio device to control the processing of audio data S114A based on controller output.

[0130] In one or more exemplary methods, processing the S106 audio data to provide an audio output includes, for example, using one or more processors of an audio device to process the S106C audio data to provide a secondary output different from the primary output using a secondary processor, wherein the audio output is based on the secondary output.

[0131] In one or more exemplary methods, processing the audio data of S106 to provide audio output includes, for example, mixing the primary output and secondary output of S106D using one or more processors of an audio device to provide a mixed output, and wherein the audio output is based on the mixed output.

[0132] In one or more exemplary methods, the training manifold includes a probability distribution, and the mapping of one or more audio parameters S110 includes, for example, using one or more processors of an audio device to map one or more audio parameters S110A to the probability distribution to provide mapping parameters.

[0133] In one or more exemplary methods, mapping one or more audio parameters of S110 includes, for example, using one or more processors of an audio device to determine the distance between one or more audio parameters of S110B and the training manifold.

[0134] In one or more exemplary methods, method 100 includes, for example, determining whether an uncertainty parameter of S104 satisfies a first criterion using one or more processors of an audio device, and, based on the uncertainty parameter that satisfies the first criterion, processing audio data of S106E according to a first signal processing scheme using one or more processors of an audio device to provide audio output.

[0135] In one or more exemplary methods, method 100 includes, for example, using one or more processors of an audio device to determine whether the uncertainty parameter of S105 satisfies a second criterion, and, based on the uncertainty parameter that does not satisfy the first criterion and / or based on the uncertainty parameter that satisfies the second criterion, processing the audio data of S106F according to a second signal processing scheme to provide audio output.

[0136] In one or more exemplary methods, method 100 includes processing S106G audio data to provide audio output according to a third signal processing scheme based on uncertainty parameters that do not meet a first criterion and uncertainty parameters that do not meet a second criterion, such as using one or more processors of an audio device.

[0137] In one or more exemplary methods, method 100 includes outputting audio output S116 via an interface and / or using one or more processors of an audio device.

[0138] Examples of audio devices, systems, and methods according to this disclosure are set forth in the following clauses:

[0139] Clause 1 An audio device includes a memory, an interface, and one or more processors, wherein the one or more processors are configured to:

[0140] Acquire audio data;

[0141] Process the audio data to provide audio output;

[0142] The audio data is processed to provide one or more audio parameters that indicate one or more characteristics of the audio data;

[0143] Mapping the one or more audio parameters to a first latent space of a first neural network to provide mapping parameters indicating whether the one or more audio parameters belong to a training manifold of the first latent space; determining uncertainty parameters indicating uncertainty in processing quality based on the mapping parameters; and

[0144] The audio data processing is controlled based on this uncertainty parameter to provide the audio output.

[0145] Clause 2 of the audio device pursuant to Clause 1, wherein processing audio data to provide one or more audio parameters includes encoding the audio data using a first neural network to provide one or more audio parameters.

[0146] Clause 3 An audio device pursuant to any of the preceding clauses, wherein the one or more processors include a digital signal processor, the digital signal processor including a second neural network, and wherein processing the audio data to provide an audio output includes providing the audio data as input to the second neural network, and processing the audio data using the second neural network based on the uncertainty parameter to provide a first-level output, and wherein the audio output is based on the first-level output.

[0147] Clause 4 is an audio device pursuant to Clause 3, wherein the second neural network includes a deep neural network.

[0148] Clause 5. An audio device pursuant to any one of Clauses 3 to 4, wherein the digital signal processor includes a controller configured to determine a controller output based on the uncertainty parameter, and wherein controlling the processing of the audio data to provide the audio output includes controlling the processing of the audio data based on the controller output.

[0149] Clause 6. An audio device pursuant to any one of Clauses 3 to 5, wherein the one or more processors include a secondary processor different from the digital signal processor, wherein processing the audio data to provide an audio output includes using the secondary processor to process the audio data to provide a secondary output different from the primary output, wherein the audio output is based on the secondary output.

[0150] Clause 7 An audio device pursuant to any one of Clauses 3 to 6, wherein the one or more processors include a mixer, and wherein processing the audio data to provide an audio output includes mixing the primary output and the secondary output to provide a mixed output, and wherein the audio output is based on the mixed output.

[0151] Clause 8 An audio device pursuant to any of the preceding clauses, wherein the training manifold includes a probability distribution, and wherein mapping the one or more audio parameters includes mapping the one or more audio parameters to the probability distribution to provide the mapping parameters.

[0152] Clause 9. An audio device pursuant to any of the preceding clauses, wherein mapping the one or more audio parameters to a first latent space of a first neural network includes determining the distance between the one or more audio parameters and the training manifold.

[0153] Clause 10 An audio device pursuant to any of the preceding clauses, wherein the one or more processors are configured to:

[0154] Determine whether the uncertainty parameter satisfies the first criterion, and based on the uncertainty parameter that satisfies the first criterion, process the audio data according to the first signal processing scheme to provide audio output.

[0155] Clause 11 pursuant to Clause 10, an audio device wherein the one or more processors are configured to: process audio data according to a second signal processing scheme to provide audio output based on an uncertainty parameter that does not satisfy the first criterion and / or an uncertainty parameter that satisfies the second criterion.

[0156] Clause 12 pursuant to Clause 11, an audio device wherein the one or more processors are configured to: process audio data according to a third signal processing scheme to provide audio output based on uncertainty parameters that do not meet the first criterion and uncertainty parameters that do not meet the second criterion.

[0157] Clause 13 An audio device pursuant to any of the preceding clauses, wherein the one or more processors are configured to output the audio output via the interface.

[0158] Clause 14 A method (100) for operating an audio device, the method comprising:

[0159] Acquire (S102) audio data;

[0160] Process (S106) the audio data to provide audio output;

[0161] Process (S108) the audio data to provide one or more audio parameters indicating one or more characteristics of the audio data;

[0162] The one or more audio parameters are mapped (S110) to the first latent space of the first neural network to provide mapping parameters indicating whether the one or more audio parameters belong to the training manifold of the first latent space;

[0163] Based on this mapping parameter, an uncertainty parameter indicating the uncertainty of the processing quality is determined (S112); and

[0164] Based on this uncertainty parameter, the processing of the audio data is controlled (S114) to provide the audio output.

[0165] Clause 15 pursuant to the method (100) of Clause 14, wherein processing (S108) the audio data to provide one or more audio parameters includes encoding (S108A) the audio data using the first neural network to provide the one or more audio parameters.

[0166] Clause 16 pursuant to any one of Clauses 14 to 15, the method (100) wherein processing (S106) the audio data to provide an audio output includes providing the audio data as input (S106A) to a second neural network, and processing (S106B) the audio data using the second neural network based on the uncertainty parameter to provide a first-level output, and wherein the audio output is based on the first-level output.

[0167] Clause 17 pursuant to the method (100) of Clause 16, wherein the second neural network comprises a deep neural network.

[0168] Clause 18 pursuant to any one of Clauses 16 to 17, method (100), wherein controlling (S114) the processing of audio data to provide audio output includes controlling (S114A) the processing of audio data based on controller output.

[0169] Clause 19 pursuant to any one of Clauses 16 to 18, method (100), wherein processing (S106) the audio data to provide an audio output includes processing (S106C) the audio data using a secondary processor to provide a secondary output different from the primary output, wherein the audio output is based on the secondary output.

[0170] Clause 20 pursuant to any one of Clauses 16 to 19, method (100), wherein processing (S106) the audio data to provide audio output includes mixing (S106D) the primary output and the secondary output to provide a mixed output, and wherein the audio output is based on the mixed output.

[0171] Clause 21 The method (100) according to any one of Clauses 14 to 20, wherein the training manifold includes a probability distribution, and wherein mapping (S110) the one or more audio parameters includes mapping (S110A) the one or more audio parameters to the probability distribution to provide the mapping parameters.

[0172] Clause 22 pursuant to any one of Clauses 14 to 21, method (100), wherein mapping (S110) the one or more audio parameters includes determining (S110B) the distance between the one or more audio parameters and the training manifold.

[0173] Clause 23 pursuant to any one of Clauses 14 to 22, the method (100) comprising determining (S104) whether the uncertainty parameter satisfies a first criterion, and processing (S106E) the audio data according to a first signal processing scheme to provide the audio output based on the uncertainty parameter that satisfies the first criterion.

[0174] Clause 24 pursuant to the method (100) of Clause 23, wherein the method (100) includes determining (S105) whether the uncertainty parameter satisfies the second criterion, and processing (S106F) the audio data according to the second signal processing scheme to provide the audio output based on the uncertainty parameter that does not satisfy the first criterion and / or based on the uncertainty parameter that satisfies the second criterion.

[0175] Clause 25 pursuant to the method (100) of Clause 23, wherein the method (100) comprises: processing (S106G) the audio data according to a third signal processing scheme to provide the audio output based on the uncertainty parameter that does not satisfy the first criterion and the uncertainty parameter that does not satisfy the second criterion.

[0176] Clause 26 pursuant to any one of Clauses 14 to 25, the method (100) includes outputting (S116) the audio output.

[0177] The use of the terms "first," "second," "third," and "fourth," "level one," "level two," and "level three," etc., does not imply any particular order, but is included to identify individual elements. Furthermore, the use of the terms "first," "second," "third," and "fourth," "level one," "level two," and "level three," etc., does not indicate any order or importance, but is used to distinguish one element from another. Note that the terms "first," "second," "third," and "fourth," "level one," "level two," and "level three," etc., are used here and elsewhere for labelling purposes only and are not intended to indicate any particular spatial or temporal order. Moreover, the labeling of a first element does not imply the existence of a second element, and vice versa.

[0178] It is understood that the accompanying drawings include some circuits or operations shown in solid lines and some circuits, components, features, or operations shown in dashed lines. The circuits or operations included in solid lines are those included in the widest example. The circuits, components, features, or operations included in dashed lines are examples that can be included in or are part of the circuits, components, features, or operations of the solid-line examples, or other circuits, components, features, or operations that may be employed in addition to those of the solid-line examples. It should be understood that these operations do not need to be performed in the order shown. Furthermore, it should be understood that not all operations need to be performed. Exemplary operations can be performed in any order and in any combination. It should be understood that these operations do not need to be performed in the order shown. The circuits, components, features, or operations included in dashed lines can be considered optional.

[0179] Other operations not described herein may be combined with the exemplary operations. For example, one or more additional operations may be performed before, after, simultaneously with, or between any described operation.

[0180] Some of the features discussed above as individual implementations can also be combined to implement a single implementation. Conversely, features described as a single implementation can also be implemented individually or in any suitable sub-combination in multiple implementations. Furthermore, although features may be described above as functioning in certain combinations, in some cases, one or more features from the claimed combination may be removed from that combination, and that combination may be claimed as any sub-combination or any variation of a sub-combination.

[0181] It should be noted that the word "including" does not necessarily exclude the presence of other elements or steps besides those listed.

[0182] It should be noted that the word "one" or "a" before "component" does not preclude the existence of multiple such components.

[0183] It should also be noted that any reference numerals in the drawings do not limit the scope of the claims, examples may be implemented at least in part by means of hardware and software, and several “components,” “units,” or “devices” may be represented by the same hardware item.

[0184] While features have been shown and described, it should be understood that they are not intended to limit the scope of the claimed disclosure, and it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the scope of the claimed disclosure. Therefore, the specification and drawings are to be considered illustrative rather than restrictive. The claimed disclosure is intended to cover all alternatives, modifications, and equivalents.

[0185] References

[0186] 10 audio devices; 10A output interface

[0187] 10B Input Interface; 10C One or More Processors

[0188] 10D audio speakers; 10E1 first microphone

[0189] 10E2 second microphone; 10F first wireless transceiver

[0190] 10G Second Wireless Transceiver; 12 Uncertainty Quantization Deterministor

[0191] 12A encoder; 12B uncertainty parameter determiner

[0192] 14 signal processors; 14A controller

[0193] 14B Digital Signal Processor; 14C Mixer

[0194] 14D secondary processor; 16 first transceiver interfaces

[0195] 18 Second transceiver interface; 20 Encoder

[0196] 22 First potential space; 24 Decoder

[0197] 26 Training manifolds; 28 A First training data points

[0198] 29A First audio parameter; 29B Second audio parameter

[0199] 30A training audio data; 30B audio data

[0200] 32 Training audio output; 50 First microphone input signal

[0201] 52 inputs, 56 outputs

[0202] 58 outputs; 60 controller outputs

[0203] 61 Controller Output; 62 Controller Output

[0204] 64 Feedback Output; 66 Level 1 Output

[0205] 68 secondary outputs; 70 mixed outputs

[0206] 72 Second microphone input signal; 74 Transceiver input signal

[0207] 76 transceiver interface outputs; 78 audio speaker inputs

[0208] 80 Second transceiver input; 82 Second transceiver output

[0209] 100 Method for operating an audio device; S102 Acquiring audio data

[0210] S104 Determine whether the uncertainty parameters satisfy the first criterion.

[0211] S105 Determine whether the uncertainty parameter satisfies the second criterion.

[0212] S106 processes audio data to provide audio output.

[0213] S106A provides audio data as input to the second neural network.

[0214] S106B uses a second neural network to process audio data.

[0215] The S106C processes audio data to provide secondary output.

[0216] S106D Mixed Primary and Secondary Outputs

[0217] S106E processes audio data according to the first signal processing scheme.

[0218] S106F processes audio data according to the second signal processing scheme.

[0219] The S106G processes audio data according to a third signal processing scheme.

[0220] S108 processes audio data to provide one or more audio parameters.

[0221] S108A encoded audio data

[0222] S110 maps one or more audio parameters

[0223] S110A maps one or more audio parameters to a probability distribution.

[0224] S110b determines distance

[0225] S112 Determine the uncertainty parameters

[0226] S114 controls the processing of audio data.

[0227] S114A controls the processing of audio data.

[0228] S116 acquires audio output.

Claims

1. An audio device (10) comprising a memory, an interface, and one or more processors (10C), wherein, The one or more processors (10C) are configured to: Acquire (S102) audio data; Process (S106) the audio data to provide audio output; Process (S108) the audio data to provide one or more audio parameters indicating one or more characteristics of the audio data; Map the one or more audio parameters (S110) to a first latent space of a first neural network to provide mapping parameters indicating whether the one or more audio parameters belong to a training manifold of the first latent space; based on the mapping parameters, determine (S112) uncertainty parameters indicating uncertainty in processing quality; as well as Based on the uncertainty parameter, the processing of the audio data is controlled (S114) to provide the audio output.

2. The audio device (10) according to claim 1, wherein, Processing (S108) the audio data to provide one or more audio parameters includes encoding (S108A) the audio data using the first neural network to provide the one or more audio parameters.

3. The audio device (10) according to any one of the preceding claims, wherein, The one or more processors (10C) include a digital signal processor (14B) which includes a second neural network, and wherein processing (S106) the audio data to provide the audio output includes providing (S106A) the audio data as input to the second neural network, and using the second neural network to process (S106B) the audio data to provide a first-level output based on the uncertainty parameter, and wherein the audio output is based on the first-level output.

4. The audio device (10) according to claim 3, wherein, The second neural network includes a deep neural network.

5. The audio device (10) according to claim 3, wherein, The digital signal processor (14B) includes a controller (14A) configured to determine a controller output based on the uncertainty parameter, and wherein controlling (S114) the processing of the audio data to provide the audio output includes controlling (S114A) the processing of the audio data based on the controller output.

6. The audio device (10) according to claim 3, wherein, The one or more processors (10C) include a secondary processor (14D) different from the digital signal processor (14B), wherein processing (S106) the audio data to provide the audio output includes using the secondary processor (14D) to process (S106C) the audio data to provide a secondary output different from the primary output, wherein the audio output is based on the secondary output.

7. The audio device (10) according to claim 6, wherein, The one or more processors (10C) include a mixer (14C), and wherein processing (S106) the audio data to provide the audio output includes mixing (S106D) the primary output and the secondary output to provide a mixed output, and wherein the audio output is based on the mixed output.

8. The audio device (10) according to claim 1, wherein, The training manifold (26) includes a probability distribution, and wherein mapping (S110) the one or more audio parameters includes mapping (S110A) the one or more audio parameters to the probability distribution to provide the mapping parameters.

9. The audio device (10) according to claim 1, wherein, Mapping the one or more audio parameters to the first latent space (22) of the first neural network (S110) includes determining (S110B) the distance between the one or more audio parameters and the training manifold (26).

10. The audio device (10) according to claim 1, wherein, The one or more processors (10C) are configured to: (S104) Determine whether the uncertainty parameter satisfies the first criterion, and based on the uncertainty parameter that satisfies the first criterion, process (S106E) the audio data according to the first signal processing scheme to provide the audio output.

11. The audio device (10) according to claim 10, wherein, The one or more processors (10C) are configured to process (S106F) the audio data according to a second signal processing scheme to provide the audio output, based on the uncertainty parameter that does not meet the first criterion and / or the uncertainty parameter that meets the second criterion.

12. The audio device (10) according to claim 11, wherein, The one or more processors (10C) are configured to process (S106G) the audio data according to a third signal processing scheme to provide the audio output based on the uncertainty parameter that does not meet the first criterion and the uncertainty parameter that does not meet the second criterion.

13. The audio device (10) according to claim 1, wherein, The one or more processors (10C) are configured to output the audio output via the interface (10A).

14. A method (100) for operating an audio device, the method comprising: Acquire (S102) audio data; Process (S106) the audio data to provide audio output; Process (S108) the audio data to provide one or more audio parameters indicating one or more characteristics of the audio data; Map the one or more audio parameters (S110) to a first latent space of a first neural network to provide mapping parameters that indicate whether the one or more audio parameters belong to a training manifold of the first latent space; Based on the mapping parameters, determine (S112) the uncertainty parameters that indicate the uncertainty of the processing quality; as well as The processing of the audio data is controlled (S114) based on the uncertainty parameter to provide the audio output.

Images