Method of training a machine learning system of a hearing device, and a hearing device system

Abstract

A method of training a machine learning system (16) of a hearing device (1) for evaluating a seating of an earpiece when worn by a user. The invention also refers to a hearing device system, a method of operating a hearing device system and an internet server.

Description

[0001] Specification

[0002] Method of training a machine learning system of a hearing device, and a hearing device system

[0003] Field of the invention

[0004] The invention refers to a method of training a machine learning system of a hearing device. The invention refers also to a hearing device system. Further the invention refers to a method of operating a hearing device system and an internet server.

[0005] Hearing devices are regularly used to output a sound signal to the hearing of the wearer of this hearing device. The output is generated by means of an output transducer, usually acoustically via airborne sound by means of a loudspeaker (also referred to as "receiver"). Such hearing devices are often used as so-called hearing aids. For this purpose, hearing devices usually comprise an acoustic input transducer (in particular a microphone) and a signal processor which is set up to process the input signal (also: microphone signal) generated by the input transducer from the ambient sound. Such sound processing is done by applying at least one signal processing algorithm, usually stored in a user-specific manner, in such a way that a hearing loss of the wearer of the hearing device is at least partially compensated. In particular in the case of a hearing aid, the output transducer may be, alternatively to a loudspeaker, a so-called bone-conduction earphone or a cochlear implant, which are set up for mechanical or electrical coupling of the sound signal into the hearing of the wearer. The term hearing devices additionally includes, in particular, devices such as so-called tinnitus maskers, headsets, headphones and the like.

[0006] Typical designs of hearing devices, in particular hearing aids, are behind-the-ear ("BTE") and in-the-ear ("ITE") hearing devices. These designations refer to the intended wearing position. In-the-ear hearing aids have a (main) housing that is worn behind the auricle. A distinction can be made between models whose loudspeaker is located in this housing - the sound is usually emitted to the ear by means of a sound tube that is worn in the ear canal - and models that have an external loudspeaker that is placed in the ear canal. In-the-ear hearing aids, on the other hand, have a housing that is worn in the pinna or even completely in the ear canal.

[0007] Usually, hearing devices, especially hearing aids, have been provided by audiologists - specialists who adapt the hearing devices to the users, with respect to a physiological fitting as well as to a functional adaption to the needs of the user in terms of adjusting processing parameters. However, buying hearing aids without having an audiologist at hand may also get more and more interesting in the near future. One point are countries where many people do not have access to an audiologist. Another point are also people who already do a great deal of their shopping over the internet. In both cases, mostly nobody is there to help with physiological fitting. However, even for hearing devices having personally physiologically adapted parts, especially earpieces (typically denoted as custom earpieces), the respective user can still have struggles with achieving a correct positioning (seating) of the earpiece and, typically, this will be even more the case with earpieces that are configured for fitting a lot of people.

[0008] Summary of the invention

[0009] The invention is based on the problem to provide an easy to wear hearing device.

[0010] That problem is solved according to the invention by a method with the features of claim 1 . Also, that problem is solved according to the invention by a hearing device system with the features of claim 7. Furthermore, that problem is solved by a method of operation a hearing device system with the features of claim 10 and by an internet server with the features of claim 11 . Further advantageous embodiments and developments that may be inventive for themselves are laid out in the dependent claims and the following specification. The method according to the invention is designed and serves for training a machine learning system of a hearing device, the machine learning system being designed and serving for evaluating a seating of an earpiece of the hearing device within an ear canal of a user (i.e. , when the earpiece is worn by the user). The method according to the invention comprises the steps of providing a first data set comprising data representing settings of a feedback cancellation module of the hearing device; performing an unsupervised training step of an auto-encoder part of the machine learning system using the first data set; providing a second data set comprising data representing settings of the feedback cancellation module of the hearing device, wherein the data of the second data set are labelled with prior knowledge about the seating of the earpiece; and performing a supervised training step of an evaluation part of the machine learning system using the second data set. The evaluation part is configured to evaluate the seating of the earpiece on the basis of input data that is an output of the auto-encoder part, wherein the input data to the evaluation part more specifically is the output data from the encoder sub-part of the auto-encoder part. Optionally, within the supervised training step a layer in a latent space of the evaluation part is being trained.

[0011] Especially the step of unsupervised training of the auto-encoder has the advantage that even a small amount of supervised training data may be sufficient for a training effect. That is due to the trained auto-encoder offering a reduction in dimensionality of the latent space data, by discarding input data that is not really relevant, which facilitates the amount of supervised training required to obtain an efficient machine learning system training data.

[0012] Training of an auto-encoder, especially an auto-encoder neural network, is generally known e.g. from EP 3 596664.

[0013] According to a preferred variant of the method, the unsupervised training step of the auto-encoder part is performed before the supervised training step.

[0014] According to a further preferred variant of the method, an encoder sub-part and / or a decoder sub-part of the auto-encoder part, which is trained within the unsupervised training step, is unaffected, i.e. preferably left as it is trained, during the supervised training step.

[0015] According to a useful variant of the method, as said auto-encoder part a variable auto-encoder or a partial variable auto-encoder is used.

[0016] According to an advantageous variant of the method, the data representing settings of the feedback cancellation module comprises coefficients of an adaptive feedback cancellation filter of the feedback cancellation module (“adaptive feedback cancellation filter coefficients” or short: “feedback filter coefficients”). The data set used for the supervised training step (especially, the second data set) comprises such feedback filter coefficients that are (additionally) labelled with respect to proper or improper seating of the earpiece within the ear canal of the user. Said feedback filter coefficients determine a transfer function of the adaptive feedback cancellation filter and hereby also control how the adaptive feedback cancellation filter reacts to and reduces feedback during intended operation of the hearing device. Thus, the feedback filter coefficients resemble a measure of the seating of the earpiece within the ear canal since improper positioning of the earpiece usually leads to increased feedback.

[0017] According to an advantageous variant of the method both the first data set and the second data set comprise additional information with respect to the context of each specific setting of the feedback cancellation module, wherein said additional information comprises at least one of: a type of the hearing device, a type of an earpiece, a size of a vent of the hearing device (especially, the earpiece, an age of the user of the hearing aid (1 ), a measure regarding an experience of the user with respect to wearing hearing devices in general and a measure regarding an experience of the user with respect to wearing the presently worn hearing device (including the specific earpiece). These additional information pieces may increase efficiency and / or precision of the training of the machine learning system. Typically, such information is available in the hearing device’s memory (preferably an EEPROM) after initial fitting. According to a further variant of the method, an open loop gain value is derived from the feedback filter coefficients. Optionally, the open loop gain value is retrieved (especially also for the feedback filter) by way of a specific open loop gain measurement. Such a measurement is often performed by an audiologist, e.g., during a fitting session. Additionally or alternatively, such a measurement may also be initiated and performed automatically by the hearing device (i.e. without interference from a user or audiologist), e.g., during first initialization, within regular intervals or the like. The open loop gain value, as defined in the present context, provides information about a transfer function of the hearing device while not within its intended wearing position and, thus, can be used for evaluation of the seating of the earpiece (e.g., by providing some kind of comparison to a position completely not within the ear canal).

[0018] According to a useful advantageous variant of the method, the data representative for settings of the feedback cancellation module is derived from an initial first fitting of the hearing device (e.g., determined during such a fitting). Alternatively, these data may also be acquired during normal operation of the hearing device or as part of a specific investigation where the earpiece of the hearing device is deliberately positioned in either a correct or incorrect position or in a position between the two. Optionally, said specific setting of the feedback cancellation module consists of a plurality of feedback filter coefficients.

[0019] Preferably, for the machine learning system a number (especially a plurality) of neural networks is used. Advantageously, as such neural networks deep neural networks are used. However, in the alternative also only one neural network with different features may be used within the scope of the invention. Optionally, also more than one, in particular two, neural networks with distinct features are used. In that case neural networks realizing the encoder sub-part of the auto-encoder part and an evaluation neural network respectively are used.

[0020] The hearing device system according to the invention comprises a hearing device which in turn comprises an acoustoelectric transducer (preferably a microphone) adapted for converting a sound (especially an ambient sound) input into an (preferably electrical) input signal, a signal processor for generating an (preferably electrical) output signal on the basis of the input signal, an electroacoustic transducer (preferably a loudspeaker or receiver) for generating acoustic signals on the basis of the output signal, and an earpiece for transmitting the acoustic signals into an ear canal of the user of the hearing device. Thus, the earpiece accommodates the electroacoustic transducer (directly) or serves for placing a sound tube within the ear canal. The hearing device system further comprises a feedback cancellation module comprising an adaptive feedback cancellation filter configured to suppress acoustical feedback which is often caused by the acoustic signals in the hearing device. Also, the hearing device system comprises a machine learning system (especially the afore mentioned one) for evaluating the seating of the earpiece within the ear canal during intended use. The machine learning system is trained according to the method described before. Furthermore, the hearing device system is configured to alert (or: notify) the user if the machine learning system evaluates, based on said input data, that the earpiece is not seated correctly (also: inserted correctly).

[0021] Thus, the hearing device shares all features described before with respect to the training method.

[0022] Preferably, the hearing device is configured to provide a computing module - optionally as a part of the signal processor - where the machine learning system is executed during intended usage. Thus, the hearing device system may consist of only a single hearing device itself.

[0023] According to an optional embodiment, the hearing device system further comprises a computing system separately from the hearing device. Such a computing system may be resembled by a smart mobile phone, a tablet or even a laptop or personal computer. That computing system hosts the machine learning system (i.e. , provides a hardware and / or software environment for the machine learning system) and is configured to provide at least one of an audio alert and a visual alert if the machine learning system evaluates that the earpiece is not seated correctly. Additionally or alternatively, the hearing device system comprising the above described machine learning system may advantageously be configured to monitor the seating of the earpiece of the hearing device over a differently long terms, e.g., over a whole day of wearing (as, e.g., a “short term monitoring”) or even over many days of use (’’long term monitoring”), even with wearing pauses such as during sleep. For these use cases and according to a useful embodiment, the machine learning system is configured to monitor and hereby detect whether the position of the earpiece is somewhere between correctly positioned and incorrectly positioned.

[0024] In the case of long term monitoring, it may be derived that the earpiece is getting worn over time which may lead to a less good seating within the ear canal. The hearing device may be configured such that such a migration effect is encountered by amended sound processing parameters. E.g., a sound leakage through an air gap, due to incorrect earpiece position, may be compensated for by increasing a gain or the like. In the case of short term monitoring such as for a whole day, a migration of the earpiece may result during usage, e.g., due to impact during sports, that would lead to a less good seating, as well, and in response to a detection of such a less good position an alert can be provided to hearing device user.

[0025] Optionally, in the case of the long term monitoring, a plurality of specific earpieces (e.g., for different hearing device types) may be used to - on one side - develop new improved earpieces and - on the other side - out-phase earpiece types that tend to end up being positioned incorrectly.

[0026] Further optionally, in the case of the long term monitoring, the long term monitoring may be combined with additional information such as the user’s experience with wearing earpieces in order to assist in estimating when a hearing care professional can trust that a user wears the earpieces correctly positioned. The method of operating the above mentioned hearing device system according to the invention comprises the steps of storing, over time, a plurality of estimates of how correctly the earpiece is seated together with a time stamp for each estimate, as well as of analyzing said stored data. As a further step, in response to the analyzing step the method comprises the step of suggesting to the user to replace the earpiece in case a continuous and slowly increasing deviation from the correct seating of the earpiece is detected and in case that the deviation becomes larger than a first threshold while still being smaller than a second threshold wherein the second threshold represents a trigger threshold with respect to whether to alert the user that the earpiece is not correctly positioned.

[0027] The internet server according to the invention is configured to receive first data from a plurality of hearing device systems of the kind described before, wherein said first data is initially stored by the respective hearing device systems using the method described above. The internet server is further configured to receive, together with said first data and for each of said plurality of hearing device systems, second data comprising at least one of: the hearing device type, the earpiece type, the vent size of the hearing device, the age of the hearing device user, the experience of the hearing device user with respect to wearing hearing devices generally, the experience of the hearing device user with respect to wearing the presently worn hearing device. Also, the internet server is configured to use said received first and second data to identify earpiece types that need to be improved, especially regarding their seating.

[0028] In the following, an embodiment of the invention is explained in more detail with reference to a drawing. Therein show:

[0029] Fig. 1 in a schematic side-view a hearing device,

[0030] Fig. 2 in a schematic process diagram a method for training a machine learning system of the hearing device, and

[0031] Fig. 3 schematically an auto-encoder part together with the associated data in order to explain an unsupervised training step of a machine learning system according to the invention. Parts corresponding to each other are always provided with the same reference signs in all figures.

[0032] In the following the terms “neural network” (NN) and “deep neural network” (DNN) may be used interchangeably since the difference is somewhat blurred because the main difference is that a DNN typically consists of multiple hidden layers whereas a NN typically only has two or three layers (i.e. an input, hidden and output layer).

[0033] Fig. 1 shows in a schematic view a hearing device system comprising a hearing device 1 in the form of a hearing aid. The hearing device 1 comprises a housing 2 that is to be worn behind the auricle of a user and a receiver 4 to be worn within an ear canal 6. The hearing device 1 further comprises two microphones 8 for picking up sounds from surroundings of the hearing device 1 and transforming them into electrical microphone signals SM. Additionally, the hearing device 1 comprises a signal processor 10 for processing the microphone signals SM into a processed output signal SO that is output by the receiver 4 acoustically, cf. airborne sound AS resembled by arrows in front of the receiver 4. The signal processor 10 is configured - by software means or by circuitry means, e. g. by way of an EEPROM - to perform frequency dependent audio processing, especially filtering, selective enhancement of gain and the like. Also, the hearing device 1 comprises an earpiece 12 that houses the receiver 4 and is configured to be seated (i.e. positioned) within the ear canal 6 and hold the receiver 4 within the user’s ear canal 6.

[0034] Additionally, the earpiece 12 is configured to, at least partly, acoustically seal the ear canal 6 from the surroundings up to a prescribed level such that sound from the surroundings does not or only at a reduced level reach the eardrum of the user. Thereby the earpiece 12 also has an impact on airborne sound AS leaking from the ear canal 6 and reaching the microphones 8. Such pickup of the airborne sound AS from the receiver 4 would cause a feedback loop FL since the pickup of the “own” sound signals (i.e., the sound signals output by the receiver 4 that are in turn caused by the microphones 8 pickup) most often leads to a whistling tone. Since such a feeback loop FL is most often not completely avoidable, the hearing device 1 comprises a feedback cancellation module 14 that is - in the embodiment given by Fig. 1 - embedded within the signal processor 10. The feedback cancellation module 14 is configured in a way known to a person skilled in the art to estimate a transfer function for the feedback loop FL and based thereon determine feedback filter coefficients FBC for setting up an adaptive feedback cancellation filter - i.e. an adaptive filter for reducing or prohibiting acoustical feedback from occurring - which is part of the feedback cancellation module 14.

[0035] Since the seating of the earpiece 12 within the ear canal 6 (especially the kind of seating, e.g., where the earpiece 12 fits tightly into the ear canal 6) has an impact on the feedback loop FL, the hearing device 1 , especially the signal processor 10, comprises a machine learning system 16. The machine learning system 16 is configured to and serves for evaluating the seating of the earpiece 12 within the ear canal 6 and for enabling notifying of the user about a non-correct seating. The machine learning system 16 is fed during operation, either periodically or in response to a trigger action, with the feedback filter coefficients FBC since these bear information about the degree of feedback or in other words about the amount of airborne sound AS from the receiver 4 picked up via the microphones 8.

[0036] For that the machine learning system 16 comprises an encoder sub-part 22 (not shown in detail) and a downstream evaluation part (not shown in detail). The encoder sub-part 22 is a neural network (e.g. a deep neural network) trained to reduce the dimensionality of input data comprising the feedback filter coefficients FBC. The output of the encoder sub-part 22 is the data input to the evaluation part (not shown) which is also a neural network (such as a deep neural network) trained to estimate from the input data information about the seating of the earpiece 12, e.g. in form of a binary decision such as either proper seating or improper (incorrect) seating of the earpiece 12.

[0037] In an (optional) embodiment that decision is forwarded to the user acoustically, e.g. as some kind of welcome jingle (e.g. in response to the hearing device 1 having been powered up) when the user positions the hearing device 1 correctly in the ear.

[0038] Fig. 2 schematically shows a method for training the machine learning system 16. Within a first step S1 a first data set is gathered. Such data is representative of a general setting of the feedback cancellation module 14 of hearing devices of the same kind as the hearing device 1. In particular, this data, from a plurality of users, comprises feedback cancellation filter coefficients (e.g., the feedback filter coefficients FBC for each of the respective hearing devices 1 of said plurality of users) for the feedback cancellation module 14. Some of these filter coefficients are set during fitting sessions while others may have been logged during use of the respective hearing devices 1 .

[0039] That first data set is used within a second step S2 for training the auto-encoder part 20 (i.e. both the encoder sub-part 22 and the decoder sub-part 24) of the machine learning system 16. The training performed within step S2 is an unsupervised training.

[0040] Within a third step S3 a second data set is provided. The second data set comprises input objects of the same type as the first data set used in the first step S1 . However, the second data set is additionally associated with prior knowledge, e.g., in the form of a desired output value for each data input object (such as a vector of predictor variables). According to one embodiment the available output values are limited to: either correctly positioned or incorrectly positioned, but the available desired output values may also comprise more than two options e.g. by including desired output values representing positions between correctly and incorrectly positioned.

[0041] Additionally (and according to a preferred embodiment), both the first and the second data set comprise information about at least one of the following:

[0042] - the type of hearing device;

[0043] - the kind of earpiece used with the respective hearing device (e.g., whether the earpiece is custom fitted to the user, vented, size of the vent and the like); - the age of the user;

[0044] - the experience of the user with respect to wearing hearing devices generally and / or the experience of the user with respect to wearing the presently worn hearing device 1 ;

[0045] - an open loop gain value (provided from e.g. an initial fitting or some other semiautomatic or fully automatic test carried out by the respective hearing device 1 ).

[0046] The second data set is used within a fourth step S4 to train the evaluation part of the machine learning system 16 by a supervised training step. During that fourth step S4 the second data set is input to the encoder sub-part 22 (of the autoencoder part 20) and the output from the encoder sub-part 22 is used as input to the evaluation part of the machine learning system 16. The encoder sub-part 22 has already been trained in step S2 and is consequently not affected, i.e. not trained further. In particular, the evaluation part comprises at least one layer.

[0047] In an embodiment the evaluation part is a neural network for binary classification, i.e. the evaluation part can provide an output that represents whether or not the seating (i.e. insertion or positioning) of an earpiece 12 when worn by a user is correct.

[0048] In an embodiment the evaluation part (i.e. the evaluation neural network) consists of three layers: an input layer, a single hidden layer and an output layer. More specifically, the input layer comprises of 64 neurons, which each are adapted to receive as input the output from the encoder sub-part 22. The hidden layer may comprise 32 neurons of the same type as the input layer and all the neurons of both the input layer and the hidden layer may use as activation function ReLU (i.e. Rectified Linear Unit which is particularly computationally efficient). In an embodiment the output layer has only a single neuron with e.g. a sigmoid activation function, which is a suitable choice of activation function for binary classification (i.e. correct seating or not).

[0049] According to an alternative embodiment a multi-class classification is applied in order to e.g. enable an estimation of whether the seating is either correctly seated or not or is mostly correctly seated or mostly not correctly seated and in that case the output layer will typically consists of multiple neurons (e.g. corresponding to the number of possible classification results) and in that case it can be preferred to apply e.g. softmax activation function instead of the sigmoid activation function. However, according to yet another alternative embodiment, a single neuron in the output layer can be used to provide an output representing a percentage of “how” correct the seating of the earpiece is by applying a regression model instead of a classification model and in response hereto applying a regression loss function such as Mean Squared Error (MSE) or Mean Absolute Error (MAE), which measure the difference between the predicted and actual values.

[0050] As will readily be understood by the skilled person the selected dimensions of a neural network can vary dependent on e.g. the required precision and / or the acceptable power consumption. More specifically cross-validation can be used to find the best network architecture. In an embodiment binary cross-entropy is used as the loss function, when training the evaluation neural network.

[0051] Thus, as will be clear for the skilled person the dimensions and the exact specific details of (both) the neural networks of the present application may vary in a number of different ways all of which will be obvious for the skilled person.

[0052] Reference is now made to Fig. 3, which schematically illustrates the auto-encoder part 20 of the machine learning system 16 together with the associated data. In Fig. 3 a first data set D1 (represented by a circle) is used to train the auto-encoder part 20. The encoder sub-part 22 is represented by a trapezoid shaped figure. Data in a latent space (“latent data DL”) of the auto-encoder part 20 is also represented by a circle. A decoder sub-part 24 of the auto-encoder part 20 is represented by another trapezoid shaped figure and a reconstructed data set D2 provided by the decoder sub-part 24 (i.e. , the data set optimized to re-construct the first data set D1 ) is represented by yet another circle (description from left to right).

[0053] Thus, in an embodiment of the second step S2 of Fig. 2 the data of said first data set D1 is input to the neural network (such as e.g. a deep neural network - DNN) of the encoder sub-part 22. The encoder sub-part 22 compresses the first data set D1 into a lower-dimensional representation (the “latent data DL”) that retains the most important information comprised in the first data set D1 while discarding unnecessary details.

[0054] However, according to another embodiment the latent variables are multi-modal, which means that a combination of multiple vectors or other data structures is used, whereby the auto-encoder part 20 can be enabled to capture different modes or variations in the input signal data.

[0055] The encoder sub-part 22 typically consists of multiple layers, such as fully connected (dense) layers or other types of specialized layers, depending on the specific problem domain. The number of layers, their sizes, and activation functions are determined based on the complexity of the input data and the desired representation - as it is known in the art. In an embodiment the input layer comprises 128 neurons, one for each of the filter taps in the feedback cancellation filter, but in obvious variations the number of neurons can be e.g. 256 or 64 or some other values. Furthermore the number of neurons need not necessarily be the same as the number of filter taps in the feedback cancellation filter and finally additional neurons can be added to the input layer to accommodate additional data such as about at least one of the following types of data (already listed above above): the type of hearing device; the kind of earpiece used with the respective hearing device (e.g., whether the earpiece is custom fitted to the user, vented, size of the vent and the like); the age of the user; the experience of the user with respect to wearing hearing devices generally and / or the experience of the user with respect to wearing the presently worn hearing device 1 ; an open loop gain value (provided from e.g. an initial fitting or some other semi-automatic or fully automatic test carried out by the respective hearing device 1).

[0056] The input data (i.e. , the first data set D1 ) are passed through the layers of the encoder sub-part 22 in a forward propagation manner. Each layer applies a set of weights and biases to the input and transforms it using a non-linear activation function. This process gradually processes the input through the encoder network, extracting higher-level features at each layer (i.e. the encoder network transforms the input data into a lower-dimensional space).

[0057] As the data propagates through the layers, the network's representation typically becomes progressively smaller. This reduction in dimensionality is achieved by reducing the number of neurons or the layer size in each subsequent layer. By compressing the information, the encoder sub-part 22 effectively captures the most important features while discarding redundant or less relevant information.

[0058] The encoder sub-part’s layers are designed to learn and extract meaningful features from the input data. Through the learning process, the network adjusts its weights and biases to minimize the discrepancy between the original input (i.e. the first data set D1 ) and the reconstructed output produced by the decoder sub-part 24 as will be explained in the following.

[0059] The encoder sub-part 22 culminates in a bottleneck layer, which represents the compressed representation of the input data (i.e. the latent data DL of the latent space). This layer has a significantly lower dimensionality than the original input, and it captures the essential information needed to reconstruct the data.

[0060] Next the lower dimensional representation (the latent data DL) is input to the decoder sub-part 24 that provides the reconstructed data set D2 as output data. The decoder sub-part 24 is construed with the target to reconstruct the original (first) input data set D1 from the lower-dimensionality (i.e. encoded) representation (latent data DL).

[0061] Thus, the unsupervised training of the auto-encoder part 20 comprises comparing the reconstructed data set D2 (i.e., the data output of the decoder sub-part 24) with the original input data to the encoder sub-part 22 (i.e., the first data set D1 ) and using a loss function, such as mean squared error, to measure the difference between the input data of the first data set D1 and the reconstructed data set D2, whereby the weights (and possibly other parameters) of the network can be optimized using optimization algorithms such as stochastic gradient descent (like e.g. ADAM stochastic gradient optimization) in order to minimize the loss function.

[0062] Additionally, validation data (i.e. , input data, that was not part of the first data set D1 used for the training described above), are used to assess the performance of the trained auto-encoder part 20. This can be done by determining metrics such as a reconstruction error (that may arise because the auto-encoder part 20 does not generalize well and therefore tends to overfit the original input data or by using qualitative analysis to determine how well the auto-encoder part 20 captures the essential features of the input data.

[0063] As will be well known by the skilled person the exact neural network architecture is typically obtained based on a trial and error approach whereby the optimum solution is found with respect to e.g.: the number and types of layers (i.e. input, output and hidden layers) in the network, the number of neurons in each layer, the activation functions and the connections of the neurons between the layers.

[0064] Furthermore, it is typically required to also use a trial and error approach in order to determine the right hyperparameters (such as the learning rate and the loss function) to be used during the training.

[0065] Thus, optionally, different network architectures are tested as part of the training in order to further enhance the auto-encoder part 20 (and obviously the same is true for the evaluation part).

[0066] Thus, the unsupervised training step S2 of the auto-encoder part 20 using the first data set D1 provides a representation (in the latent space) of the settings of the feedback cancellation module 14, wherein the representation will have a reduced dimensionality and as such will foster the quality of the machine learning system 16, even with a limited amount of labelled second data available. Thus, an advantageous solution is provided with respect to the issue that labelled data requires a lot of manual work to produce and as such adds significantly to the cost and time required for training the desired machine learning system 16.

[0067] According to a further embodiment, the auto-encoder part 20 is a variational autoencoder (VAE). Generally, a VAE uses encoders to compress data into a lowerdimensional (latent space) representation just like other auto-encoders. In an embodiment, the VAE is an auto-encoder that comprises an encoder sub-part that maps the input data to the mean and variance parameters of a probability distribution (such as a Gaussian distribution) in the latent space. By subsequent sampling from said probability distribution, the VAE encoder can stochastically generate a latent representation. This latent space is typically of lower dimensionality than the input space and hereby effectively compressing the input data. The encoder's architecture, in combination with the VAE's loss function, encourages the latent space to capture the essential features and variations present in the input data.

[0068] In an embodiment said VAE encoder is a convolutional neural network (CNN) encoder. This encoder type consists of multiple convolutional layers followed by pooling layers. These layers perform operations such as feature extraction, spatial down-sampling, and non-linear transformations. The convolutional layers capture local patterns and features, while the pooling layers reduce the spatial dimensions.

[0069] According to an embodiment, the auto-encoder is a VAE, which is especially advantageous because it is adapted to provide an estimate of the probability that the estimation (i.e. evaluation) of the earpiece position is correct. This can be used to adaptively optimize when an alarm to be provided to the user should be triggered. Thus, if e.g. a user is annoyed by receiving too many warnings that the earpiece 12 may not be correct positioned, then the probability estimate can be used to adaptively change the threshold for when the estimated probability of having a correct position is high enough to provide such a warning and if on the other hand a user would prefer to receive more warnings that the earpiece 12 is not correct positioned then the (estimated) probability threshold can be lowered. According to another embodiment the ability of the VAE to provide an estimate of the probabilities that the earpiece is either correctly or in-correctly positioned can be used to provide (graduated) warnings that, e.g., the position is closer to being in-correctly than correctly positioned by comparing the probability estimates for the two (extreme) position (seating) situations (i.e. , correct positioned vs in-correct positioned).

[0070] According to another embodiment, the auto-encoder part 20 is a partial variational auto-encoder (partial VAE) that is especially advantageous for the present use case because it enables training with a partly incomplete data set. More specifically, it may be an issue that the first data set comprises incomplete data if e.g. it is not known which earpiece 12 was used when the data was produced and recorded, or if the age of the hearing aid user is not known or if the hearing aid user's experience with wearing hearing aids generally or with the presently worn earpiece specifically is not known, or if the size (if any) of the hearing aid vent is not known. However, when using a partial VAE, it is possible to train the variational auto-encoder with a mix of data where not all elements of each individual data (vector / point) of the data set are known for all the individual data and hereby benefit from particularly the additional information (such as user’s age and / or the type of earpiece) that may be part of the data.

[0071] Finally, according to yet another aspect, a variational auto-encoder is also specifically advantageous for this invention due do its ability to efficiently reduce the dimensionality of the latent space and hereby relieve the requirements to the amount of supervised training comprising labelled data.

[0072] The subject matter of the invention is not limited to the embodiment examples described above. Rather, further embodiments of the invention can be derived by the skilled person from the above description. List of reference sings

[0073] 1 hearing device

[0074] 2 housing

[0075] 4 receiver

[0076] 6 ear canal

[0077] 8 microphone

[0078] 10 signal processor

[0079] 12 earpiece

[0080] 14 feedback cancellation module

[0081] 16 machine learning system

[0082] 20 auto-encoder part

[0083] 22 encoder sub-part

[0084] 24 decoder sub-part

[0085] SM microphone signal

[0086] SO output signal

[0087] AS airborne sound

[0088] FL feedback loop

[0089] FBC feedback filter coefficient

[0090] D1 first data set

[0091] DL latent data

[0092] D2 reconstructed data set

[0093] S1-S4 method step

Claims

Claims1. A method of training a machine learning system (16) of a hearing device (1 ) for evaluating a seating of an earpiece (12) when worn by a user, the method comprising the steps of:- providing a first data set comprising data representing settings of a feedback cancellation module (14) of the hearing device (1 ),- performing an unsupervised training step (S2) of an auto-encoder part (20) of the machine learning system (16) using the first data set wherein said auto-encoder part (20) comprises an encoder sub-part (22) and a decoder sub-part (24),- providing a second data set comprising data representing settings of the feedback cancellation module (14) of the hearing device (1 ), wherein the second data set comprises prior knowledge about the seating of the earpiece (12), and- performing a supervised training step (S4) of an evaluation part of the machine learning system (16) using the second data set, the evaluation part being configured to evaluate the seating of the earpiece (12), wherein the input data to the evaluation part is the output data from the encoder sub-part (22) of the auto-encoder part (20).

2. The method according to claim 1 , wherein the unsupervised training step (S2) is performed before the supervised training step (S4), and especially wherein an encoder sub-part and / or decoder sub-part of said auto-encoder part (20) is not trained during the supervised training step (S4).

3. The method according to claim 1 or 2, wherein as said auto-encoder part a variable auto-encoder or a partial variable auto-encoder is used.

4. The method according to any preceding claims,wherein both the first data set and the second data set comprise additional information with respect to the context of each specific setting of the feedback cancellation module (14), wherein said additional information comprises at least one of: a type of the hearing device (1), a type of an earpiece of the hearing device (1 ), a size of a vent of the hearing device (1 ), an age of the user, a measure regarding an experience of the user with respect to wearing hearing devices in general and a measure regarding an experience of the user with respect to wearing the presently worn hearing device (1 ).

5. The method according to claim 4, wherein said data representing settings of the feedback cancellation module (14) of the hearing device (1 ) comprises adaptive feedback cancellation filter coefficients.

6. The method according to any preceding claims, wherein as the machine learning system (16) a number of neural networks, especially deep neural networks are used.

7. A hearing device system with at least a hearing device (1 ), the hearing device (1 ) comprising- an acoustoelectric transducer (8) adapted for converting a sound input into an input signal (SM),- a signal processor (10) for generating an output signal (SO) on the basis of the input signal (SM),- an electroacoustic transducer (4) for generating acoustic signals (AS) on the basis of the output signal (SO),- an earpiece (12) for transmitting the acoustic signals (AS) into an ear canal (6) of a user of the hearing device (1 ),- a feedback cancellation module (14) comprising an adaptive feedback cancellation filter configured to suppress acoustical feedback, and- a machine learning system (16) for evaluating a seating of the earpiece (12) within the ear canal (6) of the user during intended use based oninput data comprising at least a plurality of adaptive feedback cancellation filter coefficients, the machine learning system (16) being trained by a method according to any of the preceding claims, wherein the hearing device system (1 ) is configured to alert the user if the machine learning system evaluates, based on said input data, that the earpiece (12) is not seated correctly.

8. The hearing device system (1) according to claim 7, further comprising a computing system separately from the hearing device (1 ), the computing system hosting the machine learning system (16) and being configured to provide at least one of an audio alert and a visual alert if the machine learning system (16) evaluates that the earpiece (12) is not seated correctly.

9. A method of operating a hearing device system according to claim 7 comprising the steps of:- storing, over time, a plurality of estimates of how correctly the earpiece (12) is seated together with a time stamp for each estimate, and- analyzing said stored data and in response hereto suggest to the user to replace the earpiece (12) in case a continuous and slowly increasing deviation from the correct seating of the earpiece (12) is detected and in case the deviation becomes larger than a first threshold while still being smaller than a second threshold that determines whether to alert the user that the earpiece is not correctly positioned.

10. An internet server configured to:- receive first data from a plurality of hearing device systems according to claim 7, wherein said received first data is initially stored by the respective hearing device systems using the method according to claim 9,- receive, together with said received first data and for each of said plurality of hearing device systems, second data comprising at least one of: the hearing device type, the earpiece type, the vent size of the hearing aid, the age of the hearing aid user, the experience of the hearing aid user withrespect to wearing hearing aids generally, the experience of the hearing aid user with respect to wearing the presently worn hearing aid; and- use said received first and second data to identify earpiece types that need to be improved.

Images