Statistical audiogram processing

Abstract

Techniques and corresponding systems are disclosed for estimating an audiogram of a user of a media playback device, the techniques and corresponding systems comprising: obtaining user hearing threshold data, sample hearing threshold data, obtaining at least one of sample calibration data and sample noise data for the user; and determining an estimate of the audiogram of the user based on such data. Related techniques for estimating calibration data for a media playback device, as well as corresponding computing apparatuses, computer programs, and computer-readable storage media, are also described.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 394,017 filed on August 1, 2022 and U.S. Provisional Patent Application No. 63 / 438,669 filed on January 12, 2023, each of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates to techniques for estimating an audiogram of a user of a media playback device and techniques for estimating calibration data for a media playback device.

[0004] Although some embodiments will be described herein with particular reference to this disclosure, it will be appreciated that the disclosure is not limited to this field of use and may be applicable in a wider context. Background Art

[0005] Any discussion of the background art throughout the disclosure should in no way be considered as an admission that such art is widely known or forms part of the common general knowledge in the art.

[0006] Hearing loss is a common problem caused by exposure to noise, disease, and aging. People with hearing loss may find it difficult to carry on a conversation. They may also have difficulty understanding dialogue in media content, fully enjoying music, or interacting efficiently with systems that include user interfaces or feedback mechanisms that rely on audio (e.g., voice assistants, voice commands or controls, etc.). There are several types of hearing loss, ranging from mild loss (in which a person's sensitivity to high-pitched sounds is reduced) to severe hearing loss (in which a person can hardly hear any sound unless the sound pressure level is very high). As a person ages, hearing loss gradually emerges. Because age-related hearing loss (also known as presbycusis) is progressive, a person with presbycusis may not realize that he or she has lost some ability to hear low-level sounds.

[0007] Hearing loss is traditionally assessed by an audiologist using carefully calibrated professional equipment. The assessment is performed in a soundproof environment. In addition to other tests, the audiologist will determine a so-called audiogram, which describes the amount of hearing loss in decibels as a function of frequency. The process includes measuring the audibility threshold expressed in sound pressure level (dB SPL) using a test signal such as a sine wave or a band-limited noise signal. The audiogram or hearing loss specified as a function of frequency is then determined by calculating the difference between the measured audibility threshold and the threshold of a healthy ear. This process requires accurate calibration data to allow the digital signal level present in the sound generator or test app (e.g., the level specified relative to the digital full-scale signal, or dB re FS) to be converted into the acoustic sound pressure level reproduced by the headphones used during the assessment.

[0008] Figure 1 Depicted is an example of an audiogram estimation process performed by the audiogram estimation system 100. The following process of estimating hearing loss is repeated for a range of frequencies (e.g., at multiple discrete frequencies) and for both ears of a listener (e.g., the process requires user / listener input in response to multiple discrete stimulus signals for each ear).

[0009] According to this example audiogram estimation process, a sound generator 110 generates a signal at a specific frequency that is reproduced on headphones to a listener (user) 10. The listener 10 indicates whether the signal is audible (in the form of a subjective response 120) to determine a threshold sound level 130 in the digital domain, expressed as dB re FS (full scale). This threshold 130 is then corrected for the frequency response of the headphones and playback device (using calibration data 140) to calculate a corresponding threshold 150 in dB SPL. Finally, the normal hearing level 160 at the specific frequency is subtracted to calculate the hearing loss (at the specific frequency) that enters the audiogram 170.

[0010] The resulting audiogram can be applied in hearing loss compensation systems, such as hearing aids or applications that apply hearing loss compensation to the playback of media content based on the audiogram data (e.g., media playback devices such as mobile phones, televisions, set-top boxes, computers, etc.). Figure 2An example of such a system 200 is schematically illustrated in FIG. The system 200 receives as input an audio content 20 corresponding to an input audio signal 210, and calculates the signal level of the audio input 210 in the digital domain at a signal level calculation block (or module) 220. This process is typically performed in two or more sub-bands (not shown in the figure). Subsequently, and for each sub-band, a hearing loss compensation gain 250 is calculated based on the user's audiogram data 270 and the sub-band digital signal level at a hearing loss compensation calculation block (or module) 230. In this step, before calculating the hearing loss compensation gain 250, the device level (e.g., device playback level or volume setting) and frequency response calibration data (e.g., data characterizing the acoustic output relative to the device level) - in other words, the device calibration data 270 - are essential to ensure that the digital signal level can be converted to the associated acoustic sound pressure level. In the final stage, the hearing loss compensation gain 250 is applied to (a sub-band of) the audio signal 210 at a hearing loss compensation gain application block (or module) 260, thereby creating a reproduced audio signal 280, which is sent to the headphone, earbud or hearing aid transducer of the listener (user) 10.

[0011] Traditionally, the process of obtaining an audiogram and setting a hearing aid is performed by an audiologist. The measurement of hearing loss is performed in a supervised manner under laboratory conditions, thereby providing an accurate estimate of hearing loss. However, mobile device applications that attempt to measure and compensate for hearing loss have recently been introduced. In order to obtain accurate measurements, these apps typically require consumers (users) to wear a defined set of headphones (e.g., headphones with known calibration data and known frequency response) in a quiet environment when running a hearing test. Similar to the assessment performed by an audiologist, the process used in the mobile app is repetitive (e.g., requiring the user to input in response to many discrete stimulus signals for each ear), cumbersome (e.g., difficult to follow error-prone procedures that require a lot of attention) and inefficient (e.g., consuming time and associated computing-related resources). The process of assessing hearing loss is often referred to as 'onboarding' or 'enrolment'.

[0012] Measuring hearing loss in a consumer domain setting comes with significant challenges, such as:

[0013] Lack of supervision. The entry process on a mobile device is often unsupervised compared to a process performed by an audiologist, increasing the risk of error and reducing the accuracy of the test results.

[0014] Environmental noise. Consumers may not have access to an environment that is quiet enough to accurately measure an audiogram, thus reducing the accuracy of the measured audiogram.

[0015] Unknown playback level. Due to the large number of mobile devices, the exact conversion from the digital signal level within the app to the sound pressure level produced by the headphones can vary significantly, introducing further inaccuracies in the estimated audiogram.

[0016] Unknown headphone frequency response. The conversion from digital signal level in the mobile device to sound pressure level is altered by the frequency response of the headphones used during the measurement. Due to the wide variety of headphone types and brands, the exact frequency response of the headphones is often unknown at the time of recording.

[0017] As a result of these challenges, hearing loss estimates for specific ears and frequencies have significant error or inaccuracy compared to assessments performed by audiologists.

[0018] Therefore, there is a need for improved techniques for hearing loss estimation. In particular, there is a need for such techniques that can compensate for at least one of lack of supervision, ambient noise, unknown playback level, and unknown headphone frequency response. There is a further need for techniques that allow estimation of device calibration data or the frequency response of headphones for later use in hearing loss estimation, including in cloud-based or server-based environments. Summary of the invention

[0019] In view of the above, the present disclosure provides a method of estimating an audiogram of a user of a media playback device and a method of estimating calibration data of a media playback device, as well as corresponding apparatus, programs and computer-readable storage media having the features of the respective independent claims.

[0020] According to a first aspect of the present disclosure, a method for estimating an audiogram of a user of a media playback device is provided. The method may include obtaining user hearing threshold data of the user. The user hearing threshold data may indicate a hearing threshold for one or more frequencies and for one or both ears. The hearing threshold in the user hearing threshold data may be related to a hearing threshold measurement. The hearing measurement may be performed at the media playback device. The method may further include obtaining sample hearing threshold data. The sample hearing threshold data may indicate a hearing threshold of an individual sample set (e.g., a group). The sample hearing threshold data (statistical hearing threshold data, group hearing threshold data) may indicate a hearing threshold of a pre-stored individual sample set. The method may further include obtaining at least one of sample calibration data and sample noise data. The sample calibration data may indicate a frequency response of a media playback device sample set (e.g., a group). Further, the sample noise data may indicate a variability in playback level and / or a variability in user response during measurement of the hearing threshold. The method may further include determining an estimate of the user's audiogram based on the user hearing threshold data, the sample hearing threshold data, and at least one of the sample calibration data and the sample noise data. The method may further include outputting the determined estimate of the audiogram and / or a set of compensation gains associated with the determined estimate of the audiogram, such as to an application capable of audio playback on a media playback device, or for transmission to another device (e.g., a server or cloud-based service).

[0021] By using statistics related to the hearing thresholds of a population of individuals, and further with statistics related to device calibration and / or noise, the proposed method can relate measurements at different frequencies and / or for different ears to each other, thereby improving the accuracy of the audiogram estimate. This can in particular address deficiencies in audiogram estimates that may be caused by unknown device data, lack of supervision, and less than ideal testing environments.

[0022] In some embodiments, determining the estimate of the audiogram may be further based on normal hearing data indicating expected hearing thresholds in the absence of hearing loss.

[0023] In some embodiments, determining the estimate of the audiogram may involve applying relative weights to the user hearing threshold data and the sample hearing threshold data based on at least one of the sample calibration data and the sample noise data.

[0024] Thereby, the relative influence of pre-stored audiogram data can be weighed based on the overall quality of the measured hearing thresholds to ensure that high-quality measurements are not "diluted" by the statistics, while ensuring that the negative impact of low-quality measurements is minimized.

[0025] In some embodiments, determining the estimate of the audiogram may be based on a Bayesian Maximum A Posteriori (MAP) estimation technique.

[0026] MAP estimation techniques provide a reliable tool for using prior knowledge (eg, about group audiograms and group calibration data and expected noise) to improve the estimate of a user's audiogram.

[0027] In some embodiments, the user hearing threshold data may indicate the hearing thresholds for a plurality of frequencies for a left ear and a right ear.

[0028] In some embodiments, the hearing threshold of the user hearing threshold data may be represented by a digital signal level of a playback device.

[0029] In some embodiments, obtaining user hearing threshold data may include outputting a plurality of audio signals (audio test signals) by a media playback device at different frequencies (and at different sound pressure levels, e.g., gradually increasing or decreasing sound pressure levels). The plurality of audio signals may have different frequencies, and for each frequency, there may be one such signal for each ear. The obtaining may further include receiving user input in response to the output audio signal. The obtaining may further include generating user hearing threshold data based on the received user input.

[0030] Therefore, the method can obtain the user's subjective responses to the audio test signal to determine the user's hearing threshold based on these subjective responses.

[0031] In some embodiments, the sample hearing threshold data may indicate information about an audiogram for an individual sample set. Additionally or alternatively, the sample hearing threshold data may indicate a mean and covariance of an audiogram for an individual sample set. Additionally or alternatively, the sample hearing threshold data may indicate a mean and covariance of hearing thresholds for corresponding frequencies and ears for an individual sample set. For example, the sample hearing threshold data may indicate a mean and covariance of a hearing threshold vector for an individual sample set. Each entry of the sample hearing threshold vector may relate to a given frequency and ear. Thus, the sample hearing threshold vector may have a dimension (2N F )×1, where NF (or K as defined below) is the number of frequencies.

[0032] In some embodiments, the sample calibration data may indicate a mean and covariance of a frequency response of a media playback device sample set. For example, the sample calibration data may indicate a mean and covariance of a frequency response vector of a media playback device sample set. Each entry of the frequency response vector may relate to a given frequency and ear. For example, the frequency response vector may have a dimension (2N F )×1, where N F is the number of frequencies. The mean can be obtained by a vector of dimension (2NF )×1, where each entry represents the mean of the frequency response of the corresponding frequency-ear pair. Thus, the covariance can be represented, for example, by a matrix of dimension (2N F )×(2N F )express.

[0033] In some embodiments, the estimated value of the audiogram Can be Given, where may be optional, and where M = ∑ a (∑ a +E) -1 , where E includes ∑ h , and At least one of a is the covariance of the vector representation of the sample hearing threshold data, is the mean of the vector representation of the sample hearing threshold data, ∑ h is the covariance of the vector representation of the sample calibration data, is the mean of the vector representation of the sample calibration data, represents the variability of the user's response in measuring hearing thresholds, represents the variability of the playback level, I is the identity matrix, and J is the all-ones matrix. The vector representation may include corresponding elements for each pair consisting of one of the left ear and the right ear and a frequency from a predetermined set of frequencies.

[0034] In some embodiments, the individual sample set may be selected based on at least one user attribute of the user. For example, the individual sample set may be selected based on at least one of the user's age, gender, or place of residence. The user attribute may be derived, for example, from user input or from information about the user from other sources.

[0035] Thus, the sample hearing threshold data may be selected so that the actual hearing threshold of the user is likely to be similar to the sample hearing threshold data.

[0036] In some embodiments, determining the estimate of the audiogram may be performed at the media playback device or at a server device in communication with the media playback device.

[0037] In some embodiments, the method may further include receiving audio data for playback at a media playback device. The method may further include determining a set of compensation gains based on the determined estimate of the audiogram. The method may further include generating hearing-optimized audio data by applying the determined set of compensation gains to the audio data. The method may further include rendering the hearing-optimized audio data for playback.

[0038] In some embodiments, determining the set of compensation gains may be further based on received audio data.

[0039] According to another aspect of the present disclosure, a method for estimating calibration data of a media playback device is provided. The calibration data may indicate a frequency response of the media playback device. The method may include obtaining first sample hearing threshold data. The first sample hearing threshold data may indicate a hearing threshold of a first individual sample set and be associated with a given device type. The method may further include obtaining second sample hearing threshold data. The second sample hearing threshold data may indicate a hearing threshold of a second individual sample set that is different from the first individual sample set. The second sample hearing data may not be associated with a given single device type. For example, the second sample hearing data may include a subset of multiple sample hearing data, each subset being associated with a corresponding device type. The method may further include determining an estimate of the calibration data based on the first sample hearing threshold data, the second sample hearing threshold data, and normal hearing data indicating an expected hearing threshold when there is no hearing loss.

[0040] Thus, calibration data for a given device type (eg, the type of media playback device currently used by a user) may be estimated, for example, for more accurate audiogram estimation or for storage for later use.

[0041] In some embodiments, the method may further include obtaining user hearing threshold data of a user of the media playback device. The method may further include determining an estimate of the user's audiogram based on the user hearing threshold data, the estimate of the calibration data, and the normal hearing data.

[0042] In some embodiments, the user hearing threshold data, the first sample hearing threshold data, and the second sample hearing threshold data may each indicate a hearing threshold for one or more frequencies and for one or both ears.

[0043] In some embodiments, the user hearing threshold data may indicate hearing thresholds for a plurality of frequencies for a left ear and a right ear.Similar statements may apply to the first sample hearing threshold data and the second sample hearing threshold data.

[0044] In some embodiments, the hearing threshold of the user hearing threshold data may be represented by a digital signal level of a playback device.

[0045] In some embodiments, the second sample hearing threshold data may indicate information about an audiogram of the second individual sample set. Additionally or alternatively, the second sample hearing threshold data may indicate a mean of the audiogram of the second individual sample set. Additionally or alternatively, the second sample hearing threshold data may indicate a mean of the hearing thresholds for the corresponding frequencies and ears of the second individual sample set.

[0046] In some embodiments, the second individual sample set may be selected based on at least one user attribute of the user.

[0047] In some embodiments, determining the estimate of the audiogram may be performed at the media playback device or at a server device in communication with the media playback device.

[0048] In some embodiments, the method may further include obtaining updated user hearing threshold data for a user for a second media playback device different from the media playback device, the updated user hearing threshold data indicating an updated hearing threshold for a given frequency. The method may further include determining a deviation between the user hearing threshold at a given frequency as indicated by the user hearing threshold data and the updated hearing threshold. The method may further include determining an estimated value of second calibration data for the second media playback device based on the estimated value of the calibration data and the determined deviation.

[0049] In some embodiments, the method may further include obtaining updated user hearing threshold data of the user for a second media playback device different from the media playback device, the updated user hearing threshold data indicating updated hearing thresholds for a plurality of given frequencies. The method may further include determining an estimate of a deviation between the calibration data and second calibration data for the second media playback device based on the user hearing threshold at a plurality of given frequencies as indicated by the user hearing threshold data, the updated hearing threshold, the second sample hearing threshold data, and the sample noise data. Wherein, the sample noise data may indicate the variability of the user response in the process of measuring the hearing threshold. The method may further include determining an estimate of the second calibration data based on the estimate of the calibration data and the determined estimate of the deviation.

[0050] In some embodiments, determining the estimated value of the second calibration data may be based on a Bayesian Maximum A Posteriori MAP estimation technique.

[0051] Aspects of the present disclosure may be implemented via a computing device. The device may include at least one processor and a memory coupled to the processor. The processor may be adapted to perform methods according to aspects and embodiments of the present disclosure. For example, the memory may store instructions that, when executed by at least one processor, cause the computing device to perform methods according to aspects and embodiments of the present disclosure.

[0052] Aspects of the present disclosure may be implemented via a computer program. When the instructions of the computer program are executed by a processor (or computing device), the processor may execute aspects and embodiments of the present disclosure. A computer-readable storage medium may store a program. Such a computer-readable storage medium may include a memory device, such as those described herein, including but not limited to a random access memory (RAM) device, a read-only memory (ROM) device, etc. Therefore, some innovative aspects of the subject matter described in the present disclosure may be implemented via one or more computer-readable storage media having software stored thereon.

[0053] It should be noted that the methods and systems as outlined in this disclosure (including preferred embodiments thereof) can be used independently or in combination with other methods and systems disclosed in this document. In addition, all aspects of the methods and systems outlined in this disclosure can be combined arbitrarily. In particular, the features of the claims can be combined with each other in any manner.

[0054] It should be understood that device features and method steps can be interchanged in a variety of ways. In particular, the details of the (multiple) disclosed methods can be implemented by the corresponding device (or system), and vice versa, as will be appreciated by the skilled person. In addition, any statement in the above statements made about the (multiple) methods is understood to be equally applicable to the corresponding device (or system), and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Example embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0056] Figure 1 An example of an audiogram estimation system is schematically illustrated;

[0057] Figure 2 An example of a hearing loss compensation system is schematically illustrated;

[0058] Figure 3 schematically illustrates an example of an audiogram estimation system according to an embodiment of the present disclosure;

[0059] Figure 4 is a graph showing examples of average population headphone frequency responses and average population audiograms;

[0060] Figure 5 is a flow chart illustrating an example of a method of estimating an audiogram of a user of a media playback device according to an embodiment of the present disclosure;

[0061] Figure 6 is a flowchart illustrating an example of a method of estimating calibration data for a media playback device according to an embodiment of the present disclosure;

[0062] Figure 7 schematically illustrates an example of a system for determining an audiogram using a cloud-based database infrastructure according to an embodiment of the present disclosure;

[0063] Figure 8 is a flow chart illustrating another example of a method of estimating calibration data for a media playback device according to an embodiment of the present disclosure;

[0064] Figures 9 to 12 is a chart showing examples of audiograms using different techniques according to an embodiment of the present disclosure; and

[0065] Fig.13 is a block diagram schematically illustrating an example of a computing device for executing a method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0066] In view of the above needs, to address the challenges outlined above, and to significantly improve the accuracy of audiogram estimation in consumer settings, techniques and systems for estimation and / or application of audiograms for hearing loss compensation are described herein.

[0067] Broadly speaking, these techniques and systems combine threshold measurements with one or more additional probabilistic information sources to reduce errors in the estimated audiogram. These one or more additional information sources may include, for example:

[0068] Thresholds measured at frequencies other than the frequency at which a threshold is measured. It has been found that headphone calibration data as well as audiogram data are not frequency independent. This means that, for example, a threshold measurement at a frequency of 500 Hz also provides some probabilistic information or likelihood about thresholds at other frequencies. Therefore, accuracy can be improved by probabilistically combining threshold data for all frequencies tested, compared to the traditional method of measuring hearing loss at each frequency independently.

[0069] • Thresholds measured at one ear (eg, left ear) provide information on the probability of hearing loss in a different ear (eg, right ear). By combining the threshold data on both ears, a more accurate estimate of the audiogram can be obtained.

[0070] • Demographic data. Combining threshold data with information about demographic groups with similar demographic attributes (eg, gender, age, and / or geographic location) can yield improved estimates of the audiogram.

[0071] · A threshold value measured during another measurement period. The user can repeat one or more measurements at different times or in different environments. By combining such multiple measurements in a probabilistic manner, the accuracy of the resulting audiogram or estimated hearing loss can be improved.

[0072] Thresholds measured with different devices (e.g. different mobile phones and / or different headphones or earbuds). Probabilistic combination of measurements on multiple devices will reduce bias due to specific and often unknown device calibration characteristics, thereby improving the audiogram estimate.

[0073] Thresholds obtained by other users using the same device. Even if calibration data for a specific device (mobile phone and / or earbuds) is not available, the audiogram estimation process can be improved by combining measurement data of several users (preferably users with similar demographics) using the same device.

[0074] Headphone frequency response population information. Even if the headphones or earbuds used during the acquisition of the audiogram are unknown, the statistical properties of the frequency response of a population of headphones or earbuds can provide information about how their frequency responses vary on average and are related to frequency. This can help improve the accuracy of the audiogram estimate.

[0075] Group audiogram information. The availability of audiogram data for a group of consumers can provide information about how the audiograms relate across frequencies and across ears. This allows for more accurate estimation of the audiogram data.

[0076] Therefore, rather than using hearing loss estimates for each frequency and ear independently, the present disclosure proposes techniques that broadly use at least one of multiple measurements at different frequencies and / or at different ears, headphones and / or devices, and / or users, and combine this information with general population characteristics of audiograms and / or headphone frequency responses to calculate a more accurate estimate of hearing loss at each frequency and at each ear.

[0077] Figure 3 Schematically illustrated is an example of an audiogram estimation system 300 for hearing loss estimation. The system 300 may be adapted to estimate the audiogram of a user of a media playback device, for example in a consumer setting.

[0078] For the above system 100, a sound generator 310 generates a signal 315 of a specific frequency, which is reproduced on the headphones to the listener (user) 10. The listener 10 indicates whether the signal is audible (in the form of a subjective response 320) to determine, for example, a threshold sound level 330 in the digital domain, for example expressed as dB reFS (full scale). This is done for one or more frequencies (e.g., for multiple frequencies) and for one or more ears (e.g., for the left ear and the right ear). It should be understood that multiple threshold sound levels are determined. Preferably, the threshold sound levels are determined for multiple frequencies and for the left ear and the right ear. The sound level obtained can be represented by or included in the user hearing threshold data.

[0079] The hearing loss indication 360 of the listener 10 is determined by the statistical optimization block (or module) 350 based on the user hearing threshold data. The process may use normal hearing data indicating the expected hearing threshold when there is no hearing loss. The hearing loss indication 360 may be stored or compiled in the form of an audiogram 370 of the listener 10.

[0080] In accordance with techniques in accordance with the present disclosure, statistical optimization block 350 (probabilistically) combines user hearing threshold data with statistical information 340 (e.g., sample hearing threshold data and at least one of sample calibration data and sample noise data, as described in more detail below) to improve the accuracy of estimated audiogram 370.

[0081] The statistical information 340 can be stored locally (e.g., on a mobile device, on a playback device, etc.) or remotely (e.g., in a distributed system, a cloud service, etc.). Likewise, the optimization process can be performed locally or run as a cloud-based service. Additionally, threshold data generated by the user can contribute to the statistical information collected (e.g., locally or remotely) to improve the process of estimating the audiogram.

[0082] Example Embodiment: Combining Measurements Over Frequency and / or Over the Ear

[0083] It starts with a set of hearing threshold measurements (e.g., implemented, formed by, or included in user hearing threshold data), such as digital signal level (dB reFS) , including (eg, consisting of) threshold measurements for two ears {l, r} and for K frequencies, and is given, for example, by

[0084]

[0085] Generally, it will be understood that the set of measurements comprises at least two measurements associated with different frequencies and / or different ears. Without intending to be limiting, throughout the disclosure reference may be made to measurements for multiple frequencies and for two ears.

[0086] It is assumed that the mobile device and headphones (which may be collectively referred to as playback devices) used during the measurement are unknown, but that statistical information describing the population of headphones and mobile devices commonly used by consumers is accessible. More specifically, it is assumed that the headphone frequency response population information (e.g., in the form of sample calibration data as described in detail below) can be represented by (or composed of) an average frequency response vector representing (or consisting of) the mean of the frequency responses h for K frequencies on two headphones and the frequency response covariance matrix ∑ using the expected value operator <.> h For example, the headphone frequency response population information can be given by

[0087]

[0088] Similarly, it is assumed that a mean and covariance matrix are available for the audiograms of a population of users (e.g., in the form of sample hearing threshold data as described in detail below). Ideally, but not necessarily, the mean vector and covariance matrix can be adjusted for at least one user attribute (e.g., for the user's age and / or gender). The audiogram mean and covariance matrix are respectively given by and∑ a express.

[0089] As an example, the frequency of (Average Group Audiogram) and The value of (average group headphone frequency response) is Figure 4 The graph 410 represents , and the graph 420 represents The value of .

[0090] Finally, assume that the playback level can vary across mobile devices (or playback devices) and can be determined by the variance Alternatively, if repeated measurements are made due to subjective rating variability, the variability of the responses can be described by a zero-mean random variable with variance These variables can be represented by or included in sample noise data, such as described in detail below.

[0091] The absolute threshold in quiet for normal hearing (i.e., no hearing loss) is expressed in dB SPL. express.

[0092] Throughout the disclosure, the term 'calibration data' (e.g., sample calibration data) may be used to describe the conversion from digital signal levels to acoustic sound pressure levels, which includes both the sensitivity and frequency response of the headphones, as well as the effect of the mobile device generating the electrical signal. As described above, the combination of the mobile device (e.g., mobile phone) and the headphones may be referred to as a playback device.

[0093] Given the prior population information introduced above (e.g., sample hearing threshold data), the listener's audiogram can now be estimated based on noise measurement data (e.g., user hearing threshold data) acquired with unknown device calibration data (including frequency response data).

[0094] In particular, for example, the user's audiogram Bayesian Maximum A Posteriori (MAP) estimation of It can be given by the following formula

[0095]

[0096] As described in reference [1], the MAP estimation matrix M is given by:

[0097]

[0098] and where J is an all-ones matrix and I is the identity matrix (with appropriate dimensions).

[0099] As illustrated in equation 5), in determining the estimated value of the audiogram When , the techniques according to the present disclosure are based on the frequency response covariance matrix ∑h (e.g., included in the sample calibration data) and / or the zero-mean random variable and / or (e.g., including sample noise data) to the user's hearing threshold data and the mean vector (eg, included in sample hearing threshold data) applying relative weights.

[0100] Assuming the covariance matrix has non-zero off-diagonal elements, the estimate of the audiogram at frequency k depends not only on the measurement at frequency k, but also on all measurements over frequency and at both ears. Thus, the user's hearing thresholds at different frequencies and / or at different ears are (probabilistically) correlated according to the techniques of the present disclosure to determine an estimate of the listener's audiogram. This correlation can be based on sample hearing threshold data and at least one of sample calibration data and sample noise data.

[0101] Figure 5An example of a method 500 for estimating an audiogram of a user of a media playback device according to the above considerations is illustrated. The method 500 comprises steps S510 to S540, which may be performed at the media playback device or at a server device in communication with the media playback device.

[0102] exist Step S510 At, obtaining (e.g., determining, measuring) user hearing threshold data of the user. This can be as described above with reference to Figure 1 and Figure 3 Completed as described.

[0103] In particular, the user hearing threshold data may indicate a hearing threshold for one or more frequencies and for one or both ears. Preferably, the user hearing threshold data indicates a hearing threshold for a plurality of frequencies for a left ear and a right ear. For example, the user hearing threshold data may indicate a user hearing threshold vector (i.e., a vector of user hearing thresholds, such as the vector defined above). ). Each entry of the user hearing threshold vector may relate to a given frequency and ear. Thus, the user hearing threshold vector may have dimensions (2N F )×1, where N F =K is the number of frequencies.

[0104] As described above, the hearing threshold of the user's hearing threshold data may be represented by a digital signal level (eg, a digital signal level of a playback device).

[0105] As described above, the hearing threshold contained in or represented by the user hearing threshold data can be related to a hearing threshold measurement. The hearing measurement can be performed at the media playback device. For example, the hearing threshold or measurement result can be obtained by performing the following steps A to C, which can be regarded as examples of sub-steps of step S510.

[0106] Step A: Outputting a plurality of audio signals at different frequencies by a media playback device. For example, the plurality of audio signals may have different frequencies, and for each frequency, there may be one such signal for each ear. In addition, for each frequency and ear, different levels of audio signals may be output to determine a threshold of sound pressure level, above which the audio signal is audible to the user.

[0107] Step B: Receiving a user input in response to the output audio signal. The user input may be related to a user's subjective response as to whether the user can hear the corresponding audio output audio signal.

[0108] Step C: Generate user hearing threshold data based on the received user input.

[0109] exist Step S520At, sample hearing threshold data is obtained. The sample hearing threshold data may be statistical data indicating the hearing threshold of the individual sample set. Therefore, the sample hearing threshold data may indicate or be related to the hearing threshold of the pre-stored individual sample set.

[0110] Further, the sample hearing threshold data may indicate information about the audiogram of the individual sample set. Specifically, the sample hearing threshold data may indicate the audiogram of the individual sample set (e.g., ∑ a ) (e.g., as defined above ) and variance (e.g., covariance). The mean and variance (e.g., covariance) may be associated with the hearing thresholds for the respective frequencies and ears of the individual sample set. Thus, the sample hearing threshold data may indicate the mean and variance (e.g., covariance) of a hearing threshold vector (i.e., a vector of hearing thresholds) for the individual sample set. Each entry of the sample hearing threshold vector may relate to a given frequency and ear. Thus, the sample hearing threshold vector may have a dimension (2N F )×1, where N F =K is the number of frequencies.

[0111] It should be further understood that the individual sample set may be selected (e.g., compiled) based on at least one user attribute of the user. For example, the individual sample set may be selected based on at least one of the user's age, gender, or place of residence. The user attribute may be derived from user input (e.g., data provided by other users) or from user-related data obtained from an external source.

[0112] exist Step S530 At least one of sample calibration data and sample noise data is obtained.

[0113] Among them, the sample calibration data can be statistical data indicating the frequency response of the media playback device sample set. These frequency responses can be given for the entire playback device (including headphones). In other words, for multiple frequencies, the frequency response can relate the digital signal level to the sound pressure level.

[0114] In some embodiments, the media playback device sample set may be selected (e.g., compiled) based on device attributes, device IDs, etc. of the media playback device. For example, the media playback device sample set may be selected based on the type of headphones (e.g., open, closed, earbuds, cable-based, wireless) used by the user. Further, for example, the media playback device sample set may be selected based on the device type or ID or device brand. In general, any known property or attribute of the media playback device may be used to select the media playback device sample set.

[0115] In some embodiments, the sample calibration data may indicate a mean and variance (e.g., covariance) of the frequency response of the media playback device sample set. Specifically, the sample calibration data may indicate a mean and variance (e.g., covariance) of a frequency response vector (i.e., a vector of frequency responses) of the media playback device sample set. Each entry of the frequency response vector may relate to a given frequency and ear. For example, the frequency response vector may have dimensions (2N F )×1, where N F = K is the number of frequencies. The mean (e.g., defined above) ) can be represented by a vector of dimension (2N F )×1, where each entry represents the mean of the frequency response of the corresponding frequency-ear pair. h ) can be represented by a matrix of dimension (2N F )×(2N F )express.

[0116] Further, the sample noise data may be statistical data indicating the variability of the user's response during repeated measurements of hearing thresholds (e.g., the above-defined ) and / or statistics indicating the variability of the playback level (e.g., as defined above express).

[0117] The media playback device sample set may include a plurality of different media playback devices that a user may use to estimate their audiogram in a consumer setting. It should be understood that the different media playback devices may involve different combinations of media players, mobile phones, PDAs, handheld devices, etc., and headphones, earbuds, etc.

[0118] exist Step S540 At the step of determining (e.g., calculating) an estimate of the user's audiogram based on the user hearing threshold data, the sample hearing threshold data, and at least one of the sample calibration data and the sample noise data. It should be understood that determining the estimate of the audiogram may be further based on normal hearing data indicating an expected hearing threshold in the absence of hearing loss (e.g., ). For example, the expected hearing threshold may be subtracted from the measured threshold of the user's hearing threshold data to determine an estimate of the user's hearing loss.

[0119] Further, as indicated above, determining the estimate of the audiogram at step S540 may involve applying relative weights to the user hearing threshold data and the sample hearing threshold data based on at least one of the sample calibration data and the sample noise data. The relative weights may be determined such that for better measurement conditions (e.g., uncertainty in device calibration (e.g., ∑ h) is small and / or the influence of noise (e.g., the and / or ) is smaller), the audiogram estimate is closer to the user's measured hearing threshold (possibly obtained through frequency response and other calibration data (e.g., ) correction) and, for poor measurement conditions, is closer to the group average audiogram (e.g., ).

[0120] Further, step S540 may (probabilistically) correlate the user's hearing thresholds at different frequencies and / or for different ears (preferably, at different frequencies and for both ears) to determine an estimate of the listener's audiogram. This correlation may be based on sample hearing threshold data and at least one of sample calibration data and sample noise data.

[0121] For example, the estimated value of the audiogram determined at step S540 may be based on a Bayesian maximum a posteriori (MAP) estimation technique, which introduces the aforementioned correlation between the user's hearing thresholds at different frequencies and / or for different ears. Referring to the example of equations (4) and (5), this correlation is introduced by the off-diagonal elements of the estimation matrix M. For the non-trivial matrix ∑ h and / or The off-diagonal elements will be present.

[0122] When using the MAP estimation technique, the estimated value of the audiogram It can be given by equation (4), that is in is optional, and where M = ∑ a (∑ a +E) -1 Here, E includes ∑ h , and As defined above, ∑ a is the covariance of the vector representation of the sample hearing threshold data, is the mean of the vector representation of the sample hearing threshold data, ∑ h is the covariance of the vector representation of the sample calibration data, is the mean of the vector representation of the sample calibration data, represents the variability of the user's response in measuring hearing thresholds, represents the variability of the playback level, I is (dimension (2N F )×(2N F ) of the identity matrix, J is (dimension (2N F )×(2N F) of an all-one matrix, and wherein the vector representation includes one of the left ear and the right ear and a predetermined set of N F =The corresponding elements of each pair consisting of frequencies in K frequencies.

[0123] Despite Figure 5 Not shown, but method 500 may further include the step of outputting the determined estimate of the audiogram and / or a set of compensation gains associated with the determined estimate of the audiogram, for example to an application capable of audio playback on a media playback device, or for transmission to another device.

[0124] Table 1 shows example results of the performance (ie, root mean square error) of audiogram estimation in dB using four different methods:

[0125] · “No measurement” assumes no hearing loss at all (e.g. ).

[0126] · "Group mean" uses the group mean as the audiogram (e.g. )

[0127] "Measurement Results" use the measured data directly as an audiogram (e.g. )

[0128] "Bayesian MAP" estimation (e.g., ).

[0129] The numbers in Table 1 represent the mean error in dB, where lower values ​​are better. It can be seen that the Bayesian MAP estimator provides a more accurate estimate of the audiogram than any of the other methods included.

[0130] Age group (years) No measurement results Group average Measurement results Bayesian MAP| 20-40 14.8 11.1 7.2 5.4 40-60 24.9 15.9 7.2 5.7 60-80 39.2 21.2 7.2 5.8

[0131] Table 1 - Performance of various audiogram estimation techniques

[0132] For these examples, it is assumed that no a priori information is available about the type of headphones used during the measurements. If some information about the type of headphones is available, the The values ​​of ∑h and ∑h can improve the accuracy. For example, different statistical population values ​​can be used for circumaural and supra-aural headphones, earplugs, etc.

[0133] It should be noted that, without loss of generality, a frequency index k may represent a different measurement frequency, but may equally refer to repeated measurements at the same frequency, or a mixture of both. As long as all mean and covariance matrices are properly determined for the frequencies associated with measurement index k (i.e., as long as a consistent assignment of vector elements to frequency-ear pairs is used), the Bayesian MAP estimator will work regardless of which frequencies are associated with index k.

[0134] Example Embodiment: Hearing Optimization Using Estimated Audiograms

[0135] This embodiment can be easily combined with the above-described embodiment(s) and implementations.

[0136] The user's estimated audiogram can be used to enhance audio playback for the user by compensating for individual hearing loss. Figure 2 For example, a method for enhancing audio playback that can be performed in conjunction with the above method 500 or as an independent method can include the following steps A to D.

[0137] Step A: Receiving audio data for playback at a user's media playback device.

[0138] Step B: Determine (e.g., calculate) a set of compensation gains based on the determined estimate of the audiogram. Determining the set of compensation gains may be further based on the received audio data. For example, the compensation gains may be applied in a multi-band compression framework where the gains depend on the audiogram and the audio signal level in each frequency band.

[0139] Step C: generating hearing-optimized audio data by applying the determined set of compensation gains to the audio data (or an audio signal derived therefrom).

[0140] Step D: Rendering the hearing-optimized audio data for playback to the user.

[0141] Example Embodiment: Use of Population Statistics

[0142] Equipment calibration data estimation

[0143] This embodiment can be easily combined with the above-described embodiment(s) and implementations.

[0144] Assume that a large group of users have gone through the process of determining audibility thresholds (hearing thresholds), resulting in a threshold vector represented by digital levels for known device IDs denoted by D Database:

[0145]

[0146] These measured thresholds The statistical signal model of is now composed of the unknown device frequency response of device D Noise or variability in subjective responses Normal hearing threshold and the actual audiogram to be estimated The combination of gives,

[0147]

[0148] If the system has acquired a sufficiently large number of thresholds among users for a given device D And assuming that the acquisition of noise, audiogram, and calibration data are independent, the above equation can be rewritten in terms of expected values ​​using the expected value operator <.),

[0149]

[0150] Therefore, the estimated device calibration data can be obtained via the following formula

[0151]

[0152] In other words, the device calibration data All measurements performed by many users with the same device D can be evaluated and calculated The mean of the audiograms of the combined groups and normal hearing threshold The device calibration data can then be used when estimating the user's (or any user's) audiogram using a media playback device belonging to device type D.

[0153] Therefore, based on the measured threshold for a given user Audiogram estimate For example, given by the following formula

[0154]

[0155] Figure 6 An example of a method 600 for estimating calibration data for a media playback device based on the above considerations is illustrated. The calibration data may indicate a frequency response of the media playback device. The method 600 includes steps S610 to S630, which may be performed at the media playback device or at a server device in communication with the media playback device.

[0156] exist Step S610At the first sample hearing threshold data, first sample hearing threshold data is obtained (e.g., retrieved from a database). Here, the first sample hearing threshold data may indicate the hearing threshold of the first individual sample set and be associated with the given device type. Therefore, in one embodiment, the first sample hearing threshold data may, for example, correspond to the (large) number (e.g., a set) of thresholds among users of the given device D as defined above. or corresponding to its expected value

[0157] exist Step S620 At , second sample hearing threshold data is obtained (e.g., retrieved from a database). The second sample hearing threshold data may indicate the hearing threshold of a second individual sample set that is different from the first individual sample set. The second sample hearing data may not be associated with a given single device type. For example, the second sample hearing data may include a plurality of subsets of sample hearing data, each subset being associated with a corresponding device type. For example, the second sample hearing threshold data may indicate information about an audiogram of the second individual sample set. Further, the second sample hearing threshold data may indicate a mean of the audiogram of the second individual sample set, or in other words, indicate a mean of the hearing thresholds of the second individual sample set for corresponding frequencies and ears. Therefore, in one embodiment, the second sample hearing threshold data may, for example, correspond to a population average audiogram as defined above.

[0158] Here too, the second individual sample set may be selected based on at least one user attribute of the user. This may be done, for example, similarly to the selection described above in the context of step S520 of method 500 .

[0159] In step S630 At, based on the first sample hearing threshold data (for example, ), second sample hearing threshold data (eg, ) and normal hearing data indicating the expected hearing threshold in the absence of hearing loss (e.g., ) to determine calibration data (e.g., ) is an estimated value.

[0160] In the above, similar to the explanation above in the context of method 500, the user hearing threshold data, the first sample hearing threshold data and the second sample hearing threshold data may each indicate a hearing threshold for one or more frequencies and for one or both ears. Preferably, the user hearing threshold data indicates the hearing threshold for multiple frequencies for the left ear and the right ear. In this case, the other data defined above will also relate to multiple frequencies and the left ear and the right ear.

[0161] In the calibration data of the media playback device (for example, ) is available, the method may further include (not shown in the figure) obtaining user hearing threshold data of a user of the media playback device (e.g., ) (e.g., represented by a digital signal level of a playback device), and determining an audiogram of the user (e.g., ). The former of these steps may be performed similarly to step S510 of method 500 described above, and the latter of these steps may be performed similarly to step S540 of method 500. In some embodiments, determining the user's audiogram (e.g., ) can use equation (10) defined above or equations (13) and (14) defined below.

[0162] Further, as described above, determining an estimate of the audiogram may be performed at the media playback device or at a server device in communication with the media playback device.

[0163] Iterative improvements and cloud-based services for hearing loss compensation

[0164] The above embodiments may be implemented in a cloud-based manner.

[0165] Figure 7 A schematic overview of an example system 700 for determining an audiogram using a cloud-based database infrastructure is shown in FIG. A user 710, for example, using a mobile app 710 (e.g., running on a playback device), determines a digital threshold 730. This may be based on a user's subjective response 720 to an audio signal 715 generated by the playback device in the same manner as described above in Figure 3 The mobile app 710 sends the threshold 730 and corresponding device and / or user information 780 (e.g., device ID(s), user ID, age and / or gender, etc.) to a central, cloud-based server 790 that collects data from multiple users. The cloud-based service can then estimate the device characteristics 740 of device D and the user's audiogram 770, as outlined above.

[0166] In particular, this architecture / method can be combined with the method described in the embodiment of method 500 (Example embodiment: combining cross-frequency and / or cross-ear measurements). For example, when the service has not collected many audiograms or the user is using a new unknown device, the system can apply the method described in the previous embodiment to calculate the initial audiogram of the specific user via the following formula:

[0167]

[0168] Using the initial prediction matrix

[0169]

[0170] Over time, if a sufficient number of users have had their audiograms measured using the same device D, the system can use the estimated device frequency response to update the user's audiogram, rather than having to rely on the group average frequency response

[0171]

[0172] Using the updated prediction matrix

[0173]

[0174] Whenever the system determines a more accurate estimate of the audiogram data or device characteristics, the resulting improved audiogram and device characteristics / calibration data can be sent from the cloud-based service to the mobile device (playback device) or to a server in communication with the mobile device (playback device) to improve the hearing loss compensation algorithm by using the more accurate data.

[0175] Clustering of device data

[0176] The process of estimating device calibration data can be performed by using two or more devices D to calculate Further refinement can be achieved by ensuring that two or more devices have similar characteristics. Clustering can be applied to devices with similar characteristics, for example using k-means or multivariate Gaussian mixture models or other clustering techniques based on the similarity of the measurement data obtained for the devices. In addition, the type of headphone (earbud, over-ear or supra-aural headphone) can be used to improve the clustering process.

[0177] Example Embodiment: User Switches to a Different Headset

[0178] This example embodiment considers that the user has completed the registration and audiogram measurement on the headset D1 (or in general, the device configuration D1) to generate the threshold data And a use case where the same process is now being used on a different headset D2 (or, in general, a device configuration D2). Therefore, in this case, the calibration data associated with the device has not changed, but the calibration data associated with the headset may have changed. Therefore, in order for the user to obtain the best hearing loss compensation process for the headset D2, new calibration data needs to be obtained. One way is to use a method as described in a previous example embodiment (Example embodiment: use of population statistics), where a cloud-based system is able to provide calibration data for the new headset D2.

[0179] Alternatively, the user may undergo simplified testing with a limited number of frequencies for which new thresholds will be evaluated from which new calibration data will be derived. For example, the threshold for only one frequency may be measured using headset D2. D1，l，k 、x D2，l，k denotes the measured thresholds for a particular user l and frequency k of headphones D1 and D2, respectively, and assuming that the device calibration data of headphones D1 and D2 differ only by a scalar d in the decibel domain, the calibration difference scalar can be calculated from x, for example, via D1，l，k With x D2，l，k The threshold difference between

[0180]

[0181] If multiple measurements are performed on the headset D2, a Bayesian MAP estimator can be used to calculate an estimate of the scalar d, for example via

[0182]

[0183] in,

[0184]

[0185] Therefore, when the measurement is composed of the covariance matrix ∑a and the subjective measurement (repeatability) noise This approach will take into account the correlation between the tested frequencies when expressing the D2 threshold for the headphone.

[0186] Figure 8 An example of a method 800 for estimating calibration data for a media playback device based on the above considerations is illustrated. The calibration data may indicate a frequency response of a media playback device (including a particular headphone). The method 800 includes steps S810 to S830, which may be performed at the media playback device or at a server device in communication with the media playback device. Steps S810 to S830 of the method 800 may be performed after the steps of method 600, or as a separate method.

[0187] exist Step S810 At , updated user hearing threshold data of the user is obtained for a second media playback device (e.g., a media playback device with headphones D2) that is different from the media playback device (e.g., a media playback device with headphones D1). The updated user hearing threshold data may indicate the user's hearing threshold for a given frequency (e.g., x D2，l，k )’s updated hearing threshold.

[0188] exist Step S820At the user hearing threshold at a given frequency as indicated by the user hearing threshold data (e.g., x D1，l，k ) and the updated hearing threshold (e.g., x D2，l，k ) between them (e.g., a scalar deviation d).

[0189] exist Step S830 Based on calibration data (e.g. ) and a determined deviation (e.g., a scalar deviation d) - for example, by adding the determined deviation to the estimated value of the calibration data - to determine an estimated value of second calibration data for the second media playback device.

[0190] In addition, the updated user hearing threshold data at step S810 may indicate updated hearing thresholds for a plurality of given frequencies. Then, at step S820, an estimate of the deviation between the calibration data and the second calibration data of the second media playback device may be determined based on the user hearing threshold at a plurality of given frequencies as indicated by the user hearing threshold data, the updated hearing threshold, the second sample hearing threshold data, and the sample noise data. The sample noise data may indicate the variability of the user response in the process of measuring the hearing threshold. Finally, at step S830, an estimate of the second calibration data may be determined again based on the estimate of the calibration data and the determined estimate of the deviation. Determining the estimate of the second calibration data may be based on a Bayesian MAP estimation technique, as described above.

[0191] Revise

[0192] Although the techniques described throughout the disclosure use least mean squares and Bayesian inference techniques to improve the accuracy of audiograms and derive device calibration data, other methods may also be used. Such other methods may include, but are not limited to, for example: fuzzy logic, Dempster-Schafer theory, imprecise probability, machine learning, k-means clustering, deep learning, neural networks, etc.

[0193] It should be understood that the transmission, storage and use of personal data such as audibility thresholds, device IDs, demographic data, etc. can or should be appropriately protected by dedicated privacy protection and security mechanisms.

[0194] The Bayesian inference methods described throughout the disclosure are based on Gaussian distributions, resulting in closed-form solutions. It should be noted that for more complex distributions, and for data with unknown noise hyperparameters, various iterative algorithms exist for Bayesian MAP estimation under such conditions. Details are described, for example, in reference [1].

[0195] Additional Notes: Calibration Inference for Hearing Optimization

[0196] question

[0197] Part of the hearing optimization solution is a recording step in which the hearing level (HL) at different frequencies (in some embodiments, for example, 11 or 17 data points) is measured to produce an audiogram. Traditional audiogram testing requires calibrated equipment to reproduce the test tones at a specific level, so in order to reproduce the test with a device such as a mobile phone, the response characteristics of both the headset and the mobile phone must be known (i.e., device-specific calibration data needs to be known). In the context of the present disclosure, "calibration" can refer to (1) the sound pressure level (SPL) produced by the mobile phone and headset (e.g., collectively referred to as the playback device) for a given signal level (e.g., as specified at the OS / application level) and (2) the frequency response of the mobile phone and headset. It is not feasible to measure every mobile phone and headset on the market, so the solution must be calibration-free.

[0198] It is worth noting that the audiogram does not include any aspect of device calibration, but only includes the user's measured hearing level. In this disclosure, the test result including device calibration will be referred to as "audiogram + calibration".

[0199] Solution

[0200] Typically, once a mobile phone or headset is changed to a different model, the full audiogram + calibration test must be repeated, as described below. However, the techniques outlined in this disclosure relate to an optimized process that can be done without calibration, as long as the device used is the same as the device on which the test was performed.

[0201] Fig. 9 is a graph showing an example of a comparison of an audiogram of a known device (graph 910) and an audiogram+calibration of a new device (graph 920) as a function of frequency.

[0202] As will be appreciated by the skilled person, the following solutions S1 to S4 may relate to the above-described example embodiments and implementations thereof.

[0203] S1. Solution for SPL calibration: When a change to an unknown device is detected, a simplified test can be performed instead of performing a full "Audiogram + Calibration" test. By testing at a single frequency tested on a known device, a calibration deviation between the known device and the new device can be generated. This allows us to use a previous "Audiogram + Calibration" for the new device. Information identifying the device (such as the Bluetooth device ID) can be used to identify the device to which the updated test is applied when available.

[0204] Fig.10is a graph showing an example of a comparison of an audiogram of a known device (graph 1010) and an audiogram of a new device recalibrated with SPL + calibration (graph 1020) as a function of frequency.

[0205] S2. Solution for separating SPL calibration and audiogram: The method in S1 does not allow the "audiogram + calibration" result to be separated into headphone + mobile calibration and audiogram (i.e., it is not possible to derive separate device pair compensation data and audiogram data from S1). To this end, a Bayesian inference model can be constructed to enable the model output to be estimated incrementally using update rules as more information becomes available. As more and more users switch between devices and perform update tests, the Bayesian inference model incorporates information for updating the SPL calibration of a given device. This information can be propagated to users who have not or have not yet performed an updated test.

[0206] S3. Solution for SPL calibration and frequency response: The method in S2 allows separation of SPL calibration and audiogram + device frequency response. However, there is not enough information to separate the device frequency response from the audiogram. Instead of performing a single point test as in S1, two frequency points can be tested. This now allows the model to build both the SPL calibration and the frequency response because now as input to the model, the difference in SPL calibration between the two devices as well as the test and two different frequency points are available. An inference model can be built to separate the frequency response characteristics of the phone and the headset.

[0207] Fig.11 is a graph showing an example of a comparison of an audiogram of a known device (graph 1110), an audiogram of a new device recalibrated with SPL and response + calibration (graph 1120), and device response (graph 1130) over frequency.

[0208] S4. Solutions for SPL calibration, frequency response, and device characteristics variation: Some devices do not have consistent SPL or frequency response calibration across manufacturing runs. To address this, a k-means clustering step can be introduced to group similar devices together. The Bayesian inference model can use the information it has to understand which cluster the user's device belongs to in order to speed up onboarding.

[0209] Fig.12 is a graph showing an example of a comparison of an audiogram of a known device (graph 1210), an audiogram of a new device recalibrated with SPL and response + calibration (graph 1220), the device response (graph 1230), and a variant of the device response (graph 1230) as a function of frequency.

[0210] Initial Entry

[0211] A complete audiogram test is performed (e.g., all frequencies are tested for both the left and right ear (L&R)). The measured hearing level is then split into the SPL calibration, and for each frequency into an audiogram value (hearing level) and a device response value. A probability (e.g., Gaussian or other) distribution is assigned to each value. If the device is known, a known response and SPL can be used. In this case, the associated probability distribution may be narrow. If the device is unknown, the device response may be flat and a default SPL value may be used. The associated probability distribution is wider, making it more likely that an update will change the split between the audiogram and the calibration.

[0212] Using Bayesian Inference to Update Audiograms and Calibration

[0213] The recording establishes the audiogram and device calibration and the associated confidence. When the user switches to a new device (e.g., a mobile device and / or headphones), a subset of the frequency test points must be re-evaluated. As with the recording, the device characteristics may or may not be known. If they are known, there is a high confidence in the device calibration, and if they are not known, there is a low confidence (wide probability distribution). Bayesian inference can be used to update the probability distribution of the audiogram and calibration points for both the old and new devices. As more points are tested, this allows the model to become more certain about which portion of the test value should be assigned to the old / new device calibration and what should be assigned to the audiogram. It is worth noting that the probability distribution of the device calibration can be updated based on data from many users, allowing the device characteristics to be updated faster.

[0214] Device Clustering

[0215] As more users perform update tests, the model will be able to see trends in the data for the same device model identifier. Clustering within device identifiers allows the model to infer device variations caused by manufacturing differences or other reasons. As users perform more test updates, the model can become more certain of which cluster their device belongs to, allowing for more accurate updates to device characteristics.

[0216] advantage

[0217] In addition to any of the advantages described above, the technology according to the present disclosure may have the following advantages:

[0218] Calibration (e.g., measuring each device SPL output and frequency response) is no longer required. The proposed technology can operate on any known or unknown ear device. This simplifies user enrollment and the development of hearing-optimized solutions.

[0219] The proposed technique can easily construct libraries for device responses and calibration.

[0220] Device for implementing the method according to the present disclosure

[0221] Finally, although methods according to embodiments of the present disclosure have been described above, the present disclosure is also directed to an apparatus (e.g., a computer-implemented apparatus or computing apparatus, such as a playback device or server apparatus) for performing the methods and techniques described throughout the disclosure. Fig.13 An example of such an apparatus 1300 is shown. In particular, the apparatus 1300 includes a processor 1310 and a memory 1320 coupled to the processor 1310. The memory 1320 may store instructions for the processor 1310. The processor 1310 may receive, among other things, suitable input data 1330 (e.g., statistical information, hearing thresholds, subjective responses from a user, etc.) depending on the use case and / or implementation. The processor 1310 may be adapted to perform the methods / techniques described throughout the disclosure (e.g., Figure 5 Method 500, Figure 6 Method 600, Figure 8 ), and generates corresponding output data 1340 (e.g., an estimate of the user's audiogram, a reproduced audio signal compensated for hearing loss, etc.) depending on the use case and / or implementation.

[0222] It should be understood that the present disclosure further relates to a corresponding computer program and a computer-readable storage medium storing such a computer program.

[0223] explain

[0224] Aspects of the systems described herein can be implemented in an appropriate computer-based processing network environment (e.g., a stand-alone playback device, a server, or a cloud environment) for processing digital data. Portions of the system may include one or more networks comprising any desired number of independent machines including one or more routers (not shown) for buffering and routing data transmitted between computers. Such networks may be constructed on a variety of different network protocols and may be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof.

[0225] One or more components, blocks, processes or other functional components may be implemented by a computer program executed by a processor-based computing device that controls the system. It should also be noted that any number of combinations of hardware, firmware and / or data and / or instructions embodied in various machine-readable or computer-readable media may be used to describe the various functions disclosed herein from the perspective of their behavior, register transfers, logic components and / or other features. Computer-readable media that may embody such formatted data and / or instructions include, but are not limited to, various forms of physical (non-transitory), non-volatile storage media, such as optical, magnetic or semiconductor storage media.

[0226] In particular, it should be understood that embodiments may include hardware, software, and electronic components or modules, and for discussion purposes, may be illustrated and described as if most components were implemented solely in hardware. However, one of ordinary skill in the art, and based on a reading of this detailed description, will recognize that in at least one embodiment, the electronic-based aspects may be implemented in software (e.g., stored on a non-transitory computer-readable medium) that is executable by one or more electronic processors, such as a microprocessor and / or an application specific integrated circuit ("ASIC"). Therefore, it should be noted that embodiments may be implemented using a plurality of hardware and software-based devices, as well as a plurality of different structural components. For example, in the above Figure 1 , Figure 2 , Fig.13 , Figure 7 or Fig.13 The systems, blocks, or modules described in the context of may include one or more electronic processors, one or more computer-readable media modules, one or more input / output interfaces, and various connections (e.g., a system bus) connecting the various components.

[0227] Although one or more embodiments have been described by way of example and with respect to specific embodiments, it should be understood that one or more embodiments are not limited to the disclosed embodiments. On the contrary, it is intended to cover various modifications and similar arrangements that are obvious to those skilled in the art. Therefore, the scope of the appended claims should be given the broadest interpretation so as to cover all such modifications and similar arrangements.

[0228] Likewise, it should be understood that the phraseology and terminology used herein is for descriptive purposes and should not be regarded as limiting. The use of "including," "comprising," or "having" and variations thereof are intended to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless otherwise specified or limited, the terms "mounted," "connected," "supported," and "coupled" and variations thereof are used broadly and encompass direct and indirect mountings, connections, supports, and couplings.

[0229] Enumerated example embodiments

[0230] Various aspects and embodiments of the present disclosure may also be understood from the following enumerated example embodiments (EEE), which are not claims.

[0231] EEE1. A method for estimating a hearing threshold of a first user of a media playback device, the method comprising:

[0232] obtaining user hearing threshold data corresponding to a first user (in some embodiments, a set of hearing threshold measurements corresponding to one or more frequencies for one or both ears of the user (e.g., thresholds represented by digital signal levels));

[0233] Obtaining population hardware data corresponding to a plurality of media playback devices (in some embodiments, the hardware data includes data representing a population of listening devices (e.g., headphones, hearing aids, speakers, etc.) and media playback systems (e.g., mobile phones, televisions, etc.), data identifying distribution of listening devices and / or media playback device models, types, brands, etc.);

[0234] obtaining group hearing threshold data corresponding to a plurality of second users (in some embodiments, the hearing threshold data includes data characterizing hearing thresholds of a group of people (eg, a group corresponding to a set of demographic characteristics)); and

[0235] An estimated audiogram for the first user is determined based on: the user hearing threshold data, the population hardware data, the population hearing threshold data, and audiogram data representative of normal hearing (eg, data based on an audiogram indicating no hearing loss).

[0236] EEE2. A method as described in EEE1, wherein the user hearing threshold data includes hearing threshold measurement results expressed in digital signal levels (e.g., dB re FS) for multiple frequencies (e.g., 2 frequencies, 11 frequencies, 17 frequencies, etc.) for two ears (e.g., left and right).

[0237] EEE3. A method as described in any one of EEE1 to EEE2, wherein obtaining the user hearing threshold data comprises:

[0238] At the media playback device:

[0239] outputting a plurality of audio signals (e.g., outputting audio tones or signals corresponding to a plurality of frequencies through headphones connected to the media playback device); and

[0240] A user input corresponding to a respective audio signal of the plurality of audio signals is received (eg, a user input indicating perception of a particular audio signal); and wherein the user hearing threshold data is based on data corresponding to the received user input.

[0241] EEE4. A method as described in any one of EEE1 to EEE3, wherein the user hearing threshold data is obtained from at least one of the following:

[0242] a memory of the media playback device, the memory storing application data; and

[0243] A server is in communication with the media playback device via one or more networks.

[0244] EEE5. A method as described in any of EEE1 to EEE4, wherein the population hardware data includes statistical information describing frequency responses and associated covariances of a population of listening devices and media playback devices (e.g., devices typically used by consumers).

[0245] EEE6. A method as described in any one of EEE1 to EEE5, wherein the group hardware data includes an average frequency response vector corresponding to the mean of frequency responses of multiple frequencies.

[0246] EEE7. A method as described in any one of EEE1 to EEE6, wherein the group hearing threshold data includes statistical information describing an audiogram of a group of people (e.g., an audiogram corresponding to a demographically related group of people or users of a media playback device).

[0247] EEE8. A method as described in any one of EEE1 to EEE7, wherein the group hearing threshold data includes average audiogram data (e.g., average vector) and corresponding covariance data (e.g., covariance vector).

[0248] EEE9. A method as described in EEE8, wherein the group hearing threshold data is adjusted based on at least one of the following: age (e.g., age or age range, age threshold, hearing age, etc.) and gender of the first user.

[0249] EEE10. A method as described in any one of EEE1 to EEE9, wherein obtaining group hearing threshold data includes: receiving user input corresponding to self-identified demographic data (for example, user input indicating age or age range, gender, location, occupation, etc.); and wherein the group hearing threshold data is based on data corresponding to the received user input.

[0250] EEE11. A method as described in any one of EEE1 to EEE10, wherein determining the estimated audiogram is performed according to a Bayesian maximum a posteriori (MAP) estimation technique.

[0251] EEE12. The method of any one of EEE1 to EEE11, wherein determining the estimated audiogram is further based on one or more of the following:

[0252] Data representing playback level variance (e.g., device-to-device variance); and

[0253] Data representing the variability of user responses.

[0254] EEE13. A method as described in any one of EEE1 to EEE12, wherein determining the estimated audiogram is performed by the media playback device.

[0255] EEE14. A method as described in any one of EEE1 to EEE13, wherein determining the estimated audiogram is performed by a server device (e.g., a remote server, a supporting device, etc.) that communicates with the media playback device.

[0256] EEE15. The method as described in any one of EEE1 to EEE14, further comprising:

[0257] receiving audio for playback at the media playback device;

[0258] determining a set of gains based on the estimated audiogram;

[0259] generating hearing-optimized audio by applying the set of gains to the audio for playback; and

[0260] The media playback device is caused to playback the hearing-optimized audio.

[0261] EEE16. The method as described in any one of EEE1 to EEE15 further includes: providing data representing the estimated audiogram or a compensation gain associated with the estimated audiogram to an application (e.g., a media player app, a communication app, a game, etc.) on the media playback device.

[0262] EEE17. The method as described in any one of EEE1 to EEE16 further includes: transmitting data representing the estimated audiogram or the compensation gain associated with the estimated audiogram to another device (e.g., a server or device different from the media playback device).

[0263] EEE18. The method as described in any one of EEE1 to EEE17, further comprising:

[0264] generating a set of personalized compensatory gains based on the estimated audiogram and data representing a personalized head-related transfer function; and

[0265] Personalized hearing compensated audio is generated by applying the set of personalized compensation gains to the audio received for playback.

[0266] EEE19. A method for estimating device calibration of a first media playback device, the method comprising:

[0267] obtaining device hearing threshold data associated with a first plurality of users and device types;

[0268] obtaining group hearing threshold data corresponding to a second plurality of users (in some embodiments, the group hearing threshold data includes data characterizing hearing thresholds of a group of people (e.g., a group corresponding to a set of demographic characteristics, a group representing demographic characteristics of a group of device users or a geographic area, etc.)); and

[0269] Estimated device calibration data is determined based on the device hearing threshold data, the population hearing threshold data, and audiogram data representative of normal hearing (eg, data based on an audiogram indicating no hearing loss).

[0270] EEE20. The method as described in EEE1 further comprises:

[0271] obtaining user hearing threshold data corresponding to the first user (in some embodiments, a set of hearing threshold measurements corresponding to one or more frequencies for one or both ears of the user (e.g., thresholds represented by digital signal levels)); and

[0272] Determining an estimated audiogram of the first user based on:

[0273] The user hearing threshold data;

[0274] the estimated device calibration data, and

[0275] Audiogram data indicative of normal hearing (eg, data based on an audiogram indicating no hearing loss).

[0276] EEE21. A method as described in any one of EEE19 to EEE20, wherein the device hearing threshold data includes multiple hearing threshold vectors associated with device types and representing digital levels of multiple frequencies.

[0277] EEE22. The method of any one of EEE19 to EEE21, wherein the device type is defined in part by at least one of the following:

[0278] Device model (e.g., phone model, such as iPhone 13);

[0279] Bluetooth identification value;

[0280] Equipment brand;

[0281] Device form factor (e.g., mobile phone, hearing aid, VR / AR headset, etc.); and

[0282] Applications (e.g., gaming systems).

[0283] EEE23. A method as described in any one of EEE19 to EEE22, wherein the group hearing threshold data includes statistical information describing an audiogram of a group of users of the media playback device (e.g., an audiogram corresponding to a demographically related user group).

[0284] EEE24. A method as described in any one of EEE19 to EEE23, wherein the group hearing threshold data includes average audiogram data (e.g., average vector) and corresponding covariance (e.g., covariance vector).

[0285] EEE25. A method as described in EEE24, wherein the group hearing threshold data is adjusted based on at least one of the age (e.g., age or age range, hearing age, etc.) and gender of the first user.

[0286] EEE26. A method as described in any one of EEE19 to EEE25, wherein determining the estimated device calibration data is performed by a media playback device.

[0287] EEE27. A method as described in any one of EEE19 to EEE26, wherein determining the estimated device calibration data is performed by a server device (e.g., a remote server (e.g., a remote server, a supporting device, etc.) that communicates with the media playback device.

[0288] EEE28. A method as described in any one of EEE19 to EEE27, wherein device types and data corresponding to the group hearing threshold data are received from multiple second media playback devices.

[0289] EEE29. The method as described in any one of EEE19 to EEE28, further comprising:

[0290] Demographic data (eg, age, gender, user ID, geographic region, etc.) is received from a plurality of second media playback devices in response to user input; and wherein determining the estimated device calibration data is further based on a subset of the received demographic data.

[0291] EEE30. The method as described in any one of EEE19 to EEE29, further comprising:

[0292] Generate updated estimated device calibration data for the second media playback device by:

[0293] obtaining updated user threshold data corresponding to a single frequency (e.g., a threshold generated by a device different from the device used to determine the estimated device calibration data (e.g., a headset)); and

[0294] determining a deviation between a value of the estimated device calibration data corresponding to the single frequency and the updated user threshold data corresponding to the single frequency; and

[0295] The updated estimated device calibration data is determined by applying the bias to data corresponding to each frequency represented in the estimated device calibration data.

[0296] EEE31. The method as described in EEE30 further includes: using a Bayesian MAP estimator to optimize the deviation using additional measurements based on frequencies other than the single frequency.

[0297] EEE32. A computing device comprising:

[0298] at least one processor; and

[0299] A memory storing instructions, which, when executed by the at least one processor, cause the computing device to perform the method as described in any one of EEE1 to EEE31.

[0300] EEE33. A non-transitory computer-readable storage medium storing instructions, which, when executed by a computing device, cause the computing device to perform a method as described in any one of EEE1 to EEE31.

[0301] EEE34. A computer program comprising instructions which, when executed by a computing device, cause the computing device to perform a method as described in any one of EEE1 to EEE31.

[0302] References

[0303] [1] Rasmussen, CE and Williams, CKI (2006). Gaussian processes for machine learning. MIT Press. ISBN 0-262-18253-X.

Claims

1. A method of estimating an audiogram of a user of a media playback device, the method comprising: obtaining user hearing threshold data of the user, wherein the user hearing threshold data indicates a hearing threshold for one or more frequencies and for one or both ears; obtaining sample hearing threshold data, wherein the sample hearing threshold data indicates the hearing threshold of a sample set of individuals; obtaining at least one of sample calibration data and sample noise data, wherein the sample calibration data indicates a frequency response of a media playback device sample set, and wherein the sample noise data indicates variability in user response and / or variability in playback level in measuring hearing thresholds; and An estimate of the audiogram of the user is determined based on the user hearing threshold data, the sample hearing threshold data, and the at least one of the sample calibration data and the sample noise data.

2. The method according to claim 1, wherein: Determining the estimate of the audiogram is further based on normal hearing data indicating expected hearing thresholds in the absence of hearing loss.

3. The method according to claim 1 or 2, wherein: Determining the estimate of the audiogram involves applying relative weights to the user hearing threshold data and the sample hearing threshold data based on the at least one of the sample calibration data and the sample noise data.

4. The method according to any one of claims 1 to 3, wherein: Determining the estimated value of the audiogram is based on a Bayesian Maximum A Posteriori (MAP) estimation technique.

5. The method according to any one of claims 1 to 4, wherein: The user hearing threshold data indicates hearing thresholds for a left ear and a right ear for a plurality of frequencies.

6. The method according to any one of claims 1 to 5, wherein: The hearing threshold of the user hearing threshold data is represented by a digital signal level of the playback device.

7. The method according to any one of claims 1 to 6, wherein: Obtaining the user hearing threshold data includes: Outputting a plurality of audio signals at different frequencies by the media playback device; receiving user input in response to the outputted audio signal; and The user hearing threshold data is generated based on the received user input.

8. The method according to any one of claims 1 to 7, wherein: The sample hearing threshold data indicates information about the audiogram of the individual sample set.

9. The method according to any one of claims 1 to 8, wherein: The sample hearing threshold data indicates a mean and covariance of the audiograms of the sample set of individuals.

10. The method according to any one of claims 1 to 9, wherein: The sample hearing threshold data indicates the mean and covariance of the hearing thresholds for the respective frequencies and ears of the individual sample set.

11. The method according to any one of claims 1 to 10, wherein: The sample calibration data indicates a mean and covariance of a frequency response of the media playback device sample set.

12. The method according to any one of claims 1 to 11, wherein: The estimated value of the audiogram Given by the following formula Among them, μ h is optional, and where M=∑ a (∑ a +E) -1 , Among them, E includes ∑ h , and At least one of a is the covariance of the vector representation of the sample hearing threshold data, is the mean of the vector representation of the sample hearing threshold data, ∑ h is the covariance of the vector representation of the sample calibration data, is the mean of the vector representation of the sample calibration data, representing said variability of the user's response during said process of measuring the hearing threshold, Representing the variability of playback level, I is an identity matrix, J is an all-ones matrix, and wherein the vector representation includes corresponding elements for each pair consisting of one of the left ear and the right ear and a frequency from a predetermined set of frequencies.

13. The method according to any one of claims 1 to 12, wherein: The individual sample set is selected based on at least one user attribute of the user.

14. The method according to any one of claims 1 to 13, wherein: Determining the estimate of the audiogram is performed at the media playback device or at a server device in communication with the media playback device.

15. The method according to any one of claims 1 to 14, further comprising: receiving audio data for playback at the media playback device; determining a set of compensation gains based on the determined estimate of the audiogram; generating hearing-optimized audio data by applying the determined set of compensation gains to the audio data; as well as The hearing-optimized audio data is rendered for playback.

16. The method according to claim 15, wherein: Determining the set of compensation gains is further based on the received audio data.

17. A method for estimating calibration data for a media playback device, wherein: The calibration data indicates a frequency response of the media playback device, the method comprising: obtaining first sample hearing threshold data, wherein the first sample hearing threshold data indicates a hearing threshold of a first individual sample set and is associated with a given device type; obtaining second sample hearing threshold data, wherein the second sample hearing threshold data indicates a hearing threshold of a second individual sample set different from the first individual sample set; and An estimate of the calibration data is determined based on the first sample hearing threshold data, the second sample hearing threshold data, and normal hearing data indicating an expected hearing threshold in the absence of hearing loss.

18. The method according to claim 17, further comprising: obtaining user hearing threshold data of a user of the media playback device; as well as An estimate of an audiogram of the user is determined based on the user hearing threshold data, the estimate of the calibration data, and the normal hearing data.

19. The method according to claim 18, wherein: The user hearing threshold data, the first sample hearing threshold data, and the second sample hearing threshold data each indicate a hearing threshold for one or more frequencies and for one or both ears.

20. The method according to any one of claims 18 to 19, wherein: The user hearing threshold data indicates hearing thresholds for a left ear and a right ear for a plurality of frequencies.

21. The method according to any one of claims 18 to 20, wherein: The hearing threshold of the user hearing threshold data is represented by a digital signal level of the playback device.

22. The method according to any one of claims 17 to 21, wherein: The second sample hearing threshold data indicates information about the audiogram of the second individual sample set.

23. The method according to any one of claims 17 to 22, wherein: The second sample hearing threshold data indicates a mean of the audiograms of the second individual sample set.

24. The method according to any one of claims 17 to 23, wherein: The second sample hearing threshold data indicates a mean value of the hearing thresholds of the second individual sample set for corresponding frequencies and ears.

25. A method according to claim 18 or any one of claims 19 to 24 when dependent on claim 18, wherein The second individual sample set is selected based on at least one user attribute of the user.

26. A method according to claim 18 or according to any one of claims 19 to 25 when dependent on claim 18, wherein Determining an estimate of the audiogram is performed at the media playback device or at a server device in communication with the media playback device.

27. A method according to claim 18 or any one of claims 19 to 26 when dependent on claim 18, further comprising: obtaining updated user hearing threshold data for the user for a second media playback device different from the media playback device, the updated user hearing threshold data indicating an updated hearing threshold for a given frequency; determining a deviation between the user hearing threshold at the given frequency as indicated by the user hearing threshold data and the updated hearing threshold; as well as An estimate of second calibration data for the second media playback device is determined based on the estimate of the calibration data and the determined deviation.

28. A method according to claim 18 or any one of claims 19 to 26 when dependent on claim 18, further comprising: obtaining updated user hearing threshold data for the user for a second media playback device different from the media playback device, the updated user hearing threshold data indicating updated hearing thresholds for a plurality of given frequencies; determining an estimate of a deviation between the calibration data and second calibration data for the second media playback device based on the user hearing thresholds at the plurality of given frequencies as indicated by the user hearing threshold data, the updated hearing thresholds, the second sample hearing threshold data, and sample noise data indicating variability in user responses in measuring hearing thresholds; as well as An estimated value of the second calibration data is determined based on the estimated value of the calibration data and the determined estimated value of the bias.

29. The method according to claim 28, wherein: Determining the estimated value of the second calibration data is based on a Bayesian Maximum A Posteriori (MAP) estimation technique.

30. A computing device comprising: at least one processor; as well as A memory storing instructions which, when executed by the at least one processor, cause the computing device to perform a method according to any one of claims 1 to 29.

31. A computer program comprising instructions which, when executed by a computing device, cause the computing device to perform a method according to any one of claims 1 to 29.

32. A non-transitory computer readable storage medium storing the computer program according to claim 31.

Images