Hearing aid loudness compensation method, computer program product and hearing aid

Abstract

The embodiment of the invention provides a hearing aid loudness compensation method, a computer program product and a hearing aid. After the hearing aid collects an original sound signal, loudness compensation can be carried out on the original sound signal by using a plurality of preset loudness compensation strategies to obtain a compensation sound signal corresponding to each loudness compensation strategy; and then, a pre-trained comfort evaluation model can be utilized to evaluate the comfort level of the compensated sound signal corresponding to each loudness compensation strategy, a better loudness compensation strategy is selected based on the comfort level, and the sound signal compensated by the better loudness compensation strategy is output to a user. According to the loudness compensation method provided by the embodiment of the invention, more natural, comfortable and clear hearing experience can be provided for the user, and the performance of the hearing aid and the satisfaction degree of the user are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a hearing aid loudness compensation method, a computer program product, and a hearing aid. Background Art

[0002] A hearing aid is an electronic device used to help hearing-impaired users improve their hearing, and one of its core functions is to perform loudness compensation. The purpose of loudness compensation is to adjust the sound signal to a range that is audible and comfortable for the user, so as to improve the user's hearing and speech comprehension ability. Generally, loudness compensation can be achieved through Wide Dynamic Range Compression (WDRC) technology. WDRC technology can dynamically adjust the gain of the hearing aid according to the intensity of the input sound to compress the sound signal to a range that is audible and comfortable for the user. In related technologies, when performing loudness compensation on the original sound signal collected by the hearing aid, an audiogram is usually obtained based on the user's pure tone audiometry, and then a loudness compensation I / O curve is determined based on the audiogram and a preset fitting formula. The loudness compensation I / O curve will remain unchanged during the user's subsequent use. However, in different acoustic environments, users have different requirements for loudness comfort and clarity. A fixed loudness compensation I / O curve is difficult to achieve the best effect in all environments, resulting in poor hearing effects and affecting the user's experience. Summary of the Invention

[0003] In view of this, the present application provides a hearing aid loudness compensation method, a computer program product, and a hearing aid.

[0004] According to a first aspect of the present application, a hearing aid loudness compensation method is provided, the method comprising:

[0005] Obtain the original sound signal currently collected by the hearing aid;

[0006] performing loudness compensation on the original sound signal using multiple loudness compensation strategies from a preset loudness compensation strategy set, to obtain compensated sound signals corresponding to each loudness compensation strategy; wherein the loudness compensation strategy is used to describe a corresponding relationship between the loudness of the original sound signal and the loudness of the compensated sound signal;

[0007] For each loudness compensation strategy, a comfort level of the compensated sound signal is determined based on the original sound signal, the compensated sound signal corresponding to the loudness compensation strategy, and a pre-trained comfort evaluation model. The comfort evaluation model is trained based on multiple sets of sample data with labels, each set of sample data including a sample sound signal and a sample compensated sound signal after loudness compensation has been performed on the sample sound signal. The labels are used to indicate the comfort level of the sample compensated sound signal.

[0008] An output sound signal is selected from the compensated sound signals corresponding to the multiple loudness compensation strategies based on the comfort level, and is output to the user.

[0009] According to a second aspect of the present application, a hearing aid is provided, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method mentioned in the first aspect when executing the computer program.

[0010] According to a third aspect of the present application, a computer program product is provided, which includes a computer program, and when the computer program is executed, the method mentioned in the first aspect is implemented.

[0011] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed, the method mentioned in the first aspect is implemented.

[0012] Using the solution provided by this application, after collecting the original sound signal, the hearing aid can use multiple preset loudness compensation strategies to compensate the original sound signal for loudness, obtaining a compensated sound signal corresponding to each loudness compensation strategy. A pre-trained comfort evaluation model can then be used to evaluate the comfort level of the compensated sound signal corresponding to each loudness compensation strategy. Based on the comfort level, the optimal loudness compensation strategy is selected, and the sound signal compensated using the optimal loudness compensation strategy is output to the user. The loudness compensation method provided in this embodiment dynamically adjusts the compensation strategy based on the real-time collected original sound signal and the user's subjective experience. The pre-trained comfort evaluation model is used to accurately evaluate the compensation effects of various compensation strategies to select the optimal compensation strategy, thereby providing the user with a more natural, comfortable, and clear hearing experience, significantly improving the performance of the hearing aid and user satisfaction.

[0013] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 It is a schematic diagram of a hearing aid loudness compensation method in the related art.

[0016] Figure 2 Schematic diagram of a hearing aid loudness compensation method according to an embodiment of the present application.

[0017] Figure 3 This is a flowchart of a hearing aid loudness compensation method according to an embodiment of the present application.

[0018] Figure 4 This is a schematic diagram of dynamically screening the optimal loudness compensation strategy using a comfort evaluation model according to an embodiment of the present application.

[0019] Figure 5 3 is a schematic diagram of determining a set of compensated sound signal loudness values ​​corresponding to each original sound signal loudness value according to an embodiment of the present application.

[0020] Figure 6 FIG. 4 is a schematic diagram of a loudness compensation I / O curve sample according to an embodiment of the present application.

[0021] Figure 7 This is a schematic diagram of fusing original sound signal features and compensated sound signal features according to an embodiment of the present application.

[0022] Figure 8 It is a schematic diagram of the structure of a comfort evaluation model according to an embodiment of the present application.

[0023] Figure 9 It is a schematic diagram of the logical structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] A hearing aid is an electronic device used to help hearing-impaired users improve their hearing, and one of its core functions is to perform loudness compensation. The purpose of loudness compensation is to adjust the sound signal to a range that is audible and comfortable for the user, in order to improve the user's hearing and speech comprehension ability. Wide dynamic range compression (WDRC) technology is a commonly used loudness compensation technology. As the input sound changes, the gain of the hearing aid will also change in real time. After the WDRC algorithm, the extremely small sounds in the input signal remain almost unchanged, small sounds are amplified, loud sounds are compressed, and extremely large sounds are limited. In related technologies, once the hearing aid determines the loudness compensation I / O curve (i.e., the gain curve, which is used to characterize the mapping relationship between the loudness value of the input signal and the loudness value of the output signal) during the fitting process, the loudness compensation I / O curve will remain unchanged during the user's subsequent use stage.

[0026] For example, Figure 1 Figure 2 is a schematic diagram of a loudness compensation scheme in the related art. The original sound signal collected by the hearing aid is subjected to time-frequency decomposition into different sub-bands (for example, m sub-bands) using an analysis filter bank. When performing loudness compensation on the original sound signal, each processing channel includes a sub-band. For each processing channel (for example, a total of m channels), the sound pressure level of the sub-band within the processing channel is calculated. Loudness compensation is then performed on the sound signal of the processing channel using a predetermined loudness compensation I / O curve corresponding to the processing channel. Finally, the compensated sound signals of each processing channel after loudness compensation are synthesized using a comprehensive filter bank to obtain the final compensated sound signal.

[0027] Because users have varying needs for loudness comfort and clarity in different acoustic environments (quiet, noisy, and reverberant), the aforementioned fixed loudness compensation I / O curve cannot adapt to dynamic changes in the sound signal (such as the shift in the ratio of speech to noise from quiet to noisy environments) and subtle differences in user perception, making it difficult to achieve optimal results in all environments. Furthermore, the aforementioned compensation scheme applies fixed gains to individual processing channels (sub-bands), failing to account for inter-channel and overall channel correlations. This makes it difficult to achieve the optimal listening experience that best suits the human ear.

[0028] Based on this, the embodiment of the present application provides a hearing aid loudness compensation method, such as Figure 2As shown, after collecting the original sound signal, the hearing aid can use multiple preset loudness compensation strategies (for example, loudness compensation strategy 1, loudness compensation strategy 2, loudness compensation strategy 3, etc.) to perform loudness compensation on the original sound signal, thereby obtaining a compensated sound signal corresponding to each loudness compensation strategy. Then, the pre-trained comfort evaluation model can be used to evaluate the comfort level of the compensated sound signal corresponding to each loudness compensation strategy. Based on the comfort level, a better loudness compensation strategy is selected, and the sound signal compensated by the better loudness compensation strategy (for example, Figure 2 The loudness compensation method provided in the embodiment of the present application dynamically adjusts the compensation strategy based on the real-time collected original sound signal and the user's subjective experience, and accurately evaluates the compensation effects of various compensation strategies through a pre-trained comfort evaluation model to select the current optimal compensation strategy, thereby providing the user with a more natural, comfortable, and clear listening experience, significantly improving the performance of the hearing aid and user satisfaction.

[0029] The loudness compensation method provided in the embodiment of the present application can be applied to various types of hearing aids. Figure 3 and Figure 4 The loudness compensation method of the hearing aid is introduced in detail.

[0030] like Figure 3 As shown, the method may include the following steps:

[0031] S302, obtaining the original sound signal currently collected by the hearing aid;

[0032] In step S302, the sound signal currently collected by the hearing aid may be obtained, hereinafter referred to as the original sound signal. The original sound signal may be a single-channel sound signal collected by a single microphone in the hearing aid, or a single-channel sound signal obtained by processing the sound signals collected by two microphones in the hearing aid using beamforming technology.

[0033] S304: performing loudness compensation on the original sound signal using multiple loudness compensation strategies from a preset loudness compensation strategy set, to obtain a compensated sound signal corresponding to each loudness compensation strategy; wherein the loudness compensation strategy is used to describe a corresponding relationship between the loudness of the original sound signal and the loudness of the compensated sound signal;

[0034] In step S304, in related art, when loudness compensation is performed on the original sound signal collected by the hearing aid, a fixed loudness compensation I / O curve is typically determined based on the user's pure tone audiogram. This loudness compensation I / O curve is then used to compensate the original sound signal. Considering that the most appropriate or optimal loudness compensation strategy may vary in different scenarios, a set of loudness compensation strategies can be pre-established, which can include multiple feasible loudness compensation strategies. A loudness compensation strategy describes the correspondence between the loudness of the original sound signal and the loudness of the compensated sound signal. To more accurately determine the optimal output loudness value (i.e., the compensated sound signal loudness value) corresponding to each input loudness value (i.e., the original sound signal loudness value), multiple feasible output loudness values ​​can be set for each input loudness value. A loudness compensation strategy can then be formulated based on the output loudness values ​​corresponding to different input loudness values. The different loudness compensation strategies may have different output loudness values ​​corresponding to at least some of the input loudness values.

[0035] The corresponding relationship between the loudness of the original sound signal and the loudness of the compensated sound signal can be represented by various forms such as a curve, a data table, etc., which is not limited in the embodiment of the present application.

[0036] For example, in some embodiments, the loudness compensation strategy can be represented by a loudness compensation I / O curve, which is used to represent the mapping relationship between input loudness values ​​and output loudness values. In some embodiments, the loudness compensation strategy can also be represented by a data table, which can record the set of output loudness values ​​corresponding to each input loudness value. Of course, the loudness compensation strategy can be represented in a variety of ways, not limited to the ones listed above, and can be specifically configured based on actual needs.

[0037] S306. Determine, for each loudness compensation strategy, a comfort level of the compensated sound signal corresponding to the loudness compensation strategy based on the original sound signal, the compensated sound signal corresponding to the loudness compensation strategy, and a pre-trained comfort evaluation model; wherein the comfort evaluation model is trained based on multiple sets of sample data carrying labels, each set of sample data including a sample sound signal and a sample compensated sound signal after loudness compensation has been performed on the sample sound signal, and the label indicates the comfort level of the sample compensated sound signal.

[0038] In step S306, in order to accurately evaluate the comfort level of the sound signal after compensation by each loudness compensation strategy, a large amount of sample data can be collected and a comfort evaluation model can be trained using the sample data to evaluate the comfort level of the compensated sound signal using the trained comfort evaluation model. Figure 4As shown in the figure, the training process and application process of the comfort evaluation model are shown. Users can wear hearing aids and move around in real life environments, and manually provide feedback on the current hearing effect and give a comfort score. For example, it can be divided into 10 comfort levels, with corresponding scores of 1-10 points, where the most comfortable is 10 points and the least comfortable is 1 point. Of course, when setting the comfort level, the number of comfort levels can be flexibly set based on actual needs. For example, 5 levels, 3 levels, and so on can be set.

[0039] After the user manually provides feedback on the comfort level, the original sound signal currently collected by the hearing aid can be used as the sample sound signal, and the sound signal currently compensated and output to the user can be used as the sample compensated sound signal. The two can be used as a set of sample data. The comfort level feedback by the user can then be used as the label corresponding to the set of sample data. The preset initial model can be supervised trained using the sample data carrying the label to obtain a comfort evaluation model.

[0040] For example, in some embodiments, the sample sound signal and the sample compensated sound signal can be input into an initial model, the sound signal can be feature extracted by the initial model, and the comfort level can be predicted based on the extracted features. Then, the model parameters of the initial model can be adjusted based on the difference between the predicted comfort level and the label to train the comfort evaluation model.

[0041] In some embodiments, the sample sound signal and the sample compensated sound signal may also be preprocessed first, for example, sound features may be extracted from the two sound signals, the extracted sound features may be fused, and then input into the initial model, and the model parameters of the initial model may be adjusted based on the difference between the comfort level predicted by the initial model and the label to obtain the comfort evaluation model.

[0042] Among them, the architecture of the comfort evaluation model can be set based on actual needs. For example, some existing deep learning model architectures can be adopted, or the model architecture can be designed independently based on the characteristics of the data to be processed. It can be flexibly set based on actual needs.

[0043] After loudness compensation is performed on the original sound signal using multiple loudness compensation strategies, a compensated sound signal corresponding to each loudness compensation strategy can be obtained. For each compensated sound signal corresponding to the loudness compensation strategy, a comfort level for the compensated sound signal corresponding to that loudness compensation strategy can be determined based on the original sound signal, the compensated sound signal corresponding to that loudness compensation strategy, and a pre-trained comfort evaluation model. For example, the original sound signal and the compensated sound signal corresponding to that loudness compensation strategy can be input into the comfort evaluation model, which can then predict the comfort level of the compensated sound signal corresponding to that loudness compensation strategy. Alternatively, feature extraction can be performed on the original sound signal and the compensated sound signal corresponding to that loudness compensation strategy to obtain their respective sound signal features. The sound signal features can then be fused and input into the comfort evaluation model, which can then predict the comfort level based on the fused sound signal features.

[0044] S308 : Select an output sound signal from the compensated sound signals corresponding to the multiple loudness compensation strategies based on the comfort level, and output the output sound signal to the user.

[0045] In step S308, after the comfort level of the compensated sound signal corresponding to each loudness compensation strategy is predicted using the comfort evaluation model, the output sound signal ultimately output to the user can be selected based on the comfort level of each compensated sound signal. For example, the compensated sound signal with the highest comfort level can be selected as the output sound signal and output to the user. Alternatively, a compensated sound signal with a comfort level higher than a preset level can be randomly selected and output to the user. The specific configuration can be flexibly adjusted based on actual needs and is not limited in this embodiment of the present application.

[0046] The embodiments of the present application achieve personalized loudness compensation by dynamically evaluating the comfort levels of multiple loudness compensation strategies, thereby improving user experience and satisfaction.

[0047] In some embodiments, when constructing a loudness compensation strategy set, the user's audiogram can be obtained, wherein an audiogram is a chart used to assess hearing loss. It helps diagnose the type and degree of hearing loss by measuring an individual's hearing thresholds at different frequencies. For each original sound signal loudness value, the corresponding compensated sound signal loudness value under multiple fitting formulas can be determined based on the user's audiogram. Among them, the fitting formula (Prescriptive Fitting Formula) is a set of formulas used to determine the hearing aid gain setting in hearing aid fitting. These formulas calculate the hearing aid gain parameters that best suit the user based on the user's audiogram and other relevant information to improve the user's hearing and speech comprehension ability. Currently, there are two main series of fitting formulas commonly used in hearing aids: the NAL series and the DSL series. The NAL series mainly includes formulas such as NAL, NAL-R, NAL-RP, NAL-NL1, and NAL-NL2, and the DSL series includes DSL v5.0 and others. Currently, a single fitting formula is usually selected based on the user's age, gender and other information to obtain a fixed loudness compensation I / O curve. In order to more accurately determine the optimal loudness compensation strategy in different scenarios, the embodiment of the present application can use multiple fitting formulas for fitting, such as Figure 5 As shown, for each original sound signal loudness value, the corresponding compensated sound signal loudness value for each fitting formula can be determined based on the audiogram and other user information (e.g., gender, age, etc.), thereby obtaining multiple compensated sound signal loudness values ​​that match the original sound signal loudness value. The minimum and maximum values ​​of the multiple compensated sound signal loudness values ​​can then be determined, i.e., a range of compensated sound signal loudness values ​​that match the original sound signal loudness value can be obtained.

[0048] Multiple loudness values ​​can then be determined within the numerical range formed by the minimum and maximum values ​​to construct a set of compensated sound signal loudness values ​​corresponding to the original sound signal loudness value. For example, multiple loudness values ​​can be randomly selected within the range, or the minimum value can be used as a reference and gradually increased according to a certain step size to obtain multiple loudness values. The multiple loudness values ​​determined can be within the range, and the specific determination method can include various methods, which are not limited in this embodiment of the present application.

[0049] For example, assuming that the loudness value of the original sound signal of a certain processing channel is 20, and the loudness values ​​of the compensated sound signal obtained by fitting through fitting formula 1, fitting formula 2, and fitting formula 3 are 30, 60, and 50 respectively, then the minimum value is 30 and the maximum value is 60, that is, the range of the loudness value of the compensated sound signal adapted to the loudness value of the original sound signal is [30, 60]. Then, multiple loudness values ​​can be determined between 30-60.

[0050] After determining the set of compensated sound signal loudness values ​​corresponding to each original sound signal loudness value, multiple loudness compensation strategies can be determined based on the sets of compensated sound signal loudness values ​​corresponding to different original sound signal loudness values ​​to obtain a set of loudness compensation strategies. In each loudness compensation strategy, at least some of the original sound signal loudness values ​​correspond to different compensated sound signal loudness values. For example, assuming the loudness compensation strategy is represented by a loudness compensation I / O curve, Figure 6 A sample loudness compensation I / O curve is shown.

[0051] The embodiment of the present application determines multiple possible compensated sound signal loudness values ​​for each original sound signal loudness value by comprehensively applying multiple fitting formulas and combining the user's audiogram and other relevant information (such as gender, age, etc.), thereby constructing a set containing multiple loudness compensation strategies. This method not only takes into account the unique advantages of different fitting formulas, but also increases the diversity and flexibility of compensation strategies by determining the range of compensated loudness values ​​and selecting multiple loudness values ​​within this range. This enables the hearing aid to dynamically select the optimal loudness compensation strategy based on different usage scenarios and user needs, providing users with more accurate and personalized hearing compensation, thereby significantly improving the user's hearing experience and speech comprehension ability in various environments, and enhancing the practicality and adaptability of the hearing aid.

[0052] Considering that a user's hearing loss often varies across different frequencies—for example, high-frequency hearing loss may be more severe than low-frequency hearing loss—loudness compensation strategies for different frequency ranges can be personalized based on the user's audiogram to ensure that signals within each frequency range can be comfortably heard by the user. Therefore, in some embodiments, when formulating loudness compensation strategies, different loudness compensation strategies can be formulated for different processing channels (i.e., different frequency ranges). For example, each loudness compensation strategy can include multiple sub-strategies, each corresponding to a processing channel, describing the correspondence between the loudness of the original sound signal and the loudness of the compensated sound signal within that processing channel. A processing channel can be a specific frequency range. For example, the full frequency band of a sound signal can be divided into multiple sub-bands, with each processing channel including one or more sub-bands. When using each loudness compensation strategy to loudness compensate the original sound signal, for each processing channel of the original sound signal, the sub-strategy corresponding to that processing channel can be retrieved from the loudness compensation strategy. The sub-strategy corresponding to that processing channel is then used to compensate the original sound signal of that processing channel, resulting in a compensated sound sub-signal for that processing channel. Then, the compensation sound sub-signals of each processing channel can be synthesized to obtain the final compensation sound signal.

[0053] The embodiments of the present application can accurately compensate for the differences in the user's hearing loss at each frequency by formulating personalized loudness compensation sub-strategies for different frequency ranges (processing channels). This method of channel-by-channel compensation allows the signal in each frequency range to receive appropriate gain adjustment based on the user's actual hearing needs, thereby ensuring that the user can hear the sound comfortably and clearly across the entire frequency band. This method greatly improves the accuracy and personalization of compensation, effectively improving the user's auditory experience. Whether in a quiet environment or a complex noisy environment, it can provide users with a more natural and comfortable hearing effect, significantly improving the performance of the hearing aid and user satisfaction.

[0054] In some embodiments, the full frequency band of the sound signal may be divided into a plurality of sub-bands, and each processing channel includes a sub-band. Figure 4As shown in the figure, a WOLA (weighted overlap add) filter bank can be used for time-frequency analysis. The WOLA filter bank divides the full frequency band into several subbands. Assuming a signal sampling rate of 16 kHz and a signal bandwidth of 8 kHz, if an even-stacked WOLA filter is used to divide the signal into 64 subbands, the frequency range of each subband is 0-62.5 Hz, 62.5-187.5 Hz, 187.5-312.5 Hz, ..., 7937.5-8000 Hz. Furthermore, each subband can be treated as a channel without merging subbands, allowing for more precise frequency adjustment.

[0055] The embodiment of the present application divides the full frequency band of the sound signal into multiple sub-bands, and each sub-band is used as an independent processing channel for loudness compensation. This allows each sub-band to independently adjust the gain according to the user's hearing loss within the frequency range, thereby achieving accurate compensation for different frequency components, significantly improving the personalized adaptation capability of the hearing aid, and providing users with a more natural and comfortable listening experience.

[0056] In some embodiments, when determining the comfort level of the compensated sound signal corresponding to the loudness compensation strategy based on the original sound signal, the compensated sound signal corresponding to the loudness compensation strategy, and a pre-trained comfort evaluation model, original sound signal features can be extracted from the original sound signal, and compensated sound signal features can be extracted from the compensated sound signal corresponding to the loudness compensation strategy. The extracted sound signal features can include various features, such as time domain features (e.g., amplitude, root mean square value), frequency domain features (e.g., spectrum, mel-spectrogram, etc.), cepstral domain features, etc., and the specific features can be set based on actual needs. The extracted original sound signal features and compensated sound signal features can then be fused to obtain fused sound signal features. For example, the two can be concatenated according to different dimensions. The fused sound signal features are then input into the comfort evaluation model, and the comfort evaluation model is used to predict the comfort level of the compensated sound signal corresponding to the loudness compensation strategy.

[0057] By extracting features from both the original and compensated sound signals, fusing these features and feeding them into a pre-trained comfort evaluation model, the model learns the inherent correlation between the original and compensated sound signal features, as well as their relationship to the comfort level. This feature fusion-based approach comprehensively considers the characteristics of both the original and compensated sound signals, providing the model with more comprehensive information and enabling a more accurate assessment of the compensation effect.

[0058] Considering that the Mel spectrum is more consistent with human auditory characteristics, such as Figure 7As shown, in some embodiments, the original sound signal feature and the compensated sound signal feature both include Mel-spectrograms. When the original sound signal feature and the compensated sound signal feature are fused to obtain a fused sound signal feature, the Mel-spectrogram of the original sound signal and the Mel-spectrogram of the compensated sound signal can be stacked in the channel dimension to obtain a fused sound signal feature, wherein the fused sound signal feature is a three-dimensional tensor.

[0059] For example, the Mel-spectrogram of an original sound signal can be determined using the following method: the sound signal can be framed, the power spectrum estimated using the periodogram method, the power spectrum filtered using a Mel filter bank, the energy in each filter calculated, and finally the logarithm of each filter energy. Each frame signal is a one-dimensional Mel-spectrogram, and a segment of the original sound signal is a two-dimensional Mel-spectrogram. The length of the two-dimensional Mel-spectrogram dimension can be controlled by selecting an appropriate frame length and frame shift, ultimately achieving a size of J*K, where J is the number of frames and K is the length of the Mel-spectrogram of each frame. Similarly, the Mel-spectrogram of the compensated sound signal can be determined using the above method, and the size of the Mel-spectrogram of the compensated sound signal is also J*K. By concatenating the two in the channel dimension, a 2@J*K three-dimensional feature is obtained, which serves as the fused sound signal feature. The fused sound signal feature can then be input into a comfort assessment model to predict the comfort level.

[0060] In some embodiments, the comfort evaluation model of the embodiment of the present application can be a convolutional neural network, the structure of the convolutional neural network is as follows: Figure 8As shown, the input layer is 2@J*K. The original sound signal's mel-spectrogram and the compensated sound signal's mel-spectrogram are combined into a two-channel J*K shape, which is then fed into the convolutional neural network. One channel contains the original sound signal's mel-spectrogram, and the other channel contains the compensated sound signal's mel-spectrogram. The second layer is the Max-Pooling layer, which aims to compress features, simplify network complexity, and reduce computational effort. The third layer is the Convolution layer. The convolution operation here involves applying a cross-correlation operation to each channel's matrix from left to right (the convolution kernel is typically a 3x3 matrix) and from top to bottom (first from left to right, then from top to bottom; all convolution operations preserve positional information). The convolution kernel acts like a small window, sliding from the top left corner to the bottom right corner. The step size can be determined by adjusting parameters. Finally, the values ​​of the two channels are summed to obtain a single value. The fourth layer is the Max-Pooling layer, followed by the Softmax classification layer. Softmax is a commonly used activation function, primarily used in multi-class classification problems. It converts the input real number vector into a probability distribution so that the output value of each category is between 0 and 1, and the sum of the output values ​​of all categories is 1. The mathematical expression of the Softmax function is as follows: For the input vector z = [z1, z2, ..., z n ], the output of the Softmax layer function y=[y1,y2,...,y n ], the calculation formula is:

[0061] By stacking the mel-spectrogram of the original sound signal and the mel-spectrogram of the compensated sound signal along the channel dimension, a three-dimensional tensor is formed as the fused sound signal feature. This fusion method can fully utilize the frequency information of the original sound signal and the compensated sound signal. As a feature representation that conforms to human auditory perception, the mel-spectrogram can more naturally reflect the frequency characteristics of sound. Stacking the two not only retains the complete information of the original sound signal, but also incorporates the characteristics of the compensated signal, allowing the subsequent comfort evaluation model to make accurate predictions based on more comprehensive data. This three-dimensional tensor structure provides rich input for deep learning models, helping the model learn the complex relationship between the original signal and the compensated signal, thereby more accurately evaluating the effectiveness of different loudness compensation strategies.

[0062] In some embodiments, sample data can also be collected while the user is using the hearing aid, so as to use the sample data to fine-tune the comfort evaluation model. For example, during the user's use of the hearing aid, after detecting the user's sound adjustment operation, the comfort level of the adjusted sound signal is determined as a preset comfort level. For example, after the user increases or decreases the volume, the increased or decreased sound can be defaulted to the sound that the user feels comfortable. Therefore, the adjusted comfort level can be set by default to a certain level that represents comfort. For example, taking 10 levels as an example, it can be set to 8 or 7 (which can be set based on actual needs). Then, the original sound signal currently collected by the hearing aid and the adjusted sound signal after the user's adjustment can be obtained, and the original sound signal and the adjusted sound signal can be used as sample data, and the preset comfort level can be used as a label to construct a training sample. The training sample is then used to fine-tune the comfort evaluation model.

[0063] The embodiments of the present application achieve continuous optimization and personalized adaptation of the model by collecting sample data while the user uses the hearing aid and using this data to fine-tune the comfort evaluation model. In this way, the comfort evaluation model can continuously learn the user's preferences and habits, thereby more accurately predicting the comfort level of different sound signals. This fine-tuning mechanism based on actual user operations not only improves the accuracy and adaptability of the model, but also enhances the personalized experience of the hearing aid, enabling the hearing aid to better meet the user's auditory needs and improve user satisfaction and usage experience.

[0064] In some embodiments, considering that the number of training samples collected by a single hearing aid may be relatively small, to increase the number of training samples and prevent the leakage of user privacy data, a federated learning approach can be employed when fine-tuning the comfort assessment model. Specifically, the hearing aid can be connected to a cloud server. When fine-tuning the comfort assessment model using the training samples, original sound signal features can be extracted from the original sound signal currently collected by the hearing aid, and adjusted sound signal features can be extracted from the adjusted sound signal. The original sound signal features and the adjusted sound signal features are then fused and input into the comfort assessment model. The comfort assessment model then determines a predicted comfort level for the adjusted sound signal. A target loss is determined based on the difference between the predicted comfort level and the label, and an update gradient for the model parameters is determined based on the target loss. This updated gradient is then sent to the cloud server, which determines updated model parameters based on the updated gradients sent by each of the multiple hearing aids. The updated model parameters can then be received back from the cloud server and used to update the current model parameters of the comfort assessment model to fine-tune the comfort assessment model.

[0065] For example, users can download the latest comfort evaluation model from the server. Each user trains the model using local data, uploads encrypted gradients to the server, and then aggregates the gradients across users to update the model parameters. The server returns the updated parameters to each user, who then updates their own model. Federated learning solves the problem of insufficient model generalization due to small amounts of training data, saving new users time and effort in debugging and adapting to the product.

[0066] In some embodiments, when training the comfort evaluation model, a combination of federated learning and self-learning can be used. For example, federated learning (i.e., distributed training) can be performed every first preset time period (e.g., 6 months), and the comfort model can be trained using training samples collected from multiple hearing aids, thereby aggregating more training samples and improving the generalization ability of the model. At the same time, self-training can be performed every second preset time period (e.g., 1 week), that is, the comfort model can be fine-tuned using training samples collected locally. Through federated learning, the privacy issues of training data and the poor generalization ability caused by the small amount of data can be solved. At the same time, user preferences can be recorded during use, and self-training learning can be performed regularly, taking into account both generalization ability and personalized features.

[0067] In some embodiments, the sound adjustment operation includes one or more of a volume adjustment operation and a mode switching operation. For example, the volume adjustment operation may be to increase the volume or decrease the volume. Hearing aids typically have multiple modes to adapt to different environments and user needs. These modes optimize sound processing and improve the user's auditory experience by adjusting the parameters of the hearing aid (such as gain, noise suppression, directionality, etc.). Therefore, the sound adjustment operation may also include a mode switching operation by the user.

[0068] Typically, for users with severe high-frequency loss, the collected original sound signal needs to be down-converted. In traditional hearing aid designs, loudness compensation and down-conversion are typically handled separately for scenarios requiring down-conversion. Loudness compensation primarily improves sound loudness by adjusting the gain at different frequencies, while down-conversion requires a separate algorithm to convert high frequencies to low frequencies. This separate approach can lead to issues such as complex processing, difficult parameter adjustment, and suboptimal sound quality. To avoid these issues, in some embodiments, down-conversion and loudness compensation can be combined. For example, before loudness compensation is performed on the original sound signal using multiple loudness compensation strategies from a pre-set set of strategies to obtain compensated sound signals corresponding to each loudness compensation strategy, the original sound signal can first be down-converted. Loudness compensation is then performed on the down-converted sound signal using multiple loudness compensation strategies from the pre-set set of strategies to obtain compensated sound signals corresponding to each loudness compensation strategy. In this way, the loudness compensation process simultaneously takes into account the effect of down-conversion, thereby obtaining compensated sound signals that better suit the user's hearing perception.

[0069] The embodiments of the present application combine frequency reduction processing with loudness compensation, avoiding the complexity and limitations brought about by the separate processing of the two in traditional hearing aid design. This integrated processing method not only simplifies the signal processing flow of the hearing aid and reduces the difficulty of parameter adjustment, but also provides a more natural, clearer sound that is more in line with the user's auditory characteristics, thereby significantly improving the performance of the hearing aid and the user's auditory experience.

[0070] Among them, the solutions of the above embodiments can be freely combined to obtain new solutions when there is no conflict. Due to space constraints, they will not be listed one by one here.

[0071] Accordingly, an embodiment of the present application further provides a computer program product, which includes a computer program, and when the computer program is executed, implements the method mentioned in any of the above embodiments.

[0072] In addition, the present invention also provides a hearing aid. Figure 9 As shown, the hearing aid 90 includes a processor 91, a memory 92, and a computer program stored in the memory 92 and executable by the processor 91. When the processor 91 executes the computer program, the method described in any of the above embodiments is implemented. Of course, in addition to the above functional units, the hearing aid can also be configured with other functional units based on the actual functions of the hearing aid, such as a microphone.

[0073] Accordingly, an embodiment of the present application further provides a computer storage medium, in which a program is stored. When the program is executed by a processor, the method in any of the above embodiments is implemented.

[0074] The embodiments of the present application may take the form of a computer program product implemented on one or more storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc, read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, disk or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0075] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0076] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0077] The above is a detailed introduction to the methods and devices provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the methods and core ideas of the present application. At the same time, for those skilled in the art, based on the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of the present application should not be understood as a limitation on the present application.

Claims

1. A hearing aid loudness compensation method, characterized in that: The method comprises: Obtain the original sound signal currently collected by the hearing aid; performing loudness compensation on the original sound signal using multiple loudness compensation strategies from a preset loudness compensation strategy set, to obtain compensated sound signals corresponding to each loudness compensation strategy; wherein the loudness compensation strategy is used to describe a corresponding relationship between the loudness of the original sound signal and the loudness of the compensated sound signal; For each loudness compensation strategy, a comfort level of the compensated sound signal is determined based on the original sound signal, the compensated sound signal corresponding to the loudness compensation strategy, and a pre-trained comfort evaluation model. The comfort evaluation model is trained based on multiple sets of sample data with labels, each set of sample data including a sample sound signal and a sample compensated sound signal after loudness compensation has been performed on the sample sound signal. The labels are used to indicate the comfort level of the sample compensated sound signal. An output sound signal is selected from the compensated sound signals corresponding to the multiple loudness compensation strategies based on the comfort level, and is output to the user.

2. The method according to claim 1, characterized in that The loudness compensation strategy set is constructed based on the following method: Get the user's audiogram; For each original sound signal loudness value, determining, based on the user's audiogram, corresponding compensated sound signal loudness values ​​for the original sound signal loudness value under a plurality of fitting formulas, and determining a minimum value and a maximum value from the compensated sound signal loudness values; Determining a plurality of loudness values ​​within a numerical range formed by the minimum value and the maximum value to construct a set of compensated sound signal loudness values ​​corresponding to the original sound signal loudness value; A plurality of loudness compensation strategies are determined based on a set of compensated sound signal loudness values ​​corresponding to different original sound signal loudness values ​​to obtain the loudness compensation strategy set.

3. The method according to claim 1 or 2, characterized in that The loudness compensation is performed on the original sound signal using a plurality of loudness compensation strategies in a preset loudness compensation strategy set to obtain a compensated sound signal corresponding to each loudness compensation strategy, including: In the process of performing loudness compensation on the original sound signal using each loudness compensation strategy, for each processing channel of the original sound signal, a sub-strategy corresponding to the processing channel is obtained from the loudness compensation strategy, and the original sound signal of the processing channel is compensated using the sub-strategy corresponding to the processing channel to obtain a compensated sound sub-signal for the processing channel; wherein each loudness compensation strategy includes multiple sub-strategies, each sub-strategy corresponding to a processing channel, and is used to describe the corresponding relationship between the loudness of the original sound signal and the loudness of the compensated sound signal in the processing channel; The compensated sound signal is obtained by synthesizing the compensated sound sub-signals of each processing channel.

4. The method according to claim 3, characterized in that The full frequency band of the sound signal is divided into a plurality of sub-bands, and each of the processing channels includes one sub-band.

5. The method according to claim 1, characterized in that The determining, based on the original sound signal, the compensated sound signal corresponding to the loudness compensation strategy, and a pre-trained comfort evaluation model, of the comfort level of the compensated sound signal corresponding to the loudness compensation strategy includes: extracting original sound signal features from the original sound signal; extracting a compensation sound signal feature from the compensation sound signal corresponding to the loudness compensation strategy; Fusing the original sound signal feature and the compensated sound signal feature to obtain a fused sound signal feature; The fused sound signal features are input into the comfort evaluation model, and the comfort level of the compensated sound signal corresponding to the loudness compensation strategy is predicted by the comfort evaluation model.

6. The method according to claim 5, characterized in that The original sound signal feature and the compensated sound signal feature are both mel-spectrograms, and fusing the original sound signal feature and the compensated sound signal feature to obtain a fused sound signal feature includes: The mel-spectrogram of the original sound signal and the mel-spectrogram of the compensated sound signal are stacked in the channel dimension to obtain the fused sound signal feature, wherein the fused sound signal feature is a three-dimensional tensor.

7. The method according to claim 1, characterized in that The method further comprises: During the process of the user using the hearing aid, after detecting the user's sound adjustment operation, determining the comfort level of the adjusted sound signal as a preset comfort level; Acquiring the original sound signal currently collected by the hearing aid and the adjusted sound signal after adjustment by the user; Using the original sound signal and the adjusted sound signal as sample data and the preset comfort level as a label to construct a training sample; The comfort evaluation model is fine-tuned using the training samples.

8. The method according to claim 7, characterized in that The hearing aid is communicatively connected to a cloud server, and fine-tuning the comfort evaluation model using the training sample includes: extracting original sound signal features from the original sound signal; extracting an adjustment sound signal feature from the adjustment sound signal; fusing the original sound signal features with the adjusted sound signal features, inputting the fused features into the comfort evaluation model, and determining a predicted comfort level of the adjusted sound signal using the comfort evaluation model; determining a target loss based on a difference between the predicted comfort level and the label, and determining an update gradient of a model parameter based on the target loss; Sending the updated gradients to a cloud server, and having the cloud server determine updated model parameters based on the updated gradients sent by each of the plurality of hearing aids; Receive the updated model parameters returned by the cloud server, and use the updated model parameters to update the current model parameters of the comfort evaluation model to fine-tune the comfort evaluation model.

9. The method according to claim 7, characterized in that The sound adjustment operation includes: volume adjustment operation and mode switching operation.

10. The method according to claim 7, characterized in that Before performing loudness compensation on the original sound signal using a plurality of loudness compensation strategies in a preset loudness compensation strategy set to obtain a compensated sound signal corresponding to each loudness compensation strategy, the method further includes: Performing frequency reduction processing on the original sound signal; The loudness compensation is performed on the original sound signal using a plurality of loudness compensation strategies in a preset loudness compensation strategy set to obtain a compensated sound signal corresponding to each loudness compensation strategy, including: Loudness compensation is performed on the down-converted sound signal using a plurality of loudness compensation strategies in a preset loudness compensation strategy set, to obtain a compensated sound signal corresponding to each loudness compensation strategy.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed, the method according to any one of claims 1 to 10 is implemented.

12. A hearing aid, characterized in that: The hearing aid comprises a processor, a memory, and a computer program stored in the memory and executable by the processor, and the processor implements the method according to any one of claims 1 to 10 when executing the computer program.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which implements the method according to any one of claims 1 to 10 when executed.

Images