Sound output adjusting method of hearing aid device, and device using same

By combining the ambient sound signal and user physiological characteristic parameters, the sound output adjustment is performed using the model of the hearing assist device, which solves the problem that cannot meet personalized needs in the prior art and achieves efficient automatic adjustment effect.

WO2024245207A1PCT designated stage expired Publication Date: 2024-12-05EVOCO LABS CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/095629
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-29
Filing Date
2024-05-28
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Existing hearing assist devices cannot effectively meet users' personalized sound output adjustment needs, especially in different usage scenarios and environments, automatic gain and volume automatic control algorithms cannot meet users' diverse needs, and the equipment hardware capabilities are limited.

Method used

By obtaining multiple adjustment parameters in the ambient sound signal, the target sound output parameters are determined through the combination of the first sound adjustment model and the second sound adjustment model, and the sound output of the listening assistant device is automatically adjusted by online training and server model update. .

Benefits of technology

It realizes personalized automatic adjustment of the sound output of hearing assist devices, meets the diverse needs of different users, improves the adjustment accuracy and efficiency of the equipment under limited resources, and reduces hardware costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024095629_05122024_PF_FP_ABST
    Figure CN2024095629_05122024_PF_FP_ABST
Patent Text Reader

Abstract

The present application provides an automatic sound output adjusting method of a hearing aid device. The method comprises: acquiring at least one first sound adjusting parameter and at least one second sound adjusting parameter; inputting the first sound adjusting parameter into a first sound adjusting model, and conveying an output result of the first sound adjusting model to a second sound adjusting model; determining a target sound output parameter of the hearing aid device at least on the basis of the second sound adjusting parameter and the output result of the first sound adjusting model by means of the second sound adjusting model, wherein the target sound output parameter is an output result of the second sound adjusting model; and adjusting the sound output of the hearing aid device to have the target sound output parameter.
Need to check novelty before this filing date? Find Prior Art

Description

Method for adjusting sound output of hearing assistive device and device using the same Technical Field

[0001] The present application relates to the field of hearing assistive devices, and more particularly, to a method for adjusting the sound output of a hearing assistive device and a device using the method. Background Art

[0002] For hearing-assistive devices such as hearing aids, hearing aids, and cochlear implants, users need to adjust sound output parameters such as volume according to different usage scenarios and their own usage habits during the wearing process. However, most existing hearing-assistive devices are worn behind the ears, and the operation of the buttons on their bodies is not very convenient. Although many hearing-assistive devices are equipped with automatic gain, automatic volume control and other sound output parameter adjustment functions, the existing automatic gain or automatic volume control algorithms are obviously unable to meet the many personalized needs of users. For example, the volume or other sound output parameter requirements of users when talking to restaurant waiters and talking to friends may be inconsistent. When watching TV programs, the volume or other sound output parameter requirements for listening to news and listening to opera may also be inconsistent. It can be seen that it is difficult to meet the personalized needs of different users through a single algorithm.

[0003] To solve the above problems, some existing hearing assistive devices have further added scene selection or recognition systems. For example, by sampling external sounds, it is determined what kind of scene the user is currently in (such as outdoors, indoors, in transportation, or in a restaurant, etc.), and then the sound output parameters are automatically adjusted accordingly based on pre-set parameters. This approach often relies on complex pre-parameter settings, and it is difficult to adjust the preset parameters in a timely and convenient manner. In addition, even if this approach is adopted, the preset parameters in the same scene may not necessarily fully meet the personalized needs of different users or the personalized needs of the same user in different situations.

[0004] Furthermore, hearing-assistive devices typically have limited computing and storage capabilities and often only have Bluetooth communication capabilities. Implementing personalized sound output adjustments in these devices also requires addressing the challenges posed by limited device hardware capabilities.

[0005] Therefore, there is a need to design an improved automatic sound output adjustment method to meet the personalized needs of hearing assistive device users.

[0006] Summary of the Invention

[0007] One object of the present application is to provide a method for adjusting the sound output of a hearing assistive device and a device using the method.

[0008] In one aspect of the present application, a method for automatically adjusting the sound output of a hearing assistive device is provided, the method comprising: obtaining at least one first sound adjustment parameter and at least one second sound adjustment parameter; inputting the first sound adjustment parameter into a first sound adjustment model, and transmitting the output result of the first sound adjustment model to a second sound adjustment model; determining a target sound output parameter of the hearing assistive device through the second sound adjustment model at least based on the second sound adjustment parameter and the output result of the first sound adjustment model, wherein the target sound output parameter is the output result of the second sound adjustment model; and adjusting the sound output of the hearing assistive device to have the target sound output parameter.

[0009] In some embodiments, obtaining at least one first sound adjustment parameter and at least one second sound adjustment parameter includes: collecting an ambient sound signal through a microphone of the hearing assistive device; and extracting at least one first sound adjustment parameter and at least one second sound adjustment parameter from the obtained ambient sound signal.

[0010] In some embodiments, the first sound adjustment parameter is a non-personalized sound adjustment parameter.

[0011] In some embodiments, the first sound adjustment parameter includes the signal-to-noise ratio of the collected ambient sound signal, the voice activity detection in the collected ambient sound signal, the length of the noise in the collected ambient sound signal, the length of the human voice in the collected ambient sound signal, or the non-sound feature parameters of the user's environment.

[0012] In some embodiments, the second sound adjustment parameter is a personalized sound adjustment parameter.

[0013] In some embodiments, the second sound adjustment parameter includes the spectrum distribution of the current human voice, the spectrum distribution of the current environmental sound, the fundamental pitch of the current human voice, the current human voice print, the physiological characteristic parameters of the current user, or the status parameters of the current user.

[0014] In some embodiments, the second sound adjustment model is trained using a first training data sample set, wherein the first training sample set includes a user's manual sound adjustment instructions and at least one second sound adjustment parameter obtained by the hearing assistive device when the user issues the manual sound adjustment instructions.

[0015] In some embodiments, the generation of data in the first training data sample set includes: when the hearing assistive device obtains a manual sound adjustment instruction from the user, adding the manual sound adjustment and the second sound adjustment parameter obtained at this time to the first training sample set.

[0016] In some embodiments, the hearing assistance device is configured to perform online training on the second sound adjustment model using the first training sample set.

[0017] In some embodiments, the training of the second sound adjustment model is performed in a server that is communicatively connected to the hearing assistance device, and the hearing assistance device is configured to obtain updated model parameters of the second sound adjustment model from the server.

[0018] In some embodiments, the second sound adjustment model has initial model parameters, and determining the initial model parameters includes: obtaining current user information of the hearing assistive device; comparing the obtained current user information with information of other users pre-stored in the server; and using the parameters of the second sound adjustment model pre-used by other users who have the highest similarity to the current user as the initial model parameters.

[0019] In some embodiments, the current user information is manually input by the current user of the hearing assistance device, and the current user information includes one or more of the region, age, or gender of the current user.

[0020] In some embodiments, the current user information includes a second sound adjustment parameter obtained by the hearing assistive device.

[0021] In some embodiments, obtaining the second sound adjustment parameter through the hearing assistive device includes: collecting an ambient sound signal through a microphone of the hearing assistive device; and extracting at least one second sound adjustment parameter from the obtained ambient sound signal.

[0022] In some embodiments, the second sound adjustment model is a decision tree model, a linear regression model, a convolutional neural network model, or a recursive neural network model.

[0023] In some embodiments, the second sound adjustment model is trained online in an embedded system.

[0024] In some embodiments, the first sound adjustment model is a traditional statistical analysis module.

[0025] In some embodiments, the target sound output parameter is a target volume value, and adjusting the sound output of the hearing assistive device to have the target sound output parameter is adjusting the sound output of the hearing assistive device to have the target volume value.

[0026] In some embodiments, the target sound output parameter includes a target volume value of a sound in a specific frequency band or a specific frequency.

[0027] In another aspect of the present application, a hearing assistive device with an automatic sound output adjustment function is provided, characterized in that the hearing assistive device is configured to perform the following steps: obtaining at least one first sound adjustment parameter and at least one second sound adjustment parameter; inputting the first sound adjustment parameter into a first sound adjustment model, and transmitting the output result of the first sound adjustment model to a second sound adjustment model; determining the target sound output parameter of the hearing assistive device through the second sound adjustment model at least based on the second sound adjustment parameter combined with the output result of the first sound adjustment model; and adjusting the sound output of the hearing assistive device to have the target sound output parameter.

[0028] In some embodiments, obtaining at least one first sound adjustment parameter and at least one second sound adjustment parameter includes: collecting ambient sound signals through a microphone of the hearing assistive device; and extracting at least one first sound adjustment parameter and at least one second sound adjustment parameter from the obtained ambient sound signals.

[0029] In some embodiments, the hearing assistance device is configured to perform online training on the second sound adjustment model using a first training sample set. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and other features of the present application will be more fully understood through the specification and the appended claims, taken in conjunction with the accompanying drawings. It should be understood that these drawings only depict several embodiments of the present application and, therefore, should not be considered to limit the scope of the present application. The present application will be more clearly and fully illustrated by the accompanying drawings.

[0031] FIG1 shows a flow chart of a method for automatically adjusting sound output of a hearing assistive device according to an embodiment of the present application;

[0032] FIG2 shows a flowchart of an online training method for a second sound adjustment model in a hearing assistance device according to an embodiment of the present application;

[0033] FIG3 shows a flowchart of a method for importing initial model parameters of a sound adjustment model in a hearing assistance device according to an embodiment of the present application;

[0034] FIG4 shows a schematic structural diagram of a hearing assist device according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols generally indicate similar components, unless the context indicates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be employed, and other changes may be made, without departing from the spirit or scope of the subject matter of the present application. It is understood that various configurations, substitutions, combinations, and designs of the various aspects of the present application generally described herein and illustrated in the accompanying drawings may be made, all of which are expressly intended to form a part of the present application.

[0036] In this application, unless otherwise expressly stated, the use of the singular includes the plural. In this application, unless otherwise expressly stated, the use of "or" means "and / or". In addition, the use of the term "include" and other forms such as "comprise" and "contain" are not limiting. In addition, unless otherwise expressly stated, terms such as "element" or "component" cover elements and components comprising one unit as well as elements and components comprising more than one subunit. In addition, the section headings used herein are for organizational purposes only and should not be construed as limiting the subject matter described.

[0037] As used herein, spatially relative terms, such as "below," "below," "above," "above," "up," "upper side," "lower side," "left side," "right side," "horizontal," "vertical," and the like, may be used herein to describe the relationship of one element or feature to another element or feature as shown in the accompanying drawings. In addition to the orientation depicted in the figures, spatially relative terms are intended to cover different orientations of the device in use or operation. The device can be oriented in other ways (rotated 90 degrees or in other orientations), and the spatially relative descriptors used herein should likewise be interpreted accordingly. It should be understood that when an element is referred to as being "connected to" or "coupled to" another element, it can be directly connected to or coupled to the other element, or there may be intervening elements.

[0038] It should be noted that the hearing assistive device of the present application may be a hearing aid configured to provide the user with an amplified or adjusted version of the detected sound, a cochlear implant that provides the user with electrical stimulation that reflects the audio content, or a dual-mode hearing assistive device that provides both sound adjustment and electrical stimulation. Those skilled in the art will understand that the hearing assistive device of the present application may be a device that includes any suitable hardware, firmware, software, or a combination thereof that can achieve the above functions. For example, the hearing assistive device of the present application may have an earplug or earphone structure, and be configured to receive external sounds through a microphone or other means, and reproduce the acquired external sounds through a speaker or other device with or without changing the parameters of the external sounds.

[0039] FIG1 shows a flow chart of a method for automatically adjusting sound output of a hearing assistive device according to an embodiment of the present application. The method can be used for any suitable hearing assistive device such as a hearing aid, a hearing aid, or a cochlear implant.

[0040] As shown in Figure 1, first, in step 102, the hearing assistive device obtains one or more sound adjustment parameters. The sound adjustment parameters described herein can be any parameters that can be used as a basis for adjusting the sound output of the hearing assistive device. In some embodiments, the sound adjustment parameters can be parameters related to the environment in which the hearing assistive device is located. For example, the sound adjustment parameters can be parameters related to the ambient sound collected by the hearing assistive device, such as the signal-to-noise ratio of the ambient sound or the result of voice activity detection in the ambient sound, the length of noise in the collected ambient sound signal, the length of human voice in the collected ambient sound signal, the spectral distribution of the current human voice, the spectral distribution of the current ambient sound, the fundamental pitch of the current human voice, the voiceprint of the current human voice, and so on. For another example, the sound adjustment parameters can also be other environmental characteristic parameters unrelated to the ambient sound, such as environmental characteristics such as ambient temperature, humidity, lighting, and geographic location. Furthermore, in some embodiments, the sound adjustment parameters can also be factors related to the user's own physiological characteristics, such as personal parameters such as the user's heart rate, pulse, blood pressure, blood oxygen level, blood sugar level, and stress level.

[0041] In some embodiments, the sound adjustment parameters may be actively collected and acquired by the hearing assistive device. For example, the sound adjustment parameters may be extracted through the ambient sound collected by the hearing assistive device. In other embodiments, the sound adjustment parameters may also be passively received by the hearing assistive device. For example, the hearing assistive device may obtain the required sound adjustment parameters, such as the user's real-time heart rate, the user's real-time location, and other information, from an external device (such as a smart watch, mobile phone, computer, etc.) periodically or on an event basis through a communication connection.

[0042] It can be seen that the sound adjustment parameters that need to be considered for adjusting the sound output of the hearing assistive device are relatively complex and diverse, and the existing hearing assistive devices are obviously unable to efficiently achieve more accurate, effective, and personalized automatic adjustment of the sound output. In order to achieve the purpose of personalized adjustment, method 100 can classify the sound adjustment parameters. Specifically, in step 202, the acquired sound adjustment parameters can be divided into a first sound adjustment parameter and a second sound adjustment parameter. In some embodiments, the first sound adjustment parameter is at least partially inconsistent with the second sound adjustment parameter.

[0043] In some embodiments, the first sound adjustment parameter is a non-personalized sound adjustment parameter. The "non-personalized sound adjustment parameter" referred to herein refers to a general parameter in sound adjustment that is generally considered unrelated to personalization. In other words, a "non-personalized sound adjustment parameter" refers to a parameter that reflects the common preferences of ordinary users for sound adjustment, such as the fact that ordinary users generally dislike noise and prefer human voices, or generally prefer music. Specifically, the non-personalized sound adjustment parameter used in the first sound adjustment parameter may include the signal-to-noise ratio of the collected ambient sound signal, the detection of voice activity in the collected ambient sound signal, the length of noise in the collected ambient sound signal, the length of human voice in the collected ambient sound signal, or non-sound characteristic parameters of the user's environment. Among these, non-sound characteristic parameters of the user's environment may include temperature, lighting, location type (restaurant, gymnasium, cinema, etc.), or geographic location, etc., all of which are not directly related to the user's personalized selection. Among these non-personalized sound adjustment parameters, since sound characteristic parameters are generally more important, they can be given a higher weight, while non-sound characteristic parameters can be given a relatively lower weight. In some embodiments, the proportion of sound feature parameters in the first sound adjustment parameter is higher than the proportion of non-sound feature parameters.

[0044] On the other hand, in some embodiments, the second sound adjustment parameter may be a personalized sound adjustment parameter. The "personalized sound adjustment parameter" described herein may be a non-universal parameter that is generally considered to be related to personalization in sound adjustment. In other words, a "personalized sound adjustment parameter" refers to a parameter that reflects the user's personalized preferences, such as a special preference for ambient sounds, a preference for the voices of familiar people, or a preference for hearing birdsong around while running. These parameters are generally related to the user's personalized preferences, rather than the common needs of ordinary users. Specifically, the personalized sound adjustment parameters used by the second sound adjustment parameter may include the spectral distribution of the current human voice, the spectral distribution of the current ambient sounds, the fundamental pitch of the current human voice, the current human voiceprint, the physiological characteristic parameters of the current user, or the current user's state parameters, etc. Among them, the user's physiological characteristic parameters may be parameters such as the user's heart rate, blood pressure, body temperature, or blood oxygen. The user's state parameter may be the user's exercise state, such as walking, running, sitting still, or lying down.

[0045] It should be noted that due to differences in users or usage scenarios or environments, some "non-personalized sound adjustment parameters" and "personalized sound adjustment parameters" may overlap. In some embodiments, the user can also adjust the classification of personalized or non-personalized sound adjustment parameters set in the hearing assistive device, for example, resetting some parameters set as personalized sound adjustment parameters to non-personalized sound adjustment parameters. With respect to the first sound adjustment parameter and the second sound adjustment parameter, the hearing assistive device in the embodiment of the present application further includes a mixed sound adjustment model consisting of a first sound adjustment model and a second sound adjustment model, so that the two sound adjustment models process the first sound adjustment parameter and the second sound adjustment parameter respectively. Specifically, in step 104, the hearing assistive device can input the first sound adjustment parameter into the first sound adjustment model, and transmit the output result of the first sound adjustment model to the second sound adjustment model. The output result is the data processing result obtained after the first sound adjustment model processes the first sound adjustment parameter, which will be described in detail below. Subsequently, in step 106, the hearing assistive device processes the second sound adjustment parameter, the output result of the first sound adjustment model through the second sound adjustment model, or further processes other data, thereby determining the target sound output parameter of the hearing assistive device based at least on the second sound adjustment parameter and the output result of the first sound adjustment model, wherein the target sound output parameter is the output result of the second sound adjustment model.

[0046] In order to achieve more optimized automatic adjustment of sound output under the limited resources of the hearing assistive device, in some embodiments, the first sound adjustment model and the second sound adjustment model can both be implemented by traditional statistical analysis models, and the scale of the statistical analysis model of the first sound adjustment model can be larger than the scale of the statistical analysis model of the second sound adjustment model. Optionally, in other embodiments, the first sound adjustment model and the second sound adjustment model can also be implemented by deep learning models. It should be noted that in some embodiments, the first sound adjustment model may not participate in online training, but through the subsequent connection between the hearing assistive device and the server, the model can be upgraded online, including updating the parameters of the model. This method of transferring the training process to a remote server can save resources and power consumption of the hearing assistive device. Conversely, in some embodiments, the second sound adjustment model can be trained online on the hearing assistive device, and then the parameters of the second sound adjustment model can be updated after the training is completed. In some embodiments, the second sound adjustment model can be a statistical analysis model such as a decision tree model or a linear regression model. In other embodiments, the second sound adjustment model is a deep learning model such as a convolutional neural network model or a recursive neural network model. Of course, in some embodiments, the training of the second sound adjustment model may not be implemented by the hearing assistive device, but may be performed in a server that is communicatively connected to the hearing assistive device in a manner similar to that of the first sound adjustment model, and the hearing assistive device is configured to obtain updated model parameters of the trained second sound adjustment model from the server.

[0047] Then, in step 108, based on the target sound output parameters output by the second sound adjustment model, the hearing assistive device automatically adjusts its sound output to have the target sound adjustment parameters, for example, adjusting the volume output of the hearing assistive device to the target volume value. Of course, the target sound output parameters may also include any other type of target sound output parameters. For example, the target sound output parameters may include target volume values ​​of one or more frequency bands or one or more specific frequency sounds, so that the hearing assistive device performs EQ equalizer adjustment on the sound output. For another example, the target sound output parameters may include configurations for suppressing ambient sounds and the degree of suppression, or for enhancing or suppressing human voices and the level of enhancement or suppression, and so on. By executing method 100, the hearing assistive device can efficiently achieve personalized automatic adjustment of sound output, and this automatic adjustment method will not bring about additional substantial increases in hardware costs or hardware resources.

[0048] To achieve better personalized processing of the automatic adjustment of the sound output of the hearing assistive device, in some embodiments of the present application, the second sound adjustment model in the mixed sound adjustment model can also be targeted for online training. Figure 2 shows a flowchart of a method 200 for online training the second sound adjustment model in the hearing assistive device according to one embodiment of the present application.

[0049] As shown in Figure 2, in step 202, the hearing assist device is configured to add the manual sound adjustment instruction and at least one second sound adjustment parameter obtained at this time to the first training sample set when the user's manual sound adjustment instruction is obtained. For example, when the user's real-time heart rate is greater than 130 beats / minute, the user may manually adjust the output volume of the hearing assist device (for example, pressing a preset sound adjustment button, or instructing and triggering the hearing assist device in a voice manner). When the hearing assist device obtains the manual sound adjustment instruction to reduce the volume, it records the second sound adjustment parameter obtained at this time (for example, the user's real-time heart rate) while responding to the operation required by the sound adjustment instruction, and records these second sound adjustment parameters in the first training sample set. The second sound adjustment parameters associated with these manual sound adjustment instructions often reflect the user's preference for sound output, so using them as training samples for a statistical analysis model or a deep learning model can obtain model parameters that are more in line with the user's personal preferences. In some embodiments, the manual sound adjustment instruction may include target sound output parameters, such as the overall target output volume of the hearing assistive device after adjustment, the target output volume of one or more frequency bands of sound, or the target output volume of one or more specific frequency sounds, etc. In other embodiments, the manual sound adjustment instruction may also include a value indicating a change in the sound output parameter, such as a specific value by which the overall output volume of the hearing assistive device needs to be increased or decreased. In some embodiments, while recording the second sound adjustment parameter, the hearing assistive device also records the real-time sound output parameters of the hearing assistive device, such as the overall output volume, the output volume of one or more frequency bands of sound, or the output volume of one or more specific frequencies, etc., and adds them to the first training sample set for subsequent training.

[0050] Those skilled in the art will understand that the above description is for illustrative purposes only. The manual sound adjustment instruction can be an instruction for the user to make any adjustment to the sound output of the hearing assistive device, and the corresponding second sound adjustment parameter can also be any other parameter.

[0051] Subsequently, in step 204, the hearing assistive device is configured to perform online training on the second sound adjustment model using the first training sample set. In some embodiments, the first training sample set is stored in the hearing assistive device, and the hearing assistive device is configured to use a built-in computing module (e.g., an embedded system) to perform online training on the second sound adjustment model based on the first training sample set.

[0052] In this way, the parameters of the second sound adjustment model can be updated regularly as needed, so that the hearing aid device can improve its automatic adjustment ability in a timely and efficient manner. At the same time, in some embodiments, since only the second sound adjustment model of a relatively small scale is trained instead of the first sound adjustment model of a larger scale, the training method can effectively save hardware resources. In addition, since the second adjustment parameters of the first training sample set used for training can be personalized sound adjustment parameters, and the connection between the user's manual adjustment instructions and the personalized sound adjustment parameters is fully considered, the second sound adjustment model generated by online training can more effectively reflect the user's personalized needs.

[0053] It is understood that in some embodiments, the online training of the second sound adjustment model can be performed periodically or based on the user's manual training instructions. For example, the first training sample set can record the relevant data of the user's manual sound adjustment recorded over a period of time (such as one day, one week, one month, one year or other time lengths), and then the second sound adjustment model is regularly trained offline based on the first training set. Optionally, the user can also manually adjust or delete the data in the first training sample set, such as selecting data within a specific time period, or choosing to delete one or more abnormal data (such as data related to user misoperation).

[0054] It is understood that in some embodiments, the training of the second sound adjustment model can also be performed in a server that is communicatively connected to the hearing assistive device. For example, the hearing assistive device sends the first training sample set to the server in advance. After the remote training of the server is completed, the hearing assistive device can obtain the updated model parameters of the trained second sound adjustment model from the server and configure the parameters of the local second sound adjustment model. In some embodiments, the first sound adjustment model does not participate in online training, but is pre-deployed in the hearing assistive device. The first sound adjustment model can be upgraded when the hearing assistive device is communicatively connected to the server, and the parameters of the first sound adjustment model can be updated.

[0055] The data collection of the first training sample set used in the method 200 shown in FIG2 often relies on manual sound adjustment by the user to trigger. In order to meet the training data volume requirement, a long data collection cycle may be required. Therefore, the online training of the entire second sound adjustment model may require a relatively long process. In order to better meet the user's personalized usage requirements during the initial use of the hearing aid device, in some embodiments of the present application, the hearing aid device can also be configured to import and utilize the data parameters of the second sound adjustment model of other users on the server to improve the effectiveness and accuracy of the initial model of its second sound adjustment model, thereby enhancing the user's usage experience.

[0056] FIG3 shows a flowchart of a method for importing initial model parameters of a sound adjustment model in a hearing assistance device according to an embodiment of the present application.

[0057] As shown in Figure 3, first, in step 302, the hearing assistive device sends information about the current user of the hearing assistive device to a server. This user information can be manually entered by the user, or automatically acquired and sent to the server by the hearing assistive device. In some embodiments, the current user information may include one or more of the current user's region, age, gender, occupation, educational background, hobbies, or other similar personal information.

[0058] Subsequently, in step 304, the server compares the received information of the current user with similar information of other users pre-stored in the server, thereby determining other users with the highest similarity to the current user. For example, the other users with the highest similarity may be other users who are from the same region, have the same gender, and are the same or similar age as the current user. It will be understood that the other users with the highest similarity may be one user, or multiple users with high similarity, such as the top 10 other users in the similarity ranking, or other users with a similarity exceeding 90% (or other preset thresholds). Those skilled in the art will appreciate that the calculation of similarity may adopt any known vector similarity calculation method, which is not limited here. For another example, when performing similarity calculation, different types of user information (such as region, gender, etc.) may be weighted.

[0059] Next, in step 306, the server sends the pre-stored parameters of the second sound adjustment model previously used by the other user with the highest similarity to the current user to the hearing assistive device. It will be appreciated that the other user with the highest similarity determined in step 304 may have already used a second sound adjustment model that matches their preferences and has been configured with the relevant parameters. These parameters may be stored on the server, which may then extract and send them to the hearing assistive device.

[0060] Finally, in step 308, after receiving the initial model parameters, the hearing assistive device updates the parameters of the second sound adjustment model to include these initial model parameters, thereby providing the current user with a second sound adjustment model that better suits their needs.

[0061] Although the above embodiment mainly describes obtaining the second sound adjustment parameter based on the user's manual indication to establish the first training sample set, considering the particularity of the use scenario of the hearing assistive device, in some embodiments of the present application, the hearing assistive device can also be configured to send one or more second sound adjustment parameters obtained by the current user during use as current user information to the server for user information comparison. For example, in some embodiments, in step 302, the hearing assistive device can be configured to collect the ambient sound signal through its microphone and extract one or more second sound adjustment parameters from the obtained ambient sound signal as the current user information. More specifically, the hearing assistive device can be configured to transmit one or more categories of second sound adjustment parameters obtained over a period of time (a week or longer) as current user information to the server. Subsequently, in step 304, the second sound adjustment parameters recorded over a period of time are used to compare with the similar second sound adjustment parameters recorded by other users pre-stored on the server over a period of time, thereby determining the other users with the highest similarity to the current user. This method of determining user similarity can not only more accurately and specifically realize the judgment of user similarity, but also avoid the cumbersome need to input current user information.

[0062] FIG4 shows a schematic diagram of the structure of a hearing assistive device according to an embodiment of the present application. As previously mentioned, the hearing assistive device of the present application can be a device including any suitable hardware, firmware, software, or a combination thereof capable of implementing a hearing assistive function, such as a hearing aid, a hearing aid, or a cochlear implant. Therefore, to more clearly illustrate the content of the present invention, the hearing assistive device 400 in FIG4 only shows the modules related to the sound output adjustment function of the hearing assistive device, and does not show other modules, such as a power supply.

[0063] As shown in Figure 4, the hearing assistive device 400 includes a sound adjustment parameter acquisition module 401, which can collect environmental sound signals through the microphone of the hearing assistive device, and extract at least one first sound adjustment parameter and at least one second sound adjustment parameter from the collected environmental sound signals. In addition, the sound adjustment parameter acquisition module 401 can also obtain sound adjustment parameters regularly or on an event basis through other devices or equipment (such as user mobile phones, watches or computers, etc.). Continuing to refer to Figure 4, the hearing assistive device 400 also includes a first sound adjustment model module 402, a second sound adjustment model module 403 and a sound output adjustment module 404. The automatic adjustment method of sound output performed by the hearing assistive device 400 will be further described below in conjunction with Figure 4.

[0064] First, the sound adjustment parameter acquisition module 401 obtains information such as the user's real-time heart rate, human voice spectrum distribution, noise spectrum distribution, the signal-to-noise ratio of the ambient sound, and voice activity detection in the ambient sound. Subsequently, the non-personalized first sound adjustment parameter information such as the signal-to-noise ratio of the ambient sound and the voice activity detection in the ambient sound is sent to the first sound adjustment model module 402 as the input of the first sound adjustment model, and the personalized second sound adjustment parameters such as the user's real-time heart rate, human voice spectrum distribution, and noise spectrum distribution are sent to the second sound adjustment model module 403 as the input of the second sound adjustment model. Then, after receiving the non-personalized first sound adjustment parameter information such as the signal-to-noise ratio of the ambient sound and the voice activity detection in the ambient sound as input, the first sound adjustment model module 402 further calculates and generates the output result of the first sound adjustment model.

[0065] After receiving the input results of the first sound adjustment model, the second sound adjustment model module 403 combines the received personalized second sound adjustment parameters such as the user's real-time heart rate, human voice spectrum distribution, noise spectrum distribution, or further processes other data to further calculate and determine the target sound output parameters, and sends them to the sound output adjustment module 404. Subsequently, the sound output adjustment module 404 adjusts the sound output of the hearing assistive device 400 to have the target output parameters, such as the target volume value, based on the received target sound output parameters.

[0066] Continuing with reference to Figure 4, the hearing assistive device 400 also includes a model online training module 405, which is connected to the sound adjustment parameter acquisition module 401 and / or the second sound adjustment model module 403 to implement online training of the second sound adjustment model in the hearing assistive device 400. Specifically, the model online training module 405 is configured to add the manual sound adjustment instruction and one or more of the personalized second sound adjustment parameters such as the user's heart rate, human voice spectrum distribution, and noise spectrum distribution obtained by the sound adjustment parameter acquisition module 401 when the hearing assistive device 400 obtains the user's manual sound adjustment instruction to the first training sample set in the model online training module 405. After that, the model online training module 405 can use the first training sample set to perform online training on the second sound adjustment model 403 and update the model parameters accordingly.

[0067] Of course, as shown in the figure, the hearing assistive device 400 can also be connected to the server 500. In some embodiments, the model online training module 405 can be configured to send the stored first online training set to the server 500, complete the training of the second sound adjustment model through the model online training module 501 on the server 500, and send the updated parameters after training to the second sound adjustment model module 403.

[0068] In some embodiments, the first sound adjustment model module 402 may not participate in online training, but may be pre-arranged in the hearing assistive device 400, and may be upgraded online when the hearing assistive device 400 is in communication with the server 500, and the parameters of the first sound adjustment model may be updated. In other embodiments, the first sound adjustment model may also adopt a machine learning model, and the first sound adjustment model module 402 may also be connected to the model online training module 405, and the model online training module 405 may use the second training sample set to perform online training on it. The data of the second training sample set used for its training may include the user's manual sound adjustment instructions, and one or more of the non-personalized first sound adjustment parameters acquired by the sound adjustment parameter acquisition module 401 at this time.

[0069] Continuing with reference to Figure 4, the hearing assistive device 400 can also communicate with the server 500 to obtain the initial model parameters of the second sound adjustment model. For example, the user manually or the hearing assistive device 400 automatically sends one or more of the region, age, gender, occupation, educational background, interests and hobbies or other similar personal information of the current user of the hearing assistive device 400 to the server 500. After receiving the above-mentioned current user information, the server 500 uses the user information comparison module 502 therein to compare it with similar information of other users stored in the storage module 503 of the server 500, and determines the other users with the highest similarity to the current user. Subsequently, the server 500 sends at least part of the parameters of the second sound adjustment model of the other users with the highest similarity stored in the storage module 503 as initial model parameters to the hearing assistive device 400. After receiving the above-mentioned initial model parameters, the hearing assistive device 400 sends them to the second sound adjustment model module 403 to update the second sound adjustment model 403 to have the above-mentioned initial model parameters.

[0070] In some embodiments, the hearing assistive device 400 may also transmit one or more types of second sound adjustment parameters acquired by the sound adjustment parameter acquisition module 401 over a period of time (one week or longer) as current user information to the server 500. The user information comparison module 502 of the server 500 then compares the second sound adjustment parameters of the same type for other users over a period of time stored in the storage module 503, thereby determining the other user with the highest similarity to the current user and performing subsequent initial model parameter importing.

[0071] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0072] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0073] Those skilled in the art can understand and implement other variations to the disclosed embodiments by studying the specification, disclosure, drawings, and appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude plural reference. In the practice of this application, a single component may perform the functions of multiple technical features recited in the claims. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A method for automatically adjusting the sound output of a hearing aid device, characterized in that: The method comprises: Acquiring at least one first sound adjustment parameter and at least one second sound adjustment parameter; Inputting the first sound adjustment parameter into a first sound adjustment model, and transmitting an output result of the first sound adjustment model to a second sound adjustment model; determining a target sound output parameter of the hearing aid device by the second sound adjustment model at least based on the second sound adjustment parameter and an output result of the first sound adjustment model, wherein the target sound output parameter is an output result of the second sound adjustment model; The sound output of the hearing assistance device is adjusted to have the target sound output parameters.

2. The method according to claim 1, characterized in that The acquiring of at least one first sound adjustment parameter and at least one second sound adjustment parameter comprises: collecting environmental sound signals through the microphone of the hearing aid device; and At least one first sound adjustment parameter and at least one second sound adjustment parameter are extracted from the acquired ambient sound signal.

3. The method according to claim 2, characterized in that The first sound adjustment parameter is a non-personalized sound adjustment parameter.

4. The method according to claim 3, characterized in that The first sound adjustment parameter includes the signal-to-noise ratio of the collected ambient sound signal, the voice activity detection in the collected ambient sound signal, the length of the noise in the collected ambient sound signal, the length of the human voice in the collected ambient sound signal, or the non-sound feature parameters of the user's environment.

5. The method according to claim 2, characterized in that: The second sound adjustment parameter is a personalized sound adjustment parameter.

6. The method according to claim 5, characterized in that The second sound adjustment parameter includes the spectrum distribution of the current human voice, the spectrum distribution of the current environmental sound, the fundamental pitch of the current human voice, the current human voice print, the physiological characteristic parameters of the current user, or the status parameters of the current user.

7. The method according to claim 1, characterized in that The second sound adjustment model is trained using a first training data sample set, wherein the first training sample set includes a user's manual sound adjustment instruction and at least one second sound adjustment parameter acquired by the hearing assistive device when the user issues the manual sound adjustment instruction.

8. The method according to claim 7, characterized in that The generation of data in the first training data sample set includes: When the hearing assist device obtains a manual sound adjustment instruction from the user, the manual sound adjustment and the second sound adjustment parameter obtained at this time are added to the first training sample set.

9. The method according to claim 7, characterized in that: The hearing assistance device is configured to perform online training on the second sound adjustment model using the first training sample set.

10. The method according to claim 9, characterized in that The training of the second sound adjustment model is performed in a server that is communicatively connected to the hearing assistance device, and the hearing assistance device is configured to obtain updated model parameters of the second sound adjustment model from the server.

11. The method according to claim 1, characterized in that: The second sound adjustment model has initial model parameters, and the determination of the initial model parameters includes: Acquiring current user information of the hearing assistive device; Comparing the acquired current user information with information of other users pre-stored in the server; and The parameters of the second sound adjustment model pre-used by other users with the highest similarity to the current user are used as the initial model parameters.

12. The method according to claim 11, characterized in that The current user information is manually input by a current user of the hearing assistance device, and the current user information includes one or more of a region, an age or a gender of the current user.

13. The method according to claim 11, characterized in that The current user information includes a second sound adjustment parameter acquired by the hearing assistance device.

14. The method according to claim 13, characterized in that Acquiring the second sound adjustment parameter by the hearing aid device includes: collecting environmental sound signals through the microphone of the hearing aid device; and At least one second sound adjustment parameter is extracted from the acquired ambient sound signal.

15. The method according to claim 1, characterized in that The second sound adjustment model is a decision tree model, a linear regression model, a convolutional neural network model or a recursive neural network model.

16. The method according to claim 15, characterized in that The second sound adjustment model is trained online in the embedded system.

17. The method according to claim 1, characterized in that The first sound adjustment model is a traditional statistical analysis module.

18. The method according to claim 1, characterized in that The target sound output parameter is a target volume value, and adjusting the sound output of the hearing assist device to have the target sound output parameter is adjusting the sound output of the hearing assist device to have the target volume value.

19. The method according to claim 1, characterized in that The target sound output parameter includes a target volume value of a sound in a specific frequency band or a specific frequency.

20. A hearing aid device with automatic sound output adjustment function, characterized in that: The hearing assistance device is configured to perform the following steps: Acquiring at least one first sound adjustment parameter and at least one second sound adjustment parameter; Inputting the first sound adjustment parameter into a first sound adjustment model, and transmitting an output result of the first sound adjustment model to a second sound adjustment model; determining a target sound output parameter of the hearing assistive device by the second sound adjustment model at least based on the second sound adjustment parameter in combination with an output result of the first sound adjustment model; The sound output of the hearing assistance device is adjusted to have the target sound output parameters.

21. The hearing assistive device according to claim 20, characterized in that: The acquiring of at least one first sound adjustment parameter and at least one second sound adjustment parameter comprises: collecting surrounding sound signals through the microphone of the hearing aid device; and At least one first sound adjustment parameter and at least one second sound adjustment parameter are extracted from the acquired ambient sound signal.

22. The hearing assistive device according to claim 20, characterized in that: The hearing assistance device is configured to perform online training on the second sound adjustment model using the first training sample set.

Citation Information

Patent Citations

  • Volume adjustment method and device for audio device and storage medium

    CN111930336A

  • Audio adjustment method and device, storage medium and electronic equipment

    CN112631542A

  • Hearing aid system based on machine learning

    CN115250415A

  • Wearable hearing assist device with artifact remediation

    US20220369047A1

  • Systems and methods for dynamic noise reduction

    US20220386018A1