Audio processing device and audio processing method

Abstract

Provided is an audio processing device that performs audio processing which is primarily for a hearing aid apparatus.　This audio processing device comprises: an information estimation unit for estimating control information on the basis of hearing threshold information pertaining to the hearing thresholds of frequencies and input audio information; and a signal processing unit for processing an input audio signal on the basis of the estimated control information. The hearing threshold information is the results of threshold measurement in which the audio processing device measures the hearing thresholds of a user at frequencies. The information estimation unit estimates information pertaining to a permissible delay amount or a noise-cancellation property in the signal processing unit.

Description

[Technical Field]

[0001] The technology disclosed in this specification (hereinafter referred to as "the present disclosure") relates to an audio processing device and an audio processing method that perform audio processing mainly for hearing aid devices, and to hearing aid devices. [Background technology]

[0002] To solve the problem of hearing loss in people with hearing impairments and the like, hearing aids that collect and amplify ambient sounds have become widespread. If the ambient sounds contain noise in addition to the desired sound (such as the speaker's speech), the quality of the amplified sound will deteriorate. Therefore, a process to reduce the noise components from the collected sound signal (hereinafter also referred to as "noise reduction (NR) process" in this specification) is often performed (see, for example, Patent Document 1). In addition, based on an audiogram (hearing level for each frequency band) obtained as a result of a hearing test for the user, the amplification function is generally set to amplify sounds in the frequency band in the audible frequency range where hearing is impaired, thereby alleviating the hearing loss. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2016-537891 [Patent Document 2] Japanese Patent Application Publication No. 8-221092 [Non-patent literature]

[0004] [Non-Patent Document 1] Donald G. MacKay, Metamorphosis of a Critical Interval:Age-Linked Changes in the Delay in Auditory Feedback that Produces Maximal Disruption of Speech, The Journal of the Acoustical Society of America, 1967 Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present disclosure is to provide a sound processing device and a sound processing method that perform sound processing primarily for a hearing aid device, and a hearing aid device. [Means for solving the problem]

[0006] The present disclosure has been made in consideration of the above problems, and a first aspect thereof is: an information estimation unit that estimates control information based on hearing threshold information relating to hearing thresholds for each frequency and input audio information; a signal processing unit that processes an input audio signal based on the estimated control information; The audio processing device is provided with:

[0007] The hearing threshold information is a result of a threshold measurement that measures the hearing threshold for each frequency of the user of the sound processing device, or the hearing threshold information is information on the sound pressure level for each frequency of a measurement signal used in a hearing measurement function.

[0008] The information estimation unit estimates information relating to an allowable delay amount in the signal processing unit, or estimates noise canceling characteristic information relating to a noise canceling process performed in the signal processing unit.

[0009] Furthermore, a second aspect of the present disclosure is an information estimation step of estimating control information based on hearing threshold information relating to hearing thresholds for each frequency and input audio information; and a signal processing step of processing an input audio signal based on the estimated control information.

[0010] A third aspect of the present disclosure is a hearing aid device with a hearing measurement function, comprising: When conducting a hearing test, ambient sounds are collected from the hearing aid device; amplifying the collected ambient sound; The hearing aid outputs the amplified ambient sound from a speaker of the hearing aid or an external device connected to the hearing aid.

[0011] The hearing aid device according to the third aspect is configured to pick up ambient sounds when the test sound for hearing test is stopped. If the hearing aid device further includes a noise canceling function, information indicating that noise canceling was used during the hearing test is recorded together with the hearing test results, and when the test results are displayed, the presence or absence of noise canceling is also displayed. [Effects of the Invention]

[0012] According to the present disclosure, it is possible to provide an audio processing device, an audio processing method, and a hearing aid device that perform processing to determine the allowable delay amount and noise canceling characteristic information for signal processing such as noise reduction performed in a hearing aid device, as well as noise canceling characteristic information during hearing measurement.

[0013] It should be noted that the effects described in this specification are merely examples, and the effects brought about by the present disclosure are not limited to these. Furthermore, the present disclosure may also bring about additional effects in addition to the effects described above.

[0014] Further objects, features, and advantages of the present disclosure will become apparent from the following detailed description based on the embodiments and accompanying drawings. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a hearing aid device. [Figure 2] FIG. 2 is a diagram showing an example of the relationship between the input sound pressure level and the output sound pressure level in a hearing aid device. [Figure 3] FIG. 3 is a diagram showing an example in which the hearing threshold of a hearing-impaired person increases (deteriorates). [Figure 4] FIG. 4 shows an example of an audiogram of a person with hearing loss. [Figure 5] FIG. 5 is a diagram showing another example of the relationship between the input sound pressure level and the output sound pressure level in a hearing aid device. [Figure 6] FIG. 6 is a diagram showing an example of the flow of sound and audio signals in a hearing aid device. [Figure 7] FIG. 7 shows an example in which a user 790 of a hearing aid device 701 is face-to-face with a speaker 794 in conversation. [Figure 8] FIG. 8 is a diagram showing how a voice uttered by a speaker reaches the speaker's own ear. [Figure 9] FIG. 9 is a diagram showing an example of a feedforward comb filter. [Figure 10] FIG. 10 is a diagram showing amplitude characteristics of the comb filter shown in FIG. [Figure 11] FIG. 11 is a diagram illustrating the effect of the NR function. [Figure 12] FIG. 12 is a diagram showing an example in which the effect of the NR function is small. [Figure 13] FIG. 13 is a diagram showing another example of the configuration of a hearing aid device. [Figure 14] FIG. 14 is a diagram showing an example configuration (first embodiment) of a hearing aid device with a variable delay NR function. [Figure 15] FIG. 15 is a diagram illustrating an example of generating a trained model. [Figure 16] FIG. 16 is a diagram showing an example of a user interface when creating learning data. [Figure 17]FIG. 17 is a diagram showing an example of a user interface used when a subject (hearing-impaired person) answers the question about the amount of acceptable delay. [Figure 18] FIG. 18 is a flowchart showing the procedure for creating learning data. [Figure 19] FIG. 19 is a diagram showing another example of generating a trained model. [Figure 20] FIG. 20 is a diagram showing yet another example of generating a trained model. [Figure 21] FIG. 21 is a diagram showing an example of the configuration of the allowable delay estimation unit 2126 that does not use a trained model. [Figure 22] FIG. 22 is a flowchart showing a processing procedure for determining an allowable delay amount without using a trained model. [Figure 23] FIG. 23 is a flowchart showing the processing procedure of variable delay NR in a hearing aid device. [Figure 24] FIG. 24 is a diagram illustrating an example of the configuration of a delay amount determination unit that uses a trained model for estimation. [Figure 25] FIG. 25 is a flowchart showing the flow of processing by the delay amount determining unit 2416. [Figure 26] FIG. 26 is a diagram showing an example of the configuration of the delay amount determining unit 2616. As shown in FIG. [Figure 27] FIG. 27 is a flowchart showing the flow of processing by the delay amount determining unit 2616. [Figure 28] FIG. 28 is a flowchart showing an example of a process for suggesting to the user to update the delay amount. [Figure 29] FIG. 29 is a diagram showing an example of a method for proposing delay amount updates to a user of a hearing aid device. [Figure 30] FIG. 30 shows an example of how a hearing aid user can respond. [Figure 31] FIG. 31 is a diagram showing an example of a user interface used when proposing a delay amount update to a hearing aid device user and receiving a response from the hearing aid device user. [Figure 32]FIG. 32 is a diagram showing a specific method for displaying the recommended range. [Figure 33] FIG. 33 is a diagram showing an example of the configuration of the NR unit. [Figure 34] FIG. 34 is a diagram showing an example of variable delay buffer operation. [Figure 35] FIG. 35 is a diagram showing a second configuration example (first embodiment) of a hearing aid device with a variable delay NR function. [Figure 36] FIG. 36 is a diagram showing another example of generating a trained model. [Figure 37] FIG. 37 is a diagram showing an example in which a camera 3797 is arranged on a hearing aid device. [Figure 38] FIG. 38 is a diagram showing yet another example of generating a trained model. [Figure 39] FIG. 39 is a diagram illustrating an example of the configuration of a delay amount determination unit that uses a trained model for estimation. [Figure 40] FIG. 40 is a flowchart showing the flow of processing by the delay amount determining unit 3916. [Figure 41] FIG. 41 is a diagram illustrating the mechanism of noise canceling. [Figure 42] FIG. 42 is a diagram showing an example of the configuration of a hearing aid device equipped with a noise canceling function. [Figure 43] FIG. 43 is a diagram showing an example of the effect when the NC function is added to the NR function. [Figure 44] FIG. 44 shows an example in which the effect of the NC function is small. [Figure 45] FIG. 45 shows an example of an audiogram of a gradual gradual high-frequency impairment. [Figure 46] FIG. 46 shows an example of a mountain-shaped audiogram. [Figure 47] FIG. 47 is a diagram showing the expected noise attenuation amount that can be expected to be attenuated by noise canceling processing. [Figure 48] FIG. 48 shows an example of a horizontal audiogram. [Figure 49]FIG. 49 is a diagram showing an example of input / output characteristics of a hearing aid device. [Figure 50] FIG. 50 is a diagram showing an example of the configuration of a hearing aid device having an NC signal generation function (second embodiment). [Figure 51] FIG. 51 is a diagram illustrating an example of generating a trained model. [Figure 52] FIG. 52 is a diagram showing an example of a user interface used when a subject (hearing-impaired person) answers NC characteristic information. [Figure 53] FIG. 53 is a flowchart showing the procedure for creating learning data. [Figure 54] FIG. 54 is a diagram showing an example of the configuration of the NC characteristic estimation unit 5431 when a trained model is not generated. [Figure 55] FIG. 55 is a flowchart showing the processing procedure for estimating the NC characteristics in the NC characteristics estimating unit 5431. [Figure 56] FIG. 56 is a flowchart showing the processing procedure of the NC function in the hearing aid device 5001. [Figure 57] FIG. 57 is a diagram showing an example of the configuration of an NC characteristic determination unit that estimates NC characteristics using a trained model. [Figure 58] FIG. 58 is a diagram showing an example of the configuration of an NC characteristic determination unit that estimates NC characteristics without using a trained model. [Figure 59] FIG. 59 is a flowchart showing a processing procedure for proposing to the user the update of the NC characteristic curve and the NC strength. [Figure 60] FIG. 60 is a diagram showing an example of a user interface when proposing to the user to update the NC strength and NC characteristic curve and obtaining a response from the user. [Figure 61] FIG. 61 is a diagram showing a method for presenting the recommended range 6099. [Figure 62] FIG. 62 shows an example of conventional hearing measurement. [Figure 63] FIG. 63 is a diagram illustrating the effect of noise on hearing measurement. [Figure 64]FIG. 64 is a diagram illustrating the effect of noise when testing the hearing ability of a hearing-impaired person. [Figure 65] FIG. 65 is a diagram showing an example of a conventional additional hearing test. [Figure 66] FIG. 66 is a diagram illustrating the effect when noise is reduced during hearing measurement. [Figure 67] FIG. 67 shows an example of hearing measurement according to the third embodiment. [Figure 68] FIG. 68 is a diagram showing an example of the amount of noise attenuation. [Figure 69] FIG. 69 is a diagram showing an example in which the noise canceling characteristic curve is changed depending on the frequency of the audiometric signal. [Figure 70] FIG. 70 is a diagram showing an example of the configuration of a hearing aid device that uses the NC signal generation function for hearing measurement. [Figure 71] FIG. 71 is a diagram showing an example of generating a trained model. [Figure 72] FIG. 72 is a diagram showing an example of a user interface when creating learning data. [Figure 73] FIG. 73 is a diagram showing an example of a user interface used when a subject (hearing-impaired person) answers NC characteristic information. [Figure 74] FIG. 74 is a flowchart showing the procedure for creating learning data. [Figure 75] FIG. 75 is a diagram showing an example of the configuration of the NC characteristic estimation unit 7531 when a trained model is not generated. [Figure 76] FIG. 76 is a flowchart showing the processing procedure for estimating the NC characteristics in the NC characteristics estimating unit 7531. [Figure 77] FIG. 77 is a flowchart showing the processing procedure of the NC function in the hearing aid device 7001. [Figure 78] FIG. 78 is a diagram showing an example of the configuration of an NC characteristic determination unit that estimates NC characteristics using a trained model. [Figure 79] FIG. 79 is a diagram showing an example of the configuration of an NC characteristic determination unit that estimates NC characteristics without using a trained model. [Figure 80] FIG. 80 shows another example of a user interface used to propose updates to NC strengths and NC characteristic curves and receive responses. [Figure 81] FIG. 81 shows another example of a user interface that presents recommended ranges and obtains responses from the hearing aid device user. [Figure 82] FIG. 82 is a flowchart showing the processing procedure for suggesting a change in the sound environment to the user. [Figure 83] FIG. 83 is a diagram showing an example of notifying a user of a hearing aid device of noise that may affect hearing measurement. [Figure 84] FIG. 84 shows an example of a display of hearing test results. [Figure 85] FIG. 85 shows an example of a display of hearing test results. [Figure 86] FIG. 86 shows an example of a display of hearing test results. [Figure 87] FIG. 87 shows an example of a display of hearing test results. [Figure 88] FIG. 88 shows an example of a display of hearing test results. [Figure 89] FIG. 89 shows an example of a display of hearing test results. DETAILED DESCRIPTION OF THE INVENTION

[0016] The present disclosure will now be described with reference to the drawings.

[0017] FIG. 1 shows a schematic diagram of an example of the configuration of a hearing aid device. The illustrated hearing aid device 101 includes a microphone 102 and a speaker (also called a receiver) 103. Sound picked up by the microphone 102 is converted into an electrical signal by the microphone 102, and after signal processing, the electrical signal is converted back into sound by the speaker 103. The sound output from the speaker 103 is emitted into an ear canal 191 and reaches the eardrum at the back. An ADC (analog-to-digital converter) 104 converts the analog signal from the microphone 102 into a digital signal and sends it to a signal processor 106. The signal processor 106 performs signal processing such as adjusting the amplitude and phase characteristics and adjusting the gain. A DAC (digital-to-analog converter) 105 converts the digital signal into an analog signal and sends it to the speaker 103. In Figure 1, the hearing aid device 101 is shown shaped to cover the entrance to the ear canal, but it can also be an in-ear or behind-the-ear type, a TWS (true wireless stereo) type with a hear-through (external sound capture) function, a headphone type, a canal type that blocks the ear canal, or an open-ear type that does not completely block the ear canal.

[0018] FIG. 2 shows an example of the relationship between input sound pressure level and output sound pressure level in a hearing aid device. The input sound pressure level refers to the level of sound entering the microphone, and the output sound pressure level refers to the level of sound output from the speaker. The gain curve indicated by reference number 280 is the gain relative to the input sound pressure level. In the example shown in FIG. 2, an input sound pressure level range 281 is converted into an output sound pressure level range 282 by the gain curve 280. For example, if an input sound pressure level of 40 dB SPL ( SoundWhen the input sound pressure level is 80 dB SPL, a 30 dB gain is applied, resulting in an output sound pressure level of 70 dB SPL. When the input sound pressure level is 80 dB SPL, a 10 dB gain is applied, resulting in an output sound pressure level of 90 dB SPL. Although people with sensorineural hearing loss, such as those with age-related hearing loss, have difficulty hearing soft sounds, their perceived loudness of sounds is not significantly different from that of people with normal hearing (characteristic A1). For this reason, a method is used in which a large gain is applied to soft input sound pressure levels and a small gain is applied to soft input sound pressure levels. This method began to spread around 2000 and is used in many currently popular hearing aids. In the example shown in Figure 2, the gain curve 280 is composed of three line segments. However, this is not limited to this, and various methods exist, such as changing the number of line segments or using curved lines.

[0019] In this way, hearing aid devices have the feature (feature B1) of changing the gain depending on the input sound pressure level.

[0020] Figure 3 shows an example of an increase (deterioration) in the hearing threshold of a hearing-impaired person. The horizontal axis of Figure 3 represents frequency, and the vertical axis represents sound pressure level. The hearing threshold refers to the threshold at which a sound is barely audible. The curve indicated by reference numeral 383 represents the hearing threshold of a person with normal hearing, and the curve indicated by reference numeral 384 represents the hearing threshold of a person with a hearing impairment. Reference numeral 385 indicates a 40 dB increase in the hearing threshold 384 of the hearing-impaired person compared to the hearing threshold 383 of the person with normal hearing. In other words, the hearing of the hearing-impaired person is worsened by 40 dB. In this case, a person with normal hearing can hear pure tone 386, but a hearing-impaired person with hearing threshold 384 cannot hear pure tone 386. The intensity of a sound that is too loud to bear is called the pain threshold, which is generally around 130–140 dB SPL. The range between the hearing threshold and the pain threshold is called the auditory field, and this range is the range in which sound can be heard. Because the pain threshold is not significantly different between people with normal hearing and people with hearing loss, people with hearing loss have a narrower hearing field and a narrower range of sound intensities than people with normal hearing. For this reason, people with hearing loss have characteristic A1, and to address this, hearing aid devices have characteristic B1.

[0021] Figure 4 shows an example of an audiogram for a person with hearing loss. The audiogram shows the degree to which the hearing threshold of a person with hearing loss has deteriorated compared to that of a person with normal hearing, for each frequency. The horizontal axis of Figure 4 represents frequency, and the vertical axis represents hearing level; however, please note that these are upside down compared to Figure 3. In the example shown in Figure 4, hearing is impaired by 15 dB at 1000 Hz and 40 dB at 4000 Hz. This is generally expressed as a hearing level of 15 dB HL (Hearing Level) at 1000 Hz and 40 dB HL at 4000 Hz. Hearing aid devices often amplify signals at each frequency to improve the hearing impairment of people with hearing loss. There are various methods for determining the amplification gain, but a guideline is to set it to approximately 1 / 3 to 1 / 2 of the hearing level. If the gain for an input sound pressure level of 60 dB SPL is set to half the hearing level, then in the example shown in FIG. 4, the gain at 1000 Hz is determined to be 7.5 dB and the gain at 4000 Hz is determined to be 20 dB.

[0022] Figure 5 shows another example of the relationship between input sound pressure level and output sound pressure level in a hearing aid device (however, in the figure, reference number 580 indicates a gain curve, and input sound pressure level range 581 is converted into output sound pressure level range 582 by gain curve 580). In the case of the hearing-impaired person shown in Figure 4, for example, the gain curve at 1000 Hz will be like the gain curve indicated by reference number 580 in Figure 5. On the other hand, the gain curve at 4000 Hz will be like the gain curve indicated by reference number 280 in Figure 2.

[0023] In this way, hearing aid devices have the feature of changing the gain to suit the hearing ability of the user (feature B2).

[0024] 6 shows an example of the flow of sound and audio signals in a hearing aid device. For example, sound at position P1 reaches position P2 in ear canal 691 via two main paths. The first path passes through microphone 602, and the second path does not pass through microphone 602. Sound on the second path includes sound passing through the vent hole of hearing aid device 601, sound passing through the gap between hearing aid device 601 and the ear canal, sound passing through the housing of hearing aid device 601, and sound passing through the earpiece (also called a dome). Sound on the first path is picked up by microphone 602, passes through ADC 604, signal processor 606, and DAC 605, and is output from speaker 603. At this time, delays occur in ADC 604, signal processor 606, and DAC 605, so the sound on the first path is generally delayed compared to the sound on the second path.

[0025] In this way, the hearing aid device has the feature that there is a second path in addition to the first path (feature B3), and the feature that the sound on the first path is delayed relative to the sound on the second path (feature B4).

[0026] The above-mentioned characteristics B1 to B4 of the hearing aid device cause problems in the hearing aid device. First, the effect of characteristic B4 (the characteristic that sound from the first path is delayed) will be described.

[0027] FIG. 7 shows an example of a conversation between a user 790 of a hearing aid device 701 and a speaker 794. In the figure, the arrow indicates the audio transmitted from the speaker 794 to the hearing aid device user 790, i.e., the other person's voice. If the delay in the hearing aid device 701 is large, the hearing aid user 790 will experience a mismatch between the speaker 794's mouth movements and the audio of the speaker 794 transmitted through the hearing aid device 701. Humans are sensitive to this mismatch and perceive it as unnatural. For example, in the broadcasting field, the maximum delay between video and audio is specified. RECOMMENDATION ITU-R BT.1359-1 (1998) specifies that a delay of 125 milliseconds or more between audio and video is perceived as unnatural, with an acceptable limit of 185 milliseconds. In Europe, EBU Technical Recommendation R37-2007 specifies a maximum delay of 60 milliseconds between audio and video. For percussion, it is perceived at 60 ms, with some reports suggesting a tolerance limit of 142 ms. In broadcasting, viewers are expected to concentrate on listening to what is happening on the other side of the screen, but in the conversation example shown in Figure 7, smooth dialogue and gesture interaction are required. The acceptable sound delay may be even shorter.

[0028] Figure 8 shows how a speaker's voice reaches the speaker's ear. In the figure, the speaker's voice, or their own voice, is indicated by an arrow pointing from the mouth at the front of the head to the ear. It is believed that when a person speaks, they do not simply speak; rather, the brain utilizes the sound they hear. Delaying the sound produced by a speaker before it reaches the ear is called delayed auditory feedback (DAF), and it can cause the speaker to stutter. For adults, a delay of around 200 milliseconds is considered the most severe form of stuttering (see Non-Patent Document 1). Even a delay shorter than 200 milliseconds can make speaking difficult. This phenomenon can also be encountered in situations where your own voice is delayed during a telephone call or video conference, where it is returned later.

[0029] Next, the influence of feature B3 (the feature that a second route exists in addition to the first route) will be described.

[0030] FIG. 9 shows an example of a feedforward comb filter. The illustrated comb filter is configured such that the input signal x[n] is amplified by a factor α in a first path that includes a delay, while the input signal x[n] is amplified by a factor β in a second path that does not include a delay. These signals are then mixed to output the signal y[n]. FIG. 10 also shows the amplitude characteristics of the comb filter shown in FIG. 9 when α = 1 and β = 1, using solid lines. Large dips are periodically observed. Assuming a delay of 1 millisecond is represented by the solid line, the frequency of the first dip is 500 Hz, and the frequency of the second dip is 1500 Hz. As the delay increases, the intervals between the dips become narrower. For example, the dashed line in FIG. 10 represents a delay twice as long as that represented by the solid line. When the delay is between 1 millisecond and 25 milliseconds, the sound of the output signal y[n] becomes noticeably metallic, with a noticeable change in timbre. As the delay increases, the change in timbre becomes less noticeable, and instead, the echo becomes more noticeable. The sensitivity to the echo also depends on the sound source. Percussion-like sounds tend to be more noticeable. The first path in Figure 9 corresponds to the first path in Figure 6, and the second path in Figure 9 corresponds to the second path in Figure 6.

[0031] When a sound from position P1 in Figure 6 reaches position P2, there is a delay along the first path, so the sound along the first path arrives delayed after the sound along the second path. Depending on the amount of delay, an echo may be perceived. Echoes can interfere with speech intelligibility. Since the primary purpose of hearing aids is to improve speech intelligibility, echo suppression is a key issue. For example, for people with normal hearing, echoes are perceived after an average of 7.1 milliseconds (standard deviation 2.5 milliseconds) for other people's voices and an average of 4.8 milliseconds (standard deviation 1.6 milliseconds) for one's own voice. For people with hearing loss, these delays are slightly longer, with echoes perceived after an average of 8.9 milliseconds (standard deviation 3.8 milliseconds) for other people's voices and an average of 6.3 milliseconds (standard deviation 2.6 milliseconds) for one's own voice. Other people's voices refer to hearing someone else's voice, as shown in Figure 7, and one's own voice refers to hearing one's own voice, as shown in Figure 8.

[0032] In the comb filter shown in Figure 9, for example, let's assume α = 100 and β = 0.5. In Figure 6, this corresponds to the case where the sound intensity of the first path is much stronger than the sound intensity of the second path (high sound pressure ratio). In the case of a hearing aid device, this corresponds to a case where the amplification gain is high. In this situation, the influence of feature B3 (the existence of a second path in addition to the first path) is not significant, and the influence of feature B4 (the delay of the sound from the first path) is dominant. In other words, the challenges are to prevent discomfort and communication difficulties caused by a discrepancy between visual and auditory information, and to prevent difficulty in speaking due to delayed auditory feedback. Based on broadcasting standards, the allowable delay amount for the first path is, for example, up to 60 milliseconds.

[0033] Also, in the comb filter shown in Figure 9, let's assume α = 1 and β = 0.5. In Figure 6, this corresponds to a case where the sound intensity of the second path and the first path is not significantly different (small sound pressure ratio). In the case of a hearing aid, this corresponds to a case where the amplification gain is very small. In this situation, in addition to the influence of feature B4 (the characteristic that the sound of the first path is delayed), the effect of feature B3 (the characteristic that a second path exists in addition to the first path) is also present. In other words, preventing echoes that interfere with speech intelligibility becomes an additional challenge. The amount of delay of the first path is allowed to be up to, for example, 6 milliseconds.

[0034] As described above, the allowable delay in the signal processing of the hearing aid device varies depending on the sound pressure ratio of the first path to the second path. This sound pressure ratio is affected by features B1 and B2. For example, feature B1 (which changes the gain depending on the input sound pressure level) causes the sound pressure ratio to be large when the input sound pressure level is low, and small when the input sound pressure level is high. In the example shown in Figure 2, the gain is 5 dB at 90 dB SPL, and the sound pressure ratio is small even when the hearing level is not low. Feature B2 (which changes the gain according to the user's hearing ability) causes the sound pressure ratio to be small when the hearing level is close to that of a normal hearing person, that is, when the hearing level is low. In the examples shown in Figures 4 and 5, when the hearing level is 15 dB HL, the sound pressure ratio is small regardless of the input sound pressure level. Although the sound pressure ratio is used here, the sound pressure ratio and the sound pressure level difference are the same thing, and either expression is acceptable.

[0035] The primary function of hearing aids is to amplify sound to match the hearing ability of the hearing-impaired individual. This improves hearing during conversations in quiet environments. However, in noisy environments, such as restaurants or shopping malls, the ambient noise is amplified along with the speech. In general, many hearing-impaired individuals have difficulty hearing speech in noisy environments compared to individuals with normal hearing (Feature A2). This is believed to be largely due to a decline in the function of the retrolabyrinth (all auditory pathways posterior to the labyrinth). Given this background, many currently available hearing aids are equipped with a noise reduction (NR) function. The NR function suppresses noise contained in the audio signal picked up by the microphone. This is achieved by removing noise components from the input audio signal through signal processing. NR processing improves the signal-to-noise ratio (SNR) of the audio, thereby alleviating speech intelligibility.

[0036] Fig. 11 illustrates the effect of the NR function. Positions P1 and P2 in Fig. 11 correspond to positions P1 and P2, respectively, in Fig. 6. Also, the first path and second path in Fig. 11 correspond to the first path and second path, respectively, in Fig. 6.

[0037] The spectrogram denoted by reference numeral 1170 represents a sound at position P1. This spectrogram 1170 consists of a speech spectrum 1174 and noise 1175. The difference between the peak of speech spectrum 1174 and the peak of noise 1175 is called difference d1. The larger the difference d1, the better the SNR. The difference d1 in spectrogram 1170 is small, meaning that the speech is only slightly louder than the noise, and a person with hearing loss will have difficulty hearing it.

[0038] Reference numeral 1172 denotes a spectrogram of the sound arriving at position P2 via the second path. The shape of spectrogram 1172 is affected by the housing and earpiece of the hearing aid device, but for simplicity of explanation, these effects will be ignored here. Spectrogram 1172 is approximately equal to spectrogram 1170.

[0039] On the other hand, in the first path, an NR function indicated by reference numeral 1107 is applied together with amplification indicated by reference numeral 1177. The amplification and NR processing are performed in the signal processing unit. Reference numeral 1171 indicates a spectrogram of the sound that arrives at position P2 via the first path. The NR function widens the difference d1 in spectrogram 1170 to the difference dNR in spectrogram 1171, improving the SNR.

[0040] Spectrogram 1172 and spectrogram 1171 are mixed at position P2. Reference numeral 1173 indicates the spectrogram of the sound at position P2. The difference d2_1 of spectrogram 1173 is close to the difference dNR. It can be seen that the SNR at position P2 is significantly improved compared to position P1. In other words, d1< <d2_1である。

[0041] Fig. 12 shows an example in which the effect of the NR function is small. Positions P1 and P2 in Fig. 12 correspond to positions P1 and P2, respectively, in Fig. 6. Also, the first path and second path in Fig. 12 correspond to the first path and second path, respectively, in Fig. 6.

[0042] Spectrogram 1270 represents the sound at position P1 and consists of a speech spectrum 1274 and noise 1275. The difference d1 between the peak of speech spectrum 1274 and the peak of noise 1275 in spectrogram 1270 is small, meaning that the speech is only slightly louder than the noise, and a person with hearing loss would have difficulty hearing it.

[0043] Reference numeral 1272 indicates a spectrogram of sound arriving at position P2 via the second path. On the other hand, in the first path, the NR function is applied as indicated by reference numeral 1207, but there is no amplification (gain: 0 dB) as indicated by reference numeral 1277. Reference numeral 1271 indicates a spectrogram of sound arriving at position P2 via the first path.

[0044] Spectrogram 1272 and spectrogram 1271 mix at position P2, and the sound at position P2 becomes spectrogram 1273. The NR function widens the difference d1 in spectrogram 1270 to the difference dNR in spectrogram 1271, improving the SNR. However, the difference d2_2 in spectrogram 1273 is close to the difference d1. It can be seen that the improvement in SNR at position P2 is only slight. In other words, d1 ≒ d2_2.

[0045] While the SNR at position P2 in the example shown in Figure 11 is significantly improved, the reason for only a slight improvement in the example shown in Figure 12 can be seen from the fact that the vertical axis of the spectrogram is in dB. This is because when two sounds displayed in dB are added, the louder sound becomes dominant. Thus, when the gain of the first path is large, using the NR function significantly improves the SNR at position P2, thereby alleviating the listening difficulty of the hearing aid user. On the other hand, when the gain of the first path is small, using the NR function only slightly improves the SNR at position P2, and the listening difficulty of the hearing aid user is also slightly improved.

[0046] As described above, the effectiveness of the NR function in improving the SNR at position P2 depends on the amplification gain of the hearing aid device. As can be seen from the examples shown in Figures 2 and 5, the amplification gain depends on the hearing ability of the hearing aid user and the input sound pressure to the hearing aid device.

[0047] There are various noise reduction techniques for speech signals. While we will not go into detail here, one commonly used technique that works effectively against stationary noise is spectral subtraction (see, for example, Patent Document 2). When using NR techniques in hearing aid devices, real-time processing is essential, but generally, a longer delay is required to achieve higher noise reduction performance. Therefore, determining the acceptable range of delay is a very important factor. Attempting to forcibly suppress stationary noise without increasing the delay can result in significant side effects, worsening audibility. For example, in spectral subtraction, if an attempt is made to suppress more stationary noise by overestimating the noise spectrum to be subtracted without increasing the delay, side effects can result in the generation of a lot of musical noise or even the deletion of necessary phonemes in speech, which can actually impair audibility. To suppress more noise without adversely affecting audibility, it is effective to apply NR techniques while allowing for a longer delay.

[0048] There are other examples where allowing for a long delay is effective. If a hearing aid amplifies sudden, loud sounds (sudden sounds), such as a door slamming, in the same way as speech, the hearing aid user will be startled by the sudden, powerful sound. People with sensorineural hearing loss, such as those with age-related hearing loss, have characteristic A1 (they have difficulty hearing soft sounds, but the loudness of sounds they perceive as noisy is not much different from that of people with normal hearing). For this reason, sudden sounds are even more unpleasant for hearing aid users than for people with normal hearing. As shown in Figure 2, hearing aids operate by applying a small gain to loud sounds, but their effectiveness against sudden sounds is limited. This process has a time constant, which is due to the primary purpose of improving speech comprehension. Many currently available hearing aids also include a function to suppress sudden sounds. The sudden sound suppression function essentially monitors the level of the sound input from the microphone and reduces the gain when the sound suddenly becomes louder. However, if you try to do this with a short delay, it may happen that the gain reduction process is not completed in time. If you try to make up for it by setting a short tracking time, the side effect is that the sound will fluctuate. Generally, a longer delay is required to suppress sudden sounds more stably and reliably. For this reason, determining the acceptable range of delay is a very important factor.

[0049] As described above, a shorter delay in the first path is better for suppressing echoes and the like, but a longer delay is better for suppressing noise and sudden sounds. Conventional hearing aids typically have a fixed delay that each hearing aid manufacturer considers optimal. Specifically, the target delay is 10 to 12 milliseconds or less. However, considering features B1 to B4, making the delay variable can result in hearing aids that are more satisfying for users.

[0050] For example, for a hearing-impaired person with a hearing level of 40 dB HL, in a sound environment of 50 dB SPL, the amplification gain is large and the influence of the sound from the second path is relatively small, so there is little echo effect. In an environment where the noise level is high relative to speech, satisfaction can be increased by activating the NR function more strongly, even if it means increasing the delay. Conversely, even for a hearing-impaired person with a hearing level of 40 dB HL, in an environment of 80 dB SPL, the amplification gain is small and the influence of the sound from the second path is relatively large, resulting in echo effect. Satisfaction can be increased by shortening the delay, even if it means weakening the NR effect.

[0051] Figure 13 shows another example of the configuration of a hearing aid device. The hearing aid device 1301 shown in Figure 13 is configured by adding a sound source storage and playback unit 1312 and an external interface 1313 to the hearing aid device shown in Figure 1. The sound source storage and playback unit 1312 plays sound sources that have been pre-stored in built-in memory, for example, like an audio player. If necessary, a signal processing unit 1306 performs signal processing to suit the hearing loss, and a speaker 1303 converts the signal into sound. The sound source can also include a message indicating low battery level, as in conventional hearing aid devices. The external interface 1313 is connected wirelessly to a television or the like, for example, and receives audio from the television or the like. If necessary, the signal processing unit 1306 performs signal processing to suit the hearing loss, and a speaker 1303 converts the signal into sound. Sound Source memory When an audio signal from the playback unit 1312 or external interface 1313 is output from the speaker 1303, whether or not to also output the sound picked up by the microphone 1302 is up to the user of the hearing aid device. For example, in case someone speaks to you, you can superimpose the audio signal picked up by the microphone 1302 onto the audio signal from a television or the like. For example, if you want to concentrate on watching television, you can stop superimposing the audio signal picked up by the microphone 1302. In the latter case, no echo is generated.

[0052] As shown in Figure 13, hearing aids have the characteristic that the priority of required functions changes depending on the usage scenario (Feature B5). Because the primary purpose of hearing aids is to improve speech comprehension, Feature B2 (adjusting gain to suit the user's hearing ability) allows the hearing aid to have a gain setting that matches the user's hearing ability. Many current hearing aids control gain according to the input sound pressure, as shown in Figure 2, due to Feature B1 (adjusting gain depending on the input sound pressure level). Even hearing aids without NR functionality can be useful in usage scenarios, such as when a person with mild to moderate hearing loss enjoys conversations with family in a quiet environment such as at home. On the other hand, NR functionality is necessary to alleviate listening difficulties in usage scenarios where conversations are drowned out by noise, such as in noisy environments such as restaurants, supermarkets, and izakayas.

[0053] There are some usage scenarios where noise is loud but conversation is not a high priority. For example, when commuting on trains or buses, it is enough to be able to hear announcements on the train, and conversations are not frequent. In these cases, a high level of effectiveness in the NR function is preferred, even if it means a slightly longer delay.

[0054] There are also usage scenarios that are not location-dependent. For example, when making a phone call, the person on the other end is not directly in front of you, so there is no need to worry about the time lag between mouth movements and sound. Telephones themselves have a long delay. ITU-T Recommendation G.114 (05 / 2003) on international telephone connections and lines states that a delay of less than 150 milliseconds is acceptable for most voice applications, and recommends that it should not exceed 400 milliseconds at most. When making a phone call on a train platform, for example, it is desirable for the NR function to function properly even if the delay is somewhat longer. However, when a telephone and a hearing aid communicate wirelessly, NR is not required for the sound picked up by the microphone, but NR may be required for the telephone audio.

[0055] There are also meaningful usage scenarios. For example, when enjoying a movie, the noise contained in the movie is important for enhancing the sense of realism of the movie, and suppressing the noise too much will reduce the enjoyment of the movie. On the other hand, it is also important to be able to hear the dialogue. One possible solution is to weaken the NR function across the board, or to strengthen the NR effect only in parts with dialogue.

[0056] As mentioned above, the priority of necessary functions changes depending on the usage scenario. Therefore, appropriately adjusting the delay amount of the first path depending on the usage scenario is very important in increasing the satisfaction of hearing aid device users.

[0057] To summarize the above, people with hearing loss have two characteristics: A1 and A2.

[0058] Feature A1: Although it is difficult to hear small sounds, the loudness of sounds that people perceive as noisy is not much different from that of people with normal hearing. Feature A2: Difficulty in understanding speech in noise compared to people with normal hearing

[0059] Furthermore, the hearing aid device has five features B1 to B5.

[0060] Feature B1: The feature of changing the gain depending on the input sound pressure level Feature B2: The feature of changing the gain according to the user's hearing ability Feature B3: The existence of a second route in addition to the first route Feature B4: The sound of the first path is delayed. Feature B5: The priority of required functions changes depending on the usage scenario.

[0061] Features B1 to B4 cause discomfort due to a discrepancy between visual and auditory information, stuttering due to delayed auditory feedback, and difficulty in listening due to echoes, so it is necessary to shorten the delay. On the other hand, features A1 and A2 require a longer delay to effectively activate the NR function and sudden sound suppression function in order to alleviate listening difficulties. By appropriately varying the delay based on the volume of environmental sounds, the gain curve, the sound pressure ratio of the first path to the second path, and the like, the satisfaction of the hearing aid device user can be improved. Furthermore, as in feature B5, by taking into account the usage scenario when controlling the delay, the needs of the hearing aid device user can be more precisely met. Therefore, in the first embodiment of the present disclosure, the satisfaction of the hearing aid device user is improved by controlling the delay of the first path.

[0062] Furthermore, due to features B1 to B3, even if the NR function is used, when the amplification gain of the first path is small, the improvement in SNR at position P2 is slight. Even in such cases, taking features A1 to A2 into account, it is necessary to improve the listening experience for users of hearing aid devices. To this end, a second embodiment of the present disclosure utilizes an NC (Noise Cancelling) function. By automatically setting the NC function, automatically switching on and off, automatically switching between strengths and weaknesses, automatically changing characteristics, and presenting recommended settings on the user interface, it is possible to achieve important power savings for hearing aid devices while also improving the listening experience for users of hearing aid devices.

[0063] Furthermore, due to feature B2, the hearing aid device must be able to grasp the user's hearing ability. In recent years, hearing ability has been measured using the hearing aid device itself, instead of traditional dedicated hearing measurement equipment. However, hearing measurement must be performed in a quiet environment, such as a soundproof room. This is due to feature B3 of the hearing aid device. In a noisy environment, the noise masks the test sound, hindering accurate measurement. To improve the satisfaction of hearing aid device users, the third embodiment of the present disclosure enables hearing measurement even in somewhat noisy environments.

[0064] It should be noted that, in this specification, the NR function is a function realized by signal processing to remove noise components contained in an input audio signal (i.e., suppress noise in the audio signal), while the NC function is a function that cancels out spatial noise with an NC signal that has the same amplitude as the noise component but is phase-inverted, by superimposing the NC signal on the desired signal after audio output. Therefore, it should be noted that the NR function and the NC function are completely different functions. The NR function outputs a signal from which noise has been removed, so the NR effect is maintained even with delay. In contrast, if the delay is large, the NC function cannot cancel out the noise due to the time difference between the noise and the NC signal, and the NC effect is lost. Additionally, because earpieces can block high-frequency noise, the NC function is primarily used to cancel noise below 1 kHz, which cannot be blocked. [Example]

[0065] In the first embodiment of the present disclosure, the delay amount of the first path is controlled to improve the satisfaction of the user of the hearing aid device.

[0066] FIG. 14 shows an example configuration of a hearing aid device 1401 with a variable delay NR function according to the first embodiment. In FIG. 14, a microphone 1402, an ADC 1404, a DAC 1405, and a speaker 1403 are drawn in a straight line from left to right, but this configuration is the same as that of the hearing aid device 1301 shown in FIG. 13. The sound picked up by the microphone 1402 is converted into an electrical signal by the microphone 1402. The ADC 1404 converts the analog signal from the microphone 1402 into a digital signal. The NR unit 1407 suppresses noise in the audio signal. The NR unit 1407 is controlled by a control unit 1411, as will be described in detail later. The amplitude / phase adjustment unit 1409 adjusts the amplitude, phase, etc. of the audio signal after NR processing. The control unit 1411 configures the amplitude / phase adjustment unit 1409 using a hearing information storage unit 1415 and an audio signal input to the amplitude / phase adjustment unit 1409. Specifically, the control unit 1411 determines the gain for each frequency according to the gain curves shown in Figures 2 and 5. The hearing information stored in the hearing information storage unit 1415 may be the gain curves shown in Figures 2 and 5, or may be hearing test results such as those shown in Figure 4. In the case of hearing test results, the process also includes obtaining the gain curve from the hearing test results. The DAC 1405 converts the digital signal into an analog signal. The speaker 1403 converts the electrical signal into sound. The delay amount of the sound output from the speaker 1403 changes in the same way as the delay amount of the variable delay NR function changes.

[0067] The sound source storage and playback unit 1412 plays stored sound sources. As with conventional hearing aid devices, sound sources can be voice messages when the battery is low, voice messages when switching programs, and sound sources for hearing tests. It is also possible to store music sound sources, like an audio player.

[0068] The external interface 1413 is responsible for transmitting and receiving audio signals and other data. For example, in a hearing aid device 1401 with independent left and right ears, the external interface 1413 can be used to transmit an audio signal picked up from a microphone to the other ear. For example, when watching television via a wireless connection, the external interface 1413 can be used to transmit television audio signals and connection information indicating that the device is connected to the television. Data stored in the sound source storage and playback unit 1412, configuration information storage unit 1414, and hearing information storage unit 1415 may also be received via the external interface 1413. Data stored in the trained model storage unit 1417 may also be received via the external interface 1413. The external interface 1413 can also be used to transmit information from the hearing aid device 1401 to the user of the hearing aid device 1401 and to receive information from the user of the hearing aid device 1401.

[0069] The configuration information storage unit 1414 stores the configuration information of the hearing aid device 1401. The configuration information of the hearing aid device 1401 includes, for example, information indicating the wearing method, such as whether the hearing aid device is worn on one ear or both ears and whether it is an in-the-ear type or a behind-the-ear type, and information regarding the characteristics of the housing of the hearing aid device 1401, such as the type of earpiece (dome) and the shape of the in-the-ear type or a behind-the-ear type. Whether the hearing aid device is worn on one ear or both ears makes a difference, for example, when detecting the user's own voice. The type of earpiece greatly affects the characteristics of the sound of the second path shown in FIG. 13 (i.e., the sound that cannot be blocked by the earpiece).

[0070] The hearing information storage unit 1415 stores the hearing information of the individual user of the hearing aid device 1401. For example, this information includes information on the results of a hearing test and gain information calculated from the results. For example, the gain information may use the gain curve 280 shown in FIG. 2.

[0071] The delay amount determination unit 1416 determines the amount of delay allowed in signal processing in the NR unit 1407, amplitude / phase adjustment unit 1409, etc., based on the hearing information, input audio information, and configuration information. The delay amount determination unit 1416 can determine the amount of delay allowed using a trained model that has been trained in advance using, for example, the hearing information, input audio information, and configuration information as explanatory variables and the allowable delay amount information as a target variable.

[0072] The trained model storage unit 1417 has trained model information that is used when determining the allowable delay amount in the delay amount determination unit 1416. Trained model information may be acquired via the external interface 1413 and stored in the trained model storage unit 1417. It is also possible to store a trained model that has been retrained by the hearing aid device 1401 in the trained model storage unit 1417.

[0073] FIG. 15 shows an example of a trained model used in this embodiment. This trained model 1501 is used in the delay amount determination unit 1416. It receives hearing information, input audio information, and configuration information, estimates allowable delay information, and outputs it. That is, the trained model 1501 uses hearing information, input audio information, and configuration information as explanatory variables and the allowable delay information as a target variable. FIG. 15 shows a trained model with three input nodes, one output node, and a neural network with two intermediate layers, but this is not limited to this. The trained model 1501 can be generated by a server or personal computer separate from the hearing aid device 1401. The generated trained model 1501 may be transmitted to the hearing aid device 1401 via the external interface 1413 of the hearing aid device shown in FIG. 14. Note that the trained model 1501 may also be generated on the hearing aid device 1401 side (so-called edge AI). The explanatory variables and target variables of the trained model 1501 will now be described.

[0074] The hearing information can be measurement result information from pure tone audiometry. In this specification, no distinction is made between pure tone audiometry and pure tone audiometry. Pure tone audiometry may be at least one of air conduction audiometry and bone conduction audiometry. The audiogram example shown in FIG. 4 is an example of the measurement result of air conduction audiometry. Instead of the measurement result information from pure tone audiometry, gain information calculated using a prescription formula based on the measurement result information can also be used. Gain information refers to the gain curve 280 shown in FIG. 2, for example. Prescription formulas that can be used include NAL-NL2 (National Acoustic Laboratories non-linear 2) and DSLv5 (Desired Sensation Level version 5), which are widely used in conventional hearing aids. The prescription formula is not limited to these, and any formula can be used as long as it determines a gain that matches the hearing ability.

[0075] The hearing information may be the results of a pure tone audiometry test or a self-recorded audiometry test (in which the subject presses a button to change the volume of a sound and the change is recorded). In other words, it may be the results of a threshold test that measures the hearing threshold for each frequency.

[0076] In addition to threshold measurements, suprathreshold speech audiometry results may also be used. For example, a speech discrimination score (Speech Recognition Score) may be used. The Speech Discrimination Score is the result of a speech discrimination test. Speech audiometry, including speech discrimination tests, is outlined in ISO 8253-3. In Japan, the Japan Audiological Society has established the "Speech Audiometry 2003" guideline, based on ISO 8253-3 (1998). A test using distorted speech sounds in a standard speech audiometry is called a distorted speech audiometry. A speech discrimination score obtained from a speech discrimination test using distorted speech sounds may also be used. It is widely known that patients with neural hearing loss tend to have a lower speech discrimination score, especially the distorted speech discrimination score. Furthermore, it is widely known that patients with neural hearing loss tend to have a greater difficulty hearing words in noise. This can be used as a good reference data for determining the strength of NR, or in other words, the amount of NR delay. The speech discrimination score is a numerical value ranging from 0% to 100%. In this specification, no distinction is made between speech discrimination measurement and speech discrimination test, or speech audiometry and speech audiometry test. Here, the speech discrimination score is mentioned as a means of measuring the tendency to have difficulty hearing words in noise, but in essence, it is sufficient if it is a result that measures the degree of difficulty in hearing words in noise.

[0077] The range of hearing information training data is explained below. The range of pure-tone audiometry measurements is -20 dB HL to 120 dB HL, as shown in the audiogram in Figure 2. The gain is broadly estimated to be 0 dB to 80 dB for all input audio levels. These ranges can be used as training data ranges. For example, pure-tone audiometry measurements can be set to -20 dB HL to 120 dB HL. On the other hand, even within the range of pure-tone audiometry measurements, there is no useful information below 0 dB HL when considering use with hearing aid devices. For average hearing levels above 90 dB HL, the hearing aid effect of hearing aid devices is weak, and this level is considered the appropriate level for cochlear implants. Taking these factors into consideration, it is effective to set the training data range to, for example, 0 dB HL to 90 dB HL or 0 dB HL to 80 dB HL. Values ​​outside the range can be rounded to the boundary value.

[0078] The input audio information can be an audio signal collected from the microphone 1402 of the hearing aid device 1401. Instead of an audio signal, a level value converted to a sound pressure level or the like can also be used. Furthermore, the information may be in the form of an amplitude spectrum or a power spectrum. For frequency domain signals such as amplitude spectra and power spectra, a conversion from a time domain signal to a frequency domain signal is required. This conversion can be performed using, for example, an FFT (fast Fourier transform). In the hearing aid device 1401, signal processing tailored to hearing ability is often performed in the frequency domain, as in the amplitude / phase adjustment unit 1409 in FIG. 14 . Therefore, matching the FFT size when preparing training data to the FFT size of other processing in the hearing aid device that uses the trained model for estimation helps reduce the processing load and power consumption of the hearing aid device 1401.

[0079] When the hearing aid device 1401 is used for both ears, the input audio information is different sounds picked up by the left and right microphones 1402. The configuration may be such that the sounds from the left and right microphones 1402 are processed independently, or may be such that they are processed together. If the left and right hearing aid devices 1401 have separate housings, the picked up sounds are sent wirelessly to the other ear via the external interface 1413. In this case, the audio signal as is has a large data volume and requires a lot of power for transmission. To reduce the data volume, the data may be thinned out in the time direction. To reduce the data volume, a configuration may be adopted in which features extracted from the audio signal are transmitted. For example, level values, zero crossings, tonality, spectral roll-off, and Bark scale power values ​​may be used as features.

[0080] The range of learning data for input audio information will now be explained. Generally, the loudness of sounds received in the environment is broadly estimated to be between 0 dB SPL and 140 dB SPL. On the other hand, when considering use with a hearing aid device 1401, a narrower range is more practical. For example, the sound of leaves rustling is considered to be 20 dB SPL, and the sound of a train passing under a bridge is considered to be 100 dB SPL. The main purpose of a hearing aid device is to improve speech audibility, so it is effective to set the range of input audio levels to between 20 dB SPL and 110 dB SPL, or between 30 dB SPL and 100 dB SPL. Values ​​outside the range can be rounded to the boundary value.

[0081] The range of training data for both the hearing information and the input speech information may be limited at the data set stage, or may be limited as preprocessing when training the trained model 1501. When features of a speech signal are used for the input speech information, they may be converted into features at the data set stage, or may be converted into features as preprocessing when training the trained model.

[0082] When training the trained model 1501, any training data can be used, but it is generally useful to normalize the training data to improve discrimination performance and speed up training. Normalization is effective for both the training data for hearing information and the training data for input speech information. In this case, for example, the range of the training data described above is normalized to a range of 0.0 to 1.0. Normalization may be performed at the training dataset stage, or normalization may be performed as preprocessing when training the trained model 1501.

[0083] The configuration information includes information such as the type of earpiece, the wearing method, and the characteristics of the hearing aid device housing (as described above), and is important information for detecting the user's own voice when using the hearing aid device 1401. For example, when the user of the hearing aid device 1401 speaks when worn in both ears, the voice reaches the hearing aid devices 1401 in both ears at approximately the same time, with approximately the same volume and timbre. Using this information, the timing of the voice utterance can be easily detected. When worn in one ear, beamforming in the hearing aid device 1401 in one ear can be used to detect the voice arriving from the direction of the user's mouth, and characteristics such as its intensity and timbre can be used. When worn in both ears, the user's own voice can be detected more reliably than when worn in one ear. The configuration information includes information such as the type of earpiece of the hearing aid device 1401, the wearing method, and the characteristics of the hearing aid device housing (as described above). Examples of types of earpieces (domes) include open dome, closed dome, double dome, and tulip-shaped earpieces (domes), as well as molded earpieces, which are commonly used in conventional hearing aids. Information such as RECD (real ear to coupler difference), which is commonly used in adjusting conventional hearing aids, may also be included. For example, the intensity and timbre of the sound from the second path in Figure 13 are affected by the type of earpiece. When comparing the sound from the first path with the sound from the second path, it is important to predict the sound from the second path, and the type of earpiece and RECD are useful for this prediction.

[0084] The configuration information may be omitted in some cases. For example, the product configuration may be such that the earpiece is designed for binaural use and only one type of acoustic characteristics is available. This is expected to be the case when hearing aid functions are used with TWS, etc.

[0085] The allowable delay information may be a delay time or a number of delay samples. In Fig. 13, this allowable delay information indicates how much delay the first path can have compared to the second path and is within the allowable range for the user of hearing aid device 1401. More specifically, the delay amount in NR section 1407 is determined so that the total delay amount of the first path falls within this allowable range. For example, if the allowable range of delay for the entire first path is 6 milliseconds and a delay of 4 milliseconds occurs in parts other than the NR function, then the upper limit of the delay for the NR function is simply 2 milliseconds.

[0086] The allowable delay amount for the training data of the trained model 1501 may be the allowable delay amount DLY_ALL_MAX for the entire first path, or the allowable delay amount DLY_NR_MAX limited to the NR function. To prepare for the possibility that delay amounts other than the NR function may change after training of the trained model 1501, the allowable delay amount for the training data is preferably the allowable delay amount DLY_ALL_MAX for the entire first path. In the following description, the allowable delay amount for the training data will be mainly described as the allowable delay amount DLY_ALL_MAX for the entire first path, but is not limited to this. The allowable delay amount DLY_NR_MAX for the NR function can be easily calculated by subtracting the delay amount other than the NR function DLY_OTHER from the estimated allowable delay amount DLY_ALL_MAX for the entire first path. In other words, the allowable delay amount DLY_NR_MAX for the NR function can be calculated using the following equation (1):

[0087]

number

[0088] However, in the above formula (1), α is a non-negative numerical value, and when the delay amount of the NR unit and the delay amount of other processing are completely exclusive, α = 0. If there is a common buffer between the processing of the NR unit and other processing, α may be greater than 0. α depends on the buffer configuration of the hearing aid device 1401.

[0089] FIG. 16 shows an example of a user interface when creating training data. Hearing information and configuration information are displayed on portable information device 1664. In this example, the hearing information is an audiogram. The audiogram may be in the form of data transferred from a hearing measurement device (not shown) or manually entered by the user of portable information device 1664. In this example, the configuration information includes the type of earpiece and earmold to be used, and the selection of the left and right hearing aid devices to be used. Once the settings are complete, the setting information is transmitted to hearing aid device 1601, and left and right hearing aid devices 1601A and 1601B operate according to the settings. Hearing aid device 1601A is depicted with a shape similar to a conventional RIC (Receiver in the Canal) type hearing aid, and hearing aid device 1601B is depicted with a shape similar to a conventional TWS, but this is not limited to this. The connection between portable information device 1664 and hearing aid devices 1601A and 1601B may be wireless or wired.

[0090] FIG. 17 shows an example of a user interface used by a test subject (hearing-impaired person) to answer a question about the tolerable delay amount. The portable information device 1764 has a slider bar 1767 and radio buttons 1769 as user interfaces for adjusting the delay amount. For example, when the test subject changes the slider bar 1767, the delay amount of the hearing aid device changes accordingly. For example, the radio buttons 1769 are assigned a delay amount of 2 milliseconds to number 1 and a delay amount of 50 milliseconds to number 9. A sound environment assumed as input audio information is reproduced around the test subject, and the test subject specifies the maximum tolerable delay amount in that sound environment using the slider bar 1767 or radio buttons 1769 and presses the OK button 1765 to confirm. This adds hearing information, input audio information, configuration information, and tolerable delay amount information to the training dataset. The user interface described above is not limited to the slider bar 1767 and radio buttons 1769, and other forms of delay adjustment are also possible.

[0091] FIG. 18 shows the procedure for creating training data in the form of a flowchart. First, settings are made in the hearing aid device and a sound environment reproduction system (not shown) based on input information (step S1801). The sound environment reproduction system reproduces sound environments such as a quiet room, a hospital, a shopping mall, or a restaurant. For example, a 5.1-channel speaker system can be configured around the subject. For example, a conversation partner may be placed in front of the subject. Next, the subject specifies a delay amount using a user interface such as that shown in FIG. 17 (step S1802). Next, the hearing aid device is operated according to the delay amount specified by the subject (step S1803). Next, it is determined whether the OK button 1765 on the user interface shown in FIG. 17 has been pressed (step S1804). In other words, it is determined whether the subject has confirmed the allowable delay amount. If the OK button has not been pressed (No in step S1804), the process returns to step S1802, where the delay amount specification and operation at that delay amount are repeated. During repetition, if there is no new specification from the subject in step S1802, the previous setting may be maintained and used as is. On the other hand, if the OK button is pressed (Yes in step S1804), the delay amount specified at that time is added to the learning data set as allowable delay amount information (step S1805).

[0092] 17 and 18 show an example in which the subject specifies candidate delay amounts, but the present invention is not limited to this. For example, an automatically changed delay amount may be presented to the subject, and the subject may respond each time as to whether it is acceptable or not. Alternatively, for example, two different sets of delay amount settings may be presented to the subject, and the subject may respond as to which is more acceptable. In short, it is sufficient to obtain an answer as to what extent of delay amount is acceptable to the subject.

[0093] FIG. 19 shows another example of generating a trained model used in the delay amount determination unit 1416. Hearing information, input audio information, and configuration information are used, and allowable delay information is output. This generation example differs from the generation example shown in FIG. 15 in that trained models are generated for each piece of configuration information. For example, trained model A 1940 is generated under the condition of configuration information A, trained model B 1941 is generated under the condition of configuration information B, and trained model C 1942 is generated under the condition of configuration information C. For example, configuration information A is for monoaural wear and an open dome, and configuration information B is for binaural wear and a closed dome. When there are few combinations of configuration information elements, it is also possible to generate a trained model for each piece of configuration information in this way, and then select a trained model based on the configuration information in the estimation stage. While FIG. 19 shows an example in which three types of trained models A to C are generated using three types of configuration information A to C as conditions, hearing aid devices may be classified into four or more types of configuration information, and four or more types of trained models may be generated. The generation example shown in FIG. 15 and the generation example shown in FIG. 19 are the same in that they use hearing information, input audio information, and configuration information and output allowable delay amount information.

[0094] FIG. 20 shows yet another example of generating a trained model used in the delay amount determination unit 1416. In the illustrated example, hearing information, input speech information, and configuration information are input, and allowable delay information is output. This example differs from the example shown in FIG. 15 in that the trained model is composed of a first trained model 2040 and a second trained model 2041, and the output of the second trained model 2041 is used as the input of the first trained model 2040. Detection of the presence or absence of one's own voice is independent of the presence or absence of hearing loss, so it is possible to easily prepare a large amount of training data with the cooperation of normal-hearing individuals (those with normal hearing). First, a second trained model 2041 that detects the presence or absence of one's own voice is generated using the input speech information and configuration information. Next, when generating the first trained model 2040, the estimation results of the second trained model 2041 are used. That is, in addition to hearing information, input speech information, and configuration information, information on the presence or absence of one's own voice is input to the first trained model 2040, and allowable delay information is output. With this configuration, highly accurate own voice detection can be expected, and as a result, improved accuracy in estimating the allowable delay information of the first trained model 2040 can be expected.

[0095] Training data includes hearing information, input speech information, configuration information, and allowable delay information. The subjects to acquire training data must be hard of hearing. Many people with hearing loss are elderly, which makes it difficult for them to work for long periods of time. For this reason, generating a trained model by dividing the data into parts that must rely on the hearing loss (or parts that require the cooperation of the hearing loss) and parts that do not (or parts that do not require the cooperation of the hearing loss) is useful for efficiently constructing a training dataset.

[0096] FIG. 21 shows an example configuration of the allowable delay estimation unit 2126 when a trained model is not used. For example, the gain curve 280 shown in FIG. 2 can be used as the gain information. The first sound pressure level calculation unit 2122 calculates a first sound pressure level P_1(f), which is the output of the first path, using the frequency band level information and gain information of the input sound. f represents frequency. Next, acoustic characteristics of the earpiece and other components are acquired from the configuration information. For example, attenuation characteristics for each frequency can be used as these acoustic characteristics. Generally, high frequencies are attenuated more, and low frequencies are attenuated less. For example, sound at position P1 shown in FIG. 13 is attenuated in accordance with the attenuation characteristics of the earpiece when passing through the second path. The second sound pressure level calculation unit 2123 calculates a second sound pressure level P_2(f) in the ear canal (e.g., position P2 in FIG. 13) via the second path using the configuration information (acoustic characteristics of the earpiece and other components) and the frequency band level information of the input sound. The sound pressure level difference calculation unit 2124 calculates the difference r(f) between the first sound pressure level P_1(f) and the second sound pressure level P_2(f) according to the following equation (2).

[0097]

number

[0098] When P_1 and P_2 are considered in terms of sound pressure instead of sound pressure level, the sound pressure ratio P_1(f) / P_2(f) may be calculated. The allowable delay amount calculation unit 2125 calculates the allowable delay amount DLY_ALL_MAX from the sound pressure level difference r(f) as shown in the following equation (3), for example.

[0099]

number

[0100] Here, function f1 is a function that represents the relationship between the sound pressure level difference r(f) and the allowable delay DLY_ALL_MAX, and is calculated in advance as an approximation from statistical data. For example, as illustrated in the explanation of Figure 9, it can be calculated as 60 milliseconds when the difference is large and 6 milliseconds when the difference is small.

[0101] 22 shows, in the form of a flowchart, a processing procedure for determining an allowable delay amount in the allowable delay amount estimation unit 2126 shown in FIG. 21 without using a trained model. The first sound pressure level calculation unit 2122 calculates a first sound pressure level P_1(f), which is the output of the first path, from the input audio information and hearing information (step S2201). The second sound pressure level calculation unit 2123 calculates a second sound pressure level P_2(f) of the second path from the input audio information and configuration information (step S2202). Next, the sound pressure level difference calculation unit 2124 calculates the sound pressure level difference r(f) between the first sound pressure level P_1(f) and the second sound pressure level P_2(f) based on the above equation (2) (step S2203). Then, the allowable delay amount calculation unit 2125 calculates the allowable delay amount DLY_ALL_MAX from the sound pressure level difference r(f) based on the above equation (3) (step S2204), and outputs it (step S2205).

[0102] In the above description, the function f1 is a function that represents the relationship between the sound pressure level difference r(f) and the allowable delay amount DLY_ALL_MAX of the entire first path, but it may also be a function that represents the relationship between the sound pressure level difference r(f) and the allowable delay amount DLY_NR_MAX of the NR function. In that case, in step S2205 of Fig. 22, the allowable delay amount DLY_NR_MAX is output.

[0103] A large amount of training data is required to train a trained model. As explained with reference to Figures 16 to 18, the procedure for creating training data is time-consuming and labor-intensive. When collecting a large amount of training data, it is important to reduce manual work as much as possible. The method without using a trained model, explained with reference to Figures 21 and 22, can be used as a tool for efficiently constructing training data. First, candidates for the allowable delay amount are prepared using a method without using a trained model. Next, the candidates are manually fine-tuned and used as training data. For example, the initial values ​​of the slider bar 1767 and the radio button 1769 in Figure 17 are determined using candidates for the allowable delay amount calculated using a method without using a trained model. In this way, the subject can determine the answer by simply trying values ​​close to the initial value. In a configuration in which the subject is asked to answer multiple-choice questions, the number of multiple-choice questions required can be reduced. This method significantly improves work efficiency. Furthermore, the method that does not use a trained model, as described with reference to Figures 21 and 22, requires only a small amount of calculation and can therefore be implemented even on inexpensive hearing aid devices. In other words, it can be used as a simple alternative to estimation using a trained model.

[0104] FIG. 23 shows the processing procedure of variable delay NR in a hearing aid device in the form of a flowchart.

[0105] First, the amount of delay for NR is initialized (step S2301). Next, NR processing is performed on the input audio signal (step S2302). The audio signal that has undergone NR processing is output from the speaker after undergoing other signal processing of the hearing aid device. The amount of delay for the audio output from the speaker is affected by the amount of delay for NR processing.

[0106] Next, the recommended setting for the amount of delay for NR is calculated using the hearing ability information of the user of the hearing aid device, the input audio information of the hearing aid device, and the configuration information of the hearing aid device (step S2303). Examples of how to calculate the amount of delay will be described later with reference to Figures 24 to 27. Next, the recommended setting is presented to the user of the hearing aid device as needed (step S2304). If the recommended setting is to be applied automatically, step S2304 may be skipped.

[0107] Next, the NR delay amount is updated based on the response from the hearing aid device user (step S2305). If the recommended settings are set to be automatically applied, they are updated automatically. Examples of how to present the recommended settings to the hearing aid device user and how to respond will be described later with reference to Figures 28 to 32.

[0108] It is then determined whether to end the NR processing (step S2306). If not (No in step S2306), the process returns to step S2302 and continues. If NR processing is to be ended (Yes in step S2306), the process ends. NR processing may be ended when the user of the hearing aid device turns off the NR function or when the hearing aid device is turned off.

[0109] In Figure 23, the user is described as a user of a hearing aid device, but it may also be a family member or other person other than the user of the hearing aid device. The family member or other person may be located next to the user of the hearing aid device, or may be located at a distance from the user of the hearing aid device via a network.

[0110] FIG. 24 shows an example configuration of a delay amount determination unit according to the present disclosure that uses a trained model to determine allowable delay amount information. The delay amount determination unit 2416 shown in FIG. 24 corresponds to the delay amount determination unit 1416 shown in FIG. 14, and receives hearing information, input audio information, and configuration information as input, and outputs a delay amount. The input and output are performed via the control unit 1411 in FIG. 14. The trained model 2440 may be any of the trained models described in FIG. 15, FIG. 19, and FIG. 20. The trained model 2440 receives hearing information, input audio information, and configuration information, and outputs allowable delay amount information. The delay amount calculation unit 2427 receives the allowable delay amount information, calculates the delay amount of the NR unit, and outputs it.

[0111] FIG. 25 shows the flow of processing by the delay amount determining unit 2416 in the form of a flowchart.

[0112] First, the trained model 2440 receives the hearing information, the input audio information, and the configuration information, and estimates the allowable delay using the trained model (step S2501). Next, the delay calculation unit 2427 calculates the allowable delay DLY_NR_MAX of the NR unit from the allowable delay information (step S2502). When the allowable delay estimated by the trained model 2440 is the allowable delay DLY_ALL_MAX of the entire first path, the allowable delay DLY_NR_MAX can be calculated as in the following equation (4):

[0113]

number

[0114] In the above equation (4), DLY_OTHER is the delay amount other than the NR function. α is a non-negative number, and if the delay amount of the NR unit and the delay amount of other processing are completely exclusive, α = 0. If there is a common buffer between the NR unit processing and other processing, α may be greater than 0. If the allowable delay amount estimated by the trained model is the allowable delay amount of the NR function, DLY_NR_MAX, the allowable delay amount estimated by the trained model can be used as is.

[0115] Next, the delay amount calculation unit 2427 determines the delay amount DLY_NR of the NR unit from the allowable delay amount DLY_NR_MAX of the NR unit (step S2503). When an arbitrary delay amount is available due to the configuration of the NR unit, the delay amount DLY_NR of the NR unit is determined as shown in the following equation (5).

[0116]

number

[0117] When the delay amount possible due to the configuration of the NR unit 1407 is discrete, the delay amount DLY_NR of the NR unit 1407 can be set within a range not exceeding DLY_NR_MAX. For example, when the delay amount possible due to the configuration of the NR unit 1407 is 16 samples, 32 samples, and 64 samples, and the calculated DLY_NR_MAX is 40 samples, DLY_NR is determined to be 32 samples. The delay amount DLY_NR of the NR unit 1407 may not only be set to as large a value as possible, but may also be set to a small value. For example, when the delay amount possible due to the configuration of the NR unit 1407 is 16 samples, 32 samples, and 64 samples, and the calculated DLY_NR_MAX is 40 samples, DLY_NR may be determined to be 16 samples.

[0118] In hearing aid devices with independent left and right shapes, when the allowable delay amounts are calculated independently, it is possible to use separate delay amounts for the left and right, or to use a uniform delay amount for the left and right. When using a uniform delay amount for the left and right, for example, it may be the average of the delay amounts calculated for the left and right, or it may be set to a short delay amount. In order to suppress fluctuations in the left and right sound localization, it is preferable to use a uniform delay amount for the left and right. In order to suppress discomfort felt by the user of the hearing aid device, it is preferable to set the left and right delay amounts to a short delay amount. When integrating the left and right allowable delay amounts, the calculated delay amount can be exchanged via the external interface 1413.

[0119] FIG. 26 shows an example configuration of the delay amount determination unit 2616 according to the present disclosure. The delay amount determination unit 2616 corresponds to the delay amount determination unit 1416 shown in FIG. 14, and receives hearing information, input audio information, and configuration information as input, and outputs a delay amount. The input and output are performed via the control unit 1411 shown in FIG. 14. The allowable delay amount estimation unit 2626 receives hearing information, input audio information, and configuration information, and outputs allowable delay amount information. The delay amount determination unit 2616 differs from the delay amount determination unit 2416 shown in FIG. 24 in that the allowable delay amount estimation unit 2626 determines the allowable delay amount information without using a trained model. The allowable delay amount estimation unit 2626 may have a configuration similar to, for example, the allowable delay amount estimation unit 2126 shown in FIG. 21, but is not limited to this. The operation of the delay amount calculation unit 2627 is the same as that of the delay amount calculation unit 2427 shown in FIG. 24.

[0120] 27 is a flowchart showing the processing flow of the delay amount determination unit 2616. The processing is the same as that shown in FIG. 25 except that in step S2701, the allowable delay amount estimation unit 2626 determines the allowable delay amount information without using a trained model.

[0121] Fig. 28 shows, in the form of a flowchart, an example of a process for suggesting to the user that the delay amount be updated. When the newly calculated delay amount of the NR unit 1407 differs from the current delay amount of the NR unit, there are two methods: an automatic update method, and a method for suggesting an update to the user of the hearing aid device. When updating automatically, it is preferable to update with a certain time constant, as frequent updates may cause discomfort. When suggesting an update to the user of the hearing aid device, for example, the flowchart shown in Fig. 28 can be used.

[0122] First, a new delay amount of the NR unit 1407 is calculated (step S2801), as shown in Figures 24 and 26. Next, the newly calculated delay amount of the NR unit 1407 is compared with the current delay amount of the NR unit 1407 to check whether the delay amount of the NR unit 1407 has changed (step S2802).

[0123] If the newly calculated delay amount of the NR unit 1407 does not differ from the current delay amount of the NR unit 1407 (No in step S2802), the process returns to step S2801. On the other hand, if the newly calculated delay amount of the NR unit 1407 differs from the current delay amount of the NR unit 1407 (Yes in step S2802), the process proceeds to step S2803, where a proposal to update the delay amount of the NR unit 1407 is made to the user of the hearing aid device. Examples of the proposal method will be described later using Figs. 29, 31, and 32.

[0124] Next, a response to the proposal is obtained from the user of the hearing aid device (step S2804). Examples of response methods will be described later using Figures 30, 31, and 32. Next, in step S2805, the process branches depending on the response. If the user's response indicates a desire for an update (Yes in step S2805), the delay amount in the NR unit 1407 is updated (step S2806), and then the process ends. On the other hand, if the user's response indicates a desire not to update (No in step S2805), the delay amount in the NR unit 1407 is not updated (step S2807), and the process ends.

[0125] FIG. 29 shows an example of a method for proposing a delay amount update to a hearing aid device user. The suggestion can be made by playing a voice message from the speaker of the hearing aid device 2901 or the speaker of an externally connected portable information device 2964. In addition to voice, a specific alarm sound or music may also be used. The screen of the externally connected portable information device 2964 can also be used. Verbal messages, symbols, pictograms, etc. are also possible. Furthermore, the vibration function of the externally connected portable information device 2964 can also be used. For example, the interval between vibrations can be changed to distinguish between them.

[0126] FIG. 30 shows an example of how a hearing aid user can respond. The user 3090 can respond using the buttons or touch sensor on the hearing aid device 3001, or the user interface on the screen of the externally connected mobile information device 3064. It is also possible to receive responses from the user using an acceleration sensor on the hearing aid device 3001 or the externally connected mobile information device 3064. For example, the hearing aid device 3001 can use an acceleration sensor to detect nodding its head for "yes" or shaking its head from side to side for "no" and receive this as a response. The mobile information device 3064 can also receive a "yes" or "no" response depending on the direction in which the mobile information device 3064 is shaken. Responses from the user can also be received using a microphone on the hearing aid device 3001 or the externally connected mobile information device 3064. The user of the hearing aid device can respond verbally, and the speech picked up by the microphone can be recognized to identify whether the speech was "yes" or "no."

[0127] FIG. 31 shows another example of a user interface used to suggest a delay update to a hearing aid device user and receive a response from the hearing aid device user. A slider bar 3167 indicating the NR strength is provided, and a recommended range 3199 may also be displayed. The NR strength is another way of indicating the length of the delay. A "weak" NR strength corresponds to a "short" delay, and the strength of the NR is easier to understand than the length of the delay. The length of the delay may also be used. The recommended range 3199 is determined from the newly calculated delay. Recommended range 3199A on the left of FIG. 31 is an example of a newly calculated delay that is short, and recommended range 3199B on the right of FIG. 31 is an example of a newly calculated delay that is long. For example, the recommended range may be 80% to 100% of the newly calculated delay.

[0128] FIG. 32 shows several specific methods for displaying the recommended range. Display methods such as recommended range 3299C and recommended range 3299D may also be used. Dial 3268 may be used instead of slider bar 3167 shown in FIG. 31. In the case of a dial, the recommended range may also be displayed as recommended range 3299E and recommended range 3299F. Furthermore, the means for adjusting the NR intensity is not limited to slider bar 3167 and dial 3268, and other methods may also be used.

[0129] 31 and 32 show examples of a slider bar 3167 and a dial 3268, respectively, but the present invention is not limited to these and may be implemented in any way as long as a recommended range is automatically presented according to the newly calculated delay amount. In this way, by presenting the recommended range to the hearing aid user, the hearing aid user can set the NR intensity according to their own will while referring to the range. After updating the settings, the OK button 3165 is pressed to exit. If the hearing aid user does not want to change the settings, the OK button 3165 is pressed to exit without operating the slider bar 3167 or dial 3268. If the NR intensity, i.e., the NR delay amount, is to be set automatically, the switch 3166 may be configured to allow the automatic setting to be switched on and off.

[0130] Fig. 33 shows an example of the configuration of an NR unit, which corresponds to the NR unit 1407 in Fig. 14. Fig. 34 shows an example of variable delay buffer operation. For example, in the case of frequency domain processing such as the spectral subtraction method, the time domain is first converted to the frequency domain. When using FFT, a window function and 50% overlap are often used.

[0131] The NR unit 3307 shown in FIG. 33 receives, for example, the delay amount determined by the delay amount determination unit 1416 in FIG. 14 via the control unit 1411 in FIG. 14. The audio signal input to the NR unit 3307 first enters an input buffer 3351 (input buffer 3451 in FIG. 34). A windowing unit (pre-processing) 3353 multiplies the signal by a window function of a size appropriate for the delay amount (for example, window function 3450 in FIG. 34). An FFT 3355 converts the signal to the frequency domain using an FFT of a size appropriate for the delay amount. An NR_core 3357 performs NR processing using a spectral subtraction method or the like. The NR-processed audio signal is converted to the time domain using an IFFT (inverse fast Fourier transform) in an IFFT 3356, then multiplied by a window function in a windowing unit (post-processing) 3354, and passed to an output buffer 3352 (output buffer 3452 in FIG. 34).

[0132] FIG. 34 shows an example of variable delay buffer operation for a signal consisting of 13 frames (frames 1 through 13) with a delay varying in the order of two frames, four frames, and two frames. In real-time processing, when the delay length increases, the processed audio signal becomes insufficient, while when the delay length decreases, the processed audio signal becomes surplus. Simply changing the delay length causes discontinuities in the processed audio signal, resulting in artifacts. This issue is unique to real-time processing. For example, audio codecs also use variable frame sizes, but audio codecs convert data to a bitstream, which eliminates this issue. To address this issue, a process is required to smoothly connect the processed audio signals. In the example shown in FIG. 34, the window size is changed and crossfading is used to smoothly connect the processed audio signals when the delay length changes. While the above explanation illustrates an example in which the FFT size is changed in accordance with the delay length, the FFT size does not necessarily have to be changed. Alternatively, the number of frames used to estimate the noise spectrum may be extended in accordance with the delay length.

[0133] 33 and 34 show examples of frequency domain and frame processing, but time domain or sample processing may also be used. NR processing with multiple delay amounts may always be performed in parallel, and these may be selected as appropriate and connected by cross-fading. In the example shown in FIG. 34, cross-fading is performed over a one-frame width, but it may also be performed over multiple frames. It is sufficient that the audio signal after variable delay NR processing is smoothly connected.

[0134] Figure 35 shows a second example configuration of a hearing aid device with a variable delay NR function according to the first embodiment. Although microphone 3502, ADC 3504, DAC 3505, and speaker 3503 are drawn in a straight line from left to right, it should be understood that this is the same configuration as in Figure 13. Below, the configuration of hearing aid device 3501 will be explained, focusing on the differences from hearing aid device 1401 shown in Figure 14.

[0135] The sound source storage and playback unit 3512 corresponds to the sound source storage and playback unit 1412 in Fig. 14. The external interface 3513 corresponds to the external interface 1413 in Fig. 14, but can also handle connection information and sensor information, which will be described later. The configuration information storage unit 3514 corresponds to the configuration information storage unit 1414 in Fig. 14. The trained model storage unit 3517 has the same configuration as the trained model storage unit 1417 in Fig. 14.

[0136] Connection information storage unit 3518 stores information about external devices connected to hearing aid device 3501. For example, external devices can be telephone devices, mobile information devices, televisions, etc. Information about the type of external device can be used as connection information. Content genre information for audio transmitted from an external device to hearing aid device 3501 can also be included in the connection information. Content genre information includes movie genre, news genre, sports genre, etc.

[0137] The sensor information section 3519 is information on various sensors, such as an acceleration sensor, a gyro sensor, and a camera.

[0138] 35 uses the effect of feature B5 (the feature that the priority of required functions changes depending on the usage scenario) to estimate the allowable delay amount. To do this, at least one of connection information and sensor information is used.

[0139] FIG. 36 shows another example of generating a trained model according to the first embodiment of the present disclosure. This trained model 3640 is used in the delay amount determination unit 3516 in the hearing aid device 3501. Hearing information, input audio information, configuration information, connection information, and sensor information are input, and allowable delay information is estimated and output. That is, the trained model 3640 is obtained by adding connection information and sensor information to the trained model 1501 shown in FIG. 15 as explanatory variables. While FIG. 36 depicts the trained model 3640 as a neural network with five input nodes, one output node, and two hidden layers, this is not limiting. The trained model 3640 can be generated on a server or personal computer separate from the hearing aid device 3501. The generated trained model 3640 can be transmitted to the hearing aid device 3501 via the external interface 3513 in FIG. 35.

[0140] The hearing information, input speech information, configuration information, and allowable delay information are the same as in the example of generating a trained model shown in Fig. 15. The connection information and sensor information will be described below.

[0141] The connection information can be, for example, information about the type of connected external device. Wireless communication, such as Bluetooth (registered trademark) or Wi-Fi (registered trademark), can be used for the connection. The type of external device can be, for example, a telephone device, a mobile information device, or a television. For example, when a hearing aid is wirelessly connected to a telephone device, the sound transmitted from the telephone device, rather than the sound picked up by the microphone of the hearing aid, is output from the speaker of the hearing aid. Therefore, echoes such as those caused by Feature B3 do not occur. Delays are tolerated as long as they do not interfere with communication with the other party. According to ITU-T Recommendation G.114 (05 / 2003), a call delay of up to 150 milliseconds is tolerated. Assuming that hearing aids can tolerate up to 20% of that, roughly speaking, a delay of up to 30 milliseconds would be tolerated. For example, when a hearing aid is wirelessly connected to a television, echoes such as those caused by Feature B3 do not occur. However, as already explained with reference to FIG. 7, television has both video and audio, and if the synchronization between them is lost, viewers will experience discomfort. For example, in the case of an external device that can use a variety of content, such as a television, content genre information can also be used. Even when connected to a television, the expected behavior may differ depending on whether the video content being viewed is movie content or news content. The audio of news content is inherently clear and has a fairly constant volume, so NR functionality is rarely required. In contrast, sound effects and noise are important elements of movie content, and the audio is often difficult to hear. NR functionality is useful for people with hearing loss. For example, for television broadcasts, content genre information can be obtained from an electronic program guide (EPG, IPG). In the above explanation, when listening to audio transmitted from an external device, the audio picked up by the microphone of the hearing aid device is not output from the speaker, but both may be output simultaneously. In this case, the device will also be affected by echoes caused by feature B3.

[0142] If the connection information uses type information of the connected external device and there are not many types of devices, it is also possible to generate a trained model for each type, as in the trained model generation example shown in Figure 19. However, illustration is omitted here.

[0143] Sensor information includes information from various sensors, such as an acceleration sensor, a gyro sensor, and a camera. For example, acceleration sensor information can be added to the training data used as input for the trained model for voice presence detection shown in Figure 20. The vibrations of the user's voice travel from the vocal cords through the skull to the ear canal and auricle. These vibrations are captured and used by an acceleration sensor. Accurate capture of information from an acceleration sensor, even in noisy environments, enables more accurate voice presence detection, even in noisy environments. For example, using an acceleration sensor and a gyro sensor can estimate the body movements of a hearing aid user. For example, frequent nodding or head shaking increases the likelihood of the user communicating with someone. Figure 37 shows an example of a camera 3797 mounted on a hearing aid. For example, using a camera can detect the presence of a face in the immediate vicinity in front of the user. This also increases the likelihood of communication with someone. The acceptable delay is likely to be short. For example, if the user remains motionless and looks straight ahead, the user is likely watching television or other similar content, decreasing the likelihood of communication with someone. The acceptable delay is likely to be long. In FIG. 37, one camera 3797 is arranged on each of the left and right hearing aid devices, but this is not limited to this and there may be multiple cameras, or the number of cameras may differ between the left and right hearing aid devices.

[0144] FIG. 38 illustrates yet another example of generating a trained model according to the present disclosure. In the example of generation illustrated in FIG. 38, hearing information, input audio information, configuration information, connection information, and sensor information are input, and allowable delay information is output. This example differs from the example of generation illustrated in FIG. 36 in that the output of the second trained model 3841 is used as the input of the first trained model 3840. For example, because body movement estimation and face detection are independent of whether or not a person has hearing loss, it is possible to easily prepare a large amount of training data with the cooperation of individuals with normal hearing. First, a second trained model 3841 that detects body movement and a second trained model 3841 that detects faces are generated using sensor information. Next, when generating the first trained model 3840, the estimation results of the second trained model 3841 are used. That is, in addition to hearing information, input audio information, configuration information, connection information, and sensor information, body movement information and face information are input to the first trained model 3840, and allowable delay information is output. With this configuration, highly accurate body movement estimation and face detection can be expected, and as a result, the accuracy of the estimation of the allowable delay amount information of the first trained model 3840 can be expected to be improved.

[0145] FIG. 39 shows an example configuration of a delay amount determination unit according to the present disclosure that uses a trained model for estimation. The delay amount determination unit 3916 corresponds to the delay amount determination unit 3516 shown in FIG. 35, but receives hearing information, input audio information, configuration information, connection information, and sensor information as input, and outputs a delay amount. The input and output are performed via the control unit 3511 in FIG. 35. The trained model 3940 may be any of the trained models described in FIG. 36 or FIG. 38. The trained model 3940 receives hearing information, input audio information, configuration information, connection information, and sensor information, and outputs allowable delay amount information. The delay amount calculation unit 3927 receives the allowable delay amount information, calculates the delay amount of the NR unit, and outputs it.

[0146] FIG. 40 shows, in the form of a flowchart, the flow of processing in which the delay amount determination unit 3916 determines the amount of delay using a trained model.

[0147] First, the trained model 3940 receives the hearing information, input audio information, and configuration information, as well as connection information and sensor information, and estimates the allowable delay using the trained model (step S4001). Next, the delay calculation unit 3927 calculates the allowable delay DLY_NR_MAX of the NR unit 3507 from the allowable delay information based on the above equation (4) (step S4002).

[0148] Next, the delay amount calculation unit 3927 determines the delay amount DLY_NR of the NR unit 3507 from the allowable delay amount DLY_NR_MAX of the NR unit 3507 based on the above equation (5) (step S4003).

[0149] When the delay amount possible due to the configuration of the NR unit 3507 is discrete, the delay amount DLY_NR of the NR unit 3507 can be set within a range not exceeding DLY_NR_MAX. For example, when the delay amount possible due to the configuration of the NR unit 3507 is 16 samples, 32 samples, and 64 samples, and the calculated DLY_NR_MAX is 40 samples, DLY_NR is determined to be 32 samples. The delay amount DLY_NR of the NR unit 3507 may not only be set to as large a value as possible, but may also be set to a small value. For example, when the delay amount possible due to the configuration of the NR unit 3507 is 16 samples, 32 samples, and 64 samples, and the calculated DLY_NR_MAX is 40 samples, DLY_NR may be determined to be 16 samples.

[0150] If the newly calculated delay amount of NR section 3507 differs from the current delay amount of NR section 3507, there are two methods: an automatic update method, and a method of suggesting an update to the user of the hearing aid device. The processing for these methods is as explained with reference to Figures 28 to 32.

[0151] Changing the window size in accordance with changes in the delay amount of the NR unit 3507 and smoothly connecting the audio signal after NR processing without interruption are as explained with reference to Figures 33 and 34.

[0152] The handling of the delay amount of the NR unit 3507 in a hearing aid device with independent left and right sensors is the same as in the example configuration shown in Fig. 14. In a hearing aid device with independent left and right sensors, it is possible to use the sensor information separately for the left and right sensors, or to use the sensor information uniformly for the left and right sensors. When using the sensor information uniformly for the left and right sensors, for example, the sensor information may be averaged. When integrating the sensor information for the left and right sensors, it can be exchanged via the external interface 3513.

[0153] The above has mainly explained how the delay amount of the NR function is controlled using hearing information etc. in a hearing aid device, but as mentioned above, it is also possible to control the sudden sound suppression function in the same way.

[0154] The effects of the first embodiment are summarized below. By controlling the delay amount of the first path using hearing information, input audio information, and the like, the first embodiment makes it possible to effectively operate the NR function and the sudden sound suppression function. The effect is to improve the discomfort caused by the discrepancy between visual information and auditory information resulting from features B1 to B4, stuttering due to delayed auditory feedback, difficulty in listening due to echo, and listening difficulties resulting from features A1 and A2. Furthermore, by presenting recommended delay amount settings to the user of the hearing aid device and allowing the user to update the settings themselves based on the recommended settings, a higher level of satisfaction can be expected. [Example]

[0155] In a second embodiment, the hearing aid device utilizes NC functionality in addition to NR functionality, which allows for automatic setting of the NC functionality, automatic on / off switching, automatic intensity switching, automatic characteristic changes, and the presentation of recommended settings on the user interface, thereby enabling important power savings for hearing aid devices while also improving the listening experience for hearing aid users.

[0156] It should be noted that, in this specification, the NR function is a function realized by signal processing to remove noise components contained in an input audio signal (i.e., suppress noise in the audio signal), while the NC function is a function that cancels out spatial noise with an NC signal that has the same amplitude as the noise component but is phase-inverted, by superimposing the NC signal on the desired signal after audio output. Therefore, it should be noted that the NR function and the NC function are completely different functions. The NR function outputs a signal from which noise has been removed, so the NR effect is maintained even with delay. In contrast, if the delay is large, the NC function cannot cancel out the noise due to the time difference between the noise and the NC signal, and the NC effect is lost. Additionally, because earpieces can block high-frequency noise, the NC function is primarily used to cancel noise below 1 kHz, which cannot be blocked.

[0157] Before describing the second embodiment in detail, we will briefly explain the background and issues. Due to features B1 to B3, even if the NR function is used, as described in FIG. 12, if the amplification gain of the first path is small, the improvement in the SNR at position P2 is slight. Even in such cases, features A1 and A2 require improvements in speech intelligibility in noisy environments. Therefore, the second embodiment utilizes the NC function. FIG. 41 illustrates an example of the noise canceling mechanism. For example, assume that waveform 4145 is the waveform when sound (noise) at position P1 in FIG. 13 arrives at position P2 via the second path. In this case, if sound of waveform 4146 arrives at position P2 via the first path, waveforms 4145 and 4146 cancel each other, ideally resulting in silence. Noise reduction suppresses noise in audio signals, while noise canceling cancels noise in space; the two are completely different in their mechanisms.

[0158] Fig. 42 shows an example configuration of a hearing aid device equipped with a noise canceling function. This shows an example configuration in which a microphone 4229 is added inside the hearing aid device 4201. By using the internal microphone 4229 for the NC function, feedback-type noise canceling can also be performed. Note that the other components, such as the microphone 4202, speaker 4203, ADC 4204, DAC 4205, signal processing unit 4206, sound source storage and playback unit 4212, and external interface 4213, are the same as those of the hearing aid device 1301 shown in Fig. 13, so detailed description will be omitted here.

[0159] An example of the effect when the NC function is added to the NR function is shown in Fig. 43. Positions P1 and P2 in Fig. 43 correspond to positions P1 and P2 shown in Fig. 6 and Fig. 13, respectively.

[0160] Spectrogram 4370 represents the sound at position P1 and consists of speech spectrum 4374 and noise 4375. The larger the difference d1 between the peak of speech spectrum 4374 and the peak of noise 4375, the better the SNR. The difference d1 in spectrogram 4370 is small, meaning the speech is only slightly louder than the noise, and a person with hearing loss would have difficulty hearing it.

[0161] In the first path via the microphone 4202, the audio signal is subjected to signal processing by the signal processing unit 4206, but in the second embodiment, the signal processing includes a first processing system that activates the NR function and a second processing system that activates the NC function.

[0162] The sound that arrives at position P2 via the second path and the sound that arrives at position P2 via the second processing system of the first path are combined to form spectrogram 4372. The second processing system of the first path does not amplify the sound but instead applies NC function 4308. In other words, spectrogram 4372 represents the sound at position P2 after noise cancellation. The NC function widens the difference d1 in spectrogram 4370 to the difference dNC in spectrogram 4372, improving the SNR. Note that the shape of the spectrogram is affected by the housing and earpieces of the hearing aid device, but for simplicity's sake, these will be ignored here.

[0163] On the other hand, in the first processing system of the first path, the NR function 4307 is activated but is not amplified (gain: 0 dB) as indicated by reference numeral 4377. The sound that passes through the first processing system of the first path and reaches position P2 is shown as spectrogram 4371. The NR function widens the difference d1 in spectrogram 4370 to the difference dNR in spectrogram 4371, improving the SNR.

[0164] Then, spectrogram 4372 and spectrogram 4371 mix together at position P2, and the sound at position P2 becomes spectrogram 4373. The difference d2_3 in spectrogram 4373 is larger than d2_2 in Fig. 12. It can be confirmed that when the NC function is added to the NR function, the SNR at position P2 is greatly improved compared to when only the NR function is used.

[0165] In this way, even if the gain of the first path is small, it is possible to improve the SNR at position P2 by using the NC function in addition to the NR function.

[0166] Figure 44 shows an example in which the effect of the NC function is small. Positions P1 and P2 in Figure 44 correspond to positions P1 and P2 in Figure 6 and Figure 13, respectively. In the example shown in Figure 44, the difference d1 between the peak of speech spectrum 4474 and the peak of noise 4475 in spectrogram 4470 is small, and the speech is only slightly louder than the noise, so a person with hearing loss would have difficulty hearing it.

[0167] In the second processing system of the first path, the NC function 4408 is applied without amplification. On the other hand, in the first processing system of the first path, the NR function is applied along with amplification (large gain). The sound that passes through the first processing system of the first path and arrives at position P2 is taken as spectrogram 4471. Spectrogram 4472 and spectrogram 4471 mix at position P2, and the sound at position P2 becomes spectrogram 4473. The NR function widens the difference d1 in spectrogram 4470 to the difference dNR in spectrogram 4471, improving the SNR.

[0168] Then, spectrogram 4472 and spectrogram 4471 are mixed at position P2, and the sound at position P2 becomes spectrogram 4473. However, the difference d2_4 in spectrogram 4473 is only slightly larger than d2_1 in Fig. 11. It can be seen that even when the NC function is added to the NR function, there is only a slight further improvement in SNR at position P2 compared to the case of only the NR function.

[0169] As shown in Figure 43, when the gain of the first processing system of the first path is small, adding the NC function significantly improves the SNR at position P2, thereby alleviating the listening difficulty of the hearing aid user. On the other hand, as shown in Figure 44, when the gain of the first processing system of the first path is large, the SNR at position P2 has already been improved by the NR function, and adding the NC function provides only a small further improvement. The improvement in the listening difficulty of the hearing aid user is also small.

[0170] As described above, the effect of adding an NC function to an NR function to improve the SNR at position P2 depends on the amplification gain of the hearing aid device. As can be seen from the examples shown in Figures 2 and 5, the amplification gain depends on the hearing ability of the hearing aid user and the input sound pressure to the hearing aid device.

[0171] Another aspect of the NC function is the issue of power consumption and the discomfort caused by an overly strong NC effect. If the NC effect is set too strong in a quiet environment, hearing aid users may feel uncomfortable, such as a tingling sensation in the ears. Because hearing aids are small devices that require long periods of operation, power consumption must be kept low. NC functions tend to consume a lot of power, as they sometimes require the operation of dedicated ADCs and DACs. These issues can be alleviated by turning the NC function on and off or adjusting its strength depending on the situation.

[0172] FIG. 45 shows an example of a gradual-sloping audiogram of high-frequency hearing impairment, and FIG. 46 shows an example of a mountain-shaped audiogram. Consider, for example, a prescription that applies a gain of 1 / 2 to an input of 60 dB SPL. In the example shown in FIG. 45, the gain is 7.5 dB at 250 Hz and 25 dB at 1000 Hz. In the example shown in FIG. 46, the gain is 15 dB at 250 Hz and 5 dB at 1000 Hz. As explained with reference to FIGS. 43 and 44, when the NC function is added to the NR function, the SNR improvement effect is significant when the gain of the NR function is small. However, when the effect of the NC function is small, the SNR improvement effect is small even if the gain of the NR function is increased. Taking these factors into consideration, it can be said that noise cancellation focused on the 125 Hz to 250 Hz range in the audiogram shown in FIG. 45, and on the 500 Hz to 1000 Hz range in the audiogram shown in FIG. 46, is effective in improving listening difficulty.

[0173] 47 shows the expected noise attenuation that can be achieved by noise canceling processing. For example, curve A indicated by reference number 4734 has a noise canceling effect over a wide frequency band, although the maximum attenuation is not high. Curves B, C, and D indicated by reference numbers 4735, 4736, and 4737, respectively, have narrow frequency bands but high maximum attenuation. In this way, it is possible to adjust the curve of the expected noise attenuation by performing noise canceling processing.

[0174] As explained with reference to Figures 45 to 47, adjusting the noise canceling characteristics depending on hearing information can be said to be effective in improving listening difficulty. For example, it is effective to use curve B4735 for the audiogram shown in Figure 45, and curve D4737 for the audiogram shown in Figure 46.

[0175] Figure 48 shows an example of a horizontal audiogram. Consider a prescription that provides half the gain for a 60 dB SPL input. Since all frequencies are 40 dB HL, the gain is 20 dB at all frequencies. If we ignore the natural ear gain, the gain would be as shown by curve 4980 in Figure 49. In the case of a person with sensorineural hearing loss, even if their hearing level is high, it is common to reduce the gain for high input sound pressure levels due to feature B1. In the example input / output characteristics of a hearing aid device shown in Figure 49, the gain is low at input sound pressure levels around 90 dB SPL. Using the NC function can be said to be effective in alleviating listening difficulties. In this case, using curve A4734 in Figure 47 is effective.

[0176] Figure 50 shows an example configuration of a hearing aid device 5001 with an NC signal generation function according to a second embodiment of the present disclosure. A microphone 5002, an ADC 5004, a DAC 5005, and a speaker 5003 are drawn in a straight line from left to right, and the configuration is similar to that of the hearing aid devices shown in Figures 13 and 42. Below, the configuration of hearing aid device 5001 will be described, focusing on the differences from hearing aid device 1401 shown in Figure 14.

[0177] The amplitude / phase adjustment unit 5009 is set by the control unit 5011 using the hearing information storage unit 5015 and the audio signal input to the amplitude / phase adjustment unit 5009 (same as above). The audio signal adjusted by the amplitude / phase adjustment unit 5009 is sent to the DAC 5005.

[0178] The NC signal generation unit 5008 receives as input the audio signal from the ADC 5004 and generates a noise cancellation (NC) signal. The NC signal is a signal for spatially canceling noise that has arrived at position P2 via the second path in the hearing aid devices shown in FIGS. 13 and 42. The generated NC signal is sent to the DAC 5005. The NC signal generation unit 5008 generates the NC signal using the NC characteristics (NC characteristic curve, NC strength) determined by the NC characteristic determination unit 5030.

[0179] The DAC 5005 receives a digital signal obtained by superimposing an NC signal on an audio signal adjusted by the amplitude / phase adjustment unit 5009, and converts it into an analog signal. The speaker 5003 converts the electrical signal into sound. Therefore, the sound output from the speaker 5003 includes sound converted from the NC signal generated by the NC signal generation unit 5008, and cancels out the noise that has passed through the second path. Here, the ADC 5004 and DAC 5005 are shown as being shared by the NR unit 5007 and the NC signal generation unit 5008, but they may also be provided separately.

[0180] Fig. 51 shows an example of generating a trained model in the second embodiment of the present disclosure. Trained model 5140 receives hearing information, input audio information, and configuration information, estimates NC characteristic information, and outputs it. Fig. 51 depicts a neural network with three input nodes, one output node, and two hidden layers, but trained model 5140 is not limited to this.

[0181] The parameters of the hearing information, input audio information, and configuration information that are input to the trained model 5140 are the same as in the example of generating the trained model 1501 shown in Figure 15, and detailed explanations will be omitted here. Also, the configuration information may be omitted in some cases.

[0182] Also, as shown in FIG. 19, the trained model 5140 may generate trained models for each configuration information, but detailed explanation will be omitted here.

[0183] The NC characteristic information can use noise canceling intensity information. For example, it can be a value having a range such as 0.0 to 1.0. For example, 0.0 can be defined as noise canceling off. The noise canceling characteristic curve can be selected from multiple types. The noise canceling characteristic curve can be selected as described in FIG. 47. In this case, the NC characteristic information is the noise canceling characteristic curve. It is also possible to include intensity information in the NC characteristic information. For example, for a classification problem, the learning data can be 100% of curve A, 75% of curve A, 50% of curve A, 25% of curve A, 0% of curve A, 100% of curve B, and 75% of curve B in FIG. 47.

[0184] FIG. 52 shows an example of a user interface used by a subject (hearing-impaired person) to respond with NC characteristic information. Examples of NC characteristic information include an NC characteristic curve and NC intensity. The portable information device 5264 includes a radio button 5269 and a slider bar 5267 as a user interface for adjusting the NC characteristic curve and NC intensity. For example, when the subject changes the radio button 5269, the NC characteristic curve of the hearing aid device 5001 changes accordingly. For example, sliding the slider bar 5267 toward "strong" increases the NC intensity of the hearing aid device. In the example shown in FIG. 52, one of the options for the radio button 5269 is to select OFF for the NC function. For example, at high frequencies such as 4 kHz, the NC function is not expected to be effective, so the subject may choose not to use the NC function. The sound environment assumed as input speech information is reproduced around the subject, and in that sound environment, the subject specifies the NC characteristic curve and NC strength that are optimal for him / herself using radio buttons 5269 or slider bar 5267, and confirms by pressing OK button 5265. As a result, hearing information, input speech information, configuration information, and NC characteristic information are added to the training dataset.

[0185] FIG. 53 shows the procedure for creating training data in the form of a flowchart. First, the hearing aid device and a sound environment reproduction system (not shown) are configured based on input information (step S5301). The sound environment reproduction system reproduces sound environments such as a quiet room, a hospital, a shopping mall, or a restaurant. For example, a 5.1-channel speaker system can be configured around the subject. For example, a conversation partner may be placed in front of the subject. Next, the subject specifies NC characteristic information using a user interface such as that shown in FIG. 52 (step S5302). Next, the hearing aid device is operated according to the NC characteristic information specified by the subject (step S5303). Next, it is determined whether the OK button 5265 in the user interface shown in FIG. 52 has been pressed (step S5304). In other words, it is determined whether the subject has confirmed the NC characteristic information. If the OK button has not been pressed (No in step S5304), the process returns to step S5302, where the specification of NC characteristic information and operation using that NC characteristic information are repeated. During repetition, if there is no new specification from the subject in step S5302, the previous settings may be maintained and used as they are. On the other hand, if the OK button is pressed (Yes in step S5304), the NC characteristic information specified at that time is added to the training data set (step S5305), and this process ends.

[0186] 52 and 53 show examples in which the subject specifies candidate NC characteristics, but this is not limiting. For example, the NC characteristics of the hearing aid device may be automatically set and presented to the subject, with the subject answering whether they are comfortable or uncomfortable each time. Alternatively, for example, two different sets of NC characteristic settings may be presented to the subject, and the subject may answer which is more comfortable. In short, it is sufficient to obtain an answer regarding the NC characteristics that indicates whether they are comfortable or uncomfortable for the subject.

[0187] 54 shows an example of the configuration of the NC characteristic estimation unit 5431 when a trained model is not generated. The NC characteristic estimation unit 5431 is used in the NC characteristic determination unit 5030 of the hearing aid device 5001.

[0188] The first sound pressure level calculation unit 5422 calculates a first sound pressure level P_1(f), which is the output of the first path, using the level information for each frequency band of the input sound and the gain information. The second sound pressure level calculation unit 5423 calculates a second sound pressure level P_2(f) in the ear canal (for example, position P2 in FIGS. 13 and 48) via the second path, using the configuration information (acoustic characteristics of the earpiece, etc.) and the input sound level information. The sound pressure level difference calculation unit 5424 calculates the difference r(f) between the first sound pressure level P_1(f) and the second sound pressure level P_2(f) as shown in the above formula (2).

[0189] When P_1 and P_2 are considered in terms of sound pressure instead of sound pressure level, the sound pressure ratio P_1(f) / P_2(f) may be calculated. The NC strength / NC characteristic curve calculation unit 5432 calculates the NC strength NC_LEVEL from the sound pressure level difference r(f). For example, the NC strength NC_LEVEL can be calculated as in the following formula (6).

[0190]

number

[0191] In the above equation (6), the function f2L is a function that represents the relationship between the sound pressure level difference r(f) and the NC strength NC_LEVEL, and is approximately calculated in advance from statistical data. For example, as explained with reference to Figures 43 and 44, it can be calculated as 0.0 when the difference is large and 1.0 when the difference is small. Here, 0.0 means that the NC function is off or weak, and 1.0 means that the NC function is strong. The NC strength / NC characteristic curve calculation unit 5432 further determines the NC characteristic curve NC_CURVE from the sound pressure level difference r(f). For example, the NC characteristic curve NC_CURVE can be calculated as shown in the following equation (7).

[0192]

number

[0193] In the above equation (7), the function f2C is a function that represents the relationship between the sound pressure level difference r(f) and the NC characteristic curve NC_CURVE, and is approximately calculated in advance from statistical data. For example, characteristic curves such as curve A, curve B, and curve C shown in FIG. 47 are determined via a user interface such as that shown in FIG. 52. It is also possible to choose not to use the NC function.

[0194] Fig. 55 shows, in the form of a flowchart, the processing procedure for estimating NC characteristics in NC characteristic estimation unit 5431 shown in Fig. 54. The processing up to calculating the sound pressure level difference r(f) between first sound pressure level P_1(f) and second sound pressure level P_2(f) (steps S5501 to S5503) is the same as the processing procedure shown in Fig. 22, and therefore description thereof will be omitted here. NC strength / NC characteristic curve calculation unit 5432 calculates NC strength NC_LEVEL and NC characteristic curve NC_CURVE (step S5504). Then, NC strength NC_LEVEL and NC characteristic curve NC_CURVE are output from NC characteristic estimation unit 5431 (step S5505), terminating this processing.

[0195] A large amount of training data is required to train a trained model. As explained with reference to Figures 16, 52, and 53, the procedure for creating training data is time-consuming and labor-intensive. The key to collecting a large amount of training data is to minimize manual work. The method without using a trained model, explained with reference to Figures 54 and 55, can be used as a tool for efficiently constructing training data. First, NC characteristic candidates are prepared using a method without using a trained model. Next, the candidates are manually fine-tuned and used as training data. For example, the initial values ​​of the slider bar 5267 and radio button 5269 in Figure 52 are determined using NC characteristic candidates calculated with a method without using a trained model. This allows the test subject to determine an answer by simply trying values ​​close to the initial value. In a configuration in which the test subject answers multiple-choice questions, the number of multiple-choice questions required can be reduced. This method significantly improves work efficiency. Furthermore, the methods shown in Figures 54 and 55 that do not use trained models require light computations and can therefore be implemented even on inexpensive hearing aid devices. In other words, they can be used as a simple alternative to estimation using trained models.

[0196] FIG. 56 shows the processing procedure of the NC function in the hearing aid device 5001 in the form of a flowchart. First, the NC characteristic information is initialized (step S5601). Next, NC processing is performed on the input audio signal (step S5602). The NC signal generated by the NC processing is superimposed on the input audio signal and output from the speaker 5003. Next, recommended settings for the NC characteristic information are calculated using hearing information of the user of the hearing aid device, input audio information of the hearing aid device, and configuration information of the hearing aid device 5001 (step S5603). An example of how to calculate characteristic information will be described later with reference to FIGS. 57 and 58. Next, recommended settings are presented to the user of the hearing aid device 5001 as needed (step S5604). If the recommended settings are to be automatically applied, step S5604 may be skipped. Next, the NC characteristic information is updated based on a response from the user of the hearing aid device 5001 (step S5605). If the recommended settings are to be automatically applied, the updates are performed automatically. Examples of how to present recommended settings to the user of hearing aid device 5001 and how the user can respond will be described later with reference to Figures 29, 30, and 59 to 61. Next, a determination is made as to whether to terminate the NC process (step S5606). If the NC process is not to be terminated (No in step S5606), the process returns to step S6202 and continues. On the other hand, if the NC process is to be terminated (Yes in step S5606), the process terminates. Examples of when the NC process is to be terminated include when the user of hearing aid device 5001 turns off the NC function or when the hearing aid device 5001 is powered off. While the user has been described as the user of hearing aid device 5001, the user may be someone other than the user himself, such as a family member of the user of hearing aid device 5001. The family member may be located near the user of hearing aid device 5001, or may be located via a network.

[0197] FIG. 57 shows an example configuration of an NC characteristic determination unit that determines NC characteristics using a trained model according to the second embodiment of the present disclosure. The NC characteristic determination unit 5730 shown in FIG. 57 corresponds to the NC characteristic determination unit 5030 of the hearing aid device 5001 shown in FIG. 50. The NC characteristic determination unit 5730 receives hearing information, input audio information, and configuration information as input, and outputs NC characteristic information. The input and output are performed via the control unit 5011 in FIG. 50. The trained model 5740 can use the trained model 5140 shown in FIG. 51. The trained model 5740 receives hearing information, input audio information, and configuration information as input, and outputs NC characteristic information. The NC characteristic information is, for example, the on / off state of the NC signal generation unit 5008, the NC strength, and the NC characteristic curve.

[0198] When NC characteristic information is calculated independently for the left and right hearing aid devices 5001, it is possible to use separate NC characteristic information for the left and right, or to use unified NC characteristic information for the left and right. When unified NC characteristic information for the left and right is used, for example, it may be the average of the NC characteristic information calculated for each of the left and right, or it may be matched to NC characteristic information with weaker intensity and a wider frequency band. When the left and right hearing acuities are similar, it is preferable to use unified NC characteristic information for the left and right to reduce fluctuations in the sound localization between the left and right. In order to reduce discomfort felt by the user of the hearing aid device 5001, it is preferable to match the left and right NC characteristic information with weaker intensity and a wider frequency band when integrating the left and right NC characteristic information. When integrating the left and right NC characteristic information, the calculated NC characteristic information can be exchanged via the external interface 5013. When the left and right hearing acuities are significantly different, it is preferable to use separate NC characteristic information for the left and right.

[0199] 58 shows an example configuration of an NC characteristic determiner that determines NC characteristics without using a trained model, according to the second embodiment of the present disclosure. The NC characteristic determiner 5830 corresponds to the NC characteristic determiner 5030 of the hearing aid device 5001 shown in FIG.

[0200] Figure 59 shows, in the form of a flowchart, the processing steps for suggesting to the user that the NC characteristic curve and NC intensity be updated. If the newly calculated NC characteristic curve and NC intensity differ from the current NC characteristic curve and NC intensity, there are two methods: an automatic update method, or an update suggestion to the hearing aid user. When updating automatically, it is preferable to update with a certain time constant, as frequent updates may cause discomfort. When suggesting an update to the hearing aid user, for example, the flowchart shown in Figure 59 can be used.

[0201] First, a new NC characteristic curve and NC intensity are calculated (step S5901). This is performed using NC characteristic determination unit 5730 shown in FIG. 57 or NC characteristic determination unit 5830 shown in FIG. 58. Next, the newly calculated NC characteristic curve and NC intensity are compared with the current NC characteristic curve and NC intensity (step S5902). If the newly calculated NC characteristic curve and NC intensity do not differ from the current NC characteristic curve and NC intensity (No in step S5902), the process returns to step S5901. If the newly calculated NC characteristic curve and NC intensity differ from the current NC characteristic curve and NC intensity (Yes in step S5902), a proposal to update the NC characteristic curve and NC intensity is made to the user of the hearing aid device (step S5903). An example of the proposal method can be similar to the method described with reference to FIG. 29. Next, a response to the proposal is obtained from the user of the hearing aid device (step S5904). An example of the response method can be similar to the method described with reference to FIG. 30. The process then branches depending on the response from the user (step S5905). If the response indicates a desire to update (Yes in step S5905), the NC characteristic curve and NC strength are updated (step S5906), and the process ends. On the other hand, if the response indicates a desire not to update (No in step S5905), the NC characteristic curve and NC strength are not updated (step S5907), and the process ends.

[0202] FIG. 60 shows an example of a user interface for proposing to the user of the hearing aid device 5001 that updates the NC strength and NC characteristic curve, and for obtaining a response from the user of the hearing aid device 5001. A slider bar 6067 representing the NC strength is provided, along with a recommended range 6099. The recommended range 6099 is determined from the newly calculated NC strength. For example, the recommended range 6099A shown on the left of FIG. 60 is an example of a weaker newly calculated NC strength, and the recommended range 6099B shown on the right of FIG. 60 is an example of a stronger newly calculated NC strength. For example, the recommended range may be between 80% and 120% of the newly calculated NC strength. The user interface also includes radio buttons 6069 that allow the user to specify the NC type.

[0203] FIG. 61 shows a method for presenting the recommended range 6099. It may be as shown in FIG. 61 as recommended range 6199C or recommended range 6199D. Instead of slider bar 6067, a dial 6168 may be used to indicate the NC strength. In the case of a dial, the recommended range may be as shown in FIG. 60 as recommended range 6199E or recommended range 6199F. The "NC type" in FIGS. 60 and 61 is another way of saying the NC characteristic curve. Hearing aid device users can more easily understand the term "NC type" than the term "NC characteristic curve." The example user interface shown in FIG. 60 includes a radio button 6069 indicating the NC type, along with a recommended mark 6098. The recommended mark 6098 is determined from a newly calculated NC characteristic curve. For example, recommended mark 6098A is an example of a newly calculated NC characteristic curve called "Curve B," and recommended mark 6098B is an example of a newly calculated NC characteristic curve called "Curve A." The method of illustrating the recommendation mark 6098 may be any one of the recommendation mark 6198C, recommendation mark 6198D, recommendation mark 6198E, and recommendation mark 6198F in FIG.

[0204] In the examples shown in Figures 60 and 61, slider bar 6067 and dial 6168 are used for NC strength, and radio button 6069 is used for NC characteristic curve. However, this is not a limitation; the key is that the recommended range 6099 and recommended mark 6198 may be automatically presented according to the newly calculated NC strength and NC characteristic curve. By presenting the recommended range and recommended mark to the hearing aid device user in this way, the hearing aid device user can set the NC strength and NC characteristic curve according to their own will, while referring to the recommended range and recommended mark. After updating the settings, press OK button 6065 to exit. If you do not want to change the settings, press OK button 6065 to exit without operating slider bar 6067, dial 6168, or radio button 6069. If you want to automatically set the NC strength and NC characteristic curve (NC type), you may specify them using switch 6066.

[0205] The effects of the second embodiment are summarized below. The second embodiment makes it possible to turn the NC function on and off, adjust its strength, and propose and automatically control changes to the NC characteristic curve. Even when the NR function is used, the improvement in SNR at position P2 is slight if the amplification gain of the first path is small. In contrast, the second embodiment makes it possible to further improve the SNR, thereby alleviating the listening difficulties of users of hearing aid devices. Furthermore, the second embodiment also makes it possible to reduce power consumption, which is important for hearing aid devices. [Example]

[0206] The third embodiment of the present disclosure enables hearing measurement even in a somewhat noisy environment.

[0207] Before explaining the third embodiment, the background and the problem to be solved will be explained. Due to the feature B2 of changing the gain according to the hearing ability of the user, the hearing aid device needs to know the hearing ability of the user.

[0208] FIG. 62 shows an example of conventional hearing testing. Due to feature B3, which indicates the existence of a second path in addition to the first path, hearing cannot be accurately measured in the presence of external noise. For this reason, it is generally recommended that hearing tests be performed in a soundproof room. A subject 6295 is located in a soundproof room 6263, shielded from external noise. The subject 6295 wears a handset 6261 and holds a response button 6262. A measuring device operator 6296 operates the measuring device 6260 to send a measurement signal to the handset 6261 of the subject 6295. The subject 6295 responds using the response button 6262. The measuring device 6260 can be a device called an audiometer. In the example shown in FIG. 62, the operator 6296 and the measuring device 6260 are depicted as being outside the soundproof room 6263, but they may also be located inside the soundproof room 6263. A larger soundproof room 6263 is required to accommodate the operator 6296 and the measuring device 6260.

[0209] Figure 63 illustrates the effect of noise on hearing tests. As explained in Figure 3, the threshold at which a sound is barely audible is called the hearing threshold, and the hearing threshold indicated by reference number 6383 in Figure 63 is the hearing threshold of a person with normal hearing. The hearing threshold of a person with normal hearing is sometimes simply referred to as the hearing threshold. In pure-tone hearing tests, the level of a pure tone of a specified frequency is varied and presented to the subject 6295 through a receiver 6261 to measure the threshold. Assume that noise A indicated by reference number 6375 in Figure 63 is present during the hearing test. In this case, even a person with normal hearing who should be able to hear the pure tone (measurement signal) indicated by reference number 6386 will not be able to hear the pure tone 6386. This is because the sound pressure level of noise A 6375 is greater than that of pure tone 6386, resulting in the pure tone 6386 being masked by noise A 6375. For this reason, it is generally recommended that hearing tests be performed in a soundproof room 6263. However, the situation is different for people with hearing loss.

[0210] Figure 64 illustrates the effect of noise on hearing tests for hearing-impaired individuals. When a hearing-impaired individual's hearing threshold is hearing threshold 6484, they cannot hear pure tone (test signal) 6486. A hearing-impaired individual with hearing threshold 6484 cannot hear pure tone 6486, regardless of whether noise A6475 is present. In other words, noise A6475 does not affect the hearing test for a hearing-impaired individual with hearing threshold 6484. This is because noise A6475 is lower than the hearing threshold 6484 of the hearing-impaired individual. Thus, the amount of noise required for hearing tests depends on the subject's hearing threshold. For example, when testing the hearing of a hearing-impaired individual whose hearing is uniformly 50 dB worse than that of a person with normal hearing, it can be said that testing is possible even in an environment that is 50 dB noisier than that of a person with normal hearing. However, because the subject's hearing threshold is unknown until the hearing test is performed, the noise level that does not affect the test is also unknown until the test is performed.

[0211] FIG. 65 shows an example of a conventional supplemental hearing test. Some recent hearing aids have a hearing measurement function. A hearing aid 6501 replaces the receiver 6261 in FIG. 62. A subject 6595 wears the hearing aid 6501. An operator 6596 operates a measuring device 6560 to send a measurement signal to the hearing aid 6501 of the subject 6595. The measuring device 6560 is, for example, a PC (Personal Computer) on which hearing measurement software can be installed to play the measurement signal. Alternatively, the measuring device 6560 sends a control signal to play the measurement signal stored in the hearing aid 6501. The subject 6595 then responds verbally or with gestures. The reason why FIG. 65 is called "supplemental" is because it is based on the premise that hearing measurement will be performed in advance in a low-noise environment, as in FIG. 62. The subject 6501 is known to have a high (severe) level of hearing loss through prior testing, and it is known that the test can be performed without entering a soundproof room. The hearing test shown in Figure 65 may be used when repeat testing is desired within a few months of the previous hearing test. The reason why additional hearing tests are desired is that they are easier to perform than hearing tests in a soundproof room.

[0212] Figure 66 illustrates the effect of reducing noise during a hearing test. For example, when a person with a hearing threshold 6683 is presented with a pure tone (measurement signal) 6686 for a hearing test, the person cannot hear the pure tone 6686 if noise A, indicated by reference number 6675, is present. If noise A 6675 can be reduced to noise B 6676, the person with hearing threshold 6683 will be able to hear the pure tone 6686 because the pure tone 6686 is above the hearing threshold 6683.

[0213] To summarize, it is recommended that hearing tests be performed in a quiet environment such as a soundproof room. This is because noise affects hearing tests. The effect of noise on hearing tests depends not on the absolute level of the noise, but on the relationship between the noise and the test subject's hearing. When a test subject's hearing is impaired, louder noise is tolerated than for people with normal hearing. If noise can be reduced, even if not to the same extent as in a soundproof room, it will be possible to test the hearing of people with milder hearing loss.

[0214] Hearing tests are typically performed at otolaryngology clinics or hearing aid stores equipped with soundproof rooms. Many hearing-impaired individuals find it inconvenient to have to visit a hospital or store. There is also a need for easier use of hearing aids. For example, some existing hearing aids available at electronics retailers or online allow for setup using a simple measurement function. However, the acoustic environment of an average home is not necessarily quiet. As shown in Figures 63 and 64, the acceptable noise level increases as hearing loss progresses. However, as shown in Figures 4, 45, and 46, hearing ability varies, and even if the average hearing level is at the moderate hearing loss level, some frequency bands may still be at a normal level. Therefore, the third embodiment provides a method for more convenient and accurate hearing tests. To achieve this, a noise reduction method using noise canceling technology is used in hearing tests.

[0215] FIG. 67 shows an example of hearing measurement according to the third embodiment of the present disclosure. A subject 6795 wears a hearing aid device 6701. The subject 6795 operates a measurement device 6764 to send a measurement signal to the hearing aid device 6701 of the subject 6795. The measurement device 6764 may be, for example, a PC on which hearing measurement software is installed and which can play the measurement signal. Alternatively, the measurement device 6764 sends a control signal to play the measurement signal stored in the hearing aid device 6701. The subject 6795 then responds to the measurement signal verbally, by gesture, by pressing a button, or the like. According to the third embodiment, as shown in FIG. 67, hearing measurement can be easily performed using the hearing aid device 6701 in a place that is not necessarily quiet, such as at home.

[0216] A few considerations are relevant when considering the use of noise cancelling technology in audiometry.

[0217] FIG. 68 shows examples of passive and active noise attenuation using earpieces, headphone cushions, and the like. Earpieces and headphone cushions passively reduce noise. Generally, earpieces and the like are effective in reducing high-frequency noise. Reference numeral 6848 in FIG. 68 exemplifies passive noise attenuation using earpieces, headphone cushions, and the like. A specific example of noise attenuation 6848 is roughly 1000 Hz or higher. In contrast, the noise canceling technology shown in FIG. 41 actively reduces noise. Generally, noise canceling technology is effective in reducing low-frequency noise. Reference numeral 6847 in FIG. 68 exemplifies active noise attenuation using noise canceling technology. A specific example of noise attenuation 6847 is roughly 1000 Hz or lower. As such, there are frequency bands in which noise canceling can be expected to be effective and frequency bands in which it cannot be expected to be effective.

[0218] Pure-tone audiometry, a type of hearing test, primarily measures frequencies of 125 Hz, 250 Hz, 500 Hz, 1000 Hz, 2000 Hz, 4000 Hz, and 8000 Hz, as can be seen from the audiogram shown in Figure 4. In some cases, it may also measure frequencies of 750 Hz, 1500 Hz, 3000 Hz, and 6000 Hz. It is important to note here that the hearing test device knows the frequency of the sound being measured. Some conventional noise-canceling headphones change the noise-canceling characteristic curve depending on the type of noise in the environment. For example, they distinguish between train and airplane noise. When using noise-canceling processing with an hearing test device, not only can the characteristic curve be changed depending on the type of noise in the environment, but it can also be changed depending on the frequency of the test sound. For example, when measuring a 250 Hz tone, the goal is to reduce noise that interferes with the audibility of the 250 Hz pure tone. For example, by using characteristic curve B or characteristic curve C in FIG. 47, it is possible to increase the amount of noise attenuation in a specific frequency band compared to when characteristic curve A is used.

[0219] FIG. 69 shows an example in which the noise canceling characteristic curve changes depending on the frequency of the audiometric test signal. Noise masking the test sound is mainly at frequencies close to the test sound. For example, when a 250 MHz pure tone 6986 is the test signal, noise masking the pure tone 6986 can be effectively reduced by using characteristic curve B6935 as the noise canceling characteristic curve. Similarly, when a 500 MHz pure tone 6987 is the test signal, noise masking the pure tone 6987 can be effectively reduced by using characteristic curve C6936. On the other hand, when a 4000 MHz pure tone 6988 is the test signal, the frequency of the pure tone 6988 is outside the frequency band in which noise canceling effects can be expected, so the noise canceling function can be turned off.

[0220] In addition to changing the noise canceling characteristic curve depending on the frequency of the measurement signal, the noise canceling characteristic curve can also be changed depending on the noise in the subject's environment. For example, a characteristic curve suitable for pure tone 6986 can be changed to characteristic curve BB6938 by taking into account the noise in the subject's environment. A shape like characteristic curve BB6938 is effective when noise at frequencies lower than the measurement signal frequency is louder. Noise masking occurs over a wider range in the high-frequency range than in the low-frequency range, based on the noise frequency band. Similarly, a characteristic curve suitable for pure tone 6987 can be changed to characteristic curve CC6939 by taking into account the noise in the subject's environment.

[0221] Figure 70 shows an example configuration of a hearing aid device 7001 that uses an NC signal generation function for hearing measurement according to a third embodiment of the present disclosure. In Figure 70, a microphone 7002, an ADC 7004, a DAC 7005, and a speaker 7003 are drawn in a straight line from left to right, and this configuration is similar to the hearing aid devices shown in Figures 13 and 42. Below, we will explain the differences from hearing aid device 5001 shown in Figure 50, and mainly the function and operation of each part during hearing measurement.

[0222] The NC signal generator 7008 receives the audio signal from the ADC 7004 as input and generates a noise cancellation signal (NC signal). The NC signal is a signal used to spatially cancel noise that has traveled via the second path and arrived at position P2 in the hearing aid devices shown in FIGS. 13 and 42. The NC signal generator 7008 generates the NC signal using the NC characteristics (NC characteristic curve, NC strength) determined by the NC characteristics determiner 7030. The generated NC signal is sent to the DAC 7005.

[0223] The sound source storage and playback unit 7012 plays back a measurement signal. The played-back measurement signal may be stored PCM (pulse code modulation) data or the like, or may be generated using a formula. The level of the played-back measurement signal may be changed under control of the control unit 7011. The measurement signal may be supplied from the sound source storage and playback unit 7012 or from an external interface 7013. In this case, the frequency and level of the measurement signal are also transmitted via the external interface 7013. If not transmitted, the hearing aid device 7001 may detect them from the measurement signal. In either case, the hearing aid device 7001 knows the frequency and level of the measurement signal. The NR unit 7007 is not used for the hearing measurement function, so it does not input an audio signal from the ADC 7004, and instead outputs the measurement signal input from the sound source storage and playback unit 7012 or the external interface 7013 as is.

[0224] The amplitude / phase adjuster 7009 may be configured to adjust the amplitude of the measurement signal according to the hearing level being measured. The measurement signal output from the amplitude / phase adjuster 7009 is sent to the DAC 7005. The DAC 7005 converts the digital signal into an analog signal. The speaker 7003 converts the electrical signal into sound. The sound output from the speaker 7003 includes sound converted from the NC signal generated by the NC signal generator, and cancels out the noise that has passed through the second path. The sound output from the speaker 7003 includes the measurement signal. Here, the ADC 7004 and DAC 7005 are shown as being shared by the NR unit 7007 and the NC signal generator 7008, but they may also be configured to be provided separately.

[0225] The external interface 7013 may receive measurement signals. Data stored in the configuration information storage unit 7014 and the trained model storage unit 7017 also passes through the external interface 7013. Data stored in the measured hearing information storage unit 7915 may be transmitted to an external device via the external interface 7013. The external interface 7013 can also be used to transmit information from the hearing aid device 7001 to a user of the hearing aid device 7001, and to receive information from a user of the hearing aid device 7001.

[0226] Hearing information storage unit 7015 stores the hearing level of the user measured using hearing aid device 7001. Of course, hearing information storage unit 7015 may also store hearing measurement results such as those shown in Fig. 4. Hearing information storage unit 7015 may also store the presence or absence of noise canceling at the time of measurement, its characteristics, noise level, etc.

[0227] The sensor information section 7019 contains information on various sensors, including an acceleration sensor and a gyro sensor.

[0228] 71 shows an example of generating a trained model in the third embodiment of the present disclosure. The trained model 7140 receives measurement signal information, input audio information, and configuration information, estimates NC characteristic information, and outputs it.

[0229] In the case of pure tone audiometry, the measurement signal information may be the frequency and level of the measurement signal. The unit of the measurement signal level value may be the audiometer dial level (dB HL) or sound pressure level (dB SPL). In addition to pure tone audiometry, self-recording audiometry may also be used. In short, any measurement signal for threshold measurement that measures the hearing threshold for each frequency will suffice.

[0230] The range of training data for measurement signal information will now be described. The measurement value range of pure tone audiometry is -20 dB HL to 120 dB HL, as shown in the audiogram in FIG. 2. For example, it can be set to -20 dB HL to 120 dB HL. On the other hand, even within the range of pure tone audiometry, there is no useful information below 0 dB HL when considering use with hearing aid device 7001. For average hearing levels above 90 dB HL, the hearing aid effect of hearing aid devices is low, and this level is considered appropriate for cochlear implants. Taking these factors into consideration, it is effective to set the training data range to, for example, 0 dB HL to 90 dB HL or 0 dB HL to 80 dB HL. Values ​​outside this range can be rounded to the boundary value. When expressed in terms of ear canal sound pressure, the range is broadly estimated to be 0 dB SPL to 110 dB SPL, although this depends on the frequency.

[0231] The input audio information is an audio signal picked up by microphone 7002 of hearing aid device 7001. During hearing measurement, the input audio information refers to environmental noise. Instead of an audio signal, a level value converted into a sound pressure level or the like can also be used. Furthermore, the information may be in the form of an amplitude spectrum or a power spectrum. The naked ear gain may be added to the input audio signal to determine the ear canal sound pressure.

[0232] When hearing aid device 7001 is used for both ears, the input audio information is different sounds picked up by left and right microphones 7002. The configuration may be such that the sounds from left and right microphones 7002 are processed independently, or may be such that they are processed together, as in the example of generating a trained model shown in FIG.

[0233] The range of learning data for input audio information will now be described. Generally, the loudness of sounds received in an environment is broadly estimated to be between 0 dB SPL and 140 dB SPL. When considering use with hearing aid device 7001, it is effective to set the range of loudness of sounds received in an environment to, for example, 20 dB SPL to 100 dB SPL. In the case of ear canal sound pressure, it is effective to set it to, for example, 20 dB SPL to 120 dB SPL.

[0234] The training data range of the measurement signal information and the training data range of the input voice information may be limited at the data set stage, or may be limited as preprocessing when training the trained model 7140.

[0235] When training the trained model 7140, any training data can be used, but it is generally useful to normalize the training data to improve discrimination performance and speed up training. Normalization is effective for both the training data for hearing information and the training data for input speech information. In this case, for example, the range of the training data described above is normalized to a range of 0.0 to 1.0. Normalization may be performed at the training dataset stage, or normalization may be performed as preprocessing when training the trained model 7140.

[0236] The configuration information includes information such as the type of earpiece, wearing method, and characteristics of the hearing aid device housing (as described above). When wearing a hearing aid in one ear, care must be taken because masking of the ear opposite the measurement ear cannot be performed. It also includes information such as the type of earpiece (dome) of the hearing aid device 7001 (as described above). It may also include information such as the RECD, which is commonly used to adjust conventional hearing aids. For example, the intensity and timbre of the sound from the second path in the hearing aid devices shown in Figures 13 and 42 are affected by the type of earpiece. When comparing the measurement sound with the sound from the second path, it is important to predict the sound from the second path, and the type of earpiece and RECD are useful for this prediction.

[0237] The configuration information may be omitted in some cases. For example, the product configuration may be such that the earpiece is designed for binaural use and only one type of acoustic characteristics is available. This is expected to be the case when hearing aid functions are used with TWS, etc.

[0238] Also, as shown in Figure 19, the trained model 7140 may generate a trained model for each configuration information, but detailed explanation will be omitted here.

[0239] The NC characteristic information can use noise canceling intensity information. For example, it can be a value having a range, such as 0.0 to 1.0. For example, 0.0 may be defined as noise canceling off. The noise canceling characteristic curve may be selected from multiple types. The noise canceling characteristic curve can be selected as described with reference to Figures 47 and 69. The NC characteristic information output from the trained model 7140 is the same as in the example of generating the trained model 5140 shown in Figure 51, and a detailed description thereof will be omitted here.

[0240] FIG. 72 shows an example of a user interface when creating training data. Portable information device 7264 displays the frequencies and configuration information of the measurement signal for pure tone audiometry. For example, by checking the frequencies to be trained, the checked frequencies are executed in order. 125 Hz, 250 Hz, 500 Hz, 1000 Hz, 2000 Hz, 4000 Hz, and 8000 Hz are frequencies that are normally always measured in pure tone audiometry and may be selected by default. 750 Hz, 1500 Hz, 3000 Hz, and 6000 Hz are checked as needed. In the example shown in FIG. 72, the configuration information includes the type of earpiece and earmold to be used, and the selection of whether the hearing aid device is left or right. Once the settings are complete, the setting information is transmitted to hearing aid devices 7201A and 7201B, and hearing aid devices 7201A and 7201B operate according to the settings. Although the hearing aid devices 7201A and 7201B are depicted as conventional RIC-type hearing aids, and the hearing aid device 8201B is depicted as having a shape similar to that of a conventional TWS, this is not limitative. The connection between the hearing aid devices 7201A and 7201B and the portable information device 7264 may be wireless or wired.

[0241] FIG. 73 shows an example of a user interface used by a test subject (hearing-impaired person) to respond with NC characteristic information. The frequency of the current measurement signal may be displayed as reference information. In the example shown in FIG. 73, the frequency of the current measurement signal is 500 Hz. Examples of NC characteristic information include an NC characteristic curve and NC intensity. A portable information device 7364 includes radio buttons 7369 and a slider bar 7367 as a user interface for adjusting the NC characteristic curve and NC intensity. NC characteristic information is learned for each frequency of the measurement signal. As described with reference to FIG. 69, for example, when the measurement signal is a pure tone 6986, the test subject makes adjustments based on NC characteristic curve B 6935. Radio button 7369A is used to narrow or widen the width of the NC characteristic curve. Radio button 7369B is used to lower or raise the center frequency of the NC characteristic curve. This operation results in, for example, an NC characteristic curve BB 6938. For example, when the measurement signal is a pure tone 6987, the NC characteristic curve CC6939 is obtained by the same operation. In this example, width and frequency are selected as parameters for adjusting the NC characteristic curve, but this is not limited to these. For example, sliding the slider bar 7367 toward "strong" increases the NC strength of the hearing aid device 7001. In this example, switch 7366 allows the user to select whether to turn off the NC function. For example, the NC function is not effective at high frequencies such as 4 kHz, so one may choose not to use the NC function. The sound environment assumed as input audio information is reproduced around the subject, and in that sound environment, the subject specifies the NC characteristic curve and NC strength that are optimal for that subject using radio button 7369 or slider bar 7367 and then presses OK button 7365 to confirm. This adds the measurement signal information, input audio information, configuration information, and NC characteristic information to the training dataset.

[0242] The procedure for creating learning data is shown in the form of a flowchart in Figure 74. The illustrated processing procedure is basically the same as that shown in Figure 53, so a description thereof will be omitted here.

[0243] 73 and 74 show examples in which the subject specifies NC characteristic candidates, but this is not limiting. For example, the NC characteristics of the hearing aid device 7001 may be automatically set and presented to the subject, with the subject responding each time as to whether they are comfortable or uncomfortable. Alternatively, for example, two different sets of NC characteristic settings may be presented to the subject, and the subject may respond as to which is more comfortable. In short, it is sufficient to obtain a response regarding the NC characteristics that indicates whether they are comfortable or uncomfortable for the subject.

[0244] 75 shows an example of the configuration of the NC characteristic estimation unit 7531 when a trained model is not generated. The NC characteristic estimation unit 7531 is used in the NC characteristic determination unit 7030 of the hearing aid device 7001.

[0245] The measurement signal information may be the frequency or level of the test sound, or it may be the measurement signal itself. The first sound pressure level calculation unit 7528 calculates the sound pressure level P_1 of the test sound. If the level value input to the first sound pressure level calculation unit 7528 is already the sound pressure level of the test sound, the input is output as is. The second sound pressure level calculation unit 7523 uses the frequency information and configuration information (acoustic characteristics of the earpiece, etc.) of the test sound and the input audio level information to estimate the sound in the ear canal via the second path (for example, position P2 in the hearing aid device shown in Figures 13 and 48) and calculates the sound pressure level P_2 equivalent to narrow-band noise that may mask the test sound. For example, ISO 532-1975 can be used for this calculation. As shown in Figure 42, a microphone 4229 may be placed inside the hearing aid device 4201 to directly obtain the sound pressure level P_2 of the sound in the ear canal via the second path. The timing for collecting sound from the microphone 4229 inside the hearing aid device 4201 is preferably when no test sound is being output. If the test sound is an intermittent sound, it may be collected between the timings of the test sounds. If sound is also collected when a test sound is being output, the collected audio signal is estimated by excluding the measurement signal. The required NC strength calculation unit 7524 calculates the noise canceling strength NC_NEED required for measurement from the difference between the level of the noise via the second path and the level of the test sound, using the following equation (8):

[0246]

number

[0247] In the above equation (8), the constant β is a correction value between -5 dB and 10 dB. NC_NEED is non-negative. The NC strength / NC characteristic curve calculation unit 7532 calculates the NC strength NC_LEVEL and the NC characteristic curve NC_CURVE from the required NC strength NC_NEED. The NC strength NC_LEVEL is calculated using the function f3L as shown in the following equation (9).

[0248]

number

[0249] In the above formula (9), f3L for calculating the NC strength NC_LEVEL is expressed as in the following formula (10).

[0250]

number

[0251] In the above equation (10), th_NC is the threshold value for the effect of NC processing. When the required NC strength (NC_NEED) is greater than the threshold value th_NC, the achievable NC effect cannot meet the requirement. th_NC is a value determined from the characteristic curve of the NC function. For example, if the noise attenuation amount at 250 Hz of a certain characteristic curve is 25 dB, th_NC can be determined as 25 dB. In addition, the NC characteristic curve NC_CURVE is calculated using the function f3C as shown in the following equation (11).

[0252]

number

[0253] In the above equation (11), f3C, which calculates the NC characteristic curve, may select an NC characteristic curve so that NC_LEVEL has a sufficient value while prioritizing the width of the frequency band. For example, when the required NC strength NC_NEED is small, priority is given to the curve A4734 shown in Figure 47, and when the required NC strength NC_NEED is large, priority is given to the curve C4736 shown in Figure 47.

[0254] Fig. 76 shows, in the form of a flowchart, the processing procedure for estimating the NC characteristic in the NC characteristic estimation unit 7531 shown in Fig. 75. The first sound pressure level calculation unit 7528 calculates a first sound pressure level P_1 of the measurement sound from the measurement signal information (step S7601), and the second sound pressure level calculation unit 7523 calculates a second sound pressure level P_2 equivalent to narrowband noise masking the measurement sound from the frequency information, input audio information, and configuration information of the constant sound (step S7602). Next, the required NC strength calculation unit 7524 calculates the required NC strength NC_NEED from the first sound pressure level P_1 and the second sound pressure level P_2 (step S7603), the NC strength / NC characteristic curve calculation unit 7532 calculates the NC strength NC_LEVEL and the NC characteristic curve NC_CURVE from the required NC strength NC_NEED (step S7604), the NC characteristic estimation unit 7531 outputs the NC strength NC_LEVEL and the NC characteristic curve NC_CURVE (step S7605), and this processing is terminated.

[0255] Although Fig. 75 shows an example of a configuration in which NC characteristics are estimated using measurement signal information, input audio information, and configuration information, it is also possible to determine NC characteristic information using only measurement signal information. In other words, it may be determined like characteristic curve B 6935 or characteristic curve C 6936 described in Fig. 69.

[0256] A large amount of training data is required to train a trained model. As explained in the procedure examples with reference to Figures 72 to 74, creating training data is a time-consuming and labor-intensive task. The key to collecting a large amount of training data is to reduce the amount of manual work required as much as possible. The methods without using a trained model, as explained in Figures 75 and 76, can be used as a tool for efficiently constructing training data. First, NC characteristic candidates are prepared using a method without using a trained model. Next, the candidates are manually fine-tuned and used as training data. For example, the initial values ​​of the slider bar 7367 and radio button 7369 in Figure 73 are determined using NC characteristic candidates calculated using a method without using a trained model. This allows the test subject to determine an answer by simply trying values ​​close to the initial value. In a configuration in which the test subject answers multiple-choice questions, the number of multiple-choice questions required can be reduced. This method significantly improves work efficiency. Furthermore, the methods shown in Figures 75 and 76 that do not use trained models require light computations and can therefore be implemented even on inexpensive hearing aid devices. In other words, they can be used as a simple alternative to estimation using trained models.

[0257] Figure 77 shows in the form of a flowchart the processing procedure of the NC function in the hearing aid device 7001. This processing procedure is basically the same as in Figure 56, so a description thereof will be omitted here.

[0258] 78 shows a configuration example of an NC characteristic determination unit that uses a trained model for estimation according to the third embodiment of the present disclosure. The configuration of the NC characteristic determination unit 7830 is basically the same as that of the NC characteristic determination unit shown in FIG. 57, and therefore a description thereof will be omitted here.

[0259] Fig. 79 shows a configuration example of an NC characteristic determination unit that does not use a trained model according to the third embodiment of the present disclosure. The configuration of the NC characteristic determination unit 7930 is basically the same as that of the NC characteristic determination unit shown in Fig. 58, and therefore a description thereof will be omitted here.

[0260] The process of suggesting to the user that the NC characteristic curve and NC intensity be updated can be, for example, the process shown in the flowchart of FIG. 59. If the newly calculated NC characteristic curve and NC intensity differ from the current NC characteristic curve and NC intensity, there are two methods: an automatic update method, or an update suggestion to the user of the hearing aid device 7001. When updating automatically, it is preferable to update with a certain time constant, since frequent updates may cause discomfort. To suggest an update to the user of the hearing aid device 7001, for example, the flowchart shown in FIG. 59 can be used. In step S5901, a new NC characteristic curve and NC intensity are calculated. This is as shown in FIGS. 78 and 79. In step S5902, the newly calculated NC characteristic curve and NC intensity are compared with the current NC characteristic curve and NC intensity. If the newly calculated NC characteristic curve and NC intensity do not differ from the current NC characteristic curve and NC intensity (No in step S5902), the process returns to step S5901. If the newly calculated NC characteristic curve and NC intensity differ from the current NC characteristic curve and NC intensity (Yes in step S5902), the process proceeds to step S5903, where a proposal to update the NC characteristic curve and NC intensity is made to the user of the hearing aid device 7001. An example of the proposal method can be performed as in FIG. 29. Next, in step S5904, a response to the proposal is obtained. An example of the response method can be performed as in FIG. 30. In step S5905, the process branches depending on the response. If the response indicates a desire to update (Yes in step S5906), the process proceeds to step S5906, where the NC characteristic curve and NC intensity are updated, and the process ends. If an update is not desired (No in step S5906), the process proceeds to step S5907, where the NC characteristic curve and NC intensity are not updated, and the process ends.

[0261] FIG. 80 shows another example of a user interface used to suggest updates to the NC strength and NC characteristic curve to the user of the hearing aid device 7001 and to receive a response from the user of the hearing aid device 7001. The basic configuration of the user interface is the same as the example shown in FIG. 60. In the example shown in FIG. 80, the NC characteristic curve is adjusted, for example, by adjusting the width and frequency. A radio button 8069A representing the width and a radio button 8069B representing the frequency are provided, along with a recommendation mark 8098. The recommendation marks 8098A-D are determined from the newly calculated NC characteristic curve. For example, recommendation mark 8098A is an example of "0" indicating that the width of the newly calculated NC characteristic curve is standard, and recommendation mark 8098C ​​is an example of "+1" indicating that the width of the newly calculated NC characteristic curve is slightly wider. Recommendation mark 8098B is an example of "0" indicating that the frequency of the newly calculated NC characteristic curve is standard, and recommendation mark 8098D is also an example of "0" indicating that the frequency of the newly calculated NC characteristic curve is standard.

[0262] FIG. 81 shows another example of a user interface that presents a recommended range and receives a response from the hearing aid device user. The basic configuration of the user interface is the same as the example shown in FIG. 61. The recommendation mark 8098 shown in FIG. 80 may be the recommendation mark 8198E, 8198F, 8198G, or 8198H shown in FIG. 81. While FIGS. 80 and 81 use the slider bar 8067 and dial 8168 for the NC strength and the radio button 8069 for the NC characteristic curve as examples, the present invention is not limited to these. In short, it is sufficient if the recommended range 8099 and recommendation mark 8098 are automatically presented according to the newly calculated NC strength and NC characteristic curve. In this way, by presenting the recommended range and recommendation mark to the hearing aid device user, the hearing aid device user can set the NC strength and NC characteristic curve according to their own will, while referring to the recommended range and recommendation mark. After updating the settings, press the OK button 8065 to exit. If you do not want to change the settings, press the OK button 8065 to finish without operating the slider bar 8067, dial 8068, or radio button 8069. If you want to set the NC strength and NC characteristic curve automatically, you may make it possible to specify this using switch 8066A. The NC on / off setting may be made possible to specify using switch 8066B.

[0263] Figure 82 shows a flowchart of the process for suggesting changes to the sound environment to the user. If the effect of NC does not meet the needs, the hearing aid user can be suggested to suspend or stop the measurement. For example, if the sound of a vacuum cleaner is a problem, the user can stop the vacuum cleaner. If the traffic noise on the road in front of the room is a problem, the user can move to a room farther away from the road.

[0264] First, the required NC strength NC_NEED is calculated (step S8201). Next, the effect of NC is determined (step S8202). Specifically, the required NC strength NC_NEED is compared with a threshold value th_NC. If the required NC strength NC_NEED is greater than the threshold value th_NC, it is determined that the achievable effect does not satisfy the need, and if the required NC strength NC_NEED is equal to or less than the threshold value th_NC, it is determined that the achievable effect satisfies the need.

[0265] If NC is effective (Yes in step S8202), the process returns to step S8201. On the other hand, if NC is not effective (No in step S8202), the user of the hearing aid device is notified that NC is not effective, and a suggestion is made to the user of the hearing aid device 7001 to change the sound environment (step S8203). An example of the suggestion method can be performed in the same way as in FIG. 29. In step S8204, a response to the suggestion is obtained from the user of the hearing aid device. An example of the response method can be performed in the same way as in FIG. 30.

[0266] The process branches depending on the answer received from the user in step S8204. If the user answers that a change is desired (Yes in step S8205), the hearing test is interrupted (step S8206), and this process ends. If the hearing test is resumed in a quieter environment, the hearing test may be restarted from the beginning or continued from where it left off. Because hearing test is time-consuming, continuing from where it left off reduces the burden on the user. When the test is interrupted, a message to that effect may be displayed. On the other hand, if the user answers that a change is not desired (No in step S8205), the hearing test is stopped (step S8207), and this process ends. When the test is stopped, a message to the user may be displayed informing them that the test cannot be performed due to environmental noise.

[0267] Figure 83 shows an example of informing a hearing aid user of noises that may affect hearing tests. For example, in Figure 63, noises that may affect hearing tests refer to noise A6375 when a person with a hearing threshold 6383 undergoes a 250 Hz hearing test. It can be difficult for ordinary people who are not sound experts to recognize which sounds are environmental noises that may affect hearing tests. Therefore, environmental noises that are problematic at the hearing test site are picked up by the microphone of the hearing aid device 8301, amplified, and output to the user of the hearing aid device 8301. The speaker used for outputting the sound may be the speaker of the hearing aid device 8301 or the speaker of a portable information device 8364A connected to the hearing aid device 8301. The amplification may be limited to the problematic frequency band. For this purpose, a band-pass filter, low-pass filter, high-pass filter, band-stop filter, etc. can be used. By amplifying the environmental noise in question to a level that the user of the hearing aid device 8301 can hear and playing it to the user, the user of the hearing aid device 8301 can specifically identify the environmental noise in question and take measures to address it. Such measures include turning off the power switch of the device generating the noise or moving to a room where the noise is not present. The environmental noise output to the user of the hearing aid device 8301 may be configured to output the sound picked up by a microphone in real time, or may be configured to be recorded and then output. The person using the portable information device 8364 to check the environmental noise may be someone other than the user of the hearing aid device 8301, such as a family member. The family member may be located next to the user of the hearing aid device 8301, or may be located remotely from the user of the hearing aid device 8301 and connected to the portable information device 8364 via a network.

[0268] When a hearing test is performed using NC, this fact may be recorded along with the hearing test data. For example, it may be recorded that noise canceling is used at 125 Hz and 250 Hz, but not at other frequencies. Furthermore, the environmental noise level may be recorded along with the hearing test data. The environmental noise level may be, for example, a single value representing the entire frequency band, or a value for each frequency of the hearing test. The hearing test results can be transmitted externally to the hearing aid device via the external interface 7013. At this time, the use or non-use of noise canceling, the NC characteristic curve, the NC strength, the environmental noise level, and configuration information can be transmitted along with the hearing test data. This allows the conditions under which the test results were performed to be confirmed later. The hearing test results transmitted externally to the hearing aid device 8301 via the external interface can be displayed, for example, on the screen of a mobile information device 8364. For example, the audiogram shown in FIG. 4 is a commonly used format and is easy to understand. The external destination of the hearing aid device 8301 may be a third party information device (not shown) connected to a network.

[0269] 84 to 89 show examples of displaying hearing test results. In this example, an NC use notation 8433 is displayed inside and outside the audiogram so that the frequencies at which the hearing test was performed using NC can be identified. This example shows the hearing test results using noise canceling at 125 Hz and 250 Hz. NC use notation 8433A in FIG. 84 is an example of displaying an "N" mark near the hearing measurement value. NC use notation 8433B in FIG. 85 is an example of displaying an "N" mark at the corresponding frequency position outside the audiogram. NC use notation 8433C in FIG. 86 is an example of displaying the corresponding frequency within the audiogram with shading. NC use notation 8433D in FIG. 87 is an example of displaying the color of the mark representing the hearing test result on the audiogram in a distinctive way. NC use notation 8433E in FIG. 88 is an example of displaying the shape of the mark representing the hearing test result on the audiogram in a distinctive way. NC use notation 8433F in FIG. 89 is an example of displaying the frequency at which the hearing test was performed using NC. This is not limited to NC use notations 8433A-F, and other notation methods may be used. In short, any format that allows the user to read that the hearing test was performed using NC is sufficient. The hearing test results may be displayed on the screen of a portable information device or the like, or may be printed on paper, etc.

[0270] Although the estimation of NC characteristic information in the third embodiment of the present disclosure has been described in the form of a hearing aid device, it is also possible to add a noise canceling function to the receiver 6261 of the conventional measuring device 6260 described in Fig. 62. In this case, the user interfaces described in Fig. 29, Fig. 30, Fig. 80, and Fig. 81 can be used in the measuring device 6260 or in information equipment (not shown) operated by the measuring device operator 6296.

[0271] The effects of the third embodiment are summarized below. In a noisy environment, the noise masks the test sound, hindering accurate measurement. In contrast, the third embodiment performs noise canceling processing in accordance with the test sound, thereby efficiently reducing noise that adversely affects the measurement, thereby enabling hearing measurement even in somewhat noisy environments. This increases the opportunities for easy hearing measurement, for example, even in places like the home. Furthermore, by displaying the fact that the measurement was performed using noise canceling together with the hearing measurement results, it becomes possible to later check the conditions under which the measurement results were performed. [Industrial Applicability]

[0272] Although the present disclosure has been described in detail above with reference to specific embodiments, it is obvious that those skilled in the art can make modifications or substitutions to the embodiments without departing from the spirit and scope of the present disclosure.

[0273] The present disclosure can be applied to various hearing aid devices with different configuration information, such as the wearing method, including whether the device is worn in one ear or both ears, and whether it is an in-the-ear type or behind-the-ear type, and information regarding the characteristics of the hearing aid device housing, such as the type of earpiece (dome) and the shape of the in-the-ear type or behind-the-ear type. Furthermore, although the present specification has mainly described embodiments in which the present disclosure is applied to hearing aid devices, it can also be similarly applied to TWS with hear-through functions, headphones, earphones, sound collectors, head-mounted displays (HMDs), and the like.

[0274] In short, the present disclosure has been described in the form of examples, and the contents of the specification should not be interpreted as limiting. To determine the gist of the present disclosure, the claims should be taken into consideration.

[0275] The present disclosure may also be configured as follows.

[0276] (1) an information estimation unit that estimates control information based on hearing threshold information relating to hearing thresholds for each frequency and input audio information; a signal processing unit that processes an input audio signal based on the estimated control information; An audio processing device comprising:

[0277] (2) The hearing threshold information includes at least one of the results of a threshold measurement that measures the hearing threshold for each frequency of the user of the audio processing device, or information on the sound pressure level for each frequency of a measurement signal used in the hearing measurement function. The audio processing device according to (1) above.

[0278] (2-1) The hearing threshold information includes a measurement result of a hearing threshold measurement. The audio processing device according to (2) above.

[0279] (2-2) The hearing threshold measurement includes at least one measurement result of pure tone audiometry or self-recording geometry, The audio processing device according to (2-1) above.

[0280] (2-3) The measurement results of the pure tone audiometry include at least one of air conduction audiometry and bone conduction audiometry. The audio processing device according to (2-2) above.

[0281] (2-4) The hearing threshold information includes gain information calculated by a prescription formula from the results of a hearing test. The audio processing device according to (2) above.

[0282] (2-5) The hearing threshold information includes a measurement result at the hearing threshold, The audio processing device according to (2) above.

[0283] (2-6) The measurement on the hearing threshold includes at least one measurement result of a speech discrimination measurement or a distorted speech discrimination measurement, The audio processing device according to (2-5) above.

[0284] (2-7) The audiometry function is a hearing threshold measurement. The audio processing device according to (2) above.

[0285] (2-8) The hearing threshold measurement includes at least one measurement result of pure tone audiometry or self-recording geometry, The audio processing device according to (2-7) above.

[0286] (2-9) The measurement results of the pure tone audiometry include at least one of air conduction audiometry and bone conduction audiometry. The audio processing device according to (2-8) above.

[0287] (2-10) The frequency and sound pressure level of the test signal are limited to the frequency and sound pressure level required for hearing measurement. The audio processing device according to (2) above.

[0288] (3) The input audio information is an input audio of an audio processing device that outputs an audio signal output from the signal processing unit. The audio processing device according to any one of (1) and (2) above.

[0289] (3-1) The input voice information includes an utterance of the user's own voice by the voice processing device. The audio processing device according to (3) above.

[0290] (3-2) The input audio information includes a level value of the input audio of the audio processing device. The audio processing device according to (3) above.

[0291] (3-3) The input voice information includes at least one of an amplitude spectrum and a power spectrum of the input voice of the voice processing device. The audio processing device according to (3) above.

[0292] (4) The information estimation unit further estimates the control information based on configuration information of the audio processing device. The audio processing device according to (1) above.

[0293] (4-1) The configuration information includes information indicating whether the sound processing device is for single ear use or for double ear use. The audio processing device according to (4) above.

[0294] (4-2) The configuration information includes at least one of the type of earpiece connected to the sound processing device and acoustic characteristic information of the earpiece. The audio processing device according to (4) above.

[0295] (5) The information estimation unit estimates at least one of information on an allowable delay amount in the signal processing unit and noise canceling characteristic information on a noise canceling process performed in the signal processing unit. The audio processing device according to any one of (1) to (4) above.

[0296] (5-1) The information estimation unit estimates an allowable delay amount for the entire processing in the signal processing unit. The audio processing device according to (5) above.

[0297] (5-2) The information estimation unit estimates an allowable delay amount for a part of processing in the signal processing unit. The audio processing device according to (5) above.

[0298] (5-3) The part of the processing includes at least one of a noise reduction processing and a sudden sound suppression processing. The audio processing device according to (5-2) above.

[0299] (5-4) The allowable delay amount is a delay time or a delay sample number. The audio processing device according to (5) above.

[0300] (6) The noise canceling characteristic information includes at least one of a noise canceling characteristic curve or a noise canceling intensity. The audio processing device according to (5) above.

[0301] (7) The information estimation unit estimates the control information using a trained model trained in advance with at least the user's hearing information and the input voice information as explanatory variables and the control information as a target variable. The audio processing device according to any one of (1) to (6) above.

[0302] (8) the signal processing unit processes an input audio signal of the audio processing device; the information estimation unit estimates the control information using the trained model trained using configuration information of the voice processing device as a condition. The audio processing device according to (7) above.

[0303] (9) The trained model is composed of a first trained model and a second trained model; The output of the second trained model is used as an input of the first trained model. The audio processing device according to any one of (7) and (8) above.

[0304] (9-1) The sound processing device is applied to a hearing aid device, The training dataset for the first trained model requires the cooperation of a hearing-impaired person, and the training dataset for the second trained model does not require the cooperation of a hearing-impaired person; The audio processing device according to (9) above.

[0305] (9-2) processing an input audio signal of the audio processing device; the second trained model is trained in advance using input speech information and configuration information of the speech processing device as explanatory variables and information on the presence or absence of one's own voice as a target variable; The first trained model is trained in advance using hearing information, input speech information, configuration information of the speech processing device, and information on the presence or absence of one's own voice as explanatory variables and allowable delay information as a target variable. The audio processing device according to (9) above.

[0306] (9-3) the signal processing unit processes an input audio signal of the audio processing device, the second trained model is trained in advance using sensor information as an explanatory variable and body movement information or face information as a target variable; The first trained model is trained in advance using hearing information, input speech information, configuration information of the speech processing device, connection information of the speech processing device, sensor information, and body movement information or face information as explanatory variables, and allowable delay amount information as a target variable. The audio processing device described above (Y).

[0307] (9-3-1) The connection information includes type information of an external device that connects to the hearing aid device and transmits audio information to the hearing aid device. The audio processing device according to (9-3) above.

[0308] (9-3-2) The connection information includes content genre information of content transmitted from an external device that is connected to the audio processing device and transmits audio information to the audio processing device. The audio processing device according to (9-3) above.

[0309] (9-3-3) The sensor information includes sensor information obtained from a sensor mounted on the voice processing device. The audio processing device according to (9-3) above.

[0310] (9-3-4) The sensor information includes at least one of gyro sensor information and camera information. The audio processing device according to (9-3) above.

[0311] (10) The information estimation unit calculating a first sound pressure level from gain information calculated by a prescription formula from the measurement results and a band-specific sound pressure level of the input sound to the sound processing device; calculating a second sound pressure level from the band-specific sound pressure levels and acoustic characteristics based on configuration information of the audio processing device; determining, as the control information, an allowable delay amount in the signal processing unit or noise canceling characteristic information related to a noise canceling process performed in the signal processing unit, based on a difference between the first sound pressure level and the second sound pressure level; The audio processing device according to (2) above.

[0312] (10-1) In a user interface for creating learning data for a trained model that estimates control information, the determined control information is used as an initial value. The audio processing device according to (10) above.

[0313] (11) The information estimation unit calculating a first sound pressure level from frequency information and sound pressure level information of the measurement signal and a band-specific sound pressure level of the input sound to the sound processing device; calculating a second sound pressure level from the band-specific sound pressure levels and acoustic characteristics based on configuration information of the audio processing device; calculating a noise canceling strength required for the hearing test from a difference between the first sound pressure level and the second sound pressure level; determining, based on the required noise canceling strength, noise canceling characteristics relating to the noise canceling process performed in the signal processing unit as the control information; The audio processing device according to (2) above.

[0314] (11-1) In a user interface for creating learning data for a trained model that estimates noise canceling characteristic information, the determined noise canceling characteristics are used as initial values. The audio processing device according to (11) above.

[0315] (12) The configuration information includes at least one of the type of earpiece of the audio processing device, a wearing method, and characteristic information of a housing of the audio processing device. The voice processing device according to any one of (4), (10), and (11) above.

[0316] (13) When the first control information currently used in the speech processing device is different from the second control information newly estimated by the information estimation unit, a process for changing the control information used in the speech processing device is performed. The audio processing device according to any one of (1) to (12) above.

[0317] (13-1) automatically changing the first control information to the second control information; The audio processing device according to (13) above.

[0318] (14) recommending a change from the first control information to the second control information to a user of the voice processing device, and changing the control information based on a response from the user. The audio processing device according to (13) above.

[0319] (14-1) The recommendation is made by outputting a voice message, an alarm sound, or music from an auditory information output means of the voice processing device. The audio processing device according to (14) above.

[0320] (14-2) performing the recommendation using an output means of an external device connected to the voice processing device; The audio processing device according to (14) above.

[0321] (14-2-1) The recommendation is made by outputting a voice message, an alarm sound, or music from the auditory output means of the external device. The audio processing device according to (14-2) above.

[0322] (14-2-2) The recommendation is made by outputting a linguistic message, a symbol, or a pictogram from a visual output means of the external device. The audio processing device according to (14-2) above.

[0323] (14-2-3) outputting vibrations from the haptic output means of the external device to make the recommendation; The audio processing device according to (14-2) above.

[0324] (14-3) Acquiring a response from the user based on input to a physical button, touch sensor, acceleration sensor, or microphone of the hearing aid device. The audio processing device according to (14) above.

[0325] (14-4) acquiring a response from the user based on an input to a physical button, a user interface, an acceleration sensor, or a microphone of an external device connected to the voice processing device; The audio processing device according to (14) above.

[0326] (15) The newly estimated second control information is displayed as a recommended mark, a recommended value, or a recommended range on a screen of an external device connected to the audio processing device, together with a setting means for a user to set control information. The audio processing device according to (14) above.

[0327] (15-1) The setting means is composed of a slider bar, a dial, or a radio button. The audio processing device according to (15) above.

[0328] (16) When the noise canceling effect based on the calculated required noise canceling strength is insufficient, the system notifies the user of the sound processing device that the noise canceling effect is insufficient and suggests changing the sound environment, and determines whether to suspend or cancel the hearing test based on the response from the user. The audio processing device according to (13) above.

[0329] (16-1) After suspending the hearing test based on the response from the user, the hearing test is resumed from the midpoint in the changed sound environment. The audio processing device according to (16) above.

[0330] (17) The audio processing device is any one of an earphone, a headphone, a hearing aid, a sound collector, and a head-mounted display. The audio processing device according to any one of (1) to (16) above.

[0331] (18) an information estimation step of estimating control information based on hearing threshold information relating to hearing thresholds for each frequency and input voice information; a signal processing step of processing an input audio signal based on the estimated control information; 1. A method for processing audio comprising:

[0332] (19) A hearing aid device with a hearing measurement function, When conducting a hearing test, ambient sounds are collected from the hearing aid device; amplifying the collected ambient sound; outputting the amplified ambient sound from a speaker of the hearing aid device or an external device connected to the hearing aid device; Hearing aid devices.

[0333] (19-1) Pick up ambient sounds when the test sound for hearing measurement is stopped. The hearing aid device according to (19) above.

[0334] (20) It also has a noise canceling function. Record information indicating that noise cancellation was used during the hearing test along with the hearing test results; The hearing aid device according to (19) above.

[0335] (20-1) Display the results of the hearing test and whether noise canceling is enabled or disabled. The hearing aid device according to (20) above. [Explanation of symbols]

[0336] 101...Hearing aid device, 102...Microphone, 103...Speaker 104...ADC, 105...DAC, 106...signal processing unit 601...Hearing aid device, 602...Microphone, 603...Speaker 604...ADC, 605...DAC, 606...signal processing unit 1301...Hearing aid device, 1302...Microphone, 1303...Speaker 1304...ADC, 1305...DAC, 1306...signal processing unit 1312...sound source storage and playback unit, 1313...external interface 1401...Hearing aid device, 1402...Microphone, 1403...Speaker 1404…ADC, 1405…DAC, 1407…NR section 1409...amplitude / phase adjustment unit, 1411...control unit 1412...sound source storage and playback unit, 1413...external interface 1414: Configuration information storage unit, 1415: Hearing information storage unit 1416...delay amount determination unit, 1417...trained model storage unit 1501...Trained model 1601...Hearing aids, 1664...Portable information devices 1764...Portable information devices 1940…Trained Model A, 1941…Trained Model B 1942…Trained Model C 2040...First trained model, 2041...Second trained model 2122...First sound pressure level calculation unit 2123... second sound pressure level calculation unit, 2124... sound pressure level difference calculation unit 2125: Allowable delay amount calculation unit, 2126: Allowable delay amount estimation unit 2416: Delay amount determination unit, 2427: Delay amount calculation unit 2440...Trained model 2616: Delay amount determining unit; 2626: Allowable delay amount estimating unit 2627...Delay amount calculation unit 2901...Hearing aids, 2964...Portable information devices 3001: Hearing aids, 3064: Portable information devices 3307...NR section, 3351...input buffer, 3352...output buffer 3353...Window hanger, 3354...IFFTM, 3355...FFT 3356...IFFT, 3357...NR_core 3501...Hearing aid device, 3502...Microphone, 3503...Speaker 3504…ADC, 3505…DAC, 3507…NR section 3509...amplitude / phase adjustment unit, 3511...control unit 3512...sound source memory playback unit, 3513...external interface 3514...Configuration information storage unit, 3515...Hearing information storage unit 3516: Delay amount determination unit, 3517: Learned model storage unit 3518...Connection information storage unit, 3519...Sensor information unit 3640...Trained model 3797...Camera 3840...First trained model, 3841...Second trained model 3916: Delay amount determination unit, 3927: Delay amount calculation unit 3940...Trained model 4201...Hearing aid devices, 4202...Microphones, 4203...Speakers 4204...ADC, 4205...DAC, 4206...Signal processing unit 4212...sound source memory playback unit, 4213...external interface 4229...Microphone 5001...Hearing aid device, 5002...Microphone, 5003...Speaker 5004…ADC, 5005…DAC, 5007…NR section 5008...NC signal generation unit, 5009...Amplitude / phase adjustment unit 5011...control unit, 5012...sound source storage and playback unit 5013: external interface; 5014: configuration information storage unit 5015...Hearing memory unit, 5017...Trained model memory unit 5019: Sensor information unit, 5030: NC characteristic determination unit 5140...Trained model 5264...Mobile information devices 5422...First sound pressure level calculation unit 5423... second sound pressure level calculation unit, 5424... sound pressure level difference calculation unit 5431...NC characteristic estimation section 5432...NC strength and NC characteristic curve calculation section 5730: NC characteristic determination unit, 5740: trained model 5830...NC characteristic determination section, 5831...NC characteristic estimation section 6260...measuring device, 6261...handset, 6262...answering push button 6263...Soundproof room 6501...Hearing aids, 6560...Measuring devices 6701...Hearing aids, 6764...Measuring devices 7001...Hearing aid device, 7002...Microphone, 7003...Speaker 7004…ADC, 7005…DAC, 7007…NR section 7008: NC signal generation unit, 7009: Amplitude / phase adjustment unit 7011...control unit, 7012...sound source storage and playback unit 7013: external interface; 7014: configuration information storage unit 7015...Hearing memory unit, 7017...Trained model memory unit 7019: Sensor information section, 7030: NC characteristic determination section 7140...Trained model 7201A, 7201B...Hearing aid devices, 7264...Portable information devices 7364...Mobile information devices 7528...first sound pressure level calculation unit, 7523...Second sound pressure level calculation section 7524...Required NC strength calculation section, 7531...NC characteristic estimation section 7532...NC strength / NC characteristic curve calculation section 7830: NC characteristic determination unit, 7840: trained model 7930...NC characteristic determination section, 7931...NC characteristic estimation section 8064...Portable information devices 8301: Hearing aids, 8364: Portable information devices

Claims

1. an information estimation unit that estimates control information based on hearing threshold information relating to hearing thresholds for each frequency and input audio information; a signal processing unit that processes an input audio signal based on the estimated control information; Equipped with the information estimation unit estimates, as the control information, an allowable delay amount of processing by the signal processing unit; the signal processing unit performs noise reduction processing on the input audio signal based on the allowable delay amount. Audio processing device.

2. The hearing threshold information includes at least one of a result of a threshold measurement that measures the hearing threshold for each frequency of the user of the sound processing device, or information on a sound pressure level for each frequency of a measurement signal used in a hearing measurement function. The audio processing device according to claim 1 .

3. The input audio information is an input audio of the audio processing device that outputs an audio signal output from the signal processing unit. The audio processing device according to claim 1 .

4. the information estimation unit further estimates the control information based on configuration information of the audio processing device. The audio processing device according to claim 1 .

5. the information estimation unit further estimates noise canceling characteristic information related to the noise canceling processing performed by the signal processing unit. The audio processing device according to claim 1 .

6. The noise canceling characteristic information includes at least one of a noise canceling characteristic curve or a noise canceling intensity. The audio processing device according to claim 5 .

7. the information estimation unit estimates the control information using a trained model trained in advance with at least the user's hearing ability information and the input voice information as explanatory variables and the control information as a target variable; The audio processing device according to claim 1 .

8. the signal processing unit processes the input audio signal of the audio processing device; the information estimation unit estimates the control information using the trained model trained using configuration information of the voice processing device as a condition. The audio processing device according to claim 7 .

9. The trained model is composed of a first trained model and a second trained model, The output of the second trained model is used as an input of the first trained model. The audio processing device according to claim 7 .

10. The information estimation unit calculating a first sound pressure level from gain information calculated by a prescription formula from the measurement results and a band-specific sound pressure level of the input sound to the sound processing device; calculating a second sound pressure level from the band-specific sound pressure levels and acoustic characteristics based on configuration information of the sound processing device; determining, as the control information, an allowable delay amount in the signal processing unit or noise canceling characteristic information related to a noise canceling process performed in the signal processing unit, based on a difference between the first sound pressure level and the second sound pressure level; The audio processing device according to claim 2 .

11. The information estimation unit calculating a first sound pressure level from frequency information and sound pressure level information of the measurement signal and a band-specific sound pressure level of the input sound to the sound processing device; calculating a second sound pressure level from the band-specific sound pressure levels and acoustic characteristics based on configuration information of the sound processing device; calculating a noise canceling strength required for the hearing test from the difference between the first sound pressure level and the second sound pressure level; determining, based on the required noise canceling strength, noise canceling characteristics relating to the noise canceling process performed in the signal processing unit as the control information; The audio processing device according to claim 2 .

12. the configuration information includes at least one of a type of earpiece of the audio processing device, a wearing method, and characteristic information of a housing of the audio processing device; The audio processing device according to claim 4 .

13. When the first control information currently used in the audio processing device is different from the second control information newly estimated by the information estimation unit, a process for changing the control information used in the audio processing device is performed. The audio processing device according to claim 1 .

14. recommending a change from the first control information to the second control information to a user of the voice processing device, and changing the control information based on a response from the user; The audio processing device according to claim 13.

15. a setting means for allowing a user to set control information, and the newly estimated second control information are displayed as a recommended mark, a recommended value, or a recommended range on a screen of an external device connected to the audio processing device; The audio processing device according to claim 14.

16. When the noise canceling effect based on the calculated required noise canceling strength is not sufficient, the device notifies the user of the sound processing device that the noise canceling effect is not sufficient and suggests changing the sound environment, and determines whether to suspend or stop the hearing test based on a response from the user. The audio processing device according to claim 13.

17. The audio processing device is any one of an earphone, a headphone, a hearing aid, a sound collector, and a head-mounted display. The audio processing device according to claim 1 .

18. an information estimation step of estimating control information based on hearing threshold information relating to hearing thresholds for each frequency and input audio information; a signal processing step of processing an input audio signal based on the estimated control information; and In the information estimation step, an allowable delay amount of processing in the signal processing step is estimated as the control information; In the signal processing step, noise reduction processing is performed on the input audio signal based on the allowable delay amount. Audio processing methods.

Images