Adaptive acoustic feedback cancellation system and method

Abstract

An adaptive acoustic feedback cancellation system according to a first embodiment of the present invention comprises: a variable initialization unit for initializing a plurality of variables involved in adaptive acoustic feedback cancellation; a path input calculation unit for calculating a forward path input (e(k)) while updating moment vectors for each sample or every several samples while going through a loop for adaptive acoustic feedback cancellation on the basis of initialized variables; a learning rate updating unit for updating, with respective specific values, a plurality of learning rates preset for every several steps according to the calculated forward path input (e(k)); a path coefficient updating unit for updating predicted feedback path coefficients according to a delay time on the basis of the learning rates; and a learning rate reduction unit for reducing the learning rates at a constant rate, for every several steps, or whenever a certain condition is satisfied in order to reduce, over time, the learning rates used in the path coefficient updating unit.

Description

Adaptive acoustic feedback cancellation system and method

[0001] The present invention relates to an adaptive acoustic feedback cancellation system and method that can be applied to a product in which a speaker and a microphone are used together (e.g., a product in any field that eliminates acoustic feedback), and more particularly, to an adaptive acoustic feedback cancellation system and method that can eliminate sound distortion and howling that occur when sound entering the microphone of a product is reproduced through the speaker of the product and the reproduced sound is re-entered into the microphone.

[0002] Acoustic feedback issues are common in audio systems where the speakers and microphones are located in close proximity. For example, acoustic feedback issues can arise in a variety of situations, including hearing aids with integrated speakers and microphones, close proximity of microphones and speakers in lecture halls or karaoke rooms, and two or more participants in a video conference from close distances using their own laptops.

[0003] Hearing aids are devices designed to help people with hearing loss hear external sounds clearly. Digital hearing aids consist of an input unit containing a microphone, an amplifier unit that amplifies the input sound, and an output unit that outputs the amplified sound. The amplifier unit amplifies the signal by frequency band, tailored to the user's hearing loss, using an amplifier within a DSP IC chip. Users can also adjust the digital amplifier's operating mode using a volume control to further adjust the level of amplification depending on their environment. However, if a hearing aid completely blocks external sounds and transmits only the amplified sound to the eardrum, the user will feel a sense of confinement and hear their own voice echoing, significantly reducing the hearing aid's comfort. To mitigate this, hearing aids are designed with an air passage connecting the inner ear to the outside world. However, this air passage can cause acoustic feedback when the receiver's output is fed into the microphone. Increasing the diameter of the air passage reduces the sense of confinement, improving comfort, but also exacerbates the acoustic feedback problem. This acoustic feedback not only reduces the output sound quality, but also limits the maximum amplification level of the hearing aid to prevent howling caused by acoustic feedback.

[0004] As mentioned above, acoustic feedback in hearing aids is a major factor that interferes with the amplification performance and user listening experience. To address this issue, adaptive feedback cancellation (AFC) technology is widely used. This technology detects and eliminates unwanted acoustic feedback from hearing aids in real time, providing users with a more natural listening experience.

[0005] The present invention has been created by comprehensively considering the above-mentioned matters, and its purpose is to provide an adaptive acoustic feedback cancellation system and method that can effectively eliminate sound distortion and howling that occur when sound entering a product's microphone is reproduced through the product's speaker and the reproduced sound is re-entered into the microphone.

[0006] In order to achieve the above purpose, the adaptive acoustic feedback cancellation system according to the first embodiment of the present invention,

[0007] A system that removes acoustic feedback by applying optimization methods used in deep learning.

[0008] A variable initialization section that initializes a number of variables involved in adaptive acoustic feedback cancellation;

[0009] Based on the variables initialized by the above variable initialization section, the loop for adaptive acoustic feedback removal is performed while updating the moment vectors at every sample or a certain number of samples, and the forward path input ( ) Path input calculation unit for calculating;

[0010] Forward path input calculated by the above path input calculation unit ( ) according to a predetermined step, each of which updates a plurality of preset learning rates to a specific value;

[0011] A path coefficient update unit that updates the predicted feedback path coefficients according to the delay time based on the learning rate; and

[0012] It is characterized by including a learning rate reduction unit that reduces the learning rate at a constant rate, at certain steps, or whenever a certain condition is satisfied, in order to reduce the learning rate used in the path coefficient update unit as time passes.

[0013] Here, the above variables are of length The predicted feedback path vector ( ), length The first moment vector ( ), length The second-order primitive moment vector ( ) may be included, and the above , , can all be initialized to 0.

[0014] In addition, the above path input calculation unit inputs the forward path ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. can be calculated.

[0015] In addition, the learning rate update unit updates each of the plurality of learning rates set in advance to a specific value at each stage. Learning rate at each step class Each of them class can be renewed.

[0016] In addition, when the above path coefficient update unit updates the predicted feedback path coefficient, first, each delay time The gradient (g) of the gradient descent method can be calculated using the following mathematical relationship.

[0017]

[0018] Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.

[0019] At this time, the above slope (g) can also be calculated by the following mathematical relationship without soft clipping.

[0020]

[0021] At this time, the path coefficient update unit may also calculate the slope (g), obtain the first moment estimate, calculate the second raw moment estimate, calculate the bias-corrected first moment estimate and the second raw moment estimate, respectively, and update the feedback path coefficient using the calculated bias-corrected first moment estimate and the second raw moment estimate.

[0022] In addition, in order to achieve the above purpose, the adaptive acoustic feedback removal method according to the first embodiment of the present invention,

[0023] A method for removing acoustic feedback by applying optimization methods used in deep learning.

[0024] a) A step of initializing a number of variables involved in adaptive acoustic feedback removal, wherein the variable initialization section;

[0025] b) The forward path input calculation unit loops for adaptive acoustic feedback removal, updating the moment vectors at every sample or a certain number of samples. ) to calculate;

[0026] c) A step in which a learning rate update unit updates multiple learning rates set in advance to specific values ​​at each regular step;

[0027] d) a step of updating the predicted feedback path coefficient according to the delay time by the path coefficient update unit; and

[0028] e) The learning rate reduction unit is characterized by including a step of reducing the learning rate at a constant rate, at a constant step rate, or whenever a certain condition is satisfied, in order to reduce the learning rate used in the path coefficient update unit over time.

[0029] Here, in the above step a), the variables have lengths The predicted feedback path vector ( ), length The first moment vector ( ), length The second-order primitive moment vector ( ) may be included, and the above , , can all be initialized to 0.

[0030] In addition, in the above step b), the forward path input ( ) is calculated, the feedback signal is predicted using the feedback path coefficients and speaker input predicted in the previous step, and then subtracted from the microphone input. can be calculated.

[0031] In addition, in the step b), the path input calculation unit inputs the forward path ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. can be calculated.

[0032] In addition, in the above step c), the learning rate update unit updates each of the plurality of learning rates set in advance to a specific value at each predetermined step. Learning rate at each step class Each of them class can be renewed.

[0033] In addition, in the step d), when the path coefficient update unit updates the predicted feedback path coefficient, first, each delay time The gradient (g) of the gradient descent method can be calculated using the following mathematical relationship.

[0034]

[0035] Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.

[0036] At this time, the above slope (g) can also be calculated by the following mathematical relationship without soft clipping.

[0037]

[0038] At this time, the path coefficient update unit may also calculate the slope (g), obtain the first moment estimate, calculate the second raw moment estimate, calculate the bias-corrected first moment estimate and the second raw moment estimate, respectively, and update the feedback path coefficient using the calculated bias-corrected first moment estimate and the second raw moment estimate.

[0039] In addition, in order to achieve the above purpose, the adaptive acoustic feedback cancellation system according to the second embodiment of the present invention,

[0040] This system is a modified version of the Adam optimization method, one of the deep learning-based optimization methods.

[0041] A variable initialization section that initializes a number of variables involved in adaptive acoustic feedback cancellation;

[0042] Based on the variables initialized by the above variable initialization section, a loop for adaptive acoustic feedback removal is performed, updating the moment vector at every sample or a certain number of samples, and inputting the forward path ( ) Path input calculation unit for calculating;

[0043] Forward path input calculated by the above path input calculation unit ( ) according to a predetermined step, each of which updates a plurality of preset learning rates to a specific value;

[0044] A moment estimation value calculation unit that calculates a moment estimation value as a second-order equation of the gradient based on the learning rate updated to the above-mentioned specific value;

[0045] A path coefficient update unit that updates the predicted feedback path coefficient using the moment estimate calculated by the moment estimate calculation unit and variables that affect the learning rate; and

[0046] It is characterized by including a learning rate reduction unit that reduces the learning rate at a constant rate, at certain steps, or whenever a certain condition is satisfied, in order to reduce the learning rate used in the path coefficient update unit as time passes.

[0047] Here, the above variables are of length The predicted feedback path vector ( ), length In moment vector( ) may be included, and the above , can all be initialized to 0.

[0048] In addition, the above path input calculation unit inputs the forward path ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. can be calculated.

[0049] In addition, the learning rate update unit updates each of the plurality of learning rates set in advance to a specific value at each stage. Learning rate at each step class Each of them class can be renewed.

[0050] In addition, when the moment estimation value calculation unit calculates the moment estimation value as a second-order equation of the slope, the slope (g) can be calculated by the following mathematical relationship.

[0051]

[0052] Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.

[0053] At this time, the above slope (g) can also be calculated by the following mathematical relationship without soft clipping.

[0054]

[0055] In addition, when the moment estimation value calculation unit calculates the moment estimation value as a second-order equation of the slope, the moment estimation value can be calculated by the following mathematical relationship.

[0056]

[0057] In addition, when the path coefficient update unit updates the predicted feedback path coefficient using the moment estimate and the variables that affect the learning rate, the variables that affect the learning rate are , , may include.

[0058] In addition, in order to achieve the above purpose, the adaptive acoustic feedback removal method according to the second embodiment of the present invention,

[0059] This is a method that modifies and applies the Adam optimization method, one of the deep learning-based optimization methods.

[0060] m) A step of initializing a number of variables involved in adaptive acoustic feedback removal by a variable initialization unit;

[0061] n) The forward path input calculation unit loops for adaptive acoustic feedback removal, updating the moment vector at every sample or a certain number of samples. ) to calculate;

[0062] o) A step in which a learning rate update unit updates multiple learning rates set in advance to specific values ​​at each regular step;

[0063] p) A step of calculating a moment estimate value using a quadratic equation of a gradient based on a learning rate updated to a specific value by a moment estimate calculation unit;

[0064] q) a step of updating the predicted feedback path coefficients by using the calculated moment estimate and variables affecting the learning rate; and

[0065] r) The learning rate reduction section is characterized by including a step of reducing the learning rate at a constant rate, at a constant step rate, or whenever a certain condition is satisfied, in order to reduce the learning rate used in the path coefficient update section over time.

[0066] Here, in the above step m), the variables have lengths The predicted feedback path vector ( ), length In moment vector( ) may be included, and the above , can all be initialized to 0.

[0067] In addition, in the above step n), the path input calculation unit inputs the forward path ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. can be calculated.

[0068] In addition, in the above step o), when the learning rate update unit updates a plurality of learning rates set in advance at each certain step to a specific value, Learning rate at each step class Each of them class can be renewed.

[0069] In addition, in the step p), when the moment estimation value calculation unit calculates the moment estimation value as a quadratic equation of the slope, the slope (g) can be calculated by the following mathematical relationship.

[0070]

[0071] Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.

[0072] At this time, the above slope (g) can also be calculated by the following mathematical relationship without soft clipping.

[0073]

[0074] In addition, in the step p), when the moment estimation value calculation unit calculates the moment estimation value as a second-order equation of the slope, the moment estimation value can be calculated by the following mathematical relationship.

[0075]

[0076] In addition, the variables that affect the learning rate in the above step q) are , , may include.

[0077] According to the present invention, by decreasing the learning rate at a constant rate and initializing the step size corresponding to the learning rate of deep learning in the process of finding the solution of the mean square error periodically or when a specific condition is satisfied, sound distortion and howling that occur when the process of sound entering the product's microphone being played through the product's speaker and the played sound being re-entered into the microphone is repeated can be effectively eliminated.

[0078] Figure 1 is a block diagram illustrating an overview of a typical adaptive feedback cancellation system.

[0079] Figure 2 is a schematic diagram showing the structure of a hearing aid without a feedback removal system.

[0080] FIG. 3 is a schematic diagram illustrating the configuration of an adaptive acoustic feedback cancellation system according to a first embodiment of the present invention.

[0081] Figure 4 is a flowchart showing the execution process of an adaptive acoustic feedback removal method according to the first embodiment of the present invention.

[0082] Figure 5 is a diagram showing an algorithm that applies the Adam optimization method, one of the deep learning-based optimization methods, to updating feedback path coefficients.

[0083] FIG. 6 is a schematic diagram illustrating the configuration of an adaptive acoustic feedback cancellation system according to a second embodiment of the present invention.

[0084] Figure 7 is a flowchart showing the execution process of an adaptive acoustic feedback removal method according to the second embodiment of the present invention.

[0085] Figure 8 is a diagram showing an algorithm using an optimization method that modifies the Adam optimization method.

[0086] Figure 9 is a diagram comparing the performance of H-NLMS, ADAM-AFC, and BS-AFC.

[0087] Figure 10 is a diagram comparing the performance of ADAM-AFC and BS-AFC while increasing the step size.

[0088] Figure 11 is a diagram showing the results of an experiment to examine whether howling occurs in H-NLMS and BS-AFC while increasing the amplification size in the same feedback path.

[0089] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0090] Here, before a detailed description of the adaptive acoustic feedback cancellation system and method according to an embodiment of the present invention is given, the conventional technology that forms the basis of the present invention will first be described to help understand the present invention.

[0091] Figure 1 is a block diagram illustrating an overview of a typical adaptive feedback cancellation system.

[0092] Referring to Figure 1, represents discrete time. Microphone input is an external sound Wow feedback sound It is composed of the sum of . If we express this as a formula relationship, it is as follows.

[0093] Mathematical formula 1

[0094]

[0095] The basic concept of an adaptive feedback cancellation system is the feedback path Predictive feedback path of , and configure the microphone input Predictive feedback sound is to subtract.

[0096] Predictive feedback path A feedback path If it matches, the predicted feedback sound Ga feedback sound , so the signal to be used in the forward path is the same as is a voice (external sound) It matches with .

[0097] Microphone input Predictive feedback sound in The result of subtracting the forward path of the hearing aid Input of This corresponds to . If we express this as a formula relationship, it is as follows.

[0098] Mathematical formula 2

[0099]

[0100] Speaker output is the input of the forward path And the forward path of the hearing aid, i.e. the amplification factor is determined by . If amplified equally regardless of frequency, speaker output can be expressed by the following mathematical relationship.

[0101] Mathematical formula 3

[0102]

[0103] Here, is a discrete-time delay operator. It measures the delay time in the forward path. If you say so, Is It becomes.

[0104] Sound output from speaker (120) is a feedback path After a certain amount of time has passed, the signal enters the input of the microphone (110) in a reduced size state. Therefore, the point in time Feedback sound input into the microphone (110) It corresponds to the sum of sounds output from the speaker (120) at the previous point in time and reaching the microphone (110) after going through delay time and size attenuation.

[0105] Here, the feedback path Is can be expressed as . At this time, Is , which represents the impulse response of the feedback path. is a vector of dimensions, Is As such, this is is a vector of discrete-time delay operators of dimension. Therefore, the feedback sound can be expressed by the following mathematical relationship.

[0106] Mathematical formula 4

[0107]

[0108] Here, represents the attenuation ratio.

[0109] Meanwhile, if there is no feedback removal process as in Fig. 2, the transfer function of the closed loop system, which represents the ratio of output sound to external sound, can be expressed by the following mathematical relationship.

[0110] Mathematical Formula 5

[0111]

[0112] In a hearing aid, sound is amplified in the forward path, and the sound output from the speaker (120) is re-entered into the microphone (110) through the feedback path, a process that is repeated. In this process, if the two conditions of the following mathematical expression 6 are simultaneously satisfied, the hearing aid becomes unstable and howling occurs.

[0113] Mathematical formula 6

[0114]

[0115] Here, and are the frequency responses in the forward path and the feedback path, respectively, represents angular velocity.

[0116] Acoustic feedback not only degrades sound quality, but also limits the maximum amplification of the hearing aid to prevent howling due to acoustic feedback. Therefore, acoustic feedback cancellation is a crucial function of hearing aids. Therefore, to maintain sound quality and suppress howling, hearing aids must incorporate technologies that circumvent one of the conditions in Equation 6.

[0117] As shown in the above-described Fig. 1, the transfer function of a closed loop system with added feedback cancellation can be defined as in the following mathematical expression 7.

[0118] Mathematical formula 7

[0119]

[0120] The goal of a feedback elimination system is to predict the feedback path well. Become It is to make it happen.

[0121] If we substitute the above mathematical expression 1 into mathematical expression 2, the input of the forward path can be expressed as the following mathematical expression 8.

[0122] Mathematical formula 8

[0123]

[0124] Here, am.

[0125] Predictive feedback filter coefficient vector To obtain the mean square error The vector that minimizes is obtained using the following mathematical relationship.

[0126] Mathematical formula 9

[0127]

[0128] The LMS (Least Mean Square) algorithm is a method of obtaining a feedback filter vector by updating the feedback filter coefficient for each sample of the input sound, as shown in the following mathematical expression 10.

[0129] Mathematical formula 10

[0130]

[0131] Here, As recently is a vector consisting of samples of dogs is the step size.

[0132] The LMS method is simple, but it has the problem that it takes a long time to converge when the step size is small, and it easily diverges when the input signal is large when the step size is large.

[0133] The Normalized Least Mean Square (NLMS) algorithm attempts to address the shortcomings of the LMS algorithm by normalizing the step size. This can be expressed mathematically as follows:

[0134] Mathematical formula 11

[0135]

[0136] Here, is a small positive number to prevent division by zero.

[0137] The PEM (Prediction Error Method) algorithm uses a pre-whitened adaptive filter to reduce the bias caused by the correlation between the microphone output and the speaker input. This method Assume that is modeled as an autoregressive process of white Gaussian noise. That is, assume the following mathematical expression (12).

[0138] Mathematical formula 12

[0139]

[0140] Here, n(k) is white Gaussian noise, is a monic and stable polynomial transfer function. is predicted by the Levinson-Durbin method. Pre-whitened forward pass input and pre-amplified speaker output Is The predicted value of It is calculated by . If the above series of contents are expressed in a formula relationship, it is as follows: Mathematical formulas 13, 14, and 15.

[0141] Mathematical formula 13

[0142]

[0143] Mathematical formula 14

[0144]

[0145] Mathematical formula 15

[0146]

[0147] Here, the predicted feedback path coefficient can be obtained using the following mathematical expression 16.

[0148] Mathematical formula 16

[0149]

[0150] PEM has a smaller error rate than NLMS, but it takes longer to converge. The H-NLMS method utilizes the strengths of both PEM and NLMS. When the difference between the size of the forward input and the soft-clipping result of that input exceeds a certain threshold, NLMS is used. When the input is small, PEM is used.

[0151] As described above, adaptive feedback elimination techniques such as LMS, NLMS, PEM-AFC, PEMSC, and H-NLMS update the feedback path coefficients by applying gradient descent for each input sample to obtain the feedback path coefficients that minimize the mean square error described in Equation 9 above. Gradient descent is a method used in optimization algorithms for deep learning.

[0152] Below, an embodiment of the present invention is described based on the above-described prior art.

[0153] In the present invention, we focus on the fact that the process of updating the feedback path coefficients for each sample using gradient descent in the adaptive feedback elimination technique is similar to the process of updating the input variables for each batch in deep learning.

[0154] Deep learning trains the variables of a deep learning model using given training data. Therefore, the optimal solution for the model variables for the given training data is fixed, and the error is reduced by gradually reducing the learning rate. In contrast, in environments where adaptive feedback cancellation technology is applied, the feedback path coefficients frequently change due to actions such as head rotation after wearing a hearing aid or holding a mobile phone to the ear. To reflect these differences, the present invention resets the step size, which corresponds to the deep learning learning rate, periodically or when a specific condition is met during the process of obtaining the mean squared error solution.

[0155] FIG. 3 is a schematic diagram illustrating the configuration of an adaptive acoustic feedback cancellation system according to a first embodiment of the present invention.

[0156] Referring to FIG. 3, an adaptive acoustic feedback removal system (300) according to a first embodiment of the present invention is a system that removes acoustic feedback by applying an optimization method used in deep learning, and may be configured to include a variable initialization unit (310), a path input calculation unit (320), a learning rate update unit (330), a path coefficient update unit (340), and a learning rate decrease unit (350).

[0157] The variable initialization unit (310) initializes a number of variables involved in adaptive acoustic feedback removal. Here, the variables have a length The predicted feedback path vector ( ), length The first moment vector ( ), length The second-order primitive moment vector ( ) may be included, and the above , , can all be initialized to 0.

[0158] Also, the above variables are step size = 0.01, = 0.001, = 10 -8 , = 2, = 0.9, = 0.999, = 1, = 1024, = 64 variables can be included. Here, the values ​​of the variables are not limited to the values ​​above, and may be changed depending on the target or conditions to which the system of the present invention is applied.

[0159] The path input calculation unit (320) loops for adaptive acoustic feedback removal based on the variables initialized by the variable initialization unit (310) and updates the moment vectors for each sample or a certain number of samples while performing forward path input ( ) is calculated. The path input calculation unit (320) of this type calculates the forward path input ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. can be calculated.

[0160] The learning rate update unit (330) calculates the forward path input ( ) updates each of the plurality of preset learning rates to a specific value at each regular step. Here, when the learning rate update unit (330) updates each of the plurality of preset learning rates to a specific value at each regular step, Learning rate at each step class Each of them class can be updated with the learning rate as follows. class Each of them class It is not limited to updating, and can be updated to different values ​​depending on certain conditions or situations. Also, it is not limited to updating at certain stages, and can be updated when a given condition is satisfied.

[0161] The path coefficient update unit (340) updates the predicted feedback path coefficient according to the delay time based on the learning rate. Here, when the path coefficient update unit (340) updates the predicted feedback path coefficient, first, each delay time The gradient (g) of the gradient descent method can be calculated using the following mathematical relationship.

[0162]

[0163] Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.

[0164] At this time, the above slope (g) can also be calculated by the following mathematical relationship without soft clipping.

[0165]

[0166] At this time, the path coefficient update unit (340) may also calculate the slope (g), obtain a first moment estimate, calculate a second raw moment estimate, calculate a bias-corrected first moment estimate and a second raw moment estimate, and update the feedback path coefficient using the calculated bias-corrected first moment estimate and the second raw moment estimate.

[0167] The learning rate reduction unit (350) reduces the learning rate at a constant rate to reduce the learning rate used in the path coefficient update unit (340) over time.

[0168] Here, the variable initialization unit (310), path input calculation unit (320), learning rate update unit (330), path coefficient update unit (340), and learning rate reduction unit (350) as described above may be integrated into one and configured as a single integrated module.

[0169] Figure 4 is a flowchart showing the execution process of an adaptive acoustic feedback removal method according to the first embodiment of the present invention.

[0170] Referring to FIG. 4, the adaptive acoustic feedback removal method according to the first embodiment of the present invention is a method of removing acoustic feedback by applying an optimization method used in deep learning. First, the variable initialization unit (310) initializes a number of variables involved in the adaptive acoustic feedback removal (step S401). Here, the variables have a length of The predicted feedback path vector ( ), length The first moment vector ( ), length The second-order primitive moment vector ( ) may be included, and the above , , can all be initialized to 0.

[0171] Also, the above variables are step size = 0.01, = 0.001, = 10 -8 , = 2, = 0.9, = 0.999, = 1, = 1024, = 64 variables can be further included. Here, the values ​​of the variables are not limited to the values ​​described above, and may be changed depending on the target or conditions to which the method of the present invention is applied. The variables described above will be discussed again in FIG. 5.

[0172] After the initialization of variables as described above is completed, the path input calculation unit (320) loops for adaptive acoustic feedback removal and updates the moment vectors for each sample or a certain number of samples while performing forward path input ( )(see Fig. 1) is calculated (step S402). Here, the path input calculation unit (320) inputs the forward path ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. can be calculated. This will be explained later.

[0173] Afterwards, the learning rate update unit (330) updates each of the plurality of preset learning rates to a specific value at each regular step (step S403). Here, when the learning rate update unit (330) updates each of the plurality of preset learning rates to a specific value at each regular step, Learning rate at each step class Each of them class can be updated with the learning rate as follows. class Each of them class It is not limited to updating, and can be updated to different values ​​depending on certain conditions or situations. Also, it is not limited to updating at certain stages, and can be updated when a given condition is satisfied.

[0174] The path coefficient update unit (340) updates the predicted feedback path coefficient according to the delay time based on the learning rate (i.e., the learning rate calculated by the learning rate reduction unit (350)) (step S404). Here, when the path coefficient update unit (340) updates the predicted feedback path coefficient, first, each delay time The gradient (g) of the gradient descent method can be calculated using the following mathematical relationship.

[0175]

[0176] Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.

[0177] At this time, the above slope (g) can also be calculated by the following mathematical relationship without soft clipping.

[0178]

[0179] At this time, the path coefficient update unit (340) may also calculate the slope (g), obtain the first moment estimate, calculate the second raw moment estimate, calculate the bias-corrected first moment estimate and the second raw moment estimate, respectively, and update the feedback path coefficient using the calculated bias-corrected first moment estimate and the second raw moment estimate. Here, the path coefficient update is not limited to using the first moment estimate and the second raw moment estimate, and may be updated using other optimization methods used in deep learning techniques in addition to the Adam method.

[0180] After updating the predicted feedback path coefficients according to the delay time, the learning rate reduction unit (350) reduces the learning rate at a constant rate to decrease the learning rate over time (step S405). Note that the learning rate reduction is not limited to this constant rate, and the learning rate may be reduced at regular intervals or whenever a specific condition is met.

[0181] Figure 5 is a diagram showing an algorithm that applies the Adam optimization method, one of the deep learning-based optimization methods, to updating feedback path coefficients.

[0182] Referring to FIG. 5, Algorithm 1 is an algorithm (software program) that applies the Adam optimization method, one of the deep learning-based optimization methods, to feedback path coefficient updates. Here, this optimization method will be referred to as "ADAM-AFC." This "ADAM-AFC" is an algorithm employed to implement the adaptive acoustic feedback cancellation method according to the first embodiment of the present invention described in FIG. 4 above.

[0183] In line 1 and are microphone input and speaker output respectively, and in line 2 is the input of the forward path as a result of removing the predicted feedback from the microphone input.

[0184] Each variable is initialized to the value described in line 3. is the length is the predicted feedback path vector, class are each of length These are the first-order moment vector and the second-order primitive moment vector, and all of these vectors are initialized to 0.

[0185] ADAM-AFC performs forward pass input by looping and updating the moment vectors for each sample. Calculate .

[0186] To explain in more detail, the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker in line 6, and then subtracted from the microphone input. Calculate the feedback path coefficient is updated in the process from line 7 to line 20.

[0187] Schedule from line 7 to line 10 Learning rate at each step class are each class is updated.

[0188] Lines 11 to 18 are the process of updating the predicted feedback path coefficients according to the delay time.

[0189] Each delay time The gradient of the gradient descent method is calculated in line 12. is the soft clipping used in the H-NLMS method, and can also be calculated as g = step_size×e(k)×s(k-lag) without soft clipping.

[0190] In line 13, the first moment estimate is obtained, and in line 14, the second raw moment estimate is calculated. In lines 15 and 16, the bias-corrected first moment estimate and the second raw moment estimate are calculated, respectively. Using the bias-corrected first moment estimate and the second raw moment estimate, the feedback path coefficients are updated in line 17.

[0191] Afterwards, in order to make the learning rate smaller over time, the learning rate is decreased at a constant rate in lines 19 and 20.

[0192] FIG. 6 is a schematic diagram illustrating the configuration of an adaptive acoustic feedback cancellation system according to a second embodiment of the present invention.

[0193] Referring to FIG. 6, an adaptive acoustic feedback removal system (600) according to a second embodiment of the present invention is a system that applies the Adam optimization method, which is one of the deep learning-based optimization methods, by modifying it, and may be configured to include a variable initialization unit (610), a path input calculation unit (620), a learning rate update unit (630), a moment estimation value calculation unit (640), a path coefficient update unit (650), and a learning rate reduction unit (660).

[0194] The variable initialization unit (610) initializes a number of variables involved in adaptive acoustic feedback removal. Here, the variables have a length The predicted feedback path vector ( ), length In moment vector( ) may be included, and the above , can all be initialized to 0.

[0195] Also, the above variables are step size = 2.0, = 0.001, = 2, = 0.9, = 0.999, = 10, = 1024, = 64 variables can be included. Here, the values ​​of the variables are not limited to the values ​​above, and may be changed depending on the target or conditions to which the system of the present invention is applied.

[0196] The path input calculation unit (620) loops for adaptive acoustic feedback removal based on the variables initialized by the variable initialization unit (610) and updates the moment vector for each sample or a certain number of samples while performing forward path input ( ) is calculated. The path input calculation unit (620) calculates the forward path input ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. can be calculated.

[0197] The learning rate update unit (630) calculates the forward path input ( ) updates multiple preset learning rates to specific values ​​at each regular step. When the learning rate update unit (630) updates multiple preset learning rates to specific values ​​at each regular step, Learning rate at each step class Each of them class can be updated with the learning rate as follows. class Each of them class It is not limited to updating, and can be updated to different values ​​depending on certain conditions or situations. Also, it is not limited to updating at certain stages, and can be updated when a given condition is satisfied.

[0198] The moment estimate calculation unit (640) calculates the moment estimate as a quadratic equation of the slope based on the predicted learning rate. Here, when the moment estimate calculation unit (640) calculates the moment estimate as a quadratic equation of the slope, the slope (g) can be calculated by the following mathematical relationship.

[0199]

[0200] Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.

[0201] At this time, the above slope (g) can also be calculated by the following mathematical relationship without soft clipping.

[0202]

[0203] In addition, when the moment estimation value calculation unit (640) calculates the moment estimation value as a second-order equation of the slope, the moment estimation value can be calculated by the following mathematical relationship.

[0204]

[0205] Here, it is not limited to the quadratic equation of the slope (g) as described above, and various quadratic equations that are modified from the above quadratic equation may be used.

[0206] The path coefficient update unit (650) updates the predicted feedback path coefficient using the moment estimate value calculated by the moment estimate value calculation unit (640) and variables that affect the learning rate. Here, when the path coefficient update unit (650) updates the predicted feedback path coefficient using the moment estimate value and variables that affect the learning rate, the variables that affect the learning rate are , , may include.

[0207] The learning rate reduction unit (660) reduces the learning rate at a constant rate to reduce the learning rate used in the path coefficient update unit (650) over time. However, the learning rate reduction is not limited to a constant rate; the learning rate may be reduced at certain stages or whenever a certain condition is met.

[0208] Here, the variable initialization unit (610), path input calculation unit (620), learning rate update unit (630), moment estimation value calculation unit (640), path coefficient update unit (650), and learning rate reduction unit (660) as described above may be integrated into one as a whole and configured as a single integrated module, similar to the case of the system according to the first embodiment described above (see FIG. 3).

[0209] Figure 7 is a flowchart showing the execution process of an adaptive acoustic feedback removal method according to the second embodiment of the present invention.

[0210] Referring to Fig. 7, the adaptive acoustic feedback removal method according to the second embodiment of the present invention is a method that modifies and applies the Adam optimization method, which is one of the deep learning-based optimization methods. First, the variable initialization unit (610) initializes a number of variables involved in the adaptive acoustic feedback removal (step S701). Here, the variables have a length of The predicted feedback path vector ( ), length In moment vector( ) may be included, and the above , can all be initialized to 0. Also, the above variables have step size = 2.0, = 0.001, = 2, = 0.9, = 0.999, = 10, = 1024, = 64 variables can be further included. Here, the values ​​of the variables are not limited to the values ​​described above, and may be changed depending on the target or conditions to which the method of the present invention is applied. The above variables will be discussed again in Fig. 8.

[0211] As above, after the initialization of variables is completed, the path input calculation unit (620) loops for adaptive acoustic feedback removal and updates the moment vector for each sample or a certain number of samples while performing forward path input ( ) is calculated (step S702). Here, the path input calculation unit (620) calculates the forward path input ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. can be calculated. This will be explained later.

[0212] This way, the forward path input ( ) is calculated, the learning rate update unit (630) updates each of the plurality of preset learning rates to a specific value at each regular step (step S703). Here, when the learning rate update unit (630) updates each of the plurality of preset learning rates to a specific value at each regular step, Learning rate at each step class Each of them class can be updated with the learning rate as follows. class Each of them class It is not limited to updating, and can be updated to different values ​​depending on certain conditions or situations. Also, it is not limited to updating at certain stages, and can be updated when a given condition is satisfied.

[0213] The moment estimate calculation unit (640) calculates the moment estimate as a quadratic equation of the slope based on the predicted learning rate (step S704). Here, when the moment estimate calculation unit (640) calculates the moment estimate as a quadratic equation of the slope, the slope (g) can be calculated by the following mathematical relationship.

[0214]

[0215] Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.

[0216] At this time, the above slope (g) can also be calculated by the following mathematical relationship without soft clipping.

[0217]

[0218] Here, when the moment estimation value calculation unit (640) calculates the moment estimation value as a second-order equation of the slope, the moment estimation value can be calculated by the following mathematical relationship.

[0219]

[0220] Here, it is not limited to the quadratic equation of the slope (g) as described above, and various quadratic equations that are modified from the above quadratic equation may be used.

[0221] Afterwards, the path coefficient update unit (650) updates the predicted feedback path coefficient using the calculated moment estimate and variables affecting the learning rate (step S705). Here, the variables affecting the learning rate are , , may include.

[0222] After updating the predicted feedback path coefficients by the above, the learning rate reduction unit (660) reduces the learning rate at a constant rate or at each step or whenever a certain condition is satisfied in order to make the learning rate used in the path coefficient update unit (650) smaller over time (step S706).

[0223] Figure 8 is a diagram showing an algorithm using an optimization method that modifies the Adam optimization method.

[0224] Referring to Figure 8, Algorithm 2 is an algorithm (software program) that uses an optimization method that modifies the Adam optimization method. Here, this algorithm (optimization method) will be referred to as "BS-AFC." This "BS-AFC" is the algorithm employed to implement the adaptive acoustic feedback cancellation method according to the second embodiment of the present invention described in Figure 7 above.

[0225] As shown in lines 8 to 9, BS-AFC also periodically adjusts the learning rate class Each of them class is updated.

[0226] As described in line 12, the slope is delayed speaker output. and forward input signal , step size, etc. Instead It is also possible to use .

[0227] Experimental results show that, unlike ADAM-AFC, BS-AFC has a very wide step size range that prevents howling. For example, the step size can be set to values ​​between [0.001 and 100]. As the hearing aid amplification increases, a smaller step size becomes more effective.

[0228] The moment estimate is calculated as a quadratic equation of the slope, as in line 13. class The value of uses the value recommended by the ADAM optimization method, as can be seen in line 3.

[0229] In line 14, sign(x) is a function that is -1 if x<0 and 1 if x≥0. Feedback path coefficient affects the moment estimates and learning rate. It is updated using the back.

[0230] BS-AFC also decreases the learning rate at a constant rate in lines 16 and 17 to make the learning rate smaller over time.

[0231] Figure 9 is a diagram comparing the performance of H-NLMS, ADAM-AFC, and BS-AFC.

[0232] Referring to Fig. 9, this is the result of a feedback removal experiment for H-NLMS, ADAM-AFC, and BS-AFC, respectively, in a situation where the feedback path changes in the middle. The feedback path coefficient w at the starting point is w

[0010] = 0.4, w

[0050] = 0.4, and 0 otherwise, and in the middle, it changes to w[5] = 0.4, w

[0035] = 0.4, and 0 otherwise. The volume is amplified by two times (6 dB).

[0233] (a) is the input sound (original sound), and (b) is the doubling of the amplified sound when feedback is perfectly controlled. (c) is the result using the conventional H-NLMS method, (d) is the result using ADAM-AFC instead of PEM and NLMS, and (e) is the result using BS-AFC. As can be seen in the characteristic graphs of each result, the time required for stabilization is much shorter for ADAM-AFC and BS-AFC than for H-NLMS.

[0234] Figure 10 is a diagram comparing the performance of ADAM-AFC and BS-AFC while increasing the step size.

[0235] Referring to Figure 10, this compares the performance of ADAM-AFC (a) and BS-AFC (b) while increasing the step size. It can be seen that BS-AFC produces stable results over a wider range than ADAM-AFC. Furthermore, it can be seen that BS-AFC shows increasingly better results as the step size increases.

[0236] Figure 11 is a diagram showing the results of an experiment to examine whether howling occurs in H-NLMS and BS-AFC while increasing the amplification size in the same feedback path.

[0237] Referring to Fig. 11, it can be seen that the time required for stabilization is shorter for BS-AFC(b) than for H-NLMS(a). In addition, it can be seen that the time required for stabilization in H-NLMS(a) increases as the amplification increases. In addition, in H-NLMS(a), howling occurred intermittently at 20 dB of amplification, and from 35 dB of amplification, the howling became too loud to be heard. On the other hand, in BS-AFC(b), howling did not occur or the sound echo phenomenon was not observed even when the amplification amount was increased up to 40 dB. Therefore, it can be seen from the experimental results that the performance of BS-AFC(b) is significantly better than that of H-NLMS(a).

[0238] As described above, the adaptive acoustic feedback removal system and method according to the present invention can effectively remove sound distortion and howling that occur when sound entering the product's microphone is played back through the product's speaker and the played sound is re-entered into the microphone by reducing the learning rate at a constant rate and periodically or when a specific condition is satisfied by initializing the step size corresponding to the learning rate of deep learning in the process of obtaining the solution of the mean square error.

Claims

1. A system that removes acoustic feedback by applying the optimization method used in deep learning. A variable initialization section that initializes a number of variables involved in adaptive acoustic feedback cancellation; Based on the variables initialized by the above variable initialization section, the loop for adaptive acoustic feedback removal is performed while updating the moment vectors at every sample or a certain number of samples, and the forward path input ( ) Path input calculation unit for calculating; Forward path input calculated by the above path input calculation unit ( ) according to a predetermined step, each of which updates a plurality of preset learning rates to a specific value; A path coefficient update unit that updates the predicted feedback path coefficients according to the delay time based on the learning rate; and An adaptive acoustic feedback removal system including a learning rate reduction unit that reduces the learning rate at a constant rate, at certain steps, or whenever a certain condition is satisfied, to reduce the learning rate used in the path coefficient update unit over time.

2. In paragraph 1, The above variables are of length The predicted feedback path vector ( ), length The first moment vector ( ), length The second-order primitive moment vector ( ) and the above , , An adaptive acoustic feedback cancellation system characterized in that all inputs are initialized to 0.

3. In paragraph 1, The above path input calculation unit inputs the forward path ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. An adaptive acoustic feedback cancellation system characterized by calculating .

4. In paragraph 1, In the above learning rate update unit, each of the plurality of learning rates set in advance at a certain stage is updated to a specific value, Learning rate at each step class Each of them class An adaptive acoustic feedback cancellation system characterized by renewing.

5. In paragraph 1, When the above path coefficient update unit updates the predicted feedback path coefficient, first, each delay time The gradient (g) of the gradient descent method is calculated by the following mathematical relationship: (Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.) An adaptive acoustic feedback cancellation system characterized by:

6. In paragraph 5, The above slope (g) is calculated by the following equation relationship without soft clipping. An adaptive acoustic feedback cancellation system characterized by:

7. In paragraph 5, An adaptive acoustic feedback removal system characterized in that, after the path coefficient update unit calculates the slope (g), it obtains a first moment estimate, calculates a second raw moment estimate, calculates a bias-corrected first moment estimate and a second raw moment estimate, respectively, and updates a feedback path coefficient using the calculated bias-corrected first moment estimate and the second raw moment estimate.

8. A method for removing acoustic feedback by applying the optimization method used in deep learning. a) A step of initializing a number of variables involved in adaptive acoustic feedback removal, wherein the variable initialization section; b) The forward path input calculation unit loops for adaptive acoustic feedback removal, updating the moment vectors at every sample or a certain number of samples. ) to calculate; c) A step in which a learning rate update unit updates multiple learning rates set in advance to specific values ​​at each regular step; d) a step of updating the predicted feedback path coefficient according to the delay time by the path coefficient update unit; and e) An adaptive acoustic feedback removal method including a step of reducing the learning rate at a constant rate or at each step or whenever a certain condition is satisfied so as to make the learning rate used in the path coefficient update unit smaller over time.

9. In paragraph 8, In the above step a), the variables have lengths The predicted feedback path vector ( ), length The first moment vector ( ), length The second-order primitive moment vector ( ) and the above , , An adaptive acoustic feedback cancellation method characterized in that all are initialized to 0.

10. In paragraph 8, In the above step b), the path input calculation unit inputs the forward path ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. An adaptive acoustic feedback cancellation method characterized by calculating .

11. In paragraph 8, In the above step c), the learning rate update unit updates each of the plurality of learning rates set in advance to a specific value at each predetermined step, Learning rate at each step class Each of them class An adaptive acoustic feedback removal method characterized by updating.

12. In paragraph 8, In the above step d), when the path coefficient update unit updates the predicted feedback path coefficient, first, each delay time The gradient (g) of the gradient descent method is calculated by the following mathematical relationship: (Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.) An adaptive acoustic feedback cancellation method characterized by:

13. In paragraph 12, The above slope (g) is calculated by the following equation relationship without soft clipping. An adaptive acoustic feedback cancellation method characterized by:

14. In paragraph 12, An adaptive acoustic feedback removal method characterized in that, after the path coefficient update unit calculates the slope (g), it obtains a first moment estimate, calculates a second raw moment estimate, calculates a bias-corrected first moment estimate and a second raw moment estimate, respectively, and updates a feedback path coefficient using the calculated bias-corrected first moment estimate and the second raw moment estimate.

15. A system that applies the Adam optimization method, one of the deep learning-based optimization methods, by modifying it. A variable initialization section that initializes a number of variables involved in adaptive acoustic feedback cancellation; Based on the variables initialized by the above variable initialization section, a loop for adaptive acoustic feedback removal is performed, updating the moment vector at every sample or a certain number of samples, and inputting the forward path ( ) Path input calculation unit for calculating; Forward path input calculated by the above path input calculation unit ( ) according to a predetermined step, each of which updates a plurality of preset learning rates to a specific value; A moment estimate calculation unit that calculates the moment estimate as a quadratic equation of the gradient based on the predicted learning rate; A path coefficient update unit that updates the predicted feedback path coefficient using the moment estimate calculated by the moment estimate calculation unit and variables that affect the learning rate; and An adaptive acoustic feedback removal system including a learning rate reduction unit that reduces the learning rate at a constant rate, at certain steps, or whenever a certain condition is satisfied, to reduce the learning rate used in the path coefficient update unit over time.

16. In paragraph 15, The above variables are of length The predicted feedback path vector ( ), length In moment vector( ) and the above , An adaptive acoustic feedback cancellation system characterized in that all inputs are initialized to 0.

17. In paragraph 15, The above path input calculation unit inputs the forward path ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. An adaptive acoustic feedback cancellation system characterized by calculating .

18. In paragraph 15, In the above learning rate update unit, each of the plurality of learning rates set in advance at a certain stage is updated to a specific value, Learning rate at each step class Each of them class An adaptive acoustic feedback cancellation system characterized by renewing.

19. In paragraph 15, When the above moment estimation value calculation unit calculates the moment estimation value as a quadratic equation of the slope, the slope (g) is calculated by the following mathematical relationship. (Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.) An adaptive acoustic feedback cancellation system characterized by:

20. In paragraph 19, The above slope (g) is calculated by the following equation relationship without soft clipping. An adaptive acoustic feedback cancellation system characterized by:

21. In paragraph 19, When the above moment estimation value calculation unit calculates the moment estimation value as a second-order equation of the slope, the moment estimation value is calculated by the following mathematical relationship. An adaptive acoustic feedback cancellation system characterized by:

22. In paragraph 15, When the above path coefficient update unit updates the predicted feedback path coefficient using the moment estimate value and variables affecting the learning rate, the variables affecting the learning rate are , , An adaptive acoustic feedback cancellation system comprising:

23. A method that modifies and applies the Adam optimization method, one of the deep learning-based optimization methods. m) A step of initializing a number of variables involved in adaptive acoustic feedback removal by a variable initialization unit; n) The forward path input calculation unit loops for adaptive acoustic feedback removal, updating the moment vector at every sample or a certain number of samples. ) to calculate; o) A step in which a learning rate update unit updates multiple learning rates set in advance to specific values ​​at each regular step; p) A step of calculating a moment estimate value using a second-order gradient equation based on a predicted learning rate by a moment estimate calculation unit; q) a step of updating the predicted feedback path coefficients by using the calculated moment estimate and variables affecting the learning rate; and r) An adaptive acoustic feedback removal method including a step of reducing the learning rate at a constant rate or at each step or whenever a certain condition is satisfied so as to make the learning rate used in the path coefficient update unit smaller over time.

24. In paragraph 23, In the above step m), the variables have lengths The predicted feedback path vector ( ), length In moment vector( ) and the above , An adaptive acoustic feedback cancellation method characterized in that all are initialized to 0.

25. In paragraph 23, In the above step n), the path input calculation unit inputs the forward path ( ), the feedback signal is predicted using the predicted feedback path coefficients and the signal transmitted to the speaker, and then subtracted from the microphone input. An adaptive acoustic feedback cancellation method characterized by calculating .

26. In paragraph 23, In the above step o), the learning rate update unit updates each of the plurality of learning rates set in advance to a specific value at each predetermined step, Learning rate at each step class Each of them class An adaptive acoustic feedback removal method characterized by updating.

27. In paragraph 23, In the above step p), when the moment estimation value calculation unit calculates the moment estimation value as a quadratic equation of the slope, the slope (g) is calculated by the following equation relationship. (Here, represents soft clipping used in the H-NLMS adaptive feedback cancellation technique.) An adaptive acoustic feedback cancellation method characterized by:

28. In paragraph 27, The above slope (g) is calculated by the following equation relationship without soft clipping. An adaptive acoustic feedback cancellation method characterized by:

29. In paragraph 27, In the above step p), when the moment estimation value calculation unit calculates the moment estimation value as a quadratic equation of the slope, the moment estimation value is calculated by the following mathematical relationship An adaptive acoustic feedback cancellation method characterized by:

30. In paragraph 23, In the above step q), the variables that affect the learning rate are , , An adaptive acoustic feedback cancellation method comprising:

Images