Howling detection method and device, equipment and medium

Abstract

The invention provides a howling detection method, apparatus and device, and a medium. The method comprises the steps of obtaining a to-be-detected audio; determining candidate howling frequency points in the audio based on the amplitude change information of the audio at each frequency point; and determining a howling detection result of the audio based on the amplitude distribution information of the candidate howling frequency point in the audio and / or a relationship between the amplitude value of the candidate howling frequency point in the audio and the human ear perception amplitude range. The method, the device, the equipment and the medium provided by the invention can adapt to a complex or changing environment through dynamic analysis of amplitude characteristics, and do not need to frequently debug a threshold value. Meanwhile, the method does not depend on a large amount of annotated data, realizes howling detection through feature analysis, and reduces data dependence. According to the method, the howling signals and the non-howling signals can be accurately distinguished, false detection and missing detection are reduced, and the method has relatively high detection precision and environmental adaptability.

Description

Technical Field

[0001] The present invention relates to the field of audio processing technology, and in particular to a howling detection method, device, equipment and medium. Background Art

[0002] In audio equipment, howling is an important factor affecting sound quality and equipment stability. Howling is usually caused by excessive gain in the acoustic feedback path or amplification of a specific frequency of the audio signal. It is characterized by a sharp sound of a single frequency, which may cause equipment failure in severe cases.

[0003] Traditional howling detection methods usually rely on setting multiple thresholds to determine the howling frequency points. However, the setting of these thresholds requires a lot of verification and debugging, and is prone to failure in complex or changing environments. Different environments and device configurations may require different thresholds to be adjusted. Deep learning methods can better cope with complex scenarios and improve detection accuracy through neural network models, but they require a large amount of labeled training data. The data collection and labeling process is both labor-intensive and material-intensive, and is limited by data scenarios and environments, resulting in insufficient generalization capabilities of the model.

[0004] Therefore, there is an urgent need for a howling detection method that reduces resource consumption, avoids complex verification and adapts to changing environments. Summary of the invention

[0005] The present invention provides a howling detection method, device, equipment and medium, which are used to solve the defects of high misjudgment rate and insufficient generalization ability in related technologies.

[0006] The present invention provides a howling detection method, comprising: Get the audio to be detected; Determining candidate howling frequency points in the audio based on amplitude change information of the audio at each frequency point; A howling detection result of the audio is determined based on the amplitude distribution information of the candidate howling frequency points in the audio and / or the relationship between the amplitude values ​​of the candidate howling frequency points in the audio and the amplitude range perceived by human ears.

[0007] According to a howling detection method provided by the present invention, determining candidate howling frequency points in the audio based on amplitude change information of the audio at each frequency point includes: Based on the amplitude change trend and amplitude change acceleration of the audio at each frequency point, candidate howling frequency points and types of the candidate howling frequency points are determined from the frequency points.

[0008] According to a howling detection method provided by the present invention, when the type of the candidate howling frequency point is attenuation-type howling, the amplitude distribution information of the candidate howling frequency point in the audio includes spectrum sparsity; The determining of the howling detection result of the audio based on the amplitude distribution information of the candidate howling frequency point in the audio and / or the relationship between the amplitude value of the candidate howling frequency point in the audio and the amplitude range perceived by the human ear includes: When the frequency sparsity of the candidate howling frequency point is greater than the sparsity threshold and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range, it is determined that the howling detection result includes attenuation-type howling.

[0009] According to a howling detection method provided by the present invention, the amplitude distribution information of the candidate howling frequency point in the audio further includes amplitude dispersion; The step of determining that the howling detection result includes attenuation-type howling when the frequency sparsity of the candidate howling frequency point is greater than a sparsity threshold and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range comprises: When the frequency sparsity of the candidate howling frequency point is greater than the sparsity threshold, the amplitude dispersion of the candidate howling frequency point is less than the dispersion threshold, and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range, it is determined that the howling detection result includes attenuation-type howling.

[0010] According to a howling detection method provided by the present invention, when the type of the candidate howling frequency point is a growing howling, determining the howling detection result of the audio based on the amplitude distribution information of the candidate howling frequency point in the audio, and / or the relationship between the amplitude value of the candidate howling frequency point in the audio and the amplitude range perceived by the human ear, includes: When the amplitude value of the candidate howling frequency point in the audio is within the amplitude range perceived by the human ear, it is determined that the howling detection result includes a growing howl.

[0011] According to a howling detection method provided by the present invention, when the audio is collected based on a sound reinforcement system, the method further includes: Determining a critical gain of the sound reinforcement system based on a howling detection result of the audio and a current gain of the sound reinforcement system; When the current gain is greater than the critical gain, howling control is performed on the sound reinforcement system.

[0012] The present invention provides a training method for a howling suppression model, comprising: Synthesize candidate audio based on near-end speech and noise audio; Based on the howling detection method, performing howling detection on the candidate audio to obtain a howling detection result of the candidate audio; Based on the howling detection result of the candidate audio, filter out sample audio from the candidate audio; A howling suppression model is trained based on the sample audio and the near-end speech used to synthesize the sample audio.

[0013] According to a howling suppression model training method provided by the present invention, the howling detection result of the candidate audio is based on the candidate audio, and the sample audio is selected from the candidate audio, comprising: Determining the severity of howling of the candidate audio based on the number of howling frequency points in the howling detection result of the candidate audio; Based on the proportion of howling audio required for training and the howling severity of the candidate audio, sample audio is screened out from the candidate audio.

[0014] The present invention also provides a howling detection device, comprising the following modules: An acquisition unit, used for acquiring audio to be detected; An evaluation unit, configured to determine candidate howling frequency points in the audio based on amplitude change information of the audio at each frequency point; The detection unit is used to determine the howling detection result of the audio based on the amplitude distribution information of the candidate howling frequency point in the audio and / or the relationship between the amplitude value of the candidate howling frequency point in the audio and the amplitude range perceived by the human ear.

[0015] The present invention also provides a training device for a howling suppression model, comprising the following modules: An audio synthesis unit, used for synthesizing candidate audio based on near-end speech and noise audio; A howling detection unit, configured to perform howling detection on the candidate audio based on the howling detection method, to obtain a howling detection result of the candidate audio; A sample screening unit, configured to screen out sample audio from the candidate audio based on the howling detection result of the candidate audio; The model training unit is used to train a howling suppression model based on the sample audio and the near-end speech used to synthesize the sample audio.

[0016] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the howling detection method or the howling suppression model training method as described above is implemented.

[0017] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the howling detection method or the howling suppression model training method as described above is implemented.

[0018] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned howling detection methods or howling suppression model training methods.

[0019] The howling detection method, device, equipment and medium provided by the present invention detect the howling frequency points by analyzing the amplitude characteristics of the frequency points in combination with the perception characteristics of the human ear. Through the dynamic analysis of the amplitude characteristics, it is possible to adapt to complex or changing environments without frequent debugging of thresholds. At the same time, it does not rely on a large amount of labeled data, but achieves howling detection through feature analysis, thereby reducing data dependence. The method can accurately distinguish howling signals from non-howling signals, reduce false detections and missed detections, has high detection accuracy and environmental adaptability, and provides a stable and efficient howling detection solution for audio equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a flow chart of the howling detection method provided by the present invention.

[0022] Figure 2 It is a flow chart of the training method of the howling suppression model provided by the present invention.

[0023] Figure 3 It is a structural schematic diagram of the acoustic feedback closed-loop simulation system provided by the present invention.

[0024] Figure 4 It is a structural schematic diagram of the howling detection device provided by the present invention.

[0025] Figure 5 It is a structural schematic diagram of a training device for a howling suppression model provided by the present invention.

[0026] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] In audio equipment, howling is an important factor affecting sound quality and equipment stability. Howling is usually caused by excessive gain in the acoustic feedback path or amplification of a specific frequency of the audio signal. It is characterized by a sharp sound of a single frequency. In severe cases, it may cause equipment failure and even affect user experience and reduce equipment reliability. Howling is common in a variety of audio equipment such as sound reinforcement systems, hearing aids, and telephone conference systems. The howling problem is more serious when the acoustic environment is complex or the equipment configuration is not ideal.

[0029] Traditional howling detection methods usually rely on setting multiple thresholds to determine the howling frequency point. These methods determine whether howling exists by monitoring the amplitude, frequency characteristics and other characteristics of the audio signal. However, the setting of these thresholds requires a lot of experimental verification and debugging, and is prone to failure in complex or changing environments. In addition, the threshold setting has a great impact on the stability of the device. If the threshold is set improperly, it is easy to cause false detection or missed detection, affecting the reliability of the device.

[0030] With the development of deep learning technology, howling detection methods based on neural networks have gradually attracted attention. Deep learning methods can extract complex features of signals through self-learning, and can provide higher detection accuracy in more complex and dynamic environments. These methods optimize models through training data, can cope with changes in different environments, and improve the adaptability of models. However, a major problem with deep learning methods is their dependence on a large amount of labeled data. The data collection and labeling process consumes a lot of manpower and material resources, and is limited by data scenarios and environments, resulting in high costs for obtaining labeled data. In addition, due to the limitations of training data, the trained models may lack sufficient generalization capabilities and cannot effectively adapt to all practical application scenarios.

[0031] In view of the above problem, an embodiment of the present invention provides a howling detection method. Figure 1 FIG. 1 is a flow chart of the howling detection method provided by the present invention, such as Figure 1 As shown, the method includes: Step 110: Acquire the audio to be detected.

[0032] Specifically, the audio to be detected can come from a variety of devices, such as real-time voice signals collected by a microphone, audio files read from a storage device, or real-time signals transmitted from a loudspeaker. The audio to be detected may contain multiple components, including target signals (such as voice or music), environmental noise, and howling signals caused by the acoustic feedback path.

[0033] Step 120: Determine candidate howling frequency points in the audio based on the amplitude variation information of the audio at each frequency point.

[0034] Specifically, after obtaining the audio to be detected, the audio is first subjected to spectrum analysis. This step usually converts the audio from the time domain to the frequency domain through short-time Fourier transform or other time-frequency analysis methods (such as wavelet transform). This process decomposes the audio into different frequency components and displays the amplitude changes of each frequency point in each time frame. In this way, the spectrum diagram of the audio at different time frames and frequency points can be obtained, thereby revealing the frequency characteristics of the audio and its changes over time.

[0035] In the spectrum, the amplitude change of each frequency point can reflect the characteristics of the audio, especially when the audio contains howling components. Howling is usually manifested as a sharp increase or decrease in the amplitude of certain frequency points. In the case of growth-type howling, the amplitude will increase rapidly and remain at a high level; while in decay-type howling, the amplitude change may be manifested as a gradual decay. Although the amplitude gradually decreases, its decline process is often relatively gentle or stable. Whether it is growth-type or decay-type howling, it will show obvious amplitude change characteristics in the spectrum.

[0036] In order to accurately identify the howling frequency points, we can consider calculating the amplitude change rate for each frequency point in the audio, with special attention to the frequency points with large amplitude changes. For example, by calculating the amplitude change rate of each frequency point between adjacent time frames, we can capture the frequency points where the amplitude increases or decreases sharply. These frequency points are often related to the occurrence of howling. Then, we can set an amplitude change threshold to help screen out possible howling frequency points. For example, if the amplitude change of a certain frequency point exceeds the preset threshold, or its increase and attenuation amplitude exceeds the average amplitude change rate of the signal, then these frequency points may be howling frequency points. In addition to the simple amplitude change threshold, we can also consider using some other means. For example, the standard deviation of the amplitude can help identify frequency points with large amplitude fluctuations. By calculating the amplitude standard deviation of each frequency point within a certain time window, we can identify frequency points with large amplitude changes, which may be indicative signals of howling. Another method that can be considered is the acceleration of the amplitude change rate, that is, calculating the second-order derivative (acceleration) of the amplitude change. If the acceleration is large, it means that the amplitude change of the frequency point is more drastic, which may indicate the occurrence of howling, whether it is growth or decay.

[0037] Based on the amplitude change information of each frequency point, the frequency points of growing and decaying howling can be effectively identified and distinguished. Finally, the selected frequency points will be used as candidate howling frequency points as the basis for subsequent howling identification. After further verification and screening, these candidate frequency points will help accurately locate the howling source and perform effective suppression processing.

[0038] Step 130: determine a howling detection result of the audio based on the amplitude distribution information of the candidate howling frequency points in the audio and / or the relationship between the amplitude values ​​of the candidate howling frequency points in the audio and the amplitude range perceived by the human ear.

[0039] Specifically, after determining the candidate howling frequency points, it is necessary to further determine whether they are actual howling frequency points based on the amplitude distribution information of these frequency points. Amplitude distribution information refers to the temporal distribution characteristics of the amplitude values ​​of these frequency points in the entire audio signal. Although these candidate frequency points have shown the characteristics of drastic amplitude changes, the amplitude changes alone are not enough to fully confirm that they are actual howling frequency points. By analyzing the amplitude distribution information, those frequency points that are more consistent with the howling characteristics can be further screened out. The amplitude distribution of howling frequency points is usually manifested as the concentration and stability of energy within a certain time range. For example, the amplitude distribution of the frequency points of the growing howling will maintain a high value in multiple time frames, and even show a state of gradual enhancement and stabilization. For the frequency points of the attenuation howling, although the amplitude gradually decreases, its amplitude distribution is still regular, that is, it shows a smooth attenuation trend on the time axis. In contrast, the amplitude distribution of background noise or other non-howling frequency points is often more dispersed, even if the amplitude changes dramatically, showing random fluctuations and lack of stability.

[0040] By statistically analyzing the amplitude distribution information of these candidate frequency points, potential howling frequency points can be further screened out. One method is to analyze whether the amplitude change of the frequency point shows a certain time stability. For example, the amplitude of the frequency point of the growth-type howling should fluctuate stably within a certain range, while the amplitude of the frequency point of the decay-type howling should show a regular decreasing trend. A further approach is to divide the audio into multiple time windows and calculate the amplitude distribution characteristics of the frequency points in each time window. If a frequency point shows high energy and continuous concentration in all time windows, it is further supported as a howling frequency point.

[0041] In addition, since the human ear has a weaker perception of low-loudness signals, some frequency points, although showing amplitude changes on the spectrum, have low loudness and may not be perceived in actual hearing. Even if the amplitude changes of these frequency points are significant on the spectrum, they may not have a significant impact on the user and therefore cannot be regarded as actual howling components. In order to improve the accuracy of howling detection, candidate frequency points with low loudness and difficult to be perceived by the human ear can be eliminated. Specifically, for each candidate howling frequency point, its loudness value in the audio needs to be calculated. If the loudness of some frequency points is lower than the perception threshold of the human ear (for example, lower than a certain phon value), even if their amplitude changes are large, they should be considered to have little effect on the user's auditory experience, thereby excluding them as valid howling frequency points. This screening process helps to reduce misjudgments caused by environmental noise or non-howling sources, and ensures that the howling frequency points finally detected can accurately reflect the howling components that actually affect sound quality and stability.

[0042] In this way, low-loudness frequency points that are not easily detected by the human ear can be effectively eliminated, ensuring that the howling detection results are more accurate and reducing the risk of false detection and missed detection. In this way, the final howling detection results will be more in line with the auditory perception in actual usage scenarios, which can effectively improve the performance of audio equipment and user experience.

[0043] The howling detection method provided in the embodiment of the present invention detects the howling frequency point by analyzing the amplitude characteristics of the frequency point and combining the perception characteristics of the human ear. Through the dynamic analysis of the amplitude characteristics, it can adapt to complex or changing environments without frequent debugging of thresholds. At the same time, it does not rely on a large amount of labeled data, but realizes howling detection through feature analysis, thereby reducing data dependence. The method can accurately distinguish howling signals from non-howling signals, reduce false detection and missed detection, has high detection accuracy and environmental adaptability, and provides a stable and efficient howling detection solution for audio equipment.

[0044] Based on the above embodiment, determining the candidate howling frequency points in the audio based on the amplitude change information of the audio at each frequency point includes: Based on the amplitude change trend and amplitude change acceleration of the audio at each frequency point, candidate howling frequency points and types of the candidate howling frequency points are determined from the frequency points.

[0045] The amplitude change trend of each frequency point refers to the change direction and rate of change of the amplitude of each frequency point in the audio over time, which can be expressed as the sign and magnitude of the first-order derivative of the amplitude of each frequency point. The calculation formula of the first-order derivative is: in, Frequency point In time frame The amplitude value of Frequency point In time frame The amplitude value of is the differential of the time frame, Frequency point In time frame The amplitude change acceleration indicates the change of the amplitude change rate over time, for example, it can be expressed as the magnitude of the second-order derivative of the amplitude at each frequency point. The calculation formula for the second-order derivative is: in, Frequency point In time frame The second-order derivative of the amplitude value. By analyzing the amplitude change trend and amplitude change acceleration of the audio signal at each frequency point, the candidate howling frequency points can be effectively identified, and the types of these frequency points can be further determined. Specifically, if the first-order derivative of the amplitude of a frequency point with respect to the time frame is a constant in a continuous time frame, and the value of the second-order derivative is close to zero (for example, lower than a set threshold), it can be judged that the amplitude change rate of the frequency point is relatively slow, which is usually an important characteristic of the howling frequency point. The stability of this amplitude change rate reflects the persistence of the howling signal in the time dimension, which is significantly different from the violent fluctuations of non-howling frequency points.

[0046] Furthermore, the type of the candidate howling frequency point can be determined by the sign of the first-order derivative. When the first-order derivative is positive and constant, it means that the amplitude of the frequency point is increasing at a steady rate, and it can be classified as a candidate howling frequency point for growth-type howling. The amplitude of such frequency points continues to increase over time and exhibits a higher energy concentration. When the first-order derivative is negative and constant, it means that the amplitude of the frequency point is decreasing at a steady rate, and it can be classified as a candidate howling frequency point for decay-type howling. The amplitude of such frequency points continues to decrease over time, but still exhibits a higher energy concentration.

[0047] Through the joint analysis of the amplitude change trend (first-order derivative) and the amplitude change acceleration (second-order derivative) of each frequency point, not only can the candidate howling frequency points be accurately identified, but also the growth type howling and the decay type howling can be distinguished based on the stability and rate direction of the amplitude change. This analysis method based on mathematical features not only enhances the detection accuracy, but also improves the adaptability of the method in complex environments, providing a reliable data basis for subsequent howling suppression.

[0048] Based on the above embodiment, when the type of the candidate howling frequency point is attenuation type howling, the amplitude distribution information of the candidate howling frequency point in the audio includes spectrum sparsity; The determining of the howling detection result of the audio based on the amplitude distribution information of the candidate howling frequency point in the audio and / or the relationship between the amplitude value of the candidate howling frequency point in the audio and the amplitude range perceived by the human ear includes: When the frequency sparsity of the candidate howling frequency point is greater than the sparsity threshold and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range, it is determined that the howling detection result includes attenuation-type howling.

[0049] Specifically, when the type of howling frequency point is selected as attenuation-type howling, it is possible to further confirm whether it is the actual howling frequency point by analyzing its amplitude distribution information in the audio. The amplitude distribution information includes the important characteristic of spectrum sparsity. Spectrum sparsity refers to the concentration of the energy distribution of the signal in the frequency domain. For signals with higher sparsity, the sparsity is smaller, and its energy is mainly concentrated in a few frequency points, while for signals with lower sparsity, the sparsity is larger, and the energy distribution is more uniform and dispersed in more frequency points. In the attenuation-type howling, although the amplitude gradually decreases, the energy is often concentrated in a few frequency points, resulting in a higher spectrum sparsity. This high sparsity characteristic is an important basis for distinguishing attenuation-type howling from other non-howling signals.

[0050] There are many ways to measure spectrum sparsity. For example, the ratio of the second-order norm to the fourth-order norm can be used to reflect the concentration of frequency point energy. Specifically, the calculation formula for the ratio of the second-order norm to the fourth-order norm is: in, Indicates the number of time frames, is the ratio of the second-order norm to the fourth-order norm, is the normalized sparsity feature. When the sparsity is minimum, , at the frequency point The probability of howling is the highest; when the sparsity is the largest, , at the frequency point The probability of howling is the lowest. The energy distribution can also be evaluated by calculating the ratio of the total energy of several frequency points with the highest energy share to the overall energy. In addition, the entropy value can also be used as a way to measure the sparsity of the spectrum. The lower the entropy value, the more concentrated the energy; the higher the entropy value, the more dispersed the energy distribution. These methods can analyze the distribution characteristics of signal energy in frequency from different angles, so as to accurately determine its sparsity.

[0051] During the detection process, when the spectral sparsity of the candidate howling frequency point is higher than the set sparsity threshold and its amplitude value is within the range of human ear perception, it can be confirmed that the frequency point is an actual attenuated howling. The sparsity condition effectively eliminates signals with more dispersed energy distribution, and the combination of the human ear perception range condition further eliminates frequency points with low loudness that do not affect auditory perception. Through this joint judgment, the real howling signal can be accurately screened out from the candidate frequency points.

[0052] Based on the above embodiment, the amplitude distribution information of the candidate howling frequency point in the audio further includes amplitude dispersion; The step of determining that the howling detection result includes attenuation-type howling when the frequency sparsity of the candidate howling frequency point is greater than a sparsity threshold and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range comprises: When the frequency sparsity of the candidate howling frequency point is greater than the sparsity threshold, the amplitude dispersion of the candidate howling frequency point is less than the dispersion threshold, and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range, it is determined that the howling detection result includes attenuation-type howling.

[0053] Specifically, when the type of the candidate howling frequency point is an attenuated howling, the amplitude distribution information of the candidate howling frequency point in the audio is further analyzed to comprehensively judge whether it is an actual howling frequency point. In addition to spectrum sparsity, the amplitude distribution information also includes the characteristic of amplitude dispersion. Amplitude dispersion is an important indicator to measure the degree of fluctuation of the signal amplitude within a time frame, and is used to evaluate whether the amplitude of the frequency point is stable. For howling frequency points, especially attenuated howling, the amplitude distribution characteristics are characterized by a small fluctuation amplitude and a gradual decrease over time, so the amplitude dispersion is usually low. For ordinary noise or other non-howling signals, the amplitude fluctuation is more random and the amplitude dispersion is higher. This characteristic can help distinguish attenuated howling from other signals. Amplitude dispersion can be expressed by calculating parameters such as the standard deviation or variance of the amplitude value of the frequency point within the time frame. For example, the standard deviation is calculated as: in, It is the frequency point In time frame If the amplitude dispersion of a candidate howling frequency point is lower than the set dispersion threshold, it means that the amplitude change of this frequency point is relatively stable, which conforms to the distribution characteristics of attenuated howling. On the contrary, if the amplitude dispersion is high, it means that the amplitude fluctuation of this frequency point is large, which is more likely to be background noise or other non-howling signals. For example, when the standard deviation When it is less than 3, the current frequency point There may be howling.

[0054] During the detection process, three conditions, namely, spectrum sparsity, amplitude dispersion and human ear perception range, need to be considered jointly. When the spectrum sparsity of a candidate howling frequency point is greater than the sparsity threshold, it indicates that its energy distribution has significant concentration; at the same time, the amplitude dispersion is lower than the dispersion threshold, indicating that its amplitude distribution is relatively stable; the amplitude value is within the human ear perception range, indicating that the signal has a practical impact on auditory perception. When the above conditions are met, the candidate frequency point can be determined to be an actual attenuated howling and included in the howling detection result.

[0055] By combining the multi-feature analysis of spectrum sparsity, amplitude dispersion and human ear perception range, the detection results of attenuation-type howling are more accurate. This comprehensive judgment method not only improves the accuracy and robustness of detection, but also can adapt to the changing audio environment, providing important support for subsequent howling suppression.

[0056] Based on the above embodiment, when the type of the candidate howling frequency point is a growth type howling, the The howling detection result of the audio is determined based on the amplitude distribution information of the candidate howling frequency point in the audio, and / or the relationship between the amplitude value of the candidate howling frequency point in the audio and the amplitude range perceived by the human ear, including: When the amplitude value of the candidate howling frequency point in the audio is within the amplitude range perceived by the human ear, it is determined that the howling detection result includes a growing howl.

[0057] Specifically, the typical feature of growth-type howling is that the amplitude continues to grow over time and maintains a high degree of stability in multiple time frames. Unlike attenuation-type howling, the frequency point amplitude of growth-type howling gradually increases, the energy concentration is higher, and it usually has a more obvious impact on audio quality.

[0058] The judgment of the growth-type howling mainly depends on the relationship between the amplitude value and the human ear's perception range, and there is no need to further analyze the amplitude distribution information. The reason is that the core feature of the growth-type howling is the significant increase in the amplitude of the frequency point, and this continuous increase itself is sufficient to distinguish the howling signal from the background noise or other non-howling signals. In contrast, the amplitude of the attenuation-type howling gradually weakens and the energy may be dispersed, so it is necessary to combine the amplitude distribution information (such as frequency sparsity or amplitude dispersion) to confirm whether it meets the howling characteristics. The growth-type howling has outstanding signal characteristics, and the change in the amplitude value itself can fully reflect its law, without further use of distribution information. Therefore, by judging whether the amplitude value of the candidate frequency point is within the perception range, it can be effectively confirmed whether it is an actual howling frequency point. Specifically, when the amplitude value of a candidate frequency point is greater than the set perception threshold, it can be directly judged as a growth-type howling frequency point; if its amplitude value is lower than the perception range, even if the amplitude increases significantly, it may not have a substantial impact on the audio quality, and it will not be judged as a growth-type howling frequency point.

[0059] The method for judging the growing howling can directly confirm that the detection result contains the growing howling by combining the amplitude value with the human ear perception range. This method effectively utilizes the prominent characteristics of the growing howling signal, simplifies the judgment process while ensuring the detection accuracy, and improves the detection efficiency.

[0060] Based on the above embodiment, when the audio is collected by a sound reinforcement system, the method further includes: Determining a critical gain of the sound reinforcement system based on a howling detection result of the audio and a current gain of the sound reinforcement system; When the current gain is greater than the critical gain, howling control is performed on the sound reinforcement system.

[0061] Specifically, the critical gain is a key threshold of the gain value in the sound reinforcement system, which indicates the maximum gain value allowed by the sound reinforcement system without causing howling. When the actual gain of the sound reinforcement system exceeds the critical gain, the gain accumulation in the acoustic feedback path will cause howling, affecting the audio quality and the operating stability of the system.

[0062] The determination of critical gain depends on the howling detection method. By analyzing the howling frequency points and their energy distribution characteristics in the current audio signal and combining the gain value of the current sound reinforcement system, the critical gain range of the sound reinforcement system can be accurately identified. The howling detection method determines whether the current gain is close to the boundary state of howling by identifying the current howling signal, and accordingly confirms the maximum allowable gain value without howling.

[0063] When the current gain is greater than the critical gain, it is necessary to implement howling control on the sound reinforcement system. The means of howling control include reducing the gain value of the sound reinforcement system to below the critical gain, adjusting the feedback path of the sound reinforcement equipment, or using adaptive feedback elimination technology to weaken the impact of the feedback signal in real time. Reducing the gain is the most direct way. By making the current gain less than the critical gain, the occurrence of howling can be effectively avoided. At the same time, in order to maintain the clarity and gain effect of the audio output while controlling the howling, the acoustic characteristics of the feedback path can also be combined for detailed adjustments, such as optimizing the microphone and speaker layout of the equipment, or adjusting the delay parameters and room impulse response in the feedback path.

[0064] By dynamically determining the critical gain and adjusting the gain of the sound reinforcement system in real time through the howling detection method, the stable operation of the system can be ensured in a changing acoustic environment, avoiding the interference of howling on the audio quality. This mechanism enhances the stability and adaptability of the sound reinforcement system, while improving the user experience of the sound reinforcement system in complex scenarios.

[0065] An embodiment of the present invention provides a training method for a howling suppression model. Figure 2 is a flow chart of the training method of the howling suppression model provided by the present invention, such as Figure 2 As shown, the method includes: Step 210: synthesize candidate audio based on near-end speech and noise audio.

[0066] Near-end speech refers to the original speech signal collected by the target microphone, which is usually the main audio content that needs to be processed and enhanced in the sound reinforcement system. Noise audio simulates the background noise that may exist in the sound reinforcement environment, such as environmental noise, equipment noise, or feedback signal interference. By synthesizing near-end speech and noise audio, it is possible to simulate the complex audio signals that the sound reinforcement system may encounter in actual operation scenarios, providing diverse data samples for training the howling suppression model.

[0067] In specific operations, the near-end speech signal and the noise audio can be mixed according to a preset signal-to-noise ratio to control the relative clarity of the target speech in a noisy environment. For example, in the case of a high signal-to-noise ratio, the energy of the near-end speech is dominant, while in the case of a low signal-to-noise ratio, the noise audio has a greater interference with the overall audio. By adjusting the signal-to-noise ratio, candidate audio with different interference strengths can be generated to cover the amplification needs of various real scenarios.

[0068] Step 220: Perform a howling detection on the candidate audio based on the howling detection method to obtain a howling detection result of the candidate audio.

[0069] The synthesized candidate audio is comprehensively analyzed using the aforementioned howling detection method to identify whether there is a howling signal therein and generate the corresponding detection result. The candidate audio usually contains near-end speech, noise interference and possible howling components. The howling detection method can accurately extract the howling features in these audios and provide basic data for subsequent processing. Specifically, the howling detection method analyzes the spectral characteristics of the audio signal and makes a comprehensive judgment on the amplitude change trend, amplitude distribution information and other characteristics (such as spectral sparsity or amplitude dispersion) of each frequency point in the audio. The typical characteristics of the howling signal are the drastic change of amplitude and the concentration of energy. Therefore, the howling detection method first extracts possible candidate howling frequency points in the candidate audio. The amplitude change information of these candidate frequency points is matched with the howling characteristics to determine whether it meets the characteristics of the howling signal. Finally, the detection results will clearly identify the howling frequency points in the candidate audio, the type of howling signal (increase or decay type) and other information. These detection results can not only be used to screen training data, but also provide a reference for the optimization of the subsequent howling suppression model.

[0070] Step 230: Filter out sample audio from the candidate audio based on the howling detection result of the candidate audio.

[0071] Specifically, in this step, by analyzing the howling detection results of the candidate audio, sample audio that meets specific conditions is screened out for subsequent howling suppression model training. The purpose of screening sample audio is to ensure that the training data set has sufficient diversity and representativeness, while avoiding the interference of sample audio with too severe howling on the model training effect.

[0072] In the candidate audio, the amplification gain and room impulse response will significantly affect the generation and severity of howling in the audio. When the amplification gain is large or the feedback path characteristics cause the cumulative gain of the signal to be high, the probability and severity of howling will increase significantly. Although this kind of audio with severe howling can reflect the extreme processing capabilities of the howling suppression model, if the proportion of such data in the training set is too high, it may make it difficult for the model to recover the target speech signal from the audio with severe howling during the training process, resulting in difficulty in converging the training loss function. Therefore, it is necessary to perform statistical analysis on the howling detection results of the candidate audio, and screen out sample audio with an appropriate degree of howling based on the number of howling frequency points. For example, sample audio with a large number of howling frequency points is regarded as severe howling data, and screening out such sample audio helps to better train the model.

[0073] In addition to the number of howling frequency points, key information in the howling detection results also includes the proportion of the howling signal to the total audio signal (howling audio ratio) and the type of howling (growth type or decay type). By analyzing this information, more diverse sample audio can be provided for model training. Through this screening process, a representative sample audio collection can be generated, including howling samples of appropriate proportions and severity, and a reasonable distribution of growth type and decay type howling samples. Such a data set can effectively improve the training effect of the howling suppression model in complex environments and ensure that it has strong generalization capabilities.

[0074] Step 240: training a howling suppression model based on the sample audio and the near-end speech used to synthesize the sample audio.

[0075] Specifically, the model is trained through supervised learning using the selected sample audio as the input signal and the corresponding near-end speech as the target label. The training goal is to learn the howling features in the audio through the model, and accurately distinguish the howling signal from the target speech signal, so as to effectively suppress the howling signal.

[0076] The sample audio contains a variety of howling characteristics and noise interference. After being screened in the above steps, these samples contain a moderate proportion of howling audio and cover the different characteristics of growth-type howling and decay-type howling, which can provide diverse training data for the model. During the training process, the sample audio is input into the howling suppression model, and the model learns the characteristic structure of the audio through layer-by-layer processing, including the frequency distribution of the howling signal, the amplitude variation law, and the difference characteristics between the howling and the target speech.

[0077] The near-end speech is used as a label signal to provide a reference for the model, ensuring that the model can retain the target speech signal as completely as possible while suppressing howling. The near-end speech is usually a pure target speech signal that is not interfered by howling or noise. Therefore, it can be used to guide the model to learn how to remove the howling component while retaining the frequency characteristics, speech content and sound quality of the speech. The training loss function of the model usually uses the error index of speech signal reconstruction, such as the mean square error in the time domain or the energy spectrum error in the frequency domain. It can also be combined with the Bark frequency domain loss perceived by the human ear to ensure that the suppression effect of the model is consistent with the actual auditory perception.

[0078] In a specific implementation, a deep learning model (such as a convolutional neural network or a time-frequency mixture network) can be used to extract and reconstruct the features of the input signal. For example, a mixed signal of near-end speech, noise audio, and feedback signal is firstly Convert to complex spectrum by short-time Fourier transform , and calculate the energy spectrum In order to better conform to the auditory characteristics of the human ear, the mixed signal energy is mapped from the frequency domain to the Bark domain, and the logarithmic decibel value is taken to obtain the Bark domain energy spectrum ,in Indicates the frequency point in the Bark domain. In the feature extraction process, a feedforward sequence memory neural network model is used for training. The training goal is to generate a mask matrix Mask for howling suppression, and its calculation formula is: in, and Represent the energy spectrum of the noise audio and feedback signal in the Bark domain respectively. The mask matrix is ​​used to estimate the enhancement process of the target signal under howling interference. In order to optimize the model, the loss function is defined as the mean square error between the mask matrix predicted by the model and the true mask matrix for each frame: in, is the total number of frequency points in the Bark domain, is the true mask matrix. By minimizing this loss function, the model can learn how to accurately distinguish near-end speech, noisy audio, and feedback signals, thereby effectively suppressing howling and restoring a clear target speech signal.

[0079] Through this training process, the howling suppression model can effectively grasp the characteristic differences between the howling signal and the target speech signal, so as to identify and suppress howling in real time in practical applications while retaining the integrity and naturalness of the speech signal. Ultimately, the trained model can be applied to a variety of sound reinforcement scenarios, providing audio equipment with stable and reliable howling suppression capabilities.

[0080] Based on the above embodiment, the step of selecting a sample audio from the candidate audio based on the howling detection result of the candidate audio includes: Determining the severity of howling of the candidate audio based on the number of howling frequency points in the howling detection result of the candidate audio; Based on the proportion of howling audio required for training and the howling severity of the candidate audio, sample audio is screened out from the candidate audio.

[0081] Specifically, the severity of howling in the candidate audio is determined by analyzing the number of howling frequency points in the howling detection results. The number of howling frequency points reflects the coverage and impact intensity of the howling signal in the audio. The greater the number, the greater the severity of the howling signal. However, audio with too severe howling may cause the target speech signal to be severely masked, making it difficult for the model to accurately learn effective howling suppression characteristics during training, and may also cause the training loss function to be difficult to converge. Therefore, audio with the number of howling frequency points reaching the set upper limit needs to be eliminated during the screening process. This elimination operation can effectively filter out overly extreme audio samples to avoid negative impacts on the training effect.

[0082] While eliminating audio with severe howling, the remaining candidate audio needs to be further screened in combination with the howling audio ratio required for training. The howling audio ratio is the proportion of audio containing howling signals in the training data set, and is an important indicator for controlling the distribution and characteristics of training data. A reasonable ratio can ensure that the model can fully learn the howling suppression characteristics while maintaining good restoration capabilities for the target speech signal. In actual operation, the howling audio ratio in the sample audio can be controlled by setting the amplification gain range. As shown in Table 1 below, different gain ranges correspond to different howling audio ratios.

[0083] Table 1: Different gain ranges correspond to different proportions of howling audio

[0084] Through this screening process, we can remove the sample audio with severe howling and construct a sample audio set covering different proportions of howling audio. This method ensures the quality and diversity of the model training data, allowing the howling suppression model to show strong adaptability in complex scenarios.

[0085] The training method of the howling suppression model provided by the present invention reasonably controls the quality of the sample audio by analyzing the number of howling frequency points and the howling proportion of the candidate audio, so that the model has stronger generalization ability and robustness in complex scenarios. At the same time, there is no need to rely on a large amount of real scene data, which significantly reduces the cost of data collection and annotation. Through the training loss function optimized for the howling characteristics, the model can accurately suppress the howling signal while retaining the sound quality and integrity of the target speech, providing a stable and efficient howling suppression capability for audio devices in practical applications.

[0086] Based on the above embodiments, Figure 3 Schematic diagram of the structure of the acoustic feedback closed-loop simulation system provided by the present invention. Figure 3As shown, the input signal is composed of a mixture of near-end audio and noise audio, which is used to simulate the actual input signal in the sound reinforcement environment. The mixture of the two can be adjusted by setting the signal-to-noise ratio to generate audio signals with different interference intensities. The adaptive feedback elimination module is used to simulate the real-time suppression process of the sound reinforcement equipment on the acoustic feedback. By analyzing the feedback signal of the previous frame, the current audio is adaptively adjusted to partially eliminate the impact of feedback. It is a key function in the system to reduce howling. The delay module simulates the time delay caused by spatial propagation or equipment processing in the acoustic feedback path. The existence of the delay causes a phase difference between the feedback signal and the original signal, which may cause the amplification of certain frequencies. The gain module simulates the amplification effect of the sound reinforcement equipment. The size of the gain value determines the energy intensity of the signal in the feedback path. When the gain is too high, the feedback signal is continuously amplified, which may cause howling. The room impulse response module is used to simulate the propagation characteristics of the audio signal in the physical space, such as acoustic effects such as reflection and absorption. Its characteristics determine the frequency response and attenuation characteristics of the feedback path. After being processed by the above modules, the feedback signal is returned to the input end to form a closed-loop feedback path. When the gain is set too high and the accumulated energy of the feedback path exceeds the stability condition of the system, howling may occur in the audio signal. Through this simulation system, howling audio with different intensities and characteristic distributions can be generated, providing diverse simulation data for the training of the howling suppression model. As shown in Table 2, after the acoustic feedback control is turned on, the probability of howling phenomenon is reduced at the same gain.

[0087] Table 2: Howling audio ratio corresponding to different gains before and after acoustic feedback control is turned on

[0088] The howling detection device and the howling suppression model training device provided by the present invention are described below. The howling detection device and the howling suppression model training device described below and the howling detection method and the howling suppression model method described above can be referred to each other.

[0089] Figure 4 is a schematic diagram of the structure of the howling detection device provided by the present invention, such as Figure 4 As shown, the device comprises: An acquisition unit 410 is used to acquire audio to be detected; An evaluation unit 420, configured to determine a candidate howling frequency point in the audio based on amplitude change information of the audio at each frequency point; The detection unit 430 is used to determine the howling detection result of the audio based on the amplitude distribution information of the candidate howling frequency point in the audio and / or the relationship between the amplitude value of the candidate howling frequency point in the audio and the amplitude range perceived by the human ear.

[0090] The howling detection device provided in the embodiment of the present invention detects the howling frequency point by analyzing the amplitude characteristics of the frequency point and combining the perception characteristics of the human ear. Through the dynamic analysis of the amplitude characteristics, it can adapt to complex or changing environments without frequent debugging of thresholds. At the same time, it does not rely on a large amount of labeled data, but realizes howling detection through feature analysis, thereby reducing data dependence. The method can accurately distinguish howling signals from non-howling signals, reduce false detection and missed detection, has high detection accuracy and environmental adaptability, and provides a stable and efficient howling detection solution for audio equipment.

[0091] Based on any of the above embodiments, the evaluation unit is specifically used for: Based on the amplitude change trend and amplitude change acceleration of the audio at each frequency point, candidate howling frequency points and types of the candidate howling frequency points are determined from the frequency points.

[0092] Based on any of the above embodiments, the detection unit is specifically used for: In a case where the type of the candidate howling frequency point is attenuation-type howling, the amplitude distribution information of the candidate howling frequency point in the audio includes spectrum sparsity; The determining of the howling detection result of the audio based on the amplitude distribution information of the candidate howling frequency point in the audio and / or the relationship between the amplitude value of the candidate howling frequency point in the audio and the amplitude range perceived by the human ear includes: When the frequency sparsity of the candidate howling frequency point is greater than the sparsity threshold and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range, it is determined that the howling detection result includes attenuation-type howling.

[0093] The amplitude distribution information of the candidate howling frequency point in the audio also includes amplitude dispersion; The step of determining that the howling detection result includes attenuation-type howling when the frequency sparsity of the candidate howling frequency point is greater than a sparsity threshold and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range comprises: When the frequency sparsity of the candidate howling frequency point is greater than the sparsity threshold, the amplitude dispersion of the candidate howling frequency point is less than the dispersion threshold, and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range, it is determined that the howling detection result includes attenuation-type howling.

[0094] The howling detection result of the audio is determined based on the amplitude distribution information of the candidate howling frequency point in the audio, and / or the relationship between the amplitude value of the candidate howling frequency point in the audio and the amplitude range perceived by the human ear, including: When the amplitude value of the candidate howling frequency point in the audio is within the amplitude range perceived by the human ear, it is determined that the howling detection result includes a growing howl.

[0095] Determining a critical gain of the sound reinforcement system based on a howling detection result of the audio and a current gain of the sound reinforcement system; When the current gain is greater than the critical gain, howling control is performed on the sound reinforcement system.

[0096] Figure 5 Schematic diagram of the structure of the training device of the howling suppression model provided by the present invention. Figure 5 As shown, the device comprises: An audio synthesis unit 510, configured to synthesize candidate audio based on near-end speech and noise audio; A howling detection unit 520, configured to perform howling detection on the candidate audio based on the howling detection method, and obtain a howling detection result of the candidate audio; A sample screening unit 530, configured to screen out sample audio from the candidate audio based on the howling detection result of the candidate audio; The model training unit 540 is used to train a howling suppression model based on the sample audio and the near-end speech used to synthesize the sample audio.

[0097] The training device of the howling suppression model provided by the present invention analyzes the number of howling frequency points and the howling proportion of the candidate audio, reasonably controls the quality of the sample audio, and enables the model to have stronger generalization ability and robustness in complex scenarios. At the same time, there is no need to rely on a large amount of real scene data, which significantly reduces the cost of data collection and annotation. Through the training loss function optimized for the howling characteristics, the model can accurately suppress the howling signal while retaining the sound quality and integrity of the target speech, providing a stable and efficient howling suppression capability for audio equipment in practical applications.

[0098] Based on any of the above embodiments, the sample screening unit is specifically used for: Determining the severity of howling of the candidate audio based on the number of howling frequency points in the howling detection result of the candidate audio; Based on the proportion of howling audio required for training and the howling severity of the candidate audio, sample audio is screened out from the candidate audio.

[0099] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6As shown, the electronic device may include: a processor (processor) 610, a communication interface (Communications Interface) 620, a memory (memory) 630 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the howling detection method or the howling suppression model training method. Among them, the howling detection method includes: Get the audio to be detected; Determining candidate howling frequency points in the audio based on amplitude change information of the audio at each frequency point; A howling detection result of the audio is determined based on the amplitude distribution information of the candidate howling frequency points in the audio and / or the relationship between the amplitude values ​​of the candidate howling frequency points in the audio and the amplitude range perceived by human ears.

[0100] The training method of the howling suppression model includes: Synthesize candidate audio based on near-end speech and noise audio; Based on the howling detection method, performing howling detection on the candidate audio to obtain a howling detection result of the candidate audio; Based on the howling detection result of the candidate audio, filter out sample audio from the candidate audio; A howling suppression model is trained based on the sample audio and the near-end speech used to synthesize the sample audio.

[0101] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0102] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute a howling detection method or a howling suppression model training method. The howling detection method includes: Get the audio to be detected; Determining candidate howling frequency points in the audio based on amplitude change information of the audio at each frequency point; A howling detection result of the audio is determined based on the amplitude distribution information of the candidate howling frequency points in the audio and / or the relationship between the amplitude values ​​of the candidate howling frequency points in the audio and the amplitude range perceived by human ears.

[0103] The training method of the howling suppression model includes: Synthesize candidate audio based on near-end speech and noise audio; Based on the howling detection method, performing howling detection on the candidate audio to obtain a howling detection result of the candidate audio; Based on the howling detection result of the candidate audio, filter out sample audio from the candidate audio; A howling suppression model is trained based on the sample audio and the near-end speech used to synthesize the sample audio.

[0104] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the howling detection method or the howling suppression model training method provided by the above methods. The howling detection method includes: Get the audio to be detected; Determining candidate howling frequency points in the audio based on amplitude change information of the audio at each frequency point; A howling detection result of the audio is determined based on the amplitude distribution information of the candidate howling frequency points in the audio and / or the relationship between the amplitude values ​​of the candidate howling frequency points in the audio and the amplitude range perceived by human ears.

[0105] The training method of the howling suppression model includes: Synthesize candidate audio based on near-end speech and noise audio; Based on the howling detection method, performing howling detection on the candidate audio to obtain a howling detection result of the candidate audio; Based on the howling detection result of the candidate audio, filter out sample audio from the candidate audio; A howling suppression model is trained based on the sample audio and the near-end speech used to synthesize the sample audio.

[0106] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0107] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A howling detection method, characterized in that: include: Get the audio to be detected; Determining candidate howling frequency points in the audio based on amplitude change information of the audio at each frequency point; A howling detection result of the audio is determined based on the amplitude distribution information of the candidate howling frequency points in the audio and / or the relationship between the amplitude values ​​of the candidate howling frequency points in the audio and the amplitude range perceived by human ears.

2. The howling detection method according to claim 1, characterized in that: The determining, based on the amplitude change information of the audio at each frequency point, candidate howling frequency points in the audio includes: Based on the amplitude change trend and amplitude change acceleration of the audio at each frequency point, candidate howling frequency points and types of the candidate howling frequency points are determined from the frequency points.

3. The howling detection method according to claim 2, characterized in that: In a case where the type of the candidate howling frequency point is attenuation-type howling, the amplitude distribution information of the candidate howling frequency point in the audio includes spectrum sparsity; The determining of the howling detection result of the audio based on the amplitude distribution information of the candidate howling frequency point in the audio and / or the relationship between the amplitude value of the candidate howling frequency point in the audio and the amplitude range perceived by the human ear includes: When the frequency sparsity of the candidate howling frequency point is greater than the sparsity threshold and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range, it is determined that the howling detection result includes attenuation-type howling.

4. The howling detection method according to claim 3, characterized in that: The amplitude distribution information of the candidate howling frequency point in the audio also includes amplitude dispersion; The step of determining that the howling detection result includes attenuation-type howling when the frequency sparsity of the candidate howling frequency point is greater than a sparsity threshold and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range comprises: When the frequency sparsity of the candidate howling frequency point is greater than the sparsity threshold, the amplitude dispersion of the candidate howling frequency point is less than the dispersion threshold, and the amplitude value of the candidate howling frequency point in the audio is within the human ear perception amplitude range, it is determined that the howling detection result includes attenuation-type howling.

5. The howling detection method according to claim 2, characterized in that: In a case where the type of the candidate howling frequency point is a growing howling, determining the howling detection result of the audio based on the amplitude distribution information of the candidate howling frequency point in the audio and / or the relationship between the amplitude value of the candidate howling frequency point in the audio and the amplitude range perceived by the human ear includes: When the amplitude value of the candidate howling frequency point in the audio is within the amplitude range perceived by the human ear, it is determined that the howling detection result includes a growing howl.

6. The howling detection method according to any one of claims 1 to 5, characterized in that: In the case where the audio is collected based on a sound reinforcement system, the method further includes: Determining a critical gain of the sound reinforcement system based on a howling detection result of the audio and a current gain of the sound reinforcement system; When the current gain is greater than the critical gain, howling control is performed on the sound reinforcement system.

7. A training method for a howling suppression model, characterized in that: include: Synthesize candidate audio based on near-end speech and noise audio; Based on the howling detection method according to any one of claims 1 to 5, performing howling detection on the candidate audio to obtain a howling detection result of the candidate audio; Based on the howling detection result of the candidate audio, filter out sample audio from the candidate audio; A howling suppression model is trained based on the sample audio and the near-end speech used to synthesize the sample audio.

8. The method for training a howling suppression model according to claim 7, characterized in that: The step of selecting a sample audio from the candidate audio based on the howling detection result of the candidate audio includes: Determining the severity of howling of the candidate audio based on the number of howling frequency points in the howling detection result of the candidate audio; Based on the proportion of howling audio required for training and the howling severity of the candidate audio, sample audio is screened out from the candidate audio.

9. A howling detection device, characterized in that: include: An acquisition unit, used for acquiring audio to be detected; An evaluation unit, configured to determine candidate howling frequency points in the audio based on amplitude change information of the audio at each frequency point; The detection unit is used to determine the howling detection result of the audio based on the amplitude distribution information of the candidate howling frequency point in the audio and / or the relationship between the amplitude value of the candidate howling frequency point in the audio and the amplitude range perceived by the human ear.

10. A training device for a howling suppression model, characterized in that: include: An audio synthesis unit, used for synthesizing candidate audio based on near-end speech and noise audio; a howling detection unit, configured to perform howling detection on the candidate audio based on the howling detection method according to any one of claims 1 to 5, to obtain a howling detection result of the candidate audio; A sample screening unit, configured to screen out sample audio from the candidate audio based on the howling detection result of the candidate audio; The model training unit is used to train a howling suppression model based on the sample audio and the near-end speech used to synthesize the sample audio.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the howling detection method according to any one of claims 1 to 6, or the howling suppression model training method according to claim 7 or 8 is implemented.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the howling detection method according to any one of claims 1 to 6 or the howling suppression model training method according to claim 7 or 8 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the howling detection method according to any one of claims 1 to 6 or the howling suppression model training method according to claim 7 or 8 is implemented.

Images