A method for improving intercom voice of surveillance camera

By improving the webrtc algorithm and speaker amplification strategy, the surveillance camera's sound quality blur and echo cancellation problems under low signal-to-noise ratio conditions are solved, and clearer sound quality and stable echo cancellation effect are achieved, reducing the R&D cycle.

CN118430555BActive Publication Date: 2024-11-15ZHUHAI RAYSHARP TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410589729.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-13
Publication Date
2024-11-15
Estimated Expiration
2044-05-13

AI Technical Summary

Technical Problem

The existing surveillance cameras have a low signal-to-noise ratio microphone array and speaker structure, and the sound quality is blurred, the echo cancellation effect is poor, and the speaker power consumption is amplified, causing sound distortion, which increases the R&D and development cycle.

Method used

Improve the Ans and AEC algorithms of Webrtc, remove noise and echo through quantile estimation, Wiener filter and delay estimation, and optimize the amplification strategy from the master to the speaker to ensure clear sound quality and does not exceed the speaker's rated power.

Benefits of technology

The sound quality stability and echo cancellation effect are significantly improved under low signal-to-noise ratio conditions, reducing the speaker amplifier debugging cycle, adapting to different microphone and speaker structures, and improving intercom effect and portability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118430555B_ABST
    Figure CN118430555B_ABST
Patent Text Reader

Abstract

The present invention provides a method for improving the intercom voice of a surveillance camera, which specifically includes: S1, improving the ANS algorithm of WebRTC, removing the low frequency that causes the near-end data to be unclear, and removing the non-steady-state noise that easily causes howling in the high frequency part; S2, improving the AEC algorithm of WebRTC, improving the problem of occasional unclean echo filtering when turning on the intercom; S3, improving the amplification strategy from the main control to the speaker, improving the speaker sound quality, and reducing the research and development cycle. The present invention can better suppress noise, perform echo cancellation, and adapt to microphone arrays and speakers with low signal-to-noise ratios by improving the ANS and AEC algorithms of WebRTC, and can significantly improve the speaker sound quality by improving the amplification strategy from the main control to the speaker.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of security monitoring, and in particular to a method for improving intercom voice of a monitoring camera. Background Art

[0002] With the development of the Internet and the popularization of security equipment, audio preview, intercom, and speaker alarm functions are widely used in the field of surveillance cameras. Among them, audio noise reduction, AGC amplification, and echo cancellation algorithms based on WebRTC are also widely used, such as security monitoring pickups in shopping malls and home doorbells.

[0003] The existing audio3a technology mainly uses spectral algorithms. For example, AEC and ANS calculate the spectrum based on the original signal to estimate the noise and echo and multiply or subtract the noise and echo parts by relevant coefficients. AGC distinguishes the background and voice parts according to the spectrum to perform different amplification gains. The spectral method mainly includes the WebRTC algorithm and the Speex algorithm, both of which are open source.

[0004] WebRTC algorithm and Speex algorithm using spectrum method, WebRTC is better than Speex algorithm in echo cancellation, and WebRTC algorithm is better than Speex algorithm in nonlinear echo processing; however, most of the ARM boards are microphone arrays with low signal-to-noise ratio, and WebRTC's denoising algorithm will not remove low-frequency background noise cleanly, resulting in unclear preview sound quality; because different ARM boards have different delay times between remote data and board data when talking, the delay time setting greatly affects the echo effect, as well as the appearance and structure of mics and speakers on different boards, and the echo cancellation effect is often unsatisfactory; WebRTC's AGC amplification is used to match the power consumption amplification factor of the speaker, and the power consumption of the speaker for different sounds of different volumes often exceeds the rated power of the speaker, resulting in speaker sound distortion, and there is no complete process for speaker debugging, which increases the R&D cycle. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes a method for improving the intercom voice of a surveillance camera. By improving the ANS and AEC algorithms of WebRTC, the noise can be better suppressed, echo cancellation can be performed, and the microphone array and speaker with low signal-to-noise ratio can be adapted. By improving the amplification strategy from the main control to the speaker, the speaker sound quality can be significantly improved.

[0006] To implement the above technical solution, the present invention provides a method for improving the intercom voice of a surveillance camera, which specifically includes the following steps:

[0007] S1. Improve the ANS algorithm of WebRTC to remove the low frequency that causes unclear near-end data and the high frequency of non-steady-state noise that easily causes howling, including:

[0008] S11, using quantile estimation method to perform initial noise estimation;

[0009] S12, updating noise estimation based on likelihood ratio method;

[0010] S13, using Wiener filter to filter out noise;

[0011] S14, multiplying the coefficient by 0 when performing Wiener filtering on some high and low frequencies to reduce the background noise;

[0012] S2. Improve the AEC algorithm of WebRTC to solve the problem of occasional poor echo filtering when starting intercom, including:

[0013] S21, delay estimation;

[0014] S22, linear filtering;

[0015] S23, nonlinear filtering, removing echo residue in the near-end sound by limiting the nonlinear filtering suppression coefficient;

[0016] S3. Improve the amplification strategy from the main control to the speaker and improve the speaker sound quality, including:

[0017] S31, use 1khz sine wave audio as the alarm sound source, and use webrtc agc to limit the sound source, with default parameters target = 5, gain = 5, enable = 1;

[0018] S32, set the target of agc target = 5, and save the audio data to see whether the amplitude value after agc reaches the amplitude value corresponding to target = 5;

[0019] S33, sending the AGC sound through webrtc to the main control, the main control sets the volume to be amplified or reduced again, and finds the volume value when the main control voltage is 80% of the rated voltage. This volume corresponds to the maximum output volume value maxoutvolume that can be set. Based on the volume value maxoutvolume, the speaker amplifier amplification factor is debugged to make the speaker volume output reach the maximum rated power of the speaker. At this time, the speaker amplifier factor has been determined;

[0020] S34, taking the volume value below maxoutvolume as the default volume value defaultoutvolume, which has the maximum waveform and is not distorted;

[0021] S35. Set the intercom level parameters according to the alarm volume level parameters. Since the sound source of the intercom is smaller than the alarm sound source, the master volume setting value can be appropriately increased based on the alarm. After determining the intercom volume parameters, you can perform hardware testing. The test requirements are: the waveform is not distorted, the sound is not broken, and it is less than the rated power of the speaker. However, if the sound is increased a little bit, the waveform will be distorted. Finding the critical point is the best sound quality point.

[0022] Preferably, in step S13, the Wiener filter denoising formula of ans in the Wiener filter is:

[0023] epsilon = 1*10 -7 ;

[0024]

[0025] snrprior[n]=0.98f*prevestimate[n]+(1.0f-0.98f)*snrpost[n];

[0026]

[0027] Where prevsignalspectrum is the signal spectrum of the previous frame, prevgain is the gain of the previous frame, and prevnoisespectrum is the noise spectrum of the previous frame; overdrive is the value corresponding to the set denoising level; gain is the final denoising gain coefficient; n is the spectrum subscript converted from time domain to frequency domain, and its range is 0 to 64.

[0028] Preferably, in order to reduce the background noise, the calculation formula of the denoising gain coefficient is improved as follows:

[0029]

[0030] This algorithm removes low frequencies that obscure the near-end data.

[0031] Preferably, the nonlinear suppression coefficient calculation formula in step S23 is:

[0032] hNl[n]=hNl[n] (overDriveSm*overDriveCurve[n]) ;

[0033] efw[n]=hNl[n]*efw[n];

[0034] hNl is the nonlinear filtering suppression coefficient, overDriveSm is the set echo cancellation level, overDriveCurve is a monotonically increasing convex curve in the range of [1.0, 2.0], and efw is the estimated echo spectrum signal after linear filtering.

[0035] Preferably, in step S23, if hN1 is too large, a large amount of echo residue will be caused, so the nonlinear filtering suppression coefficient is improved:

[0036] hNl[n]=(hNl[n]*0.5) (overDriveSm*overDriveCurve[n]) ;

[0037] efw[n]=hNl[n]*efw[n];

[0038] This algorithm can remove echo residue from the near-end sound.

[0039] The beneficial effects of the surveillance camera intercom voice improvement method provided by the present invention are:

[0040] 1) The improved webrtc ans algorithm can make the sound quality clearer in the case of a microphone array with a low signal-to-noise ratio at a long pickup distance, without sacrificing the microphone's pickup distance to obtain clarity. The preview effect is more stable and the hearing is more comfortable;

[0041] 2) The improved WebRTC AEC algorithm will not produce any echo when the intercom is turned on, and can ensure that the volume of the speaker and the incoming mobile phone are loud enough. For products with poor microphone and speaker structure design, as well as ARM boards with different chips, the delay time is easier to set, with better portability and better intercom effect;

[0042] 3) Improve the whole speaker amplifier amplification strategy to ensure that the speaker sound is not distorted and does not exceed the rated power, reducing the research and development cycle of hardware colleagues and eliminating the need for repeated debugging of the speaker amplifier coefficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is the overall flow chart of the present invention.

[0044] Figure 2 It is a time domain comparison diagram of audio data after different processing methods in the present invention.

[0045] Figure 3 It is a frequency domain comparison diagram of audio data after different processing methods in the present invention.

[0046] Figure 4 This is a diagram showing the effect of echo cancellation after being processed by the WebRTC AEC algorithm in the present invention and the improved WebRTC AEC algorithm in the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary persons in the art without creative work are within the scope of protection of the present invention.

[0048] Embodiment: A method for improving the intercom voice of a surveillance camera.

[0049] Reference Figures 1 to 4 As shown in the figure, a method for improving the intercom voice of a surveillance camera mainly includes: (1) improving the ANS algorithm of WebRTC to remove the low frequency that causes unclear near-end data and the non-steady-state noise that easily causes howling in the high frequency part; (2) improving the AEC algorithm of WebRT to improve the problem of occasional unclean echo filtering when turning on the intercom; (3) improving the amplification strategy from the main control to the speaker, improving the speaker sound quality and reducing the R&D cycle.

[0050] The detailed process is as follows:

[0051] 1. Improvement of ANS algorithm based on webrtc:

[0052] The ANS algorithm of WebRTC mainly consists of two parts: the first step is noise estimation, and the second step is noise filtering. The noise estimation uses the initial noise estimation and the updated noise estimation. The initial noise estimation uses the quantile estimation method, and the updated noise estimation uses the likelihood ratio-based method. The noise filtering uses the Wiener filter. Among them, prevsignalspectrum is the signal spectrum of the previous frame, prevgain is the gain of the previous frame, and prevnoisespectrum is the noise spectrum of the previous frame; overdrive is the value corresponding to the set denoising level. Gain is the final denoising gain coefficient; n is the spectrum subscript converted from time domain to frequency domain, and its range is 0 to 64.

[0053] The Wiener filter denoising formula of ans is:

[0054] epsilon = 1*10 -7 (1);

[0055]

[0056]

[0057] snrprior[n]=0.98f*prevestimate[n]+(1.0f-0.98f)*snrpost[n] (4);

[0058]

[0059] The present invention improves the denoising coefficient:

[0060]

[0061] When performing Wiener filtering on some high and low frequencies, multiply them by a coefficient of 0 to reduce the background noise.

[0062] Reference Figure 2 and Figure 3 , compare the audio data after recording by the board mic in the time domain and spectrum. Figure 3 This is a time domain comparison chart of audio data after different processing methods. Figure 4 This is a frequency domain comparison diagram of audio data after different processing methods in the present invention. It is obvious through comparison that after the improved webrtc ans algorithm of the present invention is processed, the noise of the low-frequency and high-frequency parts of the mic recording is effectively filtered out.

[0063] 2. AEC improved algorithm based on webrtc:

[0064] The AEC algorithm of webrtc mainly consists of three parts: the first step is delay estimation, the second step is linear filtering, and the third step is nonlinear filtering. After the second step of linear filtering, the signal collected by the microphone still retains a large amount of far-end signals. If you want to completely filter out the echo signal, the third step is the core. The nonlinear filtering suppression coefficient is hNl, overDriveSm is related to the setting of the echo cancellation level, and overDriveCurve is a monotonically increasing convex curve with a range of [1.0, 2.0]. efw is the estimated echo spectrum signal after linear filtering.

[0065] The calculation formula of the nonlinear suppression coefficient of webrtc is:

[0066] hNl[n]=hNl[n] (overDriveSm*overDriveCurve[n]) (7);

[0067] efw[n]=hNl[n]*efw[n] (8);

[0068] If hNl is too large, a large amount of echo residue will occur. Therefore, this solution improves the suppression coefficient:

[0069] hNl[n]=(hNl[n]*0.5) (overDriveSm*overDriveCurve[n]) (9);

[0070] efw[n]=hNl[n]*efw[n] (10);

[0071] This algorithm can remove echo residue from the near-end sound.

[0072] Figure 4 This is the echo cancellation effect diagram after WebRTC AEC and the improvement of this case. Through comparison, it can be clearly seen that when the far end is speaking, the near end of this case has less echo returned.

[0073] 3. Improve the amplification strategy from the main control to the speaker amplifier:

[0074] In the first step, a 1kHz sine wave audio is used as the alarm sound source, and the audio source is compressed by the AGC of WebRTC. The default parameters are target=5, gain=5, and enable=1.

[0075] The second step is to set the AGC target target = 5. Be sure to save the audio data to see if the amplitude value after AGC reaches the amplitude value corresponding to target = 5. If there is still a lot of room from the amplitude value corresponding to target = 5, the speaker will be overpowered when you change to a high-volume audio next time.

[0076] The third step is to send the AGC sound through webrtc to the main control, which will set the volume to amplify or reduce it again. At this time, the volume value should be set as wide as possible for the hardware colleagues to find the volume value when the main control voltage is 80% of the rated voltage. This volume corresponds to the maximum output volume value maxoutvolume that can be set for this board. Based on the volume value maxoutvolume, debug the speaker amplifier amplification factor to make the speaker volume output reach the maximum rated power of the speaker. At this time, the speaker amplifier factor has been determined;

[0077] Step 4: Take the volume value below maxoutvolume as the default volume value (defaultoutvolume). This value must be the maximum waveform without distortion.

[0078] The fifth step is to set the intercom level parameters according to the alarm volume level parameters. Since the sound source of the intercom is smaller than the alarm sound source, the master volume setting value can be appropriately increased based on the alarm. After determining the intercom volume parameters, you can test it with your hardware colleagues. The requirements are that the waveform is not distorted, the sound is not broken, and it is less than the rated power of the speaker. If you want the sound to be louder, the waveform may be distorted.

[0079] Compared with the prior art, the present invention improves the WebRTC ANS algorithm in the case of a microphone array with a low signal-to-noise ratio, so that the sound quality is clearer at a long pickup distance, and there is no need to sacrifice the microphone pickup distance to obtain clarity, the preview effect is more stable, and the hearing is more comfortable; through the improved WebRTC AEC algorithm, when the intercom is turned on, there will be no echo, and the volume of the speaker and the input to the mobile phone can be guaranteed to be large enough. For products with poorly designed microphone and speaker structures, and ARM boards with different chips, the delay time is easier to set, the portability is better, and the intercom effect is better; the whole set of speaker power amplifier amplification strategies can ensure that the speaker sound is not distorted and does not exceed the rated power, which reduces the R&D development cycle of hardware colleagues and does not require repeated debugging of the speaker power amplifier coefficient.

[0080] The above description is only a preferred embodiment of the present invention, but the present invention should not be limited to the contents disclosed in the embodiment and the drawings, so any equivalent or modification completed without departing from the spirit disclosed in the present invention shall fall within the scope of protection of the present invention.

Claims

1. A method for improving the intercom voice of a surveillance camera, characterized in that The specific steps include: S1. Improve the ANS algorithm of WebRTC to remove the low frequency that causes unclear near-end data and the high frequency of non-steady-state noise that easily causes howling, including: S11, using quantile estimation method to perform initial noise estimation; S12, updating noise estimation based on likelihood ratio method; S13, using Wiener filter to filter out noise; S14, multiplying the coefficient by 0 when performing Wiener filtering on some high and low frequencies to reduce the background noise; S2. Improve the AEC algorithm of WebRTC to solve the problem of occasional poor echo filtering when starting intercom, including: S21, delay estimation; S22, linear filtering; S23, nonlinear filtering, removing echo residue in the near-end sound by limiting the nonlinear filtering suppression coefficient; The calculation formula of nonlinear suppression coefficient is: hNl[n]=hNl[n] (overDriveSm*overDriveCurve[n]) ; efw[n]=hNl[n]*efw[n]; hNl is the nonlinear filtering suppression coefficient, overDriveSm is the set echo cancellation level, overDriveCurve is a monotonically increasing convex curve with a range of [1.0, 2.0], and efw is the estimated echo spectrum signal after linear filtering; If hNl is too large, a large amount of echo residue will occur, so the nonlinear filtering suppression coefficient is improved: hNl[n]=(hNl[n]*0.5) (overDriveSm*overDriveCurve[n]) ; efw[n]=hNl[n]*efw[n]; This algorithm can remove echo residue from near-end sound; S3. Improve the amplification strategy from the main control to the speaker and improve the speaker sound quality, including: S31, use 1khz sine wave audio as the alarm sound source, and use webrtc agc to limit the sound source, with default parameters target = 5, gain = 5, enable = 1; S32, set the target of agc target = 5, and save the audio data to see whether the amplitude value after agc reaches the amplitude value corresponding to target = 5; S33, sending the AGC sound through webrtc to the main control, the main control sets the volume to be amplified or reduced again, and finds the volume value when the main control voltage is 80% of the rated voltage. This volume corresponds to the maximum output volume value maxoutvolume that can be set. Based on the volume value maxoutvolume, the speaker amplifier amplification factor is debugged to make the speaker volume output reach the maximum rated power of the speaker. At this time, the speaker amplifier factor has been determined; S34, taking the volume value below maxoutvolume as the default volume value defaultoutvolume, which has the maximum waveform and is not distorted; S35. Set the intercom level parameters according to the alarm volume level parameters. Since the sound source of the intercom is smaller than the alarm sound source, the master volume setting value needs to be appropriately increased based on the alarm. After determining the intercom volume parameters, perform hardware testing. The test requirements are: the waveform is not distorted, the sound is not broken, and it is less than the rated power of the speaker. However, if the sound is increased a little bit, the waveform will be distorted. Find the critical point as the best sound quality point.

2. The method for improving the intercom voice of a surveillance camera as claimed in claim 1, characterized in that: In step S13, the Wiener filter denoising formula of ans in the Wiener filter is: epsilon=1*10 -7 ; snrprior[n]=0.98f*prevestimate[n]+(1.0f-0.98f)*snrpost[n]; Where prevsignalspectrum is the signal spectrum of the previous frame, prevgain is the gain of the previous frame, and prevnoisespectrum is the noise spectrum of the previous frame; overdrive is the value corresponding to the set denoising level; gain is the final denoising gain coefficient; n is the spectrum subscript converted from time domain to frequency domain, and its range is 0 to 64.

3. The method for improving the intercom voice of a surveillance camera as claimed in claim 2, characterized in that: In order to reduce the background noise, the calculation formula of the denoising gain coefficient is improved as follows: This algorithm removes low frequencies that obscure the near-end data.