A listening system

Abstract

The application discloses a monitoring system, a monitoring microphone is used for collecting the sound outside the shell and transmitting the collected sound outside the shell to an acoustic signal collecting unit through the acoustic signal collecting unit; the acoustic signal collecting unit is used for eliminating the correlation between the sound outside the shell input by the monitoring microphone and the monitoring output signal and suppressing the howling in real time; a communication microphone is used for collecting the sound of a speaker inside the shell and transmitting the collected sound of the speaker to an acoustic echo canceling unit through the acoustic signal collecting unit; the acoustic echo canceling unit is used for eliminating the playing sound transmitted by a loudspeaker; and the loudspeaker is used for outputting the sound signal processed by the acoustic howling suppressing unit and the acoustic echo canceling unit. The monitoring system provided by the application realizes the monitoring of the sound field outside the shell in the shell, such as a helmet or a vehicle, and improves the safety.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of real-time acoustic processing, in particular to a listening system. BACKGROUND

[0002] The conventional helmet system does not have a listening function. After wearing the helmet, it is difficult to hear the details of the external sound, causing communication barriers with the outside world and related safety problems. The conventional helmet also has the problem of poor telephone communication experience. On the one hand, the remote speaker cannot be heard clearly. On the other hand, the acoustic echo problem caused by the helmet environment causes the remote user to hear the echo. These problems also exist in the automotive scenario. On the one hand, in the case of closed vehicle doors and windows, especially in the case of playing music in the vehicle, it is difficult for people inside the vehicle to hear the details of the sound outside the vehicle, which can cause a variety of safety problems. On the other hand, when in a communication state, it often causes the remote user to hear the echo problem. SUMMARY

[0003] Therefore, the present application provides a listening system to solve the problem of poor communication experience in the helmet or vehicle and the difficulty in hearing the sound outside the helmet or vehicle, which causes safety hazards.

[0004] In order to achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0005] A listening system, comprising an acoustic I / O device and a sound signal processing device, the acoustic I / O device comprising a listening microphone, a communication microphone and a loudspeaker, the listening microphone being arranged outside a shell, the communication microphone, the sound signal processing device and the loudspeaker being arranged inside the shell;

[0006] The sound signal processing device comprises an acoustic signal acquisition unit, an acoustic howling suppression unit and an acoustic echo cancellation unit, the listening microphone is used to acquire the sound outside the shell and transmit the acquired sound outside the shell to the acoustic signal acquisition unit and the acoustic howling suppression unit;

[0007] The acoustic howling suppression unit is used to eliminate the correlation between the sound outside the shell input by the listening microphone and the listening output signal and to suppress howling in real time;

[0008] The communication microphone is used to acquire the sound of a speaker inside the shell and transmit the acquired sound of the speaker to the acoustic signal acquisition unit and the acoustic echo cancellation unit;

[0009] The acoustic echo cancellation unit is used to eliminate the playing sound transmitted by the loudspeaker;

[0010] The loudspeaker is used to output the sound signal processed by the acoustic howling suppression unit and the acoustic echo cancellation unit.

[0011] Further, the acoustic howling suppression unit is used to eliminate the correlation between the shell external sound input by the monitoring microphone and the monitoring output signal by delay output or linear prediction de-correlation.

[0012] Further, when de-correlated by linear prediction, a linear prediction of 2 to 20 orders is adopted.

[0013] Further, when suppressing howling, the acoustic howling suppression unit adopts LMS or NLMS adaptive algorithm.

[0014] Further, the communication microphone is used to collect the speaker's sound inside the shell, and the collected speaker's sound is transmitted to the acoustic echo cancellation unit through the acoustic signal collection unit.

[0015] After the speaker's sound collected by the communication microphone is transmitted to the beam forming unit to obtain a spatially enhanced signal and then transmitted to the acoustic echo cancellation unit.

[0016] Further, the acoustic echo cancellation unit includes a linear adaptive filter, which is used to eliminate the playing sound transmitted by the loudspeaker and perform different switching according to different working states.

[0017] Further, the working states include a double-talking state and a single-talking state, in the double-talking state, the adaptive filter stops coefficient updating or reduces the updating speed; in the single-talking state, the adaptive filter updates in real time and performs echo suppression in real time.

[0018] Further, the sound signal processing device further includes an acoustic signal real-time noise reduction processing unit, which is used to receive the sound signal transmitted by the acoustic howling suppression unit and the acoustic echo cancellation unit and filter out the noise in the sound signal.

[0019] Further, the acoustic signal real-time noise reduction processing unit adopts a spectral subtraction algorithm or a noise reduction algorithm based on a deep neural network to filter out the noise in the sound signal.

[0020] Further, the sound signal processing device further includes an acoustic signal real-time playback unit, after the sound collected by the monitoring microphone and the communication microphone passes through the acoustic signal real-time playback unit, the sound is mixed and played through the loudspeaker, and the proportion of mixing or the gain of the two sounds can be independently controlled.

[0021] Compared with the prior art, the present application has at least the following beneficial effects:

[0022] The application provides a monitoring system, a monitoring microphone is arranged outside a shell, a communication microphone, a sound signal processing device and a loudspeaker are arranged inside the shell; the monitoring microphone is used for collecting sound outside the shell and transmitting the collected sound outside the shell to an acoustic howling suppression unit through an acoustic signal collecting unit; the acoustic howling suppression unit is used for eliminating the correlation between the sound outside the shell input by the monitoring microphone and a monitoring output signal and suppressing howling in real time; the communication microphone is used for collecting the sound of a speaker inside the shell and transmitting the collected sound of the speaker to an acoustic echo cancellation unit through the acoustic signal collecting unit; the acoustic echo cancellation unit is used for eliminating the playing sound transmitted by the loudspeaker; and the loudspeaker is used for outputting the sound signal processed by the acoustic howling suppression unit and the acoustic echo cancellation unit. The monitoring system provided by the application realizes the monitoring of the sound field outside the shell in the shell of a helmet, a vehicle or the like, and improves the safety. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more directly illustrate the prior art and the application, several exemplary drawings are given below. It should be understood that the specific shapes, structures shown in the drawings should not be regarded as the limiting conditions for implementing the application; for example, based on the technical concept disclosed by the application and the exemplary drawings, those skilled in the art can easily make routine adjustments or further optimization on the increase / decrease / assignment of certain units (components), the specific shape, the positional relationship, the connection mode, the size ratio relationship and the like.

[0024] Figure 1 A structural block diagram of the monitoring system provided by the application is shown in the figure;

[0025] Figure 2 A structural schematic diagram of the monitoring system provided by the application applied to a helmet is shown in the figure;

[0026] Figure 3 A processing flowchart of the acoustic howling suppression unit provided by the application is shown in the figure.

[0027] Explanation of reference signs:

[0028] 1, monitoring microphone; 2, communication microphone; 3, loudspeaker; 4, sound signal processing device; 41, acoustic signal collecting unit; 42, acoustic echo cancellation unit; 43, acoustic howling suppression unit; 44, acoustic signal real-time noise reduction processing unit; 45, acoustic signal real-time gain equalization unit; 46, acoustic signal real-time playback unit; 5, shell. DETAILED DESCRIPTION

[0029] The application will be further described in detail below by combining the drawings and specific embodiments.

[0030] In the description of the present application: unless otherwise specified, the meaning of "a plurality of" is two or more. The terms "first", "second", "third" and the like in the present application are intended to distinguish the objects referred to, and do not have a special meaning in the technical connotation aspect (for example, it should not be understood as emphasizing importance or order, etc.). The expressions such as "include", "contain", "have" and the like also mean "not limited to" (certain units, components, materials, steps, etc.).

[0031] The terms such as "upper", "lower", "left", "right", "intermediate" and the like referred to in the present application are generally used for the convenience of intuitive understanding with reference to the drawings, and are not an absolute limitation on the positional relationship in the actual product. Changes in these relative positional relationships are also considered within the scope of the present application without departing from the technical concept disclosed in the present application.

[0032] Please refer to Figure 1 and Figure 2 The present application provides a listening system, which comprises an acoustic I / O device and a sound signal processing device, the acoustic I / O device comprises a listening microphone 1, a communication microphone 2 and a loudspeaker 3, the listening microphone 1 is arranged outside the shell 5, the communication microphone 2, the acoustic signal processing device 4 and the loudspeaker 3 are arranged inside the shell 5, it should be noted that the helmet and the automobile shell in the present application are collectively referred to as the shell 5, and the listening microphone 1, the communication microphone 2 and the loudspeaker 3 can be arranged one or more, and the arrangement of the listening microphone 1 needs to consider the wind shield and the waterproof treatment and the like.

[0033] The listening microphone 1 is used for real-time acquisition of external sound and transmission of the acquired sound to the acoustic howling suppression unit 43.

[0034] Specifically, the listening system is actually a kind of loudspeaker, when the loudspeaker 3 volume is larger, or the shell of the shell 5 is thinner, or the window is opened, etc. It may cause howling, therefore the listening microphone 1 needs to do howling suppression processing after the acoustic signal acquisition unit 41 acquires the sound, and the processing algorithm includes frequency shift, phase shift and adaptive howling suppression algorithm.

[0035] The adaptive howling suppression algorithm is the core component of the acoustic howling suppression unit 43, and cannot directly apply the adaptive algorithm such as LMS, NLMS and the like similar to echo cancellation algorithm, because the input and output signals are highly correlated.

[0036] Please refer to Figure 3 , the howling suppression needs to do decorrelation algorithm first, delay output, linear prediction decorrelation and the like can be used to eliminate the correlation between the input signal of the listening microphone 1 and the listening output signal.

[0037] The present application preferably uses linear prediction processing of 2nd to 20th order, specifically,

[0038] The decorrelation signal is calculated according to formula (1):

[0039]

[0040] Wherein, X(n) is the input signal of the monitoring microphone 1, the prediction residual signal u(n) is obtained through P-order linear prediction, P is generally 2-20 orders, G is an amplitude gain constant, generally G=1.

[0041] The adaptive filter part is the core of the whole howling suppression, the adaptive filter length M is generally consistent with the actual acoustic path length, the adaptive filter residual signal e(n) of the decorrelation signal and the adaptive filter feedback signal is obtained according to formula (2)

[0042]

[0043] Wherein, f i Is the adaptive filter coefficient.

[0044] The robustness of the adaptive filter core of the application is derived from the innovative robust signal (long-time prediction residual average power and long-time filter residual average power) estimation,

[0045] Wherein, the long-time prediction residual average power is:

[0046] Sp(n)=bl*Sp(n)+(1-bl)*u(n)*u(n)

[0047] The long-time filter residual average power is:

[0048] Se(n)=bl*Se(n)+(1-bl)*e(n)*e(n)

[0049] Wherein, b1 is a memory factor, generally b1>0.9.

[0050] The filter update step size μ is obtained according to the long-time prediction residual average power and the long-time filter residual average power

[0051]

[0052] In formula (3), ε is a small number to prevent division by zero, from formula (3), when the filter residual power relative to the prediction residual power becomes large, it indicates that the adaptive filter mismatch is aggravated, at this time, according to formula (3), the update step size is adaptively increased, so that the adaptive filter f i Quickly follow the actual change. At the same time, the long-time prediction residual average power signal is cached:

[0053] S bp =[S p (n)Sp (n-1)…S p (n-M)]

[0054] Adaptive filter update:

[0055]

[0056] where ε is a small number to prevent division by zero.

[0057] Finally, the signal with the correlation eliminated is used for adaptive algorithms such as LMS, NLMS, etc. to achieve real-time howling suppression effect.

[0058] When it is not desired to amplify all the sound outside the shell 5, it is necessary to filter out the noise in it through the acoustic signal real-time noise reduction processing unit 44. The real-time noise reduction algorithm can use the spectral subtraction algorithm, or the noise reduction algorithm based on deep neural network. Whether it is spectral subtraction or deep neural network noise reduction, it is to model the background noise and the useful signal to be preserved differently, so as to distinguish the noise and the useful signal in the time-frequency space, and suppress the noise component with the noise reduction factor. Finally, the signal of the monitoring playback can be amplified in different volumes according to the needs, or different amplification can be performed on different frequency bands, and finally the acoustic signal real-time playback unit 46 is used for playback. It should be noted that the order of the acoustic howling suppression unit 43, the acoustic signal real-time noise reduction processing unit 44, and the acoustic signal real-time gain equalization unit 45 in the present application can be reversed.

[0059] The communication microphone 2 is used to collect the voice of the speaker in the vehicle in real time, and then transmit to the acoustic echo cancellation unit 42. If there are multiple communication microphones 2, a spatially enhanced signal can be obtained by beamforming unit first, and then the echo cancellation and noise reduction processing is performed. When the computing resources are sufficient, the echo cancellation processing can be performed on each communication microphone 2, and then the spatial beam enhancement is performed. The acoustic echo cancellation unit 42 has a linear adaptive filter part and a nonlinear processing part. The adaptive filter part mainly has LMS, NLMS and other adaptive filters. The adaptive filter part can maximize the elimination of the playing sound transmitted by the loudspeaker 3 from the communication microphone 2 signal. The adaptive filter can be switched according to different working states. When the person in the shell 5 and the remote person in the loudspeaker speak at the same time, it is called double speaking state. When the double speaking detector detects the double speaking state, the coefficient update of the adaptive filter is stopped or the update speed is greatly reduced, and the nonlinear echo suppression unit is closed. When there is only one person speaking in the shell 5 or remotely, the adaptive filter can be updated in real time, and the nonlinear echo suppression unit is opened. That is, if the person in the shell 5 speaks and the remote person does not speak, the signal from the remote end is completely suppressed or greatly suppressed. If the remote person is speaking and the person in the shell is not speaking, the uplink signal is completely suppressed or greatly suppressed. The noise reduction and gain control system of the communication acoustic processing system can be shared with the monitoring system.

[0060] In addition, the monitoring system can work in parallel with the communication system. At this time, the acoustic signal real-time playback unit 46 needs to mix and play two kinds of signals, that is, the sound outside the shell 5 and the voice of the remote person can be mixed and played. The mixing ratio or the gain of the two kinds of sound can be independently controlled.

[0061] The monitoring system provided in the application realizes the monitoring of the sound field outside the shell in the shell of the helmet, vehicle and the like, and improves the safety. The communication system in the shell and the sound field monitoring outside the shell can work at the same time, and the volume of each can be independently adjusted.

[0062] The technical features of the above embodiments can be combined in any way (as long as the combination of the technical features does not exist contradiction). In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described. The embodiments not explicitly written are also considered as the scope of the present application.

[0063] The application is described in detail above by general description and specific embodiments. It should be understood that based on the technical concept of the application, some conventional adjustments or further innovations can be made to the specific embodiments; as long as the technical concept of the application is not deviated, the technical solutions obtained by the conventional adjustments or further innovations also fall within the protection scope of the claims of the application.

Claims

1. A listening system, characterized in that The application relates to an acoustic I / O device and a sound signal processing device, the acoustic I / O device comprising a listening microphone, a communication microphone and a speaker, the listening microphone being arranged outside a housing, the communication microphone, the sound signal processing device and the speaker being arranged inside the housing; The sound signal processing device comprises an acoustic signal acquisition unit, an acoustic howling suppression unit and an acoustic echo cancellation unit, the listening microphone being used to acquire the sound outside the housing and transmit the acquired sound outside the housing to the acoustic signal acquisition unit and the acoustic howling suppression unit; The acoustic howling suppression unit is used to eliminate the correlation between the sound outside the housing input by the listening microphone and the listening output signal and suppress howling in real time; The communication microphone is used to acquire the sound of a speaker inside the housing and transmit the acquired sound of the speaker to the acoustic signal acquisition unit and the acoustic echo cancellation unit; The acoustic echo cancellation unit is used to eliminate the playing sound transmitted by the speaker; The speaker is used to output the sound signal processed by the acoustic howling suppression unit and the acoustic echo cancellation unit.

2. The listening system of claim 1, wherein, When eliminating the correlation between the sound outside the housing input by the listening microphone and the listening output signal, the acoustic howling suppression unit is used to output by delay or linear prediction decorrelation.

3. The listening system of claim 2, wherein, When the linear prediction decorrelation is used, the linear prediction of 2 to 20 orders is adopted.

4. The listening system of claim 1, wherein, When the acoustic howling suppression unit suppresses howling, the LMS or NLMS adaptive algorithm is adopted.

5. The listening system of claim 1, wherein, Before the communication microphone is used to acquire the sound of a speaker inside the housing and transmit the acquired sound of the speaker to the acoustic signal acquisition unit and the acoustic echo cancellation unit, the following step is further included: The sound of the speaker acquired by the communication microphone is transmitted to a beam forming unit to obtain a spatially enhanced signal and then transmitted to the acoustic echo cancellation unit.

6. The listening system of claim 1, wherein, The acoustic echo cancellation unit comprises a linear adaptive filter, the linear adaptive filter being used to eliminate the playing sound transmitted by the speaker and switched in different manners according to different working states.

7. The listening system of claim 6, wherein, The working states include a double-talking state and a single-talking state, in the double-talking state, the adaptive filter stops coefficient updating or reduces the updating speed, in the single-talking state, the adaptive filter updates in real time and performs echo suppression in real time.

8. The listening system of claim 1, wherein, The sound signal processing device further comprises an acoustic signal real-time noise reduction processing unit, the acoustic signal real-time noise reduction processing unit being used to receive the sound signal transmitted by the acoustic howling suppression unit and the acoustic echo cancellation unit and filter out the noise in the sound signal.

9. The listening system of claim 8, wherein, The acoustic signal real-time noise reduction processing unit adopts a spectral subtraction algorithm or a noise reduction algorithm based on a deep neural network to filter out the noise in the sound signal.

10. The listening system of claim 1, wherein, The sound signal processing device further comprises an acoustic signal real-time playback unit, the sound acquired by the listening microphone and the communication microphone is mixed and played by the speaker through the acoustic signal real-time playback unit, and the proportion of mixing or the gain of the two kinds of sound can be independently controlled.

Images