Array signal processing method applied to teaching scene far-field pickup

Abstract

The invention belongs to the technical field of pickup audio signal processing, and discloses an array signal processing method applied to teaching scene far-field pickup. Comprising the following steps: S1, acquiring a sound field by using an integrated microphone array to obtain multiple paths of audio signals; s2, preprocessing the multiple paths of audio signals, and performing beam forming processing on the preprocessed multiple paths of audio signals to obtain multiple paths of beam audio signals; s3, performing direction-of-arrival estimation on the multiple paths of beam audio signals by using an array signal processing algorithm so as to obtain direction and distance information of a sound source pointed by each path of beam audio signal; by integrating a microphone array and an advanced array signal processing algorithm, accurate sound source positioning, efficient echo cancellation and significant environmental noise suppression of far-field pickup in a classroom scene are realized, and thus the accuracy and stability of speech recognition can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of sound pickup audio signal processing, and more specifically, to an array signal processing method applied to far-field sound pickup in teaching scenarios. Background Art

[0002] With the continuous development of audio technology, far-field sound pickup technology is increasingly used in various fields, such as conference rooms, classrooms, lecture halls and other scenes that need to capture sound from a distance. Traditional far-field sound pickup methods are often limited by factors such as microphone sensitivity, environmental noise and sound propagation attenuation, resulting in poor sound pickup effects, especially in noisy environments, where the target sound source is easily drowned out by background noise, making it difficult to extract clear sound information;

[0003] Therefore, how to solve the problems of insufficient sound source localization accuracy, severe echo interference and significant influence of environmental noise caused by far-field sound pickup in classroom scenarios to ensure the smooth progress of subsequent speech recognition has become an urgent problem to be solved.

[0004] In view of this, the present invention proposes an array signal processing method for far-field sound pickup in teaching scenarios to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: an array signal processing method applied to far-field sound pickup in a teaching scene, comprising the following steps:

[0006] S1, using an integrated microphone array to collect the sound field, thereby obtaining multi-channel audio signals;

[0007] S2, preprocessing the multi-channel audio signals, and performing beamforming processing on the preprocessed multi-channel audio signals, thereby obtaining multi-channel beamformed audio signals;

[0008] S3, using an array signal processing algorithm to estimate the direction of arrival of the multi-channel beam audio signals, and then obtain the direction and distance information of the sound source to which each beam audio signal points;

[0009] S4, using a preset wake-up engine to detect the wake-up word on the multi-channel beam audio signal, and then obtain a wake-up result;

[0010] S5. According to the wake-up result, the sound pickup function of the microphones in other directions is attenuated to a closed state, and the multi-channel beam audio signals are combined and processed to obtain an audio stream.

[0011] Preferably, the step of collecting the sound field using an integrated microphone array to obtain multi-channel audio signals includes:

[0012] Integrate multiple microphones according to a preset layout and perform spatial sampling of the sound field in the classroom;

[0013] Then, multiple audio signals are obtained simultaneously, wherein each audio signal corresponds to a different microphone and a different sound source direction.

[0014] Preferably, after obtaining the multi-channel audio signals, ADC conversion is performed on the multi-channel audio signals.

[0015] Preferably, the step of preprocessing the multi-channel audio signals and performing beamforming processing on the preprocessed multi-channel audio signals to obtain the multi-channel beamformed audio signals comprises:

[0016] De-noise and filter multi-channel audio signals;

[0017] The preprocessed multi-channel audio signals are beamformed using a beamforming algorithm to obtain multi-channel beam audio signals.

[0018] Preferably, the step of using an array signal processing algorithm to estimate the direction of arrival of multiple beam audio signals, and then obtaining the direction and distance information of the sound source to which each beam audio signal points, includes:

[0019] The array signal processing algorithm includes multiple signal classification and rotation invariant technology signal parameter estimation;

[0020] The direction of the sound source to which each audio signal received by the integrated microphone array points is estimated by using any algorithm of multiple signal classification and rotation invariant technology signal parameter estimation;

[0021] Obtaining the time point at which each microphone in the integrated microphone array receives the audio signal, thereby obtaining the time difference between each microphone in the integrated microphone array;

[0022] Using the time difference between each microphone and the speed of sound, calculate the distance difference that the sound source travels between the microphones;

[0023] The distance information of the sound source pointed by each audio signal is obtained through the direction and distance difference of the sound source pointed by each audio signal.

[0024] Preferably, the step of using a preset wake-up engine to detect the wake-up word on the multi-channel beam audio signal to obtain the wake-up result includes:

[0025] Preset wake-up engine and multiple wake-up words;

[0026] Decode the multi-channel beam audio signals and input them into the wake-up engine to perform multi-channel wake-up operations;

[0027] When the wake-up engine detects that one of the multiple beam audio signals contains a wake-up word, the beam audio signal containing the wake-up word is marked to obtain a wake-up result.

[0028] Preferably, the step of attenuating the sound pickup function of microphones in other directions to a closed state according to the wake-up result, and merging and processing the multi-channel beam audio signals to obtain the audio stream includes:

[0029] When a beam audio signal with a wake-up word is detected, the pickup function of the microphones in other directions is attenuated to the off state, and only the current microphone is monitored;

[0030] Subsequently, the multi-path beam audio signals are combined into an audio stream using signal weighting, averaging and filtering.

[0031] Preferably, in said S5, it further includes:

[0032] The microphone pointed by the beam audio signal of the detected wake-up word is used as the current microphone, and echo cancellation processing is performed on the audio signal picked up by the current microphone.

[0033] Preferably, in said S5, it further includes:

[0034] Preset the spatial domain filter and adjust the beam shape of the spatial domain filter;

[0035] Make the direction of the main lobe consistent with the direction of the current microphone and reduce the gain of the side lobe.

[0036] Preferably, the S5 further includes:

[0037] When a wake-up word is detected in one of the multiple beam audio signals, a wake-up signal is sent to the upper computer;

[0038] Then, the audio stream and the direction and distance information of each sound source in the audio stream are output.

[0039] The technical effects and advantages of the array signal processing method for far-field sound pickup in teaching scenarios of the present invention are as follows:

[0040] By integrating a microphone array and advanced array signal processing algorithms, accurate sound source positioning, efficient echo cancellation and significant ambient noise suppression of far-field sound pickup in classroom scenarios can be achieved, improving the accuracy and stability of speech recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of an array signal processing method for far-field sound pickup in teaching scenarios according to the present invention;

[0042] Figure 2It is a schematic diagram of the process of the present invention;

[0043] Figure 3 The present invention is a structural schematic diagram of an array signal processing system for far-field sound pickup in teaching scenarios. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Example 1

[0046] See also Figure 1 and Figure 2 As shown, the array signal processing method for far-field sound pickup in a teaching scene described in this embodiment includes the following steps:

[0047] S1, using an integrated microphone array to collect the sound field, thereby obtaining a multi-channel audio signal step, including:

[0048] S11. Integrate multiple microphones according to a preset layout (such as linear, planar or stereo array) and perform spatial sampling of the sound field in the classroom, wherein the integrated microphone array can capture sound waves from different directions, and each microphone receives an audio signal containing sound field information;

[0049] S12: Multiple audio signals can be obtained simultaneously, wherein each audio signal corresponds to a different microphone and a different sound source direction.

[0050] It should be noted that by spatially sampling the sound field through an integrated microphone array, multiple audio signals can be obtained simultaneously, each of which corresponds to a different microphone and a different sound source direction, greatly improving the ability to capture sound field information;

[0051] Among them, spatial sampling refers to the use of multiple microphones to collect the sound field in the classroom according to a preset layout to obtain information about the sound field at different spatial positions; in spatial sampling, spatial parameter estimation of the source signal (original sound signal / audio signal of one or more sound sources) is performed, usually corresponding to angle estimation for the far-field model and azimuth coordinate estimation for the near-field model; for the far-field model, the spatial parameter estimation of the source signal usually corresponds to angle estimation. Under far-field conditions, the distance between the sound source and the microphone array is much larger than the aperture of the array and the wavelength of the sound source, so it can be assumed that the sound wave arrives at the array in the form of a plane wave; at this time, by measuring the phase difference or time difference between the signals received by different microphones, the azimuth and elevation of the sound source relative to the microphone array can be estimated; for the near-field model, the spatial parameter estimation of the source signal corresponds to azimuth coordinate estimation. Under near-field conditions, the distance between the sound source and the microphone array is relatively close, and the sound wave arrives at the array in the form of a spherical wave.

[0052] Furthermore, after obtaining the multi-channel audio signals, ADC conversion is performed on the multi-channel audio signals.

[0053] It should be noted that the audio signal is captured by the integrated microphone array. At this time, the audio signal is in analog form and is converted into a digital audio signal for subsequent digital signal processing. ADC conversion is the process of converting the audio signal from analog to digital, which is a preparatory step in advance for the subsequent preprocessing of the audio signal.

[0054] S2, preprocessing the multi-channel audio signals, and performing beamforming processing on the preprocessed multi-channel audio signals to obtain the multi-channel beam audio signals, comprising:

[0055] S21, preprocessing the multi-channel audio signals, wherein the preprocessing includes denoising and filtering;

[0056] S22. Perform beamforming processing on the preprocessed multi-channel audio signals using a beamforming algorithm to obtain multi-channel beamformed audio signals.

[0057] Specifically, beamforming is a method of weighted summation of the received signals of each array element, which enables the system to have different responses to different spaces or directions. Therefore, beamforming is called spatial filtering. Since the integrated microphone array can perform spatial filtering, the spatial filter can be designed so that its main lobe is aligned with the target signal and the side lobe is as low as possible, thereby achieving the purpose of extracting the target signal and suppressing interference. The biggest advantage of beamforming over monophonic speech enhancement is that it can use linear processing methods to improve the signal-to-noise ratio and introduce very little signal distortion.

[0058] It should be noted that, by setting the spatial domain filter, the audio signal from a specific direction can be enhanced while suppressing interference from other directions; that is, multi-channel beam audio signals can be extracted from multi-channel audio signals to improve the quality of the audio signals; by preprocessing and beamforming the collected multi-channel audio signals, the signal strength of the target sound source is effectively enhanced, while background noise and interference are suppressed, thereby improving the signal-to-noise ratio of the signal.

[0059] S3, using an array signal processing algorithm to estimate the direction of arrival of multiple beam audio signals, and then obtaining the direction (angle, etc.) and distance information of the sound source to which each beam audio signal points, including:

[0060] S31. Use array signal processing algorithms, such as Multiple Signal Classification (MUSIC) and Estimation of Signal Parameters Using Rotational Invariant Techniques (ESPRIT), which can estimate the direction of arrival of a sound source using the audio signal received by an integrated microphone array;

[0061] S32, obtaining the time point at which each microphone in the integrated microphone array receives the audio signal, and then obtaining the time difference between each microphone in the integrated microphone array;

[0062] S33, using the time difference between each microphone and the speed of sound, calculate the distance difference of the sound source propagating between the microphones. For example, if it takes 0.0001 seconds longer for the sound to propagate from the first microphone to the second microphone, then the distance difference between the two microphones is the speed of sound multiplied by the time difference. By analyzing these distance differences, it is possible to determine from which direction the sound is coming, that is, the approximate location of the speaker / microphone / sound source.

[0063] S34. Obtain the distance information of the sound source to which each audio signal points based on the direction and distance difference of the sound source to which each audio signal points.

[0064] Specifically, direction of arrival estimation is a technology used to determine the direction in which a signal arrives at each microphone in an array. By calculating the time difference or phase difference between the arrival of beam audio signals at different microphones, the elevation angle, azimuth angle and other directional information of the signal arriving at the array can be estimated. In addition, combined with parameters such as the speed of sound and propagation time, the distance information from the target sound source to the integrated microphone array can be further estimated. In this way, the direction and distance information of the sound source to which each beam audio signal points can be obtained, providing important spatial parameters for subsequent processing.

[0065] It should be noted that by having different responses to the directions of different sound sources, the beam audio signal can be extracted and enhanced to facilitate the subsequent suppression of interference signals.

[0066] S4, using a preset wake-up engine to detect a wake-up word on a multi-channel beam audio signal, and then obtaining a wake-up result, comprising:

[0067] S41, preset wake-up engine and multiple wake-up words;

[0068] S42, decoding the multi-channel beam audio signals and inputting them into the wake-up engine to perform a multi-channel wake-up operation;

[0069] S43. When the wake-up engine detects that one of the multiple beam audio signals contains a wake-up word, the beam audio signal containing the wake-up word is marked, thereby obtaining a wake-up result to facilitate triggering subsequent processes.

[0070] It should be noted that the wake-up word can be some specific words, such as "Microphone No. 1, start recording", etc. When the wake-up engine detects the wake-up word, it will immediately trigger the algorithm to enter the waiting command state, and then execute the subsequent processes. Furthermore, the preset wake-up engine will perform wake-up word detection on the multi-channel beam audio signal, and the relevant processing flow will only be activated when the wake-up word is detected, effectively reducing system power consumption and unnecessary resource occupation.

[0071] Specifically, the wake-up engine analyzes multiple beam audio signals one by one. When it is detected that one or more beam audio signals contain a preset wake-up word, the wake-up result is triggered. The wake-up result may include one of the signal channel where the wake-up word is detected, the marked beam audio signal, the timestamp and other information, providing a basis for subsequent processing.

[0072] S5, according to the wake-up result, attenuating the sound pickup function of the microphones in other directions to a closed state, and merging and processing the multi-channel beam audio signals to obtain an audio stream, including:

[0073] S51. When a beam audio signal with a wake-up word is detected, the pickup function of the microphones in other directions is attenuated to a closed state, and only the current microphone is monitored, thereby reducing interference and noise;

[0074] S52: Subsequently, the multi-path beam audio signals are combined into an audio stream by using signal weighting, averaging and filtering.

[0075] It should be noted that after detecting the wake-up word, the microphone pickup function in other directions is attenuated to the off state, and only the current microphone is monitored, which improves the system processing efficiency and accuracy. By merging and processing multi-channel beam audio signals and using signal weighting, averaging and filtering technologies, the merged audio stream is ensured to have high quality and low distortion.

[0076] Furthermore, in S5, it also includes:

[0077] The microphone pointed by the beam audio signal of the wake-up word detected is used as the current microphone, and the audio signal picked up by the current microphone is echo-cancelled, which can eliminate echo and sidelobe interference and improve the clarity of the audio stream.

[0078] Furthermore, in S5, it also includes:

[0079] Preset the spatial domain filter and adjust the beam shape of the spatial domain filter;

[0080] Make the direction of the main lobe consistent with the direction of the current microphone and reduce the gain of the side lobe

[0081] Furthermore, in S5, it also includes:

[0082] When a wake-up word is detected in one of the multiple beam audio signals, a wake-up signal is sent to the host computer to notify the host computer to perform the speech recognition stage;

[0083] Subsequently, the audio stream and the direction and distance information (position) of each sound source in the audio stream are output.

[0084] In this embodiment, a group of integrated microphone arrays are used to collect the sound field in the classroom, so that the spatial information of the sound source can be recorded. On the one hand, spatial domain filtering is used for signal extraction and enhancement, and on the other hand, the direction of arrival of the sound source can be estimated; the integrated microphone array decodes multiple audio signals and sends them to the wake-up engine. When a wake-up word is detected in a certain audio channel, the pickup function of the microphone in other directions is attenuated to the off state, and only the currently awakened microphone is monitored; at the same time, a wake-up signal is sent to the host computer, and the merged audio stream is thrown out for use in speech recognition, so that the sound source positioning in the classroom can be obtained, and the angle and distance of the sound source can be calculated; and the sound played by itself is removed for echo cancellation, and the noise can be reduced, filtered and the sound of microphones in other directions can be suppressed, thereby improving the accuracy and stability of speech recognition.

[0085] Example 2

[0086] See also Figure 3 As shown, the array signal processing system for far-field sound pickup in a teaching scenario described in this embodiment includes:

[0087] The audio signal acquisition module uses an integrated microphone array to collect the sound field and obtain multi-channel audio signals;

[0088] An audio signal processing module pre-processes the multi-channel audio signals and performs beamforming processing on the pre-processed multi-channel audio signals to obtain multi-channel beamformed audio signals;

[0089] The information acquisition module uses an array signal processing algorithm to estimate the direction of arrival of multiple beam audio signals, and then obtains the direction and distance information of the sound source to which each beam audio signal points;

[0090] The wake-up detection module uses a preset wake-up engine to detect the wake-up word of the multi-channel beam audio signal, and then obtains the wake-up result;

[0091] The audio signal merging module attenuates the sound pickup function of microphones in other directions to a closed state according to the wake-up result, and merges the multi-channel beam audio signals to obtain an audio stream.

[0092] In this embodiment, accurate sound source positioning, efficient echo cancellation, and significant ambient noise suppression of far-field sound pickup in classroom scenarios can be achieved, thereby improving the accuracy and stability of speech recognition.

[0093] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0094] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only one, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0095] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0096] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An array signal processing method for far-field sound pickup in teaching scenarios, characterized in that: The following steps are involved: S1, using an integrated microphone array to collect the sound field, thereby obtaining multi-channel audio signals; S2, preprocessing the multi-channel audio signals, and performing beamforming processing on the preprocessed multi-channel audio signals, thereby obtaining multi-channel beamformed audio signals; S3, using an array signal processing algorithm to estimate the direction of arrival of the multi-channel beam audio signals, and then obtain the direction and distance information of the sound source to which each beam audio signal points; S4, using a preset wake-up engine to detect the wake-up word on the multi-channel beam audio signal, and then obtain a wake-up result; S5. According to the wake-up result, the sound pickup function of the microphones in other directions is attenuated to a closed state, and the multi-channel beam audio signals are combined and processed to obtain an audio stream.

2. According to claim 1, an array signal processing method for far-field sound pickup in teaching scenarios is characterized in that: The step of collecting the sound field using the integrated microphone array to obtain multi-channel audio signals includes: Integrate multiple microphones according to a preset layout and perform spatial sampling of the sound field in the classroom; Then, multiple audio signals are obtained simultaneously, wherein each audio signal corresponds to a different microphone and a different sound source direction.

3. The array signal processing method for far-field sound pickup in teaching scenarios according to claim 2 is characterized in that: After obtaining the multi-channel audio signals, ADC conversion is performed on the multi-channel audio signals.

4. The array signal processing method for far-field sound pickup in teaching scenarios according to claim 1 is characterized in that: The step of preprocessing the multi-channel audio signals and performing beamforming processing on the preprocessed multi-channel audio signals to obtain the multi-channel beam audio signals includes: De-noise and filter multi-channel audio signals; The preprocessed multi-channel audio signals are beamformed using a beamforming algorithm to obtain multi-channel beam audio signals.

5. The array signal processing method for far-field sound pickup in teaching scenarios according to claim 1 is characterized in that: The step of using an array signal processing algorithm to estimate the direction of arrival of multiple beam audio signals, and then obtaining the direction and distance information of the sound source to which each beam audio signal points, includes: The array signal processing algorithm includes multiple signal classification and rotation invariant technology signal parameter estimation; The direction of the sound source to which each audio signal received by the integrated microphone array points is estimated by using any algorithm of multiple signal classification and rotation invariant technology signal parameter estimation; Obtaining the time point at which each microphone in the integrated microphone array receives the audio signal, thereby obtaining the time difference between each microphone in the integrated microphone array; Using the time difference between each microphone and the speed of sound, calculate the distance difference that the sound source travels between the microphones; The distance information of the sound source pointed by each audio signal is obtained through the direction and distance difference of the sound source pointed by each audio signal.

6. The array signal processing method for far-field sound pickup in teaching scenarios according to claim 1 is characterized in that: The step of using a preset wake-up engine to detect the wake-up word on the multi-channel beam audio signal to obtain the wake-up result includes: Preset wake-up engine and multiple wake-up words; Decode the multi-channel beam audio signals and input them into the wake-up engine to perform multi-channel wake-up operations; When the wake-up engine detects that one of the multiple beam audio signals contains a wake-up word, the beam audio signal containing the wake-up word is marked to obtain a wake-up result.

7. The array signal processing method for far-field sound pickup in teaching scenarios according to claim 6 is characterized in that: The step of attenuating the sound pickup function of microphones in other directions to a closed state according to the wake-up result, and merging and processing the multi-channel beam audio signals to obtain the audio stream includes: When a beam audio signal with a wake-up word is detected, the pickup function of the microphones in other directions is attenuated to the off state, and only the current microphone is monitored; Subsequently, the multi-path beam audio signals are combined into an audio stream using signal weighting, averaging and filtering.

8. The array signal processing method for far-field sound pickup in teaching scenarios according to claim 7 is characterized in that: In the S5, it also includes: The microphone pointed by the beam audio signal of the detected wake-up word is used as the current microphone, and echo cancellation processing is performed on the audio signal picked up by the current microphone.

9. The array signal processing method for far-field sound pickup in teaching scenarios according to claim 8, characterized in that: In the S5, it also includes: Preset the spatial domain filter and adjust the beam shape of the spatial domain filter; Make the direction of the main lobe consistent with the direction of the current microphone and reduce the gain of the side lobe.

10. The array signal processing method for far-field sound pickup in teaching scenarios according to claim 1, characterized in that: In the S5, it also includes: When a wake-up word is detected in one of the multiple beam audio signals, a wake-up signal is sent to the upper computer; Then, the audio stream and the direction and distance information of each sound source in the audio stream are output.

Images