Voice processing method, system, storage medium and terminal voice device

Abstract

A speech processing method, system, storage medium and terminal speech device, the speech processing method comprises: obtaining first audio data collected by an internal microphone and multiple second audio data collected by multiple external microphones respectively; converting the first audio data and multiple second audio data into the frequency domain to obtain a first amplitude-frequency characteristic corresponding to the first audio data and a second amplitude-frequency characteristic corresponding to the multiple second audio data; generating an equalization filter according to the first amplitude-frequency characteristic and the multiple second amplitude-frequency characteristics; performing beamforming processing on the multiple second audio data to obtain a beamformed audio signal; and inputting the beamformed audio signal into the equalization filter. The present invention can effectively reduce the noise interference caused by the direction of the undesired sound source, and can realize the dynamic adjustment of the equalization filter according to the change of the amplitude-frequency characteristic of the audio data collected by the microphone, thereby realizing effective adaptation to environmental changes.

Description

Technical Field

[0001] The present invention belongs to the field of signal processing, and in particular relates to a voice processing method, system, storage medium and terminal voice equipment. Background Art

[0002] Different rooms or different locations in a room have different acoustic characteristics. When using a conference speaker in an ordinary conference room, part of the near-end speaker's voice is directly picked up by the speaker's microphone, and the other part is picked up by the microphone after being reflected by the wall, desktop, etc., resulting in the change of the spectrum characteristics of the sound finally collected by the microphone compared with the speaker's original voice. For example, speaking in the corner of the room, due to the strong reflection of the wall, the low frequency of the sound will be over-enhanced and the sound will be muddy. Speaking in an empty hall, due to the weak reflection, the sound will sound thin. In cases like these, the microphone needs to be equalized to improve the naturalness and authenticity of the picked up sound.

[0003] The traditional processing method is usually to debug a set of filters in a specific room to filter the sound signals picked up by the microphone. Once the filter is debugged, it is fixed and will not change with the changes in the environment where the microphone is located. This method is simple to implement and occupies less computing resources. However, it cannot adapt to changes in different environments and it is difficult to eliminate interference caused by undesired sound source directions. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a speech processing method, which solves the problem that the traditional microphone equalization method cannot adapt to environmental changes and is difficult to eliminate interference caused by undesired sound source directions. The present invention also proposes a speech processing system, a terminal speech device and a computer-readable storage medium for executing the above-mentioned speech processing method.

[0005] The speech processing method according to the first embodiment of the present invention is applied to a terminal speech device, wherein the terminal speech device is provided with an internal microphone inside and a plurality of external microphones outside.

[0006] The speech processing method comprises the following steps:

[0007] Acquire first audio data collected by the internal microphone and multiple second audio data collected by multiple external microphones respectively;

[0008] Convert the first audio data and the plurality of second audio data into a frequency domain to obtain a first amplitude-frequency characteristic corresponding to the first audio data and a second amplitude-frequency characteristic corresponding to the plurality of second audio data;

[0009] generating an equalization filter according to the first amplitude-frequency characteristic and a plurality of the second amplitude-frequency characteristics;

[0010] Performing beamforming processing on the plurality of second audio data to obtain beamformed audio signals;

[0011] The beamformed audio signal is input to the equalization filter.

[0012] According to the speech processing method of the embodiment of the present invention, at least the following technical effects are achieved: microphones are arranged inside and outside the terminal voice device, providing a hardware basis for executing the speech processing method of the embodiment of the present invention; and then the amplitude-frequency characteristic curve of the audio data collected by multiple external microphones and internal microphones can be used to generate an equalization filter; and the use of multiple external microphones to collect audio data to assist in generating the equalization filter can further improve the noise reduction effect of the equalization filter. At the same time, by beamforming the multiple second audio data, the noise interference caused by the direction of the undesired sound source can be eliminated, so that the beamforming audio signal finally formed contains only the required audio as much as possible. After the beamforming audio signal is input into the equalization filter, clearer audio data can be obtained. Compared with the traditional processing method, the speech processing method of the embodiment of the present invention can effectively reduce the noise interference caused by the direction of the undesired sound source, and at the same time, the dynamic adjustment of the equalization filter can be realized according to the change of the amplitude-frequency characteristic of the audio data collected by the microphone, thereby realizing effective adaptation to environmental changes.

[0013] According to some embodiments of the present invention, generating an equalization filter according to the first amplitude-frequency characteristic and a plurality of the second amplitude-frequency characteristics comprises the following steps:

[0014] Dividing the first amplitude-frequency characteristic and the plurality of the second amplitude-frequency characteristics into a plurality of real-time sub-bands;

[0015] Obtaining amplitude mean values ​​of a plurality of the second amplitude-frequency characteristics in each of the real-time sub-bands;

[0016] Subtracting the amplitude mean value corresponding to each of the real-time sub-bands from the corresponding amplitude value of the first amplitude-frequency characteristic in sequence to obtain a real-time sub-band amplitude difference value group;

[0017] Determine the original amplitude-frequency curve of the filter according to the real-time sub-band amplitude difference group and the standard sub-band amplitude difference group, wherein the standard sub-band amplitude difference group is obtained by the terminal voice device according to the swept frequency tone;

[0018] The equalization filter is determined according to the original amplitude-frequency curve of the filter.

[0019] According to some embodiments of the present invention, the standard sub-band amplitude difference value group is obtained by the following steps:

[0020] Acquire standard internal audio data collected by the internal microphone, and acquire standard external audio data collected by multiple external microphones, wherein the standard internal audio data and the multiple standard external audio data are both obtained based on the swept frequency tone;

[0021] Converting the standard internal audio data and the plurality of standard external audio data into a frequency domain to obtain a standard internal amplitude-frequency characteristic corresponding to the standard internal audio data and a standard external amplitude-frequency characteristic corresponding to the plurality of standard external audio data;

[0022] Dividing the standard internal amplitude-frequency characteristic and the plurality of standard external amplitude-frequency characteristics into a plurality of corresponding standard sub-bands, wherein the plurality of standard sub-bands correspond one-to-one to the plurality of real-time sub-bands;

[0023] Obtaining an amplitude mean value of a plurality of the standard external amplitude-frequency characteristics in each of the standard sub-bands;

[0024] Subtracting the amplitude mean corresponding to each standard sub-band from the amplitude of the corresponding standard internal amplitude-frequency characteristic in turn to obtain a standard sub-band amplitude difference group.

[0025] According to some embodiments of the present invention, determining the original amplitude-frequency curve of the filter according to the real-time sub-band amplitude difference group and the standard sub-band amplitude difference group comprises the following steps:

[0026] Subtracting each difference in the real-time sub-band amplitude difference group from each difference in the standard sub-band amplitude difference group in turn to obtain a sub-band difference sequence;

[0027] The sub-band difference sequence is fitted based on the frequency domain to obtain the original amplitude-frequency curve of the filter.

[0028] According to some embodiments of the present invention, determining the equalization filter according to the original amplitude-frequency curve of the filter comprises the following steps:

[0029] Selecting at least one peak of the original amplitude-frequency curve of the filter, and determining a set of corresponding gain, center frequency, and quality factor according to each peak;

[0030] Generate an intermediate equalization filter according to each group of gain, center frequency, and quality factor;

[0031] synthesizing the equalization filter according to the intermediate equalization filter;

[0032] Wherein, each of the wave peaks is selected according to the amplitude, and each of the intermediate equalization filters is used to improve the filtering effect of the equalization filter.

[0033] According to some embodiments of the present invention, generating an intermediate equalization filter according to each group of the gain, center frequency, and quality factor comprises the following steps:

[0034] The gain, center frequency, and quality factor are adjusted multiple times according to multiple groups of adjustment parameters, and a target amplitude-frequency characteristic curve corresponding to a transition filter generated according to the gain, center frequency, and quality factor after each adjustment is obtained, wherein the multiple groups of adjustment parameters are input into the terminal voice device by a user;

[0035] The total difference between each target amplitude-frequency characteristic curve and the original amplitude-frequency curve of the filter is obtained in sequence, and the intermediate equalization filter is generated according to the gain, center frequency, and quality factor corresponding to the target amplitude-frequency characteristic curve with the smallest total difference.

[0036] According to some embodiments of the present invention, performing beamforming processing on the plurality of second audio data comprises the following steps:

[0037] Acquire time data and frequency data of a plurality of the second audio data;

[0038] Determine the original sound source direction according to the time data and the frequency data;

[0039] Calculating the signal power corresponding to each of the original sound source directions, and determining the expected sound source direction according to the magnitude of each of the signal powers;

[0040] The outputs of the two external microphones are constrained based on a linear constrained minimum variance criterion so that signals not in the direction of the desired sound source are filtered out to obtain the beamforming audio signal.

[0041] The speech processing system according to the second aspect of the present invention is applied to a terminal speech device, wherein the terminal speech device is provided with an internal microphone inside and a plurality of external microphones outside.

[0042] The speech processing system comprises:

[0043] A data acquisition unit, used to acquire first audio data collected by the internal microphone and a plurality of second audio data collected by a plurality of external microphones respectively;

[0044] an amplitude-frequency conversion unit, configured to convert the first audio data and the plurality of second audio data into a frequency domain, so as to obtain a first amplitude-frequency characteristic corresponding to the first audio data and a second amplitude-frequency characteristic corresponding to the plurality of second audio data;

[0045] a filter generating unit, configured to generate an equalizing filter according to the first amplitude-frequency characteristic and a plurality of the second amplitude-frequency characteristics;

[0046] A beam constraining unit, configured to perform beamforming processing on the plurality of second audio data to obtain a beamformed audio signal;

[0047] A data output unit is used to input the beamforming audio signal into the equalization filter.

[0048] According to the speech processing system of the embodiment of the present invention, at least the following technical effects are achieved: microphones are arranged inside and outside the terminal speech device; and then the amplitude-frequency characteristic curves of the audio data collected by multiple external microphones and the internal microphone can be used to generate an equalization filter; and the use of multiple external microphones to collect audio data to assist in generating the equalization filter can further improve the noise reduction effect of the equalization filter. At the same time, by beamforming the multiple second audio data, the noise interference caused by the direction of the undesired sound source can be eliminated, so that the beamforming audio signal finally formed contains only the required audio as much as possible. After the beamforming audio signal is input into the equalization filter, clearer audio data can be obtained. The speech processing system of the embodiment of the present invention can effectively reduce the noise interference caused by the direction of the undesired sound source, and at the same time can realize the dynamic adjustment of the equalization filter according to the change of the amplitude-frequency characteristic of the audio data collected by the microphone, thereby realizing effective adaptation to environmental changes.

[0049] According to the terminal voice device of the embodiment of the third aspect of the invention, it includes:

[0050] case;

[0051] A speaker is disposed on the housing, wherein the speaker and the housing form a sealed chamber;

[0052] an internal microphone, disposed in the sealed chamber;

[0053] A plurality of external microphones are disposed on the outer surface of the housing;

[0054] A processing unit is connected to the speaker, the internal microphone and the plurality of external microphones respectively, and the processing unit is used to execute the above-mentioned speech processing method.

[0055] The terminal voice device according to the embodiment of the present invention has at least the following beneficial effects: microphones are arranged both inside and outside the terminal voice device, thereby providing an environmental basis for the processing unit to execute the above-mentioned voice processing method, so that the terminal voice device of the embodiment of the present invention can effectively reduce the noise interference caused by the direction of undesired sound sources, and at the same time can realize dynamic adjustment of the equalization filter according to the changes in the amplitude-frequency characteristics of the audio data collected by the microphone, thereby realizing effective adaptation to environmental changes.

[0056] According to the computer-readable storage medium of the embodiment of the fourth aspect of the invention, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned speech processing method.

[0057] The computer-readable storage medium according to the embodiment of the present invention has at least the following beneficial effects: the storage medium can facilitate the storage and transfer of computer-executable instructions.

[0058] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The above or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0060] Figure 1 is a flow chart of a speech processing method according to an embodiment of the present invention;

[0061] Figure 2 is a schematic diagram of an expectation of beamforming processing according to an embodiment of the present invention;

[0062] Figure 3 is a system block diagram of a speech processing system according to an embodiment of the present invention;

[0063] Figure 4 It is a schematic diagram of the structure of a terminal voice device according to an embodiment of the present invention.

[0064] Reference numerals:

[0065] Housing 100 , speaker 200 , internal microphone 300 , external microphone 400 . DETAILED DESCRIPTION

[0066] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0067] In the description of the present invention, if there is a description of first, second, third, fourth, etc., it is only for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the sequence of the indicated technical features.

[0068] In the description of the present invention, unless otherwise clearly defined, words such as setting and connecting should be understood in a broad sense, and technicians in the technical field can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0069] refer to Figure 1 4, a speech processing method according to an embodiment of the first aspect of the present invention is described. The speech processing method according to the embodiment of the present invention is applied to a terminal speech device, wherein an internal microphone 300 is arranged inside the terminal speech device, and a plurality of external microphones 400 are arranged outside the terminal speech device;

[0070] The speech processing method comprises the following steps:

[0071] Acquire first audio data collected by the internal microphone 300 and multiple second audio data collected by multiple external microphones 400;

[0072] Convert the first audio data and the plurality of second audio data into a frequency domain to obtain a first amplitude-frequency characteristic corresponding to the first audio data and a second amplitude-frequency characteristic corresponding to the plurality of second audio data;

[0073] generating an equalization filter according to the first amplitude-frequency characteristic and a plurality of second amplitude-frequency characteristics;

[0074] Performing beamforming processing on the plurality of second audio data to obtain a beamformed audio signal;

[0075] The beamformed audio signal is input to an equalization filter.

[0076] refer to Figures 1 to 4 , multiple external microphones 400 are arranged outside the terminal voice device and on the same horizontal plane, and the internal microphone 300 is arranged inside the terminal voice device. When the external sound source makes a sound, both the internal microphone 300 and the external microphone 400 can collect audio data.

[0077] The collected audio data is converted from the time domain to the frequency domain, so that a first amplitude-frequency characteristic corresponding to the first audio data and a second amplitude-frequency characteristic corresponding to the plurality of second audio data can be obtained. After obtaining the amplitude-frequency characteristic curve, the difference between the second amplitude-frequency characteristic and the first amplitude-frequency characteristic can be used to generate an equalization filter. It should be noted here that usually the external sound source does not frequently change its position after making a sound, especially in a conference room environment, and the position of the sound source may not change during the entire meeting. Therefore, the first amplitude-frequency characteristic and the second amplitude-frequency characteristic are unlikely to change suddenly over time, so there is no need to frequently use the collected audio data to obtain the amplitude-frequency characteristic curve, and there is no need to frequently generate or adjust the equalization filter. Usually, the equalization filter can be generated or adjusted once at intervals, for example, once every five minutes.

[0078] The use of an equalizing filter can effectively filter out noise, but it cannot filter out interference from the direction of undesired sound sources. When using a microphone to collect sound, the microphone is usually placed as close to the sound source as possible. In most cases, it can be basically determined that the signal source with the largest sound amplitude is the desired sound source. Based on this, after obtaining multiple second audio data, the multiple second audio data can be beamformed, so that the direction of the desired sound source can be quickly and effectively determined, and the sound from the direction of the undesired sound source can be filtered out to obtain a beamformed audio signal that contains as much sound from the direction of the desired sound source as possible. After this beamformed audio signal is input into the equalizing filter for equalization and noise filtering, the audio data of the ideal desired sound source can be obtained.

[0079] According to the speech processing method of the embodiment of the present invention, microphones are set inside and outside the terminal voice device, providing a hardware basis for executing the speech processing method of the embodiment of the present invention; and then the amplitude-frequency characteristic curve of the audio data collected by multiple external microphones 400 and the internal microphone 300 can be used to generate an equalization filter; and the use of multiple external microphones 400 to collect audio data to assist in generating an equalization filter can further improve the noise reduction effect of the equalization filter. At the same time, by beamforming the multiple second audio data, the noise interference caused by the direction of the undesired sound source can be eliminated, so that the final beamforming audio signal contains only the required audio as much as possible. After the beamforming audio signal is input into the equalization filter, clearer audio data can be obtained. Compared with the traditional processing method, the speech processing method of the embodiment of the present invention can effectively reduce the noise interference caused by the direction of the undesired sound source, and at the same time, the dynamic adjustment of the equalization filter can be realized according to the change of the amplitude-frequency characteristic of the audio data collected by the microphone, thereby realizing effective adaptation to environmental changes.

[0080] In some embodiments of the present invention, generating an equalization filter according to a first amplitude-frequency characteristic and a plurality of second amplitude-frequency characteristics comprises the following steps:

[0081] The first amplitude-frequency characteristic and the plurality of second amplitude-frequency characteristics are correspondingly divided into a plurality of real-time sub-bands;

[0082] Obtaining the amplitude mean value of the plurality of second amplitude-frequency characteristics in each real-time sub-band;

[0083] Subtracting the amplitude mean value corresponding to each real-time sub-band from the amplitude value of the corresponding first amplitude-frequency characteristic in turn to obtain a real-time sub-band amplitude difference value group;

[0084] Determine the original amplitude-frequency curve of the filter according to the real-time sub-band amplitude difference group and the standard sub-band amplitude difference group, where the standard sub-band amplitude difference group is obtained by the terminal voice device according to the swept frequency tone;

[0085] The equalization filter is determined according to the original amplitude-frequency curve of the filter.

[0086] The first amplitude-frequency characteristic and the multiple second amplitude-frequency characteristics are divided into multiple real-time sub-bands. For example, the frequency range of the sound collection frequency that both the internal microphone 300 and the external microphone 400 collect is 20HZ to 20000HZ, and the frequency domain range of the first amplitude-frequency characteristic and the multiple second amplitude-frequency characteristics can also be determined to be 20HZ to 20000HZ. Then, a real-time sub-band can be divided at every fixed frequency interval. The specific division size can be set manually based on experience, or it can be divided into a sub-band at every 1 / N times the frequency interval, for example, a sub-band can be divided at every 1 / 3 or 1 / 6 times the frequency interval. After the real-time sub-band division is completed, the multiple second amplitude-frequency characteristics will be averaged and assigned in each divided real-time sub-band, that is, the amplitude mean, and then the amplitude mean of the real-time sub-band is used to make a difference with the amplitude of the first amplitude-frequency characteristic corresponding to the real-time sub-band to obtain the real-time sub-band amplitude difference. After the amplitude mean and the amplitude of the corresponding first amplitude-frequency characteristic in each real-time sub-band are subtracted in turn, a group of real-time sub-band amplitude differences can be obtained, which can be recorded as a real-time sub-band amplitude difference group. It should be noted here that the use of a second amplitude-frequency characteristic can also make a difference between the amplitude and the amplitude of the first amplitude-frequency characteristic in each real-time sub-band, but compared with the method of averaging multiple second amplitude-frequency characteristics, the stability of the generated equalization filter is worse.

[0087] After obtaining the real-time sub-band amplitude difference group, each difference in the real-time sub-band amplitude difference group can be made different from each difference in the standard sub-band amplitude difference group one by one, and the original amplitude-frequency curve of the filter can be determined using this group of difference results, so that the post-adjustment equalization filter can be generated based on the original amplitude-frequency curve of the filter. It should be noted that the standard sub-band amplitude difference group is obtained based on the swept frequency tone. For example, if it is necessary to collect sounds from 20HZ to 20000HZ, the swept frequency tone will play sounds from 20HZ to 20000HZ in sequence, so that the standard sub-band amplitude difference group generated in this way can better assist the real-time sub-band amplitude difference group in obtaining the original amplitude-frequency curve of the filter. In some embodiments of the present invention, the swept frequency tone can also be replaced by white noise or sounds of other preset frequency bands, but the effect is poorer than that of the swept frequency tone.

[0088] In some embodiments of the present invention, the standard sub-band amplitude difference set is obtained by the following steps:

[0089] Acquire standard internal audio data collected by the internal microphone 300, and acquire standard external audio data collected by multiple external microphones 400, wherein the standard internal audio data and multiple standard external audio data are both obtained based on the swept frequency sound;

[0090] Converting the standard internal audio data and the plurality of standard external audio data into the frequency domain to obtain a standard internal amplitude-frequency characteristic corresponding to the standard internal audio data and a standard external amplitude-frequency characteristic corresponding to the plurality of standard external audio data;

[0091] The standard internal amplitude-frequency characteristic and the plurality of standard external amplitude-frequency characteristics are divided into a plurality of corresponding standard sub-bands, and the plurality of standard sub-bands correspond to the plurality of real-time sub-bands one by one;

[0092] Obtaining the amplitude mean value of multiple standard external amplitude-frequency characteristics in each standard sub-band;

[0093] Subtract the amplitude mean value corresponding to each standard sub-band from the amplitude value of the corresponding standard internal amplitude-frequency characteristic in turn to obtain a standard sub-band amplitude difference value group.

[0094] When a terminal voice device (such as a conference audio system with a microphone) is provided with a speaker 200 inside, the terminal voice device can be placed in a soundproof room, and then the speaker 200 is used to play the sweeping tone, and then the standard internal audio data collected by the internal microphone 300 and the standard external audio data collected by multiple external microphones 400 are sufficient. It should be noted that different types of terminal voice devices can choose the method of obtaining standard external audio data and standard internal audio data according to actual conditions, and directly using the built-in speaker 200 to obtain cannot be used as a limitation of the present invention.

[0095] After obtaining the standard external audio data and the standard internal audio data, a standard sub-band amplitude difference group can be generated in a similar manner to the generation of the real-time sub-band amplitude difference group. First, the standard internal audio data and multiple standard external audio data are converted to the frequency domain to obtain the standard internal amplitude-frequency characteristics corresponding to the standard internal audio data and the standard external amplitude-frequency characteristics corresponding to the multiple standard external audio data; then the standard internal amplitude-frequency characteristics and the multiple standard external amplitude-frequency characteristics are divided into standard sub-bands. It should be noted here that the standard sub-band division needs to correspond to the real-time sub-band division, that is, which frequency bands are divided into the real-time sub-band, and the standard sub-band can only be divided into multiple frequency bands. After the standard sub-band division is completed, the amplitudes of the multiple standard external amplitude-frequency characteristics in each standard sub-band are averaged to obtain the amplitude mean of the standard external amplitude-frequency characteristics in each standard sub-band, and the amplitude mean in each standard sub-band is used to make a difference with the amplitude of the standard internal amplitude-frequency characteristics corresponding to the standard sub-band one by one to obtain multiple standard sub-band amplitude differences, and the multiple standard sub-band amplitude differences are recorded as a standard sub-band amplitude difference group.

[0096] In some embodiments of the present invention, determining the original amplitude-frequency curve of the filter according to the real-time sub-band amplitude difference group and the standard sub-band amplitude difference group includes the following steps:

[0097] Subtracting each difference in the real-time sub-band amplitude difference value group from each difference in the standard sub-band amplitude difference value group in turn to obtain a sub-band difference value sequence;

[0098] The sub-band difference sequence is fitted based on the frequency domain to obtain the original amplitude-frequency curve of the filter.

[0099] By subtracting the difference in the real-time sub-band amplitude difference group from the difference in the standard sub-band amplitude difference group one by one, a sub-band difference sequence can be obtained. The sub-band difference sequence represents the difference between the sound in the real-time environment and the sound in the anechoic room environment. After that, the sub-band difference sequence is fitted in the frequency domain to obtain the original amplitude-frequency curve of the filter. After the filter is generated using the original amplitude-frequency curve of the filter, it can be used to eliminate noise interference in the environment.

[0100] In some embodiments of the present invention, determining an equalization filter according to an original amplitude-frequency curve of the filter comprises the following steps:

[0101] Selecting at least one peak of the original amplitude-frequency curve of the filter, and determining a set of corresponding gain, center frequency, and quality factor according to each peak;

[0102] Generate an intermediate equalization filter according to each group of gain, center frequency, and quality factor;

[0103] synthesizing an equalizing filter based on the intermediate equalizing filter;

[0104] Each peak is selected according to its amplitude, and each intermediate equalization filter is used to improve the filtering effect of the equalization filter.

[0105] The equalization filter is essentially a filter group composed of at least one intermediate equalization filter. After obtaining the original amplitude-frequency curve, the higher the peak, the greater the amplitude. Therefore, it is necessary to give priority to the design of the intermediate equalization filter. Therefore, when actually designing the intermediate equalization filter, a set of gain, center frequency, and quality factor will be determined according to the maximum peak of the original amplitude-frequency curve of the filter. The gain can be understood as the height of the peak, the center frequency is the horizontal coordinate frequency corresponding to the peak, and the quality factor can use the preset value. Determining a set of gain, center frequency, and quality factor can generate the first intermediate equalization filter. After generating the first intermediate equalization filter, the peak with the second amplitude will be selected, and the intermediate equalization filter will be generated in the same way as above. Then, the filtering effect of the filter group composed of the two intermediate equalization filters will be evaluated. If the noise filtering effect is better than that of only the first intermediate equalization filter, the intermediate equalization filter will be retained, otherwise the intermediate equalization filter will be abandoned. After that, the peaks can be processed one by one according to the amplitude from large to small until the filter group formed by multiple intermediate equalization filters meets the expected filtering requirements. At this time, the filter group is recorded as an equalization filter.

[0106] It should be noted that in some embodiments, the step of judging whether the filter group meets the expected filtering requirements is: obtaining the target amplitude-frequency characteristic curve of the filter group, selecting some points (e.g., equally spaced selection) on the target amplitude-frequency characteristic curve and the original amplitude-frequency curve of the filter to make a difference, obtaining multiple differences, and then calculating the total difference of the multiple differences, and judging that the total difference is less than the preset filtering threshold requirement, if the total difference is less than the filtering threshold, it is determined that the filter group meets the expected filtering requirements. It should be noted that the filtering threshold can be set by the user according to the applicable objects faced by different products.

[0107] In some embodiments of the present invention, generating an intermediate equalization filter according to each set of gain, center frequency, and quality factor comprises the following steps:

[0108] The gain, center frequency, and quality factor are adjusted multiple times according to multiple groups of adjustment parameters, and a target amplitude-frequency characteristic curve corresponding to a transition filter generated according to the gain, center frequency, and quality factor after each adjustment is obtained, wherein the multiple groups of adjustment parameters are input into the terminal voice device by the user;

[0109] The total difference between each target amplitude-frequency characteristic curve and the original amplitude-frequency curve of the filter is obtained in turn, and an intermediate equalization filter is generated according to the gain, center frequency, and quality factor corresponding to the target amplitude-frequency characteristic curve with the smallest total difference.

[0110] The intermediate equalization filter is not obtained directly. After obtaining the gain, center frequency, and quality factor, a transition filter will be generated according to the gain, center frequency, and quality factor. Then, the transition filter will be appropriately adjusted by fine-tuning the gain, center frequency, and quality factor. Each fine-tuning of the gain, center frequency, and quality factor will be adjusted according to a set of adjustment parameters. Then, the target amplitude-frequency characteristic curve corresponding to the adjusted transition filter is obtained, and points are taken corresponding to the target amplitude-frequency characteristic curve and the original amplitude-frequency curve of the filter (for example, equidistant selection). After the difference between these points is obtained, the total difference can be obtained using these differences. The smaller the total difference, the better the filtering effect, so that a set of transition filters generated by the gain, center frequency, and quality factor with the smallest total difference among multiple adjustment parameters can be selected as the intermediate equalization filter. It should be noted that the number of adjustment parameters is not unlimited, and can be selected according to actual needs, for example, 10 groups or 20 groups can be selected.

[0111] In some embodiments of the present invention, beamforming processing is performed on a plurality of second audio data, comprising the following steps:

[0112] Acquire time data and frequency data of a plurality of second audio data;

[0113] Determine the original sound source direction based on the time data and the frequency data;

[0114] Calculate the signal power corresponding to each original sound source direction, and determine the expected sound source direction based on the magnitude of each signal power;

[0115] The outputs of the two external microphones 400 are constrained based on a linear constrained minimum variance criterion so that the signals in the direction of undesired sound sources are filtered out to obtain a beamformed audio signal.

[0116] The external microphones 400 are installed at different locations, so when receiving the same sound signal from the sound source, there will be a time difference. By analyzing the time data of multiple second audio data, the time difference when multiple microphones obtain the same audio can be determined, and then the direction of the sound source can be determined based on the time difference. The frequency data can reflect the power of the sound obtained. Usually, there are differences in power between different sound sources. Therefore, the time difference and frequency data can be used to determine the direction of all original sound sources in the horizontal plane where the multiple external microphones 400 are located.

[0117] Normally, the volume of noise will be lower than the volume of the expected sound source that needs to be collected. Therefore, multiple original sound source directions can be sorted according to power size, and a threshold can be set to determine the original sound source direction that exceeds this threshold as the expected sound source direction, otherwise it will be treated as noise. The specific number of expected sound source directions needs to be determined based on the actual usage scenario. For example, in a meeting scenario, when the leader speaks, only one expected sound source direction needs to be determined. In an interview scenario, multiple expected sound source directions need to be obtained; of course, there is no limit on the number, and all original sound source directions that exceed the threshold can be determined as the expected sound source direction. In some embodiments, the power size is graded, and one or several original sound source directions of the highest level are determined as the expected sound source direction.

[0118] After the sound source direction is determined, the outputs of the two external microphones 400 can be constrained according to the linear constrained minimum variance criterion. The constraint process is mainly to adjust the amplitude or phase weighting coefficient of the microphone output signal to appropriately increase the output power of the beamformer, thereby retaining more sound from the desired sound source direction and filtering out the sound from other undesired sound source directions. For example: for a single desired sound source direction, adjusting the amplitude or phase weighting coefficient can adaptively minimize the output power of the beamformer, which is equivalent to minimizing the noise power of the undesired sound source direction in the microphone output signal, thereby enhancing the desired sound source direction signal. For multiple desired sound source directions, such as when speakers from two different directions speak at the same time, by adjusting the amplitude or phase weighting coefficient, the output power of the beamformer is appropriately increased, thereby retaining more sound from the two directions, avoiding excessive elimination of one of them, and filtering out noise from other directions.

[0119] In some embodiments of the present invention, before constraining the outputs of two external microphones 400 based on the linear constrained minimum variance criterion, delay compensation is performed on the second audio data collected by the external microphones 400, so that the second audio data collected by multiple external microphones 400 remain consistent to facilitate subsequent constraint processing.

[0120] The speech processing system according to the second embodiment of the present invention is applied to a terminal speech device, wherein an internal microphone 300 is arranged inside the terminal speech device, and a plurality of external microphones 400 are arranged outside the terminal speech device;

[0121] The speech processing system comprises a data acquisition unit, an amplitude-frequency conversion unit, a filter generation unit, a beam constraint unit and a data output unit.

[0122] A data acquisition unit, used to acquire first audio data collected by the internal microphone 300 and a plurality of second audio data collected by the plurality of external microphones 400;

[0123] an amplitude-frequency conversion unit, configured to convert the first audio data and the plurality of second audio data into a frequency domain, so as to obtain a first amplitude-frequency characteristic corresponding to the first audio data and a second amplitude-frequency characteristic corresponding to the plurality of second audio data;

[0124] A filter generating unit, configured to generate an equalizing filter according to the first amplitude-frequency characteristic and a plurality of second amplitude-frequency characteristics;

[0125] A beam constraining unit, configured to perform beamforming processing on the plurality of second audio data to obtain a beamformed audio signal;

[0126] The data output unit is used to input the beamformed audio signal into the equalization filter.

[0127] refer to Figures 1 to 4 , multiple external microphones 400 are arranged outside the terminal voice device and on the same horizontal plane, and the internal microphone 300 is arranged inside the terminal voice device. When the external sound source makes a sound, both the internal microphone 300 and the external microphone 400 can collect audio data, and these audio data will be acquired by the data acquisition unit.

[0128] The amplitude-frequency conversion unit converts the collected audio data from the time domain to the frequency domain, so as to obtain a first amplitude-frequency characteristic corresponding to the first audio data and a second amplitude-frequency characteristic corresponding to the plurality of second audio data. After obtaining the amplitude-frequency characteristic curve, the filter generation unit can generate an equalization filter using the difference between the second amplitude-frequency characteristic and the first amplitude-frequency characteristic. It should be noted here that usually the external sound source does not frequently change its position after making a sound, especially in a conference room environment, and the position of the sound source may not change during the entire meeting. Therefore, the first amplitude-frequency characteristic and the second amplitude-frequency characteristic are unlikely to change suddenly over time, so there is no need to frequently use the collected audio data to obtain the amplitude-frequency characteristic curve, and there is no need to frequently generate or adjust the equalization filter. Usually, the equalization filter can be generated or adjusted once at intervals, for example, once every five minutes.

[0129] The use of an equalizing filter can effectively filter out noise, but it cannot filter out interference from the direction of undesired sound sources. When using a microphone to collect sound, the microphone is usually placed as close to the sound source as possible. In most cases, it can be basically determined that the signal source with the largest sound amplitude is the desired sound source. Therefore, after obtaining multiple second audio data, the beam constraint unit can perform beamforming processing on the multiple second audio data, so that the direction of the desired sound source can be quickly and effectively determined, and the sound from the direction of the undesired sound source can be filtered out to obtain a beamformed audio signal that contains as much sound from the direction of the desired sound source as possible. After the data output unit inputs this beamformed audio signal into the equalizing filter for equalization and noise filtering, the ideal audio data of the desired sound source can be obtained.

[0130] According to the voice processing system of the embodiment of the present invention, microphones are set inside and outside the terminal voice device; and then the amplitude-frequency characteristic curve of the audio data collected by multiple external microphones 400 and the internal microphone 300 can be used to generate an equalization filter; and the use of multiple external microphones 400 to collect audio data to assist in generating the equalization filter can further improve the noise reduction effect of the equalization filter. At the same time, by beamforming the multiple second audio data, the noise interference caused by the direction of the undesired sound source can be eliminated, so that the final beamforming audio signal contains only the required audio as much as possible. After the beamforming audio signal is input into the equalization filter, clearer audio data can be obtained. The voice processing system of the embodiment of the present invention can effectively reduce the noise interference caused by the direction of the undesired sound source, and at the same time can realize the dynamic adjustment of the equalization filter according to the change of the amplitude-frequency characteristic of the audio data collected by the microphone, thereby realizing effective adaptation to environmental changes.

[0131] In some embodiments of the present invention, the filter generation unit includes a sub-division module, a mean value calculation module, a difference determination module, an amplitude-frequency curve generation unit, and a filter generation module.

[0132] A sub-division module, used for dividing the first amplitude-frequency characteristic and the plurality of second amplitude-frequency characteristics into a plurality of real-time sub-bands;

[0133] A mean value calculation module, used for obtaining the amplitude mean value of the plurality of second amplitude-frequency characteristics in each real-time sub-band;

[0134] A difference determination module, used for sequentially performing a subtraction operation on the amplitude mean corresponding to each real-time sub-band and the amplitude of the corresponding first amplitude-frequency characteristic to obtain a real-time sub-band amplitude difference group;

[0135] An amplitude-frequency curve generating unit, used to determine the original amplitude-frequency curve of the filter according to the real-time sub-band amplitude difference group and the standard sub-band amplitude difference group, wherein the standard sub-band amplitude difference group is obtained by the terminal voice device according to the swept frequency tone;

[0136] The filter generation module is used to determine the equalization filter according to the original amplitude-frequency curve of the filter.

[0137] The sub-division module divides the first amplitude-frequency characteristic and the multiple second amplitude-frequency characteristics into multiple real-time sub-bands. For example, the frequency region of the sound collection frequency that both the internal microphone 300 and the external microphone 400 will collect is 20HZ to 20000HZ, so the frequency domain range of the first amplitude-frequency characteristic and the multiple second amplitude-frequency characteristics can also be determined to be 20HZ to 20000HZ. Then, a real-time sub-band can be divided at every fixed frequency interval. The specific division size can be set manually based on experience, or it can be divided into a sub-band at intervals of 1 / N times the frequency, for example, a sub-band is divided at intervals of 1 / 3 or 1 / 6 times the frequency. After the real-time sub-band division is completed, the mean calculation module will average and assign multiple second amplitude-frequency characteristics in each divided real-time sub-band, that is, the amplitude mean, and then the difference determination module uses the amplitude mean of the real-time sub-band to make a difference with the amplitude of the first amplitude-frequency characteristic corresponding to the real-time sub-band to obtain the real-time sub-band amplitude difference. After the amplitude mean and the amplitude of the corresponding first amplitude-frequency characteristic in each real-time sub-band are subtracted in turn, a group of real-time sub-band amplitude differences can be obtained, which can be recorded as a real-time sub-band amplitude difference group. It should be noted here that the use of a second amplitude-frequency characteristic can also make a difference between the amplitude and the amplitude of the first amplitude-frequency characteristic in each real-time sub-band, but compared with the method of averaging multiple second amplitude-frequency characteristics, the stability of the generated equalization filter is worse.

[0138] After obtaining the real-time sub-band amplitude difference group, the amplitude-frequency curve generation unit can make a difference between each difference in the real-time sub-band amplitude difference group and each difference in the standard sub-band amplitude difference group one by one, and use this group of difference results to determine the original amplitude-frequency curve of the filter, so that the filter generation module can generate and adjust the equalization filter according to the original amplitude-frequency curve of the filter. It should be noted that the standard sub-band amplitude difference group is obtained based on the swept frequency tone. For example, if it is necessary to collect 20HZ to 20000HZ sound, the swept frequency tone will play 20HZ to 20000HZ sound in sequence, so that the standard sub-band amplitude difference group generated in this way can better assist the real-time sub-band amplitude difference group to obtain the original amplitude-frequency curve of the filter. In some embodiments of the present invention, the swept frequency tone can also be replaced by white noise or other sounds of preset frequency bands, but the effect is poorer than the swept frequency tone.

[0139] In some embodiments of the present invention, the beam constraining unit includes an information acquisition module, an original direction determination module, a desired direction determination module, and a beam forming module.

[0140] An information acquisition module, used to acquire time data and frequency data of a plurality of second audio data;

[0141] An original direction determination module, used to determine the original sound source direction according to time data and frequency data;

[0142] The expected direction determination module is used to calculate the signal power corresponding to each original sound source direction and determine the expected sound source direction according to the magnitude of each signal power;

[0143] The beamforming module constrains the outputs of the two external microphones 400 based on a linear constrained minimum variance criterion so as to filter out signals from undesired sound source directions and obtain a beamformed audio signal.

[0144] The external microphones 400 are installed at different positions, so there will be a time difference when receiving the same sound signal from the sound source. By analyzing the time data of multiple second audio data, the time difference when multiple microphones obtain the same audio can be determined, and then the direction of the sound source can be determined based on the time difference. The frequency data can reflect the power of the sound obtained. Usually, there are differences in power between different sound sources. Therefore, the original direction determination module can determine the direction of all original sound sources in the horizontal plane where the multiple external microphones 400 are located using the time difference and frequency data.

[0145] Normally, the volume of noise will be lower than the volume of the expected sound source that needs to be collected. Therefore, multiple original sound source directions can be sorted according to power size, and a threshold can be set. The expected direction determination module can determine the original sound source direction that exceeds this threshold as the expected sound source direction. The specific number of expected sound source directions needs to be determined based on the actual usage scenario. For example, in a meeting scenario, when the leader speaks, only one expected sound source direction needs to be determined. In an interview scenario, multiple expected sound source directions need to be obtained; of course, there is no limit on the number, and all original sound source directions that exceed the threshold can be determined as the expected sound source direction. In some embodiments, the power size will be graded, and one or several original sound source directions of the highest level will be determined as the expected sound source direction.

[0146] After determining the direction of the sound source, the beamforming module can constrain the outputs of the two external microphones 400 according to the linear constrained minimum variance criterion. The constraint process is mainly to adjust the amplitude or phase weighting coefficient of the microphone output signal to appropriately increase the output power of the beamformer, thereby retaining more sound from the desired sound source direction and filtering out the sound from other undesired sound source directions. For example: for a single desired sound source direction, adjusting the amplitude or phase weighting coefficient can adaptively minimize the output power of the beamformer, which is equivalent to minimizing the noise power of the undesired sound source direction in the microphone output signal, thereby enhancing the desired sound source direction signal. For multiple desired sound source directions, such as when speakers from two different directions speak at the same time, by adjusting the amplitude or phase weighting coefficient, the output power of the beamformer is appropriately increased, thereby retaining more sound from the two directions, avoiding excessive elimination of one of them, and filtering out noise from other directions.

[0147] The terminal voice device according to the embodiment of the third aspect of the invention includes a housing 100, a speaker 200, an internal microphone 300, a plurality of external microphones 400, and a processing unit.

[0148] The speaker 200 is disposed on the housing 100, and the speaker 200 and the housing 100 form a sealed chamber;

[0149] An internal microphone 300 is disposed in the sealed chamber;

[0150] A plurality of external microphones 400 are disposed on the outer surface of the housing 100;

[0151] The processing unit is connected to the speaker 200, the internal microphone 300 and the plurality of external microphones 400 respectively, and is used to execute the above-mentioned speech processing method.

[0152] refer to Figure 4 When multiple external microphones 400 are arranged on the outer surface of the shell 100, they need to be kept on the same horizontal plane as much as possible, and when the number of external microphones 400 exceeds two, they should be arranged at equal angles and equal intervals as much as possible, so as to improve the accuracy of the processing unit when executing the voice processing method as described above. The speaker 200 can be used as a playback device, and the speaker 200 can also be used when obtaining a standard sub-band amplitude difference group. In addition, in order to ensure the effect of the equalization filter as much as possible, the speaker 200 and the shell 100 will form a sealed chamber, and the internal microphone 300 will be placed in the sealed chamber, so as to effectively reduce interference. It should be noted that the internal microphone 300 and the multiple external microphones 400 can all use omnidirectional microphones, or directional microphones can be used in combination with the specific structure of the shell 100.

[0153] According to the terminal voice device of the embodiment of the present invention, microphones are arranged inside and outside the terminal voice device, thereby providing an environmental basis for the processing unit to execute the above-mentioned voice processing method, so that the terminal voice device of the embodiment of the present invention can effectively reduce the noise interference caused by the direction of undesired sound sources, and at the same time can realize dynamic adjustment of the equalization filter according to the changes in the amplitude-frequency characteristics of the audio data collected by the microphone, thereby realizing effective adaptation to environmental changes.

[0154] In some embodiments of the present invention, the internal microphone 300 may be replaced by a distance sensor or an acceleration sensor. According to the vibration law of the distance sensor or the acceleration sensor, the amplitude and frequency may also be used to assist the external microphone 400 in completing the setting of the equalization filter.

[0155] According to the computer-readable storage medium of the embodiment of the fourth aspect of the invention, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned speech processing method.

[0156] According to the computer-readable storage medium of the embodiment of the present invention, the storage and transfer of computer-executable instructions can be facilitated through the storage medium.

[0157] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0158] Although the embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, the present invention is not limited to the above embodiments. Those skilled in the art will understand that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and purpose of the present invention. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. A voice processing method, applied to a terminal voice device, characterized in that: An internal microphone is arranged inside the terminal voice device, and a plurality of external microphones are arranged outside the terminal voice device; The speech processing method comprises the following steps: Acquire first audio data collected by the internal microphone and multiple second audio data collected by multiple external microphones respectively; Convert the first audio data and the plurality of second audio data into a frequency domain to obtain a first amplitude-frequency characteristic corresponding to the first audio data and a second amplitude-frequency characteristic corresponding to the plurality of second audio data; generating an equalization filter according to the first amplitude-frequency characteristic and a plurality of the second amplitude-frequency characteristics; Performing beamforming processing on the plurality of second audio data to obtain beamformed audio signals; inputting the beamformed audio signal into the equalization filter; The method of generating an equalization filter according to the first amplitude-frequency characteristic and a plurality of the second amplitude-frequency characteristics comprises the following steps: Dividing the first amplitude-frequency characteristic and the plurality of the second amplitude-frequency characteristics into a plurality of real-time sub-bands; Obtaining amplitude mean values ​​of a plurality of the second amplitude-frequency characteristics in each of the real-time sub-bands; Subtracting the amplitude mean value corresponding to each of the real-time sub-bands from the corresponding amplitude value of the first amplitude-frequency characteristic in sequence to obtain a real-time sub-band amplitude difference value group; Determine the original amplitude-frequency curve of the filter according to the real-time sub-band amplitude difference group and the standard sub-band amplitude difference group, wherein the standard sub-band amplitude difference group is obtained by the terminal voice device according to the swept frequency tone; The equalization filter is determined according to the original amplitude-frequency curve of the filter.

2. The speech processing method according to claim 1, characterized in that: The standard sub-band amplitude difference group is obtained by the following steps: Acquire standard internal audio data collected by the internal microphone, and acquire standard external audio data collected by multiple external microphones, wherein the standard internal audio data and the multiple standard external audio data are both obtained based on the swept frequency tone; Converting the standard internal audio data and the plurality of standard external audio data into a frequency domain to obtain a standard internal amplitude-frequency characteristic corresponding to the standard internal audio data and a standard external amplitude-frequency characteristic corresponding to the plurality of standard external audio data; Dividing the standard internal amplitude-frequency characteristic and the plurality of standard external amplitude-frequency characteristics into a plurality of corresponding standard sub-bands, wherein the plurality of standard sub-bands correspond one-to-one to the plurality of real-time sub-bands; Obtaining an amplitude mean value of a plurality of the standard external amplitude-frequency characteristics in each of the standard sub-bands; Subtracting the amplitude mean corresponding to each standard sub-band from the amplitude of the corresponding standard internal amplitude-frequency characteristic in turn to obtain a standard sub-band amplitude difference group.

3. The speech processing method according to claim 1 or 2, characterized in that: The method of determining the original amplitude-frequency curve of the filter according to the real-time sub-band amplitude difference group and the standard sub-band amplitude difference group comprises the following steps: Subtracting each difference in the real-time sub-band amplitude difference group from each difference in the standard sub-band amplitude difference group in turn to obtain a sub-band difference sequence; The sub-band difference sequence is fitted based on the frequency domain to obtain the original amplitude-frequency curve of the filter.

4. The speech processing method according to claim 1, characterized in that: Determining the equalization filter according to the original amplitude-frequency curve of the filter comprises the following steps: Selecting at least one peak of the original amplitude-frequency curve of the filter, and determining a set of corresponding gain, center frequency, and quality factor according to each peak; Generate an intermediate equalization filter according to each group of gain, center frequency, and quality factor; synthesizing the equalization filter according to the intermediate equalization filter; Wherein, each of the wave peaks is selected according to the amplitude, and each of the intermediate equalization filters is used to improve the filtering effect of the equalization filter.

5. The speech processing method according to claim 4, characterized in that: Generating an intermediate equalization filter according to each group of gain, center frequency, and quality factor includes the following steps: The gain, center frequency, and quality factor are adjusted multiple times according to multiple groups of adjustment parameters, and a target amplitude-frequency characteristic curve corresponding to a transition filter generated according to the gain, center frequency, and quality factor after each adjustment is obtained, wherein the multiple groups of adjustment parameters are input into the terminal voice device by a user; The total difference between each target amplitude-frequency characteristic curve and the original amplitude-frequency curve of the filter is obtained in sequence, and the intermediate equalization filter is generated according to the gain, center frequency, and quality factor corresponding to the target amplitude-frequency characteristic curve with the smallest total difference.

6. The speech processing method according to claim 1, characterized in that: The performing beamforming processing on the plurality of the second audio data comprises the following steps: Acquire time data and frequency data of a plurality of the second audio data; Determine the original sound source direction according to the time data and the frequency data; Calculating the signal power corresponding to each of the original sound source directions, and determining the expected sound source direction according to the magnitude of each of the signal powers; The outputs of the plurality of external microphones are constrained based on a linear constrained minimum variance criterion so that signals not in the direction of the desired sound source are filtered out to obtain the beamforming audio signal.

7. A voice processing system, applied to a terminal voice device, characterized in that: An internal microphone is arranged inside the terminal voice device, and a plurality of external microphones are arranged outside the terminal voice device; The speech processing system comprises: A data acquisition unit, used to acquire first audio data collected by the internal microphone and a plurality of second audio data collected by a plurality of external microphones respectively; an amplitude-frequency conversion unit, configured to convert the first audio data and the plurality of second audio data into a frequency domain, so as to obtain a first amplitude-frequency characteristic corresponding to the first audio data and a second amplitude-frequency characteristic corresponding to the plurality of second audio data; a filter generating unit, configured to generate an equalizing filter according to the first amplitude-frequency characteristic and a plurality of the second amplitude-frequency characteristics; A beam constraining unit, configured to perform beamforming processing on the plurality of second audio data to obtain a beamformed audio signal; a data output unit, configured to input the beamformed audio signal into the equalization filter; The method of generating an equalization filter according to the first amplitude-frequency characteristic and a plurality of the second amplitude-frequency characteristics comprises the following steps: Dividing the first amplitude-frequency characteristic and the plurality of the second amplitude-frequency characteristics into a plurality of real-time sub-bands; Obtaining amplitude mean values ​​of a plurality of the second amplitude-frequency characteristics in each of the real-time sub-bands; Subtracting the amplitude mean value corresponding to each of the real-time sub-bands from the corresponding amplitude value of the first amplitude-frequency characteristic in sequence to obtain a real-time sub-band amplitude difference value group; Determine the original amplitude-frequency curve of the filter according to the real-time sub-band amplitude difference group and the standard sub-band amplitude difference group, wherein the standard sub-band amplitude difference group is obtained by the terminal voice device according to the swept frequency tone; The equalization filter is determined according to the original amplitude-frequency curve of the filter.

8. A terminal voice device, characterized in that: include: case; A speaker is disposed on the housing, wherein the speaker and the housing form a sealed chamber; an internal microphone, disposed in the sealed chamber; A plurality of external microphones are disposed on the outer surface of the housing; A processing unit is connected to the speaker, the internal microphone and the plurality of external microphones respectively, and the processing unit is used to execute the speech processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a speech processing method as described in any one of claims 1 to 6.

Images