Speech enhancement method, system and device based on particle velocity sensor microarray

Abstract

The present application discloses a speech enhancement method, system and device based on a particle velocity sensor microarray. The method is based on a novel particle velocity sensor microarray. Based on an in-depth study of the dipole directivity of the particle velocity sensitive unit, the method can realize the adaptive beam pointing and adaptive concave point pointing of the particle velocity sensor microarray by changing the guiding azimuth angle. By innovatively acquiring the beam pickup signal and concave point suppression signal of the target speech and its frequency domain amplitude spectrum, the energy distribution function of the target speech at different frequencies is obtained. By filtering the frequency domain of the sound pressure amplitude spectrum of the particle velocity sensor microarray, the speech signal in a specific direction can be enhanced in a noisy environment, while suppressing environmental noise and interference in other directions, thereby improving the intelligibility and quality of the speech.

Description

Technical Field

[0001] The present application belongs to the field of speech signal processing technology, and in particular, relates to a speech enhancement method, system and device based on a particle velocity sensor microarray. Background Art

[0002] In the field of modern communication and audio processing, especially in voice communication and recognition applications in complex environments, environmental noise and voice interference have always been the main obstacles affecting voice quality and recognition accuracy. Speech enhancement technology refers to the technology of extracting pure target speech signals from noisy speech signals. Its purpose is to improve the target speech quality, intelligibility and recognition rate. It has a wide range of applications in speech recognition, voice communication and human-computer interaction.

[0003] Traditional speech enhancement technology usually uses a single sound pressure sensor or a sound pressure sensor array. Among them, a single sound pressure sensor can only provide sound intensity information, but not direction information, so it is difficult to accurately locate and enhance the target speech in a noisy environment; the sound pressure sensor array is based on the principle of spatial filtering, and weighted combination of speech signals from different directions is performed to achieve the purpose of suppressing noise and interference. However, this type of method has some difficult-to-overcome defects, including limited ability to suppress low-frequency noise, high requirements for array size and spacing, and sensitivity to array structure errors, which greatly limits its application scenarios. Summary of the invention

[0004] The embodiments of the present application provide a speech enhancement method, system and device based on a particle velocity sensor microarray, which are used to at least solve the problem of limited application scenarios in related technologies.

[0005] In a first aspect, an embodiment of the present application provides a speech enhancement method based on a particle velocity sensor microarray, wherein the particle velocity sensor microarray has two particle velocity channels that are perpendicular to each other in horizontal directions;

[0006] The method comprises:

[0007] By picking up the speech with a particle velocity sensor microarray, the sound pressure signal and particle velocity signal corresponding to the target speech and the interference speech in the sound field are obtained;

[0008] Determine a first horizontal azimuth angle of the target speech and a second horizontal azimuth angle of the interference speech according to a trigonometric function relationship between the sound pressure signal and the particle velocity signal;

[0009] The guiding azimuth angle of the rotation of the particle velocity channel is set to a first horizontal azimuth angle, and a target voice beam pickup signal corresponding to the adaptive beam pointing of the target voice and a target voice concave point suppression signal corresponding to the adaptive concave point pointing of the target voice are obtained, wherein the target voice beam pickup signal and the target voice concave point suppression signal are both time domain signals;

[0010] Obtaining an energy distribution function of the target speech at different frequencies according to the target speech beam picking signal and the target speech concave suppression signal;

[0011] Obtaining a sound pressure amplitude spectrum after frequency domain filtering according to the energy distribution function and the sound pressure amplitude spectrum corresponding to the sound pressure component of the particle velocity sensor microarray;

[0012] The sound pressure amplitude spectrum after the frequency domain filtering is subjected to inverse Fourier transform to enhance the target speech.

[0013] In a second aspect, an embodiment of the present application provides a speech enhancement system based on a particle velocity sensor microarray, the system comprising a particle velocity sensor microarray and a speech enhancement device based on the particle velocity sensor microarray;

[0014] The particle velocity sensor microarray is used to obtain the sound pressure signal and particle velocity signal corresponding to the target voice and the interference voice in the sound field through voice pickup;

[0015] The speech enhancement device based on the particle velocity sensor microarray comprises:

[0016] A determination module, used to determine a first horizontal azimuth angle of the target speech and a second horizontal azimuth angle of the interference speech according to a trigonometric function relationship between the sound pressure signal and the particle velocity signal;

[0017] A rotation module is used to set the guiding azimuth angle of the rotation of two particle velocity channels in the particle velocity sensor microarray that are perpendicular to each other in the horizontal direction to a first horizontal azimuth angle, so as to obtain a target speech beam pickup signal corresponding to the adaptive beam pointing of the target speech, and a target speech concave point suppression signal corresponding to the adaptive concave point pointing of the target speech, wherein both the target speech beam pickup signal and the target speech concave point suppression signal are time domain signals;

[0018] A function module, used for obtaining an energy distribution function of the target speech at different frequencies according to the target speech beam picking signal and the target speech concave suppression signal;

[0019] A filtering module, used to obtain a sound pressure amplitude spectrum after frequency domain filtering according to the energy distribution function and the sound pressure amplitude spectrum corresponding to the sound pressure component of the particle velocity sensor microarray;

[0020] The transformation module is used to perform inverse Fourier transformation on the sound pressure amplitude spectrum after the frequency domain filtering to enhance the target speech.

[0021] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the steps of the speech enhancement method based on a particle velocity sensor microarray as described in any one of the embodiments of the first aspect are implemented.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the steps of the speech enhancement method based on a particle velocity sensor microarray as described in any one of the embodiments of the first aspect are implemented.

[0023] In a fifth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of a speech enhancement method based on a particle velocity sensor microarray as provided in the first aspect of the embodiment of the present application.

[0024] The speech enhancement method, system and device based on the particle velocity sensor microarray of the embodiment of the present application are based on a novel particle velocity sensor microarray. After in-depth research on the directivity of the dipole of the particle velocity sensitive unit, the adaptive beam pointing and adaptive concave point pointing of the particle velocity sensor microarray can be realized by changing the guiding azimuth angle. The energy distribution function of the target speech at different frequencies is obtained by innovatively acquiring the beam pickup signal and concave point suppression signal of the target speech and its frequency domain amplitude spectrum. By filtering the frequency domain of the sound pressure amplitude spectrum of the particle velocity sensor microarray, the speech signal in a specific direction can be enhanced in a noisy environment, while suppressing environmental noise and interference in other directions, thereby improving the intelligibility and quality of the speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] Figure 1 It is a flow chart of a speech enhancement method based on a particle velocity sensor microarray provided in an embodiment of the present application;

[0027] Figure 2 Schematic diagram of the dipole directivity of a particle velocity sensitive unit in a particle velocity sensor microarray provided in an embodiment of the present application;

[0028] Figure 3 is a beam pattern of an interference speech suppression function A0cot(θ2-θ1) provided in an embodiment of the present application;

[0029] Figure 4 It is a structural schematic diagram of a speech enhancement device based on a particle velocity sensor microarray provided in an embodiment of the present application;

[0030] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0031] Reference numerals:

[0032] A speech enhancement device 400 based on a particle velocity sensor microarray, a determination module 401, a rotation module 402, a function module 403, a filtering module 404, a transformation module 405,

[0033] Electronic device 500 , processor 501 , memory 502 , communication interface 503 , bus 510 . DETAILED DESCRIPTION

[0034] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0035] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0036] In the field of modern communication and audio processing, especially in voice communication and recognition applications in complex environments, environmental noise and voice interference have always been the main obstacles affecting voice quality and recognition accuracy. Speech enhancement technology refers to the technology of extracting pure target speech signals from noisy speech signals. Its purpose is to improve the target speech quality, intelligibility and recognition rate. It has a wide range of applications in speech recognition, voice communication and human-computer interaction.

[0037] Traditional speech enhancement technology usually uses a single sound pressure sensor or a sound pressure sensor array. Among them, a single sound pressure sensor can only provide sound intensity information, but not direction information, so it is difficult to accurately locate and enhance the target speech in a noisy environment; the sound pressure sensor array is based on the principle of spatial filtering, and weighted combination of speech signals from different directions is performed to achieve the purpose of suppressing noise and interference. However, this type of method has some difficult-to-overcome defects, including limited ability to suppress low-frequency noise, high requirements for array size and spacing, and sensitivity to array structure errors, which greatly limits its application scenarios.

[0038] The patent "A method for target speech enhancement of a single acoustic vector sensor based on time-frequency mask (CN104103277B)" obtains the difference Δθ between the estimated value θ1 of the target speech arrival angle and the actual azimuth angle θ0 of the target speech, and then sets an allowable angle range δ (which can be a number in the range of 0° to 90°, and 25° is taken in the patent). When |Δθ|<δ, it is determined that the frequency domain amplitude spectrum of the frequency point comes from the target speech. At this time, the time-frequency point mask coefficient is set to 1, otherwise the mask coefficient is set to ρ (which can be a number between 0 and 1, and 0.07 is taken in the patent). Although this method can enhance the target speech and suppress interfering speech and background noise in a complex environment with multiple speakers, the target speech enhancement effect is greatly affected by factors such as the actual azimuth angle θ0 of the target speech, the allowed angle range δ, and the selected value of the time-frequency mask coefficient ρ. For example, when the difference between the azimuth angle of the interfering speech and the actual azimuth angle of the target speech is within the allowed angle range, this method cannot suppress the interfering speech. In addition, if the actual azimuth angle of the target speech is near 45°, the background noise cannot be suppressed because it is close to the arrival angle of the component represented by the noise.

[0039] In order to solve the problems of the related art, the embodiments of the present application provide a speech enhancement method, system and device based on a particle velocity sensor microarray.

[0040] The speech enhancement method based on the particle velocity sensor microarray provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in combination with the accompanying drawings.

[0041] It should be noted that the speech enhancement method based on the particle velocity sensor microarray of the embodiment of the present application can be applied to application fields such as speech recognition, voice communication, smart home, conference microphone, etc. Moreover, the speech enhancement method based on the particle velocity sensor microarray of the embodiment of the present application does not depend on specific environmental settings or sound source positions, has strong applicability and flexibility, and can be widely used in speech enhancement scenarios in a variety of noisy environments.

[0042] Figure 1 A flow chart of a speech enhancement method based on a particle velocity sensor microarray according to an embodiment of the present application is shown.

[0043] like Figure 1 As shown, the speech enhancement method based on the particle velocity sensor microarray may specifically include the following steps:

[0044] S101, obtaining sound pressure signals and particle velocity signals corresponding to the target voice and the interference voice in the sound field through the particle velocity sensor microarray voice pickup;

[0045] S102, determining a first horizontal azimuth angle of the target speech and a second horizontal azimuth angle of the interference speech according to a trigonometric function relationship between the sound pressure signal and the particle velocity signal;

[0046] S103, setting the guiding azimuth angle of the particle velocity channel rotation to a first horizontal azimuth angle, obtaining a target voice beam pickup signal corresponding to the adaptive beam pointing of the target voice, and a target voice concave point suppression signal corresponding to the adaptive concave point pointing of the target voice, wherein both the target voice beam pickup signal and the target voice concave point suppression signal are time domain signals;

[0047] S104, obtaining an energy distribution function of the target speech at different frequencies according to the target speech beam picking signal and the target speech concave suppression signal;

[0048] S105, obtaining a sound pressure amplitude spectrum after frequency domain filtering according to the energy distribution function and the sound pressure amplitude spectrum corresponding to the sound pressure component of the particle velocity sensor microarray;

[0049] S106: Perform inverse Fourier transform on the sound pressure amplitude spectrum after the frequency domain filtering to enhance the target speech.

[0050] Therefore, based on the new type of particle velocity sensor microarray, on the basis of in-depth research on the dipole directivity of the particle velocity sensitive unit, the adaptive beam pointing and adaptive concave point pointing of the particle velocity sensor microarray can be realized by changing the guiding azimuth angle. By innovatively acquiring the beam pickup signal and concave point suppression signal of the target speech and its frequency domain amplitude spectrum, the energy distribution function of the target speech at different frequencies is obtained. By filtering the frequency domain of the sound pressure amplitude spectrum of the particle velocity sensor microarray, the speech signal in a specific direction can be enhanced in a noisy environment, while suppressing the environmental noise and interference in other directions, thereby improving the intelligibility and quality of the speech.

[0051] The specific implementation methods of the above steps are introduced below.

[0052] It should be noted that the speech sound field has the characteristics of both scalar field (sound pressure) and vector field (particle velocity). Both the speech sound pressure and particle velocity sound fields carry rich speech information. By simultaneously acquiring the sound pressure and particle velocity information of the target speech, better target speech enhancement can be achieved compared to a single sound pressure microphone.

[0053] Therefore, in the embodiment of the present application, the particle velocity sensor microarray is used to directly, synchronously, and co-point measure the sound pressure and particle velocity information in the target speech, which has the advantages of a small number of channels, a high signal-to-noise ratio, a small size, and the ability to effectively achieve 360-degree all-round voice pickup. Compared with the particle velocity sensor microarray, the method of calculating the speech particle velocity information based on the sound pressure gradient in the related art is limited by the conditions such as the amplitude, phase consistency and physical spacing between the sound pressure microphones. The sound pressure gradient has the disadvantages of low low-frequency sensitivity, narrow response frequency band, and large measurement error. Therefore, the particle velocity sensor microarray has a better development prospect in the field of speech signal processing.

[0054] Specifically, the particle velocity sensor microarray is composed of a sound pressure sensitive unit and a two-dimensional particle velocity sensitive unit. It is a new type of far-field voice pickup device that can not only measure the sound pressure component in the sound field, but also measure the two-dimensional particle velocity component in the spatial sound field in a co-point, synchronous and independent manner. Therefore, the particle velocity sensor microarray can output the sound pressure signal and particle velocity signal corresponding to the target voice and the interference voice in the sound field. Further, the azimuth of the target voice and the interference voice can be detected through the trigonometric function relationship between them, that is, the first horizontal azimuth of the target voice and the second horizontal azimuth of the interference voice.

[0055] When the particle velocity sensor microarray is performing voice pickup, the sound pressure and two-dimensional particle velocity signals measured are:

[0056]

[0057] In formula (1), x1(t) is the sound pressure signal of the target speech; x2(t) is the sound pressure signal of the interference speech; θ1 is the first horizontal azimuth angle of the target speech, and θ2 is the second horizontal azimuth angle of the interference speech, where the positive direction of the x-axis is 0°; n p (t), n x (t) and n y (t) are the noise signals (including self-noise) received by the sound pressure and two-dimensional particle velocity sensitive units respectively.

[0058] It should be understood that speech signals have good sparsity in the time-frequency domain. When there are multiple speakers in a speech, the speech signal energy of different speakers has a discrete distribution characteristic in the time-frequency domain. It can be approximately considered that only one speaker's speech signal is dominant, and the contribution of other sound sources can be ignored. This property is called the time-frequency domain sparsity of speech.

[0059] Furthermore, in some embodiments, since the particle velocity sensor microarray has two particle velocity channels that are perpendicular to each other in the horizontal direction, a combined directivity v can be formed by rotating a certain angle. c (t) and v s (t).

[0060] Specifically, replace v in formula (1) x (t) and v y (t) by rotation angle get:

[0061]

[0062] Among them, n c (t),n s (t) is white noise, and:

[0063]

[0064] It should be noted that after rotation, v c (t) and v s (t) Still orthogonal dipole directivity, by changing the guiding azimuth The value can change the directivity of the particle velocity sensor microarray, and adaptive beam pointing and adaptive concave point pointing can be achieved. When the value is the first horizontal azimuth angle θ1 of the target speech, beam picking and notch suppression of the target speech can be achieved.

[0065] Right now:

[0066]

[0067] Among them, θ2-θ1 is the angle between the target speech and the interference speech.

[0068] In this way, the target speech beam picking signal v corresponding to the adaptive beam pointing of the target speech can be obtained. c (t), and the adaptive concave point of the target speech points to the corresponding target speech concave point suppression signal v s (t), and it can be understood that the target speech beam picking signal v c (t) and the target speech concave suppression signal v s (t) are all time domain signals.

[0069] Furthermore, in some embodiments, the target speech beam picking signal and the target speech concave point suppression signal are respectively subjected to fast Fourier transform to obtain a first frequency domain amplitude spectrum and a second frequency domain amplitude spectrum; the amplitude ratio of the first frequency domain amplitude spectrum and the second frequency domain amplitude spectrum at a preset frequency point is calculated; based on the amplitude ratio, the gain of the component represented by the sound pressure signal of the target speech is normalized to obtain the energy distribution function of the target speech at different frequencies. Thus, according to the target speech beam picking signal and the target speech concave point suppression signal, the energy distribution function of the target speech at different frequencies can be obtained.

[0070] In the specific implementation, v c (t) and v s (t) Perform fast Fourier transform to obtain its frequency domain amplitude spectrum v c (ω) and v s (ω):

[0071]

[0072] Where, =x1(ω1)+ represents the amplitude spectrum of the target speech; +x2(ω2) represents the amplitude spectrum of the interfering speech; white noise n c (t),n s The amplitude spectrum of (t) c (ω),n s (ω) is a constant.

[0073] It should be noted that the depth attenuation value of the concave point of the particle velocity sensitive unit in the particle velocity sensor microarray is not an ideal negative infinity. Figure 2 , is a schematic diagram of the dipole directivity of the particle velocity sensitive unit in the particle velocity sensor microarray. Figure 2 As shown, the dB value corresponding to the pit depth (cos90° or sin0°) is not an ideal negative infinity, but is determined based on the pit depth measurement value of the particle velocity sensitive unit, for example, -30 dB.

[0074] Furthermore, A0 is defined as the depth attenuation value of the particle velocity sensitive unit concave point, then formula (5)

[0075] Can be simplified to:

[0076]

[0077] Furthermore, v c (ω) and v s (ω) The corresponding frequency points in the amplitude spectrum are ratioed to obtain the amplitude ratio r(ω):

[0078]

[0079] It can be seen that the component represented by the target speech x1(t) (at the frequency ω1 point) can achieve a gain of 1 / A0, and the component represented by the interference speech x2(t) (at the frequency ω2 point) can achieve a gain of cot(θ2-θ1). If the noise n c (ω) and n s (ω) is white noise, and the gain of the component it represents is 1.

[0080] Furthermore, by multiplying the constant A0 on both sides of equation (7), the gain of the component represented by the target speech x1(t) can be normalized to obtain the function r g (ω), this function can reflect the energy distribution of the target speech at different frequencies. That is:

[0081]

[0082] Thus, after normalization, the energy distribution function r g (ω) can achieve A0cot(θ2-θ1) times suppression of the component represented by the interfering speech x2(t), and A0 times suppression of the component represented by the noise.

[0083] When A0 = -30dB, Figure 3 The beam diagram of the interference speech suppression function A0cot(θ2-θ1) is shown, and the following Table 1 shows the suppression effect of the interference speech when the angle between the interference speech and the target speech is different. Figure 3 As shown, a narrow beam of about 5.2° (-3dB attenuation) can be achieved.

[0084] Table 1

[0085]

[0086]

[0087] Furthermore, in some embodiments, the frequency domain amplitude spectrum of the sound pressure component p(t) of the particle velocity sensor microarray, that is, the sound pressure amplitude spectrum p(ω) is obtained:

[0088] p(ω)=x1(ω1)+x2(ω2)+n p (ω) (9)

[0089] Among them, n p (ω) represents the sound pressure noise amplitude spectrum.

[0090] Then, the sound pressure amplitude spectrum p(ω) and the energy distribution function r g The corresponding frequency points in (ω) are multiplied respectively to obtain the sound pressure amplitude spectrum p after frequency domain filtering. f (ω):

[0091]

[0092] In this way, the frequency domain filtering of the sound pressure amplitude spectrum of the particle velocity sensor microarray is realized. f (ω), an inverse Fourier transform is performed to finally achieve speech enhancement.

[0093] Therefore, based on the in-depth study of the dipole directivity of the particle velocity sensitive unit, the adaptive beamforming directivity and adaptive concave point directivity characteristics of the particle velocity sensor are used to obtain the energy distribution function of the target speech at different frequencies, which can effectively distinguish and suppress the target speech and the interference speech, and has the advantage of narrow beam. Compared with the time-frequency mask method, it is not easily affected by factors such as the actual azimuth angle of the target speech, the allowed angle range, and the selected value of the time-frequency mask coefficient. Even when the target speech and the interference speech are close in direction, it can maintain good speech enhancement performance and significantly improve the signal-to-noise ratio and intelligibility of the target speech.

[0094] In other words, by adopting adaptive beam pointing technology of target speech, adaptive concave point pointing technology of target speech and frequency domain filtering technology of sound pressure amplitude spectrum, it is possible to enhance speech signals in a specific direction in a noisy environment, while suppressing environmental noise and interference in other directions, thereby improving the intelligibility and quality of speech.

[0095] In addition, the embodiment of the present application also provides another speech enhancement method based on a particle velocity sensor microarray. The specific implementation method includes: using the sound pressure signal and the particle velocity signal output by the particle velocity sensor microarray, the azimuth angle θ1 of the target speech and the azimuth angle θ2 of the interfering speech can be detected through the trigonometric function relationship between them; by rotating the angle θ1, the adaptive beam pointing of the target speech is realized to obtain the target speech beam picking signal v c(t); By rotating the angle θ1, the adaptive concave point pointing of the target speech is realized, and the concave point suppression signal v of the target speech is obtained s (t); Using fast Fourier transform, based on the obtained v c (t) and v s (t) time domain signal, and obtain its frequency domain amplitude spectrum v c (ω) and v s (ω); calculate the frequency domain amplitude spectrum v c (ω) and v s (ω) is the amplitude ratio r(ω) of the corresponding frequency points; based on the amplitude ratio r(ω), the gain of the component represented by the target speech x1(t) is normalized to obtain the function r g (ω), function r g (ω) reflects the energy distribution of the target speech at different frequencies; the function r g (ω) is multiplied by the sound pressure amplitude spectrum p(ω) in the particle velocity sensor microarray p(ω)*r g (ω), and obtain the sound pressure amplitude spectrum p after frequency domain filtering f (ω); finally, the sound pressure amplitude spectrum p f (ω) Perform inverse Fourier transform to achieve speech enhancement.

[0096] It can be seen that the beam picking signal of the target speech and the concave suppression signal of the target speech are obtained based on the particle velocity sensor microarray pickup, the energy distribution function of the target speech at different frequencies is obtained, and the frequency domain filtering of the sound pressure amplitude spectrum is realized. The energy distribution difference between the target speech and the interference speech at different frequencies and different spatial orientations is fully utilized to achieve more refined speech enhancement. Compared with the traditional time domain or space domain processing methods, it can better retain the detailed information of the target speech and improve the naturalness and clarity of the speech.

[0097] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0098] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a speech enhancement system based on a particle velocity sensor microarray.

[0099] Specifically, the speech enhancement system based on the particle velocity sensor microarray includes a particle velocity sensor microarray and a speech enhancement device 400 based on the particle velocity sensor microarray.

[0100] The particle velocity sensor microarray is used to obtain the sound pressure signals and particle velocity signals corresponding to the target speech and the interference speech in the sound field through speech pickup.

[0101] Specifically, the particle velocity sensor microarray in this embodiment has the advantages of few channels, high signal-to-noise ratio, small size, and the ability to effectively achieve 360-degree all-round voice pickup. It is easy to integrate into miniaturized devices. Compared with traditional speech enhancement methods based on large microphone arrays, it has lower hardware cost, power consumption and computational complexity, and is more suitable for application scenarios such as mobile devices, smart homes, and robot hearing.

[0102] refer to Figure 4 , is a schematic diagram of the structure of a speech enhancement device 400 based on a particle velocity sensor microarray. Figure 4 As shown, the speech enhancement device 400 based on the particle velocity sensor microarray includes:

[0103] A determination module 401 is used to determine a first horizontal azimuth angle of the target speech and a second horizontal azimuth angle of the interference speech according to a trigonometric function relationship between the sound pressure signal and the particle velocity signal;

[0104] The rotation module 402 is used to set the guiding azimuth angle of the rotation of two particle velocity channels in the particle velocity sensor microarray that are perpendicular to each other in the horizontal direction to a first horizontal azimuth angle, and obtain a target voice beam pickup signal corresponding to the adaptive beam pointing of the target voice, and a target voice concave point suppression signal corresponding to the adaptive concave point pointing of the target voice, wherein both the target voice beam pickup signal and the target voice concave point suppression signal are time domain signals;

[0105] Function module 403, used for obtaining the energy distribution function of the target speech at different frequencies according to the target speech beam picking signal and the target speech concave suppression signal;

[0106] A filtering module 404 is used to obtain a sound pressure amplitude spectrum after frequency domain filtering according to the energy distribution function and the sound pressure amplitude spectrum corresponding to the sound pressure component of the particle velocity sensor microarray;

[0107] The transformation module 405 is used to perform inverse Fourier transformation on the sound pressure amplitude spectrum after the frequency domain filtering to enhance the target speech.

[0108] In some optional embodiments, the function module 403 is specifically used to perform fast Fourier transform on the target speech beam picking signal and the target speech concave suppression signal, respectively, to obtain a first frequency domain amplitude spectrum and a second frequency domain amplitude spectrum; calculate the amplitude ratio of the first frequency domain amplitude spectrum and the second frequency domain amplitude spectrum at a preset frequency point; normalize the gain of the component represented by the sound pressure signal of the target speech based on the amplitude ratio, to obtain the energy distribution function of the target speech at different frequencies.

[0109] Optionally, the rotation module 402 is specifically configured to obtain a target speech beam picking signal and a target speech concave suppression signal according to the following formula:

[0110]

[0111] Among them, v c (t) represents the target speech beam picking signal; v s (t) represents the target speech concave suppression signal; x1(t) represents the target speech sound pressure signal; x2(t) represents the interfering speech sound pressure signal; θ1 represents the first horizontal azimuth of the target speech; θ2 represents the second horizontal azimuth of the interfering speech; n c (t),n s (t) is white noise.

[0112] Optionally, the function module 403 is specifically used to obtain the energy distribution function r of the target speech at different frequencies according to the following formula: g (ω):

[0113]

[0114] Where A0 represents the depth attenuation value of the particle velocity sensitive unit concave point; r(ω) represents the amplitude ratio; θ1 represents the first horizontal azimuth angle of the target speech; θ2 represents the second horizontal azimuth angle of the interference speech; n c (ω),n s (ω) represents white noise n c (t),n s (t) corresponds to the amplitude spectrum, which is a constant; x1(ω1)+ represents the amplitude spectrum of the target speech; +x2(ω2) represents the amplitude spectrum of the interfering speech.

[0115] Optionally, the filtering module 404 is specifically configured to obtain the sound pressure amplitude spectrum p after frequency domain filtering according to the following formula: f (ω):

[0116]

[0117] Where p(ω) represents the sound pressure amplitude spectrum; n p (ω) represents the sound pressure noise amplitude spectrum.

[0118] It should be noted that, for the convenience of description, the above device is described in various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0119] The device of the above embodiment is used to implement the corresponding speech enhancement method based on particle velocity sensor microarray in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0120] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an electronic device.

[0121] Figure 5 A more specific schematic diagram of the hardware structure of an electronic device provided by this embodiment is shown.

[0122] The electronic device 500 may include a processor 501 and a memory 502 storing computer program instructions.

[0123] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0124] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.

[0125] In certain embodiments, the memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present application.

[0126] The processor 501 reads and executes the computer program instructions stored in the memory 502 to implement any one of the speech enhancement methods based on the particle velocity sensor microarray in the above embodiments.

[0127] In some examples, the electronic device 500 may further include a communication interface 503 and a bus 510. Figure 5 As shown, the processor 501, the memory 502, and the communication interface 503 are connected via a bus 510 and communicate with each other.

[0128] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0129] Bus 510 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus 510 may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 510 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0130] Exemplarily, the electronic device 500 may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA).

[0131] Based on the same technical concept, corresponding to any of the above-mentioned embodiments, the present application also provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any of the speech enhancement methods based on the particle velocity sensor microarray in the above-mentioned embodiments is implemented. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0132] Based on the same technical concept, corresponding to any of the above-mentioned embodiments, the present application also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor execute the speech enhancement method based on a particle velocity sensor microarray. Corresponding to the execution subject corresponding to each step in each embodiment of the speech enhancement method based on a particle velocity sensor microarray, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0133] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0134] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0135] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0136] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0137] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A speech enhancement method based on a particle velocity sensor microarray, characterized in that: The particle velocity sensor microarray has two particle velocity channels which are perpendicular to each other in horizontal directions; The method comprises: By picking up the speech with a particle velocity sensor microarray, the sound pressure signal and particle velocity signal corresponding to the target speech and the interference speech in the sound field are obtained; Determine a first horizontal azimuth angle of the target speech and a second horizontal azimuth angle of the interference speech according to a trigonometric function relationship between the sound pressure signal and the particle velocity signal; The guiding azimuth angle of the rotation of the particle velocity channel is set to a first horizontal azimuth angle, and a target voice beam pickup signal corresponding to the adaptive beam pointing of the target voice and a target voice concave point suppression signal corresponding to the adaptive concave point pointing of the target voice are obtained, wherein the target voice beam pickup signal and the target voice concave point suppression signal are both time domain signals; Obtaining an energy distribution function of the target speech at different frequencies according to the target speech beam picking signal and the target speech concave suppression signal; Obtaining a sound pressure amplitude spectrum after frequency domain filtering according to the energy distribution function and the sound pressure amplitude spectrum corresponding to the sound pressure component of the particle velocity sensor microarray; The sound pressure amplitude spectrum after the frequency domain filtering is subjected to inverse Fourier transform to enhance the target speech.

2. The method according to claim 1, characterized in that The step of obtaining the energy distribution function of the target speech at different frequencies according to the target speech beam picking signal and the target speech concave suppression signal comprises: Performing fast Fourier transform on the target speech beam picking signal and the target speech concave suppression signal respectively to obtain a first frequency domain amplitude spectrum and a second frequency domain amplitude spectrum; Calculate the amplitude ratio of the first frequency domain amplitude spectrum and the second frequency domain amplitude spectrum at a preset frequency point; The gain of the component represented by the sound pressure signal of the target speech is normalized based on the amplitude ratio to obtain the energy distribution function of the target speech at different frequencies.

3. The method according to claim 1, characterized in that The guiding azimuth angle of the rotation of the particle velocity channel is set to a first horizontal azimuth angle, and a target voice beam pickup signal corresponding to the adaptive beam pointing of the target voice and a target voice concave point suppression signal corresponding to the adaptive concave point pointing of the target voice are obtained, and the target voice beam pickup signal and the target voice concave point suppression signal are both time domain signals, including: The target speech beam picking signal and the target speech concave suppression signal are obtained according to the following formula: Among them, v c (t) represents the target speech beam picking signal; v s (t) represents the target speech concave suppression signal; x1(t) represents the target speech sound pressure signal; x2(t) represents the interfering speech sound pressure signal; θ1 represents the first horizontal azimuth of the target speech; θ2 represents the second horizontal azimuth of the interfering speech; n c (t),n s (t) is white noise.

4. The method according to claim 2, characterized in that: The step of normalizing the gain of the component represented by the sound pressure signal of the target speech based on the amplitude ratio to obtain the energy distribution function of the target speech at different frequencies includes: The energy distribution function r of the target speech at different frequencies is obtained according to the following formula g (ω): Where A0 represents the depth attenuation value of the particle velocity sensitive unit concave point; r(ω) represents the amplitude ratio; θ1 represents the first horizontal azimuth angle of the target speech; θ2 represents the second horizontal azimuth angle of the interference speech; n c (ω),n s (ω) represents white noise n c (t),n s (t) corresponds to the amplitude spectrum, which is a constant; x1(ω1)+ represents the amplitude spectrum of the target speech; +x2(ω2) represents the amplitude spectrum of the interfering speech.

5. The method according to claim 4, characterized in that The method of obtaining the sound pressure amplitude spectrum after frequency domain filtering according to the energy distribution function and the sound pressure amplitude spectrum corresponding to the sound pressure component of the particle velocity sensor microarray comprises: The sound pressure amplitude spectrum p after frequency domain filtering is obtained according to the following formula f (ω): Where p(ω) represents the sound pressure amplitude spectrum; n p (ω) represents the sound pressure noise amplitude spectrum.

6. A speech enhancement system based on a particle velocity sensor microarray, characterized in that: The system includes a particle velocity sensor microarray and a speech enhancement device based on the particle velocity sensor microarray; The particle velocity sensor microarray is used to obtain the sound pressure signal and particle velocity signal corresponding to the target voice and the interference voice in the sound field through voice pickup; The speech enhancement device based on the particle velocity sensor microarray comprises: A determination module, used to determine a first horizontal azimuth angle of the target speech and a second horizontal azimuth angle of the interference speech according to a trigonometric function relationship between the sound pressure signal and the particle velocity signal; A rotation module is used to set the guiding azimuth angle of the rotation of two particle velocity channels in the particle velocity sensor microarray that are perpendicular to each other in the horizontal direction to a first horizontal azimuth angle, so as to obtain a target speech beam pickup signal corresponding to the adaptive beam pointing of the target speech, and a target speech concave point suppression signal corresponding to the adaptive concave point pointing of the target speech, wherein both the target speech beam pickup signal and the target speech concave point suppression signal are time domain signals; A function module, used for obtaining an energy distribution function of the target speech at different frequencies according to the target speech beam picking signal and the target speech concave suppression signal; A filtering module, used to obtain a sound pressure amplitude spectrum after frequency domain filtering according to the energy distribution function and the sound pressure amplitude spectrum corresponding to the sound pressure component of the particle velocity sensor microarray; The transformation module is used to perform inverse Fourier transformation on the sound pressure amplitude spectrum after the frequency domain filtering to enhance the target speech.

7. The system according to claim 6, characterized in that The particle velocity sensor microarray is composed of a sound pressure sensitive unit and a particle velocity sensitive unit.

8. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; when the processor calls the computer program instructions, the speech enhancement method based on the particle velocity sensor microarray as described in any one of claims 1 to 5 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when called by a processor, implement the speech enhancement method based on a particle velocity sensor microarray as described in any one of claims 1 to 5.

10. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the speech enhancement method based on a particle velocity sensor microarray as described in any one of claims 1 to 5.

Images