Microphone array, sound velocity estimation method, sound pickup method and related device and apparatus

Abstract

The microphone array, the sound velocity estimation method, the sound pickup method and the related device and equipment provided by the embodiment of the application are applied to the technical field of voice, the microphone array comprises: N sound vector microphones, which are used to pick up sound signals of a target sound source; wherein the distance between at least two sound vector microphones in the N sound vector microphones is within a distance range of 0.02 meters to 5 meters, and N is an integer greater than 1. The accuracy and real-time performance of sound velocity estimation are improved.

Description

Technical Field

[0001] This application relates to the field of speech technology, specifically to a microphone array, a sound velocity estimation method, a sound pickup method, and related devices and equipment. Background Technology

[0002] With the development of technology, many electronic devices are equipped with microphone arrays, which have become an important hub for human-computer voice interaction.

[0003] A microphone array refers to an arrangement of microphones. A microphone array consists of a certain number of microphones used to sample and process the control characteristics of a sound field. Existing microphone arrays typically consist of at least two omnidirectional microphones.

[0004] In existing technologies, directional beamforming can be used to better acquire speech signals and suppress noise. During directional beamforming, the sound signal picked up by the microphone array can be amplified in a specific direction by using the direction of arrival (DOA) of the microphone array, the distance between the microphones in the array, and the current sound velocity, thus achieving the purpose of directional beamforming. Since the DOA and the distance between the microphones in the array are preset, the accuracy of directional beamforming mainly depends on the estimation of the sound velocity. Currently, common sound velocity measurement methods mainly include indirect sound velocity measurement methods based on empirical formulas and direct sound velocity measurement methods based on active sound generation.

[0005] Indirect methods for measuring sound speed primarily utilize the functional relationship between the speed of sound in air and environmental temperature, humidity, and pressure. This involves measuring the ambient temperature, humidity, and pressure using instruments, and then calculating the speed of sound using empirical formulas. Generally, ambient temperature has the strongest correlation with the speed of sound; the higher the temperature, the greater the speed of sound. Commonly used empirical formulas are as follows: Where c represents the speed of sound, t represents the ambient temperature, and t0 is a preset temperature reference value. Because the speed of sound is affected by many factors in practical applications, it is impossible to calculate the speed of sound quickly and accurately when the environment changes. Therefore, the accuracy of the speed of sound measured by the above indirect measurement method is not high.

[0006] Direct sound speed measurement methods involve actively emitting sound. For example, after synchronizing the clocks of the transmitting and receiving ends, the transmitting end emits a sound signal, and the receiving end receives the sound signal. The propagation time of the sound signal is determined by the time it takes for the sound signal to be emitted and received. The speed of sound can then be estimated based on the distance between the transmitting and receiving ends and the sound signal propagation time. However, this method requires a specific sound-emitting device to perform the sound speed test. Furthermore, it cannot measure the sound speed in a timely manner when the environment changes, resulting in poor real-time performance of the sound speed estimation. Summary of the Invention

[0007] In view of this, this application provides a microphone array, a sound velocity estimation method, a sound pickup method, and related devices and equipment to solve the problem that the sound velocity cannot be estimated in real time in the prior art, resulting in low sound velocity accuracy.

[0008] In a first aspect, embodiments of this application provide a microphone array, comprising: N sound vector microphones.

[0009] N vector microphones are used to pick up the sound signal from the target sound source;

[0010] Wherein, the distance between at least two of the N sound vector microphones is in the range of 0.02 meters to 5 meters, and N is an integer greater than 1.

[0011] In this way, the acoustic signals of the target sound source are picked up by the acoustic vector microphones included in the microphone array, ensuring that at least two or N acoustic vector microphones pick up and send different acoustic signals to the processor. These acoustic signals include sound speed information, allowing the current sound speed to be estimated based on the acoustic signals picked up by the acoustic vector microphones. When the environment changes, the current sound speed can be obtained in real time by picking up the acoustic signals of the target sound source through the acoustic vector microphones included in the microphone array of this embodiment, improving the accuracy of sound speed acquisition and thus enhancing the accuracy of directional beam picking.

[0012] In one implementation of the first aspect, the N sound vector microphones are located in the same plane.

[0013] This makes the microphone array arrangement simpler, easier to implement, and provides more accurate sound pickup.

[0014] In one implementation of the first aspect, the N sound vector microphones are arranged linearly.

[0015] This makes the microphone array arrangement simpler, easier to implement, and provides more accurate sound pickup.

[0016] In one implementation of the first aspect, each of the N acoustic vector microphones is equidistant from its adjacent vector microphone.

[0017] This makes the microphone array arrangement simpler, easier to implement, and provides more accurate sound pickup.

[0018] In one implementation of the first aspect, the distances between any two adjacent acoustic vector microphones among the N acoustic vector microphones are not exactly equal.

[0019] In this way, the distance between two sound vector microphones in the N sound vector microphones in the microphone array can have multiple values, which can be better applied to the estimation of the current sound speed in different scenarios and improve the usability of the microphone array.

[0020] In one implementation of the first aspect, the N sound vector microphones include a first sound vector microphone, a second sound vector microphone, and a third sound vector microphone. The angle between the straight line containing the third sound vector microphone and the first sound vector microphone and the straight line containing the first sound vector microphone and the second sound vector microphone is greater than 0 degrees and less than 180 degrees.

[0021] In this way, the microphone array can acquire the sound signal of the sound source in two-dimensional space, which improves the accuracy of sound signal acquisition, thereby improving the accuracy of current sound speed estimation, and further improving the accuracy of directional beam picking.

[0022] In one implementation of the first aspect, the N sound vector microphones are arranged in a three-dimensional configuration.

[0023] In this way, the microphone array can acquire the sound signal of the sound source in three-dimensional space, which improves the accuracy of sound signal acquisition, thereby improving the accuracy of current sound speed estimation, and further improving the accuracy of directional beam picking.

[0024] One implementation of the first aspect also includes: at least two omnidirectional microphones.

[0025] In this way, the microphone array in this application can pick up directional beams more accurately.

[0026] In one implementation of the first aspect, the at least two omnidirectional microphones are located between the N sound vector microphones.

[0027] This makes the microphone array arrangement simpler, easier to implement, and provides more accurate sound pickup.

[0028] In one implementation of the first aspect, a processing unit is further included; the processing unit is configured to acquire the sound signals picked up by the N sound vector microphones, calculate the current sound speed based on the sound signals picked up by the N sound vector microphones and the distance between the N sound vector microphones, and output the current sound speed.

[0029] In this way, the microphone array in this application can directly calculate the current sound velocity, thereby transmitting the current sound velocity to other devices, reducing the workload of other devices, and improving the usability of the microphone array. Furthermore, by estimating the current sound velocity using the sound signal picked up by the sound vector microphone, the current sound velocity can be calculated in real time, thus improving the accuracy of directional beam picking.

[0030] In one implementation of the first aspect, the processing unit is specifically used to determine j target acoustic vector microphone pairs from the N acoustic vector microphones; wherein the distance between the two acoustic vector microphones in each target acoustic vector microphone pair is in the range of 0.02 meters to 5 meters; j is an integer greater than 1 and less than or equal to C(2, N), wherein C(2, N) represents the number of combinations containing two acoustic vector microphones taken from the N acoustic vector microphones, and C(2, N) is an integer greater than 0 and less than N;

[0031] For each of the j target acoustic vector microphone pairs, based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair, calculate the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair and the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair; based on the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair, the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, calculate the target sound speed;

[0032] Calculate the current sound speed based on j target sound speeds and output the current sound speed.

[0033] In this way, by calculating the directions of arrival (DOA) of j target sound vector microphone pairs, the target sound velocity of the j target sound vector microphone pairs can be calculated. Furthermore, based on the target sound velocities, the current sound velocity can be estimated, improving the accuracy of the current sound velocity estimation. Moreover, in this application, the sound signals collected by the sound vector microphones can be directly used to estimate the current sound velocity using the above method, allowing for real-time calculation of the current sound velocity, thereby improving the accuracy of directional beam picking.

[0034] In one implementation of the first aspect, the distance between the two sound vector microphones in each target sound vector microphone pair is on the same order of magnitude as the reference distance; wherein, the reference distance refers to the distance between the target sound source and the center point of the microphone array.

[0035] This can further improve the accuracy of estimating the current sound speed, thereby improving the accuracy of directional beam picking.

[0036] In one implementation of the first aspect, the processing unit is further configured to acquire the direction of arrival of each of at least two omnidirectional microphones preset in advance;

[0037] A reference omnidirectional microphone and at least one enhanced omnidirectional microphone are identified from at least two omnidirectional microphones;

[0038] The signal delay of the at least one enhanced omnidirectional microphone is determined based on the direction of arrival of each of the at least two omnidirectional microphones, the distance between the reference omnidirectional microphone and the at least one enhanced omnidirectional microphone, and the current speed of sound.

[0039] Based on the signal delay of the at least one enhanced omnidirectional microphone and the sound signal picked up by the at least one enhanced omnidirectional microphone, the sound signal picked up by the reference omnidirectional microphone is enhanced to obtain an enhanced sound signal.

[0040] In this way, in the embodiments of this application, the microphone array can calculate the current sound speed in real time based on the sound signal picked up by the sound vector microphone, and perform directional beam signal enhancement based on the current sound speed, thereby improving the accuracy of directional beam picking when the environment changes.

[0041] Secondly, embodiments of this application provide a microphone array, including: N sound vector microphones for picking up sound signals from a target sound source;

[0042] Wherein, the distance between at least two of the N sound vector microphones is on the same order of magnitude as the reference distance; the reference distance refers to the distance between the center point of the microphone array and the target sound source; N is an integer greater than 1.

[0043] In this way, sound signals can be collected by a microphone array in this application so as to estimate the current sound speed. That is, the current sound speed can be estimated in real time in this application. Thus, even when the environment changes, the sound speed after the environment changes can also be estimated, which improves the accuracy of sound speed estimation. Therefore, when using sound speed for directional beam picking, the accuracy of directional beam picking can be improved.

[0044] Thirdly, embodiments of this application provide a sound velocity estimation method, applied to the microphone array described in any of the first aspects above, the method comprising:

[0045] Acquire the acoustic signals picked up by N vector microphones;

[0046] The current speed of sound is estimated based on the sound signals picked up by N sound vector microphones and the distance between the N sound vector microphones.

[0047] In this way, sound signals can be collected by a microphone array in this application so as to estimate the current sound speed. That is, the current sound speed can be estimated in real time in this application. Thus, even when the environment changes, the sound speed after the environment changes can also be estimated, which improves the accuracy of sound speed estimation. Therefore, when using sound speed for directional beam picking, the accuracy of directional beam picking can be improved.

[0048] In one implementation of the third aspect, estimating the current speed of sound based on the sound signals picked up by N sound vector microphones and the distance between the N sound vector microphones includes:

[0049] Among the N acoustic vector microphones, j target acoustic vector microphone pairs are determined; wherein the distance between the two acoustic vector microphones in the target acoustic vector microphone pair is in the range of 0.02 meters to 5 meters; j is an integer greater than 1 and less than or equal to C(2, N), where C(2, N) represents the number of combinations containing two acoustic vector microphones selected from the N acoustic vector microphones, and C(2, N) is an integer greater than 0 and less than N;

[0050] For each of the j target acoustic vector microphone pairs, based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair, calculate the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair and the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair; based on the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair, the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, calculate the target sound speed;

[0051] Estimate the current sound speed based on the sound speeds of j targets.

[0052] In this way, by calculating the directions of arrival (DOA) of j target sound vector microphone pairs, the target sound velocity of the j target sound vector microphone pairs can be calculated. Furthermore, based on the target sound velocities, the current sound velocity can be estimated, improving the accuracy of the current sound velocity estimation. Moreover, in this application, the sound signals collected by the sound vector microphones can be directly used to estimate the current sound velocity using the above method, achieving real-time calculation of the current sound velocity, thereby improving the accuracy of directional beam picking.

[0053] In one implementation of the third aspect, determining j target acoustic vector microphone pairs from the N acoustic vector microphones includes:

[0054] Determine the distance between any two acoustic vector microphones among the N acoustic vector microphones;

[0055] Two acoustic vector microphones within a distance range of 0.02 meters to 5 meters are identified as a target acoustic vector microphone pair, resulting in j target acoustic vector microphone pairs. This allows for a more accurate estimation of the speed of sound.

[0056] In one implementation of the third aspect, determining two acoustic vector microphones with a distance range of 0.02 meters to 5 meters as a target acoustic vector microphone pair, and obtaining j target acoustic vector microphone pairs includes:

[0057] Two sound vector microphones within a distance range of 0.02 meters to 5 meters and of the same order of magnitude as the reference distance are identified as a target sound vector microphone pair, resulting in j target sound vector microphone pairs; wherein, the reference distance refers to the distance between the center point of the microphone array and the target sound source, which makes the estimated sound speed more accurate.

[0058] In one implementation of the third aspect, calculating the direction of arrival (DOA) of each sound vector microphone in the target sound vector microphone pair based on the sound signal picked up by each sound vector microphone in the target sound vector microphone pair includes:

[0059] The acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair is transformed into a first frequency domain signal to obtain the acoustic signal of h×k time-frequency points of each acoustic vector microphone in the target acoustic vector microphone pair; where h represents the number of frequency points, which is a positive integer; and k represents the number of frames, which is a positive integer.

[0060] For each time-frequency point of the acoustic signal at each of the h×k time-frequency points of the target acoustic vector microphone pair, the sound intensity information at that time-frequency point is obtained based on the acoustic signal at that time-frequency point.

[0061] Calculate the direction of arrival at that time and frequency point based on the sound intensity information at that time and frequency point.

[0062] By converting the time-domain acoustic signal into the frequency-domain acoustic signal, the corresponding direction of arrival can be calculated more quickly and accurately, thus improving processing efficiency.

[0063] In one implementation of the third aspect, obtaining the sound intensity information at the time-frequency point based on the sound signal at that time-frequency point includes:

[0064] Based on the sound signal at that time and frequency point, using the formula Obtain the sound intensity information at this time-frequency point; wherein, the sound signal at the time-frequency point includes the sound pressure and the sound signal in the x-axis, y-axis, and z-axis directions; I x (f,n) represents the sound intensity information along the x-axis of a time-frequency point with frequency f and frame number n; y (f,n) represents the sound intensity information of a time-frequency point with frequency f and frame number n in the y-axis direction; z (f,n) represents the sound intensity information along the z-axis at a time frequency of frequency f and frame number n; X w *(f,n) represents the complex conjugate of the sound pressure at a time frequency of f and frame number n; X x (f,n) represents the acoustic signal with frequency f and frame number n along the x-axis; X y (f,n) represents the acoustic signal with frequency f and frame number n along the y-axis; X z (f,n) represents the acoustic signal with frequency f and frame number n in the z-axis direction; f is an integer greater than 0 and not greater than h; n is an integer greater than 0 and not greater than k.

[0065] The step of calculating the direction of arrival at a given time-frequency point based on the sound intensity information at that time-frequency point includes:

[0066] Based on the sound intensity information at that time and frequency point, using the formula Calculate the direction of arrival at this time-frequency point; where, φ represents the real part. (f，n) θ represents the horizontal angle of a time frequency point with frequency f and frame number n. (f，n) The elevation angle represents the time frequency point f and the number of frames n.

[0067] This allows for faster and more accurate calculation of the direction of arrival at each time frequency point, improving processing efficiency.

[0068] In one implementation of the third aspect, before calculating the direction of arrival at the time-frequency point based on the sound intensity information at that time-frequency point, the method further includes:

[0069] Based on the sound signal at that time and frequency point, determine whether the sound signal at that time and frequency point is a speech signal;

[0070] The step of calculating the direction of arrival at a given time-frequency point based on the sound intensity information at that time-frequency point includes:

[0071] When the sound signal at the specified time and frequency point is a speech signal, the direction of arrival at that time and frequency point is calculated based on the sound intensity information at that time and frequency point.

[0072] In this way, the direction of arrival (DOA) at a given time-frequency point is calculated only when the sound signal is determined to be a speech signal, based on the sound intensity information at that time-frequency point. If the sound signal at that time-frequency point is determined to be a noise signal, the DOA is not calculated. This significantly reduces the computational workload and the consumption of hardware and software resources.

[0073] In one implementation of the third aspect, the step of calculating the acoustic signal pickup delay between two acoustic vector microphones in the target acoustic vector microphone pair based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair includes:

[0074] The acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair is transformed into a second frequency domain signal to obtain k frames of second frequency domain acoustic signal for each acoustic vector microphone in the target acoustic vector microphone pair;

[0075] Based on the k frames of second frequency domain acoustic signals from each acoustic vector microphone in the target acoustic vector microphone pair, the time delay when the two acoustic vector microphones in the target acoustic vector microphone pair pick up each frame of second frequency domain acoustic signal is calculated using a preset time delay algorithm.

[0076] By converting the time-domain acoustic signal into the frequency-domain acoustic signal, the time delay of the acoustic signal can be estimated more quickly and accurately, thus improving processing efficiency.

[0077] In one implementation of the third aspect, the preset time delay algorithm includes: a generalized cross-correlation algorithm, a generalized cross-correlation-phase transformation algorithm, or an interpolation-based time delay estimation algorithm, which can estimate the time delay of the acoustic signal faster and more accurately, thereby improving processing efficiency.

[0078] In one implementation of the third aspect, the step of calculating the target sound speed based on the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair, the time delay of the acoustic signal picked up between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair includes:

[0079] Based on the direction of arrival (DOA) of each of the h×k time-frequency points of the target acoustic vector microphone pair, the time delay of the second frequency domain acoustic signal picked up by the two acoustic vector microphones in the target acoustic vector microphone pair for each frame, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, the formula is used to... Calculate the target sound velocity at the corresponding time-frequency point of the target sound vector microphone pair; where c (f，n) τ represents the target sound speed; d represents the distance between the two sound vector microphones in the target sound vector microphone pair; n θ represents the time delay when the two acoustic vector microphones in the target acoustic vector microphone pair pick up the second frequency domain acoustic signal of the nth frame; 1(f，n) θ represents the elevation angle of the time-frequency point f of one of the target vector microphones in the target vector microphone pair, with a frame number of n. 2(f，n) This represents the elevation angle of the frequency point f and the frame number n of the other acoustic vector microphone in the target acoustic vector microphone pair.

[0080] In this embodiment of the application, after calculating the direction of arrival of the two acoustic vector microphones and the time delay between the two acoustic vector microphones by using the acoustic signals picked up by the two acoustic vector microphones in the target acoustic vector microphone pair, the target sound speed is calculated using the direction of arrival of the two acoustic vector microphones and the time delay between the two acoustic vector microphones. Then, the current sound speed can be calculated. This allows the sound speed to be estimated even when the environment changes, improving the accuracy of sound speed estimation. As a result, the accuracy of directional beam picking can be improved when using sound speed for directional beam picking.

[0081] In one implementation of the third aspect, estimating the current sound speed based on j target sound speeds includes:

[0082] Calculate the mean of j target sound speeds and determine the mean of the j target sound speeds as the current sound speed, which can improve the accuracy of the current sound speed estimation.

[0083] Fourthly, embodiments of this application provide a sound pickup method applied to a microphone array as described in any of the first aspects above. The method includes: acquiring sound signals picked up by N sound vector microphones in the microphone array, and acquiring the current speed of sound based on the sound signals picked up by the N sound vector microphones.

[0084] Obtain the direction of arrival of each omnidirectional microphone in a pre-set microphone array; wherein the microphone array contains at least two omnidirectional microphones;

[0085] A reference omnidirectional microphone and at least one enhanced omnidirectional microphone are identified from at least two omnidirectional microphones;

[0086] The signal delay of the at least one enhanced omnidirectional microphone is determined based on the direction of arrival of each of the at least two omnidirectional microphones, the distance between the reference omnidirectional microphone and the at least one enhanced omnidirectional microphone, and the current speed of sound.

[0087] Based on the signal delay of the at least one enhanced omnidirectional microphone and the sound signal picked up by the at least one enhanced omnidirectional microphone, the sound signal picked up by the reference omnidirectional microphone is enhanced to obtain an enhanced sound signal.

[0088] In this way, when performing directional sound velocity pickup, the current sound velocity can be estimated first based on the sound signal picked up by the sound vector microphones in the microphone array. After estimating the current sound velocity, the signal delay of the omnidirectional microphone in the microphone array can be calculated using the estimated current sound velocity and the direction of arrival of the omnidirectional microphone in the microphone array. Then, based on the calculated signal delay of the omnidirectional microphone, the sound signal picked up by the omnidirectional microphone can be enhanced to obtain an enhanced sound signal, thus achieving the purpose of directional beam pickup. Since the current sound velocity is estimated in real time, rather than a preset value, the accuracy of the omnidirectional microphone signal delay estimation can be improved, thereby improving the accuracy of directional beam pickup.

[0089] Fifthly, embodiments of this application provide a sound velocity estimation device, comprising:

[0090] The acquisition unit is used to acquire the acoustic signals picked up by N sound vector microphones;

[0091] The processing unit is used to estimate the current speed of sound based on the sound signals picked up by N sound vector microphones and the distance between the N sound vector microphones.

[0092] In this way, sound signals can be collected by a microphone array in this application so as to estimate the current sound speed. That is, the current sound speed can be estimated in real time in this application. Thus, even when the environment changes, the sound speed after the environment changes can also be estimated, which improves the accuracy of sound speed estimation. Therefore, when using sound speed for directional beam picking, the accuracy of directional beam picking can be improved.

[0093] Sixthly, embodiments of this application provide a sound pickup device, including:

[0094] The acquisition unit is used to acquire the sound signals picked up by N sound vector microphones in the microphone array, and to acquire the current sound speed based on the sound signals picked up by the N sound vector microphones;

[0095] The acquisition unit is also used to acquire the direction of arrival of each omnidirectional microphone in a pre-set microphone array; wherein the microphone array includes at least two omnidirectional microphones;

[0096] A determining unit is used to determine a reference omnidirectional microphone and at least one enhanced omnidirectional microphone among at least two omnidirectional microphones;

[0097] The determining unit is further configured to determine the signal delay of the at least one enhanced omnidirectional microphone based on the direction of arrival of each of the at least two omnidirectional microphones, the distance between the reference omnidirectional microphone and the at least one enhanced omnidirectional microphone, and the current speed of sound;

[0098] The processing unit is configured to perform enhancement processing on the sound signal picked up by the reference omnidirectional microphone based on the signal delay of the at least one enhanced omnidirectional microphone and the sound signal picked up by the at least one enhanced omnidirectional microphone, to obtain an enhanced sound signal.

[0099] In this way, when performing directional sound velocity pickup, the current sound velocity can be estimated first based on the sound signal picked up by the sound vector microphones in the microphone array. After estimating the current sound velocity, the signal delay of the omnidirectional microphone in the microphone array can be calculated using the estimated current sound velocity and the direction of arrival of the omnidirectional microphone in the microphone array. Then, based on the calculated signal delay of the omnidirectional microphone, the sound signal picked up by the omnidirectional microphone can be enhanced to obtain an enhanced sound signal, thus achieving the purpose of directional beam pickup. Since the current sound velocity is estimated in real time, rather than a preset value, the accuracy of the omnidirectional microphone signal delay estimation can be improved, thereby improving the accuracy of directional beam pickup.

[0100] In a seventh aspect, embodiments of this application provide an electronic device, including: a microphone array as described in any of the first or second aspects above.

[0101] Thus, by including a microphone array as described in any of the first or second aspects of the above-mentioned electronic device, the electronic device can pick up the sound signal of the target sound source through the sound vector microphones included in the microphone array, and then estimate the current sound speed based on the sound signal picked up by the sound vector microphones. When the environment changes, the sound signal of the target sound source picked up by the sound vector microphones included in the microphone array of this application embodiment can be obtained in real time, improving the accuracy of sound speed acquisition, thereby improving the accuracy of directional beam picking.

[0102] In one implementation of the seventh aspect, the electronic device includes a vehicle or terminal device, which allows the microphone array to be applied to different devices, thereby improving the usability of the microphone array.

[0103] In one implementation of the seventh aspect, when the electronic device includes a vehicle, the distance between at least two of the N sound vector microphones in the microphone array is in the range of 0.2 meters to 2 meters, so that when the microphone array is applied to the vehicle, the current sound speed can be estimated more accurately, and the accuracy of directional beam picking can be improved when performing directional beam picking.

[0104] In one implementation of the seventh aspect, when the electronic device includes a terminal device, the distance between at least two of the N sound vector microphones in the microphone array is in the range of 0.02 meters to 5 meters, so that when the microphone array is applied to the terminal device, the current sound speed can be estimated more accurately, and the accuracy of directional beam picking can be improved when performing directional beam picking.

[0105] Eighthly, embodiments of this application provide an electronic device, including a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method described in any of the third aspects above or to execute the method described in any of the fourth aspects above.

[0106] Ninthly, embodiments of this application provide a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the method described in any of the third aspects above, or to perform the method described in any of the fourth aspects above.

[0107] The solution provided in this application includes N sound vector microphones, wherein at least two of the N sound vector microphones are spaced within a range of 0.02 meters to 5 meters apart. The N sound vector microphones are used to pick up the sound signal from a target sound source, and the picked-up sound signal is used to estimate the current speed of sound. That is, in this application embodiment, the sound signal from the target sound source can be picked up by the sound vector microphones included in the microphone array, and the current speed of sound can be estimated based on the sound signal picked up by the sound vector microphones. In this way, when the environment changes, the current speed of sound can be obtained in real time by picking up the sound signal from the target sound source through the sound vector microphones included in the microphone array of this application embodiment, thus improving the accuracy of speed of sound acquisition. Attached Figure Description

[0108] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0109] Figure 1 This is a schematic diagram of the structure of a microphone array provided in an embodiment of this application;

[0110] Figure 2 This is a schematic diagram of the structure of a sound vector microphone provided in an embodiment of this application;

[0111] Figure 3 A schematic diagram of a microphone array provided in an embodiment of this application;

[0112] Figure 4 A schematic diagram illustrating another microphone array scenario provided in an embodiment of this application;

[0113] Figure 5 This is a schematic diagram of another microphone array structure provided in an embodiment of this application;

[0114] Figure 6 This is a schematic diagram of another microphone array structure provided in an embodiment of this application;

[0115] Figure 7 This is a schematic diagram of another microphone array structure provided in an embodiment of this application;

[0116] Figure 8 This is a schematic diagram of another microphone array structure provided in an embodiment of this application;

[0117] Figure 9 This is a schematic diagram of another microphone array structure provided in an embodiment of this application;

[0118] Figure 10 This is a schematic diagram of another microphone array structure provided in an embodiment of this application;

[0119] Figure 11 This is a schematic diagram of another microphone array structure provided in an embodiment of this application;

[0120] Figure 12 This is a schematic diagram of another microphone array structure provided in an embodiment of this application;

[0121] Figure 13 This is a schematic diagram of another microphone array structure provided in an embodiment of this application;

[0122] Figure 14 This is a schematic diagram of another microphone array structure provided in an embodiment of this application;

[0123] Figure 15 A schematic flowchart illustrating a sound velocity estimation method provided in an embodiment of this application;

[0124] Figure 16 A schematic diagram of a sound speed estimation method provided in an embodiment of this application;

[0125] Figure 17 A schematic diagram of a scenario for another sound speed estimation method provided in an embodiment of this application;

[0126] Figure 18a A schematic diagram of a scenario for another sound speed estimation method provided in an embodiment of this application;

[0127] Figure 18b A schematic diagram of a scenario for another sound speed estimation method provided in an embodiment of this application;

[0128] Figure 19 A schematic diagram of a scenario for another sound speed estimation method provided in an embodiment of this application;

[0129] Figure 20a A schematic diagram of a scenario for another sound speed estimation method provided in an embodiment of this application;

[0130] Figure 20b A schematic diagram of a scenario for another sound speed estimation method provided in an embodiment of this application;

[0131] Figure 21a A schematic diagram of a scenario for another sound speed estimation method provided in an embodiment of this application;

[0132] Figure 21b A schematic diagram of a scenario for another sound speed estimation method provided in an embodiment of this application;

[0133] Figure 22a A schematic diagram of a scenario for another sound speed estimation method provided in an embodiment of this application;

[0134] Figure 22b A schematic diagram of a scenario for another sound speed estimation method provided in an embodiment of this application;

[0135] Figure 23 A schematic flowchart illustrating a sound pickup method provided in an embodiment of this application;

[0136] Figure 24 A schematic diagram of a sound pickup method provided in an embodiment of this application.

[0137] Figure 25 This is a schematic diagram of the structure of a sound velocity estimation device provided in an embodiment of this application;

[0138] Figure 26 This is a schematic diagram of another sound velocity estimation device provided in an embodiment of this application;

[0139] Figure 27 This is a schematic diagram of the structure of a sound pickup device provided in an embodiment of this application;

[0140] Figure 28 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0141] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0142] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0143] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0144] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0145] The sound velocity estimation method and sound pickup method provided in this application can be applied to electronic devices. The electronic device can be any device with voice interaction capabilities, including but not limited to smart speakers, smart home appliances, smartphones, tablets, in-vehicle devices, wearable devices, and augmented reality (AR) / virtual reality (VR) devices. Specifically, the sound velocity estimation method and sound pickup method provided in this application can be stored in the electronic device as an application program or software. The electronic device implements the sound velocity estimation method and sound pickup method provided in this application by executing the application program or software.

[0146] The microphone device described in this application embodiment can also be applied to the aforementioned electronic devices.

[0147] A microphone is an energy conversion device that converts sound signals into electrical signals. A microphone array refers to the arrangement of microphones; that is, a microphone array consists of a certain number of microphones used to sample and process the control characteristics of a sound field. Microphone arrays include line arrays, cross arrays, planar arrays, spiral arrays, spherical arrays, and random arrays, among others. The number of array elements (i.e., the number of microphones) can range from two to over one. Due to cost constraints, consumer-grade microphone arrays generally do not exceed eight elements; common are six-element and four-element microphone arrays. Furthermore, some microphone arrays typically include omnidirectional microphones.

[0148] In some technologies, directional beamforming can be used to better acquire speech signals and suppress noise. During directional beamforming, the sound signal picked up by the microphone array is amplified in a specific direction by using the direction of arrival (DOA) of the microphone array, the distance between the microphones in the array, and the current speed of sound, thus achieving the purpose of directional beamforming. Since the DOA and the distance between the microphones in the array are preset, the accuracy of directional beamforming mainly depends on the estimation of the speed of sound. Currently, common methods for measuring the speed of sound mainly include indirect measurement methods based on empirical formulas and direct measurement methods based on active sound generation.

[0149] Indirect methods for measuring sound speed primarily utilize the functional relationship between the speed of sound in air and environmental temperature, humidity, and pressure. This involves measuring the ambient temperature, humidity, and pressure using instruments, and then calculating the speed of sound using empirical formulas. Generally, ambient temperature has the strongest correlation with the speed of sound; the higher the temperature, the greater the speed of sound. Commonly used empirical formulas are as follows: Where c represents the speed of sound, t represents the ambient temperature, and t0 is a preset temperature reference value. Because the speed of sound is affected by many factors in practical applications, it is impossible to calculate the speed of sound quickly and accurately when the environment changes. Therefore, the accuracy of the speed of sound measured by the above indirect measurement method is not high.

[0150] The direct method for measuring the speed of sound involves actively emitting sound. For example, after synchronizing the clocks of the transmitting and receiving ends, the transmitting end emits a sound signal, and the receiving end receives the sound signal. The propagation time of the sound signal is determined by the time it takes for the sound to be emitted and received. The speed of sound can then be estimated based on the distance between the transmitting and receiving ends and the sound signal propagation time. This method requires a specific sound-generating device to perform the sound speed test, making it relatively complex to implement.

[0151] To address the aforementioned problems, this application provides a microphone array comprising N vector acoustic microphones, wherein at least two of the N vector acoustic microphones are spaced within a distance range of 0.02 meters to 5 meters; the N vector acoustic microphones are used to pick up the acoustic signal of a target sound source, wherein the picked-up acoustic signal of the target sound source is used to estimate the current speed of sound. That is, in this application embodiment, the acoustic signal of the target sound source can be picked up by the vector acoustic microphones included in the microphone array, and the current speed of sound can then be estimated based on the picked-up acoustic signal. Thus, when the environment changes, the current speed of sound can be obtained in real time by picking up the acoustic signal of the target sound source using the vector acoustic microphones included in the microphone array of this application embodiment, improving the accuracy of speed of sound acquisition. Furthermore, the speed of sound can be estimated without setting up a dedicated sound-emitting end, making it simple and convenient to implement. A detailed description follows.

[0152] See Figure 1 This is a schematic diagram of a microphone array provided in an embodiment of this application. Figure 1 As shown, the microphone array includes N sound vector microphones 11; wherein the distance between at least two of the N sound vector microphones 11 is in the range of 0.02 meters to 5 meters. N is an integer greater than 1.

[0153] N sound vector microphones 11 are used to pick up the sound signal from the target sound source. The picked-up sound signal from the target sound source is used to estimate the current speed of sound.

[0154] It should be noted that the sound vector microphone 11, also known as a sound vector sensor, can simultaneously measure the sound pressure and particle velocity at a single point in a sound field. It is composed of a sound pressure sensor, a first particle velocity sensor, a second particle velocity sensor, and a third particle sensor. The sound pressure sensor measures the sound pressure of the sound field; the first particle velocity sensor measures the particle vibration velocity in a first direction in the sound field; the second particle sensor measures the particle vibration velocity in a second direction in the sound field; and the third particle sensor measures the particle vibration velocity in a third direction in the sound field. The first, second, and third directions are three mutually orthogonal directions in space. Therefore, in this embodiment, the sound vector microphone 11 can simultaneously measure the sound pressure and three mutually orthogonal velocity components at a single point in the sound field, such as... Figure 2 As shown.

[0155] It should be noted that, in the embodiments of this application, when the first direction, the second direction, and the third direction are represented by coordinate axes, the direction pointed to by the x-axis is typically used to represent the first direction, the direction pointed to by the y-axis to represent the second direction, and the direction pointed to by the z-axis to represent the third direction. (Refer to...) Figure 3 As shown.

[0156] Assume that the two-dimensional wave direction of arrival between the sound source S and the spatial point r at time t is (θ, φ), where θ is the pitch angle. φ is a horizontal angle, φ∈[0, 2π), reference Figure 3 As shown. At this time, the unit direction vector of the sound vector microphone. Let v(r, t) represent the particle velocity at time t at point r in space, and p(r, t) represent the sound pressure at that point at time t, then we have Where ρ0 represents the density of the medium, and c represents the speed of sound in the medium, expressed by the Euler method. For plane waves Operator is equivalent to so Since sound pressure and vibration velocity are only related to time t, the spatial point r in the expression can be omitted, hence:

[0157] From the above formula, it can be seen that, ignoring noise, the signal output by the sound pressure sensor in the sound vector microphone 11 is: x p (t) = p(t), where p(t) represents the sound pressure signal detected by the sound pressure sensor. Similarly, the signal output by the particle velocity sensor in the sound vector microphone 11 is: x vv (t)=p(t) * μ. Where x p (t) represents the signal output by the sound pressure sensor, x vv (t) represents the signal output by the particle velocity sensor.

[0158] The acoustic signal output by the aforementioned vector microphone 11 can be expressed in vector form as follows: Therefore, the guide vector of the sound vector microphone 11 is...

[0159] Where, x v (t) represents the acoustic signal output by the acoustic vector microphone 11. Since the signals output by the sound pressure sensor and the three particle velocity sensors in the acoustic vector microphone 11 satisfy the above formula, the DOA (Direction of Arrival) θ and φ of the acoustic vector microphone can be calculated from the sound pressure signal detected by the sound pressure sensor and the particle velocity vector signal detected by the three particle velocity sensors.

[0160] Assume there are two sound vector microphones 11, AVS1 and AVS2, with a distance d between them. The pitch angle between AVS1 and the target sound source is θ1, and the pitch angle between AVS2 and the target sound source is θ2. Let l be the path difference between the waves reaching AVS1 and AVS2, and l = τc, where τ is an estimate of the time delay of the received signals by AVS1 and AVS2, and c is the current speed of sound to be estimated. (Reference) Figure 4 As shown, in the triangle formed by the target sound source, sound vector microphone AVS1, and sound vector microphone AVS2, S represents the position of the target sound source, o1 represents the position of AVS1, o3 represents the position of AVS2, and o4 represents the position of AVS1 within triangle so1o3. Side so1 represents the path distance of the wave to AVS1, side so3 represents the path distance of the wave to AVS2, side o1o2 represents the difference in path distance between AVS1 and AVS2, side o1o3 represents the distance between AVS1 and AVS2, and o4 represents the midpoint between AVS1 and AVS2. Angle so1o3 is the pitch angle θ1 between AVS1 and the target sound source, θ3 represents angle o1so3, and the supplementary angle of angle so3o1 is the pitch angle θ2 between AVS2 and the target sound source. midIndicates angle so mid o3. In triangle o1o2o3, φ1 represents angle o1o2o3, φ2 represents angle o1o3o2, and side o1o2 represents the path difference between AVS1 and AVS2. In triangle so2o3, φ′1 represents angle so2o3 and is the supplementary angle to angle o1o2o3, and φ′2 represents angle so3o2. Using geometric relationships, φ1 and φ2 are expressed using estimated values ​​θ1 and θ2:

[0161] θ3=θ2-θ1

[0162]

[0163] φ1=π-φ'1

[0164] φ2=π-(φ'2+θ2)

[0165]

[0166]

[0167] Based on the triangle sine theorem, we obtain:

[0168]

[0169] Based on the known value d and the estimated values ​​τ, θ1, θ2, the speed of sound c can be estimated:

[0170] Therefore, by calculating the pitch angle θ1 using AVS1 and the pitch angle θ2 using AVS2, the speed of sound can be estimated. Thus, in this embodiment, the sound signals from the target sound source picked up by the N sound vector microphones 11 can be used to estimate the current speed of sound.

[0171] In this embodiment, to measure the current speed of sound, at least two acoustic vector microphones 11 are arranged in the microphone array, i.e., N acoustic vector microphones 11 are arranged. Since if the distance between two acoustic vector microphones 11 is too close, the time it takes for them to pick up the sound signal is essentially the same, resulting in zero time delay between them, and thus a large error in estimating the current speed of sound. Conversely, if the distance between two acoustic vector microphones 11 is too large, the pitch angle of the sound source relative to the two microphones is essentially the same, leading to a large error in estimating the current speed of sound from the sound signal output by the acoustic vector microphones 11. Therefore, to ensure the accuracy of the estimated current speed of sound, the distance between at least two of the N acoustic vector microphones 11 is within the range of 0.02 meters to 5 meters.

[0172] As one possible approach, to more accurately estimate the current speed of sound, the distance between at least two of the N sound vector microphones can be set within different ranges depending on the application scenario of the microphone array. For example, when the microphone array is applied to a vehicle, the distance between at least two of the N sound vector microphones is in the range of 0.2 meters to 2 meters. When the microphone array is applied to a terminal device, such as a speaker, the distance between at least two of the N sound vector microphones is in the range of 0.5 meters to 5 meters.

[0173] As one possible implementation, at least two of the N sound vector microphones 11 are spaced between 0.02 meters and 5 meters apart, and are on the same order of magnitude as the reference distance. The reference distance is the distance between the center point of the microphone array and the target sound source.

[0174] It should be noted that, in the context of "at least two of the N acoustic vector microphones 11 having a distance on the same order of magnitude as the reference distance," the ratio of the reference distance to the target distance is greater than 0 and not greater than 10. Here, the target distance is the distance between at least two of the N acoustic vector microphones 11.

[0175] In this embodiment, to measure the current sound velocity, at least two acoustic vector microphones 11 are arranged in the microphone array, i.e., N acoustic vector microphones 11 are arranged. The distance between at least two acoustic vector microphones 11 among the N acoustic vector microphones 11 is on the same order of magnitude as the reference distance. Since when the distance between two acoustic vector microphones 11 is on a different order of magnitude from the reference distance, for example, when the distance between two acoustic vector microphones 11 is much smaller than the distance between the center point of the microphone array and the target sound source, the pitch angle of the sound source relative to the two acoustic vector microphones 11 is basically the same. In this case, the current sound velocity estimated by the sound signal output by the acoustic vector microphones 11 has a large error. Therefore, in order to ensure the accuracy of the estimated current sound velocity, it is necessary to ensure that the distance between at least two acoustic vector microphones 11 among the N acoustic vector microphones is on the same order of magnitude as the distance between the center point of the microphone array and the target sound source.

[0176] As one possible implementation, when the distance between at least two of the N acoustic vector microphones 11 in the microphone array is on the same order of magnitude as the reference distance, and this distance is fixed after it has been determined, the reference distance can be adjusted to ensure that the distance between at least two of the N acoustic vector microphones 11 in the microphone array is on the same order of magnitude as the reference distance. That is, the distance between the N acoustic vector microphones 11 in the microphone array is set and cannot be changed. In this case, the distance between at least two of the N acoustic vector microphones 11 in the microphone array is within the range of 0.02 meters to 5 meters, for example, 0.05 meters, 0.1 meters, 0.5 meters, 1 meter, 2 meters, or 3 meters, etc. The reference distance can be set based on the distance between at least two of the N acoustic vector microphones 11 in the microphone array, which is the distance between the target sound source and the center point of the microphone array. For example, in a microphone array, if the distance between at least two microphones is 1 meter, the distance between the target sound source and the center point of the microphone array can be in the range of 0 to 10 meters. To more accurately estimate the speed of sound, the distance between the target sound source and the center point of the microphone array can be set within the range of 0.5 to 1.5 meters. That is, in this method, the reference distance is adjusted based on the distance between two sound vector microphones in the microphone array, so that the distance between the two sound vector microphones is of the same order of magnitude as the reference distance.

[0177] As one possible implementation, the distance between the N acoustic vector microphones 11 in the microphone array can be adjusted according to a reference distance. That is, given a fixed reference distance, the distance between at least two of the N acoustic vector microphones in the microphone array can be adjusted so that the adjusted distance between at least two acoustic vector microphones 11 is on the same order of magnitude as the reference distance. For example, when the reference distance is 1 meter, the distance between at least two of the N acoustic vector microphones 11 can be set to be greater than 0.1 meters. To more accurately estimate the current sound speed, the distance between at least two of the N acoustic vector microphones 11 can be set within the range of 0.7 meters to 1 meter. Alternatively, when the reference distance is within the range of 0.1 meters to 0.5 meters, to more accurately estimate the current sound speed, the distance between at least two of the N acoustic vector microphones 11 can be set within the range of 0.3 meters to 1 meter.

[0178] It should be noted that, in this embodiment of the application, among the N sound vector microphones 11 included in the microphone array, it is only necessary to ensure that the distance between at least two sound vector microphones 11 is within the range of 0.02 meters to 5 meters. There are no restrictions on the specific arrangement of the N sound vector microphones 11 in the microphone array. The possible arrangements are as follows:

[0179] As one possible implementation, N sound vector microphones 11 are located in the same plane, such as... Figures 5-9 As shown.

[0180] As one possible implementation, N sound vector microphones 11 are arranged linearly. For example, when N is 4, 4 sound vector microphones 11 are arranged linearly, as shown in the reference. Figure 5 and Figure 6 As shown, N sound vector microphones 11 are arranged in a preset order. The linear arrangement of N sound vector microphones 11 is relatively simple to implement.

[0181] As one possible implementation, in N vector microphones 11, the distance between each vector microphone 11 and its adjacent vector microphone 11 is the same. For example, when N is 4, the distance between each of the 4 vector microphones 11 and its adjacent vector microphone 11 is the same. (Refer to...) Figure 5 As shown, N sound vector microphones 11 are arranged at equal intervals. That is, the spacing between any two adjacent sound vector microphones 11 in the N sound vector microphones 11 is equal.

[0182] As one possible implementation, the distances between any two adjacent acoustic vector microphones 11 among the N acoustic vector microphones 11 are not exactly equal. For example, when N is 4, the distances between any two adjacent acoustic vector microphones 11 among the four acoustic vector microphones 11 are d1, d2, and d3, respectively. (Reference) Figure 6 As shown, d1, d2, and d3 are not equal. That is, the distances between any two adjacent acoustic vector microphones 11 among the N acoustic vector microphones 11 are not equal. In other words, the N acoustic vector microphones 11 are not evenly spaced.

[0183] As one possible implementation, N vector microphones 11 include a first vector microphone 111, a second vector microphone 112, and a third vector microphone 113. The angle between the line containing the third vector microphone 113 and the first vector microphone 111 and the line containing the first vector microphone 111 and the second vector microphone 112 is greater than 0 degrees and less than 180 degrees. (Refer to...) Figure 7As shown. That is, N vector acoustic microphones 11 are in the same plane, but not linearly arranged. There exists at least a first vector acoustic microphone 111, a second vector acoustic microphone 112, and a third vector acoustic microphone 113. The angle between the line containing the third vector acoustic microphone 113 and the first vector acoustic microphone 111 and the line containing the first vector acoustic microphone 111 and the second vector acoustic microphone 112 is greater than 0 degrees and less than 180 degrees. In other words, the first vector acoustic microphone 111, the second vector acoustic microphone 112, and the third vector acoustic microphone 113 are not on the same straight line. For example, the N vector acoustic microphones 11 can be arranged in a polygon, a circle, or other two-dimensional arrangement, as shown in the reference. Figure 8 As shown.

[0184] As one possible implementation, N sound vector microphones 11 are arranged in a three-dimensional configuration, such as... Figure 9 As shown, N sound vector microphones 11 are positioned along the x, y, and z axes. This allows for stereo acquisition of the sound signal from the target sound source, improving the accuracy of sound velocity estimation.

[0185] As one possible implementation, the N sound vector microphones 11 are arranged in a regular tetrahedral or a triangular pyramidal arrangement, such as... Figure 10 As shown.

[0186] It should be noted that the arrangement of the N acoustic vector microphones 11 in the microphone array can also be in other forms, such as spiral arrangement or spherical arrangement, as long as the distance between at least two acoustic vector microphones 11 is on the same order of magnitude as the reference distance. This application does not impose any restrictions on this.

[0187] As one possible implementation, the microphone array described above also includes at least two omnidirectional microphones 12, such as... Figure 11 As shown.

[0188] Among them, the omnidirectional microphone 12 is an omnidirectional microphone used to pick up the sound signal of the target sound source.

[0189] In this embodiment of the application, an omnidirectional microphone 12 is provided in the microphone array, which can pick up the sound signal of the target sound source and use the sound signal for voice output.

[0190] As one possible implementation, at least two omnidirectional microphones 12 are located between N sound vector microphones 11. For example, the N sound vector microphones 11 are arranged on the extension line of at least two omnidirectional microphones 12.

[0191] As one possible implementation, the microphone array described above also includes: a processing unit 13, such as... Figure 12 As shown.

[0192] The processing unit 13 is used to acquire the sound signals picked up by N sound vector microphones 11, calculate the current sound speed based on the sound signals picked up by the N sound vector microphones 11 and the distance between the N sound vector microphones 11, and output the current sound speed.

[0193] In this embodiment of the application, after the processing unit 13 acquires the sound signals picked up by N sound vector microphones 11, it can calculate the direction of arrival of each of the N sound vector microphones based on the sound signals picked up by the N sound vector microphones 11, and estimate the time delay of the sound signals picked up between the N sound vector microphones 11 based on the sound signals picked up by the N sound vector microphones 11. Thus, based on the direction of arrival of the N sound vector microphones, the time delay of the sound signals picked up between the sound vector microphones 11, and the distance between the N sound vector microphones, the current speed of sound is calculated and the current speed of sound is output.

[0194] As one possible implementation, processing unit 13 is specifically used to determine j target acoustic vector microphone pairs from N acoustic vector microphones. The distance between the two acoustic vector microphones in each target acoustic vector microphone pair is within the range of 0.02 meters to 5 meters; j is an integer greater than 1 and less than or equal to C(2, N), where C(2, N) represents the number of combinations containing two acoustic vector microphones selected from the N acoustic vector microphones, and C(2, N) is an integer greater than 0 and less than N.

[0195] For each of the j target acoustic vector microphone pairs, based on the acoustic signals picked up by each acoustic vector microphone in the target acoustic vector microphone pair, calculate the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair and the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair; based on the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair, the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, calculate the target sound speed;

[0196] Calculate the current sound speed based on j target sound speeds and output the current sound speed.

[0197] In other words, since not every pair of N acoustic vector microphones 11 is within the range of 0.02 meters to 5 meters, it is necessary to select j target acoustic vector microphone pairs from the N acoustic vector microphones 11. Each target acoustic vector microphone pair contains two acoustic vector microphones 11 whose distance is within the range of 0.02 meters to 5 meters.

[0198] As one possible implementation, the processing unit 13 can select the target acoustic vector microphone pair j from the N acoustic vector microphones 11 based on the distance between each pair of acoustic vector microphones.

[0199] In the N acoustic vector microphones 11, the processing unit 13 needs to calculate the distance between each acoustic vector microphone and the other N-1 acoustic vector microphones, so as to determine which two acoustic vector microphones are within the distance range of 0.02 meters to 5 meters, thereby identifying j target acoustic vector microphone pairs among the N acoustic vector microphones.

[0200] As one possible implementation, the distance between the two acoustic vector microphones in each target acoustic vector microphone pair is on the same order of magnitude as the reference distance. The reference distance refers to the distance between the target sound source and the center point of the microphone array. That is, among N acoustic vector microphones, two microphones whose distance is between 0.02 meters and 5 meters, and whose distance is on the same order of magnitude as the reference distance, are defined as a target acoustic vector microphone pair, thus identifying j target acoustic vector microphone pairs.

[0201] In determining j target acoustic vector microphone pairs, processing unit 13, for each target acoustic vector microphone pair, calculates the direction of arrival (DOA) of each acoustic vector microphone in the target acoustic vector microphone pair based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair, and calculates the time delay between the acoustic signals picked up by the two acoustic vector microphones in the target acoustic vector microphone pair based on the acoustic signals picked up by the two acoustic vector microphones in the target acoustic vector microphone pair. Therefore, based on the DOA of the two acoustic vector microphones in the target acoustic vector microphone pair, the time delay between the acoustic signals picked up by the two acoustic vector microphones, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, it can use the formula... Calculate the target sound speed. Where c represents the target sound speed, d is the distance between the two sound vector microphones in the target sound vector microphone pair, τ is the time delay between the sound signals picked up by the two sound vector microphones in the target sound vector microphone pair, θ1 is the pitch angle in the direction of arrival of one sound vector microphone in the target sound vector microphone pair, and θ2 is the pitch angle in the direction of arrival of the other sound vector microphone in the target sound vector microphone pair.

[0202] The above method can be used to calculate a target sound speed for each target sound vector microphone pair, thereby calculating j target sound speeds. After calculating j target sound speeds, the current sound speed can be calculated based on the j target sound speeds.

[0203] As one possible implementation, after calculating the j target sound speeds, the processing unit 13 can calculate the average of the j target sound speeds and use the calculated average as the current sound speed.

[0204] As one possible implementation, the processing unit 13 is further configured to acquire the direction of arrival of each of at least two omnidirectional microphones preset in a pre-defined manner; identify a reference omnidirectional microphone and at least one enhanced omnidirectional microphone among the at least two omnidirectional microphones; determine the signal delay of at least one enhanced omnidirectional microphone based on the direction of arrival of each of the at least two omnidirectional microphones, the distance between the reference omnidirectional microphone and at least one enhanced omnidirectional microphone, and the current sound speed; and enhance the sound signal picked up by the reference omnidirectional microphone based on the signal delay of at least one enhanced omnidirectional microphone and the sound signal picked up by at least one enhanced omnidirectional microphone to obtain an enhanced sound signal.

[0205] In this embodiment, when the microphone array includes at least two omnidirectional microphones, the current sound speed can be calculated based on the sound signals picked up by N sound vector microphones. Then, using the calculated current sound speed, directional beamforming is applied to the sound signals picked up by the at least two omnidirectional microphones to achieve directional beamforming. When the at least two omnidirectional microphones are installed in an electronic device, their directions of arrival can be set according to the actual usage scenario of the electronic device. The processing unit 13 can then acquire the direction of arrival of each of the at least two omnidirectional microphones pre-set. The processing unit 13 can determine a reference omnidirectional microphone and at least one enhanced omnidirectional microphone from the at least two omnidirectional microphones based on the desired beamforming direction. For example, the processing unit 13 can determine the omnidirectional microphone closest to the target sound source as the reference omnidirectional microphone and the other omnidirectional microphones in the microphone array as enhanced omnidirectional microphones. This allows the determination of the reference omnidirectional microphone and at least one enhanced omnidirectional microphone. Alternatively, the processing unit 13 can select any one of the at least two omnidirectional microphones as the reference omnidirectional microphone. Other omnidirectional microphones in the microphone array are identified as enhanced omnidirectional microphones. Of course, a reference omnidirectional microphone and at least one enhanced omnidirectional microphone can also be identified from at least two omnidirectional microphones in other ways, and this application does not limit this.

[0206] After calculating the current speed of sound and obtaining the direction of arrival of each of the at least two omnidirectional microphones, the processing unit 13 can calculate the signal delay of each enhanced omnidirectional microphone based on the direction of arrival of each of the at least two omnidirectional microphones, the distance between the omnidirectional microphone and each enhanced omnidirectional microphone, and the current speed of sound. After calculating the signal delay of each enhanced omnidirectional microphone, the processing unit 13 performs corresponding delay processing on the sound signal picked up by each enhanced omnidirectional microphone, and sums the delayed sound signal with the sound signal picked up by the reference omnidirectional microphone to obtain the enhanced sound signal. The processing unit 13 can output the enhanced sound signal.

[0207] As one possible implementation, in a microphone array, if the distance between any two adjacent omnidirectional microphones in at least two omnidirectional microphones is the same, such as... Figure 13 As shown, the processing unit 13, based on the direction of arrival of each of the at least two omnidirectional microphones, and referencing the distance between the omnidirectional microphone and at least one enhanced omnidirectional microphone and the current speed of sound, uses the formula... Calculate the signal delay for each enhanced omnidirectional microphone. Where, d s τ represents the distance between the s-th enhanced omnidirectional microphone and the reference omnidirectional microphone. s is greater than 0 and less than the number of omnidirectional microphones in the microphone array. s This represents the signal delay of the s-th enhanced omnidirectional microphone, and c represents the current speed of sound. This indicates the direction of arrival of the s-th enhanced omnidirectional microphone.

[0208] After calculating the signal delay of each enhanced omnidirectional microphone, the processing unit 13 can, based on the signal delay of at least one enhanced omnidirectional microphone and the sound signal picked up by at least one enhanced omnidirectional microphone, use the formula... The acoustic signal picked up by the reference omnidirectional microphone is enhanced, and the enhanced acoustic signal is calculated. Wherein, S (1*(s+1)) Let S be a vector of 1 row and s+1 columns formed by the signals picked up by the reference omnidirectional microphone and at least one enhanced omnidirectional microphone. Where S... (1*(s+1)) The first value in the vector is the acoustic signal picked up by the reference omnidirectional microphone. (*) T Let f represent the matrix transpose, f represent the sampling frequency of the omnidirectional microphone, and j represent the sampling frequency of the omnidirectional microphone. 2 = -1, where e is the base of the natural logarithm.

[0209] In this way, when the processing unit 13 performs directional enhancement on the sound signal picked up by the omnidirectional microphone, the current sound velocity used is calculated in real time from the sound signals picked up by the N sound vector microphones in the microphone array. When the environment changes, the calculated current sound velocity will be dynamically adjusted in real time, thereby improving the accuracy of directional enhancement of the sound signal picked up by the omnidirectional microphone.

[0210] refer to Figure 14 As shown in the illustration, this application provides another microphone array, the structure of which is the same as the one described above. Figures 1-13 The structure of the microphone array is the same, and can be referred to the above embodiments. The difference is that in this embodiment, the distance between at least two of the N sound vector microphones 11 is on the same order of magnitude as the reference distance.

[0211] like Figure 14 As shown, the microphone array includes N vector microphones 11 for picking up the acoustic signal from the target sound source. The distance d4 between at least two of the N vector microphones 11 is on the same order of magnitude as the reference distance d5. The reference distance d5 refers to the distance between the center point of the microphone array and the target sound source; N is an integer greater than 1.

[0212] In this embodiment, to measure the current sound velocity, at least two acoustic vector microphones 11 are arranged in the microphone array, i.e., N acoustic vector microphones 11 are arranged. The distance between at least two acoustic vector microphones 11 among the N acoustic vector microphones 11 is on the same order of magnitude as the reference distance. Since when the distance between two acoustic vector microphones 11 is on a different order of magnitude from the reference distance, for example, when the distance between two acoustic vector microphones 11 is much smaller than the distance between the center point of the microphone array and the target sound source, the pitch angle of the sound source relative to the two acoustic vector microphones 11 is basically the same. In this case, the current sound velocity estimated by the sound signal output by the acoustic vector microphones 11 has a large error. Therefore, in order to ensure the accuracy of the estimated current sound velocity, it is necessary to ensure that the distance between at least two acoustic vector microphones 11 among the N acoustic vector microphones is on the same order of magnitude as the distance between the center point of the microphone array and the target sound source.

[0213] As one possible implementation, the microphone array includes a processing unit, which is used to acquire the sound signals picked up by the N sound vector microphones, calculate the current sound speed based on the sound signals picked up by the N sound vector microphones and the distance between the N sound vector microphones, and output the current sound speed.

[0214] As one possible implementation, j target acoustic vector microphone pairs are determined from the N acoustic vector microphones; wherein the distance between the two acoustic vector microphones in each target acoustic vector microphone pair is on the same order of magnitude as the reference distance. j is an integer greater than 1 and less than or equal to C(2, N), where C(2, N) represents the number of combinations containing two acoustic vector microphones selected from the N acoustic vector microphones, and C(2, N) is an integer greater than 0 and less than N.

[0215] For each of the j target acoustic vector microphone pairs, based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair, calculate the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair and the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair; based on the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair, the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, calculate the target sound speed;

[0216] Calculate the current sound speed based on j target sound speeds and output the current sound speed.

[0217] refer to Figure 15 As shown in the embodiment of this application, a sound speed estimation method is provided, which is applied to... Figures 1-14 The microphone array shown. The method includes:

[0218] Step S1501: Obtain the sound signals picked up by N sound vector microphones.

[0219] In this embodiment of the application, after N sound vector microphones in the microphone array have picked up the sound signal, the picked-up sound signal can be obtained from the N sound vector microphones.

[0220] Step S1502: Estimate the current speed of sound based on the sound signals picked up by the N sound vector microphones and the distance between the N sound vector microphones.

[0221] In this embodiment of the application, after acquiring the sound signals picked up by N sound vector microphones, the direction of arrival of each sound vector microphone and the time delay of the sound signals picked up between the sound vector microphones can be calculated based on the sound signals picked up by the N sound vector microphones. Thus, the current speed of sound can be estimated based on the direction of arrival of each sound vector microphone, the time delay of the sound signals picked up between the sound vector microphones, and the distance between the N sound vector microphones.

[0222] As one possible implementation, the current speed of sound is estimated based on the sound signals picked up by N sound vector microphones and the distance between the N sound vector microphones, including:

[0223] Among the N acoustic vector microphones, j target acoustic vector microphone pairs are determined. The distance between the two acoustic vector microphones in each target acoustic vector microphone pair is within the range of 0.02 meters to 5 meters; j is an integer greater than 1 and less than or equal to C(2, N), where C(2, N) represents the number of combinations containing two acoustic vector microphones selected from the N acoustic vector microphones; C(2, N) is an integer greater than 0 and less than N.

[0224] For each of the j target acoustic vector microphone pairs, based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair, calculate the direction of arrival (DOA) of each acoustic vector microphone in the target acoustic vector microphone pair and the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair; based on the DOA of each acoustic vector microphone in the target acoustic vector microphone pair, the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, calculate the target sound speed.

[0225] Estimate the current sound speed based on the sound speeds of j targets.

[0226] Specifically, not every pair of N sound vector microphones is within the range of 0.02 meters to 5 meters in distance. In this application, the estimation of the current sound speed is typically performed using the sound signals picked up by two sound vector microphones. To ensure the accuracy of the estimated current sound speed, the distance between the two sound vector microphones used for estimation must be within the range of 0.02 meters to 5 meters. Therefore, j target sound vector microphone pairs need to be identified from the N sound vector microphones. Each target sound vector microphone pair contains two sound vector microphones within the range of 0.02 meters to 5 meters in distance.

[0227] As one possible implementation, determining j target acoustic vector microphone pairs from N acoustic vector microphones includes: determining the distance between every two acoustic vector microphones in the N acoustic vector microphones; identifying two acoustic vector microphones with a distance range of 0.02 meters to 5 meters as a target acoustic vector microphone pair, thus obtaining j target acoustic vector microphone pairs.

[0228] That is, among the N sound vector microphones 11, it is necessary to calculate the distance between each sound vector microphone and the other N-1 sound vector microphones, so as to determine which two sound vector microphones are within the distance range of 0.02 meters to 5 meters. The two sound vector microphones within the distance range of 0.02 meters to 5 meters are identified as a target sound vector microphone pair, and then j target sound vector microphone pairs can be identified among the N sound vector microphones.

[0229] As one possible implementation, two acoustic vector microphones within a distance range of 0.02 meters to 5 meters are defined as a target acoustic vector microphone pair, resulting in j target acoustic vector microphone pairs, including:

[0230] Two acoustic vector microphones that are within a distance range of 0.02 meters to 5 meters and are on the same order of magnitude as the reference distance are identified as a target acoustic vector microphone pair, resulting in j target acoustic vector microphone pairs.

[0231] The reference distance refers to the distance between the center point of the microphone array and the target sound source.

[0232] In this embodiment of the application, in order to estimate the current sound speed more accurately, two sound vector microphones with a distance between them ranging from 0.02 meters to 5 meters, and whose distance is on the same order of magnitude as the reference distance, can be identified as a target sound vector microphone pair, thereby obtaining j target sound vector microphone pairs from N sound vector microphones.

[0233] In identifying j target acoustic vector microphone pairs, for each pair, the direction of arrival (DOA) of each microphone is calculated based on the acoustic signal picked up by each microphone. The time delay between the two microphones in the pair is then calculated based on their respective DOAs, the time delay, and the distance between them. Therefore, the formula can be used to... Calculate the target sound speed. Where c represents the target sound speed, d is the distance between the two sound vector microphones in the target sound vector microphone pair, τ is the time delay between the sound signals picked up by the two sound vector microphones in the target sound vector microphone pair, θ1 is the pitch angle in the direction of arrival of one sound vector microphone in the target sound vector microphone pair, and θ2 is the pitch angle in the direction of arrival of the other sound vector microphone in the target sound vector microphone pair.

[0234] As one possible implementation, the direction of arrival (DOA) of each acoustic vector microphone in the target acoustic vector microphone pair is calculated based on the acoustic signal picked up by each acoustic vector microphone in the pair, including:

[0235] The acoustic signals picked up by each acoustic vector microphone in the target acoustic vector microphone pair are transformed into first frequency domain signals to obtain the acoustic signals at h×k time-frequency points of each acoustic vector microphone in the target acoustic vector microphone pair. For the acoustic signal at each of the h×k time-frequency points of the acoustic signal at each acoustic vector microphone in the target acoustic vector microphone pair, the sound intensity information at that time-frequency point is obtained based on the acoustic signal at that time-frequency point; based on the sound intensity information at that time-frequency point, the direction of arrival at that time-frequency point is calculated.

[0236] Where h represents the number of frequency points, which is a positive integer; and k represents the number of frames, which is a positive integer.

[0237] In this embodiment, the acoustic signal picked up by the acoustic vector microphone is a time-domain signal. To facilitate calculation, the time-domain signal can be converted into a frequency-domain signal. Therefore, when calculating the direction of arrival (DOA) of each acoustic vector microphone, it is necessary to first convert the acoustic signal picked up by the acoustic vector microphone into a frequency-domain signal. When calculating the DOA of the two acoustic vector microphones in each of the j target acoustic vector microphone pairs, the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair can be transformed into a first frequency-domain signal, obtaining the h×k time-frequency signals of each acoustic vector microphone in the target acoustic vector microphone pair. Due to the short-time stationary nature of acoustic signals, it is usually necessary to divide the acoustic signal into several frames of varying durations and process them in units of frames. Frame lengths have some common values ​​depending on different audio sampling rates, and can also be set according to actual needs. Based on the preset frame length, the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair can be processed into frames. The framed signal consists of sampling points. For example, at a sampling frequency of 16kHz, the true length is generally set between 10 and 35ms (milliseconds) according to the needs of short-time spectrum analysis. Assuming a frame size of 16ms, there are 256 sampling points in each frame, and the frame length is 256. After framing, a Short-Time Fourier Transform (STFT) can be performed on the framed signal to transform it into a first-frequency domain signal. The STFT involves windowing the frame signal before performing the Fourier transform. The purpose of the windowing function is to reduce frequency leakage caused by discontinuities at the frame signal boundaries during the STFT. After performing the STFT on the frame signal, the acoustic signals at h×k time-frequency points of each acoustic vector microphone can be obtained. For each time-frequency point in the acoustic signal at h×k time-frequency points of each acoustic vector microphone in the target acoustic vector microphone pair, the sound intensity information at that time-frequency point is obtained based on the acoustic signal at that time-frequency point. After obtaining the sound intensity signal at that time and frequency point, the direction of arrival at that time and frequency point can be calculated based on the sound intensity information at that time and frequency point.

[0238] It should be noted that sound intensity is the sound energy passing through a unit area per unit time in the direction perpendicular to the sound wave propagation, and it is a vector describing the direction of sound propagation. Since the direction of arrival of a sound vector microphone is related to the signal detected by the particle velocity sensor within the microphone, instantaneous sound intensity information in the time-frequency domain can be obtained based on the particle velocity vector signal detected by the particle velocity sensor.

[0239] As one possible implementation, obtaining the sound intensity information at a given time-frequency point based on the acoustic signal at that time-frequency point includes:

[0240] Based on the sound signal at that time and frequency point, using the formula Obtain the sound intensity information at that time and frequency point.

[0241] The acoustic signal at the time-frequency point includes sound pressure and acoustic signals along the x, y, and z axes. Sound pressure is a scalar, while the acoustic signals along the x, y, and z axes are vectors, specifically the particle velocity vector signals detected by the particle velocity sensor. x (f,n) represents the sound intensity information along the x-axis of a time-frequency point with frequency f and frame number n; y (f,n) represents the sound intensity information of a time-frequency point with frequency f and frame number n in the y-axis direction; z (f,n) represents the sound intensity information along the z-axis at a time frequency of frequency f and frame number n; X w * (f,n) represents the complex conjugate of the sound pressure at a time frequency of f and frame number n; X x (f,n) represents the signal with frequency f and frame number n along the x-axis; X y (f,n) represents the acoustic signal with frequency f and frame number n along the y-axis; X z (f,n) represents the acoustic signal with frequency f and frame number n in the z-axis direction; f is an integer greater than 0 and not greater than h; n is an integer greater than 0 and not greater than k.

[0242] Based on the sound intensity information at that time frequency, the direction of arrival at that time frequency is calculated as follows:

[0243] Based on the sound intensity information at that time and frequency point, using the formula Calculate the DOA at this time-frequency point; where, Indicates taking the real part, θ represents the horizontal angle of a time frequency point with frequency f and frame number n. (f，n) The elevation angle represents the time frequency point f and the number of frames n.

[0244] In this application embodiment, as can be seen from the above embodiments, Therefore, after obtaining the sound signal at that time and frequency point, the formula is used to determine the appropriate method based on the sound signal at that time and frequency point. When obtaining the sound intensity information at this time and frequency point, the sound intensity information can be simplified as follows:

[0245] After obtaining the sound intensity information, the direction of arrival at that time and frequency point can be calculated based on the sound intensity information at that time and frequency point, which means calculating the values ​​of the horizontal angle φ and the elevation angle θ at that time and frequency point.

[0246] Through formula It is possible to know Therefore, after obtaining the sound intensity information at this time and frequency point, it can be calculated according to the formula. Calculate the direction of arrival at that time frequency.

[0247] in,

[0248] The above formula can be used to calculate the direction of arrival (DOA) of each of the h×k time-frequency points of the target acoustic vector microphone pair.

[0249] It should be noted that, in the embodiments of this application, after obtaining the sound signal picked up by each sound vector microphone in the target sound vector microphone pair, the direction of arrival of the sound signal picked up by each sound vector microphone in the target sound vector microphone pair can also be calculated by other means, and this application does not limit this.

[0250] As one possible implementation, the calculation of the acoustic signal pickup delay between two acoustic vector microphones in the target acoustic vector microphone pair, based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair, includes:

[0251] The acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair is transformed into a second frequency domain signal to obtain k frames of second frequency domain acoustic signals for each acoustic vector microphone in the target acoustic vector microphone pair. Based on the k frames of second frequency domain acoustic signals picked up by each acoustic vector microphone in the target acoustic vector microphone pair, the time delay when picking up each frame of second frequency domain acoustic signal between the two acoustic vector microphones in the target acoustic vector microphone pair is calculated by a preset time delay algorithm.

[0252] In this embodiment, when calculating the current speed of sound, after calculating the direction of arrival (DOA) of each time-frequency point of each acoustic vector microphone in the target acoustic vector microphone pair, it is necessary to estimate the time delay between the two acoustic vector microphones in the target acoustic vector microphone pair. To facilitate calculation, the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair can be processed into k frames to obtain k frames of acoustic signals. The specific framing process can refer to the framing process described above when calculating the DOA of each time-frequency point of each acoustic vector microphone in the target acoustic vector microphone pair, and will not be repeated here. After processing the acoustic signal of the target acoustic vector microphone into k frames, a Fourier transform can be performed on each frame of acoustic signal, for example, a discrete Fourier transform, to obtain the second frequency domain acoustic signal of each frame. Based on the k frames of second frequency domain acoustic signals, a preset time delay algorithm is used to calculate the time delay between the two acoustic vector microphones in the target acoustic vector microphone pair when picking up each frame of the second frequency domain acoustic signal in the k frames.

[0253] It should be noted that the preset delay algorithm is a pre-set algorithm used to calculate the time delay of the audio signal between microphones.

[0254] As one possible implementation, the preset time delay algorithm includes: a generalized cross-correlation algorithm, a generalized cross-correlation-phase transform algorithm, or an interpolation-based time delay estimation algorithm. Of course, the preset time delay algorithm can also be other algorithms, and this application does not limit this.

[0255] In this embodiment of the application, when the preset delay algorithm is the Generalized Cross-Correlation (GCC) algorithm, the delay of picking up each frame of the second frequency domain sound signal between the two sound vector microphones in the target sound vector microphone pair is calculated by the preset delay algorithm based on the k frames of the second frequency domain sound signal picked up by the two sound vector microphones in the target sound vector microphone pair. That is, based on the k frames of the second frequency domain sound signal received by each sound vector microphone in the target sound vector microphone pair, the delay of picking up each frame of the second frequency domain sound signal between the two sound vector microphones in the target sound vector microphone pair is calculated by the generalized cross-correlation algorithm.

[0256] The following explanation uses the calculation of the time delay when two acoustic vector microphones in the target acoustic vector microphone pick up the second frequency domain acoustic signal of the i-th frame as an example.

[0257] At this point, the generalized cross-correlation algorithm (GCC) is used to calculate the signal delay between the second-frequency domain acoustic signal X0[i] received by one of the target acoustic vector microphones (X0, i.e., X0) and the second-frequency domain acoustic signal X1[i] received by the other acoustic vector microphone (X1, i.e., X1) of the same target acoustic vector microphone. The cross-correlation function is then:

[0258] in, The cross-power spectrum is given by k, where k is the frame number; X0[i] represents the second frequency domain acoustic signal of the i-th frame of the sound vector microphone X0; The conjugate of the second frequency domain acoustic signal of the i-th frame of the sound vector microphone X1; j 2 =-1, where e is the base of the natural logarithm; φ[i] represents the weighting function, or pre-filter, which can be preset; It is a weighted cross-power spectrum; ψ GCC [m] represents the cross-correlation function of X0[i] and X1[i], and the position corresponding to its peak value is the time delay value of the two signals X0[i] and X1[i]; m represents the time delay value to be estimated. Where argmax represents making ψ GCC When [m] reaches its maximum value, This is the estimated time delay of the acoustic signals received by the sound vector microphones X0 and X1.

[0259] The time delay between the two acoustic vector microphones in the target acoustic vector microphone when picking up the second frequency domain acoustic signal of each frame can be calculated using the above method.

[0260] As one possible implementation, when the aforementioned preset delay algorithm is a generalized cross-correlation-phase transform algorithm, the difference between the method of calculating the delay when picking up k frames of the second frequency domain acoustic signal between the two acoustic vector microphones in the target acoustic vector microphone and the aforementioned preset delay algorithm being a generalized cross-correlation algorithm is as follows: When the aforementioned preset delay algorithm is a generalized cross-correlation-phase transform algorithm, the aforementioned weighting function... This represents the cross power spectrum.

[0261] Thus, after calculating the directions of arrival (DOA) of the h×k time-frequency points for each acoustic vector microphone in the target acoustic vector microphone pair, and the k time delays when the two acoustic vector microphones in the target acoustic vector microphone pair pick up k frames of the second frequency domain acoustic signal, the target sound velocity at each of the h×k time-frequency points corresponding to the target acoustic vector microphone pair can be calculated based on the DOA of the h×k time-frequency points for each acoustic vector microphone in the target acoustic vector microphone pair, the k time delays when the two acoustic vector microphones in the target acoustic vector microphone pair pick up k frames of the second frequency domain acoustic signal, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair. Specifically, when calculating the target sound velocity at each of the h×k time-frequency points, the time-frequency points at different frequencies in the i-th frame all correspond to the time delays when the two acoustic vector microphones pick up the second frequency domain acoustic signal of the i-th frame. That is, the time delays of the signals corresponding to the time-frequency points at different frequencies in the same frame are the same.

[0262] The above method allows for the calculation of the target sound velocity at each of the h×k time-frequency points for each target sound vector microphone, thereby calculating the j target sound velocities at each of the h×k time-frequency points. After calculating the j target sound velocities at each of the h×k time-frequency points, the current sound velocity at each of the h×k time-frequency points can be calculated based on the j target sound velocities at each of the h×k time-frequency points.

[0263] As one possible implementation, the target sound speed is calculated based on the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair, the time delay of the acoustic signal picked up between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair.

[0264] Based on the direction of arrival (DOA) of each of the h×k time-frequency points of the target acoustic vector microphone pair, the time delay of the second frequency domain acoustic signal picked up by the two acoustic vector microphones in the target acoustic vector microphone pair for each frame, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, the formula is used to... Calculate the target sound velocity at the corresponding time-frequency point of the target sound vector microphone pair.

[0265] Among them, c (f，n) The target sound velocity at frequency f and frame number n represents the speed of sound at that frequency; d represents the distance between the two sound vector microphones in the target sound vector microphone pair; τ represents the speed of sound at that frequency. n θ represents the time delay when the two acoustic vector microphones in the target acoustic vector microphone pair pick up the second frequency domain acoustic signal of the nth frame; 1(f，n) θ represents the elevation angle of the time-frequency point f of one of the target vector microphones in the target vector microphone pair, with a frame number of n. 2(f，n) This represents the elevation angle of the frequency point f and the frame number n of the other acoustic vector microphone in the target acoustic vector microphone pair.

[0266] As one possible implementation, when calculating the direction of arrival (ROA) of each of the h×k time-frequency points of the target acoustic vector microphone pair, since some time-frequency points correspond to noise signals rather than valid speech signals, noise signals can be removed before calculating the ROA of each of the h×k time-frequency points in the target acoustic vector microphone pair. Only the ROA of the time-frequency points where the corresponding acoustic signals are valid speech signals can be calculated. In this case, before calculating the ROA of a time-frequency point based on its sound intensity information, the following steps are also included:

[0267] Based on the sound signal at that time and frequency point, determine whether the sound signal at that time and frequency point is a speech signal.

[0268] In this embodiment, after dividing the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair into h×k time-frequency points, the acoustic signal at each time-frequency point can be detected to determine whether it is a speech signal. For example, VAD (Voice Activity Detection) can be performed to detect whether the acoustic signal at each time-frequency point is a speech signal. VAD is also known as speech endpoint detection or speech boundary detection. Typically, VAD is used to identify and eliminate long silence periods from the audio signal stream to save voice channel resources without degrading service quality. Silence suppression can save valuable bandwidth resources and can help reduce the end-to-end latency perceived by users. In this embodiment, VAD is mainly used to detect whether the acoustic signal at each time-frequency point is a speech signal.

[0269] As one possible implementation, the IMCRA (Improved Minima Controlled Recursive Averaging) algorithm can be used to calculate the probability that the sound signal at each time-frequency point is a speech signal and the probability that it is a noise signal. Based on a preset speech threshold and the probability that the sound signal at each time-frequency point is a speech signal, after calculating the probability of the sound signal being a speech signal and the probability of the sound signal being a noise signal at each time-frequency point, a VAD mask (Voice Presence Detection Mask Suppression) can be used to discard or retain the sound signal at the time-frequency point according to the calculation results, thereby determining the time-frequency points where the sound signal is a speech signal. (Refer to...) Figure 16 As shown. Among them, in the appendix Figure 16 In the diagram, W represents the sound pressure signal picked up by the sound vector microphone, X represents the sound signal picked up by the sound vector microphone in the x-axis direction, Y represents the sound signal picked up by the sound vector microphone in the y-axis direction, and Z represents the sound signal picked up by the sound vector microphone in the z-axis direction.

[0270] The VAD mask can be a 0-1 mask: all sounds are retained when the detected sound signal is speech, and all sounds are discarded when the detected sound signal is not speech. Alternatively, the VAD mask can be a proportional mask. The VAD uses the IMCRA algorithm to output the probability that the sound signal at the current time-frequency point is a speech signal. Based on this probability, an amplification is applied to that time-frequency point, so that signals with higher probabilities are more likely to be retained, and signals with lower probabilities are more likely to be discarded.

[0271] As one possible implementation, the sound intensity information at a given time-frequency point can be used to determine whether the sound signal at that time-frequency point is a speech signal by analyzing the real and imaginary parts of the sound intensity information. For example, the ratio between the real and imaginary parts of the sound intensity information at that time-frequency point can be calculated and compared with a preset detection threshold. If the ratio is greater than the preset detection threshold, the sound signal at that time-frequency point is determined to be a speech signal. If the ratio is not greater than the preset detection threshold, the sound signal at that time-frequency point is determined to be a noise signal. This allows the formation of a set of time-frequency points for a single-source direct-path wave, i.e., a set of time-frequency points for the single-source direct-path waves of N sound vector microphones, as referenced. Figure 17 As shown, the time-frequency point of a single sound source direct wave can be selected using the above method, and the direction of arrival of the single sound source direct wave at the time-frequency point can be estimated.

[0272] It should be noted that, in Figure 17 In the middle, W n1 X represents the sound pressure signal picked up by the n1th sound vector microphone. n1 Y represents the sound signal picked up by the n1th sound vector microphone in the x-axis direction.n1 Z represents the sound signal picked up by the n1th sound vector microphone in the y-axis direction. n1 Let represent the sound signal picked up by the n1th sound vector microphone in the z-axis direction.

[0273] As one possible implementation, by estimating the direction of arrival (DOA) of the set of j single-source direct-arrival wave time-frequency points from each target acoustic vector microphone pair, the DOA of the set of j single-source direct-arrival wave time-frequency points can be obtained. A Gaussian weighted method can then be used to fuse the DOA estimates of the j single-source direct-arrival wave time-frequency points to obtain the final DOA. (Refer to...) Figure 17 As shown.

[0274] The Gaussian weighted method is achieved through the formula... To integrate. Among them, θ d φ d Let σ be any point on the angle grid, A be the smoothing step size, and h(a,b) be a two-dimensional Gaussian filter, where σ is the standard deviation and a and b are the lengths from the neighboring points within the step size to the center point.

[0275] It should be noted that other methods can also be used to detect whether the sound signal at each time and frequency point is a speech signal, and this application does not impose any restrictions on this.

[0276] At this point, the above-mentioned calculation of the direction of arrival at a time-frequency point based on the sound intensity information at that time-frequency point includes: when the sound signal at the time-frequency point is a speech signal, calculating the direction of arrival at that time-frequency point based on the sound intensity information at that time-frequency point.

[0277] In other words, the direction of arrival (DOA) at a given time-frequency point is calculated only when the sound signal is determined to be a speech signal, based on the sound intensity information at that time-frequency point. If the sound signal at that time-frequency point is determined to be a noise signal, the DOA is not calculated. This significantly reduces the computational workload and the consumption of hardware and software resources.

[0278] As one possible implementation, estimating the current sound speed based on j target sound speeds includes: calculating the mean of the j target sound speeds, and determining the mean of the j target sound speeds as the current sound speed.

[0279] That is, after calculating the j target sound velocities at each of the h×k time-frequency points, the average value of the j target sound velocities at each of the h×k time-frequency points is calculated to obtain the average value of the target sound velocities at each of the h×k time-frequency points. The average value of the target sound velocities at each of the h×k time-frequency points is then used as the current sound velocity at each of the h×k time-frequency points.

[0280] It should be noted that when the preset delay algorithm is the GCC algorithm, the delay estimated by the GCC algorithm is the integer delay. The main factors affecting the accuracy of the delay estimation by the GCC algorithm include the sampling frequency and the distance between the two acoustic vector microphones. If the distance between the two acoustic vector microphones is too small, the delay will be less than 1 / Fs, where Fs is the sampling frequency. Conversely, if the distance between the two acoustic vector microphones is too large, the correlation of the acoustic signals will weaken. Regarding the sampling frequency, if the sampling frequency is too small, the resolution of the integer delay estimation will be lower, resulting in inaccurate delay estimation.

[0281] The following uses simulation data to illustrate the impact of sampling frequency and two acoustic vector microphones on time delay estimation.

[0282] For example, the sound source is positioned relative to two sound vector microphones at different directions of arrival. Assuming that the room impulse response is generated using a mirror source model, under ideal conditions (no noise, no reverberation), when the sound source is 1 meter away from the center point of the microphone array and the array spacing is fixed at 0.04 meters, the sound source angle is set as shown in Table 1 below.

[0283] Table 1

[0284] Experiment number 1 2 3 4 5 6 7 8 9 <![CDATA[θ1]]> 9.52 19.06 28.62 38.22 47.87 57.58 67.35 77.20 87.14 <![CDATA[θ2]]> 10.52 21.02 31.49 41.91 52.26 62.54 72.73 82.84 92.86 <![CDATA[θ mid ]]> 10 20 30 40 50 60 70 80 90

[0285] Table 1 shows the direction of arrival for the sound source with different directions of arrival for two sound vector microphone settings. (Reference) Figure 4 As shown, θ1 is the direction of arrival of the sound source relative to the sound vector microphone AVS1, and θ2 is the direction of arrival of the sound source relative to the sound vector microphone AVS2. mid The direction of arrival of the sound source relative to the center of the two-element array composed of sound vector microphones AVS1 and AVS2.

[0286] The sampling frequencies were set to FS1 = 16000 Hz and FS1 = 48000 Hz respectively, and the distance between acoustic vector microphones AVS1 and AVS2 was fixed at 0.04 meters. Time delay estimation was then performed. The results of the time delay estimation are referenced. Figure 18a and Figure 18b As shown in the figure. Among them, Figure 18a This is a time delay estimation diagram when the sampling frequency is 16000Hz; Figure 18b This is a time delay estimation graph at a sampling frequency of 48000 Hz. Figure 18a and Figure 18b It can be seen that when the distance between acoustic vector microphones AVS1 and AVS2 is fixed, increasing the sampling frequency can significantly improve the accuracy of time delay estimation.

[0287] The distance between acoustic vector microphones AVS1 and AVS2 was set to 0.1 m, 0.2 m, 0.3 m, 0.4 m, 0.5 m, 0.6 m, 0.7 m, 0.8 m, 0.9 m, and 1 m, respectively. Since the true values ​​of time delay estimation differ for different microphone array distances, directly displaying the sound velocity estimation results obtained using the time delay estimate and the direction of arrival estimate is more intuitive. (Refer to...) Figure 19 As shown. Figure 19 As shown, different distances between the two sound vector microphones result in different absolute errors in the sound velocity estimation. (See attached diagram.) Figure 19 It can be seen that the average error in estimating the sound velocity when the distance between two sound vector microphones ranges from 0.7 meters to 1 meter is approximately 2 m / s. (Reference) Figure 20a and Figure 20b As shown, this is the estimation result of the sound velocity as a function of the direction of arrival of the sound source when the distance between two sound vector microphones is 1 meter and 0.1 meter, respectively. Figure 20a The figure shows the sound velocity estimation results as a function of the direction of arrival of the sound source when the distance between two sound vector microphones is 1 meter. Figure 20b The figure shows the sound velocity estimation results as the direction of arrival of the sound source changes when the distance between the two sound vector microphones is 0.1 meters.

[0288] In summary, appropriately increasing the spacing between the two acoustic vector microphones can improve the accuracy of time delay estimation, thereby improving the accuracy of sound velocity estimation. When the sound source is 1 meter away from the center of the microphone array, the estimated sound velocity error is relatively small when the spacing between the two acoustic vector microphones is in the range of 0.7 meters to 1 meter.

[0289] The following example uses a two-element linear microphone array to verify the accuracy of sound velocity estimation under reverberation conditions through simulation experiments. The experimental settings are as follows: distance between the sound source and the center point of the microphone array: 1 meter; room size: 10*8*6 meters; spacing between sound vector microphones: 1 meter; positions of the two sound vector microphones: AVS1 is [3 2.5 2], AVS2 is [4 2.5 2].

[0290] Sound source type: speech; sampling frequency: 48 kHz; duration: 1 second; device self-noise: SNR = 20 dB; ambient noise: SNR = 20 dB; T 60 = 0.6s (seconds).

[0291] By estimating the direction of arrival and signal pickup delay of the sound vector microphones AVS1 and AVS2, the estimated sound speed reference is obtained. Figure 21a and Figure 21b As shown. Among them, Figure 21a The figure shows T 60 Estimation of the speed of sound at 0.3s; Figure 21b The figure shows T60 The sound velocity estimation at T60 = 0.6s is given. The average sound velocity error at T60 = 0.3s is 1.73 m / s. The average sound velocity error at T60 = 0.6s is 2.53 m / s. Therefore, the sound velocity estimation method described in this application can guarantee the accuracy of sound velocity estimation even under reverberation conditions.

[0292] Furthermore, using a multi-element array can further improve the accuracy of sound velocity estimation. The following simulation results illustrate how a multi-element linear array improves the accuracy of sound velocity estimation, using a microphone array as a unit. The positions of the four array elements are shown in Table 2. The distance between the sound source and the center point of the microphone array is 1 meter; the room dimensions are 10*8*6 meters.

[0293] Table 2

[0294]

[0295] refer to Figure 22a and Figure 22b As shown, in Figure 22a The image shown is T. 60 Estimation of the speed of sound at 0.3s. Figure 22b The figure shows T 60 The speed of sound at 0.6s is estimated. This is achieved through... Figure 22a and Figure 22b It can be seen that the average sound velocity error at T60 = 0.3s is 1.62 m / s. Specifically, the average sound velocity error at T60 = 0.6s is 2.40 m / s. Compared to the two-element linear microphone array described above, the sound velocity errors in this embodiment are reduced. Therefore, using a multi-element array can further improve the accuracy of sound velocity estimation.

[0296] When a microphone array is placed in an electronic device, it can pick up the audio signal from within the electronic device or from the user using the device. Typically, directional beamforming is required for better acquisition of the user's audio signal. Therefore, when placing a microphone array in an electronic device, the direction of arrival of the directional beam can be preset, specifying the direction of beam pickup. Based on this, refer to... Figure 23 As shown, this application provides a sound pickup method, applied to the above-mentioned... Figures 1-13 The microphone array shown.

[0297] The method includes:

[0298] Step S2301: Obtain the sound signals picked up by N sound vector microphones in the microphone array, and obtain the current sound speed based on the sound signals picked up by the N sound vector microphones.

[0299] Specifically, please refer to steps S1401-S1402 above, which will not be repeated here.

[0300] Step S2302: Obtain the direction of arrival of each omnidirectional microphone in the pre-set microphone array.

[0301] The microphone array contains at least two omnidirectional microphones.

[0302] In this embodiment, when the microphone array includes at least two omnidirectional microphones, the acoustic signal from the sound source can be picked up by the omnidirectional microphones and then output as the picked-up speech signal. Before outputting the acoustic signal picked up by the omnidirectional microphones, directional beamforming can be applied to the acoustic signal to achieve directional beamforming. When the microphone array is installed on an electronic device, the direction of arrival (DOA) of the beam that needs directional beamforming can be pre-set according to the usage scenario of the electronic device. That is, the DOA of the omnidirectional microphones is pre-set.

[0303] For example, when the electronic device is a vehicle, such as Figure 24 As shown. A microphone array can be installed in the vehicle, such as... Figure 24 As shown, assume the microphone array contains two vector acoustic microphones and four omnidirectional microphones. The two vector microphones and four omnidirectional microphones are located on the same horizontal line, with the four omnidirectional microphones positioned between the two vector microphones. The user's voice signal from the driver's seat is used as the direction for directional beamforming. The direction of arrival (DOA) of the omnidirectional microphones in the microphone array can then be set based on the user's position in the driver's seat. The pre-set DOA of the omnidirectional microphones in the microphone array can then be obtained.

[0304] As one possible implementation, at least two omnidirectional microphones in the microphone array have the same direction of arrival.

[0305] Step S2303: Identify a reference omnidirectional microphone and at least one enhanced omnidirectional microphone among at least two omnidirectional microphones.

[0306] In this embodiment, a reference omnidirectional microphone and at least one enhanced omnidirectional microphone can be determined from at least two omnidirectional microphones based on the beam direction to be enhanced. For example, the omnidirectional microphone closest to the target sound source can be designated as the reference omnidirectional microphone. Other omnidirectional microphones in the microphone array can be designated as enhanced omnidirectional microphones. This allows the determination of the reference omnidirectional microphone and at least one enhanced omnidirectional microphone. Alternatively, one of the at least two omnidirectional microphones can be selected as the reference omnidirectional microphone. Other omnidirectional microphones in the microphone array can be designated as enhanced omnidirectional microphones.

[0307] It should be noted that the reference omnidirectional microphone and at least one enhanced omnidirectional microphone can also be determined from at least two omnidirectional microphones by other means, and this application does not limit this.

[0308] Step S2304: Based on the direction of arrival of each of the at least two omnidirectional microphones, and with reference to the distance between the omnidirectional microphone and at least one enhanced omnidirectional microphone and the current speed of sound, determine the signal delay of at least one enhanced omnidirectional microphone.

[0309] In this embodiment of the application, after calculating the current speed of sound and obtaining the direction of arrival of each of the at least two omnidirectional microphones, the signal delay of each enhanced omnidirectional microphone can be calculated based on the direction of arrival of each of the at least two omnidirectional microphones, with reference to the distance between the omnidirectional microphone and each enhanced omnidirectional microphone and the current speed of sound.

[0310] As one possible implementation, in a microphone array, if the distance between any two adjacent omnidirectional microphones in at least two omnidirectional microphones is the same, such as... Figure 24 As shown, based on the direction of arrival of each of the at least two omnidirectional microphones, and referring to the distance between the omnidirectional microphone and at least one enhanced omnidirectional microphone and the current speed of sound, the formula is used... Calculate the signal delay for each enhanced omnidirectional microphone.

[0311] Where, d s τ represents the distance between the s-th enhanced omnidirectional microphone and the reference omnidirectional microphone. s is greater than 0 and less than the number of omnidirectional microphones in the microphone array. s This represents the signal delay of the s-th enhanced omnidirectional microphone, and c represents the current speed of sound. This indicates the direction of arrival of the s-th enhanced omnidirectional microphone.

[0312] Step S2305: Based on the signal delay of at least one enhanced omnidirectional microphone and the sound signal picked up by at least one enhanced omnidirectional microphone, enhance the sound signal picked up by the reference omnidirectional microphone to obtain an enhanced sound signal.

[0313] In this embodiment of the application, after calculating the signal delay of each enhanced omnidirectional microphone, the sound signal picked up by each enhanced omnidirectional microphone is subjected to corresponding delay processing, and the delayed sound signal is summed with the sound signal picked up by the reference omnidirectional microphone to obtain the enhanced sound signal, which can be output.

[0314] As one possible implementation, after calculating the signal delay of each enhanced omnidirectional microphone, the signal delay of at least one enhanced omnidirectional microphone and the sound signal picked up by at least one enhanced omnidirectional microphone can be used to calculate the signal delay of each microphone using the formula... The sound signal picked up by the reference omnidirectional microphone is enhanced, and the enhanced sound signal is calculated.

[0315] Among them, S (1*(ns+1)) Let S be a vector of 1 row and s+1 columns formed by the signals picked up by the reference omnidirectional microphone and at least one enhanced omnidirectional microphone. Where S... (1*(s+1)) The first value in the vector is the acoustic signal picked up by the reference omnidirectional microphone. (*) T Let f represent the matrix transpose, f represent the sampling frequency of the omnidirectional microphone, and j represent the sampling frequency of the omnidirectional microphone. 2 = -1, where e is the base of the natural logarithm.

[0316] As illustrated in the example above, assuming the vehicle windows are closed, when a user in the driver's seat speaks, the sound signal picked up by the sound vector microphones in the microphone array can be used to calculate the current speed of sound. Based on the current speed of sound, the direction of arrival of each of the at least two omnidirectional microphones, and the distance between a reference omnidirectional microphone and at least one enhanced omnidirectional microphone, the sound signal picked up by the omnidirectional microphones is enhanced using directional beamforming to obtain an enhanced signal. This enhanced signal is used to amplify the voice signal of the user in the driver's seat.

[0317] When a vehicle window is open, the airflow speed inside the vehicle increases, causing a change in the speed of sound. If the speed of sound calculated when the window is closed is used for directional beam amplification, the amplified signal will deviate from the voice signal of the user in the driver's seat. In this embodiment, the voice signal of the user in the driver's seat can be re-acquired by the sound vector microphone of the microphone array, and the current speed of sound can be calculated based on this signal. That is, the current speed of sound can be updated, and the directional beam amplification can be performed using the updated current speed of sound. The resulting amplified signal is the signal that amplifies the voice signal of the user in the driver's seat. In other words, in this embodiment, the sound signal can be picked up in real time by the sound vector microphone in the microphone array, and the current speed of sound can be adjusted in real time. Then, the directional beam amplification can be performed based on the real-time adjusted speed of sound, making the directional beam amplification more accurate.

[0318] In this way, when performing directional enhancement on the sound signal picked up by the omnidirectional microphone, the current sound velocity used is calculated in real time from the sound signals picked up by the N sound vector microphones in the microphone array. When the environment changes, the calculated current sound velocity will be dynamically adjusted in real time, thereby improving the accuracy of directional enhancement on the sound signal picked up by the omnidirectional microphone.

[0319] refer to Figure 25 As shown, this application embodiment provides a sound speed estimation device, including:

[0320] Acquisition unit 2501 is used to acquire sound signals picked up by N sound vector microphones.

[0321] The processing unit 2502 is used to estimate the current speed of sound based on the sound signals picked up by N sound vector microphones and the distance between the N sound vector microphones.

[0322] In this embodiment of the application, the processing unit 2502 is specifically used to determine j target acoustic vector microphone pairs from N acoustic vector microphones; wherein the distance between the two acoustic vector microphones in the target acoustic vector microphone pair is in the range of 0.02 meters to 5 meters; j is an integer greater than 1 and less than or equal to C(2, N), where C(2, N) represents the number of combinations containing two acoustic vector microphones taken from the N acoustic vector microphones; C(2, N) is an integer greater than 0 and less than N.

[0323] For each of the j target vector microphone pairs, based on the acoustic signals picked up by each microphone in the target vector microphone pair, calculate the direction of arrival (DOA) of each microphone in the target vector microphone pair and the acoustic signal pickup delay between the two microphones in the target vector microphone pair. Based on the DOA of each microphone in the target vector microphone pair, the acoustic signal pickup delay between the two microphones in the target vector microphone pair, and the distance between the two microphones in the target vector microphone pair, calculate the target sound speed. Based on the j target sound speeds, estimate the current sound speed.

[0324] As one possible implementation, the processing unit 2502 is specifically used to determine the distance between every two acoustic vector microphones in the N acoustic vector microphones; and to determine two acoustic vector microphones with a distance range of 0.02 meters to 5 meters as a target acoustic vector microphone pair, thereby obtaining j target acoustic vector microphone pairs.

[0325] As one possible implementation, the processing unit 2502 is specifically used to identify two acoustic vector microphones that are within a distance range of 0.02 meters to 5 meters and are on the same order of magnitude as the reference distance as a target acoustic vector microphone pair, thereby obtaining j target acoustic vector microphone pairs.

[0326] The reference distance refers to the distance between the center point of the microphone array and the target sound source.

[0327] As one possible implementation, the processing unit 2502 is specifically used to transform the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair into a first frequency domain signal, thereby obtaining the acoustic signals of h×k time-frequency points of each acoustic vector microphone in the target acoustic vector microphone pair; where h represents the number of frequency points and is a positive integer; and k represents the number of frames and is a positive integer. For each time-frequency point of the acoustic signal of each of the h×k time-frequency points of the target acoustic vector microphone pair, the sound intensity information of that time-frequency point is obtained based on the acoustic signal of that time-frequency point; and the direction of arrival of that time-frequency point is calculated based on the sound intensity information of that time-frequency point.

[0328] As one possible implementation, the processing unit 2502 is specifically used to, based on the acoustic signal at that time-frequency point, use the formula Obtain the sound intensity information at this time-frequency point; wherein, the sound signal at the time-frequency point includes the sound pressure and the sound signal in the x-axis, y-axis, and z-axis directions; I x (f,n) represents the sound intensity information along the x-axis of a time-frequency point with frequency f and frame number n; y (f,n) represents the sound intensity information of a time-frequency point with frequency f and frame number n in the y-axis direction; z (f,n) represents the sound intensity information along the z-axis at a time frequency of frequency f and frame number n; X w * (f,n) represents the complex conjugate of the sound pressure at a time frequency of f and frame number n; X x (f,n) represents the acoustic signal with frequency f and frame number n along the x-axis; X y (f,n) represents the acoustic signal with frequency f and frame number n along the y-axis; X z (f,n) represents the acoustic signal with frequency f and frame number n in the z-axis direction; f is an integer greater than 0 and not greater than h; n is an integer greater than 0 and not greater than k.

[0329] Based on the sound intensity information at that time and frequency point, using the formula Calculate the direction of arrival at this time-frequency point; where, φ represents the real part. (f，n) θ represents the horizontal angle of a time frequency point with frequency f and frame number n. (f，n) The elevation angle represents the time frequency point f and the number of frames n.

[0330] As one possible implementation, the aforementioned sound speed estimation device, such as Figure 26 As shown, it also includes:

[0331] The determining unit 2503 is used to determine whether the sound signal at the time and frequency point is a speech signal based on the sound signal at that time and frequency point.

[0332] The processing unit 2502 is specifically used to calculate the direction of arrival of the sound at a time-frequency point based on the sound intensity information at that time-frequency point when the sound signal at that time-frequency point is a speech signal.

[0333] As one possible implementation, the processing unit 2502 is specifically used to transform the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair into a second frequency domain signal, thereby obtaining k frames of second frequency domain acoustic signals for each acoustic vector microphone in the target acoustic vector microphone pair. Based on the k frames of second frequency domain acoustic signals from each acoustic vector microphone in the target acoustic vector microphone pair, the time delay when the two acoustic vector microphones in the target acoustic vector microphone pair pick up each frame of the second frequency domain acoustic signal is calculated using a preset time delay algorithm.

[0334] As one possible implementation, the preset time delay algorithm includes: generalized cross-correlation algorithm, generalized cross-correlation-phase transformation algorithm, or interpolation-based time delay estimation algorithm.

[0335] As one possible implementation, the processing unit 2502 is specifically used to, based on the h×k time-frequency directions of each acoustic vector microphone in the target acoustic vector microphone pair, the time delay of the second frequency domain acoustic signal picked up by the two acoustic vector microphones in the target acoustic vector microphone pair for each frame, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, use the formula Calculate the target sound velocity at the corresponding time-frequency point of the target sound vector microphone pair.

[0336] Among them, c (f，n) τ represents the target sound speed; d represents the distance between the two sound vector microphones in the target sound vector microphone pair; n θ represents the time delay when the two acoustic vector microphones in the target acoustic vector microphone pair pick up the second frequency domain acoustic signal of the nth frame; 1(f，n) θ represents the elevation angle of the time-frequency point f of one of the target vector microphones in the target vector microphone pair, with a frame number of n. 2(f，n) This represents the elevation angle of the frequency point f and the frame number n of the other acoustic vector microphone in the target acoustic vector microphone pair.

[0337] As one possible implementation, the processing unit 2502 is specifically used to calculate the average of the j target sound speeds based on the j target sound speeds, and determine the average of the j target sound speeds as the current sound speed.

[0338] refer to Figure 27 As shown, this application embodiment provides a sound pickup device, including:

[0339] The acquisition unit 2701 is used to acquire the sound signals picked up by N sound vector microphones in the microphone array, and to acquire the current sound speed based on the sound signals picked up by the N sound vector microphones.

[0340] The acquisition unit 2701 is also used to acquire the direction of arrival of each omnidirectional microphone in a pre-set microphone array.

[0341] The microphone array contains at least two omnidirectional microphones.

[0342] The determining unit 2702 is used to determine a reference omnidirectional microphone and at least one enhanced omnidirectional microphone among at least two omnidirectional microphones.

[0343] The determining unit 2702 is further configured to determine the signal delay of at least one enhanced omnidirectional microphone based on the direction of arrival of each of the at least two omnidirectional microphones, with reference to the distance between the omnidirectional microphone and at least one enhanced omnidirectional microphone and the current speed of sound.

[0344] The processing unit 2703 is used to enhance the sound signal picked up by the reference omnidirectional microphone based on the signal delay of at least one enhanced omnidirectional microphone and the sound signal picked up by at least one enhanced omnidirectional microphone, so as to obtain an enhanced sound signal.

[0345] Corresponding to the above embodiments, this application also provides an electronic device. See also Figure 28 The diagram shown is a structural schematic of an electronic device according to an embodiment of the present invention. The electronic device may include a processor 2801, a memory 2802, and a communication unit 2803. These components communicate through one or more buses. Those skilled in the art will understand that the structure of the server shown in the figure does not constitute a limitation on the embodiment of the present invention. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0346] The communication unit 2803 is used to establish a communication channel, enabling the storage device to communicate with other devices. It can receive user data sent by other devices or send user data to other devices.

[0347] The processor 2801 serves as the control center of the storage device, connecting various parts of the electronic device via various interfaces and lines. It executes software programs and / or modules stored in the memory 2802, and retrieves data stored in the memory, to perform various functions of the electronic device and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 2801 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0348] The memory 2802 is used to store the execution instructions of the processor 2801. The memory 2802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0349] When the execution instructions in memory 2802 are executed by processor 2801, the electronic device is able to perform... Figure 15 or Figure 23 Some or all of the steps in the illustrated embodiments.

[0350] Corresponding to the above embodiments, this application also provides another electronic device. The electronic device may include: a microphone array. Wherein, the microphone array is as described above. Figures 1-14 The aforementioned microphone array.

[0351] As one possible implementation, the aforementioned electronic device includes a vehicle or a terminal device. The terminal device can be a microphone, mobile phone, computer, speaker, or other terminal device capable of voice interaction.

[0352] As one possible implementation, when the electronic device includes a vehicle, the distance between at least two of the N acoustic vector microphones in the microphone array is within the range of 0.2 meters to 2 meters. Due to the limited size of vehicles, maintaining a distance between at least two of the N acoustic vector microphones in the microphone array within the range of 0.2 meters to 2 meters allows for better application in vehicles, enabling a more accurate estimation of the current sound speed inside the vehicle and improving the accuracy of directional beam picking.

[0353] As a first possible implementation, when the electronic device includes a terminal device, the distance between at least two of the N sound vector microphones in the microphone array is within the range of 0.02 meters to 5 meters. This allows for better application to the terminal device, enabling a more accurate estimation of the current speed of sound in the environment where the terminal device is located, thereby improving the accuracy of directional beam picking.

[0354] For example, when applying a microphone array to a microphone, due to the limited size of the microphone, the distance between two vector microphones within the array can be set to 0.3 meters. When applying a microphone array to multiple speakers, one vector microphone can be placed in each speaker, forming a microphone array using at least two vector microphones from different speakers. In this case, the distance between the two speakers can be set according to the user's actual needs, such as 1 meter, 2.5 meters, or 5 meters.

[0355] In a specific implementation, the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps of the sound velocity estimation method or sound pickup method provided by the present invention. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0356] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0357] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments and terminal embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

Claims

1. A microphone array, characterized in that, include: N vector microphones are used to pick up the sound signal from the target sound source; Wherein, the distance between at least two of the N sound vector microphones is in the range of 0.02 meters to 5 meters, and N is an integer greater than 1; The microphone array further includes: a processing unit; The processing unit is used to acquire the sound signals picked up by the N sound vector microphones, calculate the current sound speed based on the sound signals picked up by the N sound vector microphones and the distance between the N sound vector microphones, and output the current sound speed. The processing unit is specifically used to determine j target acoustic vector microphone pairs from the N acoustic vector microphones; wherein the distance between the two acoustic vector microphones in each target acoustic vector microphone pair is in the range of 0.02 meters to 5 meters; j is an integer greater than 1 and less than or equal to C(2, N), where C(2, N) represents the number of combinations containing two acoustic vector microphones taken from the N acoustic vector microphones, and C(2, N) is an integer greater than 0 and less than N; For each of the j target acoustic vector microphone pairs, based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair, calculate the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair and the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair; based on the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair, the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, calculate the target sound speed; Calculate the current sound speed based on j target sound speeds and output the current sound speed.

2. The microphone array according to claim 1, characterized in that, The N sound vector microphones are located in the same plane.

3. The microphone array according to claim 2, characterized in that, The N sound vector microphones are arranged in a linear configuration.

4. The microphone array according to claim 3, characterized in that, In the N sound vector microphones, each sound vector microphone is equidistant from its adjacent sound vector microphone.

5. The microphone array according to claim 3, characterized in that, The distances between any two adjacent acoustic vector microphones in the N acoustic vector microphones are not exactly equal.

6. The microphone array according to claim 2, characterized in that, The N sound vector microphones include a first sound vector microphone, a second sound vector microphone, and a third sound vector microphone. The angle between the line containing the third sound vector microphone and the first sound vector microphone and the line containing the first sound vector microphone and the second sound vector microphone is greater than 0 degrees and less than 180 degrees.

7. The microphone array according to claim 1, characterized in that, The N sound vector microphones are arranged in a three-dimensional configuration.

8. The microphone array according to any one of claims 1 to 7, characterized in that, Also includes: At least two omnidirectional microphones.

9. The microphone array according to claim 8, characterized in that, The at least two omnidirectional microphones are located between the N sound vector microphones.

10. The microphone array according to claim 1, characterized in that, The distance between the two sound vector microphones in each target sound vector microphone pair is on the same order of magnitude as the reference distance; wherein, the reference distance refers to the distance between the target sound source and the center point of the microphone array.

11. The microphone array according to claim 10, characterized in that, The processing unit is also used to obtain the direction of arrival of each of the at least two omnidirectional microphones preset in a pre-defined manner; A reference omnidirectional microphone and at least one enhanced omnidirectional microphone are identified from at least two omnidirectional microphones; The signal delay of the at least one enhanced omnidirectional microphone is determined based on the direction of arrival of each of the at least two omnidirectional microphones, the distance between the reference omnidirectional microphone and the at least one enhanced omnidirectional microphone, and the current speed of sound. Based on the signal delay of the at least one enhanced omnidirectional microphone and the sound signal picked up by the at least one enhanced omnidirectional microphone, the sound signal picked up by the reference omnidirectional microphone is enhanced to obtain an enhanced sound signal.

12. A microphone array, characterized in that, include: N vector microphones are used to pick up the sound signal from the target sound source; Wherein, the distance between at least two of the N sound vector microphones is on the same order of magnitude as the reference distance; the reference distance refers to the distance between the center point of the microphone array and the target sound source; N is an integer greater than 1; The microphone array further includes: a processing unit; The processing unit is used to acquire the sound signals picked up by the N sound vector microphones, calculate the current sound speed based on the sound signals picked up by the N sound vector microphones and the distance between the N sound vector microphones, and output the current sound speed. The processing unit is specifically used to determine j target acoustic vector microphone pairs from the N acoustic vector microphones; wherein the distance between the two acoustic vector microphones in each target acoustic vector microphone pair is in the range of 0.02 meters to 5 meters; j is an integer greater than 1 and less than or equal to C(2, N), where C(2, N) represents the number of combinations containing two acoustic vector microphones taken from the N acoustic vector microphones, and C(2, N) is an integer greater than 0 and less than N; For each of the j target acoustic vector microphone pairs, based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair, calculate the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair and the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair; based on the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair, the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, calculate the target sound speed; Calculate the current sound speed based on j target sound speeds and output the current sound speed.

13. A method for estimating the speed of sound, characterized in that, Applied to the microphone array according to any one of claims 1-12, the method comprises: Acquire the acoustic signals picked up by N vector microphones; Estimate the current speed of sound based on the sound signals picked up by N sound vector microphones and the distance between the N sound vector microphones; The estimation of the current speed of sound based on the sound signals picked up by N sound vector microphones and the distance between the N sound vector microphones includes: Among the N acoustic vector microphones, j target acoustic vector microphone pairs are determined; wherein the distance between the two acoustic vector microphones in the target acoustic vector microphone pair is in the range of 0.02 meters to 5 meters; j is an integer greater than 1 and less than or equal to C(2, N), where C(2, N) represents the number of combinations containing two acoustic vector microphones taken from the N acoustic vector microphones, and C(2, N) is an integer greater than 0 and less than N; For each of the j target acoustic vector microphone pairs, based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair, calculate the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair and the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair; based on the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair, the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, calculate the target sound speed; Estimate the current sound speed based on the sound speeds of j targets.

14. The method according to claim 13, characterized in that, The step of determining j target acoustic vector microphone pairs from the N acoustic vector microphones includes: Determine the distance between any two acoustic vector microphones among the N acoustic vector microphones; Two acoustic vector microphones within a distance range of 0.02 meters to 5 meters are identified as a target acoustic vector microphone pair, resulting in j target acoustic vector microphone pairs.

15. The method according to claim 14, characterized in that, The step of identifying two acoustic vector microphones within a distance range of 0.02 meters to 5 meters as a target acoustic vector microphone pair, and obtaining j target acoustic vector microphone pairs, includes: Two acoustic vector microphones that are within a distance range of 0.02 meters to 5 meters and are on the same order of magnitude as the reference distance are identified as a target acoustic vector microphone pair, resulting in j target acoustic vector microphone pairs; wherein, the reference distance refers to the distance between the center point of the microphone array and the target sound source.

16. The method according to any one of claims 13-15, characterized in that, The step of calculating the direction of arrival (DOA) of each acoustic vector microphone in the target acoustic vector microphone pair based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair includes: The acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair is transformed into a first frequency domain signal to obtain the acoustic signal of h×k time-frequency points of each acoustic vector microphone in the target acoustic vector microphone pair; where h represents the number of frequency points, which is a positive integer; and k represents the number of frames, which is a positive integer. For each time-frequency point of the acoustic signal at each of the h×k time-frequency points of the target acoustic vector microphone pair, the sound intensity information at that time-frequency point is obtained based on the acoustic signal at that time-frequency point. Calculate the direction of arrival at that time and frequency point based on the sound intensity information at that time and frequency point.

17. The method according to claim 16, characterized in that, The step of obtaining the sound intensity information at a given time-frequency point based on the sound signal at that time-frequency point includes: Based on the sound signal at that time and frequency point, using the formula The sound intensity information at that time and frequency point is obtained; wherein, the sound signal at the time and frequency point includes the sound pressure and the sound signal in the x-axis, y-axis and z-axis directions; Indicates the frequency point as Frame count The sound intensity information of the time-frequency point in the x-axis direction; Indicates the frequency point as Frame count The sound intensity information of the time-frequency point in the y-axis direction; Indicates the frequency point as Frame count The sound intensity information of the time-frequency point in the z-axis direction; Indicates the frequency point as Frame count The complex conjugate of the sound pressure at the time-frequency point; The acoustic signal; Indicates the frequency point as Frame count The acoustic signal with its time-frequency point in the y-axis direction; Indicates the frequency point as Frame count The time-frequency point of the acoustic signal in the z-axis direction; f is an integer greater than 0 and not greater than h; n is an integer greater than 0 and not greater than k; The step of calculating the direction of arrival at a given time-frequency point based on the sound intensity information at that time-frequency point includes: Based on the sound intensity information at that time and frequency point, using the formula Calculate the direction of arrival at that time-frequency point; where, Indicates taking the real part, Indicates the frequency point as Frame count The horizontal angle at the time frequency point, Pitch angle.

18. The method according to claim 16, characterized in that, Before calculating the direction of arrival at the time-frequency point based on the sound intensity information at that time-frequency point, the method further includes: Based on the sound signal at that time and frequency point, determine whether the sound signal at that time and frequency point is a speech signal; The step of calculating the direction of arrival at a given time-frequency point based on the sound intensity information at that time-frequency point includes: When the sound signal at the specified time and frequency point is a speech signal, the direction of arrival at that time and frequency point is calculated based on the sound intensity information at that time and frequency point.

19. The method according to claim 16, characterized in that, The step of calculating the acoustic signal pickup delay between two acoustic vector microphones in the target acoustic vector microphone pair based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair includes: The acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair is transformed into a second frequency domain signal to obtain k frames of second frequency domain acoustic signal for each acoustic vector microphone in the target acoustic vector microphone pair; Based on the k frames of second frequency domain acoustic signals from each acoustic vector microphone in the target acoustic vector microphone pair, the time delay when the two acoustic vector microphones in the target acoustic vector microphone pair pick up each frame of second frequency domain acoustic signal is calculated using a preset time delay algorithm.

20. The method according to claim 19, characterized in that, The preset time delay algorithm includes: a generalized cross-correlation algorithm, a generalized cross-correlation-phase transformation algorithm, or a time delay estimation algorithm based on interpolation.

21. The method according to claim 19 or 20, characterized in that, The calculation of the target sound speed based on the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair, the time delay of the sound signal picked up between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair includes: Based on the direction of arrival (DOA) of each of the h×k time-frequency points of the target acoustic vector microphone pair, the time delay of the second frequency domain acoustic signal picked up by the two acoustic vector microphones in the target acoustic vector microphone pair for each frame, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, the formula is used to... The target sound velocity at the corresponding time-frequency point of the target acoustic vector microphone pair is calculated; where, d represents the target sound speed; d represents the distance between the two sound vector microphones in the target sound vector microphone pair. This represents the time delay when the two acoustic vector microphones in the target acoustic vector microphone pair pick up the second frequency domain acoustic signal of the nth frame; This indicates the target acoustic vector microphone pair, specifically one of the acoustic vector microphones. Pitch angle, This indicates the other acoustic vector microphone in the target acoustic vector microphone pair. Pitch angle.

22. The method according to any one of claims 13-15, characterized in that, The estimation of the current sound speed based on j target sound speeds includes: Calculate the mean of j target sound speeds, and determine the mean of j target sound speeds as the current sound speed.

23. A sound pickup method, characterized in that, Applied to the microphone array according to any one of claims 1-12, the method comprises: Acquire the sound signals picked up by N sound vector microphones in the microphone array, and obtain the current speed of sound based on the sound signals picked up by the N sound vector microphones; Obtain the direction of arrival of each omnidirectional microphone in a pre-set microphone array; wherein the microphone array contains at least two omnidirectional microphones; A reference omnidirectional microphone and at least one enhanced omnidirectional microphone are identified from at least two omnidirectional microphones; The signal delay of the at least one enhanced omnidirectional microphone is determined based on the direction of arrival of each of the at least two omnidirectional microphones, the distance between the reference omnidirectional microphone and the at least one enhanced omnidirectional microphone, and the current speed of sound. Based on the signal delay of the at least one enhanced omnidirectional microphone and the sound signal picked up by the at least one enhanced omnidirectional microphone, the sound signal picked up by the reference omnidirectional microphone is enhanced to obtain an enhanced sound signal.

24. A sound velocity estimation device, characterized in that, include: The acquisition unit is used to acquire the acoustic signals picked up by N sound vector microphones; The processing unit is used to estimate the current speed of sound based on the sound signals picked up by N sound vector microphones and the distance between the N sound vector microphones; The processing unit is specifically used to determine j target acoustic vector microphone pairs from the N acoustic vector microphones; wherein the distance between the two acoustic vector microphones in each target acoustic vector microphone pair is in the range of 0.02 meters to 5 meters; j is an integer greater than 1 and less than or equal to C(2, N), where C(2, N) represents the number of combinations containing two acoustic vector microphones taken from the N acoustic vector microphones, and C(2, N) is an integer greater than 0 and less than N; For each of the j target acoustic vector microphone pairs, based on the acoustic signal picked up by each acoustic vector microphone in the target acoustic vector microphone pair, calculate the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair and the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair; based on the direction of arrival of each acoustic vector microphone in the target acoustic vector microphone pair, the acoustic signal pickup delay between the two acoustic vector microphones in the target acoustic vector microphone pair, and the distance between the two acoustic vector microphones in the target acoustic vector microphone pair, calculate the target sound speed; Calculate the current sound speed based on j target sound speeds and output the current sound speed.

25. A sound pickup device, characterized in that, include: The acquisition unit is used to acquire the sound signals picked up by N sound vector microphones in the microphone array, and to acquire the current sound speed based on the sound signals picked up by the N sound vector microphones; The acquisition unit is also used to acquire the direction of arrival of each omnidirectional microphone in a pre-set microphone array; wherein the microphone array includes at least two omnidirectional microphones; A determining unit is used to determine a reference omnidirectional microphone and at least one enhanced omnidirectional microphone among at least two omnidirectional microphones; The determining unit is further configured to determine the signal delay of the at least one enhanced omnidirectional microphone based on the direction of arrival of each of the at least two omnidirectional microphones, the distance between the reference omnidirectional microphone and the at least one enhanced omnidirectional microphone, and the current speed of sound; The processing unit is configured to perform enhancement processing on the sound signal picked up by the reference omnidirectional microphone based on the signal delay of the at least one enhanced omnidirectional microphone and the sound signal picked up by the at least one enhanced omnidirectional microphone, to obtain an enhanced sound signal.

26. An electronic device, characterized in that, It includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to perform the method of any one of claims 13-22, or to perform the method of claim 23.

27. An electronic device, characterized in that, Includes the microphone array as described in any one of claims 1-12.

28. The electronic device according to claim 26, characterized in that, The electronic equipment includes vehicles or terminal devices.

29. The electronic device according to claim 27, characterized in that, When the electronic device includes a vehicle, the distance between at least two of the N acoustic vector microphones in the microphone array is in the range of 0.2 meters to 2 meters.

30. The electronic device according to claim 27, characterized in that, When the electronic device includes a terminal device, the distance between at least two of the N acoustic vector microphones in the microphone array is in the range of 0.02 meters to 5 meters.

31. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method of any one of claims 13-22, or to perform the method of claim 23.

Images