Method and device for determining sound source direction, storage medium and electronic device

Abstract

The present application discloses a method and device for determining the direction of a sound source, a storage medium and an electronic device, and relates to the field of microphone technology. The method for determining the direction of a sound source comprises: determining a time delay value for the incident sound wave to reach the two adjacent microphone elements in a microphone array when it is determined that two adjacent microphone elements in a microphone array receive an incident sound wave emitted by a sound source; performing frequency modulation conversion on the cross-power spectrum of the incident sound wave according to the time delay value to obtain a time delay spectrum function of the incident sound wave; determining a sound wave incident angle of the incident sound wave relative to the microphone array according to the time delay spectrum function, and determining the direction of the sound source relative to the microphone array according to the sound source incident angle. The above technical solution solves the problem that the direction of the sound source cannot be accurately determined in scenarios with high computational complexity.

Description

Technical Field

[0001] The present application relates to the field of microphone technology, and in particular to a method and device for determining the direction of a sound source, a storage medium, and an electronic device. Background Art

[0002] At present, with the rapid development of social economy and the continuous improvement of technological intelligence, various smart home appliances have entered thousands of households. People's understanding of smart home is no longer limited to single voice interaction. They also require voice devices to be able to perceive the position of the interactive object like people and provide a more comprehensive interactive experience. Microphone array positioning is an acoustic processing technology used to estimate the speaker's position. This acoustic processing technology can measure the speaker's direction relative to the array under the assumption of far-field plane waves, using the principle that the time delay of sound waves reaching different primitives in the microphone array changes with the incident direction.

[0003] However, in the related technologies, since microphone array positioning technology generally improves calculation accuracy at the expense of computational complexity, the application scenarios of microphone array technology are generally limited to scenarios with lower computational complexity. In addition, it also leads to the problem of being unable to accurately determine the direction of the sound source in scenarios with higher computational complexity, which seriously limits the application of microphone positioning technology in smart home scenarios.

[0004] Therefore, in the related art, there is no effective solution to the problem that the direction of the sound source cannot be accurately determined in scenarios with high computational complexity. Summary of the invention

[0005] The embodiments of the present application provide a method and device for determining the direction of a sound source, a storage medium, and an electronic device, so as to at least solve the problem in the related art that the direction of the sound source cannot be accurately determined in a scenario with high computational complexity.

[0006] According to an embodiment of the present application, a method for determining the direction of a sound source is provided, comprising: determining a time delay value for the incident sound wave to reach the two adjacent microphone elements in a microphone array when it is determined that the incident sound wave is received by the sound source; performing frequency modulation conversion on the mutual power spectrum of the incident sound wave according to the time delay value to obtain a time delay spectrum function of the incident sound wave; determining a sound wave incident angle of the incident sound wave relative to the microphone array according to the time delay spectrum function, and determining the direction of the sound source relative to the microphone array according to the sound source incident angle.

[0007] In an exemplary embodiment, determining that two adjacent microphone elements in a microphone array receive an incident sound wave emitted by a sound source is performed in the following manner, including: determining an array orientation of the microphone array, and determining a plurality of microphone elements oriented at the array orientation; grouping the plurality of microphone elements to obtain a plurality of groups of microphone elements, wherein each group of the plurality of groups of microphone elements includes at least the two adjacent microphone elements; and receiving the incident sound wave by any group of two adjacent microphone elements of the plurality of groups of microphone elements.

[0008] In an exemplary embodiment, frequency modulation conversion is performed on the mutual power spectrum of the incident sound wave according to the delay value to obtain a delay spectrum function of the incident sound wave, including: linear frequency modulation transformation is performed on the mutual power spectrum of the incident sound wave in the time domain to obtain a delay spectrum function corresponding to the mutual power spectrum of the incident sound wave.

[0009] In an exemplary embodiment, before performing frequency modulation conversion on the mutual power spectrum of the incident sound wave according to the time delay value, the method further includes: determining a digital signal obtained by sampling the incident sound wave received by the adjacent microphone element; obtaining a short-time Fourier spectrum of the digital signal by performing a short-time Fourier transform on the digital signal; determining the mutual power spectrum corresponding to the short-time Fourier spectrum based on prior knowledge of the mutual power spectrum, and determining the mutual power spectrum corresponding to the short-time Fourier spectrum as the mutual power spectrum of the incident sound wave.

[0010] In an exemplary embodiment, frequency modulation conversion is performed on the mutual power spectrum of the incident sound wave according to the time delay value to obtain a time delay spectrum function of the incident sound wave, including: determining a Fourier transform length value when performing the frequency modulation conversion; and performing a frequency modulation conversion on the mutual power spectrum in the time domain with a length equal to the Fourier transform length value according to the time delay value to obtain a time delay spectrum function of the incident sound wave.

[0011] In an exemplary embodiment, determining the sound wave incident angle of the incident sound wave relative to the microphone array based on the delay spectrum function includes: comparing the amplitudes of the delay spectrum function in the frequency domain to obtain a peak with the largest amplitude; determining a peak delay index value corresponding to the peak value, and determining a target delay value based on the correspondence between the peak delay index value and the delay value; determining the delay distance between the adjacent microphones based on the product of the target delay value and the sound wave speed; and determining the sound wave incident angle of the incident sound wave relative to the microphone array based on the delay distance.

[0012] In an exemplary embodiment, the time delay distance is used to determine the sound wave incident angle of the incident sound wave relative to the microphone array, including: obtaining a cosine calculation result corresponding to the sound wave incident angle of the incident sound wave relative to the microphone array by taking the quotient of the time delay distance and the adjacent distances of the adjacent microphone elements; and performing an inverse cosine calculation on the cosine calculation result to determine the sound wave incident angle of the incident sound wave relative to the microphone array.

[0013] According to another embodiment of the embodiments of the present application, a device for determining the direction of a sound source is also provided, including: a first determination module, for determining the time delay value of the incident sound wave reaching the two adjacent microphone elements in the microphone array when it is determined that the incident sound wave emitted by the sound source is received by the two adjacent microphone elements; an obtaining module, for performing frequency modulation conversion on the cross-power spectrum of the incident sound wave according to the time delay value to obtain a time delay spectrum function of the incident sound wave; a second determination module, for determining the sound wave incident angle of the incident sound wave relative to the microphone array according to the time delay spectrum function, and determining the direction of the sound source relative to the microphone array according to the sound source incident angle.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for determining the direction of a sound source when running.

[0015] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining the direction of the sound source through the computer program.

[0016] In an embodiment of the present application, when it is determined that two adjacent microphone elements in a microphone array receive an incident sound wave emitted by a sound source, the time delay value of the incident sound wave reaching the two adjacent microphone elements is determined; the cross-power spectrum of the incident sound wave is frequency-modulated according to the time delay value to obtain a time delay spectrum function of the incident sound wave; the sound wave incident angle of the incident sound wave relative to the microphone array is determined according to the time delay spectrum function, and the direction of the sound source relative to the microphone array is determined according to the sound source incident angle; the above-mentioned technical solution is adopted to solve the problem of being unable to accurately determine the direction of the sound source in scenarios with high computational complexity, and thus the direction of the sound source can be accurately determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 It is a hardware environment diagram of a method for determining a sound source direction according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of a method for determining a sound source direction according to an embodiment of the present application;

[0021] Figure 3 is a flow chart of a method for determining a sound source direction according to an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a physical model of far-field positioning of a microphone array according to an embodiment of the present application;

[0023] Figure 5 is a flow chart of another method for determining the direction of a sound source according to an embodiment of the present application;

[0024] Figure 6 It is a structural block diagram of a device for determining a sound source direction according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] According to one aspect of an embodiment of the present application, a method for determining the direction of a sound source is provided. The method for determining the direction of a sound source is widely used in smart home (Smart Home), smart home, smart home device ecology, smart house (IntelligenceHouse) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the above-mentioned method for determining the direction of a sound source can be applied to Figure 1 In the hardware environment composed of the terminal device 102 and the server 104 shown in FIG. Figure 1 As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0028] The network may include but is not limited to at least one of the following: wired network, wireless network. The wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 may be but is not limited to a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing device, a smart dishwasher, a smart projection device, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart fresh air device, a smart kitchen and bathroom device, a smart bathroom device, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification device, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0029] In this embodiment, a method for determining the direction of a sound source is provided, which is applied to the above-mentioned computer terminal. Figure 2 : is a flow chart of a method for determining a sound source direction according to an embodiment of the present application, the flow chart comprising the following steps:

[0030] Step S202, when it is determined that two adjacent microphone elements in the microphone array receive an incident sound wave emitted by a sound source, determine a time delay value for the incident sound wave to reach the two adjacent microphone elements;

[0031] Step S204, performing frequency modulation conversion on the cross power spectrum of the incident sound wave according to the time delay value to obtain a time delay spectrum function of the incident sound wave;

[0032] Step S206: determining the sound wave incident angle of the incident sound wave relative to the microphone array according to the time delay spectrum function, and determining the direction of the sound source relative to the microphone array according to the sound source incident angle.

[0033] Through the above steps, when it is determined that two adjacent microphone elements in the microphone array receive an incident sound wave emitted by a sound source, the time delay value of the incident sound wave reaching the two adjacent microphone elements is determined; the cross-power spectrum of the incident sound wave is frequency-modulated according to the time delay value to obtain a time delay spectrum function of the incident sound wave; the sound wave incident angle of the incident sound wave relative to the microphone array is determined according to the time delay spectrum function, and the direction of the sound source relative to the microphone array is determined according to the sound source incident angle, thereby solving the problem in the related art that the direction of the sound source cannot be accurately determined in a scenario with high computational complexity, and the direction of the sound source can be accurately determined.

[0034] In an exemplary embodiment, a technical solution is proposed, which can determine that two adjacent microphone elements in a microphone array receive an incident sound wave emitted by a sound source in the following manner, specifically including: determining the array orientation of the microphone array, and determining a plurality of microphone elements oriented at the array orientation; grouping the plurality of microphone elements to obtain a plurality of groups of microphone elements, wherein each group of the plurality of groups of microphone elements includes at least the two adjacent microphone elements; and receiving the incident sound wave by any group of two adjacent microphone elements of the plurality of groups of microphone elements.

[0035] In an exemplary embodiment, the above-mentioned step S204 performs frequency modulation conversion on the mutual power spectrum of the incident sound wave according to the delay value to obtain the delay spectrum function of the incident sound wave, which may include: performing linear frequency modulation transformation on the mutual power spectrum of the incident sound wave in the time domain to obtain the delay spectrum function corresponding to the mutual power spectrum of the incident sound wave.

[0036] It should be noted that the delay spectrum function corresponding to the cross-power spectrum of the incident sound wave represents the delay spectrum function of the incident sound wave.

[0037] In an exemplary embodiment, in order to better understand the process of performing frequency modulation conversion on the mutual power spectrum of the incident sound wave according to the delay value in the above-mentioned step S206 to obtain the delay spectrum function of the incident sound wave, the Fourier transform length value when performing the frequency modulation conversion can be determined; according to the delay value, the mutual power spectrum is subjected to frequency modulation conversion in the time domain with a length equal to the Fourier transform length value to obtain the delay spectrum function of the incident sound wave.

[0038] Optionally, in one embodiment, the process of performing frequency modulation conversion on the mutual power spectrum of the incident sound wave according to the delay value in the above step S206 to obtain the delay spectrum function of the incident sound wave may also include: performing a linear frequency modulation transform of the mutual power spectrum in the time domain with a length of the Fourier transform length value according to the delay value, and determining the delay spectrum function corresponding to the obtained mutual power spectrum of the incident sound wave as the delay spectrum function of the incident sound wave.

[0039] In an exemplary embodiment, before frequency modulation conversion is performed on the mutual power spectrum of the incident sound wave according to the time delay value, a technical solution is also proposed, and the specific steps include: determining the digital signal obtained by sampling the incident sound wave received by the adjacent microphone element; obtaining the short-time Fourier spectrum of the digital signal by performing a short-time Fourier transform on the digital signal; determining the mutual power spectrum corresponding to the short-time Fourier spectrum based on prior knowledge of the mutual power spectrum, and determining the mutual power spectrum corresponding to the short-time Fourier spectrum as the mutual power spectrum of the incident sound wave.

[0040] In an exemplary embodiment, determining the sound wave incident angle of the incident sound wave relative to the microphone array based on the delay spectrum function in the above step S206 can also be achieved in the following manner: comparing the amplitudes of the delay spectrum function in the frequency domain to obtain a peak with the largest amplitude; determining the peak delay index value corresponding to the peak value, and determining the target delay value based on the correspondence between the peak delay index value and the delay value; determining the delay distance between the adjacent microphones based on the product of the target delay value and the sound wave speed; and determining the sound wave incident angle of the incident sound wave relative to the microphone array based on the delay distance.

[0041] In an exemplary embodiment, a technical solution is further proposed for determining the sound wave incident angle of the incident sound wave relative to the microphone array using the time delay distance, which may specifically include: obtaining a cosine calculation result corresponding to the sound wave incident angle of the incident sound wave relative to the microphone array by taking the quotient of the time delay distance and the adjacent distances of the adjacent microphone elements; performing an inverse cosine calculation on the cosine calculation result, thereby determining the sound wave incident angle of the incident sound wave relative to the microphone array.

[0042] Optionally, in one embodiment, a process for determining the orientation of a microphone array can also be implemented, specifically including: obtaining a normal direction corresponding to the orientation of the microphone array; constructing a plane coordinate system with the positive direction of the horizontal coordinate axis as the orientation of the microphone array and the positive direction of the vertical coordinate axis as the normal direction; finding a target angle corresponding to the incident angle of the sound source in the angle between the horizontal coordinate axis and the vertical coordinate axis; and determining the orientation of the sound source relative to the microphone array based on the target angle.

[0043] In order to better understand the process of the above-mentioned method for determining the direction of the sound source, the implementation method flow of the above-mentioned method for determining the direction of the sound source is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.

[0044] In this embodiment, a method for determining the direction of a sound source is provided. Figure 3 is a flow chart of a method for determining a sound source direction according to an embodiment of the present application, such as Figure 3 As shown, the specific steps are as follows:

[0045] Step S301: microphone sampling signal;

[0046] Assume that the digital signals obtained by sampling the signals received by microphone elements 1 and 2 are x1(n) and x2(n) respectively, where n represents the discrete sampling time.

[0047] Step S302: short-time Fourier transform;

[0048] Among them, for the signal x m (n) to X m The short-time Fourier transform process of (k,f) is as follows:

[0049]

[0050] Wherein, m is a natural number.

[0051] Step S303: normalizing the cross power spectrum;

[0052] Performing short-time Fourier transform on x1(n) and x2(n), the short-time Fourier spectra of the signals are obtained as X1(k,f) and X2(k,f), respectively. Then the normalized cross-power spectrum between the signals (equivalent to the cross-power spectrum mentioned above) is expressed as

[0053]

[0054] Wherein, k represents the frame index and f represents the frequency index.

[0055] Based on the prior knowledge of the normalized cross power spectrum, the normalized cross power spectrum is reformulated as

[0056]

[0057] Step S304: linear frequency modulation conversion;

[0058] Based on the possible range of delay τ (equivalent to the above delay value) [-d / c, d / c], Y(k,f) is subjected to a linear frequency modulation transform (CZT transform) analysis with a length of F (equivalent to the above Fourier length value), and the delay spectrum function obtained is:

[0059] R(τ)=CZT(Y(k))0≤τ<F (4).

[0060] Among them, the process of obtaining the delay spectrum function from the normalized cross spectrum based on CZT transformation is expressed as

[0061]

[0062] in,

[0063]

[0064] Step S305: peak search;

[0065] Search for the peak value of the above delay spectrum function and obtain the delay index corresponding to the peak value:

[0066]

[0067] Among them, taking microphone 1 as the reference primitive, the relationship between the relative delay of the sound wave reaching microphone 2 and the delay peak index obtained by peak search is:

[0068]

[0069] Step S306: geometric transformation;

[0070] According to the geometric relationship, the cosine function corresponding to the incident angle of the sound source relative to the array can be expressed as follows:

[0071]

[0072] Step S307: Determine the sound source orientation (equivalent to the above-mentioned sound source direction).

[0073] Therefore, the incident angle of the sound wave from the sound source to the array can be expressed as:

[0074]

[0075] Through the above embodiments, the process of determining the incident time delay between microphone elements based on the normalized cross-correlation spectrum is realized through linear frequency modulation. Specifically, based on the cross-power spectrum, the linear frequency modulation technology is used to analyze the peak position of Y(k,f) in the transform domain. In scenarios with high computational complexity, it is also possible to accurately determine the direction of the sound source, thereby improving the accuracy of determining the direction of the sound source, and further promoting the application and development of microphone positioning technology in smart home scenarios.

[0076] In one embodiment, it is possible to combine Figure 4 The process of determining the above-mentioned sound wave incident angle is described. Figure 4is a schematic diagram of a physical model of far-field positioning of a microphone array according to an embodiment of the present application. Figure 4 As shown, due to the different incident directions of the incident sound waves, the distances that the sound waves propagate from the sound source to microphone element 1 and microphone element 2 are different. Therefore, there will be a certain time delay between the signals received by the microphones. The sound path difference is obtained by multiplying the time delay and the speed of sound. Then, the time difference of the arrival of the sound waves is estimated through the time delay estimation technology, and the arrival direction of the incident sound wave can be obtained according to the geometric principle. That is, the cosine function value of the incident angle θ is determined by the ratio of the microphone spacing d and the sound path difference.

[0077] Among them, delay estimation technology, such as microphone array positioning technology, is essentially based on the principle of generalized cross-correlation phase transformation, which is a method of estimating the incident delay between primitives through inverse Fourier transform after obtaining the normalized cross-correlation spectrum. It is limited by the array aperture and the sampling rate of the signal.

[0078] In other embodiments, a method for determining the direction of a sound source is also proposed, such as Figure 5 As shown, the specific steps are as follows: The specific steps are as follows:

[0079] Step S501: microphone sampling signal;

[0080] It should be noted that the process of determining the microphone sampling signal in step S501 is similar to the process of determining the microphone sampling signal in the above step S301. Therefore, the process of determining the microphone sampling signal in step S501 can be described with reference to the process in the above step S301, and this application will not repeat it here.

[0081] Step S502: short-time Fourier transform;

[0082] It should be noted that the short-time Fourier transform process in step S501 is similar to the short-time Fourier transform in step S301 above. Therefore, the short-time Fourier transform in step S501 can be described with reference to the process in step S301 above, and this application will not repeat it here.

[0083] Step S503: normalizing the cross power spectrum;

[0084] It should be noted that the process of cross-power spectrum normalization in step S502 is similar to the process of microphone cross-power spectrum normalization in step S302 above. Therefore, the process of cross-power spectrum normalization in step S502 can be described with reference to the process in step S302 above, and this application will not repeat it here.

[0085] Step S504: interpolation calculation;

[0086] Interpolate the normalized cross power spectrum N times to obtain the interpolated cross spectrum:

[0087] Y′(k,f)0≤f<NF (11).

[0088] Wherein, F represents the upper limit of the frequency index before interpolation, that is, the length of the Fourier transform before interpolation.

[0089] Step S505: peak search;

[0090] Perform inverse Fourier transform on the interpolated cross spectrum to obtain the cross-correlation function

[0091]

[0092] Perform peak search on the cross-correlation function and obtain the delay peak index as

[0093]

[0094] Among them, taking microphone 1 as the reference primitive, the relationship between the relative delay of the sound wave reaching microphone 2 and the delay peak index obtained by peak search is:

[0095]

[0096] Step S506: geometric transformation;

[0097] like Figure 4 As shown in Figure 2, according to the geometric relationship between the spacing between microphone elements and the incident angle of the sound wave, we have

[0098]

[0099] Step S507: Determine the direction of the sound source.

[0100] Therefore, the incident angle of the sound wave from the sound source to the array can be expressed as

[0101]

[0102] Where d is the microphone element spacing, λ s Indicates the wavelength of the sampling frequency.

[0103] This embodiment uses interpolation technology to improve the accuracy of azimuth estimation when performing inverse Fourier transform. However, the algorithm computational complexity increases linearly with the interpolation multiple, which increases the computational requirements. In other words, although the delay estimation accuracy obtained by the interpolation technology is improved by N times, the cost is that the inverse Fourier transform length becomes N times the original, and the computational complexity increases from F log(F) to NF log(NF), which expands the N log F (NF) times.

[0104] For the above scheme, when the sound wave is incident from the direction of the array normal, the time delay between the two microphones is zero, and the time delay peak of the cross-correlation function appears at the time delay center of the cross-correlation function. When the sound wave is incident from other directions, there is a time delay value between the two microphones, and the ratio of the time delay value to the sampling period is the number of discrete points at which the peak of the cross-correlation function is offset from the center of the delay. Therefore, taking the difference of the signal is equivalent to increasing the sampling frequency and shortening the sampling period. After interpolation, the number of discrete points corresponding to the same time delay becomes larger, thereby improving the accuracy of delay estimation.

[0105] Furthermore, in one embodiment, Figure 3 The sound wave incident angle and Figure 5 By comparing the incident angle of the sound wave in the Figure 5 The delay estimation accuracy of the acoustic wave incident angle in Fλ is directly improved s / 2d times.

[0106] Specifically, with sampling rate f s =16000Hz, array element spacing d = 0.1m, Fourier transform length F = 256, the delay estimation accuracy is improved by about 27 times. Figure 5 The method provided in the paper is to improve the accuracy of the position estimation by using the interpolation technique when performing the inverse Fourier transform. Figure 3 The method of estimating the incident time delay between primitives based on the normalized cross-correlation spectrum implemented by linear frequency modulation can reduce the required computational complexity and amount of calculation while maintaining the same delay estimation accuracy, and reduce the dependence on hardware computing resources. It can also improve the delay estimation accuracy of the sound wave incident angle while maintaining the same amount of calculation.

[0107] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0108] Figure 6 is a structural block diagram of a device for determining a sound source direction according to an embodiment of the present application. Figure 5 As shown, including:

[0109] A first determination module 62 is used to determine the time delay value of the incident sound wave reaching the two adjacent microphone elements in the microphone array when it is determined that the incident sound wave emitted by the sound source is received by the two adjacent microphone elements;

[0110] An obtaining module 64 is used to perform frequency modulation conversion on the cross power spectrum of the incident sound wave according to the delay value to obtain a delay spectrum function of the incident sound wave;

[0111] The second determination module 66 is used to determine the sound wave incident angle of the incident sound wave relative to the microphone array according to the time delay spectrum function, and determine the direction of the sound source relative to the microphone array according to the sound source incident angle.

[0112] Through the above-mentioned device, when it is determined that two adjacent microphone elements in the microphone array receive the incident sound wave emitted by the sound source, the time delay value of the incident sound wave reaching the two adjacent microphone elements is determined; the cross-power spectrum of the incident sound wave is frequency-modulated according to the time delay value to obtain the time delay spectrum function of the incident sound wave; the sound wave incident angle of the incident sound wave relative to the microphone array is determined according to the time delay spectrum function, and the direction of the sound source relative to the microphone array is determined according to the sound source incident angle, thereby solving the problem in the related art that the direction of the sound source cannot be accurately determined in a scenario with high computational complexity, and the direction of the sound source can be accurately determined.

[0113] In an exemplary embodiment, the first determination module 62 is further used to determine the array orientation of the microphone array, and determine a plurality of microphone elements oriented at the array orientation; group the plurality of microphone elements to obtain a plurality of groups of microphone elements, wherein each group of the plurality of groups of microphone elements includes at least the two adjacent microphone elements; and receive the incident sound wave through any group of two adjacent microphone elements of the plurality of groups of microphone elements.

[0114] In an exemplary embodiment, the obtaining module 64 is further used to perform a linear frequency modulation transform on the cross power spectrum of the incident sound wave in the time domain to obtain a delay spectrum function corresponding to the cross power spectrum of the incident sound wave.

[0115] It should be noted that the delay spectrum function corresponding to the cross-power spectrum of the incident sound wave represents the delay spectrum function of the incident sound wave.

[0116] In an exemplary embodiment, the above-mentioned sound source direction determination device also includes a third determination module, which is used to determine the digital signal obtained by sampling the incident sound wave received by the adjacent microphone element; obtain the short-time Fourier spectrum of the digital signal by performing a short-time Fourier transform on the digital signal; determine the mutual power spectrum corresponding to the short-time Fourier spectrum based on the prior knowledge of the mutual power spectrum, and determine the mutual power spectrum corresponding to the short-time Fourier spectrum as the mutual power spectrum of the incident sound wave.

[0117] In an exemplary embodiment, the above-mentioned obtaining module 64 is also used to determine the Fourier transform length value when performing the frequency modulation conversion; according to the delay value, the cross-power spectrum is subjected to a frequency modulation transformation in the time domain with a length equal to the Fourier transform length value to obtain the delay spectrum function of the incident sound wave.

[0118] Optionally, the above-mentioned obtaining module 64 is also used to perform a linear frequency modulation transform of the cross-power spectrum in the time domain with a length equal to the Fourier transform length value according to the delay value, and determine the delay spectrum function corresponding to the obtained cross-power spectrum of the incident sound wave as the delay spectrum function of the incident sound wave.

[0119] In an exemplary embodiment, the second determination module 66 is also used to compare the amplitudes of the delay spectrum function in the frequency domain to obtain a peak with the largest amplitude; determine the peak delay index value corresponding to the peak value, and determine the target delay value based on the correspondence between the peak delay index value and the delay value; determine the delay distance between the adjacent microphones based on the product of the target delay value and the speed of the sound wave; and use the delay distance to determine the sound wave incident angle of the incident sound wave relative to the microphone array.

[0120] In an exemplary embodiment, the time delay distance is used to determine the sound wave incident angle of the incident sound wave relative to the microphone array, including: obtaining a cosine calculation result corresponding to the sound wave incident angle of the incident sound wave relative to the microphone array by taking the quotient of the time delay distance and the adjacent distances of the adjacent microphone elements; and performing an inverse cosine calculation on the cosine calculation result to determine the sound wave incident angle of the incident sound wave relative to the microphone array.

[0121] In an optional embodiment, the second determination module 66 is further used to obtain the normal direction corresponding to the orientation of the microphone array; construct a plane coordinate system with the positive direction of the horizontal coordinate axis as the orientation of the microphone array and the positive direction of the vertical coordinate axis as the normal direction; find the target angle corresponding to the incident angle of the sound source in the angle between the horizontal coordinate axis and the vertical coordinate axis; and determine the orientation of the sound source relative to the microphone array based on the target angle.

[0122] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0123] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0124] S1, when it is determined that two adjacent microphone elements in the microphone array receive an incident sound wave emitted by a sound source, determining a time delay value for the incident sound wave to reach the two adjacent microphone elements;

[0125] S2, performing frequency modulation conversion on the cross power spectrum of the incident sound wave according to the delay value to obtain a delay spectrum function of the incident sound wave;

[0126] S3, determining the sound wave incident angle of the incident sound wave relative to the microphone array according to the time delay spectrum function, and determining the direction of the sound source relative to the microphone array according to the sound source incident angle.

[0127] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0128] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0129] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0130] S1, when it is determined that two adjacent microphone elements in the microphone array receive an incident sound wave emitted by a sound source, determining a time delay value for the incident sound wave to reach the two adjacent microphone elements;

[0131] S2, performing frequency modulation conversion on the cross power spectrum of the incident sound wave according to the delay value to obtain a delay spectrum function of the incident sound wave;

[0132] S3, determining the sound wave incident angle of the incident sound wave relative to the microphone array according to the time delay spectrum function, and determining the direction of the sound source relative to the microphone array according to the sound source incident angle.

[0133] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0134] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0135] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0136] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining a sound source direction, characterized in that: include: In the case where it is determined that two adjacent microphone elements in the microphone array receive an incident sound wave emitted by a sound source, determining a time delay value for the incident sound wave to reach the two adjacent microphone elements; Performing frequency modulation conversion on the cross power spectrum of the incident sound wave according to the time delay value to obtain a time delay spectrum function of the incident sound wave; Determining the sound wave incident angle of the incident sound wave relative to the microphone array according to the time delay spectrum function, and determining the direction of the sound source relative to the microphone array according to the sound source incident angle; Wherein, the acoustic wave incident angle is expressed as τ max represents the delay index corresponding to the peak value of the delay spectrum function, and F represents the Fourier length value; determining that two adjacent microphone primitives in the microphone array receive an incident sound wave emitted by a sound source in the following manner, including: Determining an array orientation of the microphone array, and determining a plurality of microphone primitives oriented in the array orientation; Grouping the plurality of microphone primitives to obtain a plurality of groups of microphone primitives, wherein each group of microphone primitives in the plurality of groups of microphone primitives includes at least the two adjacent microphone primitives; The incident sound wave is received by any two adjacent microphone elements of the plurality of groups of microphone elements.

2. The method for determining the direction of a sound source according to claim 1, characterized in that: Performing frequency modulation conversion on the cross power spectrum of the incident sound wave according to the delay value to obtain a delay spectrum function of the incident sound wave, including: The cross power spectrum of the incident sound wave is subjected to linear frequency modulation transformation in the time domain to obtain a time delay spectrum function corresponding to the cross power spectrum of the incident sound wave.

3. The method for determining the direction of a sound source according to claim 1, characterized in that: Performing frequency modulation conversion on the cross power spectrum of the incident sound wave according to the delay value to obtain a delay spectrum function of the incident sound wave, including: Determining a Fourier transform length value when performing the frequency modulation conversion; According to the time delay value, a frequency modulation transformation with a length equal to the Fourier transform length is performed on the cross power spectrum in the time domain to obtain a time delay spectrum function of the incident sound wave.

4. The method for determining the direction of a sound source according to any one of claims 1 to 3, characterized in that: Before frequency modulation conversion of the cross power spectrum of the incident sound wave according to the time delay value, the method further comprises: determining a digital signal obtained by sampling the incident sound wave received by the adjacent microphone element; Obtaining a short-time Fourier spectrum of the digital signal by performing a short-time Fourier transform on the digital signal; The cross power spectrum corresponding to the short-time Fourier spectrum is determined based on the prior knowledge of the cross power spectrum, and the cross power spectrum corresponding to the short-time Fourier spectrum is determined as the cross power spectrum of the incident sound wave.

5. The method for determining the direction of a sound source according to claim 4, characterized in that: Determining the sound wave incident angle of the incident sound wave relative to the microphone array according to the delay spectrum function includes: Comparing the amplitudes of the delay spectrum function in the frequency domain to obtain a peak value with the largest amplitude; Determine a peak delay index value corresponding to the peak value, and determine a target delay value based on a correspondence between the peak delay index value and the delay value; Determine the delay distance between the adjacent microphones based on the product of the target delay value and the speed of sound waves; The time delay distance is used to determine the sound wave incident angle of the incident sound wave relative to the microphone array.

6. The method for determining the direction of a sound source according to claim 5, characterized in that: Determining the sound wave incident angle of the incident sound wave relative to the microphone array using the time delay distance includes: Obtaining a cosine calculation result corresponding to the incident angle of the incident sound wave relative to the microphone array by taking the quotient of the time delay distance and the adjacent distance of the adjacent microphone primitives; An arccosine calculation is performed on the cosine calculation result, so as to determine the sound wave incident angle of the incident sound wave relative to the microphone array.

7. A device for determining the direction of a sound source, characterized in that: include: A first determination module is used to determine the time delay value of the incident sound wave reaching the two adjacent microphone elements in the microphone array when it is determined that the incident sound wave emitted by the sound source is received by the two adjacent microphone elements in the microphone array; An obtaining module, used for performing frequency modulation conversion on the cross power spectrum of the incident sound wave according to the time delay value to obtain a time delay spectrum function of the incident sound wave; A second determination module is used to determine the sound wave incident angle of the incident sound wave relative to the microphone array according to the time delay spectrum function, and determine the direction of the sound source relative to the microphone array according to the sound source incident angle; Wherein, the acoustic wave incident angle is expressed as τ max Represents the delay index corresponding to the peak value of the delay spectrum function, and F represents the Fourier length value; the first determination module is also used to determine the array orientation of the microphone array, and determine a plurality of microphone primitives oriented to the array orientation; group the plurality of microphone primitives to obtain a plurality of groups of microphone primitives, wherein each group of microphone primitives of the plurality of groups of microphone primitives includes at least the two adjacent microphone primitives; and receive the incident sound wave through any group of two adjacent microphone primitives of the plurality of groups of microphone primitives.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 6 when executed.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 6 through the computer program.

Images