Acoustic event positioning method, electronic device and readable storage medium

Abstract

The present application discloses an acoustic event localization method, an electronic device, and a readable storage medium, which are applied to the field of big data technology. The acoustic event localization method includes: receiving a sound signal through a preset microphone array, and determining a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array; selecting a target beam output signal that meets a preset condition from the beam output signal; determining the target sound source position of each target beam output signal based on the target beam output signal; generating an acoustic event type of each target beam output signal in a perception environment corresponding to the preset microphone array based on a frequency domain signal corresponding to the target beam output signal, and associating each acoustic event type with a corresponding target sound source position. The present application solves the technical problem of low accuracy in acoustic event localization.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to an acoustic event positioning method, an electronic device, and a readable storage medium. Background Art

[0002] With the rapid development of science and technology, big data technology has become more and more mature. At present, the acoustic event recognition model obtained by simulating the sound training in different spatial positions is used to recognize the acoustic events of the sound signal, and the recognition result is obtained, and then the sound signal is located according to the recognition result. However, this method is only applicable to single-source sound signals. When there are many sound sources of indoor sound signals, the accuracy of acoustic event recognition is low due to the low signal-to-noise ratio and high reverberation of the indoor scene, which leads to low accuracy of acoustic event positioning. Summary of the invention

[0003] The main purpose of the present application is to provide an acoustic event localization method, an electronic device and a readable storage medium, aiming to solve the technical problem of low accuracy of acoustic event localization in the prior art.

[0004] To achieve the above object, the present application provides an acoustic event localization method, which is applied to an acoustic event localization device. The acoustic event localization method includes:

[0005] Receive a sound signal through a preset microphone array, and determine a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array;

[0006] Selecting a target beam output signal that meets a preset condition from the beam output signals;

[0007] Determining the target sound source position of each target beam output signal according to the target beam output signal;

[0008] Based on the frequency domain signal corresponding to the target beam output signal, an acoustic event type of each target beam output signal in the perception environment corresponding to the preset microphone array is generated, and each acoustic event type is associated with a corresponding target sound source position.

[0009] Optionally, the step of determining a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array includes:

[0010] Obtaining a beam weight corresponding to the preset microphone array and a sound parameter corresponding to the sound signal;

[0011] The sound signal is converted into a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array according to the beam weight and the sound parameter.

[0012] Optionally, the step of selecting a target beam output signal that meets a preset condition from the beam output signals includes:

[0013] Obtaining beam weights corresponding to each of the beam output signals;

[0014] Selecting a target beam weight whose corresponding evaluation index satisfies a preset index condition from among the beam weights;

[0015] The beam output signal corresponding to the target beam weight is used as the target beam output signal.

[0016] Optionally, the evaluation index includes a beam directivity factor, a white noise gain, a beam sidelobe response, and a distortion-free response of the beam in the viewing angle direction. Before the step of selecting a target beam weight whose corresponding evaluation index satisfies a preset index condition from each of the beam weights, the step further includes:

[0017] If the distortion-free response of the beam in the viewing angle direction is a preset value, the beam directivity factor is greater than or equal to a preset first threshold, the white noise gain is less than a preset second threshold, and the beam sidelobe response is less than a preset third threshold, then it is determined that the evaluation index corresponding to the beam weight meets the preset index condition, wherein the preset first threshold is greater than the preset second threshold, and the preset first threshold is greater than the preset third threshold.

[0018] Optionally, the step of determining the target sound source position of each sound signal according to the target beam output signal includes:

[0019] According to the target beam output signal, construct a covariance matrix of the target beam output signal in a harmonic domain corresponding to the preset microphone array;

[0020] According to the covariance matrix and the preset azimuth spectrum generation model, construct an azimuth spectrum corresponding to the target beam output signal, wherein the azimuth spectrum is composed of a correlation curve between sound intensity and sound source position;

[0021] The target sound source position of each target beam output signal is generated according to the azimuth spectrum.

[0022] Optionally, the step of constructing a covariance matrix of the target beam output signal in a harmonic domain corresponding to the preset microphone array according to the target beam output signal includes:

[0023] Determining, according to the target beam output signal, the noisy relative sound pressure of each sound field corresponding to the target beam output signal;

[0024] Performing Fourier transform on the noisy relative sound pressure to obtain harmonic domain coefficients corresponding to the target beam output signal;

[0025] The covariance matrix is ​​constructed according to the harmonic domain coefficients.

[0026] Optionally, the step of constructing the azimuth spectrum corresponding to the target beam output signal according to the covariance matrix and a preset azimuth spectrum generation model includes:

[0027] Performing singular value decomposition on the covariance matrix to obtain a singular value decomposition result, wherein the singular value decomposition result includes a left singular value matrix and a right singular value matrix;

[0028] According to the left singular value matrix and the right singular value matrix, the azimuth spectrum corresponding to the target beam output signal is generated through the preset azimuth spectrum generation model.

[0029] Optionally, the singular value decomposition result also includes a singular value diagonal matrix, and the step of generating the target sound source position of each target beam output signal according to the azimuth spectrum includes:

[0030] Rank-finding processing is performed on the singular value diagonal matrix to obtain the number of sound sources of the target beam output signal;

[0031] According to the number of sound sources, the azimuth spectrum is queried to obtain the target sound source position of each target beam output signal.

[0032] To achieve the above-mentioned purpose, the present application further provides an acoustic event localization device, which is applied to an acoustic event localization device, and the acoustic event localization device comprises:

[0033] A receiving module, used to receive a sound signal through a preset microphone array, and determine a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array;

[0034] A selection module, used to select a target beam output signal that meets a preset condition from the beam output signals;

[0035] A determination module, used to determine the target sound source position of each target beam output signal according to the target beam output signal;

[0036] A generation module is used to generate the acoustic event type of each target beam output signal in the perception environment corresponding to the preset microphone array according to the frequency domain signal corresponding to the target beam output signal, and associate each acoustic event type with the corresponding target sound source position.

[0037] Optionally, the receiving module is further used for:

[0038] Obtaining a beam weight corresponding to the preset microphone array and a sound parameter corresponding to the sound signal;

[0039] The sound signal is converted into a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array according to the beam weight and the sound parameter.

[0040] Optionally, the selection module is further used for:

[0041] Obtaining beam weights corresponding to each of the beam output signals;

[0042] Selecting a target beam weight whose corresponding evaluation index satisfies a preset index condition from among the beam weights;

[0043] The beam output signal corresponding to the target beam weight is used as the target beam output signal.

[0044] Optionally, the evaluation index includes a beam directivity factor, a white noise gain, a beam sidelobe response, and a distortion-free response of the beam in the viewing direction. Before the step of selecting a target beam weight whose corresponding evaluation index satisfies a preset index condition from each of the beam weights, the acoustic event localization device is further used to:

[0045] If the distortion-free response of the beam in the viewing angle direction is a preset value, the beam directivity factor is greater than or equal to a preset first threshold, the white noise gain is less than a preset second threshold, and the beam sidelobe response is less than a preset third threshold, then it is determined that the evaluation index corresponding to the beam weight meets the preset index condition, wherein the preset first threshold is greater than the preset second threshold, and the preset first threshold is greater than the preset third threshold.

[0046] Optionally, the determining module is further used for:

[0047] According to the target beam output signal, construct a covariance matrix of the target beam output signal in a harmonic domain corresponding to the preset microphone array;

[0048] According to the covariance matrix and the preset azimuth spectrum generation model, construct an azimuth spectrum corresponding to the target beam output signal, wherein the azimuth spectrum is composed of a correlation curve between sound intensity and sound source position;

[0049] The target sound source position of each target beam output signal is generated according to the azimuth spectrum.

[0050] Optionally, the determining module is further used for:

[0051] Determining, according to the target beam output signal, the noisy relative sound pressure of each sound field corresponding to the target beam output signal;

[0052] Performing Fourier transform on the noisy relative sound pressure to obtain harmonic domain coefficients corresponding to the target beam output signal;

[0053] The covariance matrix is ​​constructed according to the harmonic domain coefficients.

[0054] Optionally, the determining module is further used for:

[0055] Performing singular value decomposition on the covariance matrix to obtain a singular value decomposition result, wherein the singular value decomposition result includes a left singular value matrix and a right singular value matrix;

[0056] According to the left singular value matrix and the right singular value matrix, the azimuth spectrum corresponding to the target beam output signal is generated through the preset azimuth spectrum generation model.

[0057] Optionally, the singular value decomposition result also includes a singular value diagonal matrix, and the generating module is further used for:

[0058] Rank-finding processing is performed on the singular value diagonal matrix to obtain the number of sound sources of the target beam output signal;

[0059] According to the number of sound sources, the azimuth spectrum is queried to obtain the target sound source position of each target beam output signal.

[0060] The present application also provides an electronic device, comprising: a memory, a processor, and a program of the acoustic event localization method stored in the memory and executable on the processor, wherein the program of the acoustic event localization method can implement the steps of the acoustic event localization method as described above when executed by the processor.

[0061] The present application also provides a computer-readable storage medium, on which is stored a program for implementing the acoustic event localization method. When the program of the acoustic event localization method is executed by a processor, the steps of the acoustic event localization method as described above are implemented.

[0062] The present application also provides a computer program product, including a computer program, which implements the steps of the acoustic event localization method as described above when executed by a processor.

[0063] The present application provides an acoustic event localization method, an electronic device, and a readable storage medium. Compared with a method for identifying acoustic events of a sound signal by using an acoustic event localization model obtained by simulating sound training at different spatial positions, the present application receives a sound signal through a preset microphone array, and determines a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array; selects a target beam output signal that meets a preset condition from the beam output signal; determines a target sound source position of each target beam output signal according to the target beam output signal; generates an acoustic event type of each target beam output signal in a perception environment corresponding to the preset microphone array according to a frequency domain signal corresponding to the target beam output signal, and associates each acoustic event type with a corresponding target sound source position, respectively; obtains a target beam signal by screening the beam output signal of the sound signal, and then determines the target sound source position of the target beam signal, and then generates an acoustic event type of each target sound source position, thereby avoiding the technical defect that when there are many sound sources of the indoor sound signal, the acoustic event localization is easily inaccurate due to the low signal-to-noise ratio and high reverberation of the indoor scene, thereby improving the accuracy of acoustic event localization. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0065] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0066] Figure 1 This is a flow chart of the first embodiment of the acoustic event localization method of the present application;

[0067] Figure 2 This is a flow chart of the second embodiment of the acoustic event localization method of the present application;

[0068] Figure 3 This is an example diagram of a scene in the acoustic event localization method of this application;

[0069] Figure 4 Schematic diagram of the structure of the device involved in the acoustic event localization method in the embodiment of the present application;

[0070] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the acoustic event localization method in the embodiment of the present application.

[0071] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0072] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of this application.

[0073] Embodiment 1

[0074] The present application provides an acoustic event localization method. In the first embodiment of the acoustic event localization method of the present application, referring to Figure 1 , the acoustic event localization method comprises:

[0075] Step S10, receiving a sound signal through a preset microphone array, and determining a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array;

[0076] Step S20, selecting a target beam output signal that meets a preset condition from the beam output signals;

[0077] Step S30, determining the target sound source position of each target beam output signal according to the target beam output signal;

[0078] Step S40, generating an acoustic event type of each target beam output signal in a perception environment corresponding to the preset microphone array according to a frequency domain signal corresponding to the target beam output signal, and associating each acoustic event type with a corresponding target sound source position.

[0079] In this embodiment, it should be noted that the preset microphone array is an arrangement of a certain number of microphones (ie, acoustic sensors) that are preset. The preset microphone array can be a linear array, a plane array, or a three-dimensional array such as a triangular array or a spherical array.

[0080] It can be understood that the three-dimensional array can better utilize spatial information than the linear array and the planar array. Among them, the triangular array is constructed as a non-planar microphone array distribution with a large reverse attenuation. The spherical array has spatial symmetry and has the function of omnidirectionally perceiving spatial information. In order to avoid the technical defect of being unable to perceive the situation in high-frequency or medium-frequency acoustic environments such as indoor environments when using the triangular array as the preset microphone array, because it is only suitable for perceiving sound signals in low-frequency acoustic environments, the embodiment of the present application preferably uses a spherical array as the preset microphone array to realize the reception of sound signals of various acoustic events.

[0081] Exemplarily, steps S10 to S40 include: obtaining a preset number of microphones and a preset microphone array, receiving a sound signal through a preset microphone array of the preset number of microphones, and determining a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array, and when the preset microphone array is a spherical array, the harmonic domain is a spherical harmonic domain; judging whether there is a target beam output signal in each of the beam output signals that satisfies a preset condition, if there is a target beam output signal in each of the beam output signals that satisfies the preset condition, collecting the target beam output signal, and returning to the execution step of judging whether there is a target beam output signal in each of the beam output signals that satisfies the preset condition, until each microphone in the preset microphone array The method further comprises the following steps: selecting the target beam output signal, and if no target beam output signal satisfies the preset condition among the beam output signals, returning to the execution step: receiving a sound signal through a preset microphone array, and determining a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array; determining a target sound source position of each target beam output signal according to the target beam output signal; determining a frequency domain signal corresponding to the target sound source position and each target beam output signal, and generating an acoustic event type of each target beam output signal according to each frequency domain signal and a preset acoustic event type recognition model; and associating each acoustic event type with a corresponding target sound source position according to a preset decision model.

[0082] Optionally, the step of determining the frequency domain signal corresponding to the target sound source position and each of the target beam output signals may specifically be:

[0083]

[0084] in, is the frequency domain signal of the target beam output signal of the qth microphone, spectrogram[·] is the spectrum calculation, audio(x q ) is the target beam output signal of the qth microphone, x q is the coordinate position of the qth microphone.

[0085] Wherein, in step S10, the step of determining the beam output signal of the sound signal in the harmonic domain corresponding to the preset microphone array includes:

[0086] Step S11, obtaining a beam weight corresponding to the preset microphone array and a sound parameter corresponding to the sound signal;

[0087] Step S12: converting the sound signal into a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array according to the beam weight and the sound parameter.

[0088] Exemplarily, steps S11 to S12 include: determining the beam weight corresponding to the preset microphone array, when the preset microphone array is a spherical array, the sound parameters corresponding to the sound signal at least include the wave number corresponding to the sound signal, the radius of the sphere corresponding to the spherical array, the sound pressure of the sound signal in the spherical harmonic domain, and one of the sound source pitch angle and sound source azimuth angle corresponding to the sound signal; based on the beam weight and the sound parameters, converting the sound signal into a beam output signal of the sound signal in the harmonic domain corresponding to the preset microphone array.

[0089] Optionally, the step of converting the sound signal into a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array according to the beam weight and the sound parameter may specifically be:

[0090]

[0091] Wherein, y is the beam output signal in the spherical harmonic domain corresponding to the preset microphone array, is the conjugate operation value of the beam weight under the preset microphone array, k is the wave number corresponding to the sound signal, r is the sphere radius corresponding to the spherical array, is the spherical harmonic function corresponding to the sound signal, is the conjugate operation of the spherical harmonic function, n is the order of the spherical harmonic function, m is the degree of the spherical harmonic function, p nm (k, r) is the sound pressure in the spherical harmonic domain, θ is the sound source pitch angle corresponding to the sound signal, is the sound source azimuth corresponding to the sound signal.

[0092] Optionally, the spherical harmonic function corresponding to the sound signal may be obtained specifically as follows:

[0093]

[0094] in, is the spherical harmonic function corresponding to the sound signal, n is the order of the spherical harmonic function, m is the degree of the spherical harmonic function, is the associated Legendre function, i is the imaginary part, is the sound source azimuth corresponding to the sound signal.

[0095] Optionally, the associated Legendre function may be obtained specifically as follows:

[0096]

[0097] in, is the associated Legendre function, m is the degree of the spherical harmonic function, n is the order of the spherical harmonic function, θ is the sound source pitch angle corresponding to the sound signal, P n (cosθ) is the associated Legendre function when m=0.

[0098] Optionally, the associated Legendre function when m=0 may be obtained as follows:

[0099]

[0100] Among them, P n (cosθ) is the associated Legendre function when m=0, n is the order of the spherical harmonic function, and θ is the sound source pitch angle corresponding to the sound signal.

[0101] It can be understood that in order to ensure that the sound pressure in the spherical harmonic domain maintains a sinusoidal characteristic on the surface of the sphere, the order of the spherical harmonic function needs to meet the condition of being greater than a preset order value, and the preset order value can be the product of the wave number corresponding to the sound signal and the radius of the sphere corresponding to the spherical array.

[0102] Since the preset beamforming formula has the following equation relationship:

[0103]

[0104] in, is the spherical harmonic function corresponding to the sound signal, is the conjugate operation of the spherical harmonic function, θ is the sound source pitch angle corresponding to the sound signal, is the sound source azimuth corresponding to the sound signal, and δ is the Dirac function.

[0105] According to the above equation, the preset beamforming formula is simplified as follows:

[0106]

[0107] Wherein, y is the beam output signal in the spherical harmonic domain corresponding to the preset microphone array, is the conjugate operation value of the beam weight under the preset microphone array, n is the order of the spherical harmonic function, m is the degree of the spherical harmonic function, k is the wave number corresponding to the sound signal, p nm (k, r) is the sound pressure in the spherical harmonic domain, and N is the upper limit of the order of the spherical harmonic function.

[0108] Wherein, in step S20, the step of determining the beam output signal of the sound signal in the harmonic domain corresponding to the preset microphone array includes:

[0109] Step S21, obtaining beam weights corresponding to the beam output signals;

[0110] Step S22, selecting a target beam weight whose corresponding evaluation index satisfies a preset index condition from among the beam weights;

[0111] Step S23: Using the beam output signal corresponding to the target beam weight as the target beam output signal.

[0112] As an example, steps S21 to S23 include: obtaining the beam weight corresponding to each of the beam output signals; obtaining the evaluation index corresponding to each of the beam weights, and selecting the target beam weight whose corresponding evaluation index meets the preset index condition from each of the beam weights; and using the beam output signal corresponding to the target beam weight as the target beam output signal.

[0113] As an example, steps S21 to S23 include: respectively constructing target optimization functions of the beam directivity factor, the white noise gain, the beam sidelobe response and the distortion-free response of the beam in the viewing angle direction; determining the target beam weight according to the target optimization function; and using the beam output signal corresponding to the target beam weight as the target beam output signal.

[0114] Optionally, the target optimization function for constructing the beam directivity factor may specifically be:

[0115] min DF -1

[0116] Wherein, DF is the beam directivity factor.

[0117] Optionally, the target optimization function for constructing the white noise gain, the beam sidelobe response, and the distortion-free response of the beam in the viewing angle direction may specifically be:

[0118] subject to Rep=1,WNG, SL

[0119] Wherein, Rep is the distortion-free response of the beam in the viewing angle direction, WNG is the white noise gain, and SL is the beam sidelobe response.

[0120] It can be understood that, since the conjugate operation value of the beam weight under the preset microphone array is a one-dimensional representation of the pitch angle and azimuth angle of the sound source in the two-dimensional space, an axisymmetric beam pattern can be used to describe the conjugate operation value of the beam weight, which can be specifically:

[0121]

[0122] in, is the conjugate operation value of the beam weight under the preset microphone array, k is the wave number corresponding to the sound signal, is the spherical harmonic function, θ l is the pitch angle of the sound source in the direction of the array viewing angle, is the azimuth of the sound source in the direction of the array viewing angle, b n (kr) is the modal function of the rigid spherical array, d n (k) is the updated beamforming weight.

[0123] Optionally, the modal function of the rigid spherical array may be obtained specifically as follows:

[0124]

[0125] Among them, b n (kr) is the modal function of the rigid spherical array, i is the imaginary part, r is the radius of the sphere corresponding to the spherical array, k is the wave number corresponding to the sound signal, j is the n (kr) is the first kind of spherical Bessel function, j′(kr a ) is the first-order derivative of the first-type spherical Bessel function, is the first-order derivative of the spherical Bessel function of the second kind, is the second-type spherical Bessel function, and n is the order of the spherical harmonic function.

[0126] Optionally, the first type of spherical Bessel function may be obtained specifically as follows:

[0127]

[0128] Among them, j n (kr) is the first kind of spherical Bessel function, n is the order of the spherical harmonic function, r is the radius of the sphere corresponding to the spherical array, and k is the wave number corresponding to the sound signal.

[0129] Optionally, the second type of spherical Bessel function may be obtained specifically as follows:

[0130]

[0131] in, is the second kind of spherical Bessel function, i is the imaginary part, r is the radius of the sphere corresponding to the spherical array, k is the wave number corresponding to the sound signal, and n is the order of the spherical harmonic function.

[0132] Wherein, in step S20, the evaluation index includes a beam directivity factor, a white noise gain, a beam sidelobe response, and a distortion-free response of the beam in the viewing angle direction, and before the step of selecting a target beam weight whose corresponding evaluation index satisfies a preset index condition from each of the beam weights, the method further includes:

[0133] Step A10, if the distortion-free response of the beam in the viewing angle direction is a preset value, the beam directivity factor is greater than or equal to a preset first threshold, the white noise gain is less than a preset second threshold, and the beam sidelobe response is less than a preset third threshold, then it is determined that the evaluation index corresponding to the beam weight meets the preset index condition, wherein the preset first threshold is greater than the preset second threshold, and the preset first threshold is greater than the preset third threshold.

[0134] In this embodiment, it should be noted that the preset first threshold is a critical value of the beam directivity factor for determining that the beam directivity factor corresponding to the beam weight satisfies a preset index condition; the preset second threshold is a critical value of the white noise gain for determining that the white noise gain corresponding to the beam weight satisfies a preset index condition; the preset third threshold is a critical value of the beam sidelobe response for determining that the beam sidelobe response corresponding to the beam weight satisfies a preset index condition.

[0135] Exemplarily, step A10 includes: when the distortion-free response of the beam in the viewing angle direction is a preset value, determining whether the beam directivity factor is greater than or equal to a preset first threshold, whether the white noise gain is less than a preset second threshold, and whether the beam sidelobe response is less than a preset third threshold; if the beam directivity factor is greater than or equal to the preset first threshold, the white noise gain is less than the preset second threshold, and the beam sidelobe response is less than the preset third threshold, then determining that the evaluation index corresponding to the beam weight meets the preset index condition; if the beam directivity factor is less than the preset first threshold, and / or the white noise gain is not less than the preset second threshold, and / or the beam sidelobe response is not less than the preset third threshold, then determining that the evaluation index corresponding to the beam weight does not meet the preset index condition.

[0136] Optionally, the correlation between the updated beamforming weight and each of the evaluation indicators is as follows:

[0137]

[0138] Where DF is the beam directivity factor, d n is the updated beamforming weight, H is the conjugate transpose operation, A is the amplitude signal, and B is the first diagonal matrix.

[0139] Optionally, the amplitude signal may be obtained by:

[0140] A=v n (Θ i )v n (Θ i ) H

[0141] Where A is the amplitude signal, v n (Θ i ) is the first frequency, H is the conjugate transpose operation, Θ i is the angle between the discretized sidelobe angle and the desired direction of the array, and i is the discretization index number of the beam sidelobe.

[0142] It can be understood that when selecting the target beam output signal, in order to improve the directivity of the beam output signal, that is, the angle between the discretized sidelobe angle and the desired direction of the array is 0, since the associated Legendre function has P n (1)=1, so the above formula can be simplified to:

[0143]

[0144] Where A is the amplitude signal, v n is the second frequency, and H is the conjugate transpose operation.

[0145] Optionally, the first diagonal matrix may be obtained specifically as follows:

[0146]

[0147] Among them, B is the first diagonal matrix, v n (Θ i ) is the first frequency, diag is the diagonalization operation, Θ i is the angle between the discretized sidelobe angle and the desired direction of the array, and i is the discretization index number of the beam sidelobe.

[0148] From the above content, we can see that the formula can be simplified to:

[0149]

[0150] Among them, B is the first diagonal matrix, v n is the second frequency, and diag is the diagonalization operation.

[0151] Optionally, the first frequency may be obtained by:

[0152]

[0153] Among them, v n (Θ i ) is the first frequency, P n is the associated Legendre function when m=0, n is the order of the spherical harmonic function, Θ i is the angle between the discretized sidelobe angle and the desired direction of the array, i is the discretization index of the beam sidelobe, and N is the upper limit of the order of the spherical harmonic function.

[0154] From the above content, we can see that the formula can be simplified to:

[0155]

[0156] Among them, v n is the second frequency, n is the order of the spherical harmonic function, and N is the upper limit of the order of the spherical harmonic function.

[0157] Optionally, the associated Legendre function when m=0 may be obtained as follows:

[0158]

[0159] Among them, P n (cosθ i ) is the associated Legendre function when m=0, and n is the order of the spherical harmonic function.

[0160]

[0161] Wherein, WNG is the white noise gain, d n is the updated beamforming weight, H is the conjugate transpose operation, A is the amplitude signal, is the second diagonal matrix.

[0162] The acquisition of the amplitude signal refers to the specific implementation process in the above content, which will not be described in detail here.

[0163] Optionally, the second diagonal matrix may be obtained specifically as follows:

[0164]

[0165] in, is the second diagonal matrix, v n (Θ i ) is the third frequency, Q is the number of microphones in the spherical array, b0, b1, …, b N is the spherical array modal function b n The results of the modal order of (kr) for different values ​​of n, diag is the diagonalization operation, Θ iis the angle between the discretized sidelobe angle and the desired direction of the array, i is the discretization index number of the beam sidelobe, N is the upper limit of the order of the spherical harmonic function, and n is the order of the spherical harmonic function.

[0166] It can be understood that when selecting the target beam output signal, in order to reduce the white noise gain and improve the directivity of the beam output signal, that is, the angle between the discretized sidelobe angle and the desired direction of the array is 0. Since the associated Legendre function has P n (1)=1, so the above formula can be simplified to:

[0167]

[0168] in, is the second diagonal matrix, Q is the number of microphones in the spherical array, n is the order of the spherical harmonics, and v n is the fourth frequency, b0, b1, ..., b N is the spherical array modal function b n The results of the modal order of (kr) for different values ​​of n, diag is the diagonalization operation, and N is the upper limit of the order of the spherical harmonics.

[0169] Optionally, the third frequency may be obtained specifically by:

[0170]

[0171] Among them, v n (Θ i ) is the third frequency, n is the order of the spherical harmonics, P n (cosθ i ) is the associated Legendre function when m = 0, n is the order of the spherical harmonic function, Θ i is the angle between the discretized sidelobe angle and the desired direction of the array, i is the discretization index of the beam sidelobe, and N is the upper limit of the order of the spherical harmonic function.

[0172] From the above content, we can see that the formula can be simplified to:

[0173]

[0174] Among them, v n is the fourth frequency, n is the order of the spherical harmonic function, and N is the upper limit of the order of the spherical harmonic function.

[0175]

[0176] Where SL is the beam sidelobe response, d n is the updated beamforming weight, v n (Θ i ) is the fifth frequency, Θi is the angle between the discretized sidelobe angle and the desired direction of the array, i is the discretization index of the beam sidelobe, I is the upper limit of the discretization index of the beam sidelobe, and H is the conjugate transpose operation.

[0177] Optionally, the fifth frequency may be obtained specifically by:

[0178]

[0179] Among them, v n (Θ i ) is the fourth frequency, n is the order of the spherical harmonic function, N is the upper limit of the order of the spherical harmonic function, P n is the associated Legendre function when m = 0, Θ i is the angle between the discretized sidelobe angle and the desired direction of the array, i is the discretization index of the beam sidelobe, and N is the upper limit of the order of the spherical harmonic function.

[0180] Optionally, the associated Legendre function when m=0 may be obtained as follows:

[0181]

[0182] Among them, P n is the associated Legendre function when m = 0, Θ i is the angle between the discretized sidelobe angle and the desired direction of the array, i is the discretization index of the beam sidelobe, and n is the order of the spherical harmonic function.

[0183] The embodiment of the present application provides an acoustic event localization method. Compared with the method of identifying acoustic events of sound signals by using an acoustic event localization model obtained by simulating sound training at different spatial positions, the embodiment of the present application receives a sound signal through a preset microphone array, and determines a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array; selects a target beam output signal that meets a preset condition from the beam output signal; determines a target sound source position of each target beam output signal according to the target beam output signal; generates an acoustic event type of each target beam output signal in a perception environment corresponding to the preset microphone array according to a frequency domain signal corresponding to the target beam output signal, and associates each acoustic event type with a corresponding target sound source position, obtains a target beam signal by screening the beam output signal of the sound signal, and then determines the target sound source position of the target beam signal, and then generates an acoustic event type of each target sound source position, thereby avoiding the technical defect that when there are many sound sources of indoor sound signals, the acoustic event localization is easily inaccurate due to the low signal-to-noise ratio and high reverberation of the indoor scene, thereby improving the accuracy of acoustic event localization.

[0184] Embodiment 2

[0185] Further, refer to Figure 2 Based on the first embodiment of the present application, in another embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be described in detail later. On this basis, in step S30, the step of determining the target sound source position of each of the sound signals according to the target beam output signal includes:

[0186] Step S31, constructing a covariance matrix of the target beam output signal in a harmonic domain corresponding to the preset microphone array according to the target beam output signal;

[0187] Step S32, constructing an azimuth spectrum corresponding to the target beam output signal according to the covariance matrix and a preset azimuth spectrum generation model, wherein the azimuth spectrum is composed of a correlation curve between sound intensity and sound source position;

[0188] Step S33: generating a target sound source position of each target beam output signal according to the azimuth spectrum.

[0189] Exemplarily, steps S31 to S33 include: constructing a covariance matrix of the target beam output signal in the harmonic domain corresponding to the preset microphone array based on the target beam output signal; decomposing the signal space corresponding to the covariance matrix into a noise subspace and a signal subspace, and constructing an azimuth spectrum corresponding to the target beam output signal based on the noise subspace, the signal subspace and a preset azimuth spectrum generation model; determining the number of sound sources corresponding to the sound signal, querying the azimuth spectrum based on the number of sound sources, and obtaining the target sound source position of each target beam output signal.

[0190] Among them, in step S31, the step of constructing a covariance matrix of the target beam output signal in the harmonic domain corresponding to the preset microphone array according to the target beam output signal includes:

[0191] Step B10, determining the noisy relative sound pressure of each sound field corresponding to the target beam output signal according to the target beam output signal;

[0192] Step B20, performing Fourier transform on the noisy relative sound pressure to obtain a harmonic domain coefficient corresponding to the target beam output signal;

[0193] Step B30: construct the covariance matrix according to the harmonic domain coefficients.

[0194] Exemplarily, step B10 to step B30 include: dividing the spatial sound field corresponding to the sound signal into a subspace sound field and a reference spatial sound field based on the preset microphone array; determining the subspace output signal of each subspace sound field based on the target beam output signal; determining the reference spatial signal of the reference microphone position corresponding to the reference spatial sound field and the subspace signal of the submicrophone position corresponding to the subspace output signal; determining the signal-to-noise ratio of the target beam output signal; determining the noisy relative sound pressure of each sound field corresponding to the target beam output signal based on the reference spatial signal at the reference microphone position, the subspace signal at the submicrophone position and the signal-to-noise ratio; performing a spherical Fourier transform on the noisy relative sound pressure to obtain the spherical harmonic domain coefficients corresponding to the target beam output signal; and constructing the covariance matrix based on the spherical harmonic domain coefficients.

[0195] Optionally, the step of determining a reference space signal at a reference microphone position corresponding to the reference space sound field and a subspace signal at a submicrophone position corresponding to the subspace output signal may specifically be:

[0196]

[0197] Among them, y0(k,x q ) is the reference spatial signal of the sub-microphone position, Q is the number of microphones in the spherical array, and y q (k,x q ) is the subspace signal at the qth microphone position, and k is the wave number corresponding to the sound signal.

[0198] Optionally, the step of determining the signal-to-noise ratio of the target beam output signal may specifically be:

[0199]

[0200] Wherein, ρ is the signal-to-noise ratio of the target beam output signal, is the signal parameter in the reference space, is the noise signal in the reference space, and k is the wave number corresponding to the sound signal.

[0201] Optionally, the acquisition of the signal parameters in the reference space may specifically be:

[0202]

[0203] in, is the signal parameter in the reference space, and y0(k, x0) is the reference space signal at the reference microphone position.

[0204] Optionally, the noise signal in the reference space may be obtained by:

[0205]

[0206] in, is the noise signal in the reference space, and n0 is the noise signal at the reference microphone.

[0207] Optionally, the step of determining the noisy relative sound pressure of each sound field corresponding to the target beam output signal based on the reference spatial signal at the reference microphone position, the subspace signal at the submicrophone position and the signal-to-noise ratio may specifically be:

[0208]

[0209] in, is the relative sound pressure of each sound field corresponding to the target beam output signal, y0(k,x0) is the reference spatial signal of the reference microphone position, y q (k,x q ) is the subspace signal at the qth microphone position.

[0210] Optionally, the steering vector corresponding to each microphone may be obtained by:

[0211]

[0212] Among them, v q (k) is the steering vector of the qth microphone, 1, 2, …, l, L are the sound sources, i is the discretization index of the beam sidelobe, and k is the wave number corresponding to the sound signal.

[0213] Optionally, the signal proportion of each sound source may be obtained as follows:

[0214]

[0215] in, is the proportion of the signal from the lth sound source in the signal received by the qth microphone, s l (k) is the output signal of L sound sources in each sound field, and L is the upper limit of the sound source.

[0216] Optionally, the vector of the spatial sound source signal may be obtained as follows:

[0217]

[0218] in, is the vector of spatial sound source signal, is the proportion of the signal from the lth sound source in the signal received by the qth microphone, is the proportion of the signal from the Lth sound source in the signal received by the qth microphone.

[0219] Therefore, according to the steering vector and the vector of the spatial sound source signal, the formula for obtaining the noisy relative sound pressure of each sound field corresponding to the target beam output signal can be converted into:

[0220]

[0221] in, is the relative sound pressure of each sound field with noise corresponding to the target beam output signal, v q (k) is the steering vector of the qth microphone, is the vector of spatial sound source signals, and ρ is the signal-to-noise ratio of the target beam output signal.

[0222] Optionally, the step of performing spherical Fourier transform on the noisy relative sound pressure to obtain the spherical harmonic domain coefficient corresponding to the target beam output signal may specifically be:

[0223]

[0224] in, is the spherical harmonic domain coefficient corresponding to the target beam output signal, j n (kr) is the first kind of spherical Bessel function, Q is the number of microphones in the spherical array, a q is the spherical sampling weight, is the relative sound pressure of each sound field with noise corresponding to the target beam output signal, θ is the sound source pitch angle corresponding to the sound signal, is the sound source azimuth corresponding to the sound signal, is the spherical harmonic function corresponding to the sound signal.

[0225] Optionally, the spherical sampling weight may be obtained by:

[0226]

[0227] Among them, a q is the spherical sampling weight, N is the upper limit of the order of the spherical harmonic function, and θ is the sound source pitch angle corresponding to the sound signal.

[0228] It can be understood that, according to the spherical harmonic domain coefficients corresponding to the smoothed target beam output signal, a covariance matrix is ​​constructed, and the specific steps may be:

[0229]

[0230] in, is the covariance matrix, K is the number of test frequency points selected in the broadband test signal, E is the mean operation, is the spherical harmonic domain coefficient corresponding to the target beam output signal, and H is the conjugate transpose operation.

[0231] Among them, in step S32, the step of constructing the azimuth spectrum corresponding to the target beam output signal according to the covariance matrix and the preset azimuth spectrum generation model includes:

[0232] Step C10, performing singular value decomposition on the covariance matrix to obtain a singular value decomposition result, wherein the singular value decomposition result includes a left singular value matrix and a right singular value matrix;

[0233] Step C20, generating an azimuth spectrum corresponding to the target beam output signal through the preset azimuth spectrum generation model according to the left singular value matrix and the right singular value matrix.

[0234] Exemplarily, steps C10 to C20 include: performing singular value decomposition on the covariance matrix to obtain a singular value decomposition result; decomposing the signal space into a noise subspace and a signal subspace based on the singular value decomposition result; and generating an azimuth spectrum corresponding to the target beam output signal through the preset azimuth spectrum generation model based on the noise subspace and the signal subspace.

[0235] Optionally, the step of performing singular value decomposition on the covariance matrix to obtain a singular value decomposition result may specifically be:

[0236]

[0237] Among them, SVD is the singular value decomposition operation, is the covariance matrix, U is the left singular value matrix, V is the right singular value matrix, and ∑ is the singular value diagonal matrix.

[0238] Wherein, in step S33, the singular value decomposition result also includes a singular value diagonal matrix, and the step of generating the target sound source position of each target beam output signal according to the azimuth spectrum includes:

[0239] Step D10, performing rank processing on the singular value diagonal matrix to obtain the number of sound sources of the target beam output signal;

[0240] Step D20: query the azimuth spectrum to obtain the target sound source position of each target beam output signal according to the number of sound sources.

[0241] Exemplarily, step D10 to step D20 include: performing rank processing on the singular value diagonal matrix to obtain the rank of the singular value diagonal matrix, and using the rank as the number of sound sources of the target beam output signal; based on the number of sound sources, querying the azimuth spectrum to obtain the target sound source position of each target beam output signal.

[0242] As an example, see Figure 3 , Figure 3 Including: a preset microphone array (shown in the figure), a preset azimuth spectrum generation model (the multi-source DOA estimation algorithm based on SHD-MUSIC shown in the figure), a preset acoustic event type recognition model (the acoustic event recognition model based on the time-frequency characteristics of the auditory nerve discharge rate shown in the figure) and a preset decision model (the event decision model based on the bidirectional long short-term memory recurrent neural network shown in the figure). The beam output signal in the spherical harmonic domain is obtained by collecting the spherical microphone array (the signal representation of the spherical harmonic domain shown in the figure), the target sound source position (the acoustic event position shown in the figure) is determined by the preset azimuth spectrum generation model and the beam output signal, and the acoustic event type (the acoustic event type shown in the figure) of each beam output signal in the perception environment corresponding to the spherical microphone array is generated by the preset acoustic event type recognition model and the beam output signal, and each acoustic event type is associated with the corresponding target sound source position through the preset decision model.

[0243] The embodiment of the present application provides an acoustic event localization method. Compared with the method of identifying acoustic events of sound signals by using an acoustic event localization model obtained by simulating sound training at different spatial positions, the embodiment of the present application receives a sound signal through a preset microphone array, and determines a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array; selects a target beam output signal that meets a preset condition from the beam output signal; determines a target sound source position of each target beam output signal according to the target beam output signal; generates an acoustic event type of each target beam output signal in a perception environment corresponding to the preset microphone array according to a frequency domain signal corresponding to the target beam output signal, and associates each acoustic event type with a corresponding target sound source position, obtains a target beam signal by screening the beam output signal of the sound signal, and then determines the target sound source position of the target beam signal, and then generates an acoustic event type of each target sound source position, thereby avoiding the technical defect that when there are many sound sources of indoor sound signals, the acoustic event localization is easily inaccurate due to the low signal-to-noise ratio and high reverberation of the indoor scene, thereby improving the accuracy of acoustic event localization.

[0244] Embodiment 3

[0245] The embodiment of the present application further provides an acoustic event localization device, which is applied to an acoustic event localization device, and the acoustic event localization device includes:

[0246] A receiving module, used to receive a sound signal through a preset microphone array, and determine a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array;

[0247] A selection module, used to select a target beam output signal that meets a preset condition from the beam output signals;

[0248] A determination module, used to determine the target sound source position of each target beam output signal according to the target beam output signal;

[0249] A generation module is used to generate the acoustic event type of each target beam output signal in the perception environment corresponding to the preset microphone array according to the frequency domain signal corresponding to the target beam output signal, and associate each acoustic event type with the corresponding target sound source position.

[0250] Optionally, the receiving module is further used for:

[0251] Obtaining a beam weight corresponding to the preset microphone array and a sound parameter corresponding to the sound signal;

[0252] The sound signal is converted into a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array according to the beam weight and the sound parameter.

[0253] Optionally, the selection module is further used for:

[0254] Obtaining beam weights corresponding to each of the beam output signals;

[0255] Selecting a target beam weight whose corresponding evaluation index satisfies a preset index condition from among the beam weights;

[0256] The beam output signal corresponding to the target beam weight is used as the target beam output signal.

[0257] Optionally, the evaluation index includes a beam directivity factor, a white noise gain, a beam sidelobe response, and a distortion-free response of the beam in the viewing direction. Before the step of selecting a target beam weight whose corresponding evaluation index satisfies a preset index condition from each of the beam weights, the acoustic event localization device is further used to:

[0258] If the distortion-free response of the beam in the viewing angle direction is a preset value, the beam directivity factor is greater than or equal to a preset first threshold, the white noise gain is less than a preset second threshold, and the beam sidelobe response is less than a preset third threshold, then it is determined that the evaluation index corresponding to the beam weight meets the preset index condition, wherein the preset first threshold is greater than the preset second threshold, and the preset first threshold is greater than the preset third threshold.

[0259] Optionally, the determining module is further used for:

[0260] According to the target beam output signal, construct a covariance matrix of the target beam output signal in a harmonic domain corresponding to the preset microphone array;

[0261] According to the covariance matrix and the preset azimuth spectrum generation model, construct an azimuth spectrum corresponding to the target beam output signal, wherein the azimuth spectrum is composed of a correlation curve between sound intensity and sound source position;

[0262] The target sound source position of each target beam output signal is generated according to the azimuth spectrum.

[0263] Optionally, the determining module is further used for:

[0264] Determining, according to the target beam output signal, the noisy relative sound pressure of each sound field corresponding to the target beam output signal;

[0265] Performing Fourier transform on the noisy relative sound pressure to obtain harmonic domain coefficients corresponding to the target beam output signal;

[0266] The covariance matrix is ​​constructed according to the harmonic domain coefficients.

[0267] Optionally, the determining module is further used for:

[0268] Performing singular value decomposition on the covariance matrix to obtain a singular value decomposition result, wherein the singular value decomposition result includes a left singular value matrix and a right singular value matrix;

[0269] According to the left singular value matrix and the right singular value matrix, the azimuth spectrum corresponding to the target beam output signal is generated through the preset azimuth spectrum generation model.

[0270] Optionally, the singular value decomposition result also includes a singular value diagonal matrix, and the generating module is further used for:

[0271] Rank-finding processing is performed on the singular value diagonal matrix to obtain the number of sound sources of the target beam output signal;

[0272] According to the number of sound sources, the azimuth spectrum is queried to obtain the target sound source position of each target beam output signal.

[0273] The acoustic event localization device provided by the present application adopts the acoustic event localization method in the above embodiment to solve the technical problem of low accuracy of acoustic event localization. Compared with the prior art, the beneficial effects of the acoustic event localization device provided by the embodiment of the present application are the same as the beneficial effects of the acoustic event localization method provided by the above embodiment, and other technical features in the acoustic event localization device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0274] Embodiment 4

[0275] An embodiment of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the acoustic event localization method in the above embodiment.

[0276] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. The electronic device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0277] like Figure 5 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0278] Typically, the following systems can be connected to the I / O interface: input devices including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices including, for example, a magnetic tape, a hard disk, etc.; and communication devices. The communication device can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0279] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0280] The electronic device provided by the present application adopts the acoustic event localization method in the above embodiment to solve the technical problem of low accuracy of acoustic event localization. Compared with the prior art, the beneficial effects of the electronic device provided by the embodiment of the present application are the same as the beneficial effects of the acoustic event localization method provided by the above embodiment, and other technical features in the electronic device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0281] It should be understood that the various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0282] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0283] Embodiment 5

[0284] This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the method of the acoustic event localization method in the above embodiment.

[0285] The computer-readable storage medium provided in the embodiment of the present application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a system or a device or used in combination with it. The program code contained on the computer-readable storage medium can be transmitted with any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0286] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0287] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the electronic device: receives a sound signal through a preset microphone array, and determines a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array; selects a target beam output signal that meets preset conditions from the beam output signal; determines the target sound source position of each target beam output signal based on the target beam output signal; generates an acoustic event type of each target beam output signal in a perception environment corresponding to the preset microphone array based on a frequency domain signal corresponding to the target beam output signal, and associates each acoustic event type with a corresponding target sound source position.

[0288] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0289] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0290] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.

[0291] The computer-readable storage medium provided by the present application stores computer-readable program instructions for executing the above-mentioned acoustic event localization method, which solves the technical problem of low accuracy of acoustic event localization. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the embodiment of the present application are the same as the beneficial effects of the acoustic event localization method provided by the above-mentioned implementation, and will not be repeated here.

[0292] Embodiment 6

[0293] The present application also provides a computer program product, including a computer program, which implements the steps of the acoustic event localization method as described above when executed by a processor.

[0294] The computer program product provided by the present application solves the technical problem of low accuracy in acoustic event localization. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as the beneficial effects of the acoustic event localization method provided by the above embodiment, which will not be described in detail here.

[0295] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. An acoustic event localization method, characterized in that: The acoustic event localization method comprises: Receive a sound signal through a preset microphone array, and determine a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array; Obtaining beam weights corresponding to each of the beam output signals; Selecting a target beam weight whose corresponding evaluation index satisfies a preset index condition from each of the beam weights, wherein the evaluation index includes a beam directivity factor, a white noise gain, a beam sidelobe response, and a distortion-free response of the beam in a viewing angle direction; Using the beam output signal corresponding to the target beam weight as the target beam output signal; Determining the target sound source position of each target beam output signal according to the target beam output signal; Generating, according to the frequency domain signal corresponding to the target beam output signal, an acoustic event type of each target beam output signal in the perception environment corresponding to the preset microphone array, and associating each acoustic event type with a corresponding target sound source position; Before the step of selecting a target beam weight whose corresponding evaluation index satisfies a preset index condition from each of the beam weights, the method further includes: If the distortion-free response of the beam in the viewing angle direction is a preset value, the beam directivity factor is greater than or equal to a preset first threshold, the white noise gain is less than a preset second threshold, and the beam sidelobe response is less than a preset third threshold, then it is determined that the evaluation index corresponding to the beam weight meets the preset index condition, wherein the preset first threshold is greater than the preset second threshold, and the preset first threshold is greater than the preset third threshold.

2. The acoustic event localization method according to claim 1, characterized in that: The step of determining a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array comprises: Obtaining a beam weight corresponding to the preset microphone array and a sound parameter corresponding to the sound signal; The sound signal is converted into a beam output signal of the sound signal in a harmonic domain corresponding to the preset microphone array according to the beam weight and the sound parameter.

3. The acoustic event localization method according to claim 1, characterized in that: The step of determining the target sound source position of each sound signal according to the target beam output signal comprises: Constructing a covariance matrix of the target beam output signal in a harmonic domain corresponding to the preset microphone array according to the target beam output signal; According to the covariance matrix and the preset azimuth spectrum generation model, construct an azimuth spectrum corresponding to the target beam output signal, wherein the azimuth spectrum is composed of a correlation curve between sound intensity and sound source position; The target sound source position of each target beam output signal is generated according to the azimuth spectrum.

4. The acoustic event localization method according to claim 3, characterized in that: The step of constructing a covariance matrix of the target beam output signal in a harmonic domain corresponding to the preset microphone array according to the target beam output signal comprises: Determining, according to the target beam output signal, the noisy relative sound pressure of each sound field corresponding to the target beam output signal; Performing Fourier transform on the noisy relative sound pressure to obtain harmonic domain coefficients corresponding to the target beam output signal; The covariance matrix is ​​constructed according to the harmonic domain coefficients.

5. The acoustic event localization method according to claim 3, characterized in that: The step of constructing the azimuth spectrum corresponding to the target beam output signal according to the covariance matrix and the preset azimuth spectrum generation model comprises: Performing singular value decomposition on the covariance matrix to obtain a singular value decomposition result, wherein the singular value decomposition result includes a left singular value matrix and a right singular value matrix; According to the left singular value matrix and the right singular value matrix, the azimuth spectrum corresponding to the target beam output signal is generated through the preset azimuth spectrum generation model.

6. The acoustic event localization method according to claim 5, characterized in that: The singular value decomposition result also includes a singular value diagonal matrix. The step of generating the target sound source position of each target beam output signal according to the azimuth spectrum includes: Rank-finding processing is performed on the singular value diagonal matrix to obtain the number of sound sources of the target beam output signal; According to the number of sound sources, the azimuth spectrum is queried to obtain the target sound source position of each target beam output signal.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the acoustic event localization method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for implementing the acoustic event localization method, and the program for implementing the acoustic event localization method is executed by a processor to implement the steps of the acoustic event localization method according to any one of claims 1 to 6.

Images