Method, device, equipment and computer-readable storage medium for locating sound source orientation

Through microphone linear array and signal processing technology, combined with delay estimation and MVDR algorithm, interference is suppressed, and the high accuracy of sound source positioning is achieved, solving the problem of low sound source positioning accuracy in the prior art.

CN113687305BActive Publication Date: 2024-11-29ZHEJIANG DAHUA TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202110846646.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-26
Publication Date
2024-11-29
Estimated Expiration
2041-07-26

AI Technical Summary

Technical Problem

The prior art has low accuracy in sound source positioning, especially when the sampling rate is low or when there is interference, it is difficult to accurately determine the sound source orientation.

Method used

The microphone linear array is used to receive audio signals, and the time domain data is obtained through preset sampling frequency. Fourier transform is performed after frame-by-frame window processing. Combined with delay estimation algorithm and MVDR algorithm, the airspace matrix filter is used to suppress interference and accurately bit Sound source location.

Benefits of technology

The accuracy and accuracy of sound source positioning are improved, and the precise orientation of the sound source can be efficiently determined in the presence of interference.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113687305B_ABST
    Figure CN113687305B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device, equipment and computer-readable storage medium for locating the orientation of a sound source. The method for locating the orientation of a sound source includes: obtaining an audio signal received by a microphone linear array, and sampling the audio signal at a preset sampling frequency to obtain time domain data of multiple channels; performing frame windowing processing on the time domain data of multiple channels to obtain time domain segmented data of multiple channels; performing Fourier transform on the time domain segmented data of multiple channels to obtain frequency domain segmented data of multiple channels; determining a preliminary search range for the orientation of the sound source based on the frequency domain segmented data of multiple channels; using a preset spatial matrix filter to process the covariance matrix of the frequency domain segmented data of multiple channels to obtain a covariance matrix after spatial filtering; based on the preliminary search range and the covariance matrix after spatial filtering, using the MVDR algorithm to determine the actual position of the orientation of the sound source. The above scheme can efficiently determine the precise orientation of the sound source.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of sound source localization, and in particular to a method, device, equipment and computer-readable storage medium for localizing a sound source. Background Art

[0002] Microphones are a common acoustic-electric transducer for collecting sound signals, especially voice signals. They are widely used in multimedia conferencing, teaching, communications, mechanical equipment vibration and noise detection, military command and reconnaissance, and other fields. Currently, audio acquisition related products on the market are mainly products with a single microphone as the audio signal acquisition unit. Some high-end products use various forms of microphone arrays as sensors for audio signal acquisition and processing. For example, in traditional meetings, a single microphone is usually used to collect the speaker's voice information, and the meeting recording content is manually sorted and proofread after the meeting. However, this is far from meeting the requirements of modern meetings, and the intelligent construction of conference rooms has become a new trend.

[0003] Regarding the positioning of the sound source, the time delay difference-based method can be used to estimate the sound source position, but the accuracy is low. When the sampling rate is low, the angular estimation accuracy of the sound source orientation will drop significantly. The initial angle range can also be preset. When the sound source is not within the preset angle range or there is strong interference in other directions, the positioning performance will decrease. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a method, device, equipment and computer-readable storage medium for locating the direction of a sound source, which can efficiently determine the precise direction of the sound source.

[0005] In order to solve the above problems, the first aspect of the present application provides a method for locating the orientation of a sound source, and the method for locating the orientation of a sound source includes: acquiring an audio signal received by a microphone linear array, and sampling the audio signal at a preset sampling frequency to obtain time domain data of multiple channels; performing frame windowing processing on the time domain data of the multiple channels to obtain time domain segmented data of the multiple channels; performing Fourier transform on the time domain segmented data of the multiple channels to obtain frequency domain segmented data of the multiple channels; based on the frequency domain segmented data of the multiple channels, using a time delay estimation algorithm to determine a preliminary search range of the sound source orientation; using a preset spatial domain matrix filter to process the covariance matrix of the frequency domain segmented data of the multiple channels to obtain a covariance matrix after spatial filtering; based on the preliminary search range and the covariance matrix after spatial filtering, using an MVDR algorithm to determine the actual position of the sound source orientation.

[0006] To solve the above problems, the second aspect of the present application provides a device for locating the orientation of a sound source, the device comprising: a time domain data acquisition module, the time domain data acquisition module is used to acquire an audio signal received by a microphone linear array, and sample the audio signal at a preset sampling frequency to obtain time domain data of multiple channels; a data processing module, the data processing module is used to perform frame windowing processing on the time domain data of the multiple channels to obtain time domain segmented data of the multiple channels; and perform Fourier transform on the time domain segmented data of the multiple channels to obtain multiple The invention relates to a method for determining the frequency domain segmented data of the multiple channels by using a time delay estimation algorithm based on the frequency domain segmented data of the multiple channels; a preliminary positioning module, wherein the preliminary positioning module is used to determine the preliminary search range of the sound source orientation by using a time delay estimation algorithm based on the frequency domain segmented data of the multiple channels; an interference suppression module, wherein the interference suppression module is used to process the covariance matrix of the frequency domain segmented data of the multiple channels by using a preset spatial domain matrix filter to obtain the covariance matrix after spatial domain filtering; and a final positioning module, wherein the final positioning module is used to determine the actual position of the sound source orientation by using an MVDR algorithm based on the preliminary search range and the covariance matrix after spatial domain filtering.

[0007] To solve the above problems, the third aspect of the present application provides a device for locating the orientation of a sound source, wherein the device for locating the orientation of a sound source comprises a linear array of microphones, a processor and a memory connected to each other; the linear array of microphones is used to collect audio signals; the memory is used to store program instructions, and the processor is used to execute the program instructions to implement the method for locating the orientation of a sound source of the first aspect.

[0008] To solve the above-mentioned problem, the fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implements the method for locating the sound source orientation of the first aspect mentioned above.

[0009] The beneficial effects of the present invention are as follows: different from the prior art, after acquiring the audio signal received by the microphone linear array, the present application can use a preset sampling frequency to sample the audio signal to obtain time domain data of multiple channels, and then perform frame windowing processing on the time domain data of multiple channels to obtain time domain segmented data of multiple channels, and perform Fourier transform on the time domain segmented data of multiple channels to obtain frequency domain segmented data of multiple channels; then based on the frequency domain segmented data of multiple channels, a time delay estimation algorithm can be used to determine the preliminary search range of the sound source orientation; then a preset spatial domain matrix filter is used to process the covariance matrix of the frequency domain segmented data of multiple channels to obtain the covariance matrix after spatial domain filtering; then, based on the preliminary search range and the covariance matrix after spatial domain filtering, the MVDR algorithm can be used to determine the actual position of the sound source orientation. Through the above method, a time delay estimation algorithm is first used to pre-estimate the preliminary search range of the sound source orientation, and the regional range of the divided space to which the sound source belongs is determined. Then, the matrix filter designed offline is used to process the covariance matrix of the frequency domain segmented data, which can suppress the influence of interference or noise in other areas. Finally, the MVDR algorithm is used to perform a detailed search within the preliminary search range of the estimated sound source orientation, so that the precise orientation of the sound source can be efficiently determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flow chart of an embodiment of a method for locating the sound source orientation of the present application;

[0011] Figure 2 yes Figure 1 A schematic diagram of a flow chart of an embodiment of step S14;

[0012] Figure 3 is a schematic diagram of the arrangement of the microphone linear array of the present application;

[0013] Figure 4 yes Figure 1 A schematic diagram of a flow chart of an embodiment of step S15;

[0014] Figure 5 and Figure 6 It is a schematic diagram of the amplitude response curve of the passband sector azimuth angles at [25°, 65°] and [85°, 125°] in an application scenario of the present application;

[0015] Figure 7 It is a structural schematic diagram of an embodiment of a device for locating the sound source orientation of the present application;

[0016] Figure 8 It is a structural schematic diagram of an embodiment of a sound source positioning device of the present application;

[0017] Fig. 9 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0018] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0019] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0020] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0021] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of a method for locating the sound source orientation of the present application. The method for locating the sound source orientation in this embodiment includes the following steps:

[0022] S11: Acquire an audio signal received by a microphone linear array, and sample the audio signal using a preset sampling frequency to obtain time domain data of multiple channels.

[0023] A microphone array is a sound collection system that uses multiple microphones to collect sounds from different spatial directions. After multiple microphones are arranged according to specified requirements, the problem of sound source localization can be solved by adding corresponding algorithms. Different arrangements of microphones correspond to different algorithms. In this embodiment, a microphone linear array can be used, that is, multiple microphones are arranged in a row. Figure 3 , Figure 3 It is a schematic diagram of the arrangement of the microphone linear array of the present application. The microphone linear array can receive audio signals corresponding to speech. The received audio signals include audio signals between the sound source and each microphone. Then each array element synchronously amplifies the received audio signals at a preset sampling frequency and transmits them to the signal processing unit for processing to convert the audio signals into processable digital signals, thereby obtaining time domain data of multiple channels.

[0024] S12: performing frame and window processing on the time domain data of the multiple channels to obtain time domain segmented data of the multiple channels.

[0025] After acquiring the continuous time domain data X(t) of multiple channels received by the microphone linear array, the time domain data X(t) of each channel can be framed and windowed using a window of finite length, thereby obtaining time domain segmented data.

[0026] Specifically, the speech signal is a time-varying non-stationary signal, but it has short-term stability. Generally speaking, it is considered that within the time range of 10-30ms, its characteristics are relatively stable. Therefore, it is necessary to perform frame and window processing on the continuous time domain data X(t) of multiple channels, so that each frame signal can be approximately considered as a stable signal. In order to increase the coherence between two frames of data, the framing method used can be the overlapping segmentation method, and the overlap between frames is the frame shift, and the value of the frame shift can be half of the frame length. In addition, the window function used can be a rectangular window, a Hanning window, a Hamming window, a triangular window, etc. For example, the window type can select a Hamming window. The low-pass characteristic of the Hamming window is relatively smooth, which can well reflect the frequency characteristics of short-term signals such as speech.

[0027] S13: Performing Fourier transform on the time-domain segmented data of the multiple channels to obtain frequency-domain segmented data of the multiple channels.

[0028] The time domain segmented data of multiple channels obtained after the frame segmentation and windowing process are subjected to N-point Fourier transform to obtain the frequency domain segmented data X(ω k ).

[0029] Specifically, the audio signal collected by the microphone is a real signal, and the time domain segmented data is transformed by Fourier transform. Assuming that the number of transformation points is N fft , only N fft / 2 points of data have practical significance, and the remaining N fft / 2 points are symmetrical data, and each frequency point can be considered as a narrowband signal. Therefore, the time domain segmented data of each channel can be selected to obtain the first N fft / 2 points of frequency domain data as the transformation result, that is, as the frequency domain segmented data X(ω k ).

[0030] S14: Based on the frequency domain segmented data of the multiple channels, a time delay estimation algorithm is used to determine a preliminary search range of the sound source orientation.

[0031] There are many methods of delay estimation algorithms, such as generalized correlation method, generalized phase spectrum method and bispectral method. Among them, the generalized correlation delay estimation method estimates the time difference between the delays of the signals through the peak value of the autocorrelation function of the signal; for example, the generalized weighted correlation delay estimation algorithm (GCC) mainly improves the signal-to-noise ratio. The correlation function and the power spectrum density function are a pair of Fourier transform pairs. The power spectrum density function is processed in the frequency domain using different weighting functions, thereby realizing the pre-whitening of the signal, that is, enhancing the signal-to-noise ratio.

[0032] Based on the frequency domain segmented data of multiple channels, the relative delay between any two channels can be estimated by using the delay estimation algorithm, and then the approximate azimuth angle θ' of the sound source can be further calculated. Then, with this estimated angle as the center, the initial search range of the sound source azimuth is determined by adding a disturbance angle Δθ to the left and right.

[0033] S15: Using a preset spatial domain matrix filter to process the covariance matrix of the frequency domain segmented data of the multiple channels to obtain a covariance matrix after spatial domain filtering.

[0034] According to the frequency domain segmented data of multiple channels, the corresponding covariance matrix can be determined, and then the covariance matrix of the frequency domain segmented data of multiple channels can be processed by a preset spatial domain matrix filter to obtain the covariance matrix after spatial domain filtering. By introducing the covariance matrix of the data processed by the preset spatial domain matrix filter, the influence of interference or noise within the stop band sector on the direction positioning of the sound source can be suppressed.

[0035] S16: Based on the preliminary search range and the covariance matrix after the spatial filtering, the actual position of the sound source orientation is determined using the MVDR algorithm.

[0036] It can be understood that after obtaining the preliminary search range of the sound source orientation and the covariance matrix after spatial filtering, the spatial spectrum search matrix of the MVDR algorithm can be constructed to obtain the spatial spectrum function. The maximum value search of the function is performed in the spatial spectrum. The θ value corresponding to the spectrum peak is the estimated value of the sound source orientation, and the actual position of the sound source orientation is obtained. At this time, a more accurate sound source localization result is obtained.

[0037] In the above scheme, after acquiring the audio signal received by the microphone linear array, the audio signal can be sampled at a preset sampling frequency to obtain time domain data of multiple channels, and then the time domain data of multiple channels can be framed and windowed to obtain time domain segmented data of multiple channels, and the time domain segmented data of multiple channels can be Fourier transformed to obtain frequency domain segmented data of multiple channels; then based on the frequency domain segmented data of multiple channels, a time delay estimation algorithm can be used to determine the preliminary search range of the sound source orientation; then a preset spatial domain matrix filter is used to process the covariance matrix of the frequency domain segmented data of multiple channels to obtain the covariance matrix after spatial domain filtering; then, based on the preliminary search range and the covariance matrix after spatial domain filtering, the MVDR algorithm can be used to determine the actual position of the sound source orientation. By first using a time delay estimation algorithm to pre-estimate the preliminary search range of the sound source orientation and determining the area range of the divided space to which the sound source belongs, and then using an offline designed matrix filter to process the covariance matrix of the frequency domain segmented data, the influence of interference or noise in other areas can be suppressed. Finally, the MVDR algorithm is used to perform a detailed search within the preliminary search range of the pre-estimated sound source orientation, so that the precise orientation of the sound source can be efficiently determined.

[0038] Please combine Figure 2 , Figure 2 yes Figure 1 Schematic diagram of a flow chart of an embodiment of step S14. In one embodiment, the above step S14 specifically includes:

[0039] S141: Select frequency domain segmented data of the first channel and the second channel, and use a generalized weighted delay estimation algorithm to obtain a relative delay difference.

[0040] In the far field (the sound source is far away from the microphone), it is generally believed that the waveform of the sound is a plane wave, so the audio signal received by each microphone in the microphone linear array will be the same at the same time, that is, there is a relative time delay.

[0041] Furthermore, in one embodiment, the above step S141 may specifically include: obtaining the cross-power spectrum between the frequency domain segmented data of the first channel and the second channel, and obtaining the corresponding time domain correlation function; the maximum value of the time domain correlation function is used as the relative delay difference between the first channel and the second channel.

[0042] Specifically, the relative delay is the time difference between two microphones obtaining the same signal. This embodiment can use a generalized weighted correlation delay estimation algorithm to obtain the cross-power spectrum between the two signals through the first channel and the second channel, and give a certain weight in the frequency domain, and then inversely transform to the time domain to obtain the cross-correlation function between the two signals, and the peak position of the cross-correlation function is the relative delay between the two signals. For example, the first channel mentioned above is the first channel, the second channel is the last channel, the microphone linear array includes M microphones, and the frequency domain segmented data corresponding to the first microphone and the Mth microphone in the microphone linear array are selected, and the cross-power spectrum between the two signals is calculated as: Then the cross-correlation function between the two signals is obtained as: Then the relative time delay between the 1st microphone and the Mth microphone is the maximum value of the cross-correlation function, that is:

[0043] S142: Calculate a preliminary search range of the sound source orientation by using the relative delay difference, the sampling frequency, and the microphone spacing information between the first channel and the second channel.

[0044] Specifically, the above-mentioned step S142 may specifically include: obtaining an estimated angle of the sound source orientation according to the relative delay difference, the sampling frequency and the microphone spacing information between the first channel and the second channel; based on the estimated angle of the sound source orientation, adding an error angle to obtain a preliminary search range of the sound source orientation.

[0045] Please combine Figure 3 , the relative delay between the two microphones The distance d between the two microphones has the following relationship: Then we can get the estimated angle of the sound source relative to the microphone linear array: Where c is the propagation speed of sound. In addition, due to various interferences in the real environment, there may be a large error between the real angle θ of the sound source orientation and the estimated angle θ'. Therefore, a disturbance angle Δθ should be added to the estimated angle θ' with the estimated angle θ' as the center, and the initial search range of the sound source orientation is [θ-Δθ,θ+Δθ]. It can be understood that the larger the selected value of the disturbance angle Δθ, the greater the corresponding amount of calculation. When the noise interference is small, the estimated angle θ' is closer to the real angle θ, and a smaller disturbance angle Δθ can be selected, so that a higher estimation accuracy can be achieved based on a smaller amount of calculation.

[0046] Please combine Figure 4 , Figure 4 yes Figure 1Schematic diagram of a flow chart of an embodiment of step S15. In one embodiment, the above step S15 specifically includes:

[0047] S151: Obtain the covariance matrix of the frequency domain segmented data of the multiple channels.

[0048] Specifically, the above step S151 specifically includes: dividing the frequency domain segmented data of the multiple channels into N s sub-bands, each of which contains multiple frequency points; the covariance matrix of each frequency point in each sub-band is averaged to obtain the covariance matrix of each sub-band

[0049] S152: Divide the spatial angle into a number of passband regions, obtain a preset spatial domain matrix filter corresponding to each passband region, and determine a corresponding spatial domain filter matrix.

[0050] In the process of locating the sound source, there is often noise and interference from other directions. The spatial matrix filter can allow the signal in the direction of interest to pass through, and suppress the signal in other directions of no interest. Its output is still an array signal. Therefore, an N×N matrix G can be introduced as a filter to filter the audio signal received by the microphone linear array. In order to make the matrix filter have the characteristics of spatial filtering, so that the array signal after spatial filtering can suppress the signal in the direction of no interest, and allow the signal in the direction of interest to pass without distortion, the following constraints need to be met:

[0051]

[0052] where Θ p represents the azimuth of the sector of interest, containing the information that is expected to be processed, called the passband sector, Θ s The sector angle where the signal needs to be suppressed is called the stopband sector.

[0053] Therefore, for the design of spatial matrix filter, it is necessary to first determine the angle range of passband sector and stopband sector and discretize the azimuth. The design criteria can be the minimum mean square error criterion, the stopband constrained passband minimum mean square error criterion, etc. Taking the stopband constrained passband minimum mean square error criterion as an example, the spatial matrix filter can be expressed as:

[0054]

[0055] The above formula can limit the noise attenuation rate within the stopband sector and the noise power after filtering.

[0056] In an application scenario, the spatial angle can be divided into four regions [25°, 65°], [55°, 95°], [85°, 125°] and [115°, 155°], and the four angle regions are used as passband regions to design the spatial matrix filter offline. Then the filter is solved offline through the MATLAB toolkit CVX. For example, the number of microphones in the microphone linear array is 4, the sampling frequency is 16000Hz, the distance between two adjacent microphones is 35mm, and the frequency is selected between 1kHz-3KHz. The amplitude response curves of the passband fan azimuth angles at [25°, 65°] and [85°, 125°] can be obtained, as shown in Figure 5 and Figure 6 As shown, Figure 5 It is a schematic diagram of the amplitude response curve when the passband fan azimuth angle is [25°, 65°]. Figure 6 It is a schematic diagram of the amplitude response curve when the passband fan azimuth angle is [85°, 125°].

[0057] S153: Determine a target passband region corresponding to the preliminary search range of the sound source orientation.

[0058] S154: Using the spatial domain filter matrix corresponding to the target passband region, the covariance matrix of the frequency domain segmented data of the multiple channels is processed to obtain a covariance matrix after spatial domain filtering.

[0059] In the above steps, the estimated angle θ' of the sound source orientation can be obtained, and the preliminary search range of the sound source orientation can be expressed as [θ-Δθ, θ+Δθ]. Therefore, it can be determined that the preliminary search range of the sound source orientation belongs to a certain passband area divided in step S152. Then, the spatial domain filter matrix Gω) corresponding to the passband area can be selected to perform covariance matrices of different frequency bands. Perform spatial filtering to obtain the final covariance matrix after spatial filtering:

[0060] Further, in one embodiment, the above step S16 may specifically include: inverting the covariance matrix of each sub-band respectively to construct a spatial spectrum search matrix; wherein each sub-band corresponds to one spatial spectrum search matrix; for any angle, calculating the spatial spectrum function value of each sub-band according to the spatial spectrum search matrix, and performing weighted averaging on the spatial spectrum function values ​​of all sub-bands to obtain a final spatial spectrum function value of the angle; and taking the angle corresponding to the largest final spatial spectrum function value as the actual position of the sound source orientation.

[0061] It can be understood that for any search angle, after obtaining the covariance matrix of each sub-band After that, the covariance matrix R(ω) of each sub-band can be calculated separately. k ) is inverted to construct the spatial spectrum search matrix P of the MVDR algorithm. MVDR (θ), since each sub-band corresponds to a spatial spectrum search matrix, each sub-band can calculate a spatial spectrum function value, then N s There are N sub-bands corresponding to s The spatial spectrum search matrix is ​​N s The spatial spectrum function value, for N s The spatial spectrum function values ​​corresponding to the spatial spectrum search matrices are weighted averaged to obtain the final spatial spectrum function value corresponding to the search angle. Then, all the searched spectrum function values ​​are sorted to obtain the maximum value, and the angle corresponding to the maximum value is the actual position of the sound source.

[0062] In addition, there are two ways to search for the actual position of the sound source: one is to estimate the position of each sub-band. s A total of N sub-bands can be estimated s The final sound source angle estimate can be obtained by weighted averaging the sound source position coordinates. w k is the weight coefficient of the kth sub-band; another method is to convert N s The spatial spectrum search matrix at each sub-band is weighted averaged, and the final spatial spectrum function obtained after weighted average The angle corresponding to the maximum value of the final spatial spectrum function is taken as the actual position of the sound source.

[0063] See also Figure 7 , Figure 7It is a structural diagram of an embodiment of a device for locating the sound source orientation of the present application. The device for locating the sound source orientation 70 in this embodiment includes a time domain data acquisition module 700, a data processing module 702, a preliminary positioning module 704, an interference suppression module 706 and a final positioning module 708 that are interconnected; the time domain data acquisition module 700 is used to acquire the audio signal received by the microphone linear array, and sample the audio signal at a preset sampling frequency to obtain time domain data of multiple channels; the data processing module 702 is used to perform frame and window processing on the time domain data of the multiple channels to obtain time domain segmented data of the multiple channels; and the time domain data of the multiple channels are processed by the time domain processing module 702. The frequency domain segmented data is subjected to Fourier transformation to obtain frequency domain segmented data of multiple channels; the preliminary positioning module 704 is used to determine the preliminary search range of the sound source orientation by using a delay estimation algorithm based on the frequency domain segmented data of the multiple channels; the interference suppression module 706 is used to process the covariance matrix of the frequency domain segmented data of the multiple channels by using a preset spatial domain matrix filter to obtain the covariance matrix after spatial domain filtering; the final positioning module 708 is used to determine the actual position of the sound source orientation by using the MVDR algorithm based on the preliminary search range and the covariance matrix after spatial domain filtering.

[0064] In one embodiment, the preliminary positioning module 704 executes the steps of determining the preliminary search range of the sound source orientation based on the frequency domain segmented data of the multiple channels by using a time delay estimation algorithm, including: selecting the frequency domain segmented data of the first channel and the second channel, and using a generalized weighted delay estimation algorithm to obtain a relative delay difference; and calculating the preliminary search range of the sound source orientation by using the relative delay difference, the sampling frequency, and the microphone spacing information between the first channel and the second channel.

[0065] In one embodiment, the preliminary positioning module 704 executes the steps of selecting the frequency domain segmented data of the first channel and the second channel, and using the generalized weighted delay estimation algorithm to obtain the relative delay difference, including: obtaining the cross-power spectrum between the frequency domain segmented data of the first channel and the second channel, and obtaining the corresponding time domain correlation function; and taking the maximum value of the time domain correlation function as the relative delay difference between the first channel and the second channel.

[0066] In one embodiment, the preliminary positioning module 704 executes the steps of calculating a preliminary search range of the sound source orientation through the relative delay difference, the sampling frequency, and the microphone spacing information between the first channel and the second channel, including: obtaining an estimated angle of the sound source orientation according to the relative delay difference, the sampling frequency, and the microphone spacing information between the first channel and the second channel; and adding an error angle on the basis of the estimated angle of the sound source orientation to obtain a preliminary search range of the sound source orientation.

[0067] In one embodiment, the interference suppression module 706 executes the steps of processing the covariance matrix of the frequency domain segmented data of the multiple channels using a preset spatial matrix filter to obtain the covariance matrix after spatial filtering, including: obtaining the covariance matrix of the frequency domain segmented data of the multiple channels; dividing the spatial angle into a plurality of passband regions by using a preset spatial matrix filter; determining the target passband region corresponding to the preliminary search range of the sound source orientation; and processing the covariance matrix of the frequency domain segmented data of the multiple channels using the spatial filter matrix corresponding to the target passband region to obtain the covariance matrix after spatial filtering.

[0068] In one embodiment, the interference suppression module 706 performs the step of obtaining the covariance matrix of the frequency domain segmented data of the multiple channels, including: dividing the frequency domain segmented data of the multiple channels into N s sub-bands, each sub-band contains multiple frequency points; the covariance matrix of each sub-band is obtained by averaging the covariance matrices of each frequency point in each sub-band.

[0069] In one embodiment, the final positioning module 708 performs the step of determining the actual position of the sound source orientation using the MVDR algorithm based on the preliminary search range and the covariance matrix after the spatial filtering, including: inverting the covariance matrix of each sub-band respectively to construct a spatial spectrum search matrix; wherein each sub-band corresponds to one spatial spectrum search matrix; performing weighted averaging on all spatial spectrum search matrices to obtain a final spatial spectrum function; and taking the angle corresponding to the maximum value of the final spatial spectrum function as the actual position of the sound source orientation.

[0070] For specific contents of the method for locating the sound source orientation implemented by the sound source orientation locating device 70 of the present application, please refer to the contents in the above-mentioned embodiment of the method for locating the sound source orientation, which will not be repeated here.

[0071] See also Figure 8 , Figure 8 1 is a schematic diagram of the structure of an embodiment of the sound source positioning device of the present application. The sound source positioning device 80 in this embodiment includes a microphone linear array 800, a processor 802 and a memory 804 connected to each other; the microphone linear array 800 is used to collect audio signals, the memory 804 is used to store program instructions, and the processor 802 is used to execute the program instructions stored in the memory 804 to implement the steps of any of the above-mentioned sound source positioning method embodiments. In a specific implementation scenario, the sound source positioning device 80 may include but is not limited to: a microcomputer, a server.

[0072] Specifically, the processor 802 is used to control itself and the memory 804 to implement the steps of any of the above-mentioned sound source orientation positioning method embodiments. The processor 802 can also be called a CPU (Central Processing Unit). The processor 802 may be an integrated circuit chip with signal processing capabilities. The processor 802 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 802 can be implemented by an integrated circuit chip.

[0073] In the above scheme, after acquiring the audio signal received by the microphone linear array 800, the processor 802 can sample the audio signal using a preset sampling frequency to obtain time domain data of multiple channels, and then perform frame and window processing on the time domain data of the multiple channels to obtain time domain segmented data of the multiple channels, and perform Fourier transform on the time domain segmented data of the multiple channels to obtain frequency domain segmented data of the multiple channels; then based on the frequency domain segmented data of the multiple channels, a time delay estimation algorithm can be used to determine the preliminary search range of the sound source orientation; then a preset spatial domain matrix filter is used to process the covariance matrix of the frequency domain segmented data of the multiple channels to obtain the covariance matrix after spatial domain filtering; thereafter, based on the preliminary search range and the covariance matrix after spatial domain filtering, the MVDR algorithm can be used to determine the actual position of the sound source orientation. By first using a time delay estimation algorithm to pre-estimate the preliminary search range of the sound source orientation and determining the area range of the divided space to which the sound source belongs, and then using an offline designed matrix filter to process the covariance matrix of the frequency domain segmented data, the influence of interference or noise in other areas can be suppressed. Finally, the MVDR algorithm is used to perform a detailed search within the preliminary search range of the pre-estimated sound source orientation, so that the precise orientation of the sound source can be efficiently determined.

[0074] See also Fig. 9 , Fig. 9 The computer-readable storage medium 90 of the present application stores program instructions 900 thereon, which implement the steps in any of the above-mentioned sound source orientation positioning methods when executed by a processor.

[0075] The computer-readable storage medium 90 may specifically be a medium that can store the program instructions 900, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or may be a server that stores the program instructions 900. The server may send the stored program instructions 900 to other devices for execution, or may execute the stored program instructions 900 by itself.

[0076] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and apparatuses can be implemented in other ways. For example, the above-described device and apparatus implementation methods are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0077] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0078] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0079] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. A method for locating the direction of a sound source, characterized in that: The method for locating the sound source orientation includes: Acquire an audio signal received by a microphone linear array, and sample the audio signal using a preset sampling frequency to obtain time domain data of multiple channels; Performing frame and window processing on the time domain data of the multiple channels to obtain time domain segmented data of the multiple channels; Performing Fourier transform on the time-domain segmented data of the multiple channels to obtain frequency-domain segmented data of the multiple channels; Based on the frequency domain segmented data of the multiple channels, a time delay estimation algorithm is used to determine a preliminary search range of the sound source orientation; Using a preset spatial domain matrix filter to process the covariance matrix of the frequency domain segmented data of the multiple channels to obtain a covariance matrix after spatial domain filtering; Based on the preliminary search range and the covariance matrix after the spatial domain filtering, the actual position of the sound source orientation is determined by using the MVDR algorithm; The method of determining the preliminary search range of the sound source position based on the frequency domain segmented data of the multiple channels by using a time delay estimation algorithm includes: Selecting the frequency domain segmented data of the first channel and the second channel, and using a generalized weighted delay estimation algorithm to obtain a relative delay difference; Calculate a preliminary search range of the sound source orientation by using the relative delay difference, the sampling frequency, and the microphone spacing information between the first channel and the second channel; The step of calculating a preliminary search range of the sound source orientation by using the relative delay difference, the sampling frequency, and the microphone spacing information between the first channel and the second channel comprises: Obtaining an estimated angle of the sound source orientation according to the relative delay difference, the sampling frequency, and the microphone spacing information between the first channel and the second channel; Based on the estimated angle of the sound source orientation, the error angle is added to obtain a preliminary search range of the sound source orientation.

2. The method for locating the sound source orientation according to claim 1, characterized in that: The selecting the frequency domain segmented data of the first channel and the second channel and using the generalized weighted delay estimation algorithm to obtain the relative delay difference includes: Obtaining a cross power spectrum between the frequency domain segmented data of the first channel and the second channel, and obtaining a corresponding time domain correlation function; The maximum value of the time domain correlation function is used as the relative delay difference between the first channel and the second channel.

3. The method for locating the sound source orientation according to claim 1, characterized in that: The method of using a preset spatial domain matrix filter to process the covariance matrix of the frequency domain segmented data of the multiple channels to obtain a covariance matrix after spatial domain filtering includes: Obtaining a covariance matrix of the frequency domain segmented data of the multiple channels; Divide the spatial angle into a number of passband regions, obtain a preset spatial matrix filter corresponding to each passband region, and determine a corresponding spatial filter matrix; Determining a target passband region corresponding to a preliminary search range of the sound source orientation; The covariance matrix of the frequency domain segmented data of the multiple channels is processed by using the spatial domain filter matrix corresponding to the target passband region to obtain the covariance matrix after spatial domain filtering.

4. The method for locating the sound source orientation according to claim 3, characterized in that: The obtaining of the covariance matrix of the frequency domain segmented data of the multiple channels includes: The frequency domain segmented data of the multiple channels are divided into N s sub-bands, each sub-band contains multiple frequency points; The covariance matrix of each sub-band is obtained by averaging the covariance matrices of each frequency point in each sub-band.

5. The method for locating the sound source orientation according to claim 4, characterized in that: The method of determining the actual position of the sound source orientation by using the MVDR algorithm based on the preliminary search range and the covariance matrix after the spatial domain filtering includes: The covariance matrix of each sub-band is inverted to construct a spatial spectrum search matrix; each sub-band corresponds to one spatial spectrum search matrix; For any angle, the spatial spectrum function value of each sub-frequency band is calculated according to the spatial spectrum search matrix, and the spatial spectrum function values ​​of all sub-frequency bands are weighted averaged to obtain the final spatial spectrum function value of the angle; The angle corresponding to the maximum final spatial spectrum function value is used as the actual position of the sound source orientation.

6. A device for locating the direction of a sound source, characterized in that: The device for locating the sound source orientation comprises: A time domain data acquisition module, the time domain data acquisition module is used to acquire the audio signal received by the microphone linear array, and sample the audio signal using a preset sampling frequency to obtain time domain data of multiple channels; A data processing module, the data processing module is used to perform frame windowing processing on the time domain data of the multiple channels to obtain time domain segmented data of the multiple channels; and perform Fourier transform on the time domain segmented data of the multiple channels to obtain frequency domain segmented data of the multiple channels; A preliminary positioning module, the preliminary positioning module is used to determine a preliminary search range of the sound source position by using a time delay estimation algorithm based on the frequency domain segmented data of the multiple channels; An interference suppression module, the interference suppression module is used to process the covariance matrix of the frequency domain segmented data of the multiple channels using a preset spatial domain matrix filter to obtain a covariance matrix after spatial domain filtering; A final positioning module, the final positioning module is used to determine the actual position of the sound source orientation using the MVDR algorithm based on the preliminary search range and the covariance matrix after the spatial filtering; The preliminary positioning module is used to determine the preliminary search range of the sound source orientation based on the frequency domain segmented data of the multiple channels by using a time delay estimation algorithm, including: selecting the frequency domain segmented data of the first channel and the second channel, and using a generalized weighted time delay estimation algorithm to obtain a relative delay difference; calculating the preliminary search range of the sound source orientation by using the relative delay difference, the sampling frequency and the microphone spacing information between the first channel and the second channel; The preliminary positioning module executes the steps of calculating a preliminary search range of the sound source orientation through the relative delay difference, the sampling frequency and the microphone spacing information between the first channel and the second channel, including: obtaining an estimated angle of the sound source orientation according to the relative delay difference, the sampling frequency and the microphone spacing information between the first channel and the second channel; and adding an error angle on the basis of the estimated angle of the sound source orientation to obtain the preliminary search range of the sound source orientation.

7. A device for locating the direction of a sound source, characterized in that: The sound source orientation positioning device includes a microphone linear array, a processor and a memory connected to each other; The microphone linear array is used to collect audio signals; the memory is used to store program instructions, and the processor is used to execute the program instructions to implement the method for locating the sound source orientation as described in any one of claims 1-5.

8. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method for locating the sound source orientation described in any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Method for sound source direction estimation based on time frequency masking and deep neural network

    CN109839612A

  • MVDR target sound source directional pickup method for microphone array

    CN111044973A

  • Sound source positioning method and device, storage medium and electronic equipment

    CN112799017A