SRP-PHAT sound source localization method and device based on FPGA

Abstract

The invention discloses an SRP‑PHAT sound source localization method and device based on FPGA, and relates to the technical field of sound source localization. The method comprises: receiving multi-channel audio data of a single-frame microphone array, grouping and cache rearrangement of input data according to channel sequence numbers, performing FFT operations and bandpass filtering operations in sequence from a first group of cached data streams to an nth group of cached data streams, each group of cached data streams being used as a subarray for cross-power spectrum operation, and outputting filtered data forming an n×n array from the subarrays; calculating the cross-power spectrum results of each subarray above the main diagonal of the n×n array, obtaining the cross-power spectrum of the entire array by taking a conjugate, and reordering the cross-power spectrum results; calculating characteristic frequency points of parallel input data, calculating a steering vector, and performing a plane scanning operation with the cross-power spectrum results. The invention realizes the SRP‑PHAT sound source localization method on the FPGA by optimizing the operation structure and method inside the FPGA.

Description

Technical Field

[0001] The present invention relates to a sound source localization technology, and in particular to a sound source localization method and device based on FPGA. Background Art

[0002] Sound is one of the important ways for humans to perceive the world. In daily life, the human ear can easily determine the source of sound and distinguish the sound characteristics within a certain frequency band. However, in professional fields such as industrial production, military, and aerospace, most noise and abnormal sounds exist in a wide frequency band and low power. Some even exist in the infrasound (frequency less than 20 Hz) or ultrasonic band (frequency greater than 20,000 Hz), which cannot be identified by the human ear alone. This requires the use of more accurate and sensitive test equipment. In many scenarios, there is a need for accurate and rapid sound source positioning, such as mechanical fault diagnosis, factory gas leak identification, power system partial discharge detection, submarine sonar underwater tracking, etc.

[0003] The sound source localization technology uses multiple microphones to form a sensor array. There is a phase difference when the sound signal reaches different microphones, and the algorithm is used to obtain the relative position of the sound source in space. However, the SRP-PHAT algorithm needs to scan the entire receiving plane block by block. As the number of microphone array channels and image pixels increase, the amount of data calculation increases exponentially. Figure 1 The existing method uses a powerful PC or accelerator card to perform real-time calculations on a large number of microphone channels. At the same time, the hardware system needs to be equipped with a parallel analog-to-digital conversion driver for the microphone array. The entire computing system is relatively complex, the equipment is large, and it is difficult to use flexibly. Figure 2 On this basis, the algorithm was optimized. The use of space shrinkage or clustering methods can effectively reduce the amount of calculation, but the positioning sensitivity and accuracy of this method will be greatly affected. Summary of the invention

[0004] The present invention aims at the shortcomings of the prior art and provides a SRP-PHAT sound source localization method based on FPGA. In order to solve the above technical problems, the present invention is solved by the following technical scheme: A SRP-PHAT sound source localization method based on FPGA, comprising the following steps:

[0005] Receive multi-channel audio data of a single frame of microphone array, where the data depth N is determined by a preset FFT point number;

[0006] The input data is grouped and cached according to the channel sequence number to form several groups of data streams. The first group of cached data streams to the nth group of cached data streams are sequentially subjected to FFT operations and bandpass filtering operations. Each group of cached data streams is used as a subarray for cross-power spectrum operation. The output filtered data is composed of subarrays to form an n×n array.

[0007] Calculate the cross power spectrum of each subarray above the main diagonal of the n×n array, and supplement the uncalculated part below the main diagonal by taking the conjugate method to obtain the cross power spectrum of the entire array, and reorder the cross spectrum power results according to the matrix row × column order;

[0008] The characteristic frequency points of the parallel input data are calculated, the steering vector is calculated according to the microphone coordinate data, a plane scanning operation is performed with the cross-power spectrum result, and then the sound pressure distribution diagram of the sound in the plane is obtained through sound pressure level conversion.

[0009] The present application also proposes an FPGA sound source localization device, comprising the following structure:

[0010] An input unit receives multi-channel audio data of a single frame of a microphone array, wherein the data depth N is determined by a preset FFT point number;

[0011] A reordering unit, grouping and cache reordering the input data according to the channel sequence number to form a plurality of groups of data streams, the input data including the first group of cache data streams to the nth group of cache data streams;

[0012] An FFT and filtering operation unit is used to receive the first group of cache sequences to the nth group of cache sequences in time sequence, and perform FFT operation and bandpass filtering operation, and output the filtered data into an n×n array to the cross-power spectrum operation unit;

[0013] A cross power spectrum calculation unit calculates the cross power spectrum result of each sub-array above the main diagonal in the n×n array;

[0014] A rearrangement operation unit, connected to the cross-power spectrum operation unit, is used to supplement the uncalculated part below the main diagonal by taking a conjugate method to obtain the cross-power spectrum of the entire array, and reorder the cross-spectrum power results according to the matrix row × column order;

[0015] A characteristic frequency point calculation unit calculates the characteristic frequency point of each frame of parallel input data;

[0016] A steering vector calculation unit, which calculates a steering vector according to the microphone coordinate data;

[0017] The plane scanning unit performs plane scanning operation according to the steering vector and the cross power spectrum results, and then obtains the sound pressure distribution diagram of the sound in the plane through sound pressure level conversion.

[0018] The beneficial effects of the present invention are as follows: through the algorithm flow and structure processing at the FPGA level, a multi-level buffered pipeline operation is adopted to enable the deployment on the FPGA to be satisfied without losing the calculation accuracy and speed. At the same time, the array decomposition operation is performed by using the conjugated property of the mutual power spectrum, which saves the computing resources and provides a further advantage for satisfying the deployment on the FPGA.

[0019] Preferably, the FFT operation unit includes a butterfly operation unit for butterfly operation of FFT, and in each step of butterfly operation, an adaptive scaling factor is determined according to the current calculation result, and the butterfly operation of each step is calculated according to the adaptive scaling factor calculated each time. This solution can further save computing resources and speed up the computing speed.

[0020] Preferably, a BRAM multiplexing unit is also included, which is used to group and cache the steering vectors generated by the imaginary sound source in a multi-channel parallel manner, and each group of data is sequentially subjected to steering vector calculation according to the timing, and the steering vector results obtained are returned to the original cache space for caching, and when all steering vector results are calculated, they are output to perform plane scanning. The method of the BRAM multiplexing unit can also further save computing resources.

[0021] Preferably, the plane scanning unit is also used to perform scanning operations in a multi-channel parallel manner, and perform scanning operations according to the multi-channel output guidance vector results. Preferably, it also includes an image reorganization unit, which is used to reorganize the multi-channel parallel output after the plane scanning into a row × column format pixel image. The use of a multi-channel parallel method can further speed up the operation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 It is a schematic diagram of the first prior art solution disclosed in the background technology;

[0024] Figure 2 is a schematic diagram of a second prior art solution disclosed in the background art;

[0025] Figure 3 It is a schematic diagram of the FPGA sound source localization device disclosed in this application;

[0026] Figure 4 It is a flow chart of the SRP-PHAT sound source localization method based on FPGA implemented in this application;

[0027] Figure 5 It is a schematic diagram of pipeline FFT operation in the prior art;

[0028] Figure 6 It is a schematic diagram of the pipeline FFT operation disclosed in the present application;

[0029] Figure 7It is a schematic diagram of the principle of realizing cross-power spectrum calculation through conjugation properties disclosed in the present application;

[0030] Figure 8 It is a schematic diagram of the result of realizing the cross power spectrum operation through the conjugate property disclosed in the present application;

[0031] Fig. 9 It is a schematic diagram of the BRAM cross-multiplexing structure;

[0032] Fig.10 This is a schematic diagram of the sound pressure distribution results of the SRP-PHAT sound source localization method implemented based on FPGA. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items. The "first" and "second" involved in the present invention do not represent specific quantities and sequences, but are merely used to distinguish names.

[0035] At present, the commonly used SRP-PHAT algorithm realizes the sound source point positioning process as follows:

[0036] Calculate the function sum of the GCC-PHAT (generalized cross-correlation-phase transform) algorithm of the received signals of all microphones in the array, traverse the entire sound source space to image the plane sound pressure distribution, and the point that can make the SRP value the largest is the sound source position.

[0037] The number of sensors in the microphone array is M. The signal received at the center of the array is s(t). τ is the time it takes for the sound source to propagate to the microphone, and n(t) is the noise collected by the microphone. The signal received by the jth sensor at time t is:

[0038] Formula 1

[0039] In Formula 1, j = 1, ..., M.

[0040] Perform FFT calculation on the received signal to obtain , frequency domain filtering is performed according to the target analysis frequency band, f1 is the starting frequency, and f2 is the ending frequency:

[0041] Formula 2

[0042] Formula 3

[0043] Formula 4

[0044] Taking the conjugate gives the cross-power spectrum:

[0045] Formula 5

[0046] Calculate the steering vector: define q as the spatial point position coordinate, and the controllable response power P(q) is:

[0047] Formula 6

[0048] in represents the arrival time difference of the imaginary sound source to the lth and mth microphones, , Respectively represent the rectangular coordinate vectors of the two primitives, and c is the speed of sound propagation in the atmosphere:

[0049] Formula 7

[0050] The above algorithm process contains a large number of complex matrix operations. According to the algorithm derivation process, it cannot be implemented using FPGA considering resource occupation. If a pipeline method is used, the implementation process will take a long time and the resulting image will not meet the minimum dynamic recognition frame number required by the human eye. Therefore, it is necessary to design the algorithm process in combination with the FPGA structure.

[0051] The following discloses an SRP-PHAT sound source localization method based on FPGA, which is used to solve the problem that the SRP-PHAT sound source localization algorithm contains a large number of complex matrix operations and cannot be implemented using FPGA due to resource occupation. The disclosed method completely deploys the SRP-PHAT algorithm on FPGA, and realizes the application of multi-channel real-time sound source localization technology on FPGA chip by optimizing the algorithm flow at the FPGA level.

[0052] The present disclosure uses FPGA to design and implement the SRP-PHAT algorithm process, and takes the implementation of 256-channel sound source localization on AMD Xilinx ZYNQ7000 series chips as an example to illustrate the solution.

[0053] The specific method is as follows:

[0054] A SRP-PHAT sound source localization method based on FPGA, such as Figure 3 and Figure 4, including the following steps:

[0055] Receive multi-channel audio data of a single frame of microphone array, where the data depth N is determined by a preset FFT point number;

[0056] The input data is grouped and cached according to the channel sequence number to form several groups of data streams. The first group of cached data streams to the nth group of cached data streams are sequentially subjected to FFT operations and bandpass filtering operations. Each group of cached data streams is used as a subarray for cross-power spectrum operation. The output filtered data is composed of subarrays to form an n×n array.

[0057] Calculate the cross power spectrum of each subarray above the main diagonal of the n×n array, and supplement the uncalculated part below the main diagonal by taking the conjugate method to obtain the cross power spectrum of the entire array, and reorder the cross power spectrum results according to the matrix row × column order;

[0058] The characteristic frequency points of the parallel input data are calculated, the steering vector is calculated according to the microphone coordinate data, a plane scanning operation is performed with the cross-power spectrum result, and then the sound pressure distribution diagram of the sound in the plane is obtained through sound pressure level conversion.

[0059] Taking 256 input channels as an example, according to the rearrangement of the input data cache, several groups of data streams are formed and specifically implemented as follows:

[0060] The 256-channel microphone array sampling data of the parallel input module is cached for one frame according to the FFT points, and then divided into 8 groups of new sequences. Each group of sequences consists of 32 microphone sampling signals, and then they are input into the FFT conversion module and the filtering module in sequence to achieve the purpose of time-division multiplexing resources and reduce resource consumption.

[0061] like Figure 5 and Figure 6 By comparison, the cross-power spectrum data processing structure in the prior art and the previous FFT operation part are ordinary. Each data point performs FFT operation in a pipeline manner, and then all parallel data performs cross-power spectrum operation, which will occupy a large amount of FPGA computing resources.

[0062] The present invention divides the sampling signals of 256 microphone arrays into 8 groups, each group consists of 32 parallel signal points. Each group of sequences is operated and translated in sequence according to the timing. During the operation and translation process, the FFT operation function and filtering function in the FPGA are reused in sequence by all sequences. Experiments show that the pipeline operation method has the same result as the parallel operation of 256 signal points, but the occupied resources are greatly reduced.

[0063] The preferred FFT operation method, or the FFT operation method on FPGA includes: in each step of butterfly operation, determining the scaling factor according to the current calculation result, and adjusting the final output according to the scaling factor to achieve the purpose of adaptive quantization, reduce resource consumption and reduce the precision loss caused by quantization.

[0064] Exemplary description: In each level of butterfly operation, the Twiddle factor W is calculated and a butterfly operation is performed.

[0065] After each butterfly operation, the maximum absolute value max_abs of the current result is detected. A suitable scaling factor scale_factor is determined based on the maximum absolute value max_abs. If the maximum absolute value max_abs is greater than a preset threshold (e.g., 1.0), the scaling factor is 1.0 / max_abs, otherwise it is 1.0. This method is used in this solution to further optimize resource utilization.

[0066] The power spectrum calculation method includes:

[0067] The power spectrum operation unit receives 8 sets of data sequences output by filtering, namely G1 (E1-E32), G2 (E33-E64), G2 (E65-E96), G2 (E97-E128), G2 (E129-E160), G6 (E161-E192), G7 (E193-E224), G8 (E225-E256). The method of making cross power spectrum for the output sequence is:

[0068] 1) The 8 output sequences are sorted by row-column conversion to form a data array of 32 rows × 32 columns × 64 groups (G1-G64 shown in the figure). The array is 8×8 groups of array data, including 64 groups of sub-arrays.

[0069] 2) Calculate the cross power spectrum results of 36 groups of 32 rows × 32 columns diagonally across the 8 × 8 array data; as shown in the darkened part of the array data in the upper right corner of the figure.

[0070] 3) According to the conjugate symmetry property of the cross power spectrum, the remaining cross power spectrum data is directly converted according to the cross power spectrum results of 36 groups of 32 rows × 32 columns at the diagonal angles, so that a cross power spectrum result of 256 × 256 can be obtained.

[0071] In the calculation of the cross power spectrum, taking 256 channels as an example, 256 × 256 × N complex multiplications are required, where N is the number of effective frequency domain points after filtering. Considering the completion cycle of the complex multiplication, the algorithm time will have a serious lag. The cross power spectrum matrix is ​​decomposed and designed. By using the conjugate symmetry of the cross power spectrum matrix, more than 40% of FPGA logic resources can be saved, and the calculation speed is faster. The algorithm optimization framework is as follows: Figure 7 Figure 8 As shown, M represents the number of channels (number of sensors) and N represents the number of effective frequency domain points.

[0072] Due to time division multiplexing in the front stage of the cross power spectrum calculation, disorder has occurred during the cross power spectrum operation. The ideal cross power spectrum matrix result is 256 rows × 256 columns, which is transformed into 32 rows × 32 columns × 64 groups after grouping. Combined with the conjugate symmetry property of the target matrix, the actual matrix operation is 32 rows × 32 columns × 36 groups. The calculation results need to be rearranged twice, and the groups that have not been operated are restored according to the conjugate symmetry, and finally 256 rows × 256 columns are output. Figure 7 and Figure 8 .

[0073] The cross-power spectrum operation block of FPGA is divided into several operation units, each of which performs the operation of the cross-power spectrum in parallel. Each operation unit only calculates the half-angle data of the signal and directly obtains the other half of the cross-power spectrum data through the conjugate property.

[0074] Preferably, a cross-multiplexing structure is added during the plane scanning process. Since there is a dependency between the steering vector calculation and the plane scanning process, the buffer implemented by the BRAM logic resource is multiplexed to store the steering vector calculation results and the scanning matrix intermediate variables.

[0075] Specifically, the steering vectors generated by the imaginary sound source are grouped and cached in a multi-channel parallel manner, such as the s groups in the figure. Each group of data is sequentially subjected to steering vector calculations according to the timing, and the steering vector results obtained are returned to the original cache space for caching. After all steering vector results are calculated, the plane scan is performed. Similarly, the scanning operation is performed in a multi-channel parallel manner, and the scanning operation is performed according to the steering vector results output by multiple channels.

[0076] Furthermore, it also includes an image reorganization method, which reorganizes the multi-channel parallel output after plane scanning into a row×column format pixel image, and then obtains the sound pressure distribution map of the sound in the plane through sound pressure level conversion.

[0077] refer to Fig. 9 The input spatial coordinate data is cached after input, and the steering vector operation is performed in sequence according to the timing. After the steering vector operation is completed, it returns to the original cache space for caching. When all steering vector calculations are completed, the output is used to execute plane scanning.

[0078] Specifically, the first steering vector cache space caches the input spatial coordinate data, the spatial coordinate data is output to the steering vector operation space for steering vector operation, and the steering vector after the operation is output to the first steering vector cache space for cache. Based on the principle of spatial reuse, the spatial coordinate data A1 corresponds to the calculation result as the steering vector A2, and the steering vector A2 is returned to the first steering vector cache space to replace the original spatial coordinate data A1 storage space for BRAM secondary reuse.

[0079] The scanning result obtained by the multiplexing structure is different from the order of the final desired frame of scanning image. The data is converted into a two-dimensional image array through buffer rearrangement to obtain the sound pressure thermodynamic map of the plane scan.

[0080] Embodiment 2: An FPGA sound source localization device is disclosed. The device can also be considered as an FPGA chip structure for implementing a sound source localization method, including:

[0081] An input unit receives multi-channel audio data of a single frame of a microphone array, wherein the data depth N is determined by a preset FFT point number;

[0082] A reordering unit, grouping and cache reordering the input data according to the channel sequence number to form a plurality of groups of data streams, the input data including the first group of cache data streams to the nth group of cache data streams;

[0083] An FFT and filtering operation unit is used to receive the first group of cache sequences to the nth group of cache sequences in time sequence, and perform FFT operation and bandpass filtering operation, and output the filtered data into an n×n array to the cross-power spectrum operation unit;

[0084] A cross power spectrum calculation unit calculates the cross power spectrum result of each sub-array above the main diagonal in the n×n array;

[0085] A rearrangement operation unit, connected to the cross-power spectrum operation unit, is used to supplement the uncalculated part below the main diagonal by taking a conjugate method to obtain the cross-power spectrum of the entire array, and reorder the cross-spectrum power results according to the matrix row × column order;

[0086] A characteristic frequency point calculation unit calculates the characteristic frequency point of each frame of parallel input data;

[0087] A steering vector calculation unit, which calculates a steering vector according to the microphone coordinate data;

[0088] The plane scanning unit performs plane scanning operation according to the steering vector and the cross power spectrum results, and then obtains the sound pressure distribution diagram of the sound in the plane through sound pressure level conversion.

[0089] It is further explained that if Figure 3In the structure shown in , the FFT operation unit includes an FFT operation space and a filter operation space connected in sequence.

[0090] The cross power spectrum operation unit does not perform cross power spectrum operation on all the received filtered data. It only performs row-column conversion on the subarray below the diagonal of the n×n array, that is, converts the data stream output by the filter into a subarray structure. The rearrangement operation unit directly calculates it according to the conjugate property.

[0091] Among them, the FFT operation unit and the cross-power spectrum operation unit both belong to the spatial multiplexing structure, which saves the internal operation resources of the FPGA and processes 8 groups of data streams in sequence.

[0092] (t) and (f) described in the figure represent time domain data and frequency domain data respectively.

[0093] Furthermore, as a preferred solution, the FFT operation unit includes a butterfly operation unit, which is used to determine the scaling factor according to the current calculation result in each step of the butterfly operation, and adjust the final output according to the scaling factor, so as to achieve the purpose of adaptive quantization, reduce resource consumption and reduce the precision loss caused by quantization.

[0094] As a preferred solution, it also includes a BRAM multiplexing unit, which is used to group and cache the steering vectors generated by the imaginary sound source in a multi-channel parallel manner. Each group of data is subjected to steering vector operation in sequence according to a timing sequence. The steering vector results obtained are returned to the original cache space for caching. After all the steering vector results are calculated, they are output to execute plane scanning.

[0095] like Fig. 9 The steering vector data generated by s groups of imaginary sound sources shown in the figure are input in parallel into s BRAM multiplexing units, and the steering vector results are output to the plane scanning unit according to the spatial multiplexing method.

[0096] The plane scanning unit performs plane scanning according to the steering vector and the cross power spectrum results. The plane scanning operation unit also adopts a spatial multiplexing structure and performs scanning operations in a multi-channel parallel manner. The scanning operations are performed according to the steering vector results output by the multi-channels. For example, the scanning operations are performed in parallel in s groups.

[0097] It also includes an image reorganization unit for reorganizing the multi-channel parallel output after plane scanning into a row × column format pixel image, and then converting the sound pressure level to obtain the sound pressure distribution map in the plane, referring to Fig. 9 , 10 .

[0098] As a preferred solution, the input unit includes 256 channel input terminals, such as the parameters disclosed in this application.

[0099] In the calculation of the cross power spectrum, especially when processing a large amount of channel data, the amount of calculation is very large, which may cause the time lag of the algorithm. In order to optimize the calculation process and reduce resource consumption, the characteristics of the cross power spectrum matrix can be used to effectively decompose and design it.

[0100] In this way, by partitioning parallel processing and utilizing conjugate symmetry, the amount of calculation can be significantly reduced, thereby improving computing efficiency and reducing FPGA logic resources.

[0101] In addition, it should be noted that the shapes and names of the parts and components of the specific embodiments described in this specification may be different. Any equivalent or simple changes made based on the structure, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. The technicians in the technical field of the present invention can make various modifications or supplements to the specific embodiments described or replace them in a similar manner, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A SRP-PHAT sound source localization method based on FPGA, characterized in that: The following steps are involved: Receive multi-channel audio data of a single frame of microphone array, where the data depth N is determined by a preset FFT point number; The input data is grouped and cached according to the channel sequence number to form several groups of data streams. The first group of cached data streams to the nth group of cached data streams are sequentially subjected to FFT operations and bandpass filtering operations. Each group of cached data streams is used as a subarray for cross-power spectrum operation. The output filtered data is composed of subarrays to form an n×n array. Calculate the cross power spectrum of each subarray above the main diagonal of the n×n array, and supplement the uncalculated part below the main diagonal by taking the conjugate method to obtain the cross power spectrum of the entire array, and reorder the cross power spectrum results according to the matrix row × column order; The characteristic frequency points of the parallel input data are calculated, the steering vector is calculated according to the microphone coordinate data, a plane scanning operation is performed with the cross-power spectrum result, and then the sound pressure distribution diagram of the sound in the plane is obtained through sound pressure level conversion.

2. The SRP-PHAT sound source localization method based on FPGA as claimed in claim 1, characterized in that: The FFT operation method comprises: In each step of the butterfly operation, a scaling factor is determined based on the current calculation result, and the final output is adjusted according to the scaling factor.

3. The SRP-PHAT sound source localization method based on FPGA as claimed in claim 1, characterized in that: The method for calculating the steering vector according to the microphone coordinate data comprises: The steering vectors generated by the imaginary sound source are grouped and cached in a multi-channel parallel manner. Each group of data is subjected to steering vector calculation in sequence, and the steering vector results obtained are returned to the original cache space for caching. When all steering vector results are calculated, they are output and plane scanning is performed.

4. The FPGA-based SRP-PHAT sound source localization method according to claim 3, characterized in that: The method for plane scanning operation comprises: The scanning operation is performed in a multi-channel parallel manner, and the scanning operation is performed according to the multi-channel output guidance vector results.

5. The SRP-PHAT sound source localization method based on FPGA as claimed in claim 1, characterized in that: The invention also includes an image reorganization method, which reorganizes the multi-channel parallel outputs after plane scanning into a row×column format pixel image.

6. An FPGA sound source localization device, characterized in that: Includes the following structures: An input unit receives multi-channel audio data of a single frame of a microphone array, wherein the data depth N is determined by a preset FFT point number; A reordering unit, grouping and cache reordering the input data according to the channel sequence number to form a plurality of groups of data streams, the input data including the first group of cache data streams to the nth group of cache data streams; An FFT and filtering operation unit is used to receive the first group of cache sequences to the nth group of cache sequences in time sequence, and perform FFT operation and bandpass filtering operation, and output the filtered data into an n×n array to the cross-power spectrum operation unit; A cross power spectrum calculation unit calculates the cross power spectrum result of each sub-array above the main diagonal in the n×n array; A rearrangement operation unit, connected to the cross-power spectrum operation unit, is used to supplement the uncalculated part below the main diagonal by taking a conjugate method to obtain the cross-power spectrum of the entire array, and reorder the cross-spectrum power results according to the matrix row × column order; A characteristic frequency point calculation unit calculates the characteristic frequency point of each frame of parallel input data; A steering vector calculation unit, which calculates a steering vector according to the microphone coordinate data; The plane scanning unit performs plane scanning operation according to the steering vector and the cross power spectrum results, and then obtains the sound pressure distribution diagram of the sound in the plane through sound pressure level conversion.

7. The FPGA sound source localization device according to claim 6, characterized in that: The FFT operation unit includes a butterfly operation unit, which is used to determine a scaling factor according to a current calculation result in each step of the butterfly operation, and adjust a final output according to the scaling factor.

8. The FPGA sound source localization device according to claim 6, characterized in that: It also includes a BRAM multiplexing unit, which is used to group and cache the steering vectors generated by the imaginary sound source in a multi-channel parallel manner. Each group of data is subjected to steering vector operation in sequence according to the timing. The steering vector results obtained are returned to the original cache space for caching. When all steering vector results are calculated, they are output to execute plane scanning.

9. The FPGA sound source localization device according to claim 6, characterized in that: The plane scanning unit is also used to perform scanning operations in a multi-path parallel manner and perform scanning operations based on the guidance vector results of the multi-path outputs.

10. The FPGA sound source localization device according to claim 6, characterized in that: It also includes an image reorganization unit for reorganizing the multi-channel parallel outputs after plane scanning into a row×column format pixel image.

Images