A method, system, device and medium for localizing multiple sound sources with DOA estimation association

Abstract

The present application discloses a multi-sound source localization method, system, device and medium associated with DOA estimation, the method comprising: constructing a voiceprint localization node network, and selecting a node in a single node network as a central node; wherein the node is provided with a signal processing unit and a wireless communication unit; after the node captures a voiceprint signal, a plurality of DOAs of the voiceprint signal are calculated by the signal processing unit, and a frequency histogram of the frequency components in the voiceprint signal belonging to each DOA is calculated by a preset method, and the frequency histogram corresponding to the DOA is used as a frequency feature set of the node, and is transmitted to the central node through the wireless communication unit after low-bit rate compression processing; the central node associates the received frequency feature set through a greedy algorithm based on feature similarity, aggregates the DOAs belonging to the same sound source to obtain a DOA set, and then calculates the positions of the plurality of sound sources according to the DOA set.

Description

Technical Field

[0001] The present application relates to the field of voiceprint positioning technology, and in particular to a method, system, device and medium for positioning multiple sound sources associated with DOA estimation. Background Art

[0002] In the sound source localization technology solution, the node can be composed of a single microphone or a microphone array (i.e., multiple microphones arranged in a predetermined geometric shape) and distributed at different locations in the monitoring area. At present, most methods use multiple voiceprints to collect information received by the node and associate this information with the location of the relevant sound source, such as energy, time, directional characteristics, and spatial likelihood functions. However, the voiceprint sensor network itself has limiting factors such as the limited data processing capabilities of the nodes and possible real-time processing requirements. The nodes cannot perform very complex computationally intensive operations, and the bandwidth usage restrictions also limit the amount of information that can be transmitted in the network. At the same time, because the nodes are running independently, the voiceprint signals required by different nodes cannot be synchronized.

[0003] The above problems are usually solved by the following methods in the existing technology: each node estimates and transmits the estimated value of the direction of arrival (DOA) of the effective sound source. The DOA estimate describes the propagation direction of the sound relative to the node at each moment in time. The location of the sound source can be estimated at the fusion center by fusing DOA measurements (also called sonar measurements), such as the MUSIC and ESPRIT algorithms. Although such methods require increasing the computational complexity of the node to perform DOA estimation, since only the DOA estimate needs to be transmitted, the bandwidth utilization is low. In addition, since the DOA estimation is performed separately at each node, the audio signals of different nodes do not need to be completely synchronized.

[0004] However, the commonly used methods mentioned above often overlook a basic problem: when there are multiple sound sources, multiple DOA estimates are received from each node (one DOA for each detected sound source), and we cannot know which sound source each DOA belongs to. This is the so-called data association problem. Figure 1 Schematic diagram of the problem of DOA mismatch in multi-source positioning solved by the present invention, as shown in Figure 1 As shown in Figure 1, the correct association of the DOAs of nodes corresponding to the same sound source must be found, otherwise the position estimation will result in a "false" sound source, that is, a position that does not correspond to the real sound source, which will lead to an erroneous positioning result. In addition, when multiple sound sources are in a moving state, some nodes may not be able to detect the voiceprint information of some sound sources, resulting in missed detection. The number of detected voiceprints, that is, the number of DOAs estimated by a single node, will vary depending on the node position and time. The problem of missed detection often occurs in practice.

[0005] Therefore, improving the DOA association and information transmission performance of multiple nodes for multi-source data is the focus that needs to be improved in current technical methods and solutions. Summary of the invention

[0006] The embodiments of the present application provide a multi-sound source localization method, system, device and medium associated with DOA estimation, which solve the technical problems that the existing sound source localization technology under multiple sound sources is prone to produce "false" sound sources, resulting in erroneous positioning results, and the limitation of bandwidth usage also restricts the amount of information that can be transmitted in the network.

[0007] In the first aspect, an embodiment of the present application provides a multi-sound source localization method associated with a low bit rate strategy and DOA estimation, comprising: constructing a voiceprint localization node network, and selecting a node in the node network as a central node; wherein the node is provided with a signal processing unit and a wireless communication unit; after the node captures the voiceprint signal, the signal processing unit calculates several DOAs of the voiceprint signal, and calculates a frequency histogram of the frequency components in the voiceprint signal belonging to each DOA through a preset method, and uses the frequency histogram corresponding to the DOA as a frequency feature set of the node, performs low bit rate compression processing, and then transmits it to the central node through the wireless communication unit; the central node associates the received frequency feature set through a greedy algorithm based on feature similarity, aggregates the DOAs belonging to the same sound source to obtain a DOA set, and then calculates the positions of several sound sources according to the DOA set.

[0008] Furthermore, a voiceprint positioning node network is constructed, specifically including: using a plurality of microphones as nodes, arranging them according to a preset geometric structure to form a voiceprint positioning node network, and using the first node in the upper left corner of the geometric structure as the central node.

[0009] Furthermore, the frequency histogram of the frequency components in the voiceprint signal belonging to each DOA is calculated by a preset method, specifically including: the voiceprint signal obtained at a time The expression of the voiceprint signal calculated according to the STFT transformation algorithm ; Where p is the node index, τ is the time index, is the frequency component index of the voiceprint signal; through the broadband DOA angle estimation method, the DOAs of several sound sources corresponding to the voiceprint signal are calculated according to the frequency components, and a DOA set is formed ; Where k is the number of DOAs; By using the narrowband DOA angle estimation method, the frequency components of a node at a time belonging to each DOA are calculated probability, and assign the frequency component to the DOA with the largest probability; the number of each frequency component assigned to a DOA within a preset time length is accumulated respectively to obtain a frequency histogram of the DOA, and all frequency histograms of a node are defined as a frequency feature set of the node; wherein the ordinate of the frequency histogram is the cumulative sum of the number of allocations, and the abscissa is the frequency component assigned to the DOA; the frequency feature set contains k frequency histograms corresponding to k DOAs.

[0010] Furthermore, after low bit rate compression processing, the data is transmitted to the central node through the wireless communication unit. The compression processing method is: The frequency histogram is compressed; wherein, and They are the frequency histogram before compression and the frequency histogram after compression, d is the preset compression factor, and B is the cumulative sum of the number of distributions of the frequency components.

[0011] Furthermore, after the DOAs belonging to the same sound source are aggregated to obtain a DOA set, the positions of several sound sources are calculated based on the DOA set, which specifically includes: the central node receives the frequency feature set of N nodes, that is, N*k frequency histograms, and traverses the N nodes in turn to calculate the frequency histogram in the frequency feature set of the current node Combine the SSIM values ​​of each frequency histogram in the next node and find the frequency histogram with the maximum SSIM value , and With frequency histogram where p and q are the node numbers, b is the bth frequency histogram on the p node, and v is the vth frequency histogram on the q node. After traversing the k frequency feature histograms on N nodes in turn according to the above process, the frequency histogram on any node is uniquely associated with a frequency histogram on the next node. The associated frequency histograms are combined to obtain k DOA sets corresponding to k sound sources. , and record the maximum SSIM value in the set; where i is the sequence number of the frequency histogram from the first node in the set; calculate the DOA set in sequence The SSIM value of each frequency histogram and all frequency histograms of other sets, that is, the calculation set A frequency histogram of The SSIM value of each frequency histogram in the remaining sets. If the calculated SSIM value is greater than the maximum SSIM value recorded, the frequency histogram corresponding to the SSIM value replaces the set The frequency histograms of the same node are exchanged to obtain the exchanged DOA set ; According to the k DOA sets after exchange All frequency histograms in the θ are used to determine the DOA of each sound source corresponding to different nodes, and the corresponding k sound source positions are obtained through the sound source localization algorithm.

[0012] Further, The calculation formula is

[0013]

[0014] in, and are the mean values ​​of the pixel brightness of the frequency histogram b and the frequency histogram v, respectively. and are the variance of the pixel brightness of the frequency histogram b and the frequency histogram v, respectively, reflecting the contrast of the image, is the covariance of the frequency histogram b and the frequency histogram v, indicating the correlation of the brightness changes of the two images. and is the preset stability factor.

[0015] On the second aspect, the embodiment of the present application also provides a multi-sound source positioning system associated with DOA estimation, including: a network construction module, which constructs a voiceprint positioning node network and selects a node in the node network as a central node; wherein the node is provided with a signal processing unit and a wireless communication unit; a DOA calculation module, after the node captures the voiceprint signal, calculates several DOAs of the voiceprint signal through the signal processing unit, and calculates the frequency histogram of the frequency components in the voiceprint signal belonging to each DOA through a preset method, and uses the frequency histogram corresponding to the DOA as the frequency feature set of the node, performs low bit rate compression processing, and then transmits it to the central node through the wireless communication unit; an aggregation processing module, in which the central node associates the received frequency feature set through a greedy algorithm based on feature similarity, aggregates the DOAs belonging to the same sound source to obtain a DOA set, and then calculates the positions of several sound sources according to the DOA set.

[0016] In a third aspect, an embodiment of the present application further provides a multi-sound source localization device associated with DOA estimation, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: implement any one step of the above-mentioned multi-sound source localization method associated with DOA estimation.

[0017] In a fourth aspect, an embodiment of the present application further provides a non-volatile computer storage medium for multi-sound source localization associated with DOA estimation, which stores computer-executable instructions, and the computer-executable instructions are configured to: be able to implement any one of the steps of the above-mentioned multi-sound source localization method associated with DOA estimation.

[0018] The embodiment of the present application provides a multi-sound source localization method and system associated with a low bit rate strategy and DOA estimation. The method and system can more effectively distinguish real sound sources from interference signals by associating data of multiple nodes through a greedy algorithm of feature similarity, thereby significantly reducing the generation of "false" sound sources and improving the accuracy of positioning; and by compressing and processing frequency feature sets at a low bit rate, the bandwidth required for data transmission is greatly reduced; the embodiment of the present application improves the positioning accuracy while reducing the bandwidth usage requirements, and solves the technical problems that the existing sound source localization technology is prone to generate "false" sound sources when there are multiple sound sources, resulting in erroneous positioning results, and the limitation of bandwidth usage also restricts the amount of information that can be transmitted in the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A schematic diagram of the problem of DOA mismatch in multi-source positioning solved by the present invention;

[0021] Figure 2 A method flow chart of a multi-sound source localization method and system associated with DOA estimation provided in an embodiment of the present application;

[0022] Figure 3 It is a schematic diagram of a simulation node network of the present invention;

[0023] Figure 4 The frequency histogram before and after compression of the low bit rate scheme of the present invention;

[0024] Figure 5 This is a comparison chart of correlation accuracy under different SNR (signal-to-noise ratio) conditions of the method of the present invention;

[0025] Figure 6 This is a comparison chart of the positioning accuracy of the method of the present invention under different SNR conditions;

[0026] Figure 7 This is a comparison chart of positioning accuracy under different compression factors and SNR conditions of the present invention;

[0027] Figure 8 A schematic diagram of the structure of a multi-sound source localization system associated with DOA estimation provided in an embodiment of the present application;

[0028] Fig. 9 A schematic diagram of the internal structure of a multi-sound source localization device associated with DOA estimation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0030] The embodiments of the present application provide a multi-sound source localization method, system, device and storage medium associated with DOA estimation, which are used to solve the technical problems that the existing sound source localization technology is prone to produce "false" sound sources when there are multiple sound sources, resulting in erroneous positioning results, and the limitation of bandwidth usage also restricts the amount of information that can be transmitted in the network.

[0031] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0032] Figure 2 A flowchart of a method for localizing multiple sound sources associated with DOA estimation provided in an embodiment of the present application. Figure 2 As shown, a method for localizing multiple sound sources associated with DOA estimation provided in an embodiment of the present application specifically includes the following steps:

[0033] Step 1: construct a voiceprint positioning node network, and select a node in the single node network as a central node; wherein the node is provided with a signal processing unit and a wireless communication unit.

[0034] Specifically, several microphones are used as nodes and arranged according to a preset geometric structure to form the voiceprint positioning node network, and the first node in the upper left corner of the geometric structure is used as the central node.

[0035] Step 2: After the node captures the voiceprint signal, it calculates several DOAs of the voiceprint signal through the signal processing unit, and calculates the frequency histogram of the frequency components in the voiceprint signal belonging to each DOA, and uses the frequency histogram corresponding to the DOA as the frequency feature set of the node, performs low bit rate compression processing, and then transmits it to the central node through the wireless communication unit.

[0036] Step 201: The voiceprint signal obtained at a moment The expression of the voiceprint signal is calculated according to the STFT (Short Time Fourier Transform) transformation algorithm: ; Wherein, p is the node index, τ is the time index, is the frequency component index of the voiceprint signal.

[0037] Step 202: Calculate the DOAs of the voiceprint signal corresponding to the sound sources according to the frequency components by using a broadband DOA angle estimation method, and form a DOA set. ; where k is the number of DOAs.

[0038] It should be noted that each node has k DOAs corresponding to k sound sources, and N nodes have a total of N*k DOAs.

[0039] Step 203: Calculate the frequency component of a node at a time that belongs to a DOA by using a narrowband DOA angle estimation method. and assigns the frequency component to the DOA with the largest probability.

[0040] Step 204: Accumulate the number of times each frequency component is allocated to a DOA within a preset time length to obtain a frequency histogram of the DOA, and define all frequency histograms of a node as a frequency feature set of the node; wherein the ordinate of the frequency histogram is the cumulative sum of the number of allocations, and the abscissa is the frequency component allocated to the DOA; the frequency feature set includes k frequency histograms corresponding to k DOAs.

[0041] Furthermore, after low bit rate compression processing, the data is transmitted to the central node through the wireless communication unit, and the compression processing method is:

[0042] By formula The frequency histogram is compressed; wherein, and They are the frequency histogram before compression and the frequency histogram after compression, d is the preset compression factor, and B is the cumulative sum of the number of distributions of the frequency components.

[0043] Specifically, the process of extracting the frequency histogram is performed within the frequency range within the frequency maximum value range. represents the number of frequency points that can be processed, that is, the frequency components. Therefore, the number of bits required to transmit the histogram is log2(B). An embodiment of the present application reduces the number of frequency points in the histogram by performing a compression process similar to that of the compression process, thereby reducing the transmission requirements. Figure 4 The frequency histograms before and after compression of the low bit rate scheme of the present invention are shown in FIG. 1 , wherein (a) is the frequency histogram without compression, and (b) is the frequency histogram after compression of the low bit rate scheme. Figure 4 As shown in Figure (a), the initial histogram horizontal axis has 16 categories, and its maximum cardinality is 10, so it is necessary Bits are transferred. By factor After compression, the number of types on the horizontal axis in Figure (b) is 8, and the maximum base is 20, so it is reduced to 35 bits for transmission. Therefore, the compressed frequency histogram can effectively reduce the data size of the transmitted information.

[0044] Step 3: The central node performs association processing on the received frequency feature set through a greedy algorithm based on feature similarity, aggregates the DOAs belonging to the same sound source to obtain a DOA set, and then calculates the positions of several sound sources.

[0045] Step 301: The central node receives the frequency feature set of N nodes, that is, N*k frequency histograms, and traverses the N nodes in turn to calculate the frequency histogram in the frequency feature set of the current node. Combine the SSIM values ​​of each frequency histogram in the next node and find the frequency histogram with the maximum SSIM value , and With the frequency histogram are associated; wherein p and q are the serial numbers of the nodes, b is the bth frequency histogram on the p node, and v is the vth frequency histogram on the q node.

[0046] Step 302: After traversing the k frequency feature histograms on N nodes in sequence according to the above process, the frequency histogram on any node is uniquely associated with a frequency histogram on the next node, and the associated frequency histograms are combined to obtain k DOA sets corresponding to k sound sources. , and record the maximum SSIM value in the set; where i is the sequence number of the frequency histogram from the first node in the set.

[0047] In one embodiment of the present application, based on the frequency histogram A1 in the frequency feature set of node A, the frequency histogram B4 with the largest SSIM value (highest image similarity) calculated with it in node B is found through calculation, and then A1 and B4 are combined into a set , and find the frequency histogram C2 with the largest SSIM value in the node C set according to B4, and add C2 to the set , get the set , and in the collection There are two SSIM values, namely the SSIM of A1 and B4 and the SSIM of B4 and C2. The SSIM of A1 and B4 is the largest. Then the record set The maximum SSIM value is the SSIM value of A1 and B4.

[0048] Step 303: Calculate the DOA set in sequence The SSIM value of each frequency histogram and all frequency histograms of other sets, that is, the calculation set A frequency histogram of If the calculated SSIM value is greater than the maximum SSIM value of the record, the frequency histogram corresponding to the SSIM value is replaced with the set The frequency histograms of the same node are exchanged to obtain the exchanged DOA set .

[0049] In one embodiment of the present application, the set The maximum SSIM value is the SSIM value of A3 and B1, but the calculated SSIM value of A3 and a set If the SSIM value of D1 in is greater than the SSIM values ​​of A3 and B1, then swap the positions of D2 and D1, and the resulting set is , .

[0050] Step 304: Based on the exchanged k DOA sets All frequency histograms in the θ are used to determine the DOA of each sound source corresponding to different nodes, and the corresponding k sound source positions are obtained through the sound source localization algorithm.

[0051] Specifically, in a DOA set In the set, there are N frequency histograms from N nodes, each frequency histogram corresponds to a DOA angle, that is, the set contains N DOAs of N nodes relative to a sound source; through the sound source localization algorithm, the sound source is located according to the N DOAs; the sound source localization algorithm includes the MUSIC algorithm, the ESPRIT algorithm, etc. In other words, the present application re-aggregates the k DOA angles received by the node according to the similarity of the frequency histogram, determines which DOAs correspond to which sound sources, and then locates the sound source according to the k DOAs, thereby solving the technical problem that the existing sound source localization technology is prone to produce "false" sound sources when there are multiple sound sources, resulting in erroneous positioning results.

[0052] Furthermore, in order to evaluate the proposed method, an embodiment of the present application simulates a square unit with a size of V = 8 meters, nodes M = 2, 4, 8, 12, 16 nodes, and sound sources S = 2, 3. Figure 3 The simulation node network diagram of the present invention is shown in the attached figure. Figure 3 As shown. Each node is a uniform circular microphone element with a radius of 5cm, and the sound source is a speech recording with a duration of 2 seconds, sampled at 44.1kHz. The SNR (signal-to-noise ratio) is measured based on the ratio of the power of each sound source signal located at the center of the square unit to the power of the noise signal.

[0053]

[0054] in, and are the signal power and noise power respectively.

[0055] In order to simulate different signal-to-noise ratio values, an embodiment of the present application adds white Gaussian noise that is uncorrelated with the sound source signal and the noise on other microphones to each microphone.

[0056] During the processing, an embodiment of the present application uses 2048 sampling points with an overlap rate of 50%. The FFT (Fast Fourier Transform) size is 2048. An embodiment of the present application sets the maximum value to 4kHz, and uses the length of the first 21 moments as the cumulative sum of the frequency histogram in the feature extraction link, which is equivalent to 0.5 seconds.

[0057] Figure 5 The following is a comparison chart of the correlation accuracy under different SNR conditions in the embodiment of the present invention. Figure 5 As shown in Figure (a) and Figure (b) are the signal-to-noise ratio-positioning accuracy results when the number of nodes S=2 and S=3, respectively. It can be seen that there are five groups of data, each corresponding to a different parameter combination M and S. From left to right, the parameter values ​​of these five groups are M=16, S=2; M=12, S=2; M=8, S=2; M=4, S=2; M=2, S=2. The color of each point represents a different SNR value (signal-to-noise ratio), including 0dB, 5dB, 10dB, 15dB and 20dB. The more nodes there are, the higher the association accuracy will be. At the same time, as the signal-to-noise ratio increases, its accuracy will also be improved to a certain extent.

[0058] Figure 6 The comparison diagram of positioning accuracy under different SNR conditions in the embodiment of the present invention is shown in the attached figure. Figure 6 As shown, sub-graph (a) represents the distribution result of sound source S=2, and sub-graph (b) represents the distribution result of sound source S=3. It can be seen from the figure that as the SNR increases, the RMSE (root mean square error) gradually decreases. This means that a higher SNR can improve the estimation accuracy. In addition, under the same SNR conditions, a larger number of nodes usually leads to a lower RMSE, indicating that more node values ​​help improve the performance of the positioning system.

[0059] Figure 7 The comparison diagram of positioning accuracy under different compression factors and SNRs in the embodiment of the present invention is shown in the attached figure. Figure 7 As shown, d=1 represents an uncompressed frequency histogram, and the positioning accuracy does not drop drastically with the increase of the compression factor d, which also proves the feasibility of the low bit rate solution of the present invention.

[0060] The above is an embodiment of the method proposed in this application. Based on the same inventive concept, an embodiment of this application also provides a multi-sound source localization system associated with DOA estimation.

[0061] Figure 8 A schematic diagram of a multi-sound source localization system associated with DOA estimation provided in an embodiment of the present application. Figure 8 As shown, the system includes:

[0062] A network construction module, which constructs a voiceprint positioning node network and selects a node in the node network as a central node; a DOA calculation module, after the node captures the voiceprint signal, calculates the sound source DOA estimation values ​​of several voiceprint signals through a preset method, and calculates the frequency histogram of the frequency components in the voiceprint signal belonging to each sound source through a preset method, takes the frequency histogram corresponding to the DOA as the frequency feature set of the node, performs low bit rate compression processing and then transmits it to the central node through a wireless communication unit; an aggregation processing module, the central node performs association processing on the received frequency feature set through a greedy algorithm based on feature similarity, aggregates the DOAs belonging to the same sound source to obtain a DOA set, and then calculates the positions of several sound sources based on the DOA set.

[0063] Based on the same inventive concept, an embodiment of the present application also provides a multi-sound source localization device associated with DOA estimation.

[0064] Fig. 9 A schematic diagram of the internal structure of a multi-sound source localization device associated with DOA estimation provided in an embodiment of the present application. Fig. 9 As shown, the device includes:

[0065] at least one processor 201;

[0066] and, a memory 202 communicatively connected to the at least one processor;

[0067] Among them, the memory 202 stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor 201 so that at least one processor 201 can: implement the steps of the method described in any one of steps 1-3 of the embodiment of the present application.

[0068] Some embodiments of the present application provide corresponding Figure 1 A non-volatile computer storage medium for multi-sound source localization associated with DOA estimation, storing computer executable instructions, wherein the computer executable instructions are configured to: be able to implement the steps of any one of steps 1-3 of the method described in the embodiment of the present application.

[0069] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the IoT device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0070] The system and medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the system and medium also have similar beneficial technical effects to the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the system and medium will not be repeated here.

[0071] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0072] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0073] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0075] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0076] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0077] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0078] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0079] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for localizing multiple sound sources associated with DOA estimation, characterized in that: The method comprises: Constructing a voiceprint positioning node network, and selecting a node in the node network as a central node; wherein the node is provided with a signal processing unit and a wireless communication unit; After the node captures the voiceprint signal, it calculates several DOAs of the voiceprint signal through the signal processing unit, and calculates the frequency histogram of the frequency components in the voiceprint signal belonging to each DOA through a preset method, and uses the frequency histogram corresponding to the DOA as the frequency feature set of the node, performs low bit rate compression processing, and then transmits it to the central node through the wireless communication unit; The central node performs association processing on the received frequency feature set through a greedy algorithm based on feature similarity, aggregates the DOAs belonging to the same sound source to obtain a DOA set, and then calculates the positions of several sound sources according to the DOA set; The frequency histogram of the frequency components in the voiceprint signal belonging to each DOA is calculated by a preset method, specifically including: The voiceprint signal obtained at a moment The expression of the voiceprint signal is calculated according to the STFT transformation algorithm: ; Wherein, p is the node index, τ is the time index, is the frequency component index of the voiceprint signal; By using a broadband DOA angle estimation method, the DOAs of the voiceprint signal corresponding to the sound sources are calculated according to the frequency components, and a DOA set is formed. ; Where k is the number of DOAs; By using the narrowband DOA angle estimation method, the frequency component of a node at a time belongs to each DOA. and assigning the frequency component to the DOA with the highest probability; The number of times each frequency component is allocated to a DOA within a preset time length is accumulated respectively to obtain a frequency histogram of the DOA, and all frequency histograms of a node are defined as a frequency feature set of the node; wherein the ordinate of the frequency histogram is the cumulative sum of the frequency allocation times, and the abscissa is the frequency component allocated to the DOA; the frequency feature set includes k frequency histograms corresponding to k DOAs.

2. The method for localizing multiple sound sources associated with DOA estimation according to claim 1, characterized in that: Construct a voiceprint positioning node network, including: A number of microphones are used as nodes and arranged according to a preset geometric structure to form the voiceprint positioning node network, and the first node in the upper left corner of the geometric structure is used as the central node.

3. The method for multiple sound source localization associated with DOA estimation according to claim 1, characterized in that: After low bit rate compression processing, it is transmitted to the central node through the wireless communication unit, and the compression processing method is: By formula The frequency histogram in the frequency feature set is compressed; wherein, and They are the frequency histogram before compression and the frequency histogram after compression, d is the preset compression factor, and B is the cumulative sum of the number of distributions of the frequency components.

4. The method for multiple sound source localization associated with DOA estimation according to claim 1, characterized in that: After the DOAs belonging to the same sound source are aggregated to obtain a DOA set, the positions of several sound sources are calculated based on the DOA set, including: The central node receives the frequency feature set of N nodes, that is, N*k frequency histograms, and traverses the N nodes in turn to calculate the frequency histogram in the frequency feature set of the current node. Combine the SSIM values ​​of each frequency histogram in the next node and find the frequency histogram with the maximum SSIM value , and With the frequency histogram associated; where p and q are the serial numbers of the nodes, b is the bth frequency histogram on the p node, and v is the vth frequency histogram on the q node; After traversing the k frequency feature histograms on N nodes in turn according to the above process, the frequency histogram on any node is uniquely associated with a frequency histogram on the next node. The interrelated frequency histograms are combined to obtain k DOA sets corresponding to k sound sources. , and record the maximum SSIM value in the set; where i is the serial number of the frequency histogram from the first node in the set; Calculate the DOA set sequentially The SSIM value of each frequency histogram and all frequency histograms of other sets, that is, the calculation set A frequency histogram of If the calculated SSIM value is greater than the maximum SSIM value of the record, the frequency histogram corresponding to the SSIM value is replaced with the set The frequency histograms of the same node are exchanged to obtain the exchanged DOA set ; According to the k DOA sets after exchange All frequency histograms in the θ are used to determine the DOA of each sound source corresponding to different nodes, and the corresponding k sound source positions are obtained through the sound source localization algorithm.

5. The method for localizing multiple sound sources associated with DOA estimation according to claim 4, characterized in that: Said The calculation formula is in, and are the mean values ​​of the pixel brightness of the frequency histogram b and the frequency histogram v, respectively. and are the variance of the pixel brightness of the frequency histogram b and the frequency histogram v, respectively, reflecting the contrast of the image, is the covariance of the frequency histogram b and the frequency histogram v, indicating the correlation of the brightness changes of the two images. and is the preset stability factor.

6. A multi-sound source localization system associated with DOA estimation, characterized in that: include: A network construction module is used to construct a voiceprint positioning node network and select a node in the node network as a central node; wherein the node is provided with a signal processing unit and a wireless communication unit; DOA calculation module, after the node captures the voiceprint signal, calculates several DOAs of the voiceprint signal through the signal processing unit, and calculates the frequency histogram of the frequency components in the voiceprint signal belonging to each DOA through a preset method, and uses the frequency histogram corresponding to the DOA as the frequency feature set of the node, performs low bit rate compression processing, and then transmits it to the central node through the wireless communication unit; Aggregation processing module, the central node performs association processing on the received frequency feature set through a greedy algorithm based on feature similarity, aggregates the DOAs belonging to the same sound source to obtain a DOA set, and then calculates the positions of several sound sources according to the DOA set; The DOA calculation module also includes: converting the voiceprint signal obtained at a moment The expression of the voiceprint signal is calculated according to the STFT transformation algorithm: ; Wherein, p is the node index, τ is the time index, is the frequency component index of the voiceprint signal; By using a broadband DOA angle estimation method, the DOAs of the voiceprint signal corresponding to the sound sources are calculated according to the frequency components, and a DOA set is formed. ; Where k is the number of DOAs; By using the narrowband DOA angle estimation method, the frequency component of a node at a time belongs to each DOA. and assigning the frequency component to the DOA with the highest probability; The number of times each frequency component is allocated to a DOA within a preset time length is accumulated respectively to obtain a frequency histogram of the DOA, and all frequency histograms of a node are defined as a frequency feature set of the node; wherein the ordinate of the frequency histogram is the cumulative sum of the frequency allocation times, and the abscissa is the frequency component allocated to the DOA; the frequency feature set includes k frequency histograms corresponding to k DOAs.

7. A multi-sound source localization device associated with DOA estimation, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; Wherein, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: implement the steps of the method described in any one of claims 1-5.

8. A non-volatile computer storage medium for localizing multiple sound sources associated with DOA estimation, storing computer executable instructions, characterized in that: The computer executable instructions are configured to implement the steps of any one of claims 1 to 5.

Images