Multi-sound source fusion positioning method, device, equipment, storage medium and product

By combining the sound source positioning algorithm with beam energy, subspace and channel correlation, accurate sound source positioning in the case of multi-sound source reverberation is achieved, solving the problem of low positioning accuracy in traditional technology and improving the overall system robustness.

CN118671700BActive Publication Date: 2024-12-06MIDEA GRP (SHANGHAI) CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410963868.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-12-06
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

Traditional sound source positioning technology is difficult to distinguish multiple sound sources under the reverberation of multiple sound sources, resulting in reduced positioning accuracy and inability to accurately locate the sound source position.

Method used

The multi-sound source fusion positioning method is adopted, combined with three sound source positioning algorithms based on beam energy, subspace and inter-channel correlations for fusion. Through peak detection and angular range judgment, accurate positioning of multiple sound sources is achieved.

Benefits of technology

In the case of multi-sound source reverberation, the accuracy and robustness of sound source positioning are improved, and the main sound source can be more accurately identified and positioned, overcoming the shortcomings of a single algorithm in complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118671700B_ABST
    Figure CN118671700B_ABST
Patent Text Reader

Abstract

The present application relates to the field of sound source localization technology, and provides a method, device, equipment, storage medium and product for fusion localization of multiple sound sources, the method comprising: using a first sound source localization algorithm to perform peak detection on sound data to obtain a first peak result; in the case of at least two peaks, determining the maximum energy sound source in at least two peaks as a preliminary main sound source; detecting whether there is a peak in the second peak result and the third peak result that falls within a first angle range to obtain a first detection result; the first angle range is determined according to the angular position of the preliminary main sound source; the second peak result is obtained by performing peak detection on sound data using a second sound source localization algorithm; the third peak result is obtained by performing peak detection on sound data using a third sound source localization algorithm; and determining the localization result of the main sound source according to the first detection result. The present application can achieve good localization effects and improve overall robustness in the case of multiple sound source reverberation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of sound source localization, and in particular to a method, device, equipment, storage medium and product for fusion localization of multiple sound sources. Background Art

[0002] Sound source localization technology refers to processing the received signal to obtain the location information of the natural sound source or the artificial sound source. In actual scenes, there may be multiple noise sources. In this case, traditional sound source localization technology will have problems in locating multiple sound sources. It cannot distinguish multiple sound sources, and the accuracy of sound source localization is reduced. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a multi-sound source fusion positioning method, which can achieve good positioning effect and improve overall robustness in the case of multi-sound source reverberation.

[0004] The present application also proposes a method, device, equipment, storage medium and program product for fusion positioning of multiple sound sources.

[0005] According to the first aspect of the present application, the fusion positioning method of multiple sound sources includes: using a first sound source positioning algorithm to perform peak detection on sound data to obtain a first peak result; when the first peak result includes at least two peaks, determining the maximum energy sound source in the at least two peaks as a preliminary main sound source; detecting whether there are peaks in the second peak result and the third peak result falling into a first angle range to obtain a first detection result; wherein the first angle range is determined according to the angular position of the preliminary main sound source; the second peak result is obtained by using a second sound source positioning algorithm to perform peak detection on sound data; the third peak result is obtained by using a third sound source positioning algorithm to perform peak detection on sound data; determining the positioning result of the main sound source according to the first detection result; wherein the first sound source positioning algorithm is a sound source positioning algorithm based on beam energy; the second sound source positioning algorithm is a sound source positioning algorithm based on subspace; and the third sound source positioning algorithm is a sound source positioning algorithm based on channel correlation.

[0006] According to the multi-sound source fusion positioning method of the embodiment of the present application, by fusing the three sound source localization algorithms, the sound source is roughly positioned using the first sound source localization algorithm, and then the rough positioning result is corrected according to the second sound source localization algorithm and the third sound source localization algorithm. Therefore, the advantages of the three sound source localization algorithms can be combined, and in the case of multi-sound source reverberation, a good positioning effect can be achieved, thereby improving the robustness of the overall system.

[0007] According to one embodiment of the present application, determining the positioning result of the main sound source based on the first detection result includes: if in the first detection result, there is at least one peak in the second peak result and the third peak result that falls within the first angle range, then determining the positioning result of the main sound source based on at least one peak that falls within the first angle range.

[0008] According to one embodiment of the present application, the positioning result of the main sound source is determined based on at least one peak falling within the first angle range, including: when only one peak in the second peak result and the third peak result falls within the first angle range, the angular position of the peak falling within the first angle range is used as the positioning result of the main sound source.

[0009] According to one embodiment of the present application, the positioning result of the main sound source is determined based on at least one peak falling within the first angle range, including: when there is a first peak falling within the first angle range in the second peak result, and there is a second peak falling within the first angle range in the third peak result, taking the average of the angular position of the first peak and the angular position of the second peak as the positioning result of the main sound source.

[0010] According to one embodiment of the present application, the positioning result of the main sound source is determined according to the first detection result, including: if in the first detection result, neither the second peak result nor the third peak result has a peak falling within the first angle range, then detecting whether there is a peak in the second peak result and the third peak result falling within the second angle range to obtain a second detection result; wherein the second angle range is determined based on the angular position of the preliminary main sound source; the second angle range is greater than the first angle range; and the positioning result of the main sound source is determined according to the second detection result.

[0011] According to one embodiment of the present application, the positioning result of the main sound source is determined according to the second detection result, including: if in the second detection result, neither the second peak result nor the third peak result has a peak falling within the second angle range, or the angle difference between the peak falling within the second angle range in the second peak result and the peak falling within the second angle range in the third peak result is greater than or equal to a first preset value, then the angular position of the preliminary main sound source is used as the positioning result of the main sound source.

[0012] According to one embodiment of the present application, the positioning result of the main sound source is determined according to the second detection result, including: if in the second detection result, both the second peak result and the third peak result have peaks falling within the second angle range, and the angle difference between the peak in the second peak result that falls within the second angle range and the peak in the third peak result that falls within the second angle range is less than a first preset value, then the average of the angular position of the peak in the second peak result that falls within the second angle range and the angular position of the peak in the third peak result that falls within the second angle range is used as the positioning result of the main sound source.

[0013] According to an embodiment of the present application, it also includes: when the first peak result is a single peak, determining the positioning result of the main sound source according to the second peak result and the third peak result.

[0014] According to one embodiment of the present application, the positioning result of the main sound source is determined according to the second peak result and the third peak result, including: determining the main peak of the second peak result and the main peak of the third peak result; when the phase difference angle value between the main peak of the second peak result and the main peak of the third peak result is less than a second preset value, determining the positioning result of the main sound source according to the angular position of the main peak of the second peak result and the angular position of the main peak of the third peak result.

[0015] According to one embodiment of the present application, the positioning result of the main sound source is determined according to the second peak result and the third peak result, including: determining the number of peaks of the second peak result, the main peak of the second peak result, the number of peaks of the third peak result and the main peak of the third peak result; when the phase difference angle value between the main peak of the second peak result and the main peak of the third peak result is greater than or equal to a second preset value, based on the number of peaks of the second peak result and the number of peaks of the third peak result, the angular position of the main peak of the second peak result or the angular position of the main peak of the third peak result is used as the positioning result of the main sound source.

[0016] According to an embodiment of the present application, the first sound source localization algorithm is better than the second sound source localization algorithm and the third sound source localization algorithm in determining the relative values ​​of energy peaks at each angle when there are two sound sources; the computing power required by the third sound source localization algorithm is greater than the computing power required by the first sound source localization algorithm, and less than the computing power required by the second sound source localization algorithm; the directivity of the third sound source localization algorithm is better than the directivity of the first sound source localization algorithm, and weaker than the directivity of the second sound source localization algorithm.

[0017] According to an embodiment of the present application, the first sound source localization algorithm is a phase conversion controllable beam response algorithm; the second sound source localization algorithm is a minimum variance distortion response algorithm; and the third sound source localization algorithm is a generalized cross-correlation-phase transformation weighted algorithm.

[0018] According to the second aspect of the present application, a fusion positioning device for multiple sound sources includes: a first peak result module, which is used to use a first sound source positioning algorithm to perform peak detection on sound data to obtain a first peak result; a preliminary main sound source module, which is used to determine the maximum energy sound source in at least two peaks as the preliminary main sound source when the first peak result includes at least two peaks; a first detection result module, which is used to detect whether there is a peak in the second peak result and the third peak result that falls into a first angle range to obtain a first detection result; wherein the first angle range is determined according to the angular position of the preliminary main sound source; the second peak result is obtained by using a second sound source positioning algorithm to perform peak detection on sound data; the third peak result is obtained by using a third sound source positioning algorithm to perform peak detection on sound data; a main sound source positioning result module, which is used to determine the positioning result of the main sound source according to the first detection result; wherein the first sound source positioning algorithm is a sound source positioning algorithm based on beam energy; the second sound source positioning algorithm is a sound source positioning algorithm based on subspace; and the third sound source positioning algorithm is a sound source positioning algorithm based on channel correlation.

[0019] According to an electronic device of an embodiment of the third aspect of the present application, the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for fusion positioning of multiple sound sources as described above is implemented.

[0020] According to the non-transitory computer-readable storage medium of the fourth aspect of the present application, a computer program is stored thereon, and when the computer program is executed by a processor, any of the above-mentioned multi-sound source fusion positioning methods is implemented.

[0021] The computer program product according to the fifth aspect of the present application includes a computer program, which, when executed by a processor, implements any of the above-mentioned methods for fusion positioning of multiple sound sources.

[0022] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0023] By fusing the three sound source localization algorithms, the first sound source localization algorithm is used to roughly locate the sound source, and then the rough localization result is corrected according to the second and third sound source localization algorithms. By combining the advantages of the three sound source localization algorithms, a good localization effect can be achieved in the case of multi-sound source reverberation, thereby improving the robustness of the overall system.

[0024] The algorithms that need to be fused include a sound source localization algorithm based on beam energy as the first sound source localization algorithm, a sound source localization algorithm based on subspace as the second sound source localization algorithm, and a sound source localization algorithm based on inter-channel correlation as the third sound source localization algorithm; these three algorithms perform sound source localization from three different perspectives, respectively, which can further improve the fusion effect;

[0025] Furthermore, the present application utilizes the advantage of the phase conversion controllable beam response algorithm that the relative value of energy is relatively accurate, and can accurately select the main sound source from multiple sound sources, thus overcoming the problem of inaccurate angle positioning accuracy of the phase conversion controllable beam response algorithm;

[0026] Furthermore, the present application utilizes the advantage of the minimum variance distortion response algorithm in terms of accurate angle positioning accuracy, and overcomes the problem of inaccurate peak energy of the minimum variance distortion response algorithm;

[0027] Furthermore, the present application utilizes the characteristic of accurate single sound source positioning of the generalized cross-correlation-phase transform weighted algorithm, and overcomes the characteristic of poor positioning accuracy of the generalized cross-correlation-phase transform weighted algorithm when the energies of two sound sources are similar.

[0028] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 It is one of the flow charts of the multi-sound source fusion positioning method provided in the embodiment of the present application.

[0031] Figure 2 It is a schematic diagram of the fusion of multiple sound source localization algorithms provided by an embodiment of the present invention.

[0032] Figure 3 This is the second flow chart of the multi-sound source fusion positioning method provided in the embodiment of the present application.

[0033] Figure 4 This is the third flow chart of the multi-sound source fusion positioning method provided in the embodiment of the present application.

[0034] Figure 5 This is the fourth flow chart of the multi-sound source fusion positioning method provided in the embodiment of the present application.

[0035] Figure 6 This is the fifth flow chart of the multi-sound source fusion positioning method provided in the embodiment of the present application.

[0036] Figure 7This is the sixth flow chart of the multi-sound source fusion positioning method provided in the embodiment of the present application.

[0037] Figure 8 This is the seventh flow chart of the multi-sound source fusion positioning method provided in the embodiment of the present application.

[0038] Fig.9A This is one of the schematic diagrams of energy distribution at different angles of the SRP-PHAT algorithm provided in the embodiment of the present application.

[0039] Fig. 9B This is one of the energy distribution diagrams at different angles of the MVDR algorithm provided in the embodiment of the present application.

[0040] Fig. 9C This is one of the schematic diagrams of the cross-correlation values ​​of the GCC-PHAT algorithm provided in the embodiments of the present application.

[0041] Fig. 10A This is the second schematic diagram of energy distribution at different angles of the SRP-PHAT algorithm provided in the embodiment of the present application.

[0042] Fig. 10B This is the second schematic diagram of energy distribution at different angles of the MVDR algorithm provided in the embodiment of the present application.

[0043] Fig. 10C This is the second schematic diagram of the cross-correlation value of the GCC-PHAT algorithm provided in the embodiment of the present application.

[0044] Fig.11 It is a schematic diagram of the overall process of the multi-sound source fusion positioning method provided in an embodiment of the present application.

[0045] Fig.12 It is a schematic diagram of the structure of a multi-sound source fusion positioning device provided in an embodiment of the present application.

[0046] Fig.13 It is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following is a further detailed description of the implementation of the present application in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but cannot be used to limit the scope of the present application.

[0048] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0049] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located, and with the authorization given by the owner of the corresponding device.

[0050] In related technologies, a single algorithm may have problems in locating multiple sound sources. For example, when the number of microphones in the array is small and the spacing is low, some high-robustness and low-directivity sound source localization algorithms cannot distinguish multiple sound sources, and the sound source localization accuracy is reduced; when the reverberation is severe, some cross-correlation-based sound source localization algorithms have reduced effects; when there are multiple sound sources, some super-resolution sound source localization algorithms cannot distinguish which direction of the sound source is the main intensity sound source.

[0051] Based on this, an embodiment of the present application provides a multi-sound source fusion positioning method, which can solve complex situations, such as the problem of inaccurate sound source positioning in the presence of multiple sound sources and reverberation, achieve good sound source positioning effect, and improve overall robustness.

[0052] See also Figure 1 , Figure 1 This is one of the flowcharts of the method for fusion positioning of multiple sound sources provided in the embodiment of the present application. In this embodiment, the method for fusion positioning of multiple sound sources may include steps 110 to 140, and each step is specifically as follows.

[0053] Step 110: Use a first sound source localization algorithm to perform peak detection on the sound data to obtain a first peak result.

[0054] Step 120: When the first peak result includes at least two peaks, determine a maximum energy sound source in the at least two peaks as a preliminary main sound source.

[0055] Step 130: Detect whether any peak in the second peak result and the third peak result falls within the first angle range to obtain a first detection result.

[0056] Among them, the first angle range is determined according to the angle position of the preliminary main sound source; the second peak result is obtained by performing peak detection on the sound data using the second sound source localization algorithm; and the third peak result is obtained by performing peak detection on the sound data using the third sound source localization algorithm.

[0057] Step 140: Determine the positioning result of the main sound source according to the first detection result.

[0058] In the fusion localization method of multiple sound sources in this embodiment, the localization result of the main sound source is obtained by fusing three sound source localization algorithms. Specifically, the sound source is roughly localized using the first sound source localization algorithm, and then the rough localization result is corrected using the second sound source localization algorithm and the third sound source localization algorithm.

[0059] The first sound source localization algorithm, the second sound source localization algorithm and the third sound source localization algorithm are different, and can combine the advantages of the three sound source localization algorithms. In the case of multi-sound source reverberation, a good positioning effect can also be achieved, thereby improving the robustness of the overall system.

[0060] Among them, the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

[0061] The beam energy-based sound source localization algorithm is a method to determine the location of the sound source through the energy distribution in the sound wave propagation path. This method is applicable to microphone arrays or sensor arrays, and the direction of the sound source can be inferred by using the difference in the intensity of the sound wave signals received by sensors at different positions in the array.

[0062] Optionally, the sound source localization algorithm based on beam energy may include a Steered Response Power - Phase Transform (SRP-PHAT) algorithm.

[0063] The subspace-based sound source localization algorithm is a method for determining the location of the sound source using the subspace information of the signal received by the sensor array. The method is based on the spatial characteristics of the signal in the sensor array and the subspace decomposition theory. Optionally, the subspace-based sound source localization algorithm may include a minimum variance distortion response (MVDR algorithm) or a multiple signal classification (MUSIC) algorithm.

[0064] Both the MUSIC algorithm and the MVDR algorithm can estimate the spatial spectrum of the received signal and use the subspace characteristics of the signal in the sensor array to determine the direction of the sound source. The MUSIC algorithm can achieve high-resolution angle estimation by analyzing the eigenvector of the signal, while the MVDR algorithm focuses more on minimizing interference in directions of non-interest.

[0065] The sound source localization algorithm based on inter-channel correlation is a method that uses the signal correlation between each channel (or sensor) in the sensor array to determine the location of the sound source. This method relies on the temporal and spatial correlation caused by the signal propagating in the sensor array.

[0066] Optionally, the sound source localization algorithm based on inter-channel correlation may include a generalized cross correlation-phase transform weighted (Generalized Cross Correlation - Phase Transform, GCC-PHAT) algorithm or a normalized least mean squares (Normalized Least Mean Squares, NLMS) algorithm.

[0067] The NLMS algorithm and the GCC-PHAT algorithm can determine the direction of the sound source by calculating the signal delay and amplitude difference in the sensor array. Similar to GCC-PHAT, the NLMS algorithm also relies on the signal correlation between different sensors to infer the location of the sound source, but NLMS uses a different mathematical and algorithmic framework, mainly by gradually updating the filter weights.

[0068] In some embodiments, the first sound source localization algorithm can be a sound source localization algorithm with high robustness and low directivity, but it is relatively easy to distinguish the energy differences of multiple sound sources; the second sound source localization algorithm can be an algorithm with better directivity, but it requires relatively large computing power; the third sound source localization algorithm can be an algorithm with medium computing power and medium directivity.

[0069] Optionally, the first sound source localization algorithm is better than the second sound source localization algorithm and the third sound source localization algorithm in determining the relative values ​​of energy peaks at each angle when there are two sound sources; the computing power required by the third sound source localization algorithm is greater than the computing power required by the first sound source localization algorithm, and less than the computing power required by the second sound source localization algorithm; the directivity of the third sound source localization algorithm is better than the directivity of the first sound source localization algorithm, and weaker than the directivity of the second sound source localization algorithm.

[0070] Furthermore, the first sound source localization algorithm may be an SRP-PHAT algorithm; the second sound source localization algorithm may be an MVDR algorithm; and the third sound source localization algorithm may be a GCC-PHAT algorithm.

[0071] The SRP-PHAT algorithm requires very little computing power, but has poor directivity and relatively large angle deviation, and the effect will be relatively poor under severe reverberation conditions. However, this algorithm can easily distinguish the energy differences of multiple sound sources. When there are two sound sources, the relative values ​​of the energy peaks at various angles are also relatively accurate, and the sound source with large energy will be significantly higher than the sound source with small energy.

[0072] The MVDR algorithm requires more computing power, but has much better directivity and smaller angle deviation. It works poorly in severe reverberation. When there are multiple sound sources, the relative value of the energy of the two sound sources is inaccurate, and a sound source with high energy may be considered to have low energy.

[0073] The GCC-PHAT algorithm requires moderate computing power, has good directivity, and has good effects under severe reverberation conditions. However, it has poor effects under low signal-to-noise ratio conditions, and when the energy of two sound sources is close, it has poor effects.

[0074] In this embodiment, three sound source localization methods are fused, and by utilizing the advantages of these three sound source localization methods, a method for localizing the main sound source with better comprehensive performance can be obtained.

[0075] See also Figure 2 , Figure 2 It is a schematic diagram of the fusion of multiple sound source localization algorithms provided by an embodiment of the present invention.

[0076] After obtaining the sound data, after inputting multiple channels of audio, it is processed by a bandpass filter to allow signals within a specific frequency range to pass through and remove the noise of the sound data. In the processing of each channel of audio, it is processed by the MVDR algorithm, SRP-PHAT algorithm and GCC-PHAT algorithm respectively.

[0077] Specifically, the energy at each angle can be obtained through the SRP-PHAT algorithm, so the SRP-PHAT is used to execute the peak detection algorithm to detect multiple peaks in the sound data to obtain the first peak result.

[0078] The energy at each angle can be obtained through the MVDR algorithm, so the MVDR is used to execute the peak detection algorithm to detect multiple peaks in the sound data to obtain the second peak result.

[0079] The GCC-PHAT algorithm can be used to obtain the cross-correlation value of each microphone combination. Therefore, the GCC-PHAT is used to execute the peak detection algorithm to detect multiple peak combinations in the sound data to obtain possible sound source points and obtain the third peak result.

[0080] The fusion judgment module can perform fusion judgment according to the first peak result, the second peak result and the third peak result to obtain the positioning result of the main sound source.

[0081] Optionally, three bandpass filters of the same specification can be set to be responsible for filtering different branches respectively, that is, Figure 2 In some other embodiments, a bandpass filter may be provided, and the sound data may be divided into three branches after being filtered.

[0082] In the fusion judgment module, the number of peaks in the first peak result is first determined. When the first peak result includes at least two peaks, the maximum energy sound source is selected from the at least two peaks as the preliminary main sound source. Then, the first angle range is determined according to the angle position of the preliminary main sound source, and it is detected whether there are peaks in the second peak result and the third peak result that fall into the first angle range.

[0083] Optionally, the first angle range is determined by extending the first angle distance in the positive and negative directions with the preliminary main sound source as the center. For example, when the angle position of the preliminary main sound source is 70 degrees and the first angle is 10 degrees, the first angle range is 60 degrees to 80 degrees; when the angle position of the preliminary main sound source is 40 degrees and the first angle is 5 degrees, the first angle range is 35 degrees to 45 degrees.

[0084] Finally, a first detection result is obtained according to whether any peak in the second peak result and the third peak result falls within the first angle range, and a positioning result of the main sound source is determined according to the first detection result.

[0085] The first detection result may include four situations:

[0086] 1. There is a peak in the second peak result that falls within the first angle range, and there is no peak in the third peak result that falls within the first angle range;

[0087] 2. There is no peak in the second peak result that falls within the first angle range, and there is a peak in the third peak result that falls within the first angle range;

[0088] 3. Both the second peak result and the third peak result have peaks falling within the first angle range;

[0089] 4. There is no peak in the second peak result and the third peak result that falls within the first angle range.

[0090] Based on any of the above embodiments, determining the positioning result of the main sound source according to the first detection result includes: if in the first detection result, there is at least one peak in the second peak result and the third peak result that falls within the first angle range, then determining the positioning result of the main sound source according to at least one peak that falls within the first angle range.

[0091] In this embodiment, for the above situations 1-3, if at least one peak in the second peak result and the third peak result falls within the first angle range, the positioning result of the main sound source can be determined based on the at least one peak falling within the first angle range.

[0092] Optionally, there are multiple ways to determine based on at least one peak falling within the first angle range, such as selecting the angular position of the closest peak as the positioning result of the main sound source, or using the angular position of the average value of multiple peaks that meet the requirements as the positioning result of the main sound source, or using the angular position obtained according to a certain weight ratio of multiple peaks that meet the requirements as the positioning result of the main sound source.

[0093] Based on any of the above embodiments, the positioning result of the main sound source is determined according to at least one peak falling within the first angle range, including: when only one peak in the second peak result and the third peak result falls within the first angle range, the angular position of a peak falling within the first angle range is used as the positioning result of the main sound source.

[0094] See also Figure 3 , Figure 3 This is the second flow chart of the method for fusion positioning of multiple sound sources provided in the embodiment of the present application. In this embodiment, the method for fusion positioning of multiple sound sources may include steps 310 to 340, and each step is specifically as follows.

[0095] Step 310: Use a first sound source localization algorithm to perform peak detection on the sound data to obtain a first peak result.

[0096] Step 320: When the first peak result includes at least two peaks, determine the maximum energy sound source in the at least two peaks as the preliminary main sound source.

[0097] Step 330: Detect whether any peak in the second peak result and the third peak result falls within the first angle range to obtain a first detection result.

[0098] Step 340: If in the first detection result, only one peak of the second peak result and the third peak result falls within the first angle range, the angular position of the peak falling within the first angle range is used as the positioning result of the main sound source.

[0099] Among them, the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

[0100] In this embodiment, a method for determining the positioning result of the main sound source when the first peak result includes a preliminary main sound source and only one peak among the second peak result and the third peak result falls within the first angle range is specifically introduced, that is, the angular position of a peak falling within the first angle range is used as the positioning result of the main sound source.

[0101] Based on any of the above embodiments, the positioning result of the main sound source is determined according to at least one peak falling within the first angle range, including: when there is a first peak falling within the first angle range in the second peak result, and there is a second peak falling within the first angle range in the third peak result, taking the average of the angular position of the first peak and the angular position of the second peak as the positioning result of the main sound source.

[0102] See also Figure 4 , Figure 4 This is the third flow chart of the method for fusion positioning of multiple sound sources provided in the embodiment of the present application. In this embodiment, the method for fusion positioning of multiple sound sources may include steps 410 to 440, and the details of each step are as follows.

[0103] Step 410: Use a first sound source localization algorithm to perform peak detection on the sound data to obtain a first peak result.

[0104] Step 420: When the first peak result includes at least two peaks, determine the maximum energy sound source in the at least two peaks as the preliminary main sound source.

[0105] Step 430: Detect whether any peak in the second peak result and the third peak result falls within the first angle range to obtain a first detection result.

[0106] Step 440: If in the first detection result, there is a first peak in the second peak result that falls within the first angle range, and there is a second peak in the third peak result that falls within the first angle range, the average of the angular position of the first peak and the angular position of the second peak is used as the positioning result of the main sound source.

[0107] Among them, the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

[0108] In this embodiment, a method for determining the positioning result of the main sound source is specifically introduced when the first peak result includes a preliminary main sound source, the second peak result has a first peak falling within the first angle range, and the third peak result has a second peak falling within the first angle range, that is, taking the average of the angular position of the first peak and the angular position of the second peak as the positioning result of the main sound source.

[0109] It should be noted that if there are multiple peaks in the second peak result that fall within the first angle range, the peak closest to the preliminary main sound source is selected as the second peak. Similarly, if there are multiple peaks in the third peak result that fall within the first angle range, the peak closest to the preliminary main sound source is selected as the third peak.

[0110] Based on any of the above embodiments, determining the positioning result of the main sound source according to the first detection result includes: if in the first detection result, neither the second peak result nor the third peak result has a peak falling within the first angle range, then detecting whether there is a peak in the second peak result and the third peak result falling within the second angle range to obtain a second detection result; wherein the second angle range is determined based on the angular position of the preliminary main sound source; the second angle range is greater than the first angle range; and determining the positioning result of the main sound source according to the second detection result.

[0111] In this embodiment, for the above situation 4, if neither the second peak result nor the third peak result has a peak falling within the first angle range, the search range of the peak in the second peak result and the third peak result is expanded.

[0112] A second angle range is determined according to the angular position of the preliminary main sound source, and the second angle range is greater than the first angle range.

[0113] Optionally, the second angle range is determined by extending the second angle distance in positive and negative directions with the preliminary main sound source as the center. The second angle is greater than the first angle.

[0114] Exemplarily, when the angular position of the preliminary main sound source is 70 degrees, the first angle is 10 degrees, and the second angle is 15 degrees, the first angle range is 60 degrees to 80 degrees, and the second angle range is 55 degrees to 85 degrees.

[0115] It is detected whether there is a peak in the second peak result and the third peak result that falls within the second angle range to obtain a second detection result; wherein the second detection result is used to determine the positioning result of the main sound source.

[0116] The second test result may include four situations:

[0117] 1. There is a peak in the second peak result that falls within the second angle range, and there is no peak in the third peak result that falls within the second angle range;

[0118] 2. There is no peak in the second peak result that falls within the second angle range, and there is a peak in the third peak result that falls within the second angle range;

[0119] 3. Both the second peak result and the third peak result have peaks falling within the second angle range;

[0120] 4. There is no peak in the second peak result and the third peak result that falls within the second angle range.

[0121] Based on any of the above embodiments, the positioning result of the main sound source is determined according to the second detection result, including: if in the second detection result, neither the second peak result nor the third peak result has a peak falling within the second angle range, or the angle difference between the peak in the second peak result that falls within the second angle range and the peak in the third peak result that falls within the second angle range is greater than or equal to a first preset value, then the angular position of the preliminary main sound source is used as the positioning result of the main sound source.

[0122] In this embodiment, if both the second peak result and the third peak result have peaks falling within the second angle range, that is, the above-mentioned situation 4, the angular position of the preliminary main sound source is used as the positioning result of the main sound source.

[0123] If only one of the second peak result and the third peak result falls within the second angle range, that is, the above cases 1 and 2, the angular position of the preliminary main sound source is used as the positioning result of the main sound source.

[0124] If both the second peak result and the third peak result have peaks falling within the second angle range, that is, the above situation 3, it is necessary to further determine whether the peaks of the second peak result and the third peak result meet the conditions.

[0125] Specifically, if the phase difference angle value between the peak value in the second peak result that falls within the second angle range and the peak value in the third peak result that falls within the second angle range is greater than or equal to the first preset value, it is considered that the condition is not met, and therefore the angular position of the preliminary main sound source is used as the positioning result of the main sound source.

[0126] If the phase difference angle between the peak in the second peak result that falls within the second angle range and the peak in the third peak result that falls within the second angle range is less than a first preset value, it is considered that the condition is met and the positioning result of the main sound source can be determined based on the peak in the second peak result that falls within the second angle range and the peak in the third peak result that falls within the second angle range.

[0127] See also Figure 5 , Figure 5 This is the fourth flow chart of the method for fusion positioning of multiple sound sources provided in the embodiment of the present application. In this embodiment, the method for fusion positioning of multiple sound sources may include steps 510 to 560, and each step is specifically as follows.

[0128] Step 510: Use a first sound source localization algorithm to perform peak detection on the sound data to obtain a first peak result.

[0129] Step 520: When the first peak result includes at least two peaks, determine the maximum energy sound source in the at least two peaks as the preliminary main sound source.

[0130] Step 530: Detect whether any peak in the second peak result and the third peak result falls within the first angle range to obtain a first detection result.

[0131] Step 540: If in the first detection result, neither the second peak result nor the third peak result has a peak falling within the first angle range, detect whether there is a peak in the second peak result or the third peak result falling within the second angle range to obtain a second detection result.

[0132] Step 550: If in the second detection result, neither the second peak result nor the third peak result has a peak value falling within the second angle range, the angular position of the preliminary main sound source is used as the positioning result of the main sound source.

[0133] Step 560: If in the second detection result, the phase difference angle between the peak value in the second peak result that falls within the second angle range and the peak value in the third peak result that falls within the second angle range is greater than or equal to a first preset value, then the angular position of the preliminary main sound source is used as the positioning result of the main sound source.

[0134] Among them, the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

[0135] In this embodiment, the first sound source localization algorithm is used to determine the preliminary main sound source in the case of multiple sound sources, and then the second peak result and the third peak result are detected to see whether there is a peak falling within the first angle range. The result is that neither the second peak result nor the third peak result has a peak falling within the first angle range, so the peak search range is expanded to further detect whether there is a peak falling within the second angle range in the second peak result and the third peak result. If neither the second peak result nor the third peak result has a peak falling within the second angle range, or the peak falling within the second angle range does not meet the conditions, the angular position of the preliminary main sound source is still used as the positioning result of the main sound source.

[0136] Based on any of the above embodiments, the positioning result of the main sound source is determined according to the second detection result, including: if in the second detection result, both the second peak result and the third peak result have peaks falling within the second angle range, and the angle difference between the peak in the second peak result that falls within the second angle range and the peak in the third peak result that falls within the second angle range is less than a first preset value, then the average of the angular position of the peak in the second peak result that falls within the second angle range and the angular position of the peak in the third peak result that falls within the second angle range is used as the positioning result of the main sound source.

[0137] See also Figure 6 , Figure 6 This is the fifth flow chart of the method for fusion positioning of multiple sound sources provided in the embodiment of the present application. In this embodiment, the method for fusion positioning of multiple sound sources may include steps 610 to 650, and each step is specifically as follows.

[0138] Step 610: Use a first sound source localization algorithm to perform peak detection on the sound data to obtain a first peak result.

[0139] Step 620: When the first peak result includes at least two peaks, determine the maximum energy sound source in the at least two peaks as the preliminary main sound source.

[0140] Step 630: Detect whether any peak in the second peak result and the third peak result falls within the first angle range to obtain a first detection result.

[0141] Step 640: If in the first detection result, neither the second peak result nor the third peak result has a peak falling within the first angle range, detect whether there is a peak in the second peak result or the third peak result falling within the second angle range to obtain a second detection result.

[0142] Step 650: If in the second detection result, both the second peak result and the third peak result have peaks falling within the second angle range, and the angle difference between the peak in the second peak result that falls within the second angle range and the peak in the third peak result that falls within the second angle range is less than a first preset value, then the average of the angular position of the peak in the second peak result that falls within the second angle range and the angular position of the peak in the third peak result that falls within the second angle range is taken as the positioning result of the main sound source.

[0143] Among them, the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

[0144] In this embodiment, neither the second peak result nor the third peak result has a peak that falls within the first angle range, so further detection is performed to determine whether the second peak result and the third peak result have a peak that falls within the second angle range. In the second detection result, both the second peak result and the third peak result have a peak that falls within the second angle range and meets the condition, so the average of the angle position of the peak that falls within the second angle range in the second peak result and the angle position of the peak that falls within the second angle range in the third peak result is used as the positioning result of the main sound source.

[0145] Based on any of the above embodiments, the method further includes: when the first peak result is a single peak, determining the positioning result of the main sound source according to the second peak result and the third peak result.

[0146] In this embodiment, if there are no multiple peaks in the result of peak detection of the sound data using the first sound source localization algorithm, it may be that there is only one peak in the sound data, or it may be that there are multiple peaks in the sound data but they cannot be detected using the first sound source localization algorithm. In this case, the localization result of the main sound source can be determined according to the second peak result and the third peak result.

[0147] Based on any of the above embodiments, the positioning result of the main sound source is determined according to the second peak result and the third peak result, including: determining the main peak of the second peak result and the main peak of the third peak result; when the phase difference angle value between the main peak of the second peak result and the main peak of the third peak result is less than a second preset value, determining the positioning result of the main sound source according to the angular position of the main peak of the second peak result and the angular position of the main peak of the third peak result.

[0148] See also Figure 7 , Figure 7 This is the sixth flow chart of the method for fusion positioning of multiple sound sources provided in the embodiment of the present application. In this embodiment, the method for fusion positioning of multiple sound sources may include steps 710 to 730, and each step is specifically as follows.

[0149] Step 710: Use a first sound source localization algorithm to perform peak detection on the sound data to obtain a first peak result.

[0150] Step 720: When the first peak result is a single peak, determine the main peak of the second peak result and the main peak of the third peak result.

[0151] Step 730: When the phase difference angle between the main peak of the second peak result and the main peak of the third peak result is less than a second preset value, determine the positioning result of the main sound source based on the average of the angular position of the main peak of the second peak result and the angular position of the main peak of the third peak result.

[0152] Among them, the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

[0153] In this embodiment, when the first peak result is a single peak, the main peak of the second peak result and the main peak of the third peak result are determined.

[0154] If there is only one peak in the second peak result, the only peak is taken as the main peak. If there are multiple peaks in the second peak result, the highest peak is taken as the main peak, or the first peak result and the third peak result are combined to determine the main peak of the second peak result. Similarly, if there is only one peak in the third peak result, the only peak is taken as the main peak. If there are multiple peaks in the third peak result, the highest peak is taken as the main peak, or the first peak result and the second peak result are combined to determine the main peak of the third peak result.

[0155] Detect whether the main peak of the second peak result and the main peak of the third peak result meet the conditions. Specifically, if the phase difference angle value between the main peak of the second peak result and the main peak of the third peak result is less than a second preset value, it is considered that the conditions are met, and the positioning result of the main sound source is determined based on the average value of the angular position of the main peak of the second peak result and the angular position of the main peak of the third peak result.

[0156] Based on any of the above embodiments, the positioning result of the main sound source is determined according to the second peak result and the third peak result, including: determining the number of peaks of the second peak result, the main peak of the second peak result, the number of peaks of the third peak result and the main peak of the third peak result; when the phase difference angle value between the main peak of the second peak result and the main peak of the third peak result is greater than or equal to a second preset value, based on the number of peaks of the second peak result and the number of peaks of the third peak result, the angular position of the main peak of the second peak result or the angular position of the main peak of the third peak result is used as the positioning result of the main sound source.

[0157] See also Figure 8 , Figure 8 This is the seventh flow chart of the method for fusion positioning of multiple sound sources provided in the embodiment of the present application. In this embodiment, the method for fusion positioning of multiple sound sources may include steps 810 to 840, and each step is specifically as follows.

[0158] Step 810: Use a first sound source localization algorithm to perform peak detection on the sound data to obtain a first peak result.

[0159] Step 820: Determine the number of peaks of the second peak result, the main peak of the second peak result, the number of peaks of the third peak result, and the main peak of the third peak result.

[0160] Step 830: When the phase difference angle value between the main peak of the second peak result and the main peak of the third peak result is greater than or equal to a second preset value, and at least one of the number of peaks of the second peak result and the number of peaks of the third peak result is greater than 2, the angular position of the main peak of the second peak result is used as the positioning result of the main sound source.

[0161] Step 840: When the phase difference angle between the main peak of the second peak result and the main peak of the third peak result is greater than or equal to the second preset value, and the number of peaks of the second peak result and the number of peaks of the third peak result are both 1, the angular position of the main peak of the third peak result is used as the positioning result of the main sound source.

[0162] Among them, the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

[0163] In this embodiment, if the phase difference angle between the main peak of the second peak result and the main peak of the third peak result is greater than or equal to the second preset value, it is considered that the condition is not met. The judgment is further made in combination with the peak number of the second peak result and the peak number of the third peak result.

[0164] If at least one of the number of peaks of the second peak result and the number of peaks of the third peak result is greater than 2, the angular position of the main peak of the second peak result is used as the positioning result of the main sound source; if the number of peaks of the second peak result and the number of peaks of the third peak result are both 1, the angular position of the main peak of the third peak result is used as the positioning result of the main sound source.

[0165] Two specific examples are given below to illustrate the effect of the multi-sound source fusion positioning method of the embodiment of the present application.

[0166] See also 9A to 9C , Fig.9A This is one of the schematic diagrams of energy distribution at different angles of the SRP-PHAT algorithm provided in the embodiment of the present application; Fig. 9B This is one of the schematic diagrams of energy distribution at different angles of the MVDR algorithm provided in the embodiment of the present application; Fig. 9C This is one of the schematic diagrams of the cross-correlation values ​​of the GCC-PHAT algorithm provided in the embodiments of the present application.

[0167] Suppose a scene includes two sound sources, one at an angle of 90 degrees and the other at 180 degrees. The 90-degree sound source is larger, so the actual positioning angle is 90 degrees.

[0168] like Fig.9AAs shown, the first peak result is obtained by using the SRP-PHAT algorithm. When the directivity of the SRP-PHAT algorithm is not enough, there is only one peak, and the positioning angle is 110 degrees.

[0169] like Fig. 9B As shown in the figure, the second peak result obtained by the MVDR algorithm can locate two peaks, 90 degrees and 180 degrees. However, the energy is wrong, and 180 degrees is considered to be the largest sound source.

[0170] like Fig. 9C As shown in the figure, the third peak result obtained by the GCC-PHAT algorithm can locate two peaks, 100 degrees and 180 degrees, although the energy at 180 degrees is very small. The GCC-PHAT algorithm can accurately locate the maximum angle of the sound source, but the angle is 100 degrees, which is not as accurate as the 90 degrees of MVDR.

[0171] When the multi-sound source fusion positioning method of the embodiment of the present invention is used, the main peak value of the MVDR algorithm is 90 degrees, and the main peak value of the GCC-PHAT algorithm is 100 degrees. The phase difference angle values ​​of the two algorithms are less than 15 degrees (the second preset value). Therefore, by taking the average of the two, a result of 95 degrees can be obtained in the end, which is better than the result of a single algorithm.

[0172] See also FIG. 10A to FIG. 10C , Fig. 10A This is the second schematic diagram of energy distribution at different angles of the SRP-PHAT algorithm provided in the embodiment of the present application; Fig. 10B This is the second schematic diagram of energy distribution at different angles of the MVDR algorithm provided in the embodiment of the present application; Fig. 10C This is the second schematic diagram of the cross-correlation value of the GCC-PHAT algorithm provided in the embodiment of the present application.

[0173] Suppose a scene includes two sound sources, one at an angle of 90 degrees and the other at 180 degrees. The 90-degree sound source is larger, so the actual positioning angle is 90 degrees.

[0174] like Fig. 10A As shown, the first peak result obtained by the SRP-PHAT algorithm, the SRP-PHAT algorithm detects two peaks, 80 degrees and 220 degrees, of which 80 degrees has the highest energy and is used as the preliminary main sound source.

[0175] like Fig. 10B As shown, the second peak result obtained by using the MVDR algorithm can locate two peaks, 90 degrees and 180 degrees.

[0176] like Fig. 10C As shown, the third peak result obtained by using the GCC-PHAT algorithm can locate a peak of 120 degrees.

[0177] However, when using the multi-sound source fusion positioning method of the embodiment of the present invention, the first angle range is 65 to 95 degrees. The 90-degree peak detected by the MVDR algorithm falls within the first angle range, and the peak detected by the GCC-PHAT algorithm does not fall within the first angle range. Therefore, the 90 degrees detected by the MVDR algorithm is used as the positioning result, which is consistent with the actual situation.

[0178] The main peak of the MVDR algorithm is 90 degrees, and the main peak of the GCC-PHAT algorithm is 100 degrees. The phase difference between the two algorithms is less than 15 degrees. Therefore, by taking the average of the two, we can finally get a result of 95 degrees, which is better than the result of a single algorithm.

[0179] The multi-sound source fusion positioning method provided in the embodiment of the present application can be applied to scenarios such as sound source positioning in nearby wake-up and sound source positioning of sweepers to obtain a main sound source positioning with better comprehensive performance.

[0180] See also Fig.11 , Fig.11 It is a schematic diagram of the overall process of the multi-sound source fusion positioning method provided in an embodiment of the present application.

[0181] In this embodiment, it is preferred to check whether the SRP-PHAT algorithm has multiple peaks:

[0182] 1) If there are multiple peaks, the maximum energy sound source located by the SRP-PHAT algorithm is used as the rough positioning. Then, from the multiple peaks detected by the MVDR algorithm and the GCC-PHAT algorithm, search for the one that is within a specified angle (for example, 15 degrees) from the rough positioning, and find the closest one within the specified range from the perspective of the MVDR algorithm and the perspective of the GCC-PHAT algorithm.

[0183] A: If both can be found, take the average of the two angles as the positioning result of the main sound source.

[0184] B: If only one of them is found, use that angle as the positioning result of the main sound source.

[0185] C: If neither is found, at a larger positive or negative specified angle (for example, set to 30 degrees), check whether there are sound source points with a difference of less than 5 degrees between the MVDR algorithm and the GCC-PHAT algorithm; if so, use the average of the angles of the MVDR algorithm and the GCC-PHAT algorithm; if not, use the coarse positioning angle as the positioning result of the main sound source.

[0186] 2) If the SRP-PHAT algorithm does not have multiple peaks, check whether the MVDR algorithm and the GCC-PHAT algorithm have multiple peaks.

[0187] A: If the MVDR algorithm has multiple sound sources, regardless of whether the GCC-PHAT algorithm has a single sound source or multiple sound sources, if the difference between the two sound source angles is lower than the specified value (15 degrees), the average of the two sound source angles will be used. If it exceeds the specified value, the positioning result of the MVDR algorithm will be used first.

[0188] B: If the MVDR algorithm uses a single sound source and the GCC-PHAT algorithm uses multiple sound sources, and the difference between the sound source angle of GCC-PHAT and the sound source angle of MVDR is lower than the specified value (15 degrees), the average of the two is used. If it exceeds the value, the positioning result of MVDR is used first.

[0189] C: If the MVDR algorithm uses a single sound and the GCC-PHAT algorithm uses a single sound source, and the difference between the sound source angle of the GCC-PHAT algorithm and the sound source angle of the MVDR algorithm is lower than the specified value (15 degrees), the average of the two is used. If it exceeds the value, the positioning result of the GCC-PHAT algorithm is used first.

[0190] In this embodiment, when the SRP-PHAT algorithm is a single peak, it is detected whether the second peak result of the MVDR algorithm and the third peak result of the GCC-PHAT algorithm have multiple peaks.

[0191] If at least one of the second peak result and the third peak result has multiple peaks, and the difference between the sound source angle of the GCC-PHAT algorithm and the sound source angle of the MVDR algorithm is lower than a specified value, the positioning result of the main sound source is determined based on the average value of the sound source angle of the GCC-PHAT algorithm and the sound source angle of the MVDR algorithm.

[0192] If at least one of the second peak result and the third peak result has multiple peaks, and the difference between the sound source angle of the GCC-PHAT algorithm and the sound source angle of the MVDR algorithm is greater than or equal to the specified value, the positioning result of the MVDR algorithm is used.

[0193] If neither the second peak result nor the third peak result has multiple peaks, and the difference between the sound source angle of the GCC-PHAT algorithm and the sound source angle of the MVDR algorithm is lower than the specified value, the positioning result of the main sound source is determined based on the average value of the sound source angle of the GCC-PHAT algorithm and the sound source angle of the MVDR algorithm.

[0194] If neither the second peak result nor the third peak result has multiple peaks, but the difference between the sound source angle of the GCC-PHAT algorithm and the sound source angle of the MVDR algorithm is greater than or equal to the specified value, the positioning result of the GCC-PHAT algorithm is used.

[0195] In summary, the multi-sound source fusion positioning method provided in the embodiment of the present application fuses multiple sound source positioning algorithms: it can take advantage of the relatively accurate relative energy value of SRP-PHAT to accurately select the main sound source from multiple sound sources, thus overcoming the problem of inaccurate angle positioning accuracy of SRP-PHAT. It takes advantage of the accurate angle positioning accuracy of MVDR to overcome the problem of inaccurate peak energy of MVDR. It takes advantage of the accurate single sound source positioning feature of GCC-PHAT to overcome the feature of poor positioning accuracy of GCC-PHAT when the energies of two sound sources are almost the same. Therefore, the embodiment of the present application can improve the overall robustness, especially to better locate the main sound source among multiple sound sources.

[0196] On the other hand, the present application also provides a multi-sound source fusion positioning device, see Fig.12 , Fig.12 12 is a schematic diagram of the structure of the multi-sound source fusion positioning device provided in an embodiment of the present application. In this embodiment, the multi-sound source fusion positioning device may include: a first peak result module 1210, a preliminary main sound source module 1220, a first detection result module 1230 and a main sound source positioning result module 1240.

[0197] A first peak result module 1210 is used to perform peak detection on the sound data using a first sound source localization algorithm to obtain a first peak result;

[0198] A preliminary main sound source module 1220, configured to determine a sound source with the largest energy among the at least two peaks as a preliminary main sound source when the first peak result includes at least two peaks;

[0199] The first detection result module 1230 is used to detect whether a peak in the second peak result and the third peak result falls within a first angle range to obtain a first detection result; wherein the first angle range is determined according to the angle position of the preliminary main sound source; the second peak result is obtained by performing peak detection on the sound data using a second sound source localization algorithm; and the third peak result is obtained by performing peak detection on the sound data using a third sound source localization algorithm;

[0200] The main sound source localization result module 1240 is used to determine the localization result of the main sound source according to the first detection result.

[0201] In this embodiment, the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

[0202] Based on any of the above embodiments, the main sound source localization result module 1240 can be specifically used to: if in the first detection result, at least one peak in the second peak result and the third peak result falls within the first angle range, then determine the localization result of the main sound source based on at least one peak falling within the first angle range.

[0203] Based on any of the above embodiments, the main sound source localization result module 1240 may include a first processing module, which can be specifically used to: when only one peak in the second peak result and the third peak result falls within the first angle range, use the angular position of a peak falling within the first angle range as the localization result of the main sound source.

[0204] Based on any of the above embodiments, the main sound source localization result module 1240 may include a second processing module, which can be specifically used to: when there is a first peak in the second peak result that falls within the first angle range, and there is a second peak in the third peak result that falls within the first angle range, take the average of the angular position of the first peak and the angular position of the second peak as the localization result of the main sound source.

[0205] Based on any of the above embodiments, the main sound source positioning result module 1240 can be specifically used for: if in the first detection result, neither the second peak result nor the third peak result has a peak falling within the first angle range, then detecting whether there is a peak in the second peak result and the third peak result falling within the second angle range to obtain a second detection result; wherein the second angle range is determined based on the angular position of the preliminary main sound source; the second angle range is greater than the first angle range; and determining the positioning result of the main sound source based on the second detection result.

[0206] Based on any of the above embodiments, the main sound source positioning result module 1240 may include a third processing module, which can be specifically used for: if in the second detection result, neither the second peak result nor the third peak result has a peak falling within the second angle range, or the angle difference between the peak in the second peak result that falls within the second angle range and the peak in the third peak result that falls within the second angle range is greater than or equal to a first preset value, then the angular position of the preliminary main sound source is used as the positioning result of the main sound source.

[0207] Based on any of the above embodiments, the main sound source localization result module 1240 may include a fourth processing module, which may be specifically used for: if in the second detection result, both the second peak result and the third peak result have peaks falling within the second angle range, and the angle difference between the peak in the second peak result that falls within the second angle range and the peak in the third peak result that falls within the second angle range is less than a first preset value, then the average of the angle position of the peak in the second peak result that falls within the second angle range and the angle position of the peak in the third peak result that falls within the second angle range is used as the localization result of the main sound source.

[0208] Based on any of the above embodiments, the main sound source localization result module 1240 may also be used to determine the localization result of the main sound source according to the second peak result and the third peak result when the first peak result is a single peak.

[0209] Based on any of the above embodiments, the main sound source localization result module 1240 may include a fifth processing module, which can be specifically used to: determine the main peak value of the second peak result and the main peak value of the third peak result; when the phase difference angle value between the main peak value of the second peak result and the main peak value of the third peak result is less than a second preset value, determine the localization result of the main sound source according to the angular position of the main peak of the second peak result and the angular position of the main peak of the third peak result.

[0210] Based on any of the above embodiments, the main sound source localization result module 1240 may include a sixth processing module, which can be specifically used to: determine the number of peaks of the second peak result, the main peak of the second peak result, the number of peaks of the third peak result, and the main peak of the third peak result; when the phase difference angle value between the main peak of the second peak result and the main peak of the third peak result is greater than or equal to a second preset value, based on the number of peaks of the second peak result and the number of peaks of the third peak result, the angular position of the main peak of the second peak result or the angular position of the main peak of the third peak result is used as the positioning result of the main sound source.

[0211] Based on any of the above embodiments, the first sound source localization algorithm is better than the second sound source localization algorithm and the third sound source localization algorithm in determining the relative values ​​of energy peaks at each angle when there are two sound sources; the computing power required for the third sound source localization algorithm is greater than the computing power required for the first sound source localization algorithm, and less than the computing power required for the second sound source localization algorithm; the directivity of the third sound source localization algorithm is better than the directivity of the first sound source localization algorithm, and weaker than the directivity of the second sound source localization algorithm.

[0212] Based on any of the above embodiments, the first sound source localization algorithm is a phase conversion controllable beam response algorithm; the second sound source localization algorithm is a minimum variance distortion response algorithm; and the third sound source localization algorithm is a generalized cross-correlation-phase transformation weighted algorithm.

[0213] In another aspect, the present application also provides an electronic device. Fig.13 , Fig.13 1 is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present application, and the electronic device may include: a processor 1310, a communication interface 1320, a memory 1330, and a communication bus 1340, wherein the processor 1310, the communication interface 1320, and the memory 1330 communicate with each other through the communication bus 1340. The processor 1310 may call the logic instructions in the memory 1330 to execute the following method:

[0214] A first sound source localization algorithm is used to perform peak detection on sound data to obtain a first peak result; when the first peak result includes at least two peaks, the maximum energy sound source in the at least two peaks is determined as a preliminary main sound source; it is detected whether there is a peak in the second peak result and the third peak result that falls within a first angle range to obtain a first detection result; wherein the first angle range is determined according to the angular position of the preliminary main sound source; the second peak result is obtained by performing peak detection on sound data using a second sound source localization algorithm; the third peak result is obtained by performing peak detection on sound data using a third sound source localization algorithm; the localization result of the main sound source is determined according to the first detection result; wherein the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

[0215] In addition, the logic instructions in the above-mentioned memory 1330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0216] In another aspect, an embodiment of the present application discloses a computer program product, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer can perform the methods provided by the above-mentioned method embodiments, for example, including:

[0217] A first sound source localization algorithm is used to perform peak detection on sound data to obtain a first peak result; when the first peak result includes at least two peaks, the maximum energy sound source in the at least two peaks is determined as a preliminary main sound source; it is detected whether there is a peak in the second peak result and the third peak result that falls within a first angle range to obtain a first detection result; wherein the first angle range is determined according to the angular position of the preliminary main sound source; the second peak result is obtained by performing peak detection on sound data using a second sound source localization algorithm; the third peak result is obtained by performing peak detection on sound data using a third sound source localization algorithm; the localization result of the main sound source is determined according to the first detection result; wherein the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

[0218] On the other hand, an embodiment of the present application further provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the methods provided in the above embodiments. For details, please refer to the above embodiments and will not be repeated here.

[0219] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0220] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0221] Finally, it should be noted that the above implementation modes are only used to illustrate the present application, rather than to limit the present application. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that various combinations, modifications or equivalent substitutions of the technical solutions of the present application do not depart from the spirit and scope of the technical solutions of the present application, and should be included in the scope of the claims of the present application.

Claims

1. A multi-sound source fusion positioning method, characterized in that: include: Using a first sound source localization algorithm to perform peak detection on the sound data to obtain a first peak result; In the case where the first peak result includes at least two peaks, determining a maximum energy sound source among the at least two peaks as a preliminary main sound source; Detecting whether a peak in the second peak result and the third peak result falls within a first angle range to obtain a first detection result; wherein the first angle range is determined according to the angle position of the preliminary main sound source; the second peak result is obtained by performing peak detection on the sound data using a second sound source localization algorithm; and the third peak result is obtained by performing peak detection on the sound data using a third sound source localization algorithm; Determine a positioning result of a main sound source according to the first detection result; Among them, the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

2. The multi-sound source fusion positioning method according to claim 1, characterized in that: The determining the positioning result of the main sound source according to the first detection result includes: If, in the first detection result, at least one peak value among the second peak value result and the third peak value result falls within the first angle range, the positioning result of the main sound source is determined according to the at least one peak value falling within the first angle range.

3. The multi-sound source fusion positioning method according to claim 2, characterized in that: The determining the positioning result of the main sound source according to at least one peak value falling within the first angle range includes: In the case that only one peak of the second peak result and the third peak result falls within the first angle range, the angular position of the peak falling within the first angle range is used as the positioning result of the main sound source.

4. The multi-sound source fusion positioning method according to claim 2, characterized in that: The determining the positioning result of the main sound source according to at least one peak value falling within the first angle range includes: When there is a first peak in the second peak result that falls within the first angle range, and there is a second peak in the third peak result that falls within the first angle range, the average value of the angular position of the first peak and the angular position of the second peak is taken as the positioning result of the main sound source.

5. The multi-sound source fusion positioning method according to claim 3, characterized in that: The determining the positioning result of the main sound source according to at least one peak value falling within the first angle range includes: When there is a first peak in the second peak result that falls within the first angle range, and there is a second peak in the third peak result that falls within the first angle range, the average value of the angular position of the first peak and the angular position of the second peak is taken as the positioning result of the main sound source.

6. The multi-sound source fusion positioning method according to claim 1, characterized in that: The determining the positioning result of the main sound source according to the first detection result includes: If in the first detection result, neither the second peak result nor the third peak result has a peak falling within the first angle range, detecting whether the second peak result and the third peak result have a peak falling within a second angle range to obtain a second detection result; wherein the second angle range is determined according to the angle position of the preliminary main sound source; and the second angle range is larger than the first angle range; The positioning result of the main sound source is determined according to the second detection result.

7. The multi-sound source fusion positioning method according to claim 6, characterized in that: The determining the positioning result of the main sound source according to the second detection result includes: If in the second detection result, neither the second peak result nor the third peak result has a peak falling within the second angle range, or the angle difference between the peak in the second peak result that falls within the second angle range and the peak in the third peak result that falls within the second angle range is greater than or equal to a first preset value, then the angular position of the preliminary main sound source is used as the positioning result of the main sound source.

8. The multi-sound source fusion positioning method according to claim 6, characterized in that: The determining the positioning result of the main sound source according to the second detection result includes: If in the second detection result, both the second peak result and the third peak result have peaks falling within the second angle range, and the angle difference between the peak in the second peak result that falls within the second angle range and the peak in the third peak result that falls within the second angle range is less than a first preset value, then the average of the angular position of the peak in the second peak result that falls within the second angle range and the angular position of the peak in the third peak result that falls within the second angle range is taken as the positioning result of the main sound source.

9. The multi-sound source fusion positioning method according to claim 7, characterized in that: The determining the positioning result of the main sound source according to the second detection result includes: If in the second detection result, both the second peak result and the third peak result have peaks falling within the second angle range, and the angle difference between the peak in the second peak result that falls within the second angle range and the peak in the third peak result that falls within the second angle range is less than a first preset value, then the average of the angular position of the peak in the second peak result that falls within the second angle range and the angular position of the peak in the third peak result that falls within the second angle range is taken as the positioning result of the main sound source.

10. The multi-sound source fusion positioning method according to claim 1, characterized in that: Also includes: In the case that the first peak result is a single peak, the positioning result of the main sound source is determined according to the second peak result and the third peak result.

11. The multi-sound source fusion positioning method according to claim 10, characterized in that: The determining the positioning result of the main sound source according to the second peak result and the third peak result includes: determining a main peak value of the second peak result and a main peak value of the third peak result; When the phase difference angle value between the main peak of the second peak result and the main peak of the third peak result is less than a second preset value, the positioning result of the main sound source is determined according to the angular position of the main peak of the second peak result and the angular position of the main peak of the third peak result.

12. The multi-sound source fusion positioning method according to claim 10, characterized in that: The determining the positioning result of the main sound source according to the second peak result and the third peak result includes: determining the number of peaks of the second peak result, the main peak of the second peak result, the number of peaks of the third peak result, and the main peak of the third peak result; When the phase difference angle value between the main peak of the second peak result and the main peak of the third peak result is greater than or equal to a second preset value, the angular position of the main peak of the second peak result or the angular position of the main peak of the third peak result is used as the positioning result of the main sound source according to the number of peaks of the second peak result and the number of peaks of the third peak result.

13. The method for fusion positioning of multiple sound sources according to claim 11, characterized in that: The determining the positioning result of the main sound source according to the second peak result and the third peak result includes: determining the number of peaks of the second peak result, the main peak of the second peak result, the number of peaks of the third peak result, and the main peak of the third peak result; When the phase difference angle value between the main peak of the second peak result and the main peak of the third peak result is greater than or equal to a second preset value, the angular position of the main peak of the second peak result or the angular position of the main peak of the third peak result is used as the positioning result of the main sound source according to the number of peaks of the second peak result and the number of peaks of the third peak result.

14. The multi-sound source fusion positioning method according to any one of claims 1 to 13, characterized in that: The first sound source localization algorithm is better than the second sound source localization algorithm and the third sound source localization algorithm in determining the relative values ​​of energy peaks at each angle when there are two sound sources; The computing power required by the third sound source localization algorithm is greater than the computing power required by the first sound source localization algorithm, and less than the computing power required by the second sound source localization algorithm; The directivity of the third sound source localization algorithm is better than the directivity of the first sound source localization algorithm, and weaker than the directivity of the second sound source localization algorithm.

15. The method for fusion positioning of multiple sound sources according to claim 14, characterized in that: The first sound source localization algorithm is a phase conversion controllable beam response algorithm; the second sound source localization algorithm is a minimum variance distortion response algorithm; and the third sound source localization algorithm is a generalized cross-correlation-phase transformation weighted algorithm.

16. A multi-sound source fusion positioning device, characterized in that: include: A first peak result module, used to perform peak detection on the sound data using a first sound source localization algorithm to obtain a first peak result; A preliminary main sound source module, configured to determine, when the first peak result includes at least two peaks, a sound source with the largest energy among the at least two peaks as a preliminary main sound source; A first detection result module is used to detect whether a peak in the second peak result and the third peak result falls within a first angle range to obtain a first detection result; wherein the first angle range is determined according to the angle position of the preliminary main sound source; the second peak result is obtained by performing peak detection on the sound data using a second sound source localization algorithm; and the third peak result is obtained by performing peak detection on the sound data using a third sound source localization algorithm; A main sound source positioning result module, used to determine the positioning result of the main sound source according to the first detection result; Among them, the first sound source localization algorithm is a sound source localization algorithm based on beam energy; the second sound source localization algorithm is a sound source localization algorithm based on subspace; and the third sound source localization algorithm is a sound source localization algorithm based on inter-channel correlation.

17. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the multi-sound source fusion positioning method as described in any one of claims 1 to 15 is implemented.

18. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for fusion positioning of multiple sound sources as described in any one of claims 1 to 15 is implemented.

19. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for fusion positioning of multiple sound sources as described in any one of claims 1 to 15 is implemented.