Adaptive frequency band selection method and system based on sound source localization

Abstract

The invention discloses a self-adaptive frequency band selection method and system based on sound source localization, and the method comprises the steps: collecting sound in a target region, carrying out the frequency band division based on frequency spectrum information, carrying out the frequency band division through employing a dichotomy, and carrying out the frequency band division through employing a frequency band division method; the signal-to-noise ratio is used as a first weight factor, the spectrum flatness is used as a second weight factor, and the coherence is used as a third weight factor; and solving the target frequency through the reference frequency in each sub-band. The self-adaptive frequency selection mode is suitable for various scenes, can dynamically and adaptively adjust the positioning frequency band, adapts to different sound field environments, avoids manual frequency selection errors, and improves the reliability of sound source positioning. In the self-adaptive frequency selection mode, sub-bands are divided by using a dichotomy method, and the target frequency is solved in combination with a multi-modal influence factor, so that the efficiency of searching the target frequency and the positioning precision are improved.

Description

Technical Field

[0001] The present invention relates to the field of frequency band selection, and in particular to an adaptive frequency band selection method and system for sound source localization. Background Art

[0002] Sound source localization technology uses a microphone array to obtain sound signals in the area, collects sound signals in real time, aligns the phases and delays the sound signals picked up by each microphone on the array, and combines the microphone coordinate information to obtain the spatial distribution information of the sound source. After combining it with the optical image captured by the camera, the sound source can be visualized to intuitively obtain the position of the sound source in the sound field.

[0003] When using the device, the microphone array collects real-time sound source signals in the test sound field. Different sound sources may have different frequency components. Common audible sound frequency bands include low, mid, and high frequencies, as well as ultrasonic frequencies beyond the human ear's audible range. Therefore, in actual engineering operations, it is necessary to set the frequency band range to better accommodate the positioning and imaging of the sound source being measured.

[0004] Patent document CN106488358A discloses an “optimized sound field imaging positioning method and system”. This method uses a microphone array to collect multi-channel sound field samples synchronized with the video, obtains time-frequency distribution characteristics from the sound field samples, and then fuses the sound field state corresponding to the time-frequency at the analysis moment in real time with the optical camera image in the form of a cloud map. The specific positions of multiple sound sources and the real-time frequencies of each sound source can be obtained intuitively and quickly. However, this technology still requires manual setting of the frequency bands required for positioning, and manual setting of frequency parameters depends on user experience or technical level. The manually set frequency may introduce human errors, resulting in inaccurate measurement results. Therefore, an intelligent frequency band selection algorithm can automatically identify and select the frequency band that is most effective for sound source positioning, reducing errors caused by human operation. Therefore, there is a need for an adaptive frequency band selection method and system for sound source positioning. Summary of the Invention

[0005] The embodiments of the present application provide an adaptive frequency band selection method and system for sound source localization, which intelligently and automatically identifies and selects the frequency band that is most effective for sound source localization, reduces errors caused by human intervention, and can dynamically adjust the frequency band according to the real-time environment to better adapt to the usage requirements of changing scenarios.

[0006] The present invention provides an adaptive frequency band selection method for sound source localization, comprising the following steps:

[0007] Collect the sound in the target area, obtain the corresponding time domain signal, perform spectrum analysis on the collected time domain signal, and obtain the spectrum information of each microphone time domain signal in real time, including frequency components or amplitude information;

[0008] Dividing the spectrum into frequency bands based on the spectrum information, the frequency bands including an audible sound band, an ultrasonic band, and a full frequency band; dividing the frequency bands using a dichotomy method, using a signal-to-noise ratio as a first weighting factor, a spectrum flatness as a second weighting factor, and a coherence as a third weighting factor;

[0009] The target frequency is obtained by solving the reference frequency in each sub-band;

[0010] The target frequency is compared with the set threshold to determine whether it is the required retained adaptive frequency band. If the target frequency is within the set threshold, the sub-band is retained; if it exceeds the threshold, the band is removed; then determine whether the retained sub-bands are adjacent frequency bands. If they are adjacent, they are merged into one frequency band and no longer undergo binary division. If they are not adjacent, binary division is used to further divide and determine the non-adjacent sub-bands;

[0011] When the target frequency is obtained, the frequency threshold in each sub-band is set, and the target frequency is compared with the threshold frequency. When the target frequency is within the set frequency threshold range, the sub-band is retained; otherwise, the sub-band is determined to be an invalid band and is eliminated (merged into right 1, and cited in the following claims).

[0012] Furthermore, specifically by obtaining the sub-band after binary division, obtaining the power corresponding to the peak frequency of the sub-band and marking it as fs, marking the power corresponding to the background noise frequency of the peak frequency as Pn, and defining its signal-to-noise ratio as SNR:

[0013]

[0014] The SNR is marked as the influence factor for obtaining the positioning frequency band, and the influence factor weight is marked as w1.

[0015] Furthermore, according to the actual sound signals collected by the acoustic array probe, when an abnormal sound source exists, it has certain distinguishing characteristics relative to the background noise, and has a certain degree of unevenness. Here, the unevenness is used as an influencing factor affecting the adaptive selection of the frequency band, marked as sf:

[0016]

[0017] Among them, the sub-band spectrum after binary division is marked as X[k], where k = 1, 2, ..., N, N represents the kth frequency component in the frequency band. When the spectrum is completely flat, sf is equal to 1. The smaller the sf value, the less flat the spectrum is. At this time, the impact factor weight is marked as w2.

[0018] Furthermore, coherence is used to evaluate the spatial consistency of the sound array. A pair of microphones is randomly selected from the microphone array that receives the sound source signal. The coherence is used to measure the correlation between the two groups of signals, which is defined as r 2 xy (k), calculated as follows:

[0019]

[0020] Among them, P xx (k), P yy (k) are the power spectral densities of a pair of microphones, P xy (k) is the cross-power spectral density between a pair of microphones.

[0021] Furthermore, in order to reduce the error in calculating the coherence coefficient between any two groups of signals, N pairs of microphones on the array probe are selected, and the average value of the N pairs of microphones is used as the influencing factor affecting the positioning accuracy, which is marked as The calculation method is:

[0022]

[0023] And mark the weight of the impact factor as w3.

[0024] Furthermore, during the detection process, the frequency band interval is selected based on the main frequency of the sound radiated by the detected object, and then adaptive frequency band selection is performed in the selected frequency band interval to reduce the computational complexity of binary decomposition to find the optimal frequency band.

[0025] Furthermore, the target frequency is calculated as follows:

[0026]

[0027] The reference frequency is the mean of the peak frequencies within the sub-band, marked as The target frequency is marked as f target .

[0028] Furthermore, to determine whether the retained adjacent frequency bands need to be merged, the calculation formula for defining the difference between adjacent sub-bands is:

[0029]

[0030] where f target_a and ftarget_b The target frequencies of the adjacent frequency bands are retained respectively, and the difference threshold is set at the same time. If the difference RD of the adjacent sub-bands is within the set threshold range, in order to facilitate the management of the adaptive frequency bands, the adjacent frequency bands are merged into one frequency band for sound source localization and no further division is performed; if the difference RD of the adjacent sub-bands is not within the set threshold range, they are treated as two separate positioning frequency bands and further divided using the binary method. The number of cyclic judgment and division using the binary method does not exceed 10 times.

[0031] An adaptive frequency band selection system for sound source localization, comprising:

[0032] Microphone array module: used to collect sounds in the target area, obtain the corresponding time domain signals, perform spectrum analysis on the collected time domain signals, and obtain the spectrum information of each microphone's time domain signal in real time, including frequency components or amplitude information;

[0033] A signal processing module is configured to divide the spectrum into frequency bands based on the spectrum information, wherein the frequency bands include an audible sound band, an ultrasonic band, and a full frequency band; the frequency bands are divided using a dichotomy method, with the signal-to-noise ratio as a first weighting factor, the spectrum flatness as a second weighting factor, and the coherence as a third weighting factor;

[0034] Frequency band screening module: obtains the target frequency by solving the reference frequency in each sub-band;

[0035] Set the frequency threshold within each sub-band, compare the target frequency with the threshold frequency, and if the target frequency is within the set frequency threshold range, retain the sub-band; otherwise, determine that the sub-band is invalid and remove it;

[0036] The binary division module is used to compare the target frequency with the set threshold to determine whether it is the required retained adaptive frequency band. If the target frequency is within the set threshold, the sub-frequency band is retained; if it exceeds the threshold, the frequency band is eliminated; and then determine whether the retained sub-frequency bands are adjacent frequency bands. If they are adjacent, they are merged into one frequency band and no longer undergo binary division. If they are not adjacent, binary division is used to further divide and determine the non-adjacent sub-frequency bands;

[0037] When the target frequency is obtained, the frequency threshold in each sub-band is set, and the target frequency is compared with the threshold frequency. If the target frequency is within the set frequency threshold range, the sub-band is retained; otherwise, the sub-band is determined to be an invalid band and is removed.

[0038] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0039] The adaptive frequency selection mode of the present invention is applicable to a variety of scenarios and can dynamically and adaptively adjust the positioning frequency band to adapt to different sound field environments, avoid human-induced frequency selection errors, and improve the reliability of sound source localization. In this adaptive frequency selection mode, the sub-bands are divided using a binary method and the target frequency is solved by combining multimodal influence factors, which improves the efficiency of finding the target frequency and the positioning accuracy. At the same time, the retained adjacent target frequency bands are merged into a single frequency band, reducing the frequency of further division and enhancing the robustness of obtaining the adaptive target frequency band. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flowchart of an adaptive frequency selection mode based on an adaptive frequency band selection method for sound source localization provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0042] like Figure 1 As shown, this is a flow chart of an adaptive frequency band selection method for sound source localization provided by an embodiment of the present application. Step S21: operate the microphone array to collect sound in the target area and obtain its corresponding time domain signal;

[0043] Step S22: performing spectrum analysis on the collected time domain signal to obtain spectrum information of each microphone time domain signal in real time, including frequency components or amplitude information;

[0044] Step S23: dividing the spectrum into frequency bands based on the spectrum information, including audible sound band, ultrasonic band and full frequency band;

[0045] The frequency range below 20 kHz is defined as the audible frequency band, primarily encompassing frequencies audible to the human ear. Another frequency range, defined as the ultrasonic frequency band, is defined as frequencies above 20 kHz. Furthermore, there is a range covering the entire frequency range. This allows the target frequency to be determined within the frequency bands of the relationship, accelerating the acquisition process.

[0046] Step S24: selecting a frequency band (audible sound band, ultrasonic band, or full frequency band) and dividing the frequency band using a binary method;

[0047] In the above step S24, the frequency band is divided based on the binary method to find the target frequency band. The binary method is used to continuously narrow the search range, which can quickly locate the target frequency band and improve the search efficiency.

[0048] Step S25: Based on the frequency bands divided by the dichotomy method, the signal-to-noise ratio, spectrum flatness, and coherence are marked as weight factors affecting their positioning accuracy, and the weights of the influencing factors are marked as w1, w2, and w3 respectively;

[0049] In the above step S25, specifically by obtaining the sub-band after the binary division, obtaining the power corresponding to the peak frequency of the sub-band and marking it as fs, marking the power corresponding to the background noise frequency of the non-peak frequency as Pn, and defining its signal-to-noise ratio as SNR:

[0050]

[0051] The SNR is marked as the frequency band of positioning accuracy to obtain the impact factor, and the impact factor weight is marked as w1.

[0052] Furthermore, according to the actual sound signals collected by the acoustic array probe, when an abnormal sound source exists, it has certain distinguishing characteristics relative to the background noise, and has a certain degree of unevenness. Here, the unevenness is used as an influencing factor affecting the adaptive selection of the frequency band, marked as sf:

[0053]

[0054] The sub-band spectrum after binary division is labeled X[k], where k = 1, 2, ..., N, where N represents the kth frequency component within the band. When the spectrum is completely flat, sf equals 1. The smaller the sf value, the less flat the spectrum. In this case, the impact factor weight is labeled w2.

[0055] Furthermore, in the process of sound source localization, in general, for an ideal target sound source, its signal shows a certain regularity in the distribution of the microphone array, while the interference source or background noise generally does not have this characteristic, that is, whether the sound source signal is consistent at each point in the microphone array, which is used to measure the similarity or synchronization between the signals received by different microphones of the target sound source. Therefore, consider using coherence to evaluate the spatial consistency of the sound array. A pair of microphones is randomly selected from the microphone array that receives the sound source signal, and the coherence is used to measure the correlation between the two groups of signals, which is defined as r 2 xy (k), calculated as follows:

[0056]

[0057] Among them, P xx (k), P yy (k) are the power spectral densities of a pair of microphones, P xy (k) is the cross-power spectral density between a pair of microphones.

[0058] In order to reduce the error in the calculated coherence coefficient between any two groups of signals, N pairs of microphones on the array probe are randomly selected, and the average value of the N pairs of microphones is used as the influencing factor affecting the positioning accuracy, which is marked as The calculation method is:

[0059]

[0060] And mark the weight of the impact factor as w3.

[0061] Step S26: Obtain the target frequency by solving the reference frequency in each sub-band;

[0062] In step S26, the reference frequency may be the mean of the peak frequencies within the sub-band, which is marked as The target frequency is marked as f target , which is calculated as follows:

[0063]

[0064] Step S27: Set the frequency threshold f in each sub-band y , the target frequency f target With the threshold frequency f y When the target frequency is within the set frequency threshold f y If the sub-band is within the range, the sub-band is retained; otherwise, the sub-band is determined to be an invalid band and is removed.

[0065] Step S28: When using the binary division method to divide the sub-bands, it is possible that the frequency components of the sound sources in two adjacent frequency bands are similar. In this case, in order to facilitate bandwidth management, it is necessary to determine whether the retained sub-bands are adjacent frequency bands and whether the frequency components are consistent. If they are adjacent frequency bands, the target frequencies of the adjacent sub-bands are determined to be close. If so, they are merged into one frequency band. If not, the non-adjacent sub-bands are further divided using the binary division method.

[0066] In step S28, it is determined whether the retained adjacent frequency bands need to be merged. To this end, the difference between adjacent sub-bands is defined as RD:

[0067]

[0068] where f target_a and f target_bThe target frequencies of the adjacent frequency bands are retained respectively, and a difference threshold is set at the same time. If the difference RD of the adjacent sub-bands is within the set threshold range, in order to facilitate the management of the adaptive frequency band, the adjacent frequency bands are merged into one frequency band for sound source localization and no further division is performed; if the difference RD of the adjacent sub-bands is not within the set threshold range, they are respectively treated as two separate positioning frequency bands and further divided by binary division (return to step S25). Preferably, the number of cyclic judgment and division by binary division does not exceed 10 times.

[0069] In addition, the weights of influencing factors can be optimized through machine learning or artificial intelligence algorithms to reduce positioning errors and improve positioning accuracy.

[0070] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0072] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0074] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0075] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for adaptive frequency band selection for sound source localization, characterized in that: The following steps are involved: Collect the sound in the target area, obtain the corresponding time domain signal, perform spectrum analysis on the collected time domain signal, and obtain the spectrum information of each microphone time domain signal in real time, including frequency components or amplitude information; Dividing the spectrum into frequency bands based on the spectrum information, the frequency bands including an audible sound band, an ultrasonic band, and a full frequency band; dividing the frequency bands using a dichotomy method, using a signal-to-noise ratio as a first weighting factor, a spectrum flatness as a second weighting factor, and a coherence as a third weighting factor; The target frequency is obtained by solving the reference frequency in each sub-band; The target frequency is compared with the set threshold to determine whether it is the required retained adaptive frequency band. If the target frequency is within the set threshold, the sub-band is retained; if it exceeds the threshold, the band is removed; then determine whether the retained sub-bands are adjacent frequency bands. If they are adjacent, they are merged into one frequency band and no longer undergo binary division. If they are not adjacent, binary division is used to further divide and determine the non-adjacent sub-bands; When the target frequency is obtained, the frequency threshold in each sub-band is set, and the target frequency is compared with the threshold frequency. If the target frequency is within the set frequency threshold range, the sub-band is retained; otherwise, the sub-band is determined to be an invalid band and is removed.

2. The adaptive frequency band selection method for sound source localization according to claim 1, characterized in that: Specifically, by obtaining the sub-band after binary division, obtaining the power corresponding to the peak frequency of the sub-band and marking it as fs, marking the power corresponding to the background noise frequency of the peak frequency as Pn, and defining its signal-to-noise ratio as SNR: The SNR is marked as the influence factor for obtaining the positioning frequency band, and the influence factor weight is marked as w1.

3. The adaptive frequency band selection method for sound source localization according to claim 1, characterized in that: According to the actual sound signals collected by the acoustic array probe, when there is an abnormal sound source, it has certain distinguishing characteristics relative to the background noise, and has a certain degree of unevenness. Here, the unevenness is used as an influencing factor affecting the adaptive selection of the frequency band, marked as sf: Among them, the sub-band spectrum after binary division is marked as X[k], where k = 1, 2, ..., N, N represents the kth frequency component in the frequency band. When the spectrum is completely flat, sf is equal to 1. The smaller the sf value, the less flat the spectrum is. At this time, the impact factor weight is marked as w2.

4. The adaptive frequency band selection method for sound source localization according to claim 1, wherein: Coherence is used to evaluate the spatial consistency of the sound array. A pair of microphones is randomly selected from the microphone array that receives the sound source signal. The coherence is used to measure the correlation between the two groups of signals, which is defined as r 2 xy (k), calculated as follows: Among them, P xx (k), P yy (k) are the power spectral densities of a pair of microphones, P xy (k) is the cross-power spectral density between a pair of microphones.

5. The adaptive frequency band selection method for sound source localization according to claim 4, characterized in that: In order to reduce the error in the calculated coherence coefficient between any two groups of signals, N pairs of microphones on the array probe are randomly selected, and the average value of the N pairs of microphones is used as the influencing factor affecting the positioning accuracy, which is marked as The calculation method is: And mark the weight of the impact factor as w3.

6. The adaptive frequency band selection method for sound source localization according to claim 1, characterized in that: During the detection process, the frequency band interval is selected based on the main frequency of the sound radiated by the detected object, and then adaptive frequency band selection is performed in the selected frequency band interval to reduce the computational complexity of binary decomposition to find the optimal frequency band.

7. The adaptive frequency band selection method for sound source localization according to claim 1, characterized in that: The target frequency is calculated as follows: The reference frequency is the mean of the peak frequencies within the sub-band, marked as The target frequency is marked as f target .

8. The adaptive frequency band selection method for sound source localization according to claim 1, characterized in that: To determine whether the retained adjacent frequency bands need to be merged, the calculation formula for defining the difference between adjacent sub-bands is: where f target_a and f target_b The target frequencies of the adjacent frequency bands are retained respectively, and the difference threshold is set at the same time. If the difference RD of adjacent sub-bands is within the set threshold range, the adjacent frequency bands are merged into one frequency band for sound source localization and no further division is performed; if the difference RD of adjacent sub-bands is not within the set threshold range, they are treated as two separate positioning frequency bands and further divided using the binary method. The number of cyclic judgment and division using the binary method shall not exceed 10 times.

9. An adaptive frequency band selection system for sound source localization, used to execute the adaptive frequency band selection method for sound source localization according to claims 1-8, characterized in that: include: Microphone array module: used to collect sounds in the target area, obtain the corresponding time domain signals, perform spectrum analysis on the collected time domain signals, and obtain the spectrum information of each microphone's time domain signal in real time, including frequency components or amplitude information; A signal processing module is configured to divide the spectrum into frequency bands based on the spectrum information, wherein the frequency bands include an audible sound band, an ultrasonic band, and a full frequency band; the frequency bands are divided using a dichotomy method, with the signal-to-noise ratio as a first weighting factor, the spectrum flatness as a second weighting factor, and the coherence as a third weighting factor; Frequency band screening module: obtains the target frequency by solving the reference frequency in each sub-band; Set the frequency threshold within each sub-band, compare the target frequency with the threshold frequency, and if the target frequency is within the set frequency threshold range, retain the sub-band; otherwise, determine that the sub-band is invalid and remove it; The binary division module is used to compare the target frequency with the set threshold to determine whether it is the required retained adaptive frequency band. If the target frequency is within the set threshold, the sub-frequency band is retained; if it exceeds the threshold, the frequency band is eliminated; and then determine whether the retained sub-frequency bands are adjacent frequency bands. If they are adjacent, they are merged into one frequency band and no longer undergo binary division. If they are not adjacent, binary division is used to further divide and determine the non-adjacent sub-frequency bands; When the target frequency is obtained, the frequency threshold in each sub-band is set, and the target frequency is compared with the threshold frequency. If the target frequency is within the set frequency threshold range, the sub-band is retained; otherwise, the sub-band is determined to be an invalid band and is removed.

Images