Sound source separation method and system for microphone array to pick up voice signals

Abstract

The invention discloses a sound source separation method and system for a microphone array to pick up voice signals. The method comprises the following steps: picking up voice signals sent by a target reactor sound source and an interference sound source in a space by using the microphone array; performing segmentation and Fourier transform on the voice signal to obtain a time-frequency domain signal; calculating the time difference of arrival and the phase difference between each pair of microphones, and obtaining the direction estimation result of each sound source in the space by combining the geometric arrangement information of the microphone array; generating a time-frequency mask, applying the time-frequency mask to the time-frequency domain signal, and separating the time-frequency domain signal of each sound source; performing inverse short-time Fourier transform and fragment splicing to obtain a corresponding continuous time domain voice signal; and outputting to different audio output channels. According to the invention, the specific sound source signal generated by the reactor can be effectively separated, the operation state of the reactor can be accurately analyzed, the abnormal condition can be timely found, and the reliability and precision of monitoring the reactor can be further improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of reactor monitoring, and relates to a method and system for separating sound sources of voice signals picked up by a microphone array. Background Art

[0002] The high-voltage shunt reactor is one of the core devices in the long-distance power transmission and transformation network, and has many functions such as adjusting reactive power, limiting overvoltage, quickly cutting off arcs, and maintaining the stability of the power grid. The condition detection of the high-voltage shunt reactor during operation is crucial for ensuring the safety of the equipment and even the high-voltage power transmission network. The state detection technology based on sound signals has the advantages of low cost and non-contact, and is a feasible means for carrying out the state detection of high-voltage shunt reactors under operating conditions. However, in complex acoustic environments such as substations, there are problems such as large noise interference and high similarity between the sound signals of non-target reactors and target reactors, resulting in difficulty in accurately extracting the sound signals of target reactors.

[0003] The sound source separation technology can separate, enhance, and locate specific sound sources in a multi-source environment to improve the clarity and accuracy of voice signals. However, traditional sound source separation methods are difficult to effectively separate multiple sound sources in complex environments, and there is a problem of low accuracy of sound source separation in a multi-source environment. At the same time, the accuracy of sound source localization of the reactor is affected by environmental noise and equipment errors. Traditional sound source separation methods cannot dynamically adapt to changes in the position of the sound source, have poor adaptability to dynamic sound sources, and are difficult to effectively highlight the characteristics of the sound source of the target reactor in the presence of interference signals. Summary of the Invention

[0004] To solve the deficiencies in the prior art, the present invention provides a method and system for separating sound sources of voice signals picked up by a microphone array. The method collects signals through a microphone array, combines time-frequency analysis to clearly separate different sound sources in the time-frequency domain, and performs localization based on directionality estimation and time difference of arrival, effectively extracting and distinguishing the azimuth information of different sound sources. Based on the information of time difference and phase difference, the positioning deviation caused by noise and equipment errors is reduced. By weighted averaging to fuse the azimuth information of different time-frequency points, precise positioning in a dynamic environment is achieved. By initializing and dynamically adjusting the weights of the time-frequency mask, the area matching the azimuth of the sound source of the target reactor has a higher weight, suppressing interference signals, improving the focusing and anti-interference ability of the sound source of the target reactor, and enhancing the adaptability to dynamic sound sources.

[0005] The present invention adopts the following technical solutions.

[0006] The first aspect of the present invention proposes a method for separating sound sources of voice signals picked up by a microphone array, including:

[0007] Use a microphone array to pick up the speech signals emitted by the target reactor sound source and interference sound sources in space; segment and perform Fourier transform on the speech signals picked up by each microphone channel to obtain the corresponding time-frequency domain signals;

[0008] Based on the time-frequency domain signals corresponding to each microphone channel, calculate the time difference of arrival and phase difference of the sound source signals between each pair of microphones, and combine with the geometric arrangement information of the microphone array to obtain the direction estimation result of each sound source in space;

[0009] Generate corresponding time-frequency masks for each sound source based on the direction estimation results; apply the time-frequency masks corresponding to each sound source to the time-frequency domain signals corresponding to each microphone channel to separate the time-frequency domain signals of each sound source;

[0010] Perform inverse short-time Fourier transform and segment splicing on the separated time-frequency domain signals of each sound source to obtain the continuous time-domain speech signals corresponding to each sound source; output the continuous time-domain speech signals corresponding to each sound source to different audio output channels and store the speech signals of each sound source independently.

[0011] Preferably, the step of using a microphone array to pick up the speech signals emitted by the target reactor sound source and interference sound sources in space includes:

[0012] Arrange the microphone array according to the desired sound source localization accuracy and frequency range, adjust the microphone spacing and positions, and perform clock synchronization processing on each microphone channel in the microphone array;

[0013] The microphone array receives the time-domain signals emitted by the target reactor sound source and interference sound sources in space, and digitally samples the time-domain signals received by each microphone channel at a preset sampling frequency to obtain discrete multi-channel digital speech signals.

[0014] Preferably, the step of segmenting and performing Fourier transform on the speech signals picked up by each microphone channel to obtain the corresponding time-frequency domain signals includes:

[0015] Segment the speech signals picked up by each microphone channel through an overlapping window and divide them into multiple short-time windows;

[0016] Apply the fast Fourier transform to the signals of each short-time window to obtain the corresponding time-frequency domain signals.

[0017] Preferably, the step of calculating the time difference of arrival and phase difference of the sound source signals between each pair of microphones based on the time-frequency domain signals corresponding to each microphone channel, and combining with the geometric arrangement information of the microphone array to obtain the direction estimation result of each sound source in space includes:

[0018] Calculate the time difference of arrival (TDOA) and phase difference of arrival (PDOA) of the sound source signal between each pair of microphones based on the time-frequency domain signals corresponding to each microphone channel.

[0019] Calculate the relative distance difference between the sound source and each microphone of the microphone array according to the TDOA. Combine the geometric position of the microphone array and the PDOA of the sound source signal to derive and calculate the relative azimuth angle and elevation angle of the sound source corresponding to each time-frequency point. Convert the calculation results of the azimuth angle and elevation angle into spatial coordinate representations to determine the relative azimuth information of the sound source.

[0020] Allocate weights according to the energy magnitude and signal-to-noise ratio (SNR) of the time-frequency points to perform weighted averaging on the relative azimuth information of each sound source at different time-frequency points, and obtain the direction estimation result of each sound source in space.

[0021] Preferably, the allocation of weights according to the energy magnitude and SNR of the time-frequency points is as follows:

[0022]

[0023] Among them, W represents the weight of the time-frequency point, E represents the energy of the current time-frequency point, E max represents the maximum energy among all time-frequency points, SNR represents the SNR of the current time-frequency point, SNT max represents the maximum SNR among all time-frequency points, α represents the energy weight coefficient, and β represents the SNR weight coefficient.

[0024] Preferably, generating a corresponding time-frequency mask for each sound source based on the direction estimation result includes:

[0025] Initialize the time-frequency mask weight of each sound source based on the direction estimation result of each sound source in space.

[0026] Based on the initial time-frequency mask weight, perform normalized dynamic adjustment on the mask weight according to the energy characteristics of the target reactor sound source in the time-frequency domain, and obtain the time-frequency mask through binarization or ratioization.

[0027] Perform smoothing processing on the time-frequency mask so that the mask shows continuous high-weight regions and low-weight regions in the time-frequency domain.

[0028] Preferably, the normalized dynamic adjustment of the mask weight according to the energy characteristics of the target reactor sound source in the time-frequency domain based on the initial time-frequency mask weight is as follows:

[0029]

[0030] Among them, M(t,f) represents the initial time-frequency mask weight, M′(t,f) represents the adjusted time-frequency mask weight, γ represents the adjustment factor, E represents the energy at the current time-frequency point, and E max represents the maximum energy among all time-frequency points, t is time, and f is frequency.

[0031] Preferably, applying the time-frequency mask corresponding to each sound source to the time-frequency domain signal corresponding to each microphone channel to separate the time-frequency domain signal of each sound source includes:

[0032] Multiplying the time-frequency mask corresponding to each sound source with the time-frequency domain signal corresponding to each microphone channel point by point to strengthen the target reactor sound source signal component in the high-weight area and suppress the interfering sound source signal component in the low-weight area, and separating the time-frequency domain signal of each sound source applied with the mask;

[0033] Performing noise estimation and threshold filtering on the time-frequency domain signal of each sound source applied with the mask to suppress the remaining noise components and obtain the enhanced time-frequency domain signal of each sound source;

[0034] Performing gain control on the enhanced time-frequency domain signal of each sound source, and compressing the amplitude of the signal within a preset range through dynamic range to obtain the finally separated time-frequency domain signal of each sound source.

[0035] Preferably, performing inverse short-time Fourier transform and segment splicing on the separated time-frequency domain signal of each sound source to obtain the continuous time-domain speech signal corresponding to each sound source includes:

[0036] Performing inverse short-time Fourier transform on the time-frequency domain signal of each sound source to convert the complex spectrum in the time-frequency domain back to the signal segments in the time domain, overlapping and splicing the signal segments in the time domain, performing weighted averaging on adjacent segments according to their overlapping parts, and performing amplitude normalization processing to obtain the corresponding continuous time-domain speech signal.

[0037] Preferably, outputting the continuous time-domain speech signal corresponding to each sound source to different audio output channels and independently storing the speech signal of each sound source includes:

[0038] Associating the continuous time-domain speech signal corresponding to each sound source with different audio output channels;

[0039] Performing format conversion on the time-domain speech signal in each audio output channel, converting it into a predetermined audio output format, and storing the format-converted speech signal in the corresponding storage medium and device to generate independent audio files or data streams for each sound source signal.

[0040] The second aspect of the present invention proposes a sound source separation system for a microphone array to pick up speech signals, including:

[0041] A signal acquisition module, configured to pick up voice signals emitted by a target reactor sound source and an interference sound source in space by using a microphone array;

[0042] A time-frequency transformation module, configured to segment and perform Fourier transform on the voice signals picked up by each microphone channel to obtain corresponding time-frequency domain signals;

[0043] A direction estimation module, configured to calculate the time difference of arrival and phase difference of the sound source signals between each pair of microphones based on the time-frequency domain signals corresponding to each microphone channel, and combine with the geometric arrangement information of the microphone array to obtain the direction estimation result of each sound source in space;

[0044] A mask generation module, configured to generate a corresponding time-frequency mask for each sound source based on the direction estimation result;

[0045] A signal enhancement module, configured to apply the time-frequency mask corresponding to each sound source to the time-frequency domain signals corresponding to each microphone channel to separate the time-frequency domain signals of each sound source;

[0046] An inverse time-frequency domain transformation module, configured to perform inverse short-time Fourier transform and segment splicing on the separated time-frequency domain signals of each sound source to obtain continuous time-domain voice signals corresponding to each sound source;

[0047] A voice output module, configured to output the continuous time-domain voice signals corresponding to each sound source to different audio output channels and store the voice signals of each sound source independently.

[0048] A third aspect of the present invention proposes a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method.

[0049] A fourth aspect of the present invention proposes a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method are implemented.

[0050] The beneficial effects of the present invention are as follows. Compared with the prior art, the present invention collects signals at multiple different positions through a microphone array and uses time-frequency analysis to clearly represent each sound source in the time-frequency domain, thereby realizing accurate separation of sound sources. Sound source localization is carried out based on directionality estimation and time difference of arrival, enabling effective extraction and differentiation of the azimuth information of different sound sources. By performing inverse transformation on the separated time-frequency signals, they are restored to continuous time-domain waveforms, and the signals of different sound sources are respectively output to different audio channels, facilitating subsequent storage, processing, and application, and meeting the requirements of multi-channel speech processing. Utilizing the spatial acquisition ability of the microphone array, it can be applied to much more complex multi-sound-source environments, including scenarios with echo or background noise, improving the practicability and robustness in different application scenarios. In the reactor monitoring, it can effectively separate the specific sound source signals generated by the reactor, extract useful signals from background noise and other interference signals, contribute to accurately analyzing the operating state of the reactor and promptly detecting abnormal situations, and further enhancing the reliability and accuracy of reactor monitoring. Further, the beneficial effects of the present invention also include:

[0051] 1. When collecting signals of the present invention, by optimizing the arrangement of the microphone array and selecting appropriate microphone spacing and positions, the spatial resolution of the microphone array can be effectively improved on the premise of meeting the sound source localization accuracy and frequency range, enabling more accurate capture of the spatial information of the sound source, making the sound source localization accuracy higher. Clock synchronization is performed on the microphone array channels to ensure that the receiving time of signals in all microphone channels is consistent, eliminating the time deviation between channels and avoiding localization deviation caused by synchronization errors, further enhancing the accuracy of sound source localization.

[0052] The signals of the microphone channels are digitally sampled at a preset sampling frequency to generate multi-channel digital signals, which can ensure the stability and quality of the collected signals. The discrete signals obtained by digital sampling are stored in the processing unit, facilitating subsequent data processing and storage, and ensuring the consistency and integrity of the data.

[0053] 2. When performing time-frequency transformation in the present invention, the signal is processed in segments. By using the overlapping short-time window method to increase the time resolution, the time-varying characteristics of the speech signal can be captured more precisely, which helps to identify the characteristics of different sound source signals in the frequency domain and improves the subsequent sound source separation effect. By applying a window function to each short-time window, spectral leakage is reduced, making the transformed frequency-domain signal more accurate, enhancing the frequency resolution, and contributing to improving the separation effect of the signal in the time-frequency domain.

[0054] The fast Fourier transform is applied to convert the time-domain signal into a frequency-domain signal. Compared with directly calculating the Fourier transform, it improves the calculation efficiency, shortens the signal processing time, enables the system to process multi-channel speech signals in real time or near real time in practical applications, and improves the overall processing speed.

[0055] 3. When the present invention performs direction estimation, it performs sound source localization based on the time difference and phase difference information received by each microphone of the microphone array. By calculating the time difference of arrival and phase difference between each pair of microphones, the subtle differences of the sound source signal can be captured more precisely. Combining with the geometric arrangement information of the microphone array helps to improve the azimuth estimation accuracy of the sound source. Calculating the time difference of arrival and phase difference can effectively reduce the positioning deviation caused by environmental noise or equipment errors, making the directivity estimation more reliable and stable.

[0056] By calculating the azimuth information at different time-frequency points and performing weighted average processing, the azimuth information at each time-frequency point can be effectively fused together, accurately distinguishing the positions of different sound sources. The separation effect in a multi-source environment is particularly significant, enabling the method to better adapt to complex sound source scenarios.

[0057] By performing real-time estimation of the time difference of arrival and phase difference, the system can quickly respond to the dynamic changes of the sound source position. Even if the sound source moves or changes direction in space, it can re-locate faster and perform corresponding direction estimation, improving the robustness and response speed of the system in real-time application scenarios. In the application of reactor monitoring, the sound source localization technology can be used to identify the direction of the noise source of the reactor, thereby distinguishing the internal fault noise of the reactor from the external environmental noise, helping the operation and maintenance personnel quickly locate potential problems and improving the maintenance efficiency.

[0058] 4. When the present invention generates a mask, by initializing the time-frequency mask weight, the weight of the area matching the azimuth of the target reactor sound source is higher, and the weight of the non-matching area is lower. This can effectively highlight the time-frequency information of the target reactor sound source and suppress interfering sound sources. Especially in a multi-source environment, this initialization process helps to improve the focusing ability on the target reactor sound source and significantly reduce the influence of interfering signals.

[0059] By dynamically adjusting the mask weight based on the energy characteristics of the target reactor sound source in the time-frequency domain, it is ensured that the mask can adapt to the changes of the sound source. As the energy characteristics of the target reactor sound source change, the mask weight can be updated in real-time, enhancing the adaptability to dynamic sound sources. This adaptive adjustment method helps to obtain a more stable and clear separation effect in a complex environment.

[0060] The present invention combines direction estimation and energy to calculate weights, rather than a single signal-to-noise ratio method, making the mask more adaptive, enhancing the generalization ability, improving the accuracy of sound source separation, reducing noise interference, and at the same time reducing speech damage.

[0061] By smoothing the mask, the high-weight and low-weight regions in the time-frequency mask become more continuous, avoiding discrete mutations caused by noise or rapid changes. This smoothing process can effectively reduce the intermittent artifacts in the mask, thereby enhancing the naturalness and smoothness of the separated signal and making the final voice output clearer.

[0062] By using the time-frequency mask matrix in combination with direction information and assigning weights according to the matching degree between the energy characteristics and the direction angle, the weight ratio of the mask can be controlled more precisely. In a complex sound field, the difference in the direction angle between different sound sources and the target reactor sound source may be small. By adjusting the weight parameters and the reference weight in the formula, the direction discrimination ability can be significantly improved, effectively filtering out non-target reactor sound sources. In reactor monitoring, by highlighting the specific sound source signal of the target reactor, the abnormal sound characteristics during the operation of the reactor can be separated and analyzed more efficiently, supporting dynamic adjustment to adapt to the sound source changes under different operating conditions of the reactor, and further improving the monitoring accuracy of the reactor operating state.

[0063] 5. When the present invention performs signal enhancement, by multiplying the time-frequency mask with the time-frequency signal point by point and strengthening the high-weight region, the signal component of the target reactor sound source is effectively enhanced, while suppressing the interference signal. It can highlight the target voice in a complex environment, making the voice clearer and easier to understand. By noise estimation and threshold filtering, the residual noise signal components after mask application are further removed, thereby improving the signal-to-noise ratio at a more subtle level, reducing the interference of background noise on the output signal, and making the signal separation more thorough. The introduction of gain control ensures that the signal remains stable within the preset dynamic range, and the amplitude is controlled by dynamic range compression to avoid distortion that may be caused by excessive signal amplitude fluctuations, which helps to improve the stability and consistency of the sound quality.

[0064] 6. When the present invention performs the inverse transform in the time-frequency domain, by performing the inverse short-time Fourier transform on the enhanced time-frequency signal, the complex spectrum in the time-frequency domain is accurately restored to the time-domain signal segment, ensuring the integrity of the signal. It can ensure that when converting from the frequency domain back to the time domain, the signal distortion rate is extremely low, achieving a high-fidelity effect. During the overlapping splicing process, a weighted average method is used to smoothly connect adjacent signal segments, making the reconstructed time-domain signal continuous and natural, avoiding discontinuity or non-smoothness at the splicing point, and improving the fluency and audibility of the voice signal. The amplitude normalization process ensures that the amplitude of the final time-domain signal is uniformly distributed within a reasonable range, which helps to improve the consistency of the volume, making the sound quality of the signal more balanced during playback or storage and easier for subsequent processing.

[0065] 7. When the present invention performs voice output, each separated sound source signal is associated with a specific output channel, ensuring independent output of multiple sound sources. This not only helps in analyzing each sound source individually but also can be used in audio devices with multi-channel output, facilitating subsequent processing or monitoring. Format conversion is performed on the separated signals to ensure that the output audio conforms to a predetermined format, enhancing the universality and compatibility of the file and making it convenient to use on different devices and platforms. The separated signals are stored as independent audio files or data streams, facilitating subsequent archiving, analysis, and processing. At the same time, the generation of independent audio files is conducive to post-analysis in multi-source scenarios and helps to achieve diverse applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a schematic flowchart of a method for separating sound sources of voice signals picked up by a microphone array according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. The embodiments described in this application are only a part of the embodiments of the present invention, rather than all embodiments. Based on the spirit of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0068] As Figure 1 shown, Embodiment 1 of the present invention provides a method for separating sound sources of voice signals picked up by a microphone array, including the following steps:

[0069] Step 1: Use a microphone array to pick up voice signals emitted by a target reactor sound source and interfering sound sources in space;

[0070] Further preferably, in this step, microphone array signal acquisition is performed. An array structure including multiple microphones is used to simultaneously collect the sound source signals of the target reactor in space. The microphone array receives the sound waveforms of the target reactor sound source through multiple microphones at different positions, forming a multi-channel time-domain signal. Specifically, it includes:

[0071] Arrange the microphone array according to the desired sound source localization accuracy and frequency range, adjust the microphone spacing and position, and perform clock synchronization processing on each microphone channel in the microphone array to eliminate the time deviation between each microphone channel;

[0072] Digitally sample the time-domain signals received by each microphone channel at a preset sampling frequency to generate discrete multi-channel digital signals, and store the collected multi-channel signals in the processing unit.

[0073] When collecting the above signals, in scenarios with high sound source localization accuracy and a wide frequency range, and in applications that require precise separation of sound sources, the microphone array can be arranged as a linear array, and the spacing between microphones can be set to about 0.05 meters to meet the resolution requirements of high-frequency sound waves and ensure the spatial resolution ability of the array for the target sound source. In scenarios with a relatively low frequency range and general localization accuracy, to capture low-frequency sound source signals (such as equipment noise), a larger microphone spacing (such as 0.2 meters to 0.5 meters) can be selected and a circular array structure can be adopted to improve the sensitivity of the array to low-frequency sound waves.

[0074] When collecting the above signals, by optimizing the arrangement of the microphone array, selecting appropriate microphone spacing and positions, it is possible to effectively improve the spatial resolution of the microphone array while meeting the sound source localization accuracy and frequency range, capture the spatial information of the sound source more accurately, make the sound source localization accuracy higher, synchronize the clocks of the microphone array channels to ensure that the time when all microphone channels receive signals is consistent, eliminate the time deviation between channels, avoid the localization deviation caused by synchronization errors, and further improve the accuracy of sound source localization.

[0075] The signals of the microphone channels are digitally sampled at a preset sampling frequency to generate multi-channel digital signals, which can ensure the stability and quality of the collected signals. The discrete signals obtained by digital sampling are stored in the processing unit, which is convenient for subsequent data processing and storage, and ensures the consistency and integrity of the data.

[0076] Step 2: Segment and perform Fourier transform on the voice signals picked up by each microphone channel to obtain the corresponding time-frequency domain signals;

[0077] Further preferably, when performing this time-frequency transformation, the time-domain signals received by each microphone channel are processed. The time-domain signals are segmented and the Fourier transform is applied to obtain the distribution of the time-domain signals in time and frequency, and the signals of each channel of the microphone array are all converted into time-frequency domain representations. Specifically, it includes:

[0078] The time-domain signals received by each microphone channel are segmented. The continuous time-domain signals are divided into multiple short-time windows. Each short-time window contains a certain number of sampling points, and the time resolution is increased by overlapping windows during the segmentation process;

[0079] A window function is applied to the time-domain signals of each segmented short-time window, and the fast Fourier transform is applied to each segment of the time-domain signals of each microphone channel to convert the segmented time-domain signals into frequency-domain signals and obtain the time-frequency representations of each segment of the signals.

[0080] When performing the above time-frequency transformation, the signal is processed in segments. By overlapping short-time windows, the time resolution is increased, enabling more precise capture of the time-varying characteristics of the speech signal. This helps in identifying the characteristics of different sound source signals in the frequency domain, enhancing the subsequent sound source separation effect. By applying a window function to each short-time window, spectral leakage is reduced, making the transformed frequency-domain signal more accurate, enhancing the frequency resolution, and contributing to improving the separation effect of the signal in the time-frequency domain.

[0081] The fast Fourier transform is applied to convert the time-domain signal into a frequency-domain signal. Compared with directly calculating the Fourier transform, the computational efficiency is improved, and the signal processing time is shortened. This enables the system to process multi-channel speech signals in real-time or near real-time in practical applications, enhancing the overall processing speed.

[0082] Step 3: Based on the time-frequency domain signals corresponding to each microphone channel, calculate the time difference of arrival and phase difference of the sound source signals between each pair of microphones, and combine them with the geometric arrangement information of the microphone array to obtain the direction estimation result of each sound source in space;

[0083] Further preferably, in this step, directivity estimation is performed. According to the time-frequency domain signals, the directivity information of multiple sound sources in space is estimated, and based on the geometric arrangement of the microphone array, the direction of the sound source is located in combination with the time difference of arrival; specifically including:

[0084] Step 3.1: Based on the time-frequency domain signals at the time-frequency points corresponding to each microphone channel, calculate the time difference of arrival and phase difference of arrival of the sound source signals between each pair of microphones;

[0085] Among them, the time difference of arrival of the sound source signal is calculated by the following formula:

[0086]

[0087] Among them, Δt represents the time difference of arrival of the sound source signal, dn, d n+1 represent the distances from the sound source to the nth and (n + 1)th microphones, c represents the speed of sound, and n represents the serial number of the microphone in the array;

[0088] The phase difference of arrival of the sound source signal is calculated by the following formula:

[0089]

[0090] Among them, represents the phase difference of arrival of the sound source signal, f represents the signal frequency, and T represents the signal period;

[0091] Step 3.2: Combine the time difference of arrival and phase difference of arrival of the sound source signal between each pair of microphones with the geometric arrangement information of the microphone array to obtain the relative azimuth information of the sound source in space corresponding to each time-frequency point;

[0092] Specifically: Calculate the relative distance difference between the sound source and each microphone of the microphone array according to the time difference of arrival, and combine the geometric positions of the microphone array and the phase difference of arrival of the sound source signal to derive and calculate the relative azimuth angle and elevation angle of the sound source corresponding to each time-frequency point, convert the calculation results of the azimuth angle and elevation angle into spatial coordinate representations, and determine the relative azimuth information of the sound source;

[0093] Step 3.3: Assign weights according to the energy magnitude and signal-to-noise ratio of the time-frequency points to perform weighted averaging on the relative azimuth information of each sound source at different time-frequency points to obtain the direction estimation result of each sound source in space.

[0094] Among them, the weight of the time-frequency point is calculated by the following formula:

[0095]

[0096] Among them, W represents the weight of the time-frequency point, E represents the energy of the current time-frequency point, E max represents the maximum energy among all time-frequency points, SNR represents the signal-to-noise ratio of the current time-frequency point, SNT max represents the maximum signal-to-noise ratio among all time-frequency points, α represents the energy weight coefficient, and β represents the signal-to-noise ratio weight coefficient.

[0097] During the above direction estimation, sound source localization is performed based on the time difference and phase difference information received by each microphone of the microphone array. By calculating the time difference of arrival and phase difference of arrival between each pair of microphones, the subtle differences of the sound source signal can be captured more accurately. Combining the geometric arrangement information of the microphone array helps to improve the azimuth estimation accuracy of the sound source. Calculating the time difference of arrival and phase difference of arrival can effectively reduce the positioning deviation caused by environmental noise or equipment errors, making the directionality estimation more reliable and stable.

[0098] By calculating the azimuth information of different time-frequency points and performing weighted averaging processing, the azimuth information of each time-frequency point can be effectively fused together, accurately distinguishing the positions of different sound sources, and the separation effect in a multi-source environment is particularly significant, enabling the method to better adapt to complex sound source scenarios.

[0099] By performing real-time estimation of the time difference of arrival and the phase difference, the system can quickly respond to the dynamic changes in the sound source position. Even if the sound source moves or changes direction in space, it can re-locate relatively quickly and perform corresponding direction estimation, improving the robustness and response speed of the system in real-time application scenarios. In the application of reactor monitoring, the sound source localization technology can be used to identify the direction of the noise source of the reactor, so as to distinguish the internal fault noise of the reactor from the external environmental noise, helping the operation and maintenance personnel quickly locate potential problems and improve the maintenance efficiency.

[0100] Step 4: Generate corresponding time-frequency masks for each sound source based on the direction estimation result;

[0101] Further preferably, in this step of mask generation, based on the result of the directivity estimation, corresponding time-frequency masks are generated for each sound source. Through the directivity information, the region in the time-frequency domain belonging to the sound source of the target reactor is marked with a high weight, and the region of the interfering sound source is marked with a low weight; specifically including:

[0102] Step 4.1: Initialize the time-frequency mask weight of each sound source based on the azimuth information of each sound source;

[0103] Among them, for the region in the time-frequency domain that matches the azimuth of the sound source of the target reactor, the initial weight is set to a high value, and for the non-matching region, the initial weight is set to a low value. For example, when the direction estimation result is the azimuth angle θ of the sound source of the target reactor, according to this direction estimation result, the estimated azimuth angle γ′ of each time-frequency point in the time-frequency domain is compared; if |γ′ - θ| < δ, it is determined that this time-frequency point is a region that matches the azimuth of the sound source of the target reactor, that is, this time-frequency point belongs to the azimuth region of the sound source of the target reactor, and the initial frequency mask weight of this time-frequency point is set to a high value. Otherwise, it is determined as a non-matching region, that is, an interference region, and the initial frequency mask weight of this time-frequency point is set to a low value, where δ is a set threshold.

[0104] Step 4.2: Based on the initial time-frequency mask weight, dynamically adjust the mask weight according to the energy characteristics of the sound source of the target reactor in the time-frequency domain, and obtain the time-frequency mask through binarization or ratio-ization; specifically, dynamically adjust the time-frequency mask weight based on energy normalization. The adjusted time-frequency mask weight is calculated by the following formula:

[0105]

[0106] Among them, M(t,f) represents the initial time-frequency mask weight, M′(t,f) represents the adjusted time-frequency mask weight, γ represents the adjustment factor, the value range of the adjustment factor is [0.3 - 0.7], E represents the energy of the current time-frequency point, and E max represents the maximum energy among all time-frequency points.

[0107] Step 4.3: Smooth the time-frequency mask so that the mask appears as continuous high-weight and low-weight regions in the time-frequency domain.

[0108] The generation process of the time-frequency mask is achieved by calculating and adjusting weights. The generation of the time-frequency mask does not directly obtain a binary mask (i.e., simply 0 or 1), but a weight matrix. These weights are used to indicate the belonging probability of each time-frequency point in the time-frequency domain. When generating the above mask, in Step 4.1, based on the direction estimation result of the sound source, by initializing the time-frequency mask weights, the regions matching the azimuth of the target reactor sound source have higher weights, and the regions not matching have lower weights, starting to construct the preliminary form of the time-frequency mask. This can effectively highlight the time-frequency information of the target reactor sound source and suppress interfering sound sources. Especially in a multi-source environment, this initialization process helps to improve the focusing ability on the target reactor sound source and significantly reduce the influence of interference signals. In Step 4.2, based on factors such as the energy characteristics of the target reactor sound source in the time-frequency domain, the weights are normalized and adjusted. After the time-frequency mask weights are calculated, the time-frequency mask is obtained through binarization or ratioization, ensuring that the mask can adapt to the changes of the sound source. As the energy characteristics of the target reactor sound source change, the mask weights can be updated in real time, enhancing the adaptability to dynamic sound sources. This adaptive adjustment method helps to obtain a more stable and clear separation effect in a complex environment.

[0109] By smoothing the mask, the high-weight and low-weight regions in the time-frequency mask become more continuous, avoiding discrete mutations caused by noise or rapid changes. This smoothing process can effectively reduce the intermittent artifacts in the mask, thereby enhancing the naturalness and fluency of the separated signal and making the final speech output clearer.

[0110] Using the time-frequency mask matrix combined with direction information, and assigning weights according to the matching degree between the energy characteristics and the direction angle can more precisely control the weight ratio of the mask. In a complex sound field, the direction angles of different sound sources may have a small difference from that of the target reactor sound source. By adjusting the weight parameters and reference weights in the formula, the direction discrimination ability can be significantly improved, effectively filtering out non-target reactor sound sources. In the monitoring of reactors, by highlighting the specific sound source signals of the target reactor, the abnormal sound characteristics during the operation of the reactor can be separated and analyzed more efficiently, supporting dynamic adjustment to adapt to the sound source changes under different working conditions of the reactor, and further improving the monitoring accuracy of the reactor operating state.

[0111] Step 5: Apply the time-frequency mask corresponding to each sound source to the time-frequency domain signal corresponding to each microphone channel to separate the time-frequency domain signal of each sound source;

[0112] Further preferably, in this step, signal enhancement is performed, and the generated time-frequency mask is applied to the time-frequency signal to strengthen the time-frequency components of the target reactor sound source; specifically, it includes:

[0113] Step 5.1: Multiply the generated time-frequency mask and the time-frequency signal point by point to strengthen the signal components of the target reactor sound source in the high-weight region, and simultaneously suppress the interference signals in the low-weight region;

[0114] Step 5.2: Process the time-frequency signal to which the mask has been applied, and suppress the remaining noise components through noise estimation and threshold filtering;

[0115] Step 5.3: Perform gain control on the enhanced time-frequency signal, and compress the amplitude of the signal within a preset range through dynamic range compression.

[0116] During the above signal enhancement, by multiplying the time-frequency mask and the time-frequency signal point by point, the high-weight region is strengthened, effectively improving the signal components of the target reactor sound source, while suppressing interference signals, which can highlight the target voice in a complex environment, making the voice clearer and easier to understand. Through noise estimation and threshold filtering, the remaining noise signal components after mask application are further removed, thereby improving the signal-to-noise ratio at a finer level, reducing the interference of background noise on the output signal, and making the signal separation more complete. The introduction of gain control ensures that the signal remains stable within the preset dynamic range, and the amplitude is controlled through dynamic range compression, avoiding distortion that may be caused by excessive signal amplitude fluctuations, which helps to improve the stability and consistency of the sound quality.

[0117] Step 6: Perform inverse short-time Fourier transform and segment splicing on the time-frequency domain signals of each separated sound source to obtain the continuous time-domain voice signal corresponding to each sound source;

[0118] Further preferably, in this step, inverse time-frequency domain transformation is performed. The separated time-frequency signal is subjected to inverse short-time Fourier transform to restore it from the time-frequency domain signal to the time-domain signal, and the inverse transform is performed on each separated sound source signal respectively to obtain a continuous time-domain waveform; specifically, it includes:

[0119] Perform inverse short-time Fourier transform on the time-frequency signal of each sound source after enhancement processing to convert the complex spectrum in the time-frequency domain back to the signal segments in the time-domain. Overlap and splice the time-domain signal segments after inverse Fourier transform, and perform weighted averaging on adjacent segments according to their overlapping parts;

[0120] Perform amplitude normalization processing on the spliced time-domain signal.

[0121] During the above-mentioned inverse time-frequency transformation, by performing an inverse short-time Fourier transform on the enhanced time-frequency signal, the complex spectrum in the time-frequency domain is accurately restored to a time-domain signal segment, ensuring the integrity of the signal. It can ensure that when converting from the frequency domain back to the time domain, the signal distortion rate is extremely low, achieving a high-fidelity effect. During the overlapping splicing process, a weighted average method is used to smoothly connect adjacent signal segments, making the reconstructed time-domain signal continuous and natural, avoiding discontinuity or non-smoothness at the splicing point, and improving the fluency and audibility of the speech signal. The amplitude normalization process ensures that the amplitude of the final time-domain signal is uniformly distributed within a reasonable range, helping to improve the consistency of the volume, making the sound quality of the signal more balanced during playback or storage and facilitating subsequent processing.

[0122] Step 7: Output the continuous time-domain speech signal corresponding to each sound source to different audio output channels, and store the speech signals of each sound source independently.

[0123] Further preferably, for this step of speech output, the separated time-domain speech signals are output to different audio channels, and the speech signals of each sound source are stored independently. Specifically, it includes:

[0124] Based on the result of sound source separation, associate each separated sound source signal with a specific output channel;

[0125] Convert the format of the time-domain signal in each audio channel to a predetermined audio output format;

[0126] Store the audio signal after format conversion into the corresponding storage medium and device, and generate independent audio files or data streams for each sound source signal.

[0127] During the above-mentioned speech output, associating each separated sound source signal with a specific output channel ensures the independent output of multiple sound sources. This not only helps to analyze each sound source separately but also can be used in audio devices with multi-channel output, facilitating subsequent processing or monitoring. Converting the format of the separated signal ensures that the output audio conforms to a predetermined format, enhancing the generality and compatibility of the file and making it convenient to use on different devices and platforms. Storing the separated signal as an independent audio file or data stream is convenient for subsequent archiving, analysis, and processing. At the same time, the generation of independent audio files facilitates post-analysis in multi-source scenarios and helps to achieve diverse applications.

[0128] In the above embodiments, the present invention collects signals at multiple different positions through a microphone array and uses time-frequency analysis to clearly represent each sound source in the time-frequency domain, thereby achieving accurate separation of sound sources. Sound source localization is performed based on direction estimation and time difference of arrival, enabling effective extraction and differentiation of the azimuth information of different sound sources. By generating a time-frequency mask, the region belonging to the target reactor sound source in the time-frequency domain is marked with high weight, while the region of the interfering sound source is marked with low weight, suppressing the energy of the interfering sound source, enhancing the signal-to-noise ratio of the target reactor sound source, and achieving a clearer separation effect. The time-frequency mask is applied to strengthen the time-frequency components of the target reactor sound source, thereby enhancing the signal quality of the target reactor sound source and improving the intelligibility of speech and audio quality. By performing inverse transformation on the separated time-frequency signal, it is restored to a continuous time-domain waveform, and the signals of different sound sources are output to different audio channels respectively, facilitating subsequent storage, processing, and application, meeting the requirements of multi-channel speech processing. Utilizing the spatial acquisition ability of the microphone array, it can be applied to much more complex multi-sound source environments, including scenarios with echo or background noise, improving the practicability and robustness in different application scenarios. In reactor monitoring, it can effectively separate the specific sound source signals generated by the reactor, extract useful signals from background noise and other interference signals, contribute to accurately analyzing the operating state of the reactor and promptly detecting abnormal situations, and further enhancing the reliability and accuracy of reactor monitoring.

[0129] Embodiment 2 of the present invention provides a sound source separation system for a microphone array to pick up speech signals, including:

[0130] A signal acquisition module for picking up speech signals emitted by a target reactor sound source and an interfering sound source in space using a microphone array;

[0131] A time-frequency transformation module for segmenting and performing Fourier transformation on the speech signals picked up by each microphone channel to obtain corresponding time-frequency domain signals;

[0132] A direction estimation module for calculating the time difference of arrival and phase difference of the sound source signals between each pair of microphones based on the time-frequency domain signals corresponding to each microphone channel, and combining with the geometric arrangement information of the microphone array to obtain the direction estimation result of each sound source in space;

[0133] A mask generation module for generating corresponding time-frequency masks for each sound source based on the direction estimation result;

[0134] A signal enhancement module for applying the time-frequency mask corresponding to each sound source to the time-frequency domain signals corresponding to each microphone channel to separate the time-frequency domain signals of each sound source;

[0135] The time-frequency domain inverse transform module is used to perform inverse short-time Fourier transform and segment splicing on the time-frequency domain signals of each separated sound source to obtain the continuous time-domain speech signal corresponding to each sound source;

[0136] The speech output module is used to output the continuous time-domain speech signal corresponding to each sound source to different audio output channels and store the speech signals of each sound source independently.

[0137] Embodiment 3 of the present invention provides a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method.

[0138] Embodiment 4 of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method are implemented.

[0139] The present disclosure may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0140] The computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punched card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as an instantaneous signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0141] The computer-readable program instructions described herein can be downloaded to various computing / processing devices from a computer-readable storage medium or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0142] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet connection using an Internet service provider). In some embodiments, by using the state information of the computer-readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the present disclosure.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific embodiments of the present invention, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered by the protection scope of the claims of the present invention.

Claims

1. A method for separating sound sources of a microphone array for picking up voice signals, characterized in that Including: Using a microphone array to pick up the voice signals emitted by the target reactor sound source and interference sound sources in space; Segmenting and performing Fourier transform on the voice signals picked up by each microphone channel to obtain corresponding time-frequency domain signals; Based on the time-frequency domain signals corresponding to each microphone channel, calculating the time difference of arrival and phase difference of the sound source signals between each pair of microphones, and combining with the geometric arrangement information of the microphone array to obtain the direction estimation result of each sound source in space; Generating a corresponding time-frequency mask for each sound source based on the direction estimation result; Applying the time-frequency mask corresponding to each sound source to the time-frequency domain signals corresponding to each microphone channel to separate the time-frequency domain signals of each sound source; Performing inverse short-time Fourier transform and segment splicing on the separated time-frequency domain signals of each sound source to obtain the continuous time-domain voice signal corresponding to each sound source; outputting the continuous time-domain voice signal corresponding to each sound source to different audio output channels and storing the voice signals of each sound source independently.

2. The method for separating sound sources of voice signals picked up by a microphone array according to claim 1, wherein: The step of using a microphone array to pick up the voice signals emitted by the target reactor sound source and interference sound sources in space includes: Arranging the microphone array according to the desired sound source localization accuracy and frequency range, adjusting the microphone spacing and positions, and performing clock synchronization processing on each microphone channel in the microphone array; The microphone array receives the time-domain signals emitted by the target reactor sound source and interference sound sources in space, and digitizes the time-domain signals received by each microphone channel at a preset sampling frequency to obtain discrete multi-channel digital voice signals.

3. The method for separating sound sources of voice signals picked up by a microphone array according to claim 1, wherein: The step of segmenting and performing Fourier transform on the voice signals picked up by each microphone channel to obtain corresponding time-frequency domain signals includes: Segmenting the voice signals picked up by each microphone channel through an overlapping window and dividing them into multiple short-time windows; Applying fast Fourier transform to the signals of each short-time window to obtain the corresponding time-frequency domain signals.

4. The method for separating sound sources of voice signals picked up by a microphone array according to claim 1, wherein: The step of calculating the time difference of arrival and phase difference of the sound source signals between each pair of microphones based on the time-frequency domain signals corresponding to each microphone channel, and combining with the geometric arrangement information of the microphone array to obtain the direction estimation result of each sound source in space includes: Calculating the time difference of arrival and phase difference of the sound source signals between each pair of microphones based on the time-frequency domain signals of the time-frequency points corresponding to each microphone channel; Calculating the relative distance difference between the sound source and each microphone of the microphone array according to the time difference of arrival, and combining with the geometric positions of the microphone array and the phase difference of the sound source signal arrival, deriving and calculating the relative azimuth angle and elevation angle of the sound source corresponding to each time-frequency point, converting the calculation results of the azimuth angle and elevation angle into spatial coordinate representations, and determining the relative azimuth information of the sound source; Weights are assigned according to the energy magnitude and signal-to-noise ratio of time-frequency points to perform weighted averaging on the relative azimuth information of different time-frequency points of each sound source, thereby obtaining the direction estimation result of each sound source in space.

5. The method for separating sound sources of a microphone array for picking up speech signals according to claim 4, characterized in that: The assignment of weights according to the energy magnitude and signal-to-noise ratio of time-frequency points is specifically as follows: Among them, W represents the weight of the time-frequency point, E represents the energy of the current time-frequency point, and E max represents the maximum energy among all time-frequency points, SNR represents the signal-to-noise ratio of the current time-frequency point, and SNT max represents the maximum signal-to-noise ratio among all time-frequency points, α represents the energy weight coefficient, and β represents the signal-to-noise ratio weight coefficient.

6. The method for separating sound sources of a microphone array for picking up speech signals according to claim 1, characterized in that: Generating a corresponding time-frequency mask for each sound source based on the direction estimation result includes: Based on the direction estimation result of each sound source in space, initializing the time-frequency mask weight of each sound source; Based on the initial time-frequency mask weight, according to the energy characteristics of the target reactor sound source in the time-frequency domain, dynamically adjust the mask weight through normalization, and obtain the time-frequency mask by means of binarization or ratio conversion; Perform smoothing processing on the time-frequency mask so that the mask shows continuous high-weight regions and low-weight regions in the time-frequency domain.

7. The method for separating sound sources of a microphone array for picking up speech signals according to claim 6, characterized in that: The dynamic adjustment of the mask weight through normalization based on the initial time-frequency mask weight according to the energy characteristics of the target reactor sound source in the time-frequency domain is specifically as follows: Among them, \(M(t,f)\) represents the initial time-frequency mask weight, \(M'(t,f)\) represents the adjusted time-frequency mask weight, \(\gamma\) represents the adjustment factor, \(E\) represents the energy at the current time-frequency point, and \(E max represents the maximum energy among all time-frequency points, \(t\) is time, and \(f\) is frequency.

8. The method for separating sound sources of a microphone array for picking up speech signals according to claim 1, characterized in that: Applying the time-frequency mask corresponding to each sound source to the time-frequency domain signal corresponding to each microphone channel to separate the time-frequency domain signal of each sound source includes: Multiplying the time-frequency mask corresponding to each sound source by the time-frequency domain signal corresponding to each microphone channel point by point to strengthen the target reactor sound source signal component in the high-weight region and simultaneously suppress the interfering sound source signal component in the low-weight region, and separating to obtain the time-frequency domain signal of each sound source after applying the mask; Perform noise estimation and threshold filtering on the time-frequency domain signal of each sound source after applying the mask to suppress the remaining noise components and obtain the enhanced time-frequency domain signal of each sound source; Perform gain control on the enhanced time-frequency domain signal of each sound source, and compress the amplitude of the signal within a preset range through dynamic range compression to obtain the finally separated time-frequency domain signal of each sound source.

9. The method for separating sound sources of a microphone array for picking up speech signals according to claim 1, characterized in that: Performing inverse short-time Fourier transform and segment splicing on the separated time-frequency domain signal of each sound source to obtain the continuous time-domain speech signal corresponding to each sound source includes: Performing inverse short-time Fourier transform on the time-frequency domain signal of each sound source to convert the complex spectrum in the time-frequency domain back to the signal segments in the time domain, performing overlapping splicing on the signal segments in the time domain, performing weighted averaging on adjacent segments according to their overlapping parts, and performing amplitude normalization processing to obtain the corresponding continuous time-domain speech signal.

10. The method for separating sound sources of a microphone array for picking up speech signals according to claim 1, characterized in that: Outputting the continuous time-domain speech signals corresponding to each sound source to different audio output channels and storing the speech signals of each sound source independently includes: Associating the continuous time-domain speech signals corresponding to each sound source with different audio output channels; Performing format conversion on the time-domain speech signals in each audio output channel, converting them into a predetermined audio output format, and storing the speech signals after format conversion in the corresponding storage medium and device to generate independent audio files or data streams for each sound source signal.

11. A sound source separation system for a microphone array to pick up voice signals, using the method according to any one of claims 1-10, characterized in that, The system includes: A signal acquisition module for picking up the speech signals emitted by the target reactor sound source and the interference sound source in space using a microphone array; A time-frequency transformation module for segmenting and performing Fourier transformation on the speech signals picked up by each microphone channel to obtain the corresponding time-frequency domain signals; A direction estimation module for calculating the time difference of arrival and phase difference of the sound source signals between each pair of microphones based on the time-frequency domain signals corresponding to each microphone channel, and combining with the geometric arrangement information of the microphone array to obtain the direction estimation result of each sound source in space; A mask generation module for generating a corresponding time-frequency mask for each sound source based on the direction estimation result; A signal enhancement module for applying the time-frequency mask corresponding to each sound source to the time-frequency domain signals corresponding to each microphone channel to separate the time-frequency domain signals of each sound source; An inverse time-frequency domain transformation module for performing inverse short-time Fourier transformation and segment splicing on the separated time-frequency domain signals of each sound source to obtain the continuous time-domain speech signals corresponding to each sound source; A speech output module for outputting the continuous time-domain speech signals corresponding to each sound source to different audio output channels and storing the speech signals of each sound source independently.

12. A terminal, including a processor and a storage medium; characterized in that: The storage medium is used for storing instructions; The processor is used to operate according to the instructions to execute the steps of the method according to any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the steps of the method according to any one of claims 1-10 are implemented.

Images