Overlapping bird song separation method based on phase matrix decomposition and related apparatus

Abstract

The present application relates to the technical field of audio signal processing, in particular to a method for separating overlapping bird songs based on phase matrix decomposition and related devices. The method comprises the following steps: obtaining an overlapping bird song audio signal to be processed; performing short-time Fourier transform on the overlapping bird song audio signal to obtain a time-frequency representation; extracting phase gradients from the time-frequency representation and constructing a two-dimensional phase gradient matrix based on the phase gradients. The present application mines and utilizes the phase gradient information in the time-frequency representation to construct a two-dimensional phase gradient matrix, which represents the phase variation law of the bird song signal in the time and frequency dimensions. Compared with the method of directly decomposing the time-frequency representation or the ordinary phase matrix, the phase gradient matrix can more effectively reflect the phase differences between different bird sound sources, because the frequency variation of the same bird sound source has certain continuity and regularity, while the frequency variation patterns of different bird sound sources usually have significant differences.

Description

Technical Field

[0001] This invention relates to the field of audio signal processing technology in ecological monitoring, specifically to a method and related apparatus for separating overlapping bird calls based on phase matrix decomposition. Background Technology

[0002] Separating overlapping bird calls has long been a challenge for researchers in ecological monitoring and wildlife surveys. When two or more birds call simultaneously, the audio signal captured by the acquisition equipment is a mixture of multiple bird call sources. Effective separation of this mixture is of great significance for species identification, population statistics, and ecological behavior analysis.

[0003] In existing technologies, signal separation methods based on amplitude spectrum estimation dominate. Independent component analysis decomposes mixed signals using the criterion of maximizing statistical independence, and nonnegative matrix decomposition decomposes the signal matrix into the product of two nonnegative matrices. These methods have shown some effectiveness in processing signals rich in amplitude information. However, amplitude spectrum estimation-based methods have an inherent drawback: during signal reconstruction, the recovery of phase information is often incomplete or inaccurate.

[0004] In existing technologies, the time-frequency representation X(f,t) obtained by short-time Fourier transform contains two parts: amplitude spectrum A(f,t) and phase spectrum θ(f,t), i.e., X(f,t) = A(f,t)·exp(j·θ(f,t)). Traditional methods typically utilize only the amplitude spectrum information during the separation process, while the phase spectrum information is obtained through approximate estimation after separation. This approximate estimation leads to phase distortion, resulting in significant waveform distortion in the reconstructed time-domain signal.

[0005] More importantly, bird calls exhibit significant time-varying characteristics and frequency modulation features, with their phase information carrying the frequency variation patterns and temporal envelope information of the calls. When phase information is missing or inaccurate, the intelligibility and naturalness of the separated bird call signals decrease significantly, failing to meet the requirements for subsequent species identification. Therefore, how to achieve effective separation of overlapping bird calls while maintaining phase consistency has become a pressing technical problem to be solved in this field. Summary of the Invention

[0006] This disclosure proposes an overlapping bird song separation method and related apparatus based on phase matrix decomposition, aiming to overcome at least one of the defects existing in the prior art.

[0007] To achieve the above objectives, the technical solution disclosed in this invention is as follows: According to one aspect of this disclosure, a method for separating overlapping bird calls based on phase matrix decomposition is provided, comprising the following steps: Acquire the overlapping bird song audio signal to be processed; Perform a short-time Fourier transform on the overlapping bird song audio signal to obtain the time-frequency representation X(f,t); The phase gradient is extracted from the time-frequency representation X(f,t), and a two-dimensional phase gradient matrix G is constructed based on the phase gradient; The two-dimensional phase gradient matrix G is subjected to singular value decomposition to obtain the left singular vector matrix U, the singular value matrix Σ and the right singular vector matrix V; The phase consistency index is calculated based on the singular value decomposition results, and the primary signal source and secondary signal source are determined accordingly. Based on the determination result, the time-frequency components of the main signal source and the time-frequency components of the secondary signal source are separated from the time-frequency representation X(f,t); The separated time-frequency components are subjected to inverse short-time Fourier transform to obtain the separated time-domain bird song signal.

[0008] Furthermore, the step of extracting the phase gradient from the time-frequency representation X(f,t) to construct the two-dimensional phase gradient matrix G includes: Calculate the phase difference between adjacent time frames in the time-frequency representation X(f,t) to obtain the time-dimensional phase gradient Δθt; Calculate the phase difference between adjacent frequency bins in the time-frequency representation X(f,t) to obtain the frequency-dimensional phase gradient Δθf; The time-dimensional phase gradient Δθt and the frequency-dimensional phase gradient Δθf are organized into a two-dimensional phase gradient matrix G, wherein the time-dimensional phase gradient constitutes the columns of the two-dimensional phase gradient matrix, and the frequency-dimensional phase gradient constitutes the rows of the two-dimensional phase gradient matrix.

[0009] Furthermore, the step of performing singular value decomposition on the two-dimensional phase gradient matrix G includes: Perform singular value decomposition on the two-dimensional phase gradient matrix G to obtain G=U·Σ·V^T; The primary and secondary signal components are determined based on the magnitude of the singular values. The singular vectors corresponding to the top N largest singular values ​​are retained, where N is a positive integer greater than or equal to 2. The retained singular vectors are reassembled into a low-rank approximate matrix.

[0010] Furthermore, the step of calculating the phase consistency index based on the singular value decomposition result, and determining the primary signal source and secondary signal source accordingly, includes: Calculate the phase consistency index γi=|(U_i)^T·V_i| for each singular value; A phase consistency threshold γ_th is set. The signal components corresponding to the singular values ​​of the phase consistency index γi that are greater than the threshold γ_th are determined as the main signal source, and the signal components corresponding to the singular values ​​of the phase consistency index γi that are less than the threshold γ_th are determined as the secondary signal source or noise components.

[0011] Furthermore, the step of separating the time-frequency components of the main signal source from the time-frequency representation X(f,t) based on the determination result includes: Based on the reconstruction result of the low-rank approximation matrix, the time-frequency components of the main signal source are separated from the original time-frequency representation X(f,t); The remaining time-frequency components are identified as secondary signal sources or noise components; The time-frequency components of the main signal source and the time-frequency components of the secondary signal source are output separately.

[0012] Furthermore, prior to performing singular value decomposition on the two-dimensional phase gradient matrix G, the method further includes: The two-dimensional phase gradient matrix G is normalized. And / or perform mean-removal processing on the two-dimensional phase gradient matrix G; And / or perform frequency band filtering on the two-dimensional phase gradient matrix G, retaining only phase information within the frequency band range of 200Hz to 15000Hz.

[0013] Furthermore, prior to the step of acquiring the overlapping bird song audio signal to be processed, the method further includes: A pre-trained sound activity detection model is used to detect the input audio and determine whether there are overlapping bird calls. The phase matrix decomposition and separation process is triggered when overlapping bird call signals are detected. And / or pre-filter the input audio to remove frequency components that are outside the range of birdsong frequencies; And / or use an automatic gain control module to adjust the dynamic range of the input audio signal to a preset level.

[0014] Furthermore, the step of performing inverse short-time Fourier transform on the separated time-frequency components to obtain the separated time-domain bird song signal includes: The same window function as used during framing is applied to the separated time-frequency components for overlapping and addition. The separated time-frequency components are converted back from the time-frequency domain to the time domain using the inverse short-time Fourier transform; And / or perform post-filtering on the separated time-domain bird call signal.

[0015] According to another aspect of this disclosure, an overlapping bird song separation system based on phase matrix decomposition is provided for implementing the overlapping bird song separation method based on phase matrix decomposition as described above, comprising: The audio signal acquisition module is used to acquire overlapping bird song audio signals to be processed; The time-frequency transformation module is used to perform a short-time Fourier transform on the overlapping bird song audio signal to obtain a time-frequency representation; A phase gradient extraction module is used to extract the phase gradient from the time-frequency representation and construct a two-dimensional phase gradient matrix; A singular value decomposition module is used to perform singular value decomposition on the two-dimensional phase gradient matrix. The signal separation module is used to calculate the phase consistency index based on the singular value decomposition results and perform signal separation; The time-domain reconstruction module is used to perform inverse short-time Fourier transform on the separated time-frequency components; The audio output module is used to output the separated time-domain bird song signal; The preprocessor module is used to perform sound activity detection, pre-filtering, and automatic gain control on the input audio.

[0016] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the overlapping bird calls separation method based on phase matrix decomposition as described above.

[0017] According to another aspect of this disclosure, an electronic product is provided, including a processor, a memory, an audio acquisition module, and a display module. The audio acquisition module includes at least one microphone for acquiring overlapping bird song audio signals to be processed. The processor is used to acquire the overlapping bird song audio signals, perform a short-time Fourier transform on the overlapping bird song audio signals to obtain a time-frequency representation X(f,t), extract phase gradients from the time-frequency representation X(f,t) to construct a two-dimensional phase gradient matrix G, perform singular value decomposition on the two-dimensional phase gradient matrix G, calculate a phase consistency index based on the singular value decomposition result, determine the primary signal source and the secondary signal source accordingly, and separate the time-frequency components of the primary signal source and the secondary signal source from the time-frequency representation X(f,t) based on the determination result. The separated time-frequency components are then subjected to inverse short-time Fourier transforms to obtain separated time-domain bird song signals. The display module is used to output and display the analysis results of the separated time-domain bird song signals.

[0018] The beneficial effects of this invention are: This invention constructs a two-dimensional phase gradient matrix by mining and utilizing phase gradient information in the time-frequency representation. This matrix characterizes the phase variation patterns of bird song signals in both the time and frequency dimensions. Compared to methods that directly decompose the time-frequency representation or ordinary phase matrix, the phase gradient matrix can more effectively reflect the phase differences between different bird song sources. This is because the frequency variation of the same bird song source has a certain continuity and regularity, while the frequency variation patterns between different bird song sources usually show significant differences.

[0019] This invention performs singular value decomposition on a two-dimensional phase gradient matrix and calculates a phase consistency index based on the decomposition results. This index can quantify the difference in phase structure between the primary and secondary signal sources. Since the singular vectors corresponding to the primary signal source have high phase consistency in both the time and frequency dimensions, while the singular vectors corresponding to the secondary signal source or noise components have low phase consistency, the primary and secondary signal sources can be accurately identified by setting a phase consistency threshold, thus achieving effective signal separation.

[0020] This invention fully preserves the phase information of the main signal source during signal separation, avoiding signal distortion caused by inaccurate phase estimation in traditional methods. The separated bird calls retain the temporal envelope characteristics and frequency modulation patterns of the original signals, exhibiting good intelligibility and naturalness, which is beneficial for subsequent species identification and individual differentiation.

[0021] The iterative optimization mechanism in this invention can further improve the signal separation quality by iteratively optimizing the phase consistency index until convergence. Experiments show that when using the method of this invention to process overlapping bird song signals, the separation accuracy of the main signal source can reach over 92%, the signal-to-noise ratio can be improved by 8 to 15 dB, and the phase distortion rate can be reduced to below 5%, which is significantly better than existing methods based on amplitude spectrum estimation.

[0022] The method of this invention relies solely on phase gradient information for signal decomposition during processing, eliminating the need for complex statistical modeling of the amplitude spectrum. Therefore, its computational complexity is moderate, making it easy to implement in real-time on embedded platforms. Furthermore, the sound activity detection, automatic gain control, and pre-filtering in the preprocessing module effectively improve the quality of the input signal, providing a solid data foundation for subsequent phase gradient matrix construction and singular value decomposition.

[0023] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the overlapping bird calls separation method based on phase matrix decomposition.

[0025] Figure 2 This is a time-frequency representation of the overlapping bird song audio signal after a short-time Fourier transform.

[0026] Figure 3 This is a diagram showing the distribution of singular values ​​after singular value decomposition of the two-dimensional phase gradient matrix.

[0027] Figure 4 The diagram shows the time-frequency representation of the separated main signal source.

[0028] Figure 5 This is a comparison diagram of the signal waveforms before and after separation.

[0029] Figure 6 This is a schematic diagram showing the relationship between the amplitude spectrum and phase spectrum of the STFT.

[0030] Figure 7 A time synchronization comparison diagram of the separated signal waveforms.

[0031] Figure 8 The convergence curve is used for iterative optimization. Detailed Implementation

[0032] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0033] The term "comprising," and any variations thereof, used in the specification and claims of this application, is intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus. Furthermore, the use of "and / or" in the specification and claims indicates at least one of the connected objects, such as A and / or B, indicating the inclusion of A alone, B alone, or both A and B.

[0034] In embodiments of the present invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0035] The present invention provides the following preferred embodiments: In one embodiment, combined Figure 1 and Figure 2This paper describes the basic process of an overlapping bird call separation method based on phase matrix decomposition. This embodiment is applicable to separating individual bird call signals from field recordings containing simultaneous calls of two or more birds, and is suitable for various application scenarios such as ecological monitoring, wildlife surveys, and nature reserve management.

[0036] This embodiment first uses a high-sensitivity directional microphone to collect overlapping bird song audio signals in a natural forest. The sampling frequency is set to 44100Hz, the quantization precision is 16-bit, and the recording duration is 30 seconds. The collection environment is during the peak bird activity period in the early morning, approximately between 5:00 AM and 7:00 AM. At this time, the temperature is suitable, the air humidity is high, and the background noise is low, making it the most active period for birds. During this collection period, there are at least three bird calls simultaneously, including mixed calls from tits, corvids, and passerines. The original audio signal obtained is denoted as x(t), which is a mono signal.

[0037] Under these acquisition conditions, the background ambient noise was approximately 35 dB, mainly originating from wind, leaf rustling, and distant flowing water. The microphone was approximately 3 meters from the nearest bird call source and approximately 15 meters from the farthest. Attenuation effects due to foliage obstruction occurred between the microphone and each sound source, resulting in a signal-to-noise ratio of approximately 10 to 15 dB. Pre-filtering can further reduce background noise interference during subsequent processing.

[0038] After obtaining the overlapping bird song audio signal x(t), a short-time Fourier transform is performed on it. A Hanning window is used as the analysis window function, with a window length of 2048 sampling points, a frame shift of 512 sampling points, and an overlap rate of 75%. A short-time Fourier transform is performed on each frame of the signal to obtain the complex time-frequency representation X(f,t) = A(f,t)·exp(j·θ(f,t)), where A(f,t) represents the amplitude spectrum, θ(f,t) represents the phase spectrum, f represents the frequency index, and t represents the time frame index.

[0039] The resulting time-frequency representation is a two-dimensional matrix with dimensions F×T, where F is the number of frequency bins (1024) and T is the number of time frames (approximately 2580 for a 30-second audio signal). This time-frequency representation contains complete spectral and phase information of the original overlapping bird song audio signal, providing a data foundation for the subsequent construction of the phase gradient matrix.

[0040] like Figure 2 As shown, the time-frequency representation X(f,t) is displayed in the form of a heatmap, with the horizontal axis representing time and the vertical axis representing frequency. The color depth represents the amplitude energy. From Figure 2 It can be seen that the time-frequency representation of overlapping bird song signals presents a complex energy distribution, with the spectra of multiple bird song sources intertwined in the time and frequency dimensions, making them difficult to distinguish directly. Figure 2 The images show obvious frequency modulation trajectories, which correspond to different bird call patterns and exhibit different sweep speeds and frequency modulation characteristics.

[0041] Subsequently, phase information is extracted from the time-frequency representation X(f,t) to construct a phase gradient matrix. Specifically, the phase difference between adjacent time frames is calculated as the time-dimensional phase gradient: Δθt(f,t) = θ(f,t+1) - θ(f,t). The phase difference between adjacent frequency bins is calculated as the frequency-dimensional phase gradient: Δθf(f,t) = θ(f+1,t) - θ(f,t). The time-dimensional and frequency-dimensional phase gradients are organized into a two-dimensional phase gradient matrix G. This matrix fully records the phase change pattern of the original overlapping bird song signal in the time-frequency domain, and its dimension is the same as the time-frequency representation, i.e., F×T. The construction of the two-dimensional phase gradient matrix is ​​one of the core steps of this invention. By mining the phase change patterns in the time and frequency dimensions, the phase characteristics of different bird song sources can be effectively distinguished.

[0042] In one embodiment, combined Figure 3 This document details the specific process of singular value decomposition (SVD) of a two-dimensional phase gradient matrix and its application in signal separation. This embodiment describes in detail the mathematical principles of SVD, the decomposition process, and the calculation method for the phase consistency index.

[0043] This embodiment performs a complete singular value decomposition on a two-dimensional phase gradient matrix G of dimension F×T. The singular value decomposition expression is G=U·Σ·V^T, where U is the left singular vector matrix of dimension F×F, Σ is the singular value matrix of dimension F×T (a diagonal matrix), and V is the right singular vector matrix of dimension T×T. The diagonal elements of the singular value matrix Σ, i.e., the singular values, are arranged in descending order, denoted as σ1≥σ2≥σ3≥...≥σr, where r is the rank of the phase gradient matrix G.

[0044] Combination Figure 3 The singular value distribution plot shows the arrangement of singular values ​​by magnitude. From Figure 3 It can be seen that the first few singular values ​​(σ1, σ2, σ3) are significantly larger than the subsequent singular values, forming a clear step-like distribution. This distribution characteristic indicates that the main signal source in the overlapping bird song signal has a significant energy concentration characteristic in the phase gradient domain, while the singular values ​​corresponding to the secondary signal sources and noise components are smaller and have a relatively flat distribution. Figure 3 It can also be observed that the energy of the main signal source region (corresponding to the first three singular values) accounts for the majority of the total energy, while the energy distribution of the secondary signal source region and the noise region is relatively uniform.

[0045] Based on the above decomposition results, this embodiment sets a singular value threshold Th and a minimum number of signal sources Nmin, retaining the singular vectors corresponding to the top N largest singular values, where N is the larger of the number of components greater than Th in the singular value sequence and Nmin, and N is not less than 2. By retaining the top N singular values ​​and their corresponding left and right singular vectors, the phase gradient matrix G can be approximately reconstructed into a low-rank approximation matrix: , where σ i For the i-th singular value, u i For the corresponding i-th left singular vector, v i T Let G_lowrank be the transpose of the i-th right singular vector, and N be the number of singular values ​​retained. The low-rank approximation matrix G_lowrank characterizes the main signal source features in the overlapping bird song mixture signal, while the remaining singular values ​​correspond to secondary signal sources or noise components. Based on this, the phase consistency index corresponding to each singular value is calculated: γi=|(U_i)^T·V_i|, which reflects the degree of phase correlation between the left singular vector U_i and the right singular vector V_i. A phase consistency threshold γ_th is set. When γi>γ_th, the corresponding signal component is identified as the main signal source; when γi<γ_th, the corresponding signal component is identified as a secondary signal source or noise component.

[0046] The method for separating the time-frequency component X1(f,t) of the main signal source from the original time-frequency representation X(f,t) is as follows: the phase information in X(f,t) is replaced with the phase of the corresponding element in the low-rank approximation matrix G_lowrank, while keeping the original amplitude spectrum A(f,t) unchanged. Specifically, for each frequency f and time frame t, the reconstructed phase angle θ_recon(f,t)=arg(G_lowrank(f,t)) is calculated, and then the separated main signal source time-frequency representation X1(f,t) is set to A(f,t)·exp(j·θ_recon(f,t)). The remaining time-frequency component X2(f,t) is calculated by X2(f,t)=X(f,t)-X1(f,t), representing the mixed component of secondary signal sources and noise. The above separation process ensures that the main signal source retains the same amplitude characteristics as the original mixed signal, while the phase information is dominated by the phase structure of the main signal source, effectively avoiding the signal distortion problem caused by amplitude and phase mismatch in traditional methods.

[0047] This embodiment also analyzes the impact of different threshold settings on the separation effect. Experiments show that the best separation effect can be obtained when the phase consistency threshold γ_th is set between 0.7 and 0.8. A threshold that is too low may cause some secondary signal sources to be misidentified as primary signal sources, while a threshold that is too high may cause some primary signal sources to be misidentified as secondary signal sources or noise. In practical applications, the phase consistency threshold can be appropriately adjusted according to the specific signal characteristics and separation requirements.

[0048] In one embodiment, combined Figure 3 This document details the preprocessing steps performed on the two-dimensional phase gradient matrix before singular value decomposition to improve signal separation. This embodiment specifically describes the principles, implementation methods, and practical effects of three preprocessing operations: normalization, mean removal, and frequency band filtering.

[0049] This embodiment adds three preprocessing operations to Embodiment 2: normalization, mean removal, and frequency band filtering. These preprocessing operations can eliminate the interference of phase drift and amplitude differences on the singular value decomposition results, improving the identification accuracy of the main signal source. The addition of the preprocessing process is one of the important improvements of this invention, which can significantly improve the separation effect under low signal-to-noise ratio conditions.

[0050] The specific steps of normalization are as follows: Calculate the sum of the magnitudes of all elements in the phase gradient matrix G, and then calculate the normalization coefficients: In this equation, F represents the total number of frequency bins, T represents the total number of time frames, and |G(f,t)| represents the magnitude of the phase gradient matrix element G(f,t). Multiplying each element of the phase gradient matrix G by the normalization coefficient λ yields the normalized phase gradient matrix G_norm = G·λ. Since the elements of the phase gradient matrix are complex numbers, their magnitude represents the intensity of the phase change. The purpose of normalization is to adjust the overall energy scale of the phase gradient matrix, making different frequency components comparable in subsequent singular value decomposition. Normalization eliminates amplitude differences between different frequency components, allowing the magnitude of the singular values ​​to more accurately reflect the relative importance of each frequency component in the signal composition.

[0051] The specific steps for mean removal are as follows: For each row of the phase gradient matrix G_norm, i.e., each frequency bin, calculate the mean vector μ(f) of all elements in that row: Where T is the total number of time frames, and G_norm is the normalized phase gradient matrix. Then, the mean vector of each row in the original phase gradient matrix is ​​subtracted to obtain the mean-reduced phase gradient matrix G_dc(f,t) = G_norm(f,t) - μ(f). The purpose of mean reduction is to eliminate the influence of the DC component caused by phase drift. When bird calls experience frequency drift during recording, linear or nonlinear phase shifts accumulate in the phase gradient matrix. This shift manifests as large low-frequency singular values ​​in singular value decomposition, interfering with the correct separation of the main signal source. Mean reduction effectively suppresses this interference, making the singular value distribution more clearly reflect the actual signal source composition.

[0052] The specific steps of frequency band filtering are as follows: Set the lower limit frequency fl = 200Hz and the upper limit frequency fu = 15000Hz for the bird song frequency band. Set the frequency components in the time-frequency representation X(f,t) whose actual frequencies are outside the range of fl to zero, retaining only the phase information within the preset frequency band to eliminate environmental noise and low-frequency interference. Bird calls typically range from 200Hz to 15000Hz. Frequency components outside this range are mainly composed of wind noise, machine noise, or other animal sounds. Frequency band filtering can significantly improve the targeting of signal separation and reduce the interference of irrelevant noise on the singular value decomposition results. Experimental data shows that after frequency band filtering, the rank of the phase gradient matrix decreases from approximately 50 to approximately 30, indicating that irrelevant noise components are effectively suppressed.

[0053] The preprocessed phase gradient matrix is ​​then fed into the singular value decomposition (SVD) module for further processing. The combination of preprocessing and SVD can further improve the accuracy of overlapping bird calls separation. Experimental results show that, compared with the unprocessed scheme, the preprocessing scheme can improve the separation accuracy by about 5 to 8 percentage points, especially under low signal-to-noise ratio conditions, where the improvement is more significant.

[0054] In one embodiment, combined Figure 4 , Figure 5 and Figure 6 This document details the specific process of iterative optimization and its application in improving signal separation quality. Specifically, it describes the triggering conditions, iteration process, convergence criteria, and post-filtering methods for iterative optimization.

[0055] When the separated time-domain bird call signal does not meet the preset signal-to-noise ratio (SNR) threshold or signal quality evaluation index, this embodiment uses an iterative optimization method to improve the separation result. The trigger condition for iterative optimization is: when the ratio of the energy of the time-frequency component X1 of the separated main signal source to the energy of the time-frequency component X2 of the secondary signal source is lower than the preset SNR threshold SNR_th, the iterative optimization process is started. The setting of the SNR threshold SNR_th needs to be adjusted according to the specific application scenario and separation requirements. Generally speaking, a better separation effect can be obtained when SNR_th is set between 10dB and 15dB.

[0056] Combination Figure 4 The time-frequency representation of the separated main signal source is displayed in the form of a heatmap. From Figure 4 As can be seen, the frequency modulation trajectories of each bird song source are clearly distinguishable in the separated time-frequency representation, which contrasts sharply with the complex spectral distribution of the original mixed signal. Figure 4 Three clear frequency modulation trajectories can be seen, corresponding to the call patterns of three different birds. The sweep speed, modulation frequency, and time distribution of these trajectories are significantly different, indicating that the method of the present invention can effectively separate overlapping bird call signals and preserve the time-frequency characteristics of each signal source.

[0057] The iterative optimization process is as follows: In the k-th iteration, based on the current separation results X1^k and X2^k, the phase gradient matrix is ​​recalculated: , where X1 k (f,t) represents the time-frequency representation of the main signal source obtained in the k-th iteration, and arg represents the complex phase angle, i.e., a new phase gradient matrix is ​​constructed using the phase angle of the current main signal source time-frequency representation. Singular value decomposition is performed on G_iter, retaining the singular vectors corresponding to the top N largest singular values, and reconstructing the low-rank approximation matrix G_iter_lowrank. The time-frequency representation of the main signal source is recalculated based on G_iter_lowrank. A1 k Let X2 be the amplitude spectrum of the main signal source at the k-th iteration, and G_iter_lowrank be the phase gradient matrix reconstructed at the k-th iteration. k+1 =X k -X1 k+1 Evaluation of X1 k+1 If the signal quality index meets the convergence condition, the iteration stops; otherwise, it continues to the next iteration.

[0058] The convergence condition is defined as the relative change in singular values ​​between two consecutive iterations being less than a preset threshold ε. |σ1 k+1 -σ1k | / σ1 k The iterations are considered converged when the sum of the first singular values ​​between two consecutive iterations is less than 1%, or when the number of iterations reaches the preset maximum number of iterations, K_max. The preset threshold ε is usually set to 0.01, meaning that the iteration is considered converged when the change in the first singular value between two consecutive iterations is less than one percent. The maximum number of iterations, K_max, is usually set to 10 to 20 to prevent excessively long iterations from wasting computational resources.

[0059] Combination Figure 5 The comparison of signal waveforms before and after separation illustrates the difference in signal quality before and after iterative optimization. From Figure 5 It can be seen that after iterative optimization, the waveform of the separated time-domain bird song signal is clearer, the noise residue is significantly reduced, and the signal-to-noise ratio of the signal is significantly improved. Figure 5 The upper subplot shows the time-domain waveform of the original mixed signal, where multiple bird calls overlap and are difficult to distinguish. The lower subplot shows the waveforms of the individual signals obtained after iterative optimization, where each signal has a clear outline and distinct boundaries. Experiments show that for typical overlapping bird calls, the signal separation quality can reach convergence after 3 to 5 iterations, with a signal-to-noise ratio improvement of approximately 2 to 5 dB compared to a single separation.

[0060] This embodiment also illustrates the specific steps of the post-filtering process. The post-filtering process uses the same Hanning window function as during framing to overlap and add the separated time-frequency representations, smoothing signal boundaries and eliminating spectral leakage caused by framing. Specifically, for the separated main signal source time-frequency representation X1(f,t), each frame is weighted using a Hanning window w(n), and the weighted frame signal is calculated: Where N is the number of points in the Discrete Fourier Transform (equal to the window length), k is the frequency index, m is the time index, n is the frame index, and X1(k,n) are the complex coefficients in the time-frequency matrix with frequency index k and time frame n. The kernel function is the inverse discrete Fourier transform. The overlapping portions of adjacent frames are then added to obtain the time-domain signal y1(t). Post-filtering can further remove residual noise from the separated signal using a bandpass filter. The passband range of the bandpass filter is set to 200Hz to 15000Hz, the stopband attenuation is not less than 40dB, and the passband ripple is not greater than 3dB.

[0061] In another embodiment, combined with Figure 7 This document details the application of sound activity detection and automatic gain control in the preprocessor module. Specifically, it describes the architecture of the sound activity detection model, the automatic gain control algorithm, and the detailed implementation of the pre-filtering process.

[0062] This embodiment utilizes a deep neural network-based sound activity detection model to perform real-time detection of input audio, determining the presence and degree of overlapping bird calls. The sound activity detection model employs a cascaded architecture of a convolutional neural network and a long short-term memory network. The input is a Mel-spectrum graph, and the output is the sound activity probability and overlap estimate for each time frame. When the detected overlap exceeds a preset threshold, the phase matrix decomposition separation process described in this invention is triggered. When only a single bird call signal or no bird call signal is detected, the separation process is skipped, and the original signal is directly output to reduce computational resource consumption and signal processing latency. The training data for the sound activity detection model includes 10,000 recordings of wild birds, covering call samples from 200 bird species. The model achieves a sound activity detection accuracy of 96.3% on the validation set, with a mean absolute error of 0.08 for overlap estimation.

[0063] Combination Figure 7 The time synchronization comparison chart of the separated signal waveforms demonstrates the working effect of the sound activity detection module. From Figure 7 As can be seen, the orange vertical lines mark key synchronization points. These points correspond to important acoustic events identified by the sound activity detection model, such as the start and end times of bird calls, and the peak times during the calls. By marking these synchronization points on the separated signal waveform, the consistency between the separation results and the original acoustic events can be verified, thereby evaluating the effectiveness of the separation algorithm.

[0064] The specific steps of the automatic gain control module are as follows: Calculate the short-time energy sequence E(n) of the input audio signal x(t): Where N is the window length, H is the frame shift length, x[m] is the discrete audio signal, w[m] is the window function, and n is the frame index. The effective level of the signal is calculated based on the short-time energy sequence: Where M is the total number of frames and E[n] is the short-time energy sequence. The effective value level is compared with the preset target level RMS_target to calculate the gain coefficient: G = RMS_target / RMS. If the gain coefficient G is greater than the preset maximum gain limit G_max, then G is set to G_max to prevent over-amplification; if the gain coefficient G is less than the preset minimum gain limit G_min, then G is set to G_min to prevent excessive attenuation. Multiplying the input audio signal by the gain coefficient G yields the dynamically adjusted audio signal x_agc(t) = G·x(t). Through automatic gain control, the dynamic range of the audio signal is adjusted to a preset level, significantly improving the numerical stability of the subsequent short-time Fourier transform and making the construction of the phase gradient matrix more stable and reliable. The preset target level RMS_target is usually set between 50% and 70% of full scale, the maximum gain limit G_max is generally no more than 20dB, and the minimum gain limit G_min is no less than -10dB.

[0065] The pre-filtering process employs a cascaded structure of a second-order high-pass filter and a second-order low-pass filter to form a bandpass filter, limiting the frequency band of the input audio signal. The high-pass filter has a cutoff frequency of 200Hz to filter out low-frequency environmental noise and power frequency interference, while the low-pass filter has a cutoff frequency of 15000Hz to filter out high-frequency sampling noise and unwanted signals beyond the bird call frequency range. The bandpass filter is designed with a passband gain of 0dB and a stopband attenuation of no less than 30dB. The filtered signal is then sent to the automatic gain control module for further processing. Experiments show that after processing by the pre-processor module, the singular values ​​of the phase gradient matrix of the audio signal are more concentrated in the first few largest singular values, which is beneficial for the identification and separation of the main signal source. Without pre-processing, the first singular value of the phase gradient matrix typically accounts for only about 30% to 40% of the total singular value energy, while after pre-processing, this proportion can be increased to about 50% to 65%, indicating a significant increase in the energy concentration of the main signal source and a corresponding reduction in the difficulty of separation.

[0066] In another embodiment, the deployment and application of the method of the present invention in a practical ecological monitoring device are described in detail. This embodiment covers various aspects such as device hardware configuration, software implementation, algorithm optimization, and performance testing.

[0067] This embodiment integrates the overlapping bird call separation method based on phase matrix decomposition described in this invention into a portable ecological monitoring device. The device includes a high-sensitivity microphone array, an embedded processor, and a wireless communication module. The microphone array adopts a four-element cross array structure to collect overlapping bird call signals in space, with a sampling frequency set to 48000Hz and a quantization accuracy of 24 bits. The embedded processor uses an ARM Cortex-A series processor with a main frequency of 1.2GHz, equipped with 512MB of RAM and 4GB of storage space. The device is battery-powered, with a battery life of over 8 hours, making it suitable for long-term deployment in outdoor environments.

[0068] The device's workflow is as follows: A microphone array collects ambient sound, and an analog-to-digital converter converts the analog audio signal into a digital audio signal. The digital audio signal first enters the preprocessor module for sound activity detection. When overlapping bird call signals are detected, the preprocessor module outputs a trigger signal to initiate the phase matrix decomposition and separation process. The embedded processor loads the separation algorithm program of this invention, performing a series of processes on the overlapping bird call audio signals, including short-time Fourier transform, phase gradient extraction, singular value decomposition, and signal separation. Finally, it outputs the time-domain waveforms of each independent bird call signal after separation. The separation results are transmitted in real time to a remote monitoring server via a wireless communication module. The server identifies the species and distinguishes individual birds from the received bird call signals. The device supports both 4G and Wi-Fi wireless communication modes, with a transmission latency of less than 200ms and a total processing time of no more than 150ms for a single separation process, meeting the requirements of real-time monitoring.

[0069] The algorithm executed on the device side has been optimized for embedded platforms: the window length of the short-time Fourier transform has been reduced from 2048 sampling points to 1024 sampling points to reduce computational complexity; fixed-point arithmetic has been used instead of floating-point arithmetic, and the DSP extended instruction set of the ARM processor has been utilized to accelerate matrix operations; singular value decomposition adopts a block-based iterative algorithm, processing a 64×64 sub-matrix each time, reducing memory usage to about one-third of the original; the signal separation results are output at an 8kHz sampling rate, further reducing data transmission bandwidth while maintaining intelligibility. Experiments show that the optimized algorithm has a single-frame processing time of 12ms on the embedded platform, which meets the requirements of real-time processing, and the signal separation quality is no more than 5% different from the original algorithm on the PC platform.

[0070] This embodiment also verifies the applicability of the method of the present invention in different bird combination scenarios. Overlap separation experiments were conducted on the calls of birds from three families: Passeriformes, Titidae, and Corvidae. Each experiment included a mixture of calls from two different bird species and a mixture of calls from three birds. The results show that for signals from two overlapping birds, the separation accuracy of the method of the present invention reached 92.3%, with a signal-to-noise ratio improvement of 8.7 dB; for signals from three overlapping birds, the separation accuracy was 84.6%, with a signal-to-noise ratio improvement of 6.2 dB. The separation accuracy is defined as the proportion of time frames in which the correlation coefficient between the spectrum of each separated signal source and the spectrum of the corresponding original independent signal source exceeds 0.8. The experimental results verify the effectiveness and robustness of the method of the present invention in various overlap scenarios.

[0071] In another embodiment, combined with Figure 8 This paper details the extended application of the method of the present invention in multi-channel audio signal scenarios and the verification of its iterative optimization convergence characteristics. This embodiment describes in detail the multi-channel joint decomposition strategy and the analysis method of the iterative convergence curve.

[0072] When the input signal is multi-channel audio, such as a dual-channel stereo or a four-channel microphone array, the method of this invention processes the singular value decomposition of the multi-channel phase gradient matrix through channel expansion. Specifically, for an input signal with C channels, each channel undergoes a short-time Fourier transform to obtain a time-frequency representation Xc(f,t), where c = 1 to C. The time-frequency representations of each channel are concatenated along the channel dimension to form a three-dimensional tensor X(F,T,C). Then, the phase angle is extracted from the time-frequency representation of each channel to construct the phase gradient matrix Gc(f,t). The construction method of the multi-channel phase gradient matrix is ​​similar to that of the single channel, with the phase gradient matrix Gc of each channel having a dimension of F×T, ultimately resulting in C phase gradient matrices.

[0073] Combination Figure 8 The convergence curve of the iterative optimization illustrates the relationship between the number of iterations and the signal separation quality index. From Figure 8 As can be seen, with the increase of the number of iterations, the signal-to-noise ratio (SNR) gradually improves from the initial approximately 5 dB, eventually converging to approximately 18 dB, exhibiting a typical logarithmic curve shape during the convergence process. The iterative convergence curve clearly demonstrates the effectiveness of the iterative optimization module in the method of this invention. The SNR improvement is most significant in the first 10 iterations, and then gradually stabilizes. When the number of iterations reaches 20, the improvement in SNR is close to saturation, and the benefit of further iterations is negligible. Therefore, in practical applications, the maximum number of iterations K_max can be set to 15 to 20 to achieve a balance between separation quality and computational efficiency.

[0074] The singular value decomposition (SVD) of the multi-channel phase gradient matrix employs a joint decomposition strategy: the phase gradient matrices of the C channels are stacked along the channel dimension to form an augmented phase gradient matrix P_aug with dimensions F×(T·C). SVD is then performed on P_aug to obtain a joint singular value matrix S_aug. Larger singular values ​​correspond to signal components representing the common main signal source across all channels, while smaller singular values ​​correspond to channel-specific noise or interference. By analyzing the distribution of singular values ​​along the channel dimension, spatially shared signals and spatially unique signals can be distinguished, thus achieving bird call signal separation under multi-channel conditions. Experiments show that in a four-channel scenario, the separation accuracy using the joint decomposition strategy is approximately 8 to 12 percentage points higher than single-channel processing, indicating that utilizing multi-channel information can significantly improve separation quality.

[0075] The reconstruction process of the separated results is similar to that of a single channel: the low-rank approximation matrix of each channel is reconstructed based on the joint singular value decomposition results; the reconstructed phase of each channel is combined with the original amplitude spectrum to obtain the time-frequency representation of each channel after separation; then, inverse short-time Fourier transform is performed on the time-frequency representation of each channel to obtain the time-domain bird song signal after separation of each channel. The separation results of each channel can be further used for subsequent processing such as spatial localization and sound source tracking.

[0076] In another embodiment, a detailed explanation of the integration of the method of the present invention with an existing automatic bird identification system is provided. This embodiment covers the architecture design, performance evaluation, and practical deployment scheme of the integrated system.

[0077] The output of the method of this invention can be used as input to an automatic bird identification system, enabling automatic identification of bird species from overlapping recordings. The workflow of the integrated scheme is as follows: The overlapping bird song audio signals are first processed by the phase matrix decomposition and separation method described in this invention, outputting the time-domain waveforms of each independent bird song signal; then, acoustic features are extracted from each independent bird song signal, including Mel-spectrum features, spectral centroid, zero-crossing rate, and fundamental frequency trajectory; the extracted acoustic features are input into a pre-trained deep learning model for bird species identification, which is based on a convolutional neural network architecture, and the training dataset contains 200,000 labeled song samples of 500 bird species; the model outputs the bird species identification result and confidence level corresponding to each independent bird song signal.

[0078] Performance evaluation results of the integrated system show that in overlapping scenarios involving the simultaneous calls of three birds, the species identification accuracy after separation and re-identification is 87.2%, while the accuracy of directly identifying species from the original overlapping signals is only 43.5%. This comparison demonstrates that pre-separating signals using the method of this invention can significantly reduce the interference of overlapping signals on the species identification algorithm, thereby improving the reliability of automatic identification. In scenarios with two overlapping birds, the accuracy of re-identification after separation reaches 93.6%, a substantial improvement over the 58.2% accuracy of direct identification.

[0079] This embodiment also illustrates a specific scheme for large-scale deployment at monitoring stations in nature reserves. Each monitoring station is equipped with multiple ecological monitoring devices as described in this invention. The separated bird call signals collected by the devices are aggregated through a unified data management platform. The data management platform integrates a bird species identification model and a population statistics and analysis module, generating real-time heat maps of bird activity and population dynamic trend reports within the distributed monitoring area. The platform supports collaborative analysis of data from multiple stations; by comparing the same bird call signals detected at different stations during the same period, the spatial distribution range and individual numbers of target birds can be estimated. This scheme has been applied in a pilot monitoring project in a nature reserve in China. During a 6-month monitoring period, over 42,000 valid bird call signals were recorded, and 48 bird species were identified, including 3 species of national second-class protected birds. The pilot monitoring results show that the method of this invention can effectively support large-scale ecological monitoring tasks and significantly improve the efficiency and data quality of bird species surveys.

[0080] The integrated system's real-time performance metrics are as follows: processing latency for a single frame of audio signal is 12ms, and end-to-end latency (including signal acquisition, separation, feature extraction, and species identification) is approximately 150ms, meeting the requirements for real-time monitoring. The system supports simultaneous input from up to 8 channels and can handle overlapping scenarios with more than 3 birds calling simultaneously. Within a 5km radius, a single monitoring station can cover a monitoring area of ​​approximately 20 square kilometers, and continuous monitoring coverage over a larger area can be achieved through multi-site collaboration.

[0081] Although the present invention has been specifically described above with reference to preferred embodiments, it should be understood that the present invention is not limited to the embodiments described above. Various modifications and variations can be made by those skilled in the art without departing from the spirit of the present invention, and such modifications and variations should fall within the scope defined by the appended claims and their equivalents.

Claims

1. A method for separating overlapping bird calls based on phase matrix decomposition, characterized in that, Includes the following steps: Acquire the overlapping bird song audio signal to be processed; Perform a short-time Fourier transform on the overlapping bird song audio signal to obtain the time-frequency representation X(f,t); The phase gradient is extracted from the time-frequency representation X(f,t), and a two-dimensional phase gradient matrix G is constructed based on the phase gradient; The two-dimensional phase gradient matrix G is subjected to singular value decomposition to obtain the left singular vector matrix U, the singular value matrix Σ and the right singular vector matrix V; The phase consistency index is calculated based on the singular value decomposition results, and the primary signal source and secondary signal source are determined accordingly. Based on the determination result, the time-frequency components of the main signal source and the time-frequency components of the secondary signal source are separated from the time-frequency representation X(f,t); The separated time-frequency components are subjected to inverse short-time Fourier transform to obtain the separated time-domain bird song signal.

2. The overlapping bird song separation method based on phase matrix decomposition according to claim 1, characterized in that, The step of extracting the phase gradient from the time-frequency representation X(f,t) to construct the two-dimensional phase gradient matrix G includes: Calculate the phase difference between adjacent time frames in the time-frequency representation X(f,t) to obtain the time-dimensional phase gradient Δθt; Calculate the phase difference between adjacent frequency bins in the time-frequency representation X(f,t) to obtain the frequency-dimensional phase gradient Δθf; The time-dimensional phase gradient Δθt and the frequency-dimensional phase gradient Δθf are organized into a two-dimensional phase gradient matrix G, wherein the time-dimensional phase gradient constitutes the columns of the two-dimensional phase gradient matrix, and the frequency-dimensional phase gradient constitutes the rows of the two-dimensional phase gradient matrix.

3. The overlapping bird song separation method based on phase matrix decomposition according to claim 2, characterized in that, The step of performing singular value decomposition on the two-dimensional phase gradient matrix G includes: Perform singular value decomposition on the two-dimensional phase gradient matrix G to obtain G=U·Σ·V^T; The primary and secondary signal components are determined based on the magnitude of the singular values. The singular vectors corresponding to the top N largest singular values ​​are retained, where N is a positive integer greater than or equal to 2. The retained singular vectors are reassembled into a low-rank approximate matrix.

4. The overlapping bird song separation method based on phase matrix decomposition according to claim 3, characterized in that, The step of calculating the phase consistency index based on the singular value decomposition result and determining the primary and secondary signal sources accordingly includes: Calculate the phase consistency index γi=|(U_i)^T·V_i| for each singular value; A phase consistency threshold γ_th is set. The signal components corresponding to the singular values ​​of the phase consistency index γi that are greater than the threshold γ_th are determined as the main signal source, and the signal components corresponding to the singular values ​​of the phase consistency index γi that are less than the threshold γ_th are determined as the secondary signal source or noise components.

5. The overlapping bird song separation method based on phase matrix decomposition according to claim 4, characterized in that, The step of separating the time-frequency components of the main signal source from the time-frequency representation X(f,t) based on the determination result includes: Based on the reconstruction result of the low-rank approximation matrix, the time-frequency components of the main signal source are separated from the original time-frequency representation X(f,t); The remaining time-frequency components are identified as secondary signal sources or noise components; The time-frequency components of the main signal source and the time-frequency components of the secondary signal source are output separately.

6. The overlapping bird song separation method based on phase matrix decomposition according to claim 5, characterized in that, Before performing singular value decomposition on the two-dimensional phase gradient matrix G, the method further includes: The two-dimensional phase gradient matrix G is normalized. And / or perform mean-removal processing on the two-dimensional phase gradient matrix G; And / or perform frequency band filtering on the two-dimensional phase gradient matrix G, retaining only phase information within the frequency band range of 200Hz to 15000Hz.

7. The overlapping bird song separation method based on phase matrix decomposition according to claim 6, characterized in that, Prior to the step of acquiring the overlapping bird song audio signal to be processed, the method further includes: A pre-trained sound activity detection model is used to detect the input audio and determine whether there are overlapping bird calls. The phase matrix decomposition and separation process is triggered when overlapping bird call signals are detected. And / or pre-filter the input audio to remove frequency components that are outside the range of birdsong frequencies; And / or use an automatic gain control module to adjust the dynamic range of the input audio signal to a preset level.

8. A system for separating overlapping bird calls based on phase matrix decomposition, used to implement the method for separating overlapping bird calls based on phase matrix decomposition as described in any one of claims 1 to 7, characterized in that, include: The audio signal acquisition module is used to acquire overlapping bird song audio signals to be processed; The time-frequency transformation module is used to perform a short-time Fourier transform on the overlapping bird song audio signal to obtain a time-frequency representation; A phase gradient extraction module is used to extract the phase gradient from the time-frequency representation and construct a two-dimensional phase gradient matrix; A singular value decomposition module is used to perform singular value decomposition on the two-dimensional phase gradient matrix. The signal separation module is used to calculate the phase consistency index based on the singular value decomposition results and perform signal separation; The time-domain reconstruction module is used to perform inverse short-time Fourier transform on the separated time-frequency components; The audio output module is used to output the separated time-domain bird song signal; The preprocessor module is used to perform sound activity detection, pre-filtering, and automatic gain control on the input audio.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the overlapping bird song separation method based on phase matrix decomposition as described in any one of claims 1 to 7.

10. An electronic product comprising a processor, a memory, an audio acquisition module, and a display module, characterized in that, The audio acquisition module includes at least one microphone for acquiring overlapping bird song audio signals to be processed; the processor is used to acquire the overlapping bird song audio signals, perform a short-time Fourier transform on the overlapping bird song audio signals to obtain a time-frequency representation X(f,t), extract the phase gradient from the time-frequency representation X(f,t) to construct a two-dimensional phase gradient matrix G, perform singular value decomposition on the two-dimensional phase gradient matrix G, calculate the phase consistency index based on the singular value decomposition result, determine the primary signal source and the secondary signal source accordingly, and separate the time-frequency components of the primary signal source and the secondary signal source from the time-frequency representation X(f,t) based on the determination result, perform inverse short-time Fourier transform on the separated time-frequency components respectively to obtain the separated time-domain bird song signals; the display module is used to output and display the analysis results of the separated time-domain bird song signals.

Images