A multi-source periodic audio signal processing method and device and a storage medium

Abstract

The application discloses a multi-source periodic audio signal processing method, device and equipment and a storage medium. The method comprises the following steps: performing noise reduction and DC offset removal processing on a first audio signal and a second audio signal; performing resampling on the first audio signal and the second audio signal by using an interpolation algorithm to make the sampling frequencies consistent; determining the period of the audio signal based on the main frequency of the audio signal, performing signal alignment on the audio signal by taking a single period as the minimum processing unit, wherein the signal alignment comprises peak signal-based alignment and short-time average amplitude value-based alignment; and performing amplitude standardization on the two aligned periodic audio signals to ensure that the amplitude ranges of the two signals are consistent. The application introduces a peak alignment method based on periodic characteristics and a short-time average amplitude difference (STAAD) sequence calculation, thereby improving the alignment accuracy and the calculation efficiency in time sequence alignment.

Description

Technical Field

[0001] This invention relates to the field of signal technology, and specifically to a method, apparatus, and storage medium for processing multi-source periodic audio signals. Background Technology

[0002] Audio signal processing is the foundation of time-domain analysis. Existing technologies for time-series processing of periodic audio signals have the following problems:

[0003] Amplitude differences in multi-source signals: When processing multi-source audio signals, amplitude differences often occur due to variations in microphone amplification or placement. Traditional methods often result in significant amplitude differences when processing multi-source signals, thus affecting alignment and normalization.

[0004] Multi-source signal alignment problem: When multiple different audio acquisition devices record simultaneously, time shifts may occur in the signals due to device transmission delays or other factors. Traditional processing methods are inefficient and inaccurate in resolving time shift problems.

[0005] Integrity issues of periodic signals: For audio signals with long periods (such as industrial machine sounds or natural sounds lasting more than a few seconds), traditional standardization methods may ignore the periodicity of the signal, resulting in the loss of important periodic information during data processing. This not only affects the usability of the signal but may also lead to inaccurate analysis results. Summary of the Invention

[0006] To address the problems existing in the prior art, the present invention provides a method, apparatus, and storage medium for processing multi-source periodic audio signals, the technical solution of which is as follows:

[0007] Firstly, a method for processing multi-source periodic audio signals is provided, comprising the following steps:

[0008] Multiple audio signals are acquired, one of which is used as a reference signal and denoted as the first audio signal, and the remaining signals are used as signals to be processed and denoted as the second audio signal.

[0009] Noise reduction and DC offset removal are performed on the first and second audio signals.

[0010] An interpolation algorithm is used to resample the first audio signal and the second audio signal. The sampling frequency after resampling is the least common multiple of the sampling rates of the first audio signal and the second audio signal.

[0011] Audio feature extraction is performed on the first audio signal and the second audio signal, including extracting the main frequency of the audio signal;

[0012] The period of the audio signal is determined based on the main frequency of the audio signal, and the audio signal is aligned by taking a single period as the smallest processing unit. The signal alignment includes alignment based on the peak signal and alignment based on the short-time average amplitude value of the period.

[0013] The amplitude of the two aligned periodic audio signals is normalized to ensure that the amplitude range of the two signals is consistent.

[0014] In some implementations, extracting the dominant frequency of the audio signal includes:

[0015] The spectrum of the signal is calculated using Fast Fourier Transform (FFT), and the frequency with the largest amplitude in the spectrum is extracted as the main frequency f0.

[0016] In some implementations, the signal alignment of the audio signal in units of a single cycle includes:

[0017] For the first audio signal and the second audio signal, the period T0 is determined according to the main frequency f0 of the audio signal. Based on the period T0, the start and end points of the period of the first audio signal and the second audio signal are found in the time domain, and the complete period segments of the first audio signal and the second audio signal are extracted respectively.

[0018] For the first audio signal and the second audio signal, the peak points in each period are obtained respectively, forming the peak sequence of the first audio signal, which is denoted as the first peak sequence, and the peak sequence of the second audio signal, which is denoted as the second peak sequence. The peak points in each period are determined based on the extreme value method.

[0019] Based on the first peak sequence and the second peak sequence, the preliminary time offset of the first audio signal and the second audio signal is calculated. Where N is the number of peak signals, t ref,i It is the time of the i-th peak of the reference signal, i.e., the first audio signal, where ref represents the reference signal, and t target,i The time of the i-th peak of the signal to be processed, i.e., the second audio signal, is represented by , and target represents the target signal;

[0020] Based on the calculated preliminary time offset Δt peak Apply a time offset, i.e., a time axis shift, to the target signal to perform initial peak alignment with the reference signal.

[0021] In some embodiments, the signal alignment of the audio signal in units of a single cycle further includes:

[0022] After initial peak alignment between the target signal and the reference signal...

[0023] For the first audio signal and the second audio signal, they are respectively divided into multiple first windows according to their respective periods, and each first window includes a complete period segment;

[0024] For the first window, the signal is further divided into two segments by using a second window that is shorter than the length of the first window, thus obtaining multiple second window segments from the first window.

[0025] For the second window, the short-time average amplitude of the signal within the second window is calculated based on the signal amplitude of all samples within the second window;

[0026] The short-time average amplitude of the second window is weighted and then fused to obtain the weighted average amplitude value of the first window; the window-weighted average amplitude sequence of the first audio signal and the window-weighted average amplitude sequence of the second audio signal are obtained.

[0027] Periodic window alignment is performed based on the window-weighted average amplitude sequence of the first audio signal and the window-weighted average amplitude sequence of the second audio signal. The alignment process involves fixing the first audio signal and gradually adjusting the time axis of the second audio signal through time shifting to minimize the error between their window-weighted average amplitude sequences. The error between their window-weighted average amplitude sequences is... Where M is the number of periodic windows in the aligned segment after the first and second audio signals are aligned, and k is the kth periodic window in the aligned segment.

[0028] In some implementations, the short-term average amplitude of the second window is weighted, and the second window containing the peak is given a larger weight, that is, the second window closer to the peak has a larger weight.

[0029] In some embodiments, the noise reduction and DC offset removal processing for the first audio signal and the second audio signal includes:

[0030] Perform wavelet transform on the signal to obtain the wavelet transform coefficients;

[0031] Based on a preset wavelet coefficient threshold, remove the portion of wavelet coefficients whose amplitude is lower than the preset wavelet coefficient threshold;

[0032] Perform inverse wavelet transform to recover the denoised signal;

[0033] Calculate the mean of the signal and subtract it from the signal;

[0034] The signal is center-clipping to remove the portion exceeding a preset signal amplitude threshold.

[0035] In some embodiments, the resampling of the signals using an interpolation algorithm on the first and second audio signals, wherein the resampled sampling frequency is the least common multiple of the sampling rates of the first and second audio signals, includes:

[0036] Find the least common multiple f of the sampling rates of the first and second audio signals. LCM ;

[0037] The first and second audio signals were resampled based on the B-spline interpolation algorithm.

[0038] Secondly, a multi-source periodic audio signal processing device is provided, comprising:

[0039] An audio signal acquisition unit is used to acquire multiple audio signals, using one of the multiple audio signals as a reference signal and denoted as the first audio signal, and the remaining signals as signals to be processed and denoted as the second audio signal;

[0040] The preprocessing unit is used to perform noise reduction and DC offset removal processing on the first audio signal and the second audio signal;

[0041] The resampling unit is used to resample the first audio signal and the second audio signal using an interpolation algorithm. The sampling frequency after resampling is the least common multiple of the sampling rates of the first audio signal and the second audio signal.

[0042] The signal feature acquisition unit is used to extract audio features from the first audio signal and the second audio signal, including extracting the main frequency of the audio signal.

[0043] The signal alignment unit is used to determine the period of the audio signal based on the main frequency of the audio signal, and to perform signal alignment on the audio signal with a single period as the smallest processing unit. The signal alignment includes alignment based on the peak signal and alignment based on the short-time average amplitude value of the period.

[0044] The signal normalization unit is used to normalize the amplitude of two aligned periodic audio signals to ensure that the amplitude range of the two signals is consistent.

[0045] In some embodiments, the signal alignment unit includes: a preliminary peak alignment unit and an optimized alignment unit.

[0046] The preliminary peak alignment unit is used to perform the following steps:

[0047] For the first audio signal and the second audio signal, the period T0 is determined according to the main frequency f0 of the audio signal. Based on the period T0, the start and end points of the period of the first audio signal and the second audio signal are found in the time domain, and the complete period segments of the first audio signal and the second audio signal are extracted respectively.

[0048] For the first audio signal and the second audio signal, the peak points in each period are obtained respectively, forming the peak sequence of the first audio signal, which is denoted as the first peak sequence, and the peak sequence of the second audio signal, which is denoted as the second peak sequence. The peak points in each period are determined based on the extreme value method.

[0049] Based on the first peak sequence and the second peak sequence, the preliminary time offset of the first audio signal and the second audio signal is calculated. Where N is the number of peak signals, t ref,i It is the time of the i-th peak of the reference signal, i.e., the first audio signal, where ref represents the reference signal, and t target,i The time of the i-th peak of the signal to be processed, i.e., the second audio signal, is represented by , and target represents the target signal;

[0050] Based on the calculated preliminary time offset Δt peak Applying a time offset, i.e., a time axis shift, to the target signal initially aligns the peak values ​​of the target signal with those of the reference signal.

[0051] The optimized alignment unit is used to perform the following steps:

[0052] After the target signal and the reference signal are initially aligned at their peaks, the first audio signal and the second audio signal are divided into multiple first windows according to their respective periods, and each first window includes a complete period segment.

[0053] For the first window, the signal is further divided into two segments by using a second window that is shorter than the length of the first window, thus obtaining multiple second window segments from the first window.

[0054] For the second window, the short-time average amplitude of the signal within the second window is calculated based on the signal amplitude of all samples within the second window;

[0055] The short-time average amplitude of the second window is weighted and then fused to obtain the weighted average amplitude value of the first window; the window-weighted average amplitude sequence of the first audio signal and the window-weighted average amplitude sequence of the second audio signal are obtained.

[0056] Periodic window alignment is performed based on the window-weighted average amplitude sequence of the first audio signal and the window-weighted average amplitude sequence of the second audio signal. The alignment process involves fixing the first audio signal and gradually adjusting the time axis of the second audio signal through time shifting to minimize the error between their window-weighted average amplitude sequences. The error between their window-weighted average amplitude sequences is... Where M is the number of periodic windows in the aligned segment after the first and second audio signals are aligned, and k is the kth periodic window in the aligned segment.

[0057] Thirdly, a computer-readable storage medium is provided having computer instructions stored thereon, which, when executed by a processor, implement the steps of the multi-source periodic audio signal processing method described in the first aspect above.

[0058] The multi-source periodic audio signal processing method, apparatus, and storage medium of the present invention have the following beneficial effects:

[0059] 1. This invention improves the accuracy and efficiency of time alignment through innovative methods such as peak alignment, short-time average amplitude difference (STAAD) sequence calculation, and periodic feature detection, especially in complex or noisy scenarios.

[0060] 2. This invention combines DC offset removal, center clipping, and a fine amplitude normalization method to effectively balance the amplitude differences of multi-source audio signals and ensure that the signals maintain a natural dynamic range during the alignment process.

[0061] 3. This invention first performs rapid preliminary alignment through peak alignment, and then uses the STAAD method to further refine the alignment accuracy. Especially in the presence of noise, small errors or irregular periodic changes, it can effectively improve the robustness and accuracy of alignment.

[0062] 4. This invention ensures that the periodic structure of long-period audio signals is not lost through periodic window division and short-time average amplitude calculation, making it particularly suitable for processing long-period audio signals in complex multi-source environments. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating a multi-source periodic audio signal processing method according to an embodiment of this application;

[0064] Figure 2 This is a flowchart illustrating the signal alignment method in an embodiment of this application;

[0065] Figure 3 This is a schematic diagram of the structure of a multi-source periodic audio signal processing device according to an embodiment of this application. Detailed Implementation

[0066] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0067] This application provides a method for processing multi-source periodic audio signals, including the following steps:

[0068] Step 1: Acquire multiple audio signals, take one of the multiple audio signals as the reference signal and record it as the first audio signal, and take the remaining signals as the signals to be processed and record them as the second audio signal;

[0069] Step 2: Perform noise reduction and DC offset removal processing on the first audio signal and the second audio signal;

[0070] Step 3: Use an interpolation algorithm to resample the first audio signal and the second audio signal. The sampling frequency after resampling is the least common multiple of the sampling rates of the first audio signal and the second audio signal.

[0071] Step 4: Extract audio features from the first and second audio signals, including extracting the main frequency of the audio signals;

[0072] Step 5: Determine the period of the audio signal based on the main frequency of the audio signal, and perform signal alignment on the audio signal with a single period as the smallest processing unit. The signal alignment includes alignment based on the peak signal and alignment based on the short-time average amplitude value of the period.

[0073] Step 6: Normalize the amplitude of the two aligned periodic audio signals to ensure that the amplitude range of the two signals is consistent.

[0074] In this embodiment, a more refined amplitude normalization method is provided by combining DC offset removal and center clipping processing, effectively balancing the amplitude differences of multi-source signals. This method can adapt to the gain differences of different recording devices and microphones, and maintains the natural dynamic range of the signal during processing, avoiding signal distortion or over-compression caused by amplitude differences.

[0075] In this embodiment, peak alignment and short-time average amplitude sequence (STAAD) alignment calculations are introduced, combined with the periodic characteristics of the signal, to achieve accurate time-series alignment of the signal. This process utilizes time-domain and frequency-domain information based on periodic characteristics, simultaneously improving the computational efficiency of time-series alignment and avoiding the reliance on large computational loads found in traditional methods. Furthermore, a correlation calculation method for the STAAD sequence is employed to improve alignment accuracy and reduce the impact of noise, thus solving the problem of low accuracy in existing technologies under noise interference.

[0076] In one embodiment, step 2 above, the noise reduction and DC offset removal processing for the first audio signal and the second audio signal, includes:

[0077] Step 21: Perform wavelet transform on the signal and obtain the wavelet transform coefficients;

[0078] Step 22: Based on a preset wavelet coefficient threshold, remove the portion of the wavelet coefficients whose amplitude is lower than the preset wavelet coefficient threshold; let x(t) be the original signal, W ψ (x(t)) are wavelet transform coefficients. A preset wavelet coefficient threshold T is used to remove the portion of the wavelet coefficients whose amplitude is lower than the preset threshold T.

[0079] Step 23: Perform inverse wavelet transform to recover the denoised signal.

[0080] Step 24: Calculate the mean of the signal and subtract it from the signal: x2(t) = x1(t) - μ(x), where μ(x) is the mean of the original signal;

[0081] Step 25: Perform center clipping on the signal to remove the portion exceeding the preset signal amplitude threshold: This yields the signal x3(t) after noise reduction and DC offset removal.

[0082] In the audio signal preprocessing, this invention adds DC offset removal after noise reduction, which effectively minimizes signal differences and thus improves the overall effect of signal alignment and standardization.

[0083] In one implementation, step 3 above, where the first audio signal and the second audio signal are resampled using an interpolation algorithm, and the resampled sampling frequency is the least common multiple of the sampling rates of the first audio signal and the second audio signal, includes:

[0084] Find the least common multiple f of the sampling rates of the first and second audio signals. LCM ;

[0085] The first and second audio signals were resampled based on the B-spline interpolation algorithm.

[0086] This invention employs B-spline interpolation technology, providing higher accuracy and smoother results during audio resampling, thus effectively solving the problem of audio quality degradation caused by inconsistent sampling rates. Replacing traditional linear interpolation with B-spline interpolation offers higher interpolation accuracy, especially in high-frequency signal processing, better maintaining signal smoothness and avoiding signal distortion. Precise sampling rate adjustment ensures seamless integration of audio signals across different devices and systems, improving overall audio quality.

[0087] In one embodiment, step 4 above, extracting the main frequency of the audio signal, includes: calculating the spectrum of the signal using Fast Fourier Transform (FFT), and extracting the frequency with the largest amplitude in the spectrum as the main frequency f0.

[0088] In one implementation, step 5 above, which involves aligning the audio signal in units of a single cycle, includes peak-based alignment as follows:

[0089] Step 501: For the first audio signal and the second audio signal, determine the period T0 according to the main frequency f0 of the audio signal respectively. According to the period T0, find the start and end points of the period of the first audio signal and the second audio signal in the time domain, and extract the complete period segments of the first audio signal and the complete period segments of the second audio signal respectively.

[0090] Step 502: For the first audio signal and the second audio signal, obtain the peak points in each period respectively, form the peak sequence of the first audio signal as the first peak sequence, and form the peak sequence of the second audio signal as the second peak sequence. The peak points in each period are determined based on the extreme value method.

[0091] Step 503: Based on the first peak sequence and the second peak sequence, calculate the preliminary time offset between the first audio signal and the second audio signal. Where N is the number of peak signals, t ref,i It is the time of the i-th peak of the reference signal, i.e., the first audio signal, where ref represents the reference signal, and t target,i The time of the i-th peak of the signal to be processed, i.e., the second audio signal, is represented by , and target represents the target signal;

[0092] Step 504, based on the calculated preliminary time offset Δt peak Apply a time offset, i.e., a time axis shift, to the target signal to perform initial peak alignment with the reference signal.

[0093] This embodiment employs peak alignment, where signal peaks represent maxima or minima within a signal period. Detecting local peaks helps identify key time points within the signal. A maximum detection algorithm is used within each period to find the peaks and obtain the main peak times for each period. For both the reference and target signals, their peak times are detected to obtain the peak sequence P. ref and P target That is, the time position of each peak. Compare the peak point sequence P. ref and P target The time offset between the two signals was initially calculated. By calculating the difference between the peak points, a preliminary alignment offset Δt was obtained. peak N is the number of peak points, and Δt is calculated. peak It is a holistic time offset.

[0094] In one implementation, step 5 above, which involves aligning the audio signal with a single cycle as the smallest processing unit, further includes alignment based on the short-time average amplitude value of the cycle, comprising the following steps:

[0095] Step 505: After the target signal and the reference signal are initially aligned at the peak, the first audio signal and the second audio signal are divided into multiple first windows according to their respective periods, and each first window includes a complete period segment.

[0096] Step 506: For the first window, the signal is further divided into second windows with a length smaller than the first window to obtain multiple second window segments in the first window;

[0097] Step 507: For the second window, calculate the short-time average amplitude of the signal within the second window based on the signal amplitude of all samples within the second window;

[0098] Step 508: Weight the short-time average amplitude of the second window, and after weighted fusion, obtain the weighted average amplitude value of the first window; obtain the window weighted average amplitude sequence of the first audio signal and the window weighted average amplitude sequence of the second audio signal;

[0099] Step 509: Perform periodic window alignment based on the window-weighted average amplitude sequence of the first audio signal and the window-weighted average amplitude sequence of the second audio signal. This alignment process involves fixing the first audio signal and gradually adjusting the time axis of the second audio signal through time shifting to minimize the error between their window-weighted average amplitude sequences. The error between their window-weighted average amplitude sequences is... Where M is the number of periodic windows in the aligned segment after the first and second audio signals are aligned, and k is the kth periodic window in the aligned segment.

[0100] In this embodiment, after peak alignment, the overall periodic characteristics of the signal are roughly aligned, but minor deviations or noise may still exist. At this stage, the STAAD (Short Time Average Amplitude) sequence is used to further optimize the signal alignment. During alignment, the time axis of the target signal is gradually adjusted by time shifting until the difference between the two STAAD sequences is minimized. A small time shift is applied to the target signal, and then the optimized MSE is calculated. This process is iterated, gradually minimizing the MSE until the difference between the two STAAD sequences converges to its minimum value.

[0101] Traditional signal normalization methods often ignore the periodicity of signals, especially in the processing of long-period signals, which can easily lead to the loss of periodic information. However, the embodiments of this application introduce methods such as periodic feature detection and intra-period windowing to maintain the periodic structure of the signal, thereby ensuring that key information is not lost during alignment and normalization. This is particularly suitable for processing long-period audio signals such as industrial machine sounds and natural environmental sounds.

[0102] In this embodiment, a rapid initial alignment is first performed using peak alignment, and the alignment accuracy is further refined using the STAAD method. This method can effectively improve the robustness and accuracy of alignment, especially in the presence of noise, small errors, or irregular periodic variations.

[0103] Specifically, in step 508 above, the short-term average amplitude of the second window is weighted, with a larger weight given to the second window containing the peak and a smaller weight given to the second window without the peak. That is, the second window that is closer to the peak has a larger weight.

[0104] In this embodiment, local features are enhanced by weighting the short-time amplitude of peak points. The STAAD method measures the signal alignment by calculating the average amplitude difference within a short-time window. To eliminate noise and instability, STAAD in this embodiment weights important windows (e.g., windows containing peaks) to optimize signal alignment. Unlike peak alignment methods, STAAD does not rely solely on a single peak point but considers the amplitude distribution of the entire signal, thus optimizing signal alignment more comprehensively. The STAAD method can effectively eliminate minor alignment errors caused by noise, inaccurate peak positions, period drift, and other factors.

[0105] In one implementation, step 6 above involves standardizing the amplitude of the two aligned periodic audio signals to ensure that the amplitude ranges of the two signals are consistent, including: Where, x aligned (t) represents the aligned signal, and μ and σ are the mean and standard deviation of the signal, respectively.

[0106] Based on the above embodiments of the multi-source periodic audio signal processing method, this application provides a multi-source periodic audio signal processing apparatus 100, including:

[0107] The audio signal acquisition unit 101 is used to acquire multiple audio signals, take one of the multiple audio signals as a reference signal and record it as the first audio signal, and take the remaining signals as signals to be processed and record them as the second audio signal;

[0108] The preprocessing unit 102 is used to perform noise reduction and DC offset removal processing on the first audio signal and the second audio signal;

[0109] The resampling unit 103 is used to resample the first audio signal and the second audio signal using an interpolation algorithm. The sampling frequency after resampling is the least common multiple of the sampling rates of the first audio signal and the second audio signal.

[0110] The signal feature acquisition unit 104 is used to extract audio features from the first audio signal and the second audio signal, including extracting the main frequency of the audio signal.

[0111] The signal alignment unit 105 is used to determine the period of the audio signal based on the main frequency of the audio signal, and to perform signal alignment on the audio signal with a single period as the smallest processing unit. The signal alignment includes alignment based on the peak signal and alignment based on the short-time average amplitude value of the period.

[0112] The signal normalization unit 106 is used to normalize the amplitude of two aligned periodic audio signals to ensure that the amplitude range of the two signals is consistent.

[0113] Specifically, the aforementioned signal alignment unit 105 includes: a preliminary peak alignment unit 1051 and an optimized alignment unit 1052.

[0114] The preliminary peak alignment unit 1051 is used to perform the following steps:

[0115] For the first audio signal and the second audio signal, the period T0 is determined according to the main frequency f0 of the audio signal. Based on the period T0, the start and end points of the period of the first audio signal and the second audio signal are found in the time domain, and the complete period segments of the first audio signal and the second audio signal are extracted respectively.

[0116] For the first audio signal and the second audio signal, the peak points in each period are obtained respectively, forming the peak sequence of the first audio signal, which is denoted as the first peak sequence, and the peak sequence of the second audio signal, which is denoted as the second peak sequence. The peak points in each period are determined based on the extreme value method.

[0117] Based on the first peak sequence and the second peak sequence, the preliminary time offset of the first audio signal and the second audio signal is calculated. Where N is the number of peak signals, t ref,i It is the time of the i-th peak of the reference signal, i.e., the first audio signal, where ref represents the reference signal, and t target,i The time of the i-th peak of the signal to be processed, i.e., the second audio signal, is represented by , and target represents the target signal;

[0118] Based on the calculated preliminary time offset Δt peak Applying a time offset, i.e., a time axis shift, to the target signal initially aligns the peak values ​​of the target signal with those of the reference signal.

[0119] The optimized alignment unit 1052 is used to perform the following steps:

[0120] After the target signal and the reference signal are initially aligned at their peaks, the first audio signal and the second audio signal are divided into multiple first windows according to their respective periods, and each first window includes a complete period segment.

[0121] For the first window, the signal is further divided into two segments by using a second window that is shorter than the length of the first window, thus obtaining multiple second window segments from the first window.

[0122] For the second window, the short-time average amplitude of the signal within the second window is calculated based on the signal amplitude of all samples within the second window;

[0123] The short-time average amplitude of the second window is weighted and then fused to obtain the weighted average amplitude value of the first window; the window-weighted average amplitude sequence of the first audio signal and the window-weighted average amplitude sequence of the second audio signal are obtained.

[0124] Periodic window alignment is performed based on the window-weighted average amplitude sequence of the first audio signal and the window-weighted average amplitude sequence of the second audio signal. The alignment process involves fixing the first audio signal and gradually adjusting the time axis of the second audio signal through time shifting to minimize the error between their window-weighted average amplitude sequences. The error between their window-weighted average amplitude sequences is... Where M is the number of periodic windows in the aligned segment after the first and second audio signals are aligned, and k is the kth periodic window in the aligned segment.

[0125] For specific limitations regarding a multi-source periodic audio signal processing device, please refer to the limitations regarding a multi-source periodic audio signal processing method mentioned above, which will not be repeated here.

[0126] This application provides a computer-readable storage medium storing computer instructions, characterized in that the instructions, when executed by a processor, constitute the steps of the aforementioned multi-source periodic audio signal processing method. The computer-readable storage medium includes: permanent and non-permanent, removable and non-removable media, and is a tangible device capable of retaining and storing instructions for use by an instruction execution device. The computer-readable storage medium includes: electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, and any suitable combination thereof. Computer-readable storage media include: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, optical disc read-only memory (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape storage, magnetic disk storage or other magnetic storage devices, memory sticks, mechanical encoding devices (such as punched cards or raised structures in grooves on which instructions are recorded), or any other non-transfer medium that can be used to store information that can be accessed by a computing device.

[0127] The following describes the experimental process and results of the multi-source periodic audio signal processing method provided in the embodiments of this application.

[0128] 1. Introduction to the dataset

[0129] The experiments used the DCASE2024 Task2 dataset. The DCASE (Detection and Classification of Acoustic Scenes and Events) task is primarily used to evaluate the performance of audio signal processing algorithms. Task 2 is an unsupervised abnormal sound detection task for machine condition monitoring. The dataset contains normal and abnormal sounds from seven real machines (fans, gearboxes, bearings, sliders, toy cars, toy trains, and valves). Each recording is a 10-second mono audio clip that includes the target machine's sound and ambient sounds.

[0130] 2. Performance Evaluation Indicators

[0131] To evaluate model performance from multiple perspectives, the AUC and PAUC metrics were calculated using the anomaly scores of the test set samples, and the normal / abnormal judgment results of the test set samples were used as evaluation metrics for the model's detection effectiveness.

[0132] AUC (source): The AUC (Area Under the Curve) metric for source domain audio samples, used to measure the ability to distinguish between normal and abnormal sounds.

[0133] AUC (target): The AUC metric for audio samples in the target domain, used to measure the model's ability to adapt to unknown machine states.

[0134] pAUC(source, target): A partial AUC metric that comprehensively evaluates the performance of samples from the source and target domains.

[0135] TOTAL score: A comprehensive performance metric that serves as an overall measure of the final model's performance.

[0136] 3. Comparative Experiment

[0137] To verify the effectiveness of the periodic audio signal processing method of this invention, this experiment conducts a performance comparison test based on the DCASE2024 Task 2 dataset to evaluate the improvement of downstream task performance (anomaly detection) after alignment and normalization.

[0138] The experiment was divided into the following two groups for comparison:

[0139] a. Baseline method: Directly use the official baseline code (baseline_MAHALA and baseline_MSE) provided by DCASE to process audio signals and extract features.

[0140] b. The method of the present invention: Based on the baseline method, the audio signal alignment and normalization method (our_MAHALA and our_MSE) of the present invention is added, including audio signal preprocessing (denoising, DC offset removal, center clipping processing), sampling rate adjustment (resampling based on B-spline interpolation method to ensure signal consistency), timing alignment (periodic signal alignment through peak detection and time offset optimization), and normalization (amplitude normalization to ensure signal consistency).

[0141] 4. Experimental Results

[0142] The method of this invention significantly improves the detection performance of audio samples in both the source and target domains compared to baseline methods, and outperforms the baseline methods overall. Results show that the method of this invention can effectively improve the alignment and normalization of periodic audio signals. Its anomaly detection performance on the DCASE2024 Task 2 dataset is significantly better than the baseline method. The method of this invention has broad application prospects and can be used to improve the robustness and accuracy of signal processing in various audio processing tasks.

[0143] Table 1 Comparison results of the method of the present invention and the baseline method

[0144]

[0145] This invention is not limited to the specific embodiments described above. Any modifications made by those skilled in the art based on the above concept without creative effort are within the scope of protection of this invention.

Claims

1. A method for processing multi-source periodic audio signals, characterized in that, Includes the following steps: Multiple audio signals are acquired, one of which is used as a reference signal and denoted as the first audio signal, and the remaining signals are used as signals to be processed and denoted as the second audio signal. Noise reduction and DC offset removal are performed on the first and second audio signals. An interpolation algorithm is used to resample the first audio signal and the second audio signal. The sampling frequency after resampling is the least common multiple of the sampling rates of the first audio signal and the second audio signal. Audio feature extraction is performed on the first audio signal and the second audio signal, including extracting the main frequency of the audio signal; The period of the audio signal is determined based on the main frequency of the audio signal, and the audio signal is aligned by taking a single period as the smallest processing unit. The signal alignment includes alignment based on the peak signal and alignment based on the short-time average amplitude value of the period. The amplitude of the two aligned periodic audio signals is normalized to ensure that the amplitude range of the two signals is consistent.

2. The multi-source periodic audio signal processing method according to claim 1, characterized in that, The extraction of the main frequency of the audio signal includes: The spectrum of the signal is calculated using Fast Fourier Transform (FFT), and the frequency with the largest amplitude in the spectrum is extracted as the main frequency f0.

3. The multi-source periodic audio signal processing method according to claim 2, characterized in that, The process of aligning the audio signal with a single cycle as the smallest processing unit includes: For the first audio signal and the second audio signal, the period T0 is determined according to the main frequency f0 of the audio signal. Based on the period T0, the start and end points of the period of the first audio signal and the second audio signal are found in the time domain, and the complete period segments of the first audio signal and the second audio signal are extracted respectively. For the first audio signal and the second audio signal, the peak points in each period are obtained respectively, forming the peak sequence of the first audio signal, which is denoted as the first peak sequence, and the peak sequence of the second audio signal, which is denoted as the second peak sequence. The peak points in each period are determined based on the extreme value method. Based on the first peak sequence and the second peak sequence, the preliminary time offset of the first audio signal and the second audio signal is calculated. Where N is the number of peak signals, t ref,i It is the time of the i-th peak of the reference signal, i.e., the first audio signal, where ref represents the reference signal, and t target,i The time of the i-th peak of the signal to be processed, i.e., the second audio signal, is represented by , and target represents the target signal; Based on the calculated preliminary time offset Δt peak Apply a time offset, i.e., a time axis shift, to the target signal to perform initial peak alignment with the reference signal.

4. The multi-source periodic audio signal processing method according to claim 3, characterized in that, The method of aligning the audio signal with a single cycle as the smallest processing unit also includes: After initial peak alignment between the target signal and the reference signal... For the first audio signal and the second audio signal, they are respectively divided into multiple first windows according to their respective periods, and each first window includes a complete period segment; For the first window, the signal is further divided into two segments by using a second window that is shorter than the length of the first window, thus obtaining multiple second window segments from the first window. For the second window, the short-time average amplitude of the signal within the second window is calculated based on the signal amplitude of all samples within the second window; The short-time average amplitude of the second window is weighted and then fused to obtain the weighted average amplitude value of the first window; the window-weighted average amplitude sequence of the first audio signal and the window-weighted average amplitude sequence of the second audio signal are obtained. Periodic window alignment is performed based on the window-weighted average amplitude sequence of the first audio signal and the window-weighted average amplitude sequence of the second audio signal. The alignment process involves fixing the first audio signal and gradually adjusting the time axis of the second audio signal through time shifting to minimize the error between their window-weighted average amplitude sequences. The error between their window-weighted average amplitude sequences is... Where M is the number of periodic windows in the aligned segment after the first and second audio signals are aligned, and k is the kth periodic window in the aligned segment.

5. The multi-source periodic audio signal processing method according to claim 4, characterized in that, The short-term average amplitude of the second window is weighted, and the second window containing the peak is given a larger weight, that is, the second window closer to the peak has a larger weight.

6. The multi-source periodic audio signal processing method according to claim 1, characterized in that, The noise reduction and DC offset removal processing for the first and second audio signals includes: Perform wavelet transform on the signal to obtain the wavelet transform coefficients; Based on a preset wavelet coefficient threshold, remove the portion of wavelet coefficients whose amplitude is lower than the preset wavelet coefficient threshold; Perform inverse wavelet transform to recover the denoised signal; Calculate the mean of the signal and subtract it from the signal; The signal is center-clipping to remove the portion exceeding a preset signal amplitude threshold.

7. The multi-source periodic audio signal processing method according to claim 1, characterized in that, The resampling of the first and second audio signals using an interpolation algorithm, wherein the resampled sampling frequency is the least common multiple of the sampling rates of the first and second audio signals, includes: Find the least common multiple f of the sampling rates of the first and second audio signals. LCM ; The first and second audio signals were resampled based on the B-spline interpolation algorithm.

8. A multi-source periodic audio signal processing device, characterized in that, include: An audio signal acquisition unit is used to acquire multiple audio signals, using one of the multiple audio signals as a reference signal and denoted as the first audio signal, and the remaining signals as signals to be processed and denoted as the second audio signal; The preprocessing unit is used to perform noise reduction and DC offset removal processing on the first audio signal and the second audio signal; The resampling unit is used to resample the first audio signal and the second audio signal using an interpolation algorithm. The sampling frequency after resampling is the least common multiple of the sampling rates of the first audio signal and the second audio signal. The signal feature acquisition unit is used to extract audio features from the first audio signal and the second audio signal, including extracting the main frequency of the audio signal. The signal alignment unit is used to determine the period of the audio signal based on the main frequency of the audio signal, and to perform signal alignment on the audio signal with a single period as the smallest processing unit. The signal alignment includes alignment based on the peak signal and alignment based on the short-time average amplitude value of the period. The signal normalization unit is used to normalize the amplitude of two aligned periodic audio signals to ensure that the amplitude range of the two signals is consistent.

9. A multi-source periodic audio signal processing device according to claim 8, characterized in that, The signal alignment unit includes: a preliminary peak alignment unit and an optimized alignment unit. The preliminary peak alignment unit is used to perform the following steps: For the first audio signal and the second audio signal, the period T0 is determined according to the main frequency f0 of the audio signal. Based on the period T0, the start and end points of the period of the first audio signal and the second audio signal are found in the time domain, and the complete period segments of the first audio signal and the second audio signal are extracted respectively. For the first audio signal and the second audio signal, the peak points in each period are obtained respectively, forming the peak sequence of the first audio signal, which is denoted as the first peak sequence, and the peak sequence of the second audio signal, which is denoted as the second peak sequence. The peak points in each period are determined based on the extreme value method. Based on the first peak sequence and the second peak sequence, the preliminary time offset of the first audio signal and the second audio signal is calculated. Where N is the number of peak signals, t ref,i It is the time of the i-th peak of the reference signal, i.e., the first audio signal, where ref represents the reference signal, and t target,i The time of the i-th peak of the signal to be processed, i.e., the second audio signal, is represented by , and target represents the target signal; Based on the calculated preliminary time offset Δt peak Applying a time offset, i.e., a time axis shift, to the target signal initially aligns the peak values ​​of the target signal with those of the reference signal. The optimized alignment unit is used to perform the following steps: After the target signal and the reference signal are initially aligned at their peaks, the first audio signal and the second audio signal are divided into multiple first windows according to their respective periods, and each first window includes a complete period segment. For the first window, the signal is further divided into two segments by using a second window that is shorter than the length of the first window, thus obtaining multiple second window segments from the first window. For the second window, the short-time average amplitude of the signal within the second window is calculated based on the signal amplitude of all samples within the second window; The short-time average amplitude of the second window is weighted and then fused to obtain the weighted average amplitude value of the first window; the window-weighted average amplitude sequence of the first audio signal and the window-weighted average amplitude sequence of the second audio signal are obtained. Periodic window alignment is performed based on the window-weighted average amplitude sequence of the first audio signal and the window-weighted average amplitude sequence of the second audio signal. The alignment process involves fixing the first audio signal and gradually adjusting the time axis of the second audio signal through time shifting to minimize the error between their window-weighted average amplitude sequences. The error between their window-weighted average amplitude sequences is... Where M is the number of periodic windows in the aligned segment after the first and second audio signals are aligned, and k is the kth periodic window in the aligned segment.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1-7.

Images