A time-frequency dual-stream lightweight intelligent vehicle siren detection method for excavating audio period characteristics

Abstract

The application discloses a kind of based on time-frequency dual-flow feature fusion light siren detection method, belong to audio signal processing and intelligent driving perception technical field.The method first acquires audio signal and carries out pretreatment;Then two kinds of complementary features are extracted in parallel: one is to extract mel-frequency spectrum diagram by short-time Fourier transform and mel filter bank, as frequency domain feature flow;Two is to calculate the normalized autocorrelation function of audio signal, and extract deep time domain periodicity feature by a light one-dimensional convolution network ACgramNet, as time domain feature flow;Two kinds of features are spliced in channel dimension, form time-frequency dual-flow fusion feature;Finally, the fusion feature is input as MobileFaceNet light classification network optimized for mobile terminal, and the classification result of a variety of siren events and background noise is output.The application fully excavates the inherent strong periodicity essential feature of siren sound by introducing autocorrelation function, and is deeply fused with frequency domain feature, significantly improves the detection robustness in complex noise environment;While using light network structure, small amount of calculation, fast inference speed, meet the real-time requirement of vehicle-mounted embedded platform.The application can be widely applied to automatic driving, intelligent networked vehicle and roadside unit acoustic perception system.

Description

Technical Field

[0001] This invention belongs to the fields of audio signal processing, machine learning and intelligent driving perception technology, and specifically relates to a lightweight siren sound detection method and system for vehicle environments, based on the fusion of autocorrelation time-domain features and Mel-spectrum frequency-domain features. Background Technology

[0002] As autonomous driving technology advances to Level 3 and above, the requirements for the completeness of vehicle environmental perception are increasing. Existing perception systems, primarily based on vision and lidar, face severe challenges in scenarios such as blind spots, adverse weather conditions, and long-distance obstructions. The siren, as a core acoustic identifier for emergency vehicles (such as police cars, fire trucks, ambulances, and emergency rescue vehicles) performing their duties, provides reliable real-time detection with non-line-of-sight, all-weather complementary perception capabilities for autonomous driving systems, which is crucial for ensuring priority passage for emergency vehicles and improving road safety.

[0003] Traditional sound event detection methods typically rely on handcrafted features such as Mel-frequency cepstral coefficients and shallow models like Gaussian mixture models and support vector machines. These methods have limited representational capabilities and their discriminative power rapidly declines in complex noisy environments. In recent years, deep learning methods have been introduced into the fields of abnormal sound detection and siren recognition, achieving some progress. For example, some studies have used Mel-frequency cepstral coefficients combined with convolutional neural networks and recurrent neural networks for classification. However, existing technologies still suffer from the following significant drawbacks: Feature extraction is often limited in scope and lacks robustness to noise: Most existing methods rely solely on Mel spectra or their derivatives. Mel spectra primarily reflect the frequency domain structure of a signal and are sensitive to environmental noise, especially high-frequency noise. In real-world road environments, wind noise, tire noise, and background traffic noise are complex and variable, making it difficult to guarantee the stability of the model under low signal-to-noise ratio conditions with a single frequency domain feature.

[0004] The key essential attribute of sound events is overlooked: the most fundamental characteristic that distinguishes sirens from other environmental sounds is their inherent strong periodicity. For example, the national standard GB8108-2014 explicitly uses "repetition pitch variation period" as a key quantitative indicator to differentiate different sirens types. Existing technologies have failed to effectively design modules specifically for extracting and utilizing this periodic feature, resulting in models failing to learn the most essential discriminative information.

[0005] Excessive model complexity makes it difficult to meet the real-time requirements of automotive applications: Some studies have improved accuracy by integrating multiple models or complex network structures, but this has led to a dramatic increase in the number of model parameters and computational load. The computing power and power consumption budget of automotive embedded devices are extremely limited, making it difficult to deploy such heavy models and meet the stringent requirements of real-time detection and low power consumption.

[0006] In summary, existing technologies have failed to achieve a good balance between robustness in complex noise environments, utilization of core periodic features, and lightweight model deployment. Therefore, there is an urgent need for a lightweight detection method that can fully exploit the essential periodic features of siren sounds while also possessing high noise robustness and high efficiency. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the present invention aims to provide a lightweight siren sound detection method and system based on time-frequency dual-stream feature fusion, in order to solve the technical problems of poor robustness of existing methods for siren sound detection, insufficient utilization of key periodic features, and difficulty in real-time deployment of models on vehicle platforms in complex noise environments on real roads.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A lightweight siren sound detection method based on time-frequency dual-stream feature fusion includes the following steps: Step S1: Acquire audio signals and perform preprocessing. Acquire single-channel audio signals of the vehicle's surrounding environment and perform preprocessing operations such as pre-emphasis, framing, and windowing.

[0009] Step S2: Extract dual-stream fusion features. Two complementary features are extracted in parallel from the preprocessed audio signal: Frequency domain feature flow: Calculates the Mel spectrum of a signal to characterize the frequency domain structure and auditory perception characteristics of an audio signal.

[0010] Temporal Feature Flow: The normalized autocorrelation function of the signal is calculated, and deep temporal periodic features are extracted through a lightweight one-dimensional convolutional network ACgramNet. The autocorrelation function is used to characterize the similarity of the signal under different time delays, so as to highlight the inherent repetition period and rhythmic pattern of the siren sound.

[0011] The extracted frequency domain features and deep time domain periodic features are concatenated along the channel dimension to obtain time-frequency dual-stream fusion features.

[0012] Step S3: Construct and train a lightweight classification network. Input the dual-stream fusion features obtained in Step S2 into a lightweight classification network optimized for mobile devices. This network uses MobileFaceNet or its variants as the backbone and outputs classification results for various siren event categories and background noise. Train this classification network end-to-end using a pre-built dataset.

[0013] Step S4: Real-time detection and recognition. Deploy the trained lightweight classification network to the vehicle computing platform, and repeat steps S1 to S3 on the real-time input audio stream to achieve real-time, high-precision detection and recognition of emergency vehicle sirens in the surrounding environment.

[0014] Furthermore, in step S2, the formula for calculating the normalized autocorrelation function R(τ) of the original audio signal x(n) is as follows:

[0015] Where N is the signal length and τ is the time delay. This function causes the correlation of periodic signals to form a significant peak at integer multiples of the period, while the correlation of random noise decays rapidly as τ increases, thus highlighting the periodic information of the signal from a strong noise background.

[0016] Furthermore, the ACgramNet network in step S2 has the following structure in sequence: A large kernel one-dimensional convolutional layer, whose kernel size and stride correspond to the frame window size and frame shift stride when extracting the Mel spectrum in step S2, to ensure dimensionality matching between temporal and frequency domain features; Multiple stacked one-dimensional convolutional blocks, each containing layer normalization, a LeakyReLU activation function, and a small kernel one-dimensional convolutional layer.

[0017] Furthermore, in step S1, the preprocessing parameters for the audio signal are: sampling rate 16000Hz, frame length 32000, and frame shift 8000. In step S2, the extraction parameters for the Mel spectrogram are: number of Mel filters 64, and Fourier transform points 512.

[0018] Furthermore, in step S3, the lightweight classification network MobileFaceNet has its input layer modified to accept fused features from two channels (Mel spectrogram channel and time-domain periodic feature channel), and its output layer modified to correspond to five categories (including, for example, police cars, fire trucks, ambulances, emergency rescue vehicles, and background noise).

[0019] A siren sound detection system, characterized in that it comprises: Audio preprocessing unit: used to acquire audio signals and perform preprocessing.

[0020] Dual-stream feature extraction unit: used to extract Mel-spectral frequency domain features and autocorrelation function-based deep time domain periodic features of audio signals in parallel, and perform feature fusion.

[0021] Lightweight classification unit: Built-in pre-trained MobileFaceNet model, used to receive fused features and output classification results.

[0022] The result output and decision interface unit is used to output the classification results to the autonomous driving decision system, providing a basis for path planning and vehicle control.

[0023] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the method described in any of the preceding descriptions.

[0024] A computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the method described in any of the preceding claims. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0026] Figure 2 This is a comparison chart of the autocorrelation function curves of four different types of emergency vehicle siren signals in this invention.

[0027] Figure 3 This is a schematic diagram of the time-frequency dual-stream feature fusion and network structure in this invention.

[0028] Figure 4 This is a graph showing the performance curves (AUC, F1, loss, accuracy) of the model training process in an embodiment of the present invention. Detailed Implementation

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. These embodiments are implemented based on the technical solution of the present invention, but the scope of protection of the present invention is not limited to the following embodiments.

[0030] Step S1: Audio signal acquisition and preprocessing In this embodiment, a four-channel linear microphone array installed at the rear of the vehicle is used to collect road environment audio, with a sampling rate set to 16kHz. The collected single-channel audio signals are then processed sequentially as follows: Pre-emphasis: Using a first-order high-pass filter H(z)=1 0.97z 1. Compensate for high-frequency attenuation.

[0031] Framing: The signal is divided into short frames with a frame length of 25ms (i.e., 400 sampling points) and a frame shift of 10ms (i.e., 160 sampling points).

[0032] Windowing: Apply a Hamming window to each frame of the signal to reduce spectral leakage.

[0033] Step S2: Extract time-frequency dual-stream fusion features like Figure 1 Process and Figure 3 As shown in the network structure, this step extracts two features in parallel.

[0034] Frequency domain feature flow (Mel spectrum): A Fast Fourier Transform (FFT) with 512 points is performed on each frame of the preprocessed signal to obtain the amplitude spectrum. This amplitude spectrum is then passed through a set of triangular Mel filter banks (64 filters), and the logarithm of the output energy of each filter is taken to finally form a Mel spectrum Fs with a dimension of time frame number × 64.

[0035] Temporal feature stream (autocorrelation deep features): The normalized autocorrelation function R(τ) is calculated for the original audio signal x(n) (length N) according to formula (4) to obtain a one-dimensional sequence.

[0036] R(τ) is input into the ACgramNet network of this invention. The network structure is described below: The first step is a large-kernel one-dimensional convolutional layer (1 input channel, 128 output channels, kernel size 400, stride 160), with an output dimension of time frame number × 128. This design matches the frame shift and frame length of the Mel spectrum, ensuring dimensionality consistency in subsequent fusion.

[0037] Then, three one-dimensional convolutional blocks are stacked, each block containing layer normalization, LeakyReLU activation, and a small kernel one-dimensional convolution (128 input / output channels, kernel size 3).

[0038] Finally, a convolutional layer is used to adjust the number of channels to 64, resulting in the deep temporal periodic feature FA, whose dimension is exactly the same as the Mel spectrogram Fs, which is the number of time frames × 64.

[0039] Feature fusion: combining log-Mel spectral features Fs and deep temporal periodic features F A The data is concatenated along the channel dimension to obtain the final time-frequency dual-stream fusion feature F. SA Its dimension is time frame number × 64 × 2 (if time and frequency are considered as two dimensions, the number of channels is 2, but more accurately, the feature dimension is 2 × time frame number × 64). This fused feature simultaneously encodes the frequency domain auditory characteristics and the time domain periodic structure of the audio.

[0040] Step S3: Construct a lightweight classification network and train it. This embodiment selects MobileFaceNet as the classification backbone network. The following modifications are made for this task: Input layer adjustment: The original MobileFaceNet received 3-channel RGB images, but in this embodiment, it is modified to receive 2-channel feature maps (corresponding to F). SA ) Output layer adjustment: The original network was used for face recognition, and the output layer corresponded to the number of identity categories. In this embodiment, the output layer is modified to correspond to 5 categories: police car, fire truck, ambulance, emergency rescue vehicle, and background noise.

[0041] The core lightweight structures in the network, such as depthwise separable convolutions, bottleneck residual blocks, and global depthwise convolutions (GDConv), remain unchanged.

[0042] Training process: Dataset: A self-built real-world road scene dataset is used. This dataset contains four types of siren sounds recorded from multiple directions and distances at stationary and different vehicle speeds (10-50km / h), and real environmental noise such as wind noise and tire noise is mixed in.

[0043] Training configuration: Adam optimizer is used, learning rate is initialized to 0.001, training epochs are 32, and cross-entropy loss is used as the loss function.

[0044] Training results: such as Figure 4 As shown, the model converges rapidly during training, with both training and validation losses decreasing synchronously, and no significant overfitting is observed. The final model achieves a best-case accuracy of 99.36% on the test set, with a weighted AUC of 0.9999 and a weighted F1 score of 0.9936, while requiring only 2.60 GOPS of computation.

[0045] Step S4: Real-time detection and recognition After training, the models (including ACgramNet and MobileFaceNet) are deployed to the SA8295PS in-vehicle embedded device. The following loop is executed on the real-time audio stream: Capture a 2-second audio clip.

[0046] Repeat steps S1 and S2 to extract time-frequency dual-stream fusion features.

[0047] Input the features into the model, perform forward inference, and output the probability of the current audio segment belonging to each category.

[0048] When the probability of a certain siren category exceeds a preset threshold, the result and the event occurrence time information are sent to the autonomous driving decision-making and planning module via an interface to trigger the priority yielding strategy.

Claims

1. A lightweight siren sound detection method based on time-frequency dual-stream feature fusion, characterized in that, Includes the following steps: Step S1: Acquire audio signals and perform preprocessing; Step S2: Extract the frequency domain feature stream and time domain feature stream of the preprocessed audio signal in parallel and fuse them; wherein, the time domain feature stream is extracted based on the normalized autocorrelation function and the lightweight one-dimensional convolutional network ACgramNet, and is used to characterize the periodicity and rhythmic pattern of the sound signal. Step S3: Construct a lightweight classification network with MobileFaceNet as the backbone, train it using the fusion features obtained in step S2, and output the classification results for various siren events and background noise. Step S4: Deploy the trained model on the vehicle platform to detect and recognize real-time audio streams.

2. The method according to claim 1, characterized in that, In step S2, the formula for calculating the normalized autocorrelation function R(τ) is as follows: Where x(n) is the audio signal, N is the signal length, and τ is the time delay.

3. The method according to claim 1, characterized in that, The ACgramNet network in step S2 specifically includes: a large kernel one-dimensional convolutional layer for initial feature mapping and dimension matching; and multiple one-dimensional convolutional blocks connected in sequence, each convolutional block containing layer normalization, LeakyReLU activation function and a small kernel one-dimensional convolutional layer.

4. The method according to claim 1, characterized in that, The frequency domain feature flow in step S2 is a Mel spectrogram, and its extraction process includes: performing a short-time Fourier transform on the preprocessed signal, passing the obtained spectrum through a set of Mel-scale triangular filter banks, and taking the logarithmic energy of the filter output.

5. The method according to claim 1, characterized in that, The feature fusion in step S2 is a channel-dimensional concatenation, and the fused feature dimension simultaneously includes the frequency domain information of the Mel spectrum and the depth time-domain periodic information extracted by ACgramNet.

6. The method according to claim 1, characterized in that, In step S3, the input layer of the MobileFaceNet network is modified to receive the number of channels of the fused features, and its output layer is modified to correspond to the 5 classification categories.

7. A lightweight siren sound detection system based on time-frequency dual-stream feature fusion, characterized in that, The method for implementing any one of claims 1 to 6 includes: an audio preprocessing unit for implementing step S1; a dual-stream feature extraction unit for implementing step S2; a lightweight classification and training unit for implementing step S3; and a real-time detection and output unit for implementing step S4.

8. The system according to claim 7, characterized in that, The dual-stream feature extraction unit specifically includes: a frequency domain feature extraction module for calculating the Mel spectrogram of the audio; a time domain periodic feature extraction module for calculating the normalized autocorrelation function of the audio and extracting deep periodic features through the built-in ACgramNet network; and a feature fusion module for concatenating the two types of features in the channel dimension.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the rights are realized.

Images