Street crash vehicle sound detection method based on heterogeneous receptive field and global temporal dependency

Abstract

This application discloses a method for detecting street-racing vehicle sounds based on heterogeneous receptive fields and global temporal dependencies, belonging to the field of audio detection technology. The method includes: constructing a feature map of street-racing soundprints based on the audio stream of the road environment; extracting heterogeneous street-racing features in parallel from the feature map using a heterogeneous receptive field parallel aggregation network; establishing full-temporal context features of street-racing sound events based on the heterogeneous street-racing features; performing interactive retrieval based on the event query vector and the full-temporal context features; and outputting the confidence, start timestamp, and end timestamp of the street-racing event based on the retrieval results using parallel prediction branch regression. This application utilizes full-temporal context features to repair broken features obscured by noise, and effectively avoids false positives and false negatives in complex sound scenes through joint optimization in the time and frequency domains, significantly improving the recognition accuracy of street-racing vehicle audio.

Description

Technical Field

[0001] This application relates to the field of audio detection technology, and in particular to a method for detecting the sound of street-racing vehicles based on heterogeneous receptive fields and global temporal dependence. Background Technology

[0002] Traffic noise pollution is becoming an increasingly prominent issue in urban environmental management. Besides the usual honking of vehicles, the roaring engines and explosive exhaust sounds produced by vehicles that "street race" have become a major source of complaints from residents. Street racing often involves illegal modifications to vehicle exhaust pipes (such as removing mufflers or installing straight exhaust pipes) or creating high-decibel noise through rapid acceleration on urban roads. This noise is highly penetrating and sudden, severely disrupting residents' normal work and rest, and is often accompanied by speeding, posing a significant traffic safety hazard.

[0003] Current off-site enforcement and supervision of vehicles that cause disturbances in urban areas mainly rely on decibel threshold detection using sound level meters or simple spectral template matching. These methods fail to fully consider the complex characteristics of disturbance sounds in the time and frequency domains, and their accuracy and anti-interference capabilities face bottlenecks when dealing with complex urban soundscapes and dynamic traffic flows. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method and related equipment for detecting street noise from vehicles based on heterogeneous receptive fields and global temporal dependencies, so as to improve the accuracy of street noise detection.

[0005] One aspect of this application provides a method for detecting loud noise from vehicles in public areas based on heterogeneous receptive fields and global temporal dependence. The method includes the following steps:

[0006] A feature map of the soundprint of street rage is constructed based on the audio stream of the road environment;

[0007] Heterogeneous street rage features are extracted in parallel from the feature map using a heterogeneous receptive field parallel aggregation network.

[0008] Based on the heterogeneous street blasting characteristics, establish full-time contextual features of street blasting sound events;

[0009] Interactive retrieval is performed based on the event query vector and the full-time-domain context features;

[0010] Based on the search results of the interactive search, the confidence level, start timestamp, and end timestamp of the street rioting event are output using parallel predictive branch regression.

[0011] In some embodiments, constructing a feature map of the loudspeaker's voiceprint based on the audio stream of the road environment includes the following steps:

[0012] Real-time audio streams of the road environment are collected and subjected to short-time time-frequency transformation to generate a two-dimensional spectrogram;

[0013] A frequency domain truncation strategy is applied to the two-dimensional acoustic spectrogram to discard high-frequency components above a preset threshold and retain the target frequency band where the exhaust noise of street-racing vehicles is concentrated.

[0014] The target frequency band is mapped to the Mel frequency domain space and a logarithmic amplitude transformation is performed to generate a logarithmic Mel spectrogram as the feature map of the street blasting soundprint.

[0015] In some embodiments, the method of extracting heterogeneous street-rage features from the feature map in parallel using a heterogeneous receptive field parallel aggregation network includes the following steps:

[0016] The feature map is input into the heterogeneous receptive field parallel aggregation network. In the feature extraction stage, the heterogeneous receptive field branch structure is used to analyze the acoustic spectrum features through receptive fields of different scales to obtain the heterogeneous street blasting features. Among them, the large-scale receptive field branch is used to capture the macroscopic frequency shift trajectory features reflecting the vehicle acceleration behavior, and the small-scale receptive field branch is used to capture the microscopic high-energy-level spectral density features reflecting exhaust resonance and backfire behavior.

[0017] In some embodiments, the heterogeneous receptive field parallel aggregation network consists of several cascaded feature extraction units. Each feature extraction unit adopts a multi-scale channel segmentation and cascade fusion structure. The feature extraction unit is used to segment the feature map into multiple feature subsets in the channel dimension and process them through cascade interaction.

[0018] Among them, the basic feature subset is processed by convolution or identity mapping to maintain the preset equivalent receptive field, and is used to extract the local acoustic texture features of high energy level spectral density in the two-dimensional acoustic spectrogram in order to identify the energy structure of harmonic stripes and the sudden pulses generated by tempering.

[0019] The cascaded feature subset is used to perform superimposed convolution using the output features of the preceding subset. Through the accumulated convolution effect, a large-scale equivalent receptive field is constructed to extract macroscopic frequency shift trajectory features across long time periods in the two-dimensional spectrogram, so as to identify the overall evolution trend of the acceleration process of street-racing vehicles.

[0020] The outputs of each feature subset are concatenated and restored along the channel dimension to obtain the heterogeneous street rage features.

[0021] In some embodiments, establishing the full-temporal context features of the street blasting sound event based on the heterogeneous street blasting features includes the following steps:

[0022] The heterogeneous street rioting features are serialized, flattened, and embedded with positional encoding to generate a feature sequence containing temporal information.

[0023] The feature sequence is input into the global temporal dependency modeling module, and the long-range correlation weights between feature frames are calculated using the global self-attention mechanism.

[0024] The full-time context features of the street noise event are established based on the long-range correlation weights to characterize the inherent temporal continuity of the street noise behavior.

[0025] In some embodiments, the global temporal dependency modeling module adopts a Transformer encoder-based architecture, including a position encoding unit and an encoder unit;

[0026] The location encoding unit is used to embed relative or absolute location information into the aggregated feature sequence to supplement the temporal location features.

[0027] The encoder unit is used to calculate the long-range correlation weights within the feature sequence using a multi-head self-attention mechanism, and then establish the full-time context features based on the long-range correlation weights.

[0028] In some embodiments, the interactive retrieval based on the event query vector and the full-time context features includes the following steps:

[0029] The event query vector is introduced by the decoder unit in the Transformer encoder architecture. The street riot event features are retrieved in parallel in the full temporal context features through a multi-head mutual attention mechanism, and then an output sequence including event category and temporal boundary information is generated as the retrieval result.

[0030] The retrieval results based on the interactive retrieval utilize parallel predictive branch regression to output the confidence level, start timestamp, and end timestamp of the street rioting event, including the following steps:

[0031] Based on the output sequence, parallel predictive branch regression is used to output multiple disordered time-domain prediction segments; wherein each time-domain prediction segment includes the confidence level of the street riot event and normalized start and end time coordinates.

[0032] Another aspect of this application embodiment provides a device for detecting the noise of street-racing vehicles based on heterogeneous receptive fields and global temporal dependence, the device comprising:

[0033] The feature map construction unit is used to construct the feature map of the street rage soundprint based on the audio stream of the road environment.

[0034] The feature extraction unit is used to extract heterogeneous street riot features from the feature map in parallel using a heterogeneous receptive field parallel aggregation network;

[0035] The context establishment unit is used to establish the full-time context features of the street blasting sound event based on the heterogeneous street blasting features;

[0036] An interactive retrieval unit is used to perform interactive retrieval based on the event query vector and the full-time-domain context features;

[0037] The street riot detection unit is used to output the confidence level, start timestamp, and end timestamp of the street riot event based on the retrieval results of the interactive retrieval using parallel predictive branch regression.

[0038] Another aspect of this application embodiment provides an electronic device, including a processor and a memory;

[0039] The memory is used to store programs;

[0040] The processor executes the program to implement any of the methods described above.

[0041] Another aspect of this application provides a computer-readable storage medium storing a program that is executed by a processor to implement the method described in any of the above embodiments.

[0042] This application includes at least the following beneficial effects:

[0043] This application constructs a feature map of street rage soundprints based on the audio stream of the road environment; extracts heterogeneous street rage features in parallel from the feature map using a heterogeneous receptive field parallel aggregation network; establishes full-temporal context features of street rage sound events based on the heterogeneous street rage features; performs interactive retrieval based on the event query vector and the full-temporal context features; and outputs the confidence, start timestamp, and end timestamp of the street rage event based on the retrieval results using parallel prediction branch regression. This application uses full-temporal context features to repair the broken features obscured by noise, and effectively avoids false alarms and false negatives in complex sound scenes through joint optimization in the time and frequency domains, significantly improving the recognition accuracy of street rage vehicle audio. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart illustrating the method for detecting the noise of street-racing vehicles based on heterogeneous receptive fields and global temporal dependence provided in this application embodiment;

[0046] Figure 2An example flowchart of a method for detecting the sound of vehicles ramming in the streets based on heterogeneous receptive field parallel aggregation and global temporal dependency modeling, provided in an embodiment of this application;

[0047] Figure 3 A schematic diagram illustrating the principle of the frequency domain truncation strategy and spectrogram construction provided in the embodiments of this application;

[0048] Figure 4 This is a schematic diagram illustrating the structural principle of the heterogeneous receptive field parallel aggregation network provided in the embodiments of this application;

[0049] Figure 5 A schematic diagram of the logical architecture of the global temporal dependency modeling module and end-to-end set prediction provided in the embodiments of this application;

[0050] Figure 6 The structural block diagram of the street noise detection device based on heterogeneous receptive field and global temporal dependence provided in the embodiments of this application is shown. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] Before providing a detailed description of the embodiments of this application, some related technologies involved in the embodiments of this application will be described first, as follows:

[0053] Specifically, the existing design has the following problems:

[0054] 1. One-sided feature extraction: Macroscopically, the rapid acceleration of street racing vehicles is manifested by a large upward frequency shift across multiple high-energy spectral lines in the acoustic spectrogram; microscopically, illegally modified exhausts cause abnormal accumulation of specific order harmonic energy, accompanied by broadband energy jumps caused by deceleration backfire. Existing single-scale convolutional networks struggle to learn multiple different spectral features simultaneously, resulting in incomplete capture of key features or blurred details.

[0055] 2. Lack of temporal correlation modeling: Street racing is a sparse sample, but the sound signal is continuous and persistent. Existing audio classification algorithms mostly use slicing, cutting the continuous audio stream into independent short segments for discrete judgment. This method ignores the inherent temporal dependencies of events, and when the street racing sound is briefly masked by other sounds, it is very easy to misjudge the complete event as interrupted or ended.

[0056] 3. Low localization accuracy: Traditional classification methods face a granularity dilemma: long-window analysis can guarantee classification accuracy but results in coarse temporal localization; short-window analysis improves temporal resolution but leads to decreased accuracy due to missing features. Existing technologies struggle to simultaneously achieve high recognition rates and millisecond-level temporal localization.

[0057] 4. Lack of multi-source decoupling capability: Traditional sound classification algorithms can only identify whether the audio within a window is from street racing, and have shortcomings in determining the number of sound sources. Therefore, it is difficult to handle the acoustic signal aliasing scenarios of street racing or parallel driving, and cannot provide clear data support for subsequent sound source localization and accurate capture.

[0058] Reference Figure 1 This application provides a method for detecting loud noise from vehicles in public areas based on heterogeneous receptive fields and global temporal dependencies, specifically including the following steps S100~S140:

[0059] S100: Construct a feature map of the soundprint of street rage based on the audio stream of the road environment;

[0060] S110: Heterogeneous street blasting features are extracted in parallel from the feature map using a heterogeneous receptive field parallel aggregation network;

[0061] S120: Establish full-time context features of the street noise event based on the heterogeneous street noise characteristics;

[0062] S130: Perform interactive retrieval based on the event query vector and the full-time-domain context features;

[0063] S140: Based on the search results of the interactive retrieval, the confidence level, start timestamp, and end timestamp of the street rioting event are output using parallel predictive branch regression.

[0064] Optionally, constructing the feature map of the loudspeaker soundprint based on the audio stream of the road environment includes the following steps:

[0065] Real-time audio streams of the road environment are collected and subjected to short-time time-frequency transformation to generate a two-dimensional spectrogram;

[0066] A frequency domain truncation strategy is applied to the two-dimensional acoustic spectrogram to discard high-frequency components above a preset threshold and retain the target frequency band where the exhaust noise of street-racing vehicles is concentrated.

[0067] The target frequency band is mapped to the Mel frequency domain space and a logarithmic amplitude transformation is performed to generate a logarithmic Mel spectrogram as the feature map of the street blasting soundprint.

[0068] Optionally, the method of extracting heterogeneous street-rage features from the feature map in parallel using a heterogeneous receptive field parallel aggregation network includes the following steps:

[0069] The feature map is input into the heterogeneous receptive field parallel aggregation network. In the feature extraction stage, the heterogeneous receptive field branch structure is used to analyze the acoustic spectrum features through receptive fields of different scales to obtain the heterogeneous street blasting features. Among them, the large-scale receptive field branch is used to capture the macroscopic frequency shift trajectory features reflecting the vehicle acceleration behavior, and the small-scale receptive field branch is used to capture the microscopic high-energy-level spectral density features reflecting exhaust resonance and backfire behavior.

[0070] Optionally, the heterogeneous receptive field parallel aggregation network consists of several cascaded feature extraction units. Each feature extraction unit adopts a multi-scale channel segmentation and cascade fusion structure. The feature extraction unit is used to segment the feature map into multiple feature subsets in the channel dimension and process them through cascade interaction.

[0071] Among them, the basic feature subset is processed by convolution or identity mapping to maintain the preset equivalent receptive field, and is used to extract the local acoustic texture features of high energy level spectral density in the two-dimensional acoustic spectrogram in order to identify the energy structure of harmonic stripes and the sudden pulses generated by tempering.

[0072] The cascaded feature subset is used to perform superimposed convolution using the output features of the preceding subset. Through the accumulated convolution effect, a large-scale equivalent receptive field is constructed to extract macroscopic frequency shift trajectory features across long time periods in the two-dimensional spectrogram, so as to identify the overall evolution trend of the acceleration process of street-racing vehicles.

[0073] The outputs of each feature subset are concatenated and restored along the channel dimension to obtain the heterogeneous street rage features.

[0074] Optionally, establishing the full-temporal context features of the street blasting sound event based on the heterogeneous street blasting features includes the following steps:

[0075] The heterogeneous street rioting features are serialized, flattened, and embedded with positional encoding to generate a feature sequence containing temporal information.

[0076] The feature sequence is input into the global temporal dependency modeling module, and the long-range correlation weights between feature frames are calculated using the global self-attention mechanism.

[0077] The full-time context features of the street noise event are established based on the long-range correlation weights to characterize the inherent temporal continuity of the street noise behavior.

[0078] Optionally, the global temporal dependency modeling module adopts a Transformer encoder-based architecture, including a position encoding unit and an encoder unit;

[0079] The location encoding unit is used to embed relative or absolute location information into the aggregated feature sequence to supplement the temporal location features.

[0080] The encoder unit is used to calculate the long-range correlation weights within the feature sequence using a multi-head self-attention mechanism, and then establish the full-time context features based on the long-range correlation weights.

[0081] Optionally, the interactive retrieval based on the event query vector and the full-time-domain context features includes the following steps:

[0082] The event query vector is introduced by the decoder unit in the Transformer encoder architecture. The street riot event features are retrieved in parallel in the full temporal context features through a multi-head mutual attention mechanism, and then an output sequence including event category and temporal boundary information is generated as the retrieval result.

[0083] The retrieval results based on the interactive retrieval utilize parallel predictive branch regression to output the confidence level, start timestamp, and end timestamp of the street rioting event, including the following steps:

[0084] Based on the output sequence, parallel predictive branch regression is used to output multiple disordered time-domain prediction segments; wherein each time-domain prediction segment includes the confidence level of the street riot event and normalized start and end time coordinates.

[0085] The following section will provide a detailed introduction and explanation of the solutions in the embodiments of this application, using specific application examples.

[0086] Reference Figure 2 This embodiment provides a method for detecting loud noise from vehicles in public areas based on heterogeneous receptive field parallel aggregation and global temporal dependency modeling, including the following steps:

[0087] S1. Constructing the sound signature feature map of street racing vehicles: Collect real-time audio streams of the road environment, perform short-time time-frequency transformation to generate a two-dimensional sound spectrum map; implement a frequency domain truncation strategy, discard high-frequency components above the preset threshold, retain the mid-low frequency band where the exhaust noise of street racing vehicles is concentrated, and map it to the Mel frequency domain space and perform logarithmic amplitude transformation to generate a logarithmic Mel sound spectrum map as the input feature map.

[0088] S2. Parallel Extraction of Heterogeneous Street Racing Features: The input feature map is fed into a heterogeneous receptive field parallel aggregation network. In the feature extraction stage, the heterogeneous receptive field branch structure is used to analyze the spectrogram features through receptive fields of different scales. The large-scale receptive field branch is used to capture the macroscopic frequency shift trajectory features reflecting the vehicle's acceleration behavior, while the small-scale receptive field branch is used to capture the microscopic high-energy spectral density features reflecting exhaust resonance and backfire behavior, so as to achieve complementary expression of the dynamic evolution and static structure of street racing sound patterns.

[0089] S3. Global Temporal Dependency Modeling: The heterogeneous street rioting features are serialized, flattened, and embedded with positional encoding to generate a feature sequence containing temporal information. This sequence is then input into the global temporal dependency modeling module, which uses a global self-attention mechanism to calculate the long-range correlation weights between feature frames. This establishes the full-temporal context features of street rioting sound events to characterize the inherent temporal continuity of street rioting behavior and ensure the integrity of long-term sound event detection.

[0090] S4. End-to-end street rioting event determination: Introducing event query vectors and full-time domain context features for interactive retrieval, and using parallel prediction branches to directly regress and output the confidence, start timestamp and end timestamp of the street rioting event based on the retrieval results.

[0091] Further, the frequency domain truncation strategy in step S1 is as follows: A frequency threshold of 2048Hz is set, and frequency domain truncation is implemented, directly discarding wind noise and irrelevant high-frequency environmental noise above this threshold, retaining only the acoustic spectrum energy within the 0Hz to 2048Hz frequency band; subsequently, a Mel filter bank is used to map the retained acoustic spectrum energy to the Mel frequency standard space, and logarithmic operations are performed to convert the amplitude to a decibel scale, ultimately generating a logarithmic Mel spectrogram. For example, Figure 3 A schematic diagram illustrating the principles of frequency domain truncation strategy and spectrogram construction.

[0092] Furthermore, the heterogeneous receptive field parallel aggregation network in step S2 consists of several cascaded feature extraction units. Each feature extraction unit adopts a multi-scale channel segmentation and cascaded fusion structure. The unit segments the input feature map into multiple feature subsets in the channel dimension and processes them through cascaded interaction. The basic feature subsets are processed by fewer layers of convolution or identity mapping to maintain a small equivalent receptive field, which is used to extract local acoustic texture features of high-energy spectral density in the two-dimensional spectrogram to identify the energy structure of harmonic stripes and sudden pulses generated by tempering. The cascaded feature subsets use the output features of the preceding subsets for superposition convolution, and construct a large-scale equivalent receptive field through the accumulated convolution effect to extract macroscopic frequency shift trajectory features across long time periods in the two-dimensional spectrogram to identify the overall evolution trend of the acceleration process of street-racing vehicles. The outputs of each feature subset are spliced ​​and restored in the channel dimension. For example, Figure 4 This is a schematic diagram illustrating the structural principle of a heterogeneous receptive field parallel aggregation network (which employs a cascaded channel segmentation and feature reuse mechanism).

[0093] Furthermore, the global temporal dependency modeling module in step S3 is based on the Transformer encoder architecture and mainly includes a position encoding unit and an encoder unit. The position encoding unit is used to embed relative or absolute position information into the aggregated feature sequence to supplement the temporal position features. The encoder unit uses a multi-head self-attention mechanism to calculate the long-range correlation weights within the feature sequence and establish the context logic of the entire temporal domain.

[0094] Furthermore, the end-to-end street rage event determination in step S4 is implemented based on a Transformer decoder architecture, including decoder units and event query vectors. The decoder unit introduces a set of learnable event query vectors and retrieves street rage event features in parallel from the context features output in step S3 through a multi-head mutual attention mechanism, generating an output sequence containing event category and temporal boundary information. The street rage determination adopts an output method based on ensemble prediction, directly outputting a set of unordered temporal prediction segments, each prediction segment containing the confidence level of the street rage event and normalized start and end time coordinates. For example, Figure 5 A schematic diagram of the logical architecture for the global temporal dependency modeling module and end-to-end set prediction.

[0095] This embodiment also provides a street noise detection system based on heterogeneous receptive field parallel aggregation and global temporal dependency modeling, including:

[0096] Audio acquisition unit: Located in the monitoring area, it is used to acquire the raw audio stream of the road traffic environment in real time and convert the analog signal into a digital audio signal;

[0097] Intelligent detection terminal: connected to the audio acquisition unit, internally storing a computer program, which implements the method of this embodiment when executed;

[0098] The intelligent detection terminal constructs feature maps, extracts heterogeneous features, and models global temporal sequences for the input digital audio signals, and determines in real time whether there are street blasting events in the audio stream.

[0099] Data output interface: Used to output the confidence level, start timestamp and end timestamp of the event as structured data.

[0100] An optional implementation method is as follows:

[0101] This embodiment provides a method for detecting loud noise from vehicles in street environments based on heterogeneous receptive field parallel aggregation and global temporal dependency modeling. This method combines global temporal dependency modeling with end-to-end ensemble prediction to address the difficulties of traditional methods in capturing long-distance dependencies and detecting overlapping sound sources in complex noise environments. The method specifically includes the following steps:

[0102] S1. Obtaining the Audio Acoustic Feature Map. First, acquire the urban environmental audio signal to be detected and resample it to a uniform sampling rate. The resampled audio signal is then framed and windowed, and a short-time Fourier transform is performed to convert the time-domain signal to the frequency-domain signal. Considering that the exhaust noise of modified vehicles is mainly concentrated in the low-frequency region, the frequency-domain signal is truncated, retaining only the effective frequency components below 2048Hz. Subsequently, the frequency-domain signal is mapped to the Mel frequency scale and the logarithm is taken to generate a two-dimensional logarithmic Mel spectrogram.

[0103] S2. Parallel Extraction and Sequencing of Heterogeneous Street Racing Features. A heterogeneous receptive field parallel aggregation network is constructed, consisting of several cascaded feature extraction units. Each feature extraction unit adopts a multi-scale channel segmentation and cascaded fusion structure. The two-dimensional log-Mel spectrogram generated in S1 is input into this network, which is segmented into multiple feature subsets along the channel dimension and processed in parallel through cascaded interaction: The basic feature subset is processed through fewer layers of convolution or identity mapping to maintain a small equivalent receptive field, used to extract local acoustic texture features of high-energy spectral density in the two-dimensional spectrogram to identify the static energy structure of harmonic stripes and the sudden pulses generated by exhaust backfire; the cascaded feature subsets use the output features of the preceding subsets for superposition convolution, and construct a large-scale equivalent receptive field through the accumulation of convolution effects, used to extract macroscopic frequency shift trajectory features across long time periods in the two-dimensional spectrogram to identify the overall frequency evolution trend of the acceleration process of street racing vehicles. The outputs of each feature subset are stitched together along the channel dimension to restore the aggregated feature map.

[0104] S3. Global Temporal Dependency Modeling. First, a feature serialization operation is performed on the aggregated feature map output from S2, flattening the two-dimensional feature map in space and converting it into a one-dimensional feature sequence. A positional encoding vector is generated, with the same dimension as the one-dimensional feature sequence, representing the temporal position information of each element in the sequence. The positional encoding vector is then element-wise added to the one-dimensional feature sequence to generate the encoder input sequence containing temporal information. A Transformer encoder is constructed, consisting of several stacked encoding layers, each containing a multi-head self-attention module and a feedforward neural network module. The encoder input sequence is fed into the Transformer encoder. In the multi-head self-attention module, the correlation weight matrix between any two time steps in the input sequence is calculated. Based on the weight matrix, the sequence features are weighted and aggregated to establish full-temporal feature dependencies. After processing through several encoding layers, the Transformer encoder outputs a feature vector containing global temporal context information, denoted as the global context feature.

[0105] S4. End-to-End Ensemble Prediction. A fixed number of learnable vectors are initialized as event query vectors, corresponding to the maximum number of events detected in a single run. A Transformer decoder is constructed, utilizing a mutual attention mechanism to interactively retrieve the event query vectors and the global context features output from S3: using the global context features as keys and values, and the event query vectors as queries, attention weights are calculated and the corresponding audio event features are aggregated to generate the decoded event feature vectors. Finally, the decoded event feature vectors are input into parallel classification and regression prediction heads, which are mapped through fully connected layers and multilayer perceptrons, respectively, directly outputting the event category confidence and normalized start and end time coordinates corresponding to each query vector.

[0106] In summary, this embodiment has the following advantages and beneficial effects compared to the prior art: This embodiment effectively solves the problem of missed detection caused by noise masking in long audio files by utilizing the global modeling capability of Transformer; through an end-to-end ensemble prediction architecture, it eliminates the complex anchor box design and cumbersome post-processing steps of traditional methods, significantly improving the simplicity of the detection process and the detection accuracy in overlapping sound source scenarios. Furthermore, the method in this embodiment has a clear structure and is easy to implement.

[0107] The beneficial effects of this embodiment include:

[0108] High recognition accuracy: Addressing the shortcomings of existing technologies in feature extraction being one-sided and lacking temporal correlation, this embodiment utilizes cascaded channel segmentation and feature reuse mechanisms in the spatial dimension, taking into account the complementary expression of macroscopic frequency shift trajectories and microscopic energy densities through convolutional accumulation effects; in the temporal dimension, it introduces global temporal dependency modeling, using full-time domain contextual features to repair fragmented features obscured by noise. This joint optimization in the time and frequency domains effectively avoids false alarms and false negatives in complex sound environments, significantly improving the recognition accuracy of street-racing vehicles.

[0109] Precise temporal boundary localization: This embodiment employs a global attention aggregation mechanism based on the Transformer decoder. This method directly outputs continuous time coordinate values, rather than discrete category labels, enabling fine-grained localization of the start and end times of street rioting events. This effectively solves the pain point of traditional methods, which can only vaguely determine the time period of an event and are unable to accurately capture transient boundaries.

[0110] Possessing multi-target decoupling capabilities and adapting to complex scenarios: This embodiment utilizes learnable event query vectors and mutual attention mechanisms to achieve independent modeling and quantitative counting of overlapping acoustic events. This enables the system to accurately distinguish the number of targets in aliased sound scenes of multi-vehicle racing, providing crucial data support for downstream sound source localization and image capture.

[0111] Reference Figure 6This application provides a device for detecting loud noise from vehicles in public areas based on heterogeneous receptive fields and global temporal dependence, including:

[0112] The feature map construction unit is used to construct the feature map of the street rage soundprint based on the audio stream of the road environment.

[0113] The feature extraction unit is used to extract heterogeneous street riot features from the feature map in parallel using a heterogeneous receptive field parallel aggregation network;

[0114] The context establishment unit is used to establish the full-time context features of the street blasting sound event based on the heterogeneous street blasting features;

[0115] An interactive retrieval unit is used to perform interactive retrieval based on the event query vector and the full-time-domain context features;

[0116] The street riot detection unit is used to output the confidence level, start timestamp, and end timestamp of the street riot event based on the retrieval results of the interactive retrieval using parallel predictive branch regression.

[0117] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0118] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0119] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional technology for an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0120] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0122] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0123] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0124] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0125] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0126] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for detecting street-racing vehicle noise based on heterogeneous receptive fields and global temporal dependence, characterized in that, The method includes the following steps: A feature map of the soundprint of street rage is constructed based on the audio stream of the road environment; Heterogeneous street rage features are extracted in parallel from the feature map using a heterogeneous receptive field parallel aggregation network. Based on the heterogeneous street blasting characteristics, establish full-time contextual features of street blasting sound events; Interactive retrieval is performed based on the event query vector and the full-time-domain context features; Based on the search results of the interactive search, the confidence level, start timestamp, and end timestamp of the street rioting event are output using parallel predictive branch regression.

2. The method for detecting street-racing vehicle sounds based on heterogeneous receptive fields and global temporal dependence according to claim 1, characterized in that, The process of constructing a feature map of the "street rage" soundprint based on the audio stream of the road environment includes the following steps: Real-time audio streams of the road environment are collected and subjected to short-time time-frequency transformation to generate a two-dimensional spectrogram; A frequency domain truncation strategy is applied to the two-dimensional acoustic spectrogram to discard high-frequency components above a preset threshold and retain the target frequency band where the exhaust noise of street-racing vehicles is concentrated. The target frequency band is mapped to the Mel frequency domain space and a logarithmic amplitude transformation is performed to generate a logarithmic Mel spectrogram as the feature map of the street blasting soundprint.

3. The method for detecting street-racing vehicle noise based on heterogeneous receptive fields and global temporal dependence according to claim 1, characterized in that, The method of extracting heterogeneous street-rage features from the feature map in parallel using a heterogeneous receptive field parallel aggregation network includes the following steps: The feature map is input into the heterogeneous receptive field parallel aggregation network. In the feature extraction stage, the heterogeneous receptive field branch structure is used to analyze the acoustic spectrum features through receptive fields of different scales to obtain the heterogeneous street blasting features. Among them, the large-scale receptive field branch is used to capture the macroscopic frequency shift trajectory features reflecting the vehicle acceleration behavior, and the small-scale receptive field branch is used to capture the microscopic high-energy-level spectral density features reflecting exhaust resonance and backfire behavior.

4. The method for detecting street-racing vehicle sounds based on heterogeneous receptive fields and global temporal dependence according to claim 3, characterized in that, The heterogeneous receptive field parallel aggregation network consists of several cascaded feature extraction units. Each feature extraction unit adopts a multi-scale channel segmentation and cascade fusion structure. The feature extraction unit is used to segment the feature map into multiple feature subsets in the channel dimension and process them through cascade interaction. Among them, the basic feature subset is processed by convolution or identity mapping to maintain the preset equivalent receptive field, and is used to extract the local acoustic texture features of high energy level spectral density in the two-dimensional acoustic spectrogram in order to identify the energy structure of harmonic stripes and the sudden pulses generated by tempering. The cascaded feature subset is used to perform superimposed convolution using the output features of the preceding subset. Through the accumulated convolution effect, a large-scale equivalent receptive field is constructed to extract macroscopic frequency shift trajectory features across long time periods in the two-dimensional spectrogram, so as to identify the overall evolution trend of the acceleration process of street-racing vehicles. The outputs of each feature subset are concatenated and restored along the channel dimension to obtain the heterogeneous street rage features.

5. The method for detecting street-racing vehicle sounds based on heterogeneous receptive fields and global temporal dependence according to claim 1, characterized in that, The step of establishing the full-temporal context features of street blasting sound events based on the heterogeneous street blasting features includes the following steps: The heterogeneous street rioting features are serialized, flattened, and embedded with positional encoding to generate a feature sequence containing temporal information. The feature sequence is input into the global temporal dependency modeling module, and the long-range correlation weights between feature frames are calculated using the global self-attention mechanism. The full-time context features of the street noise event are established based on the long-range correlation weights to characterize the inherent temporal continuity of the street noise behavior.

6. The method for detecting street-racing vehicle sounds based on heterogeneous receptive fields and global temporal dependence according to claim 5, characterized in that, The global temporal dependency modeling module adopts a Transformer encoder-based architecture, including a position encoding unit and an encoder unit; The location encoding unit is used to embed relative or absolute location information into the aggregated feature sequence to supplement the temporal location features. The encoder unit is used to calculate the long-range correlation weights within the feature sequence using a multi-head self-attention mechanism, and then establish the full-time context features based on the long-range correlation weights.

7. The method for detecting street-racing vehicle sounds based on heterogeneous receptive fields and global temporal dependence according to claim 6, characterized in that, The interactive retrieval based on the event query vector and the full-time context features includes the following steps: The event query vector is introduced by the decoder unit in the Transformer encoder architecture. The street riot event features are retrieved in parallel in the full temporal context features through a multi-head mutual attention mechanism, and then an output sequence including event category and temporal boundary information is generated as the retrieval result. The retrieval results based on the interactive retrieval utilize parallel predictive branch regression to output the confidence level, start timestamp, and end timestamp of the street rioting event, including the following steps: Based on the output sequence, parallel predictive branch regression is used to output multiple disordered time-domain prediction segments; wherein each time-domain prediction segment includes the confidence level of the street riot event and normalized start and end time coordinates.

8. A device for detecting the sound of vehicles revving in public areas based on heterogeneous receptive fields and global temporal dependence, characterized in that, The device includes: The feature map construction unit is used to construct the feature map of the street rage soundprint based on the audio stream of the road environment. The feature extraction unit is used to extract heterogeneous street riot features from the feature map in parallel using a heterogeneous receptive field parallel aggregation network; The context establishment unit is used to establish the full-time context features of the street blasting sound event based on the heterogeneous street blasting features; An interactive retrieval unit is used to perform interactive retrieval based on the event query vector and the full-time-domain context features; The street riot detection unit is used to output the confidence level, start timestamp, and end timestamp of the street riot event based on the retrieval results of the interactive retrieval using parallel predictive branch regression.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the method as described in any one of claims 1 to 7.

Images