Automotive abnormal noise data annotation system and annotation method
By combining time-frequency analysis and automatic candidate detection with manual verification, the problem of low efficiency and low accuracy in labeling abnormal noises in automobiles has been solved, achieving efficient and accurate labeling of abnormal noise data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA AUTOMOTIVE ENG RES INST
- Filing Date
- 2026-03-04
- Publication Date
- 2026-05-26
AI Technical Summary
The current method of labeling abnormal noises in automobiles relies on manual listening, which results in low efficiency, low accuracy, and poor consistency, making it difficult to meet the needs of labeling large-scale abnormal noise data.
By employing time-frequency analysis, directional filtering, and automatic candidate detection, combined with intelligent algorithms, candidate segments for abnormal noises are quickly identified and presented synchronously through time-frequency graphs. Combined with manual verification, efficient and accurate labeling is achieved.
Significantly reduces the time required to annotate a single audio track, improves annotation efficiency by 65%, achieves an annotation accuracy rate of 97%, reduces the influence of personal experience, and ensures annotation consistency.
Smart Images
Figure CN122090870A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of automotive NVH testing technology, specifically to an automotive abnormal noise data annotation system and annotation method. Background Technology
[0002] With the rapid development of the automotive industry, NVH performance has become an important indicator for measuring vehicle quality. Among these, abnormal noises are one of the main factors affecting the driving experience, and their detection and diagnosis technologies have received high attention from the industry. Currently, deep learning methods are widely used in audio classification. However, when using deep learning methods to classify abnormal noise audio, it is necessary to first accurately label the abnormal noise regions in the audio.
[0003] The current mainstream method for annotating abnormal noises mainly relies on manual listening, where inspectors manually annotate abnormal noise ranges by listening to audio recordings. However, this manual method has many drawbacks: firstly, it is extremely inefficient, with the analysis of a single audio recording taking several minutes, making it difficult to meet the needs of annotating large-scale abnormal noise data; secondly, the annotation results heavily depend on the personal experience of the inspectors, and different inspectors may have significant differences in their judgment of the same abnormal noise, resulting in poor annotation consistency; furthermore, abnormal noise signals are transient and non-periodic, and are easily affected by environmental noise, making it difficult for manual listening to accurately capture these characteristics, resulting in low annotation accuracy.
[0004] To achieve efficient and accurate labeling of automotive abnormal noise audio, it is urgent to solve the problems of time-frequency joint analysis of abnormal noise signals and signal enhancement technology under low signal-to-noise ratio conditions, and to break through the limitations of traditional manual listening methods. Summary of the Invention
[0005] This invention aims to solve the technical problems of low efficiency, low accuracy, and poor consistency caused by the reliance on manual listening inspection for labeling abnormal noises in automobiles.
[0006] To address the aforementioned problems, this application first provides a method for labeling automotive abnormal noise data, comprising the following steps: Step 1: Receive raw signals. Receive raw signal data containing abnormal noises from the vehicle collected by the sensor during vehicle operation. Step 2: Time-frequency analysis, performing time-frequency transformation on the original signal data to generate a time-frequency graph; the time-frequency graph refers to a waveform diagram showing the changes in frequency and amplitude over time; Step 3: Determine the key frequency range. Based on the time-frequency diagram, identify the key frequency range that characterizes the abnormal noise of the vehicle. Step 4: Directional filtering. Based on the key frequency range, bandpass filtering is performed on the original signal data to enhance the abnormal noise signal and suppress background noise. Step 5: Automatic candidate detection. After filtering, the signal is automatically identified and the candidate time intervals of abnormal noise are marked by a preset detection algorithm. Step 6: Output labeled data. Output the time-frequency diagrams of the labeled abnormal sound time intervals and the key frequency intervals in a structured data format.
[0007] The advantages of this solution are: through automatic candidate detection, intelligent algorithms are used to quickly locate candidate segments of abnormal sounds, greatly reducing the scope of manual annotation; combined with the synchronous presentation of audio and time-frequency graphs, the time required for annotation of a single audio segment is greatly shortened, significantly improving annotation efficiency.
[0008] Furthermore, in step 1, the sensor is a microphone or an accelerometer, installed on the part of the car to be detected, to capture acoustic and vibration signals during the operation of the car.
[0009] Beneficial effects: The system comprehensively receives raw signal data under different operating conditions, providing a complete data foundation for subsequent analysis.
[0010] Furthermore, in step 2, the time-frequency transformation adopts either short-time Fourier transform or wavelet transform; if short-time Fourier transform is adopted, the frequency resolution is 1-30Hz and the overlap rate is 90%-99%.
[0011] Beneficial effects: Users can select the time-frequency transformation mode according to different types of abnormal noises. The time-frequency transformation converts the one-dimensional time domain signal into a two-dimensional time-frequency graph, clearly presenting the amplitude distribution of the signal at different times and frequencies, making it easy to intuitively observe the frequency characteristics and time distribution patterns of the abnormal noise signal.
[0012] Further, extract the time-frequency graph. Amplitude mean sequence in the frequency dimension N is the total number of time frames in the time-frequency graph; the Otsu threshold segmentation algorithm is used for... Binarization yields an initial set of suspected frequencies, F_sus. Morphological dilation and erosion are performed on F_sus to eliminate isolated noise frequency points. Intervals with amplitude variance greater than three times the background noise variance and a number of consecutive frequency points not less than 5% of the total frequency points are selected. If multiple intervals meet these conditions, a cosine similarity algorithm is used to match a preset frequency template for automotive abnormal noises. This template includes vehicle model, fault location, abnormal noise type, operating condition, and feature frequency intervals established based on measured data. The interval with the highest similarity is selected as the final key frequency interval.
[0013] Beneficial effects: This automatic filtering algorithm can adapt to the background noise differences of different vehicle models and operating conditions, without the need for manual parameter adjustment, thus balancing efficiency and accuracy.
[0014] Furthermore, in step 4, an 8th-order butter filter is used for bandpass filtering to retain only the signal within the key frequency range and filter out background noise outside the range.
[0015] Beneficial effects: The filter parameter settings can effectively enhance the characteristics of abnormal noise signals, making them easier for the human ear to distinguish and for subsequent algorithms to identify.
[0016] Furthermore, in step 5, automatic identification and labeling includes: Step 51: Calculate the time-frequency interval corresponding to the key frequency range determined in Step 3. Figure 2 The input is a 3D matrix T(f,t), where f is the frequency index, t is the time frame index, and the matrix elements are the amplitudes at the corresponding time and frequency points. Step 52: Perform a first-order difference operation on T(f,t) in the time dimension to obtain the difference spectrum D(f,t)=|T(f,t) T(f,t 1) | (t≥1), enhances transient abnormal noise components; Step 53. Summate the difference spectrum D(f,t) along the frequency dimension to obtain the one-dimensional time series E(t):
[0017] ,in[ , [This refers to the main frequency range of the abnormal noise identified in step 3;] Step 54: Calculate the mean of E(t) N is the total length of the time series (number of data points). Step 55: Set the threshold θ = β × The value of β ranges from [1,5], indicating that the threshold is β times the time series E(t). Step 56: Mark the continuous time frames that satisfy E(t)>θ as candidate abnormal sound intervals, and output the results after time merging and minimum duration filtering.
[0018] Beneficial effects: The above steps can accurately capture the transient and non-periodic characteristics of abnormal noise signals, effectively suppress steady-state background noise, and reduce the probability of missed detection and false detection; the adaptive threshold setting can be dynamically adjusted according to the background noise level in different scenarios, improving the adaptability and robustness of the algorithm.
[0019] Furthermore, between steps 5 and 6, there is a manual verification step, which involves viewing the time-frequency diagram and signal waveform, highlighting the marked candidate intervals, manually listening to the filtered audio, and adding, deleting, or adjusting the candidate intervals.
[0020] Beneficial effects: The simultaneous presentation and linkage of multi-dimensional information eliminates the need for staff to repeatedly listen to the entire audio, allowing them to simply confirm or fine-tune the candidate intervals, significantly reducing manual verification time. At the same time, the unified judgment criteria reduce the influence of personal experience on the annotation results and improve annotation consistency.
[0021] Furthermore, in step 7, the time-frequency diagrams of the tagged abnormal sound time intervals and the key frequency intervals are output in JSON format.
[0022] Beneficial effects: The JSON format facilitates the reading and processing of subsequent deep learning models, provides standardized data support for the training of abnormal audio classification models, and improves data reusability.
[0023] This application also provides a vehicle abnormal noise data annotation system, including: The data receiving module is used to receive raw signal data with abnormal noises from the vehicle collected by the sensor during vehicle operation. The raw signal data includes acoustic signals and vibration signals. The signal processing module is used to perform time-frequency transformation on the original signal data using short-time Fourier transform or wavelet transform to generate a time-frequency graph. It also uses an 8th-order butter filter to perform bandpass filtering on the original signal data according to the key frequency range. The window length of the short-time Fourier transform is set to 512, the frequency resolution is 1-30Hz, and the overlap rate is 90%-99%. The display module is used to synchronously display the time-frequency graph, the filtered signal waveform, and the candidate time intervals of abnormal noise on the same interface, and supports synchronous display of multi-channel abnormal noise vibration signals; The intelligent labeling module automatically generates candidate time intervals for abnormal noises and adds abnormal noise labels using time axis differentiation and adaptive threshold algorithms. Specifically, it performs the following: it assigns time and frequency corresponding to key frequency intervals... Figure 2 The dimensional matrix T(f,t) is used as input. The first-order difference operation in the time dimension is performed to obtain the difference spectrum. The summation in the frequency dimension is used to obtain the one-dimensional time series E(t). The mean of E(t) is calculated and a threshold θ=β×E (β∈[1,5]) is set. The continuous time frames that satisfy E(t)>θ are marked as candidate abnormal sound intervals. After time merging and minimum duration filtering, the output is obtained. The audio playback module is used to play the original signal audio and the filtered signal audio, and supports pause, fast forward and rewind operations; The labeling output module is used to output the final abnormal sound time interval, key frequency interval and corresponding time-frequency graph with labels in a structured data format of JSON. The interactive module retrieves content from the display and audio playback modules, allowing users to add or delete meta tags for candidate time intervals and adjust candidate time regions. It also supports staff to simultaneously view time-frequency graphs and signal waveforms and listen to filtered audio to complete manual verification.
[0024] Beneficial effects: The system's various modules work together to achieve a triple-assisted annotation mode of auditory, visual, and intelligent pre-selection. The operation process is simple and intuitive, reducing the reliance on the professional skills of staff, while ensuring the efficiency of the annotation process and the accuracy of the annotation results. Attached Figure Description
[0025] Figure 1 A flowchart illustrating the method for labeling automotive abnormal noise data; Figure 2 A schematic diagram of the functional modules of a vehicle abnormal noise data annotation system. Detailed Implementation
[0026] The following detailed description illustrates the specific implementation method: Example 1: like Figure 1 As shown in the figure, in this embodiment, the method for annotating abnormal noise data in passenger cars is applied to the following scenario: Step 1: Receive the raw signal. The source of the raw signal is directly provided through the raw signal storage medium. The raw signal includes acoustic signals and vibration signals.
[0027] Step 2: Time-frequency analysis. The acquired raw signal data is transformed using short-time Fourier transform (STFT) with a frequency resolution of 1-30Hz and an overlap rate of 90%-99%, generating a time-frequency graph that clearly shows the amplitude changes of the signal at different time points and frequencies.
[0028] Step 3: Determining the key frequency range. An adaptive threshold clustering algorithm is used to automatically filter the key frequency range. For example: extract the frequency dimension amplitude mean sequence F(f) from the time-frequency map, with a total of 22050 frequency points and a preset frequency point threshold of 1102; obtain the initial suspected frequency set F_sus through Otsu threshold segmentation, corresponding to a frequency range of 450Hz-5200Hz; eliminate isolated noise points after morphological processing to obtain a continuous frequency range of 480Hz-5100Hz; calculate that the amplitude variance of this range is 3.2 times the variance of the background noise, and the continuous frequency point count is 2315, which meets the screening criteria; retrieve the preset frequency template of abnormal noise from the chassis of a family car (400Hz-5500Hz), calculate the cosine similarity to be 0.92, and finally determine the key frequency range as 480Hz-5100Hz.
[0029] Step 4: Directional filtering. Based on the determined key frequency range, an 8th-order butter bandpass filter is used to filter the original signal data, removing background noise outside the range and enhancing the characteristics of abnormal noise signals.
[0030] Step 5: Automatic candidate detection, using time axis differentiation and adaptive thresholding algorithms, specifically includes the following steps: Time frequency corresponding to the key frequency range Figure 2 The input is a 3D matrix T(f,t), where f is the frequency index, t is the time frame index, and the matrix elements are the amplitudes at the corresponding time and frequency points.
[0031] Perform a first-order difference operation along the time dimension to obtain the difference spectrum D(f,t), where D(f,t) = |T(f,t) T(f,t 1) | (t≥1), enhances transient abnormal noise components.
[0032] The one-dimensional time series E(t) is obtained by summing within the frequency index range [fmin, fmax]. The statistical mean of E(t) is set to a threshold θ = β × E(t), where the value of β is in the range [1, 5] and the value of β is set to 3. In this implementation, the statistical mean is A and the threshold is set to 3A.
[0033] Continuous time frames satisfying E(t) > 3A are marked as candidate abnormal noise intervals, with a minimum duration set to 20ms, and candidate intervals with an interval less than 10ms are merged. For example, in this embodiment, there are 3 candidate abnormal noise intervals.
[0034] Step 6: Manual verification and correction. Staff members view the time-frequency diagram of the superimposed highlighted candidate intervals and the filtered signal waveform through the display module. They repeatedly play the candidate intervals and surrounding audio through the audio playback module to confirm that the first two candidate intervals are the real abnormal noise intervals and the third is a residual interference signal, which is then deleted. Finally, two abnormal noise time interval labels are obtained.
[0035] Step 7: Output labeled data. The label output module outputs the final abnormal noise time interval labels and abnormal noise categories in JSON format.
[0036] Example 2: like Figure 1 As shown, the vehicle abnormal noise data annotation method of this embodiment is applied to the abnormal noise data annotation scenario of commercial vehicles. The specific implementation steps are as follows: Step 1: Receive the raw signal, which is provided by a microphone and accelerometer mounted on the vehicle. The raw signal includes acoustic and vibration signals.
[0037] Step 2: Time-frequency analysis. Wavelet transform is used to perform time-frequency transformation on the original signal data to generate a time-frequency diagram, which clearly presents the characteristics of the low-frequency abnormal noise signal of commercial vehicles.
[0038] Step 3: Determining the key frequency range. By combining the time-frequency graph of the display module and the original audio from the audio playback module, the staff identified the frequency range of 2kHz-6kHz as the key frequency range where the abnormal noise signal was concentrated.
[0039] Step 4: Directional filtering. An 8th-order butter bandpass filter is used to filter the original signal data, retaining the signal in the 2kHz-6kHz range and filtering out irrelevant background noise.
[0040] Step 5: Automatic candidate detection, using time axis differentiation and adaptive threshold algorithm, specifically includes the following steps. The candidate detection method is the same as in Example 1. In this example, the threshold θ=4A is set, the minimum duration is 20ms, and finally two candidate abnormal noise intervals are output.
[0041] Step 6: Manual verification and correction. Staff members view the time-frequency graph, signal waveform and highlighted candidate intervals through the interactive module, listen to the filtered audio repeatedly, confirm that both candidate intervals are real abnormal noise intervals, and there are no additions or deletions. Finally, the abnormal noise time interval label is determined.
[0042] Step 7: Output annotations, output the final annotation results in JSON format.
[0043] Example 3: like Figure 2 As shown, the annotation system in this embodiment is used to satisfy the annotation of abnormal noise data of automobiles in Embodiments 1 and 2.
[0044] The labeling system includes a data receiving module for receiving raw signal data with abnormal noises from the vehicle collected by sensors during vehicle operation. The raw signal data includes acoustic signals and vibration signals.
[0045] The signal processing module is used to perform time-frequency transformation on the original signal data by short-time Fourier transform or wavelet transform to generate a time-frequency graph, and to use an 8th-order butter filter to perform bandpass filtering on the original signal data according to the key frequency range. The window length of the short-time Fourier transform is set to 512, the frequency resolution is 1-30Hz, and the overlap rate is 90%-99%.
[0046] The display module is used to synchronously display the time-frequency graph, the filtered signal waveform, and the candidate time intervals of abnormal noises on the same interface, and supports synchronous display of multi-channel abnormal noise vibration signals.
[0047] The intelligent labeling module automatically generates candidate time intervals for abnormal noises and adds abnormal noise labels using time axis differentiation and adaptive threshold algorithms. Specifically, it performs the following: it assigns time and frequency corresponding to key frequency intervals... Figure 2 The 3D matrix T(f,t) is used as input. The first-order difference operation in the time dimension is performed to obtain the difference spectrum. The summation in the frequency dimension is used to obtain the one-dimensional time series E(t). The mean of E(t) is calculated and a threshold θ=β×E (β∈[1,5]) is set. Continuous time frames that satisfy E(t)>θ are marked as candidate abnormal sound intervals. After time merging and minimum duration filtering, the output is obtained.
[0048] The audio playback module is used to play the original audio signal and the filtered audio signal, and supports pause, fast forward, and rewind operations.
[0049] The labeling output module is used to output the final abnormal sound time interval, key frequency interval and corresponding time-frequency graph with labels in a structured data format of JSON.
[0050] The interactive module retrieves content from the display and audio playback modules, allowing users to add or delete meta tags for candidate time intervals and adjust candidate time regions. It also supports staff to simultaneously view time-frequency graphs and signal waveforms and listen to filtered audio to complete manual verification.
[0051] In this embodiment, the annotation system operates stably. For abnormal noise signals of automobiles, the annotation efficiency is improved by 65% compared with traditional methods, and the annotation accuracy reaches 97%, meeting the needs of large-scale abnormal noise data annotation.
[0052] The above are merely embodiments of the present invention. Common knowledge such as specific structures and characteristics known in the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the technical field to which the invention pertains before the application date or priority date, are able to obtain all prior art in the field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can improve and implement the solution based on the inspiration given in this application and their own capabilities. Some typical known structures or methods should not become obstacles for those skilled in the art to implement this application.
[0053] It should be noted that those skilled in the art can make various modifications and improvements without departing from the structure of this invention, and these should also be considered within the scope of protection of this invention. These modifications and improvements will not affect the effectiveness of the invention or the practicality of the patent. The scope of protection claimed in this application shall be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.
Claims
1. An automobile abnormal sound data labeling method, characterized in that, The method comprises the following steps: Step 1: receiving original signals, receiving original signal data with automobile abnormal sound collected by sensors during automobile operation; Step 2: time-frequency analysis, performing time-frequency transformation on the original signal data to generate a time-frequency graph; Step 3: key frequency interval determination, identifying a key frequency interval representing the characteristics of the automobile abnormal sound based on the time-frequency graph; Step 4: directional filtering, performing band-pass filtering on the original signal data according to the key frequency interval to enhance the abnormal sound signal and suppress background noise; Step 5: automatic candidate detection, automatically identifying and labeling the candidate time interval of the abnormal sound through a preset detection algorithm after filtering the signal; Step 6: outputting the labeled data, outputting the time-frequency graph of the key frequency interval and the labeled abnormal sound time interval in a structured data format.
2. The automobile abnormal sound data labeling method according to claim 1, characterized in that, In step 1, the sensor uses a microphone or an acceleration sensor installed on the part of the automobile to be detected to capture acoustic signals and vibration signals during automobile operation.
3. The automobile abnormal sound data labeling method according to claim 1, characterized in that, In step 2, the time-frequency transformation uses short-time Fourier transform or wavelet transform; if short-time Fourier transform is used, the window length is set to 512, the frequency resolution is 1-30 Hz, and the overlap rate is 90%-99%.
4. The automobile abnormal sound data labeling method according to claim 1, characterized in that, In step 3, the key frequency interval is the frequency range where the abnormal sound signal is concentrated, and an adaptive threshold clustering algorithm is used for automatic screening. The specific process is as follows: Extracting time-frequency maps sequence of mean amplitudes in the frequency dimension; ， N is the total number of time-frequency diagram time frames; Otsu threshold segmentation algorithm is adopted Binaryzation, get the initial suspected frequency set F_sus; Perform morphological dilation and erosion operation on F_sus to eliminate isolated noise frequency points; Screen the intervals with amplitude variance greater than 3 times the background noise variance and continuous frequency point number not less than 5% of the total frequency point number; If there are multiple intervals meeting the conditions, match the automobile abnormal sound preset frequency template through the cosine similarity algorithm, and select the interval with the highest similarity as the final key frequency interval.
5. The automobile abnormal sound data labeling method according to claim 1, characterized in that, In step 4, an 8th order butter filter is used for band-pass filtering, and only the signal in the key frequency interval is retained, and the background noise outside the interval is filtered out.
6. The automobile abnormal sound data labeling method according to claim 4, characterized in that, In step 5, automatic identification and labeling includes: Step 51: taking the time-frequency graph two-dimensional matrix T(f, t) corresponding to the key frequency interval determined in step 3 as input, where f is the frequency index, t is the time frame index, and the matrix element is the amplitude of the corresponding time-frequency point; Step 52: First-order difference operation is performed on T(f, t) in the time dimension to obtain a difference spectrogram D(f, t) = |T(f, t) T(f, t 1) |(t≥1), enhance transient abnormal sound components; Step 53. Sum the difference spectrum D(f, t) in the frequency dimension to obtain a one-dimensional time sequence E(t): ， wherein [ , ] is the identified abnormal sound dominant frequency interval in step 3; Step 54: Calculate the mean of E(t) N is the total length of the time series (number of data points). Step 55: set threshold θ = β × , β ∈ [1, 5], which means β times of the time series E(t) as the threshold; Step 56: mark the continuous time frame that satisfies E(t)>θ as a candidate abnormal sound interval, and output the result after time merging and minimum duration filtering.
7. The automobile abnormal sound data labeling method according to claim 1, characterized in that, Between step 5 and step 6, there is also a manual verification step, which views the time-frequency graph, signal waveform, highlights the labeled candidate interval, manually listens to the filtered audio, and performs addition, deletion, and adjustment operations on the candidate interval.
8. The automobile abnormal sound data labeling method according to claim 1, characterized in that, In step 6, the labeled abnormal sound time interval and the time-frequency graph of the key frequency interval are output in json format.
9. An automobile abnormal sound data labeling system, characterized by, It comprises: A data receiving module for receiving original signal data with automobile abnormal sound collected by sensors during automobile operation, the original signal data including acoustic signals and vibration signals; A signal processing module for performing short-time Fourier transform or wavelet transform time-frequency transformation on the original signal data to generate a time-frequency graph, and using an 8th order butter filter to perform band-pass filtering on the original signal data according to the key frequency interval, the short-time Fourier transform window length is set to 512, the frequency resolution is 1-30 Hz, and the overlap rate is 90%-99%; The display module is used to synchronously display the time-frequency graph, the filtered signal waveform, and the candidate time intervals of abnormal noise on the same interface, and supports synchronous display of multi-channel abnormal noise vibration signals; The intelligent labeling module is used to automatically generate candidate time intervals for abnormal noises and add abnormal noise labels through time axis differentiation and adaptive thresholding algorithms. Specifically, it takes the two-dimensional matrix T(f,t) of the time-frequency graph corresponding to the key frequency interval as input, performs a first-order difference operation on it in the time dimension to obtain the difference spectrum, sums it in the frequency dimension to obtain a one-dimensional time series E(t), calculates the mean of E(t) and sets the threshold θ=β×E (β∈[1,5]), marks the continuous time frames that satisfy E(t)>θ as candidate abnormal noise intervals, and outputs them after time merging and minimum duration filtering. The audio playback module is used to play the original signal audio and the filtered signal audio, and supports pause, fast forward and rewind operations; The labeling output module is used to output the final abnormal sound time interval, key frequency interval and corresponding time-frequency graph with labels in a structured data format of JSON. The interactive module retrieves content from the display and audio playback modules, allowing users to add or delete meta tags for candidate time intervals and adjust candidate time regions. It also supports staff to simultaneously view time-frequency graphs and signal waveforms and listen to filtered audio to complete manual verification.