Vehicle abnormal sound audio data extraction method, device and equipment and readable storage medium

Abstract

The application relates to a vehicle abnormal sound audio data extraction method and device, equipment and a readable storage medium, and relates to the technical field of data processing. The method comprises the following steps: performing MFCC analysis on each frame data in frame sequence data corresponding to a to-be-processed abnormal sound audio file to obtain an MFCC parameter sequence corresponding to each frame data; calculating the cosine similarity and the amplitude ratio between each two adjacent frame data based on the MFCC parameter sequence to obtain a cosine similarity sequence and an amplitude ratio sequence corresponding to the frame sequence data; determining an abnormal sound starting moment and an abnormal sound ending moment according to the cosine similarity sequence and the amplitude ratio sequence; and performing data extraction on the frame sequence data based on the abnormal sound starting moment and the abnormal sound ending moment to obtain abnormal sound audio data. Through the application, the extraction efficiency of abnormal sound audio data can be effectively improved, and the extraction quality of abnormal sound audio data can be ensured.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, device, and readable storage medium for extracting abnormal noise audio data from vehicles. Background Technology

[0002] With the development of the times, people's awareness of quality is increasing. In the automotive consumer sector, people are paying more and more attention to the performance of vehicles, particularly regarding unusual noises and vibrations. Therefore, the quality of these noises directly affects customers' purchasing decisions, prompting automakers to do their utmost to evaluate and control them. In related technologies, when a car exhibits unusual noise problems, staff record the noise, evaluate it, and then remove useless or irrelevant data from the recording. The remaining noise data is then extracted from the recording for further analysis.

[0003] It is evident that the current method for extracting abnormal noise audio data is still manual, that is, manually extracting audio data within the time period of the abnormal noise. This method is not only time-consuming and labor-intensive, but also inefficient, and the quality of the extracted data cannot be guaranteed. Summary of the Invention

[0004] This application provides a method, apparatus, device, and readable storage medium for extracting abnormal noise audio data from vehicles, in order to solve the problems of low efficiency and unreliable quality in the related technologies that extract abnormal noise audio data by manual means.

[0005] Firstly, a method for extracting abnormal noise audio data from vehicles is provided, including the following steps:

[0006] MFCC analysis is performed on each frame of data in the frame sequence data corresponding to the audio file with abnormal noise to be processed, and the MFCC parameter sequence corresponding to each frame of data is obtained.

[0007] Based on the MFCC parameter sequence, the cosine similarity and amplitude ratio between each pair of adjacent frames are calculated to obtain the cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data.

[0008] The start and end times of the abnormal noise are determined based on the cosine similarity sequence and the amplitude ratio sequence.

[0009] Data is extracted from the frame sequence data based on the start and end times of the abnormal noise to obtain the abnormal noise audio data.

[0010] In some embodiments, after the step of extracting data from the frame sequence data based on the start time and end time of the abnormal noise to obtain the abnormal noise audio data, the method further includes:

[0011] The abnormal noise risk index is calculated based on the duration of the abnormal noise, the cosine similarity, and the amplitude ratio corresponding to the abnormal noise audio data.

[0012] The severity of the abnormal noise is evaluated based on the abnormal noise risk index, and the abnormal noise evaluation result is obtained.

[0013] In some embodiments, the formula for calculating the abnormal noise risk index is:

[0014]

[0015] In the formula, risk_index represents the abnormal noise risk index, s_ref represents the cosine similarity reference value, s represents the cosine similarity, amp represents the amplitude ratio, amp_ref represents the amplitude ratio reference value, duration represents the duration of the abnormal noise, duration_ref represents the duration of the abnormal noise reference value, and w1, w2 and w3 represent the weight coefficients of cosine similarity, amplitude ratio and duration of the abnormal noise, respectively.

[0016] In some embodiments, the step of calculating the cosine similarity and amplitude ratio between each pair of adjacent frames of data based on the MFCC parameter sequence to obtain the cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data includes:

[0017] Substitute the MFCC parameter sequences corresponding to each two adjacent frames in the frame sequence data into the following first calculation formula to obtain the cosine similarity corresponding to each two adjacent frames, and generate the cosine similarity sequence corresponding to the frame sequence data.

[0018] The first calculation formula is:

[0019]

[0020] In the formula, s(i) represents the cosine similarity between the (i+1)th frame data and the i-th frame data, m(i,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the i-th frame data, m(i+1,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the (i+1)th frame data, and T represents the total number of MFCC parameters.

[0021] In some embodiments, the step of calculating the cosine similarity and amplitude ratio between each pair of adjacent frames of data based on the MFCC parameter sequence to obtain the cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data includes:

[0022] Substitute the MFCC parameter sequences corresponding to each two adjacent frames in the frame sequence data into the following second calculation formula to obtain the amplitude ratio corresponding to each two adjacent frames, and generate the amplitude ratio sequence corresponding to the frame sequence data.

[0023] The second calculation formula is:

[0024]

[0025] In the formula, amp(i) represents the amplitude ratio between the (i+1)th frame data and the i-th frame data, m(i,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the i-th frame data, m(i+1,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the (i+1)th frame data, and T represents the total number of MFCC parameters.

[0026] In some embodiments, determining the start and end times of the abnormal noise based on the cosine similarity sequence and the amplitude ratio sequence includes:

[0027] Each cosine similarity in the cosine similarity sequence is compared with a cosine similarity threshold, and each amplitude ratio in the amplitude ratio sequence is compared with a first amplitude ratio threshold and a second amplitude ratio threshold.

[0028] When the i-th cosine similarity in the cosine similarity sequence is less than the cosine similarity threshold and the i-th amplitude ratio in the amplitude ratio sequence is greater than the first amplitude ratio threshold, the start time of the abnormal sound is determined based on the (i+1)-th frame of data.

[0029] When the j-th cosine similarity in the cosine similarity sequence is less than the cosine similarity threshold and the j-th amplitude ratio in the amplitude ratio sequence is less than the second amplitude ratio threshold, the end time of the abnormal sound is determined based on the j-th frame data.

[0030] In some embodiments, before the step of performing MFCC analysis on each frame of data in the frame sequence data corresponding to the audio file with abnormal noise to be processed, to obtain the MFCC parameter sequence corresponding to each frame of data, the method further includes:

[0031] The audio file containing the abnormal noise is preprocessed by framing and windowing to obtain frame sequence data corresponding to the audio file containing the abnormal noise.

[0032] Secondly, a device for extracting vehicle abnormal noise audio data is provided, comprising:

[0033] The processing unit is used to perform MFCC analysis on each frame of data in the frame sequence data corresponding to the audio file with abnormal noise to be processed, and to obtain the MFCC parameter sequence corresponding to each frame of data.

[0034] The calculation unit is used to calculate the cosine similarity and amplitude ratio between each pair of adjacent frames of data based on the MFCC parameter sequence, so as to obtain the cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data.

[0035] A determining unit is used to determine the start time and end time of the abnormal noise based on the cosine similarity sequence and the amplitude ratio sequence.

[0036] An extraction unit is used to extract data from the frame sequence data based on the start time and end time of the abnormal noise to obtain abnormal noise audio data.

[0037] Thirdly, a vehicle abnormal noise audio data extraction device is provided, comprising: a memory and a processor, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the aforementioned vehicle abnormal noise audio data extraction method.

[0038] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned method for extracting abnormal vehicle noise audio data.

[0039] This application provides a method, apparatus, device, and readable storage medium for extracting vehicle abnormal noise audio data. The method includes performing MFCC analysis on each frame of data in a frame sequence corresponding to an abnormal noise audio file to be processed, obtaining an MFCC parameter sequence for each frame; calculating the cosine similarity and amplitude ratio between each pair of adjacent frames based on the MFCC parameter sequence, obtaining a cosine similarity sequence and an amplitude ratio sequence corresponding to the frame sequence data; determining the start and end times of the abnormal noise based on the cosine similarity sequence and the amplitude ratio sequence; and extracting data from the frame sequence data based on the start and end times of the abnormal noise to obtain abnormal noise audio data. This application accurately determines the start and end times of the abnormal noise based on the patterns in the abnormal noise data and by combining the cosine similarity sequence and the amplitude ratio sequence, thereby achieving precise extraction of abnormal noise audio data. This not only effectively improves the extraction efficiency of abnormal noise audio data but also ensures the extraction quality. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating a method for extracting abnormal noise audio data from a vehicle, as provided in an embodiment of this application;

[0042] Figure 2This is a schematic diagram of the structure of a vehicle abnormal noise audio data extraction device provided in an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of a vehicle abnormal noise audio data extraction device provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] This application provides a method, apparatus, device, and readable storage medium for extracting vehicle abnormal noise audio data, which can solve the problems of low efficiency and unreliable quality in related technologies that rely on manual extraction of abnormal noise audio data.

[0046] Figure 1 This application provides a method for extracting vehicle abnormal noise audio data, which includes the following steps:

[0047] Step S10: Perform MFCC analysis on each frame of data in the frame sequence data corresponding to the audio file with abnormal noise to be processed, and obtain the MFCC parameter sequence corresponding to each frame of data;

[0048] Exemplary and understandable, compared to traditional spectral analysis, MFCC (Mel Frequency Cepstrum Coefficient) takes into account the auditory characteristics of the human ear, providing a more profound description of the timbre characteristics of sound. Therefore, MFCC analysis is highly suitable for the extraction and evaluation of abnormal audio data. For example, given two frames of audio data with different spectral data, but sounding identical to the human ear, traditional spectral analysis would yield results inconsistent with subjective perception. However, MFCC analysis, considering the auditory characteristics of the human ear, would result in no difference between the two frames, aligning with subjective perception.

[0049] Therefore, this embodiment will perform MFCC analysis on each frame of the frame sequence data to calculate its MFCC parameters, and then obtain the MFCC parameter sequence corresponding to each frame of data, denoted as M; where the t-th MFCC parameter of the i-th frame of data can be denoted as m(i,t), i and t are natural numbers, 1≤i≤F, 1≤t≤T, F is the total number of frames in the frame sequence data, and T is the total number of MFCC parameters in each frame of data, that is, each frame of data has T MFCC parameters, which can be regarded as a vector.

[0050] Furthermore, before the step of performing MFCC analysis on each frame of data in the frame sequence data corresponding to the audio file with abnormal noise to be processed, and obtaining the MFCC parameter sequence corresponding to each frame of data, the method further includes:

[0051] The audio file containing the abnormal noise is preprocessed by framing and windowing to obtain frame sequence data corresponding to the audio file containing the abnormal noise.

[0052] As an example, in this embodiment, the recorded audio file of abnormal noise to be processed will undergo preprocessing including framing and windowing. It should be understood that the audio file of abnormal noise is obtained by sampling the sound signal at a certain sampling frequency fs. Since the audio data of abnormal noise is a transient signal, analyzing the audio file as a whole will not yield valuable information. Therefore, the purpose of framing in this embodiment is to segment the audio data to ensure that the analysis of abnormal noise features has a sufficiently high temporal resolution, thereby obtaining more valuable information.

[0053] Specifically, N consecutive sampling points are grouped into a data analysis unit, which is called a frame, and N is the frame length; where N is generally a power of 2, i.e., N = 2. z (z is a natural number) to facilitate subsequent feature extraction from the audio data. In addition, this embodiment will also perform a windowing operation (i.e., apply a window function) on each frame of data to reduce spectral leakage. Therefore, after preprocessing by framing and windowing, the frame sequence X can be obtained. Let the total number of data frames be F, then the i-th frame of data is denoted as x(i), where i is a natural number and 1≤i≤F.

[0054] Step S20: Calculate the cosine similarity and amplitude ratio between each pair of adjacent frames based on the MFCC parameter sequence to obtain the cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data;

[0055] As an example, it is understandable that cosine similarity is a parameter describing the closeness between two vectors. Therefore, this embodiment will use cosine similarity to describe the closeness between two adjacent MFCC vectors (i.e., the MFCC parameter sequences corresponding to two adjacent frames of data), that is, the closeness between the timbres of two adjacent frames (MFCC parameters can be understood as timbres). Therefore, the cosine similarity between each pair of adjacent frames of data in the frame sequence data will be calculated through the MFCC parameter sequence, thereby obtaining the cosine similarity sequence corresponding to the frame sequence data.

[0056] Meanwhile, it should be understood that in audio files with abnormal noise, the amplitude at the moment the abnormal noise occurs is usually larger than the amplitude at other moments (i.e., when no abnormal noise occurs). Therefore, in this embodiment, the MFCC parameter sequence of each frame of data is regarded as a vector to calculate its amplitude. By calculating the ratio of the amplitudes of the MFCC parameters of two adjacent frames of data, the amplitude ratio between the MFCC parameters corresponding to each two adjacent frames of data is obtained, thereby obtaining the value ratio sequence corresponding to the frame sequence data.

[0057] In some embodiments, the step of calculating the cosine similarity and amplitude ratio between each pair of adjacent frames of data based on the MFCC parameter sequence to obtain the cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data includes:

[0058] Substitute the MFCC parameter sequences corresponding to each two adjacent frames in the frame sequence data into the following first calculation formula to obtain the cosine similarity corresponding to each two adjacent frames, and generate the cosine similarity sequence corresponding to the frame sequence data.

[0059] The first calculation formula is:

[0060]

[0061] In the formula, s(i) represents the cosine similarity between the (i+1)th frame data and the i-th frame data, m(i,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the i-th frame data, m(i+1,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the (i+1)th frame data, and T represents the total number of MFCC parameters.

[0062] As an example, in this embodiment, the abrupt change in timbre between two adjacent frames of data will be obtained by calculating cosine similarity, that is, the cosine similarity corresponding to each pair of adjacent frames of data will be calculated based on the following first calculation formula:

[0063]

[0064] In the formula, 1≤i≤F-1, s(i) represents the cosine similarity between the (i+1)th frame data and the i-th frame data, and its value is [0,2]; m(i,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the i-th frame data, m(i+1,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the (i+1)-th frame data, and T represents the total number of MFCC parameters.

[0065] In equation (1), the first term on the right-hand side ranges from -1 to 1. The larger the value, the more similar the two vectors (i.e., the MFCC parameter sequences corresponding to the two frames of data); the smaller the value, the less similar the two vectors are. Therefore, s(i) represents the degree of timbre difference between two adjacent frames of data. The smaller the value, the greater the parameter difference between the two adjacent frames of data, and the greater the timbre abrupt change; the larger the value, the smaller the parameter difference between the two adjacent frames of data, and the smaller the timbre abrupt change. In addition, the purpose of adding 1 after the first term on the right-hand side of equation (1) in this embodiment is to ensure that s(i) is not negative, so as to facilitate the subsequent calculation and processing of the abnormal sound risk index. Then, all cosine similarities s(i) are added to the same set to form a cosine similarity sequence, which can be denoted as S.

[0066] In some embodiments, the step of calculating the cosine similarity and amplitude ratio between each pair of adjacent frames of data based on the MFCC parameter sequence to obtain the cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data includes:

[0067] Substitute the MFCC parameter sequences corresponding to each two adjacent frames in the frame sequence data into the following second calculation formula to obtain the amplitude ratio corresponding to each two adjacent frames, and generate the amplitude ratio sequence corresponding to the frame sequence data.

[0068] The second calculation formula is:

[0069]

[0070] In the formula, amp(i) represents the amplitude ratio between the (i+1)th frame data and the i-th frame data, m(i,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the i-th frame data, m(i+1,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the (i+1)th frame data, and T represents the total number of MFCC parameters.

[0071] As an example, in this embodiment, the variation characteristics of the MFCC parameter amplitude between two adjacent frames of data will be obtained by calculating the amplitude ratio, that is, the amplitude ratio corresponding to each pair of adjacent frames of data will be calculated based on the following second calculation formula:

[0072]

[0073] In the formula, 1≤i≤F-1, amp(i) represents the amplitude ratio between the (i+1)th frame data and the i-th frame data, and its value is greater than 0; m(i,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the i-th frame data, m(i+1,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the (i+1)-th frame data, and T represents the total number of MFCC parameters.

[0074] It should be noted that when the value of amp(i) is greater than 1 (i.e., the amplitude corresponding to the (i+1)th frame is greater than that of the i-th frame), the larger the value, the greater the difference between the amplitude corresponding to the (i+1)th frame and that of the i-th frame in two adjacent frames. When the value of amp(i) is greater than 0 and less than 1 (i.e., the amplitude corresponding to the (i+1)th frame is less than that of the i-th frame), the smaller the value, the greater the difference between the amplitude corresponding to the (i+1)th frame and that of the i-th frame in two adjacent frames. Adding all amplitude ratios amp(i) to the same set forms an amplitude ratio sequence, which can be denoted as A.

[0075] Step S30: Determine the start time and end time of the abnormal noise based on the cosine similarity sequence and the amplitude ratio sequence;

[0076] Specifically, determining the start and end times of the abnormal noise based on the cosine similarity sequence and the amplitude ratio sequence includes:

[0077] Each cosine similarity in the cosine similarity sequence is compared with a cosine similarity threshold, and each amplitude ratio in the amplitude ratio sequence is compared with a first amplitude ratio threshold and a second amplitude ratio threshold.

[0078] When the i-th cosine similarity in the cosine similarity sequence is less than the cosine similarity threshold and the i-th amplitude ratio in the amplitude ratio sequence is greater than the first amplitude ratio threshold, the start time of the abnormal sound is determined based on the (i+1)-th frame of data.

[0079] When the j-th cosine similarity in the cosine similarity sequence is less than the cosine similarity threshold and the j-th amplitude ratio in the amplitude ratio sequence is less than the second amplitude ratio threshold, the end time of the abnormal sound is determined based on the j-th frame data.

[0080] Exemplarily, in this embodiment, the abnormal noise characteristic time including the start time and end time of the abnormal noise will be obtained based on the abnormal noise data pattern and in combination with the cosine similarity sequence and the amplitude ratio sequence. Specifically, the cosine similarity threshold will be preset and denoted as s_limit; at the same time, the first amplitude ratio threshold and the second amplitude ratio threshold will be preset, and the first amplitude ratio threshold will be denoted as amp_limit1, and its value can be selected from natural numbers greater than 1; the second amplitude ratio threshold will be denoted as amp_limit2, and its value can be selected within (0, 1). It should be noted that the specific values of the above thresholds can be set according to actual needs or obtained after statistical processing of a large amount of abnormal noise-related data.

[0081] After setting the relevant thresholds, the cosine similarity sequence S and the amplitude ratio sequence A will be retrieved to determine the magnitude relationship between each cosine similarity s(i) in the cosine similarity sequence S and the cosine similarity threshold s_limit, and the magnitude relationship between each amplitude ratio amp(i) in the amplitude ratio sequence A and the first amplitude ratio threshold amp_limit1 and the second amplitude ratio threshold amp_limit2 respectively, and the start time and end time of the abnormal noise will be determined according to the judgment results.

[0082] Specifically, if s(i) < s_limit and amp(i) > amp_limit1, then the (i + 1)-th frame data is defined as the data at the start time of the abnormal noise, where 1 ≤ i ≤ F - 1. At this time, the start time t_start of the abnormal noise can be calculated by the following formula: t_start = i * N / fs, where N is the frame length and fs is the sampling frequency of the abnormal noise audio file, and N / fs is the time length of each frame of data; if s(j) < s_limit and amp(j) < amp_limit2, then the j-th frame data is defined as the data at the end time of the abnormal noise, where 1 ≤ j ≤ F - 1. At this time, the end time t_end of the abnormal noise can be calculated by the following formula: t_end = j * N / fs, where N is the frame length and fs is the sampling frequency of the abnormal noise audio file.

[0083] For example, assume there are 6 consecutive frames of data. The cosine similarity sequence S is (X1, X2, X3, X4, X5), and the amplitude ratio sequence A is (Y1, Y2, Y3, Y4, Y5). If s(1) = X1 < s_limit and amp(1) = Y1 > amp_limit1, then the 2nd frame data is the data at the start time of the abnormal noise; if s(5) = X5 < s_limit and amp(5) = Y5 < amp_limit2, then the 5th frame data is the data at the end time of the abnormal noise.

[0084] Step S40: Extract data from the frame sequence data based on the start time and end time of the abnormal noise to obtain the abnormal noise audio data.

[0085] As an example, it should be understood that a single audio file containing abnormal noise may contain multiple segments of abnormal noise data representing the same event (an abnormal noise caused by the same component, the same reason, or the same fault). This means there are multiple start and end times for the abnormal noise, and these start and end times appear in pairs. Therefore, the abnormal noise audio data can be accurately extracted from the frame sequence data by using these paired start and end times. It should be noted that if multiple start and end times exist, multiple abnormal noise audio data segments can be extracted from the frame sequence data.

[0086] Therefore, this embodiment prepares for abnormal noise feature processing by framing, windowing, and calculating MFCC parameters on the abnormal noise audio file; it obtains the abrupt change features of timbre between two adjacent frames by calculating cosine similarity; it obtains the change features of MFCC parameter amplitude between two adjacent frames by calculating amplitude ratio; and it obtains information such as the start and end times of abnormal noise based on the abnormal noise data patterns and in combination with cosine similarity sequence and amplitude ratio sequence, thus achieving accurate extraction of abnormal noise audio data. This not only effectively improves the extraction efficiency of abnormal noise audio data but also ensures the extraction quality of abnormal noise audio data.

[0087] Furthermore, after the step of extracting data from the frame sequence data based on the start time and end time of the abnormal noise to obtain the abnormal noise audio data, the method further includes:

[0088] An abnormal noise risk index is calculated based on the duration of the abnormal noise, cosine similarity, and amplitude ratio corresponding to the abnormal noise audio data, respectively; wherein, the formula for calculating the abnormal noise risk index is:

[0089]

[0090] In the formula, risk_index represents the abnormal noise risk index, s_ref represents the cosine similarity reference value, s represents the cosine similarity, amp represents the amplitude ratio, amp_ref represents the amplitude ratio reference value, duration represents the duration of the abnormal noise, duration_ref represents the duration of the abnormal noise reference value, and w1, w2 and w3 represent the weight coefficients of cosine similarity, amplitude ratio and duration of the abnormal noise, respectively.

[0091] The severity of the abnormal noise is evaluated based on the abnormal noise risk index, and the abnormal noise evaluation result is obtained.

[0092] Exemplarily, it can be understood that the subjective feeling of abnormal noise is related to the amplitude of abnormal noise, the degree of mutation of abnormal noise timbre, and the duration of abnormal noise. That is, the larger the amplitude of abnormal noise, the greater the mutation of abnormal noise timbre, and the longer the duration of abnormal noise, the worse the subjective feeling. At present, the subjective evaluation method is mainly used to evaluate abnormal noise. However, due to the strong subjectivity of this method, the evaluation results vary from person to person, and a unified, accurate, and convincing evaluation result cannot be obtained, and there is no objective data support. Therefore, in order to solve the above problems in this embodiment, a variable will be set to evaluate the subjective feeling of abnormal noise. This variable can be called the abnormal noise risk index, denoted as risk_index. The larger its value, the worse the subjective feeling of abnormal noise. Specifically, the calculation method of the abnormal noise risk index can be determined by combining the three parameters of cosine similarity s, amplitude ratio amp, and abnormal noise duration.

[0093] Among them, the abnormal noise duration duration can be calculated from the start time t_start and the end time t_end of the abnormal noise, that is, duration = t_end - t_start = (j - i) * N / fs, and the start time, end time, and duration of the abnormal noise can be collectively referred to as the abnormal noise characteristic time. The abnormal noise risk index risk_index can be calculated through the following calculation formula:

[0094]

[0095] In the formula, s_ref represents the cosine similarity reference value, amp_ref represents the amplitude ratio reference value, duration_ref represents the abnormal noise duration reference value, w1, w2, and w3 respectively represent the weight coefficients of cosine similarity, amplitude ratio, and abnormal noise duration, and 0 < w1 < 1, 0 < w2 < 1, 0 < w3 < 1, w1 + w2 + w3 = 1; it should be noted that the cosine similarity reference value s_ref, the amplitude ratio reference value amp_ref, and the abnormal noise duration reference value duration_ref can be obtained through statistical processing of a large amount of abnormal noise-related data.

[0096] It should be understood that in an abnormal noise audio file, there may be multiple segments of abnormal noise data for the same abnormal noise event, that is, there are multiple start times, end times, and durations of abnormal noise; and the start time of each segment of abnormal noise data corresponds to a cosine similarity and an amplitude ratio. Therefore, the cosine similarity and amplitude ratio at the start time of each segment of abnormal noise data can be averaged respectively to obtain the cosine similarity s and amplitude ratio amp in formula (3); and since each segment of abnormal noise data corresponds to an abnormal noise duration, the abnormal noise durations of each segment of abnormal noise data can be averaged to obtain the abnormal noise duration in formula (3).

[0097] Furthermore, multiplying the right side of the equal sign in equation (3) by 10 aims to ensure that the abnormal noise risk index falls between 0 and 10, thus conforming to the conventional value of evaluation results. Therefore, after determining the abnormal noise risk index corresponding to the abnormal noise audio data, the severity of the abnormal noise can be evaluated using this abnormal noise risk index to obtain the abnormal noise evaluation result.

[0098] It is understandable that the abnormal noise risk index can be directly used as the quantitative evaluation result of abnormal noise, or the evaluation result can be determined by constructing a mapping relationship between the abnormal noise risk index and the severity level of the abnormal noise. For example, the value range of the abnormal noise risk index is [0,10]. Assuming that risk_index∈[0,1) corresponds to no impact on the severity of the abnormal noise, risk_index∈[1,3) corresponds to mild severity, risk_index∈[3,6) corresponds to moderate severity, risk_index∈[6,8) corresponds to severe severity, and risk_index∈[8,10] corresponds to extremely severe severity; therefore, if the abnormal noise risk index is equal to 2.3, the evaluation result of the abnormal noise is mild severity. It should be noted that the above is only a presentation of an example, and the level division can also be carried out according to actual needs, which is not limited here.

[0099] Therefore, this embodiment analyzes the influencing factors of subjective perception of abnormal noises, and determines the calculation method for the abnormal noise risk index by combining three parameters: cosine similarity, amplitude ratio, and duration of the abnormal noise, and quantifies the severity of the abnormal noise. In summary, this embodiment is simple in principle and easy to calculate, and can automatically obtain the start and end times of abnormal noises, facilitating the extraction of abnormal noise audio data; at the same time, it uses objective test data to quantify the severity of abnormal noises, obtaining scientific, reasonable, and convincing evaluation results.

[0100] See Figure 2 As shown, this application also provides a vehicle abnormal noise audio data extraction device, including:

[0101] The processing unit is used to perform MFCC analysis on each frame of data in the frame sequence data corresponding to the audio file with abnormal noise to be processed, and to obtain the MFCC parameter sequence corresponding to each frame of data.

[0102] The calculation unit is used to calculate the cosine similarity and amplitude ratio between each pair of adjacent frames of data based on the MFCC parameter sequence, so as to obtain the cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data.

[0103] A determining unit is used to determine the start time and end time of the abnormal noise based on the cosine similarity sequence and the amplitude ratio sequence.

[0104] An extraction unit is used to extract data from the frame sequence data based on the start time and end time of the abnormal noise to obtain abnormal noise audio data.

[0105] Furthermore, the device also includes an evaluation unit, which is used for:

[0106] The abnormal noise risk index is calculated based on the duration of the abnormal noise, the cosine similarity, and the amplitude ratio corresponding to the abnormal noise audio data.

[0107] The severity of the abnormal noise is evaluated based on the abnormal noise risk index, and the abnormal noise evaluation result is obtained.

[0108] Furthermore, the formula for calculating the abnormal noise risk index is as follows:

[0109]

[0110] In the formula, risk_index represents the abnormal noise risk index, s_ref represents the cosine similarity reference value, s represents the cosine similarity, amp represents the amplitude ratio, amp_ref represents the amplitude ratio reference value, duration represents the duration of the abnormal noise, duration_ref represents the duration of the abnormal noise reference value, and w1, w2 and w3 represent the weight coefficients of cosine similarity, amplitude ratio and duration of the abnormal noise, respectively.

[0111] Furthermore, the computing unit is specifically used for:

[0112] Substitute the MFCC parameter sequences corresponding to each two adjacent frames in the frame sequence data into the following first calculation formula to obtain the cosine similarity corresponding to each two adjacent frames, and generate the cosine similarity sequence corresponding to the frame sequence data.

[0113] The first calculation formula is:

[0114]

[0115] In the formula, s(i) represents the cosine similarity between the (i+1)th frame data and the i-th frame data, m(i,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the i-th frame data, m(i+1,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the (i+1)th frame data, and T represents the total number of MFCC parameters.

[0116] Furthermore, the computing unit is specifically used for:

[0117] Substitute the MFCC parameter sequences corresponding to each two adjacent frames in the frame sequence data into the following second calculation formula to obtain the amplitude ratio corresponding to each two adjacent frames, and generate the amplitude ratio sequence corresponding to the frame sequence data.

[0118] The second calculation formula is:

[0119]

[0120] In the formula, amp(i) represents the amplitude ratio between the (i+1)th frame data and the i-th frame data, m(i,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the i-th frame data, m(i+1,t) represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the (i+1)th frame data, and T represents the total number of MFCC parameters.

[0121] Furthermore, the determining unit is specifically used for:

[0122] Each cosine similarity in the cosine similarity sequence is compared with a cosine similarity threshold, and each amplitude ratio in the amplitude ratio sequence is compared with a first amplitude ratio threshold and a second amplitude ratio threshold.

[0123] When the i-th cosine similarity in the cosine similarity sequence is less than the cosine similarity threshold and the i-th amplitude ratio in the amplitude ratio sequence is greater than the first amplitude ratio threshold, the start time of the abnormal sound is determined based on the (i+1)-th frame of data.

[0124] When the j-th cosine similarity in the cosine similarity sequence is less than the cosine similarity threshold and the j-th amplitude ratio in the amplitude ratio sequence is less than the second amplitude ratio threshold, the end time of the abnormal sound is determined based on the j-th frame data.

[0125] Furthermore, the processing unit is also used for:

[0126] The audio file containing the abnormal noise is preprocessed by framing and windowing to obtain frame sequence data corresponding to the audio file containing the abnormal noise.

[0127] It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device and each unit described above can be referred to the corresponding process in the aforementioned embodiment of the vehicle abnormal noise audio data extraction method, and will not be repeated here.

[0128] The apparatus provided in the above embodiments can be implemented as a computer program, which can be used in, for example... Figure 3 The device shown is running on the vehicle abnormal noise audio data extraction equipment.

[0129] This application embodiment also provides a vehicle abnormal noise audio data extraction device, including: a memory, a processor and a network interface connected via a system bus, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement all or part of the steps of the aforementioned vehicle abnormal noise audio data extraction method.

[0130] The network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0131] A processor can be a CPU, or other general-purpose processors, DSPs (Digital Signal Processors), ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor, or any conventional processor. The processor is the control center of a computer device, connecting all parts of the computer device through various interfaces and lines.

[0132] Memory can be used to store computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function (such as video playback, image playback, etc.), etc.; the data storage area can store data created based on the use of the mobile phone (such as video data, image data, etc.). Furthermore, memory can include high-speed random access memory (RAM), and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, SMC (Smart MediaCard), SD (Secure Digital) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0133] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements all or part of the steps of the aforementioned method for extracting abnormal vehicle noise audio data.

[0134] The embodiments of this application can implement all or part of the aforementioned processes, or they can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various methods described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, ROM (Read-Only memory), RAM (Random Access memory), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0135] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, servers, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0136] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0137] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0138] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for extracting audio data of abnormal noises from vehicles, characterized in that, Includes the following steps: MFCC analysis is performed on each frame of data in the frame sequence data corresponding to the audio file with abnormal noise to be processed, and the MFCC parameter sequence corresponding to each frame of data is obtained. Based on the MFCC parameter sequence, the cosine similarity and amplitude ratio between each pair of adjacent frames are calculated to obtain the cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data. The start and end times of the abnormal noise are determined based on the cosine similarity sequence and the amplitude ratio sequence. Based on the start and end times of the abnormal noise, data is extracted from the frame sequence data to obtain the abnormal noise audio data. The step of extracting data from the frame sequence data based on the start and end times of the abnormal noise to obtain the abnormal noise audio data further includes: An abnormal noise risk index is calculated based on the duration of the abnormal noise, cosine similarity, and amplitude ratio corresponding to the abnormal noise audio data; the formula for calculating the abnormal noise risk index is as follows: In the formula, Indicates the risk index of abnormal noise. This represents the cosine similarity reference value, where 's' represents the cosine similarity and 'amp' represents the amplitude ratio. This indicates the reference value for the amplitude ratio. Indicates the duration of the abnormal noise. The reference value for the duration of the abnormal noise is represented by w1, w2 and w3, which represent the weighting coefficients of cosine similarity, amplitude ratio and duration of abnormal noise, respectively. The severity of abnormal noise is evaluated based on the abnormal noise risk index to obtain the abnormal noise evaluation result. The step of determining the start and end times of the abnormal noise based on the cosine similarity sequence and the amplitude ratio sequence includes: Each cosine similarity in the cosine similarity sequence is compared with a cosine similarity threshold, and each amplitude ratio in the amplitude ratio sequence is compared with a first amplitude ratio threshold and a second amplitude ratio threshold. When the i-th cosine similarity in the cosine similarity sequence is less than the cosine similarity threshold and the i-th amplitude ratio in the amplitude ratio sequence is greater than the first amplitude ratio threshold, the start time of the abnormal sound is determined based on the (i+1)-th frame of data. When the j-th cosine similarity in the cosine similarity sequence is less than the cosine similarity threshold and the j-th amplitude ratio in the amplitude ratio sequence is less than the second amplitude ratio threshold, the end time of the abnormal sound is determined based on the j-th frame data.

2. The method for extracting vehicle abnormal noise audio data as described in claim 1, characterized in that, The calculation of cosine similarity and amplitude ratio between each pair of adjacent frames based on the MFCC parameter sequence yields a cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data, including: Substitute the MFCC parameter sequences corresponding to each two adjacent frames in the frame sequence data into the following first calculation formula to obtain the cosine similarity corresponding to each two adjacent frames, and generate the cosine similarity sequence corresponding to the frame sequence data. The first calculation formula is: +1 In the formula, This represents the cosine similarity between the (i+1)th frame of data and the ith frame of data. This represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the i-th frame of data. This represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the (i+1)-th frame of data, where T represents the total number of MFCC parameters.

3. The method for extracting vehicle abnormal noise audio data as described in claim 1, characterized in that, The calculation of cosine similarity and amplitude ratio between each pair of adjacent frames based on the MFCC parameter sequence yields a cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data, including: Substitute the MFCC parameter sequences corresponding to each two adjacent frames in the frame sequence data into the following second calculation formula to obtain the amplitude ratio corresponding to each two adjacent frames, and generate the amplitude ratio sequence corresponding to the frame sequence data. The second calculation formula is: In the formula, This represents the amplitude ratio between the (i+1)th frame of data and the ith frame of data. This represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the i-th frame of data. This represents the t-th MFCC parameter in the MFCC parameter sequence corresponding to the (i+1)-th frame of data, where T represents the total number of MFCC parameters.

4. The method for extracting vehicle abnormal noise audio data as described in claim 1, characterized in that, Before the step of performing MFCC analysis on each frame of data in the frame sequence data corresponding to the audio file with abnormal noise to be processed, and obtaining the MFCC parameter sequence corresponding to each frame of data, the method further includes: The audio file containing the abnormal noise is preprocessed by framing and windowing to obtain frame sequence data corresponding to the audio file containing the abnormal noise.

5. A device for extracting audio data of abnormal vehicle noises, characterized in that, include: The processing unit is used to perform MFCC analysis on each frame of data in the frame sequence data corresponding to the audio file with abnormal noise to be processed, and to obtain the MFCC parameter sequence corresponding to each frame of data. The calculation unit is used to calculate the cosine similarity and amplitude ratio between each pair of adjacent frames of data based on the MFCC parameter sequence, so as to obtain the cosine similarity sequence and amplitude ratio sequence corresponding to the frame sequence data. A determining unit is used to determine the start time and end time of the abnormal noise based on the cosine similarity sequence and the amplitude ratio sequence. An extraction unit is used to extract data from the frame sequence data based on the start time and end time of the abnormal noise to obtain abnormal noise audio data. The step of extracting data from the frame sequence data based on the start and end times of the abnormal noise to obtain the abnormal noise audio data further includes: An abnormal noise risk index is calculated based on the duration of the abnormal noise, cosine similarity, and amplitude ratio corresponding to the abnormal noise audio data; the formula for calculating the abnormal noise risk index is as follows: In the formula, Indicates the risk index of abnormal noise. This represents the cosine similarity reference value, where 's' represents the cosine similarity and 'amp' represents the amplitude ratio. This indicates the reference value for the amplitude ratio. Indicates the duration of the abnormal noise. The reference value for the duration of the abnormal noise is represented by w1, w2 and w3, which represent the weighting coefficients of cosine similarity, amplitude ratio and duration of abnormal noise, respectively. The severity of abnormal noise is evaluated based on the abnormal noise risk index to obtain the abnormal noise evaluation result. The step of determining the start and end times of the abnormal noise based on the cosine similarity sequence and the amplitude ratio sequence includes: Each cosine similarity in the cosine similarity sequence is compared with a cosine similarity threshold, and each amplitude ratio in the amplitude ratio sequence is compared with a first amplitude ratio threshold and a second amplitude ratio threshold. When the i-th cosine similarity in the cosine similarity sequence is less than the cosine similarity threshold and the i-th amplitude ratio in the amplitude ratio sequence is greater than the first amplitude ratio threshold, the start time of the abnormal sound is determined based on the (i+1)-th frame of data. When the j-th cosine similarity in the cosine similarity sequence is less than the cosine similarity threshold and the j-th amplitude ratio in the amplitude ratio sequence is less than the second amplitude ratio threshold, the end time of the abnormal sound is determined based on the j-th frame data.

6. A device for extracting audio data of abnormal vehicle noises, characterized in that, include: A memory and a processor, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the vehicle abnormal noise audio data extraction method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, implements the vehicle abnormal noise audio data extraction method according to any one of claims 1 to 4.

Images