A modified car recognition method based on neural network and feature fusion

Abstract

The invention belongs to the technical field of modified car identification, and specifically relates to a modified car identification method based on neural network and feature fusion, comprising the following steps: step one, voice input; step two, preprocessing / framing the input voice; step three, feature extraction; step four, feature fusion; step five, BP neural network training; step six, outputting the recognition result, integrating the short-time energy and the dynamic information of frames and frames into MFCC, and forming new features to improve the recognition of abnormal environmental sounds, using BP neural network to identify such complex problems, lightweighting the model, miniaturizing the model, cutting and normalizing the input samples, and obtaining a higher accuracy rate while keeping the high sensitivity of BP neurons as much as possible, making the training of the model feasible in the case of small samples, and solving the demand for sample size for complex problem identification.

Description

Technical Field

[0001] The invention belongs to the technical field of modified car identification, and in particular relates to a modified car identification method based on neural network and feature fusion. Background Art

[0002] The traditional sound recognition method first pre-processes the collected sound signal data, then extracts the characteristic parameters of the sound signal, and finally inputs the characteristic parameters into the classifier for classification and recognition. In previous studies, the features usually selected and extracted are Mel Frequency Cepstrum Coefficient (MFCC), Short-Term Energy (STE), Linear Prediction Cepstrum Coefficient (LPCC), Zero Crossing Rate (ZCR) and Autocorrelation Function (ACF), etc., among which MFCC is the most commonly used. MFCC feature parameters are extracted as abnormal sound signal features. However, MFCC is proposed based on the occurrence point of human hearing and speech, so for environmental sound events such as modified cars in complex traffic environments, in-depth research on feature extraction is still needed. Starting from the characteristics of traffic environment sound, the present invention adopts the method of feature fusion to add short-time energy as an indicator to the feature for abnormal sounds such as modified cars. And introduces the lifter and improved first-order and second-order difference algorithms.

[0003] When capturing sample data of modified cars under actual road conditions, the sample volume is small. If DNN training is used for the extracted features, the network will be underfitting. Therefore, the present invention adopts an improved BP neural network for training. To prevent underfitting in training, positive and negative samples are intercepted respectively to remove a large amount of useless information. At the same time, a dropout layer is introduced to prevent underfitting caused by too few samples.

[0004] In summary, the present invention optimizes the recognition of modified car sounds with a small sample size in a complex outdoor acoustic environment, thereby improving the model accuracy and generalization ability. Summary of the invention

[0005] In view of the problems in the prior art, the purpose of the present invention is to solve the problem of identifying the sound of modified cars with a small sample size in a complex outdoor acoustic environment. An improved MFCC feature and BP algorithm are used to provide a modified car identification method based on neural network and feature fusion.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A modified car identification method based on neural network and feature fusion includes the following steps:

[0008] S1: Signal acquisition, using a microphone array to pick up the sound signal, and then convert it into an analog signal and store it in binary txt format to facilitate data reading and storage.

[0009] S2: Sample preprocessing: convert the obtained txt file into a wav file, combine it with the corresponding video file, manually distinguish positive and negative samples, and cut off useless information in the txt sample according to the time node; try to remove useless information in the sample to reduce the interference that training will bring to the neural network.

[0010] S3: Feature extraction: divide the samples into frames and then calculate the MFCC of each frame separately to prevent data merging between samples and affect the training. Then extract the energy of each frame to form a new MFCC feature.

[0011] S4: Feature fusion: The whole is differentiated by padding to prevent information from being lost; the newly constructed MFCC features are then fused with the obtained differential features, and finally normalized to normalize the MFCC parameters of each frame of the sample training set and validation set; this can eliminate the impact of individual abnormal samples on training, speed up sample convergence, and improve training efficiency.

[0012] S5: Training the bp neural network model. Due to the small sample size, the present invention uses the bp neural network. In order to prevent the model complexity from increasing due to data parallelism, a single frame input is adopted, and a neural network with one input layer, three hidden layers, and one output layer is established. A dropout layer is added after the last hidden layer to prevent overfitting, and the Adam method is used to update the optimizer instead of the original gradient descent method.

[0013] S6: Use the trained model to identify the input samples and output two categories, one for modified cars and the other for non-modified cars. Feature extraction is performed on multiple known label samples and normalized according to the training set data. The model is then input for judgment. The model makes a judgment on each frame. However, due to the actual situation and the occurrence of misidentification, this article sets a time judgment. It is necessary to have 14 frames in an event to jointly judge it as a modified car before the time is judged as a modified car. Otherwise, it is judged as a non-modified car. The test set is then used to verify that the single-frame recognition accuracy is about 87%. If the 14-frame joint judgment method is used, the accuracy can reach 100%.

[0014] Compared with the prior art, the present invention has the following technical effects:

[0015] By using the Mel-frequency cepstral coefficients of the sound, short-time energy and time sequence information are integrated into the system; the feature fusion of short-time energy improves the system's attention to abnormal sounds; the machine is statically trained through the BP neural network algorithm, so that the recognition parameters are constantly approaching the optimal state and the recognition rate is improved. In complex outdoor environments, the sound of modified cars can be accurately identified.

[0016] It provides a feasible method for miniaturization of models and for complex recognition environments with a small number of samples, overcoming the high sample size requirements of DNN and other similar conditions, and provides an improved MFCC feature for the recognition of abnormal sounds such as modified cars. Specifically:

[0017] 1) The present invention integrates short-time energy and frame-to-frame dynamic information into MFCC, and the resulting new features improve the recognition of abnormal environmental sounds.

[0018] 2) The present invention uses BP neural network to identify such complex problems, lightens the model, miniaturizes the model, trims and normalizes the input samples, and obtains a higher accuracy rate while maintaining the high sensitivity of BP neurons as much as possible, making the model training feasible with small samples. This solves the need for sample size in complex problem identification.

[0019] 3) The present invention adopts a multi-frame joint evaluation method for the evaluation of the last event, thereby improving the accuracy of event identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is the algorithm flow chart;

[0021] Figure 2 It is a BP network model;

[0022] Figure 3 It is the flow chart of feature fusion algorithm;

[0023] Figure 4 This is a single frame accuracy chart of the experimental modified car; DETAILED DESCRIPTION

[0024] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific drawings.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0026] Example 1

[0027] S1 signal acquisition:

[0028] 30 audio signals are collected by a 30-channel microphone array with a sampling frequency of 32k. The signals are converted into digital signals and saved in binary form as txt text format. The mean of the 30 signals is calculated, and the signal with the smallest variance is taken as the sample to eliminate the interference caused by hardware reasons.

[0029] Preprocessing of S2 samples:

[0030] S2-01 Pre-emphasis processing, because of the particularity of the sound signal and the loss during its propagation, the low-frequency sound signal has high energy and the high-frequency sound signal has low energy. This time, there is a large energy difference between the low-frequency signal and the high-frequency signal, that is, there is a large gap between the signal-to-noise ratio of the low frequency and the signal-to-noise ratio of the high frequency. Therefore, we need to strengthen the high-frequency signal, which is called pre-emphasis in the industry. The implementation method is to add a first-order high-pass filter to the original signal. Suppose the sound signal sampling value at time n is x(n). The formula is as follows: where a is the pre-emphasis coefficient, and the value of a in the present invention is 0.97. y(n) is the sound sampling value after pre-emphasis.

[0031] y(n)=x(n)-ax(n-1)

[0032] S2-02 Signal framing: The analysis of sound is actually a "short-time analysis" process. This is because the sound signal is time-varying, but the characteristic changes are small within a short time range, so it is treated as a steady state, but beyond this short time range, the speech signal changes. Therefore, when the signal is read in, it is framed to ensure the continuity of its characteristics. The overlapping sampling method is adopted, that is, the overlapping part is collected between two adjacent frames. The length of the frame is called "frame length" here, and the displacement of the latter frame to the previous frame is "frame shift". The present invention uses a frame length of 1024 sampling points and a frame shift of 512 sampling points. In order to retain the original information, no windowing operation is performed here. The signal y(n) becomes y after framing. i (m), where the subscript i represents the i-th frame after framing.

[0033] S3 Feature Extraction

[0034] S3-01 FFT Fourier transform: Perform FFT transform on each frame of signal, converting time domain data into frequency domain data:

[0035] Y(i, k) = FFT[y i(m) ]

[0036] S3-02 Calculate energy spectrum

[0037] Calculate the spectral line energy for each frame of FFT data:

[0038] E(i, k) = |Y(i, k)| 2

[0039] S3-03 Calculate short-time energy

[0040] Calculate short-time energy based on the energy spectrum of each frame

[0041]

[0042] S3-04 Mel filter design

[0043] The analysis of Mel frequency cepstral coefficients is based on human auditory excitation, that is, the spectrum of speech is analyzed according to the results of human auditory experiments, hoping to obtain good speech characteristics. Since the human ear has different sensitivities to different sound wave frequencies, the relationship between the frequency perceived by the human ear and the actual frequency can be expressed by the following formula:

[0044]

[0045] Where f mel is the perceived frequency, the unit is Mel; f Hz is the actual frequency, in Hz. Set several bandpass filters H in the frequency spectrum of the speech. m (k), 0≤m<M, M is the number of filters, each filter has a triangular filtering characteristic, and its center frequency is f(m). Within the Mel frequency range, these filters are of equal bandwidth. The transfer function of each bandpass filter is:

[0046]

[0047] f(m) can be defined as follows:

[0048]

[0049] Where f l is the lowest frequency of the filter frequency range; f h is the highest frequency of the filter's frequency range; N is the length of FFT; f s is the sampling frequency; F mel The inverse function of for

[0050]

[0051] The present invention selects the number of filters as M=26, and according to Shannon sampling theorem, takes f l =0,f h =f s / 2=16000

[0052] S3-05 Calculate the energy passing through the Mel filter

[0053] Calculating the energy passing through the Mel filter is equivalent to comparing the energy spectrum E(i, k) of each frame with the frequency response H of the Mel filter. m (k) Multiply and add:

[0054]

[0055] S3-06 Calculate DCT cepstrum

[0056] The cepstrum calculation takes the logarithm of the energy of the MEL filter and calculates the DCT:

[0057]

[0058] Where S(i,m) is the filter energy; m refers to the mth filter; i refers to the i-th frame; n is the spectrum line after DCT

[0059] S3-07 Lifter

[0060] Adding a lifter helps to improve the high-frequency coefficients after DCT. The lifter formula is

[0061]

[0062] The MFCC parameters after the lifter are:

[0063] mfccL(i,n)=L·mfcc(i,n)

[0064] S4 Feature Fusion

[0065] mfccE(i,1)=En i

[0066] mfccE(i,(2,3,4...n))=mfccL(i,(2,3,4...n))

[0067] S401 Difference Calculation

[0068] First-order difference: In order to prevent the loss of the last frame information after the first frame, the first frame and the last frame are copied and inserted into the first frame and the last frame respectively, so that the new frame number is i+2.

[0069]

[0070] Second-order difference, similar to first-order difference, the first frame and the last frame are copied by two frames and inserted into the first frame and the last frame respectively, so the new frame number is i+4

[0071]

[0072] Finally, the feature columns of mfccE(i, n), Δd(i, n)1, and Δd(i, n)2 are fused in parallel to obtain a new 39-dimensional feature Fe(i, d), where i represents the i-th frame and d is the feature dimension

[0073] S402 Data normalization processing

[0074] Normalizing the training set data helps eliminate abnormal samples and accelerate model convergence

[0075]

[0076] Training of S5bp neural network:

[0077] S5-01 The activation function selected by the present invention is the rule activation function, which has the characteristics of overcoming the problem of gradient disappearance and accelerating the training speed, and the given learning rate is 0.00005, and the number of iterations is 7600

[0078] The formula for the activation function is

[0079]

[0080] S5-02: Given the first layer as the input layer, there are 39 nodes, the number of nodes is equal to the number of features; the second layer is the hidden layer, there are 78 nodes, the number of nodes is equal to twice the number of features, initialize the weights and thresholds (only executed for the first time), select Fe minmax As a sample input to the neural network, the propagation formula from the first layer to the second layer is

[0081]

[0082] In the formula is the weight from the i-th node in the first layer to the j-th node in the second layer, is the threshold from the first layer to the jth node in the second layer

[0083] S5-03 The third layer is a hidden layer with 39 nodes. Initialize the weights and thresholds (only executed for the first time). The propagation formula from the second layer to the third layer is:

[0084]

[0085] In the formula is the weight from the i-th node in the second layer to the j-th node in the third layer, is the threshold of the jth node from the second layer to the third layer

[0086] S5-04 The fourth layer is a hidden layer with 19 nodes. Initialize the weights and thresholds (only executed for the first time). The propagation formula from the third layer to the fourth layer is:

[0087]

[0088] In the formula is the weight from the ith node in the third layer to the jth node in the fourth layer, is the threshold of the jth node from the third layer to the fourth layer

[0089] S5-05 The fifth layer is a hidden layer with 2 nodes. This layer does not add an activation function. The weights and thresholds are initialized (only executed for the first time). The propagation formula from the fourth layer to the fifth layer is:

[0090]

[0091] In the formula is the weight from the ith node in the fourth layer to the jth node in the fifth layer, is the threshold of the jth node from the fourth layer to the fifth layer

[0092] S5-06Softmax layer: also known as normalized exponential function. It is a generalization of the binary classification function sigmoid in multi-classification, and its purpose is to present the results of multi-classification in the form of probability. With L4 as its input, its formula is

[0093]

[0094] In the formula, j is the jth class, L4(j) is the network output of the jth class, and the predicted label result is the class with the largest probability value in softmax.

[0095] S5-07 The present invention selects cross entropy as the loss function. Cross entropy can be used as a loss function in neural networks (machine learning). Assuming that p represents the distribution of true labels and q is the predicted label distribution of the trained model, the cross entropy loss function can measure the similarity between p and q. Another advantage of using cross entropy as a loss function is that using the sigmoid function can avoid the problem of reduced learning rate of the mean square error loss function during gradient descent, because the learning rate can be controlled by the output error. The calculation formula of cross entropy is

[0096]

[0097] In the formula, P(j) represents the true prediction of the sample in class j.

[0098] S5-08 Optimizer Selection:

[0099] The present invention uses the Adam optimizer to iterate the weights and thresholds. The Adam optimizer has

[0100] (1) High calculation efficiency

[0101] (2) Smooth and stable gradient transition, which can adapt to unstable objective functions

[0102] (3) Parameter adjustment is relatively simple, and the default parameters can handle most problems

[0103] The formula of Adam optimizer is

[0104] m t =β1m t-1 +(1-β1)g t

[0105]

[0106] In the formula, t represents the time, m t and v t are the first-order momentum term and the second-order momentum term respectively. β1 and β2 are the magnitudes of the dynamic values ​​(usually 0.9 and 0.999 respectively); are their respective correction values. t represents the parameters of the model at time t, i.e. the tth iteration; g t Represents the gradient of the cost function with respect to the weight at t iterations; ∈ is a very small number (usually 1e-8, to avoid the denominator being 0);

[0107] Optimization process formula:

[0108]

[0109] Then we will get g t Bring in the Adam optimizer to complete the back propagation of the neural network. Iterate and update the weights

[0110] S5-09 determines whether the algorithm iteration has ended. If not, returns to step S5-02.

[0111] After the algorithm iteration is completed, the model of the modified car recognition system is obtained. After the training is completed, the constructed model is verified with the test sample to verify the accuracy of the model. Experiments have proved that the model has a high evaluation accuracy. When the input features are MFCC, MFCC+energy, and the final fusion features, the accuracy of the model is increased by 2%-3% compared with the traditional BP. On the other hand, under the same model, the fusion feature has a greater advantage in the recognition of modified cars than the traditional MFCC. Finally, under the fusion feature and the improved BP algorithm, the single-frame recognition rate of modified cars is increased by 17%.

[0112] The above shows and describes the basic principles, main features and characteristics of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of the present invention to be protected is defined by the attached claims and their equivalents.

Claims

1. A modified car identification method based on neural network and feature fusion, characterized in that: The following steps are involved: S1 signal acquisition: 30 audio signals are collected by a 30-channel microphone array with a sampling frequency of 32k. They are converted into digital signals and saved in binary form as txt text format. The mean of the 30-channel signals is calculated, and the channel with the smallest variance is taken as the sample to eliminate the interference caused by hardware reasons. Preprocessing of S2 samples: S2-01 pre-emphasis processing is implemented by adding a first-order high-pass filter to the original signal. Assume that the sound signal sampling value at time n is x(n) and the formula is as follows: where a is the pre-emphasis coefficient, and a is 0.97; y(n) is the sound sampling value after pre-emphasis; y(n)=x(n)-ax(n-1) S2-02 signal framing: signal y(n) becomes y after framing i (m), where the subscript i represents the i-th frame after framing; S3 Feature Extraction S3-01FFT Fourier transform: Perform FFT transform on each frame of signal, converting time domain data into frequency domain data: Y(i,k)=FFT[y i(m) ] S3-02 Calculate energy spectrum Calculate the spectral line energy for each frame of FFT data: E(i, k) = |Y(i, k)|2 S3-03 Calculate short-time energy Calculate short-time energy based on the energy spectrum of each frame S3-04 Mel filter design Where f mel is the perceived frequency, the unit is Mel; f Hz is the actual frequency, the unit is Hz; set several bandpass filters H in the frequency spectrum range of speech m (k), 0≤m<M, M is the number of filters, each filter has a triangular filtering characteristic, and its center frequency is f(m). Within the Mel frequency range, these filters are of equal bandwidth; the transfer function of each bandpass filter is: f(m) is defined as follows: Where f l is the lowest frequency of the filter frequency range; f h is the highest frequency of the filter's frequency range; N is the length of FFT; f s is the sampling frequency; F mel The inverse function of for Select the number of filters as M = 26, and according to Shannon sampling theorem, take f l =0,f h =f s / 2=16000 S3-05 Calculate the energy passing through the Mel filter Calculating the energy passing through the Mel filter is equivalent to comparing the energy spectrum E(i, k) of each frame with the frequency response H of the Mel filter. m (k) Multiply and add: S3-06 Calculate DCT cepstrum The cepstrum calculation takes the logarithm of the energy of the MEL filter and calculates the DCT: Where S(i, m) is the filter energy; m refers to the mth filter; i refers to the i-th frame; n is the spectrum line after DCT S3-07 Lifter Adding a lifter helps to improve the high-frequency coefficients after DCT. The lifter formula is The MFCC parameters after the lifter are: mfccL(i,n)=L·mfcc(i,n) S4 Feature Fusion mfccE(i,1)=En i mfccE(i,(2,3,4…n))=mfccL(i,(2,3,4…n)) S401 Difference Calculation First-order difference, in order to prevent the loss of the last frame information after the first frame, the first frame and the last frame are copied and inserted into the first frame and the last frame respectively, so that the new frame number is i+2 Second-order difference, similar to first-order difference, the first frame and the last frame are copied by two frames and inserted into the first frame and the last frame respectively, so the new frame number is i+4 Finally, the feature columns of mfccE(i, n), Δd(i, n)1, and Δd(i, n)2 are fused in parallel to obtain a new 39-dimensional feature Fe(i, d), where i represents the i-th frame and d is the feature dimension S402 Data normalization processing Normalizing the training set data helps eliminate abnormal samples and accelerate model convergence Training of S5bp neural network: The activation function of S5-01 is the rule activation function. The activation function formula is: S5-02: Given the first layer as the input layer, there are 39 nodes, the number of nodes is equal to the number of features; the second layer is the hidden layer, there are 78 nodes, the number of nodes is equal to twice the number of features, initialize the weights and thresholds, select Fe minmax As a sample input to the neural network, the propagation formula from the first layer to the second layer is In the formula is the weight from the i-th node in the first layer to the j-th node in the second layer, is the threshold from the first layer to the jth node in the second layer S5-03 The third layer is a hidden layer with a total of 39 nodes. The propagation formula from the second layer to the third layer is initialized with weights and thresholds. In the formula is the weight from the i-th node in the second layer to the j-th node in the third layer, is the threshold of the jth node from the second layer to the third layer S5-04 The fourth layer is a hidden layer with a total of 19 nodes. The propagation formula from the third layer to the fourth layer is initialized with weights and thresholds. In the formula is the weight from the ith node in the third layer to the jth node in the fourth layer, is the threshold of the jth node from the third layer to the fourth layer S5-05 The fifth layer is a hidden layer with a total of 2 nodes. This layer does not add an activation function. The weights and thresholds are initialized. The propagation formula from the fourth layer to the fifth layer is: In the formula is the weight from the ith node in the fourth layer to the jth node in the fifth layer, is the threshold of the jth node from the fourth layer to the fifth layer S5-06Softmax layer: also known as the normalized exponential function, its formula is In the formula, j is the jth class, L4(j) is the network output of the jth class, and the predicted label result is the class with the largest probability value in softmax. S5-07 Select cross entropy as the loss function. The calculation formula of cross entropy is In the formula, P(j) represents the true prediction of the sample in class j. S5-08 Optimizer Selection: The Adam optimizer is used to iterate the weights and thresholds. The formula of Adam optimizer is m t =β1m t-1 +(1-β1)g t In the formula, t represents the time, m t and v t are the first-order momentum term and the second-order momentum term respectively; β1 and β2 are the magnitudes of the dynamic values; are their respective correction values; w t represents the parameters of the model at time t, i.e. the tth iteration; g t Represents the gradient of the cost function with respect to the weight at t iterations; the value of ∈ is 1 e-8 ; Optimization process formula: Then we will get g t Bring in the Adam optimizer to complete the back propagation of the neural network; Iteratively update weights S5-09 determines whether the algorithm iteration is finished. If not, returns to step S5-02; After the algorithm iteration is completed, the model of the modified vehicle identification system is obtained; after the training is completed, the constructed model is verified with test samples to verify the accuracy of the model.

Images