An artificial intelligence-based speech encoder feature extraction and recognition method

Abstract

The application provides a speech encoder feature extraction and recognition method based on artificial intelligence, and belongs to the field of speech encoder recognition, and comprises the following steps: generating a speech coding data set, encoding original speech through 13 speech encoders to obtain encoded files; extracting speech coding direct features to generate feature files; extracting speech coding deep features, taking the obtained original direct features as the input of a deep neural network, mapping through multiple hidden layers to obtain intermediate layer feature output as the deep features of speech coding; designing a speech coding classifier, feeding the extracted direct features and deep features into the classifier to determine the type of corresponding encoders, and realizing the classification and recognition of 13 speech encodings. The method adopts a two-stage speech encoder feature extraction method, the first stage extracts direct features to realize the recognition of speech encoders, and the second stage improves the recognition accuracy in combination with deep features for speech encodings with low recognition accuracy.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of speech codec recognition, and particularly relates to a speech encoder feature extraction and recognition method based on artificial intelligence. BACKGROUND

[0002] In speech communication, in order to improve the transmission rate of speech signals or save the storage space required by data, people usually perform speech coding on speech signals to compress and process speech data. In the process of speech signal coding and decoding, due to the different compression algorithms of different codecs, using different speech codecs will have different effects on speech signals. Speech data of different speech coding types has potential heterogeneous characteristics, and there is a linear or nonlinear correlation between the heterogeneous characteristics. These characteristics need to be deeply mined by means of artificial intelligence, so as to intelligently classify speech coding characteristics, differentially analyze speech data of different coding types, realize recognition of common speech encoders, and finally be applied to signal source processing in signal interception, improve the intelligent processing capability of intelligence, and improve the intelligent level of equipment. In the research of speech codec recognition algorithm, Hiysonmez S, Sencar H T, Avcibas I, etc. proposed to use coding randomness and chaos as features in 2011. This method is relatively complex in calculation and has too high feature dimension. Tripathi P, Raju D KP, Joka M S, etc. proposed a new algorithm in 2013, a Graphical User Interface (GUI) based decision subsystem method, which uses the Microsoft Foundation Classes (MFC) framework to realize GUI, sets the threshold based on GUI iteration, and obtains autocorrelation, central second moment, Fast Fourier Transformation (FFT) and binary bit features through GUI analysis. This method has fewer features, and needs to use the MFC framework to realize GUI, which is a relatively complex process. Bondalapati V, Raju K P, Tripathi P, etc. proposed to extract mean value, autocorrelation, second, third and fourth central moments, and binary bits from the coding bit stream as feature parameters in 2015. This algorithm has low recognition accuracy for similar encodings. The algorithm of the present application expands the number of feature parameters, and the feature parameter calculation is relatively simple. For similar encodings, the deep neural network is used to extract the intermediate layer features to improve the recognition accuracy. SUMMARY

[0003] The technical problem to be solved by the present application is to find and determine a feature combination suitable for identifying a speech encoder, while accurately classifying and identifying G.711a, G.711u, G.721, G.722, G.723, G.726, G.728, G.729, CVSD, ACELP, QCELP, AAC-LD and LPC10. The 13 speech encoders.

[0004] The technical solution adopted by the present application is:

[0005] A speech encoder feature extraction and identification method based on artificial intelligence, comprising the following steps:

[0006] S1: The original speech is encoded by a plurality of speech encoders to generate an encoded file, obtaining an original speech encoded data set, selecting a part of the data as training data and another part as test data; wherein the speech encoders include G.711a, G.711u, G.721, G.722, G.723, G.726, G.728, G.729, CVSD, ACELP, QCELP, AAC-LD and LPC10.

[0007] S2: Extracting mean, variance, K-order central moment, Fourier transform and binary ratio as direct features from the obtained encoded file; wherein K is 2, 3 and 4;

[0008] S3: Taking the obtained direct features as the input of a deep neural network, mapping through multiple hidden layers to obtain intermediate layer features as deep features;

[0009] S4: Sending the direct features and deep features extracted from the training data to a classifier, training the classifier model, and then sending the direct features and deep features extracted from the test data to the trained classifier model to realize the classification and identification of multiple speech encodings.

[0010] Further, the S2 comprises:

[0011] S21: Check the content of the encoded file obtained after passing through the speech encoder, and uniformly convert the bit stream whose content is not 01 to binary;

[0012] S22: Segmenting the binary encoded file, that is, selecting the corresponding byte segment length according to the size of the encoded file for cutting, cutting the file size by 1 / 2-2 / 3 for files not exceeding 20KB, and cutting the length to 20KB for files exceeding 20KB;

[0013] S23: Calculating the number ratio of 0 and 1 for the cut bit segment to obtain the binary ratio BRO, the calculation formula is:

[0014] BRO = ZR / OR

[0015] In the formula, ZR and OR are the number of 0 and 1 in the intercepted section, respectively;

[0016] S24: Convert the bit stream obtained in S21 into a byte sequence section using the sliding window technique, i.e., set the window size to a, first calculate the decimal value of the a-bit bit number located at the first position, then move the window to the right by 1 bit each time, and calculate the decimal value again until all the bits in the bit stream are calculated; calculate a as an unsigned integer between 0 and 255 every a bits, i.e., convert the binary sequence into a decimal sequence; segment the decimal sequence obtained after calculation to obtain a new byte sequence section, and the section length is consistent with that in S22; the formula for calculating the decimal value is:

[0017]

[0018] In the formula, b(i) is the received bit stream, s(i) is the decimal value generated by the sliding window technique, w is the window size, and n is the number of received bits;

[0019] S25: Calculate the mean M of the new byte sequence section; wherein the mean refers to the average of all data, and the formula is:

[0020]

[0021] In the formula, N is the section length of the new byte sequence section, and X(i) is the decimal value of s(i) intercepted during segmentation;

[0022] S26: Calculate the variance V of the new byte sequence section; wherein the variance is the difference between each data and the mean, and then the square is calculated and the average is obtained, and the variance is used to represent the deviation between the random variable and the mathematical expectation; the calculation formula is:

[0023]

[0024] S27: Calculate the second, third and fourth central moments Q2, Q3 and Q4 of the new byte sequence section, and the formula for the k-th moment is:

[0025]

[0026] In the formula, k is the order to be calculated, and k takes 2, 3 and 4;

[0027] S28: Calculate the Fourier transform value of the new byte sequence section, and the calculation method is: perform FFT transform on the data, take the modulus value of the first half of the values, multiply by 2, and then divide by the byte section length;

[0028] S29: Integrate and store the above features into a feature file to obtain the speech coding direct features.

[0029] Further, the S3 comprises:

[0030] The obtained direct features are taken as original inputs of the deep neural network, given training labels and test labels, and by taking minimizing cross-entropy as an objective function, parameters are adjusted by using an error back propagation algorithm, and finally after the deep neural network training is completed, the original inputs are mapped by using network parameters to obtain intermediate layer features as deep layer features.

[0031] Further, the S4 comprises:

[0032] The direct features and the deep layer features extracted by the training data are sent into the OVR SVMs, the model is trained, then the direct features and the deep layer features extracted by the test data are sent into the trained model, that is, the direct features and the deep layer features are integrated into a file and sent into the OVR SVMs for classification and recognition, and finally the type of the corresponding encoder of each feature is obtained.

[0033] Compared with the prior art, the beneficial effects of the present application are that:

[0034] 1. The speech encoder feature extraction and recognition method based on artificial intelligence of the present application is expanded in the type of speech encoder, and the intermediate layer features extracted by the deep neural network are used to improve the recognition accuracy.

[0035] 2. The speech encoder feature extraction and recognition method based on artificial intelligence of the present application uses the OVR SVMs as the classifier in the selection of the classifier, and compared with other several classifiers, the recognition accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a flowchart of the speech encoder feature extraction and recognition method based on artificial intelligence provided by the embodiment of the present application.

[0037] Figure 2 is a specific method design block diagram of the embodiment of the present application.

[0038] Figure 3 is a principle schematic diagram of the deep layer feature extraction of the embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined invention purpose, the speech encoder feature extraction and recognition method based on artificial intelligence according to the present application is described in detail in combination with the drawings and embodiments.

[0040] The foregoing and other technical contents, features and effects of the present application can be clearly presented in the following detailed description of embodiments. Through the description of the embodiments, the technical means and effects taken by the present application to achieve the predetermined purpose can be understood more deeply and specifically.

[0041] Embodiment one

[0042] Referring to Figure 1 and Figure 2 An artificial intelligence-based speech encoder feature extraction and recognition method includes the following steps:

[0043] S1: The original speech is respectively encoded by G.711a, G.711u, G.721, G.722, G.723, G.726, G.728, G.729, CVSD, ACELP, QCELP, AAC-LD and LPC10. 13 kinds of speech encoders to generate encoded files and obtain original speech encoding data sets;

[0044] Specifically, S1 includes:

[0045] Select 1000 original TIMIT speech, respectively through G.711a, G.711u, G.721, G.722, G.723, G.726, G.728, G.729, CVSD, ACELP, QCELP, AAC-LD, LPC10. 13 kinds of encoders are encoded into binary files, each kind of encoding has 1000, and 960 of them are selected as training data and 40 as test data, totaling 13000 data.

[0046] S2: Extract the mean, variance, K-order central moment (K takes 2, 3, 4), Fourier transform and binary ratio as direct features from the obtained encoded files;

[0047] Specifically, S2 includes:

[0048] S21: Check the content of the encoded file obtained after passing through the speech encoder, and uniformly convert the bit stream whose encoded file content is not 01 into binary;

[0049] S22: Segment the binary encoded file, that is, select the corresponding byte segment length according to the size of the encoded file for cutting, and cut the file size by 1 / 2-2 / 3 for files not exceeding 20KB, and cut the length to 20KB for files exceeding 20KB;

[0050] S23: Calculate the number ratio of 0 and 1 for the cut bit segment to obtain the binary ratio BRO, and the calculation formula is:

[0051] BRO=ZR / OR

[0052] wherein ZR and OR are the number of 0 and 1 in the intercepted segment, respectively;

[0053] S24: Using the sliding window technique, the bit segment intercepted in S22 is converted into a byte vector, the window size is set to a, the decimal value of the first a-bit bit number is calculated, then the window is moved to the right by 1 bit each time, and the decimal value is calculated again until all the bit numbers in the intercepted segment are calculated; every a-bit is calculated as an unsigned integer between 0 and 255, that is, the binary sequence is converted into a decimal sequence; at the same time, the byte sequence segment obtained after calculation is segmented, and the entire byte sequence segment is intercepted here; the formula for calculating the decimal value is:

[0054]

[0055] wherein b(i) is the received bit stream, X(i) is the calculated decimal value, w is the window size, and N is the number of received bits;

[0056] S25: The mean M of the new byte sequence segment is calculated; wherein the mean refers to the average of all data, and the formula is:

[0057]

[0058] wherein N is the length of the new byte sequence segment, and X(i) is the decimal value intercepted when s(i) is segmented;

[0059] S26: The variance V of the new byte sequence segment is calculated; wherein the variance is the difference between each data and the mean, then the square is calculated, and then the average is calculated, the variance is used to represent the deviation between the random variable and the mathematical expectation; the calculation formula is:

[0060]

[0061] S27: The second, third and fourth central moments Q2, Q3 and Q4 of the new byte sequence segment are calculated, and the formula for the k-th moment is:

[0062]

[0063] wherein k is the order to be calculated, and k is 2, 3 and 4;

[0064] S28: The Fourier transform value of the new byte sequence segment is calculated, and the calculation method is: the data is subjected to FFT transform, the modulus value of the first half number is multiplied by 2 and then divided by the length of the byte segment;

[0065] S29: The above features are integrated and stored in a feature file to obtain the speech coding direct features.

[0066] S3: The obtained direct features are used as input to the deep neural network and mapped through multiple hidden layers to obtain intermediate layer features, which are then used as deep layer features.

[0067] Specifically, S3 includes:

[0068] like Figure 3 As shown, a deep neural network consists of an input layer, an output layer, and n hidden layers. Features for intermediate layers can be obtained from the output of any of the hidden layers. Each layer has multiple nodes, and each connection between layers has an adjustable coefficient. These coefficients are learned from the input sample data and are updated through backpropagation until the model reaches its optimum. First, the neural network is trained using a training sample set to optimize the model. Then, the direct features obtained from the input are processed through the optimal neural network model to obtain the intermediate layer feature outputs.

[0069] Using the obtained direct features as input to the deep neural network, the original features travel from the input layer through the connection weight matrix to the first hidden layer. This is equivalent to projecting the original feature space onto a new feature space composed of nodes in each hidden layer, thus obtaining the feature representation of the first hidden layer. Each hidden layer can be viewed as a recombination of the features from the previous layer and reconstructed in this layer through a nonlinear mapping. Given training and test labels, the parameters are adjusted using the backpropagation algorithm with the objective function of minimizing cross-entropy. Finally, after the deep neural network is trained, the network parameters are used to map the original input to obtain intermediate layer features, which serve as deep layer features.

[0070] S4: The direct and deep features extracted from the training data are fed into the classifier to train the model. Then, the direct and deep features extracted from the test data are fed into the trained model to achieve the classification and recognition of various speech codes.

[0071] Specifically, S4 includes:

[0072] The direct and deep features extracted from the training data are fed into OVR SVMs to train the classifier model. Then, the direct and deep features extracted from the test data are fed into the trained model. In other words, the direct and deep features are integrated into a file and fed into OVR SVMs for classification and recognition. Finally, the encoder type corresponding to each feature is obtained.

[0073] Example 2

[0074] This embodiment conducts a simulation experiment on a speech encoder feature extraction and recognition method based on artificial intelligence, as described in Embodiment 1.

[0075] 1. Simulation experimental conditions:

[0076] Operating system: ubuntu 16.04, python3.6

[0077] Processor: Intel Core i5-7700k CPU@4.20GHZx4

[0078] Video card: NVIDIA GeForce 2070Ti GPU

[0079] Memory: 32GB

[0080] 2. Simulation experiment content:

[0081] Simulation experiment one: speech codec recognition algorithm accuracy experiment

[0082] OVR SVMs, random forest, K nearest neighbor algorithm, gradient boosting tree, decision tree and other classifiers are selected as the above obtained direct and deep features. As can be seen from Table 1, the average recognition accuracy of OVR SVMs is more than 95%, but the recognition accuracy of LPC10 and QCELP is low, only 70%; the average recognition accuracy of random forest is about 82%; the average recognition accuracy of K nearest neighbor algorithm is about 91%; the average recognition accuracy of gradient boosting tree is about 75%; the average recognition accuracy of decision tree is about 68%. In summary, the recognition accuracy of OVR SVMs is the highest, for the two kinds of encoding with low recognition rate, LPC10 and QCELP, the deep features and the direct features are combined into new features, and OVR SVMs is used as a classifier for two classification, finally the recognition accuracy of LPC10 is 97.5%, and the recognition accuracy of QCELP is 100%.

[0083] The above is a further detailed description of the present application in combination with a specific preferred embodiment, and the specific implementation of the present application cannot be limited to these descriptions. For ordinary skilled persons in the technical field to which the present application belongs, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, which should be considered as belonging to the protection scope of the present application.

Claims

1. An artificial intelligence-based speech encoder feature extraction and recognition method, characterized in that, The method comprises the following steps: S1: encode the original speech through multiple speech encoders respectively to generate encoded files, obtain original speech encoded data sets, select a part of the data as training data, and select another part as test data; wherein the speech encoders include G.711a, G.711u, G.721, G.722, G.723, G.726, G.728, G.729, CVSD, ACELP, QCELP, AAC-LD and LPC10. S2: extract the mean, variance, K-order central moment, Fourier transform and binary ratio as direct features from the obtained encoded files; wherein K is 2, 3 and 4; S3: take the obtained direct features as the input of a deep neural network, map through multiple hidden layers to obtain intermediate layer features as deep layer features; S4: send the direct features and deep layer features extracted through the training data to a classifier, train the classifier model, then send the direct features and deep layer features extracted through the test data to the trained classifier model, and realize the classification and identification of multiple speech encodings. 2.The AI-based speech encoder feature extraction and recognition method of claim 1, wherein, The S2 comprises: S21: check the content of the encoded files obtained through the speech encoders, and uniformly convert the bit streams whose content is not 01 into binary; S22: segment the encoded files converted into binary, that is, select the corresponding byte segment length according to the size of the encoded files for cutting, cut 1 / 2-2 / 3 of the file size for files not exceeding 20 KB, and cut 20 KB for files exceeding 20 KB; S23: calculate the number ratio of 0 and 1 of the cut bit segment to obtain the binary ratio BRO, and the calculation formula is: BRO=ZR / OR In the formula, ZR and OR are the numbers of 0 and 1 in the cut segment respectively; S24: use the sliding window technology to convert the bit stream obtained in S21 into a byte sequence segment, that is, set the window size as a, first calculate the decimal value of the a-bit number located at the first position, then move the window to the right by 1 bit each time, calculate the decimal value again, and calculate all the bit numbers in the bit stream; each a-bit is calculated as an unsigned integer between 0 and 255, that is, the binary sequence is converted into a decimal sequence; the decimal sequence obtained after calculation is segmented again to obtain a new byte sequence segment, and the segment length is consistent with S22; the decimal value calculation formula is: In the formula, b(i) is the received bit stream, s(i) is the decimal value generated by the sliding window technology, w is the window size, and n is the number of received bits; S25: calculate the mean M of the new byte sequence segment; wherein the mean refers to the average of all data, and the formula is: In the formula, N is the segment length of the new byte sequence segment, and X(i) is the decimal value s(i) cut in the segmentation; S26: calculate the variance V of the new byte sequence segment; wherein the variance is the difference between each data and the average, then the square is calculated, and then the average is calculated, the variance is used to represent the deviation between the random variable and the mathematical expectation; the calculation formula is: S27: calculate the second, third and fourth order central moments Q2, Q3 and Q4 of the new byte sequence segment, and the formula of the k-order moment is: In the formula, k is the order to be calculated, k takes 2, 3 and 4; S28: Fourier transform value of the new byte sequence segment is calculated, and the calculation manner is: the modulus value of the first half of the data after FFT transformation is multiplied by 2 and then divided by the byte segment length; S29: the above features are integrated and stored in a feature file to obtain speech coding direct features. 3.The AI-based speech encoder feature extraction and recognition method of claim 1, wherein, The S3 comprises: The obtained direct features are taken as original inputs of the deep neural network, given training labels and test labels, a target function is minimized by cross-entropy, parameters are adjusted by the error back propagation algorithm, finally, after the deep neural network training is completed, the original inputs are mapped by using the network parameters to obtain intermediate layer features as deep features. 4.The AI-based speech encoder feature extraction and recognition method of claim 1, wherein, The S4 comprises: The direct features and the deep features extracted by the training data are sent into the OVR SVMs, the model is trained, then the direct features and the deep features extracted by the test data are sent into the trained model, that is, the direct features and the deep features are integrated into a file and sent into the OVR SVMs for classification and identification, finally, the type of the encoder corresponding to each feature is obtained.

Images