Voice emotion recognition model training method and system

Abstract

The invention provides a voice emotion recognition model training method and system, and belongs to the field of model training. The problem of low model training efficiency is solved; the method specifically comprises the following steps: S1, obtaining an MFCC coefficient of each piece of voice information; s2, acquiring an LPCC coefficient of each piece of voice information; performing supervised training and unsupervised training on the original model; obtaining a to-be-determined model and to-be-determined output until output results of the supervised training and the unsupervised training of the original model are the same; s3, comparing the performance of the to-be-determined model and the performance of the hot model; if the performance of the to-be-determined model is weaker than that of the hot model, weak items of the to-be-determined model are analyzed, and a lifting report is obtained; otherwise, not processing; s4, summarizing the promotion report, and updating the voice information; according to the invention, through defining an extraction method of voice information features, the model can accurately capture text features and voiceprint features of different emotion voices in an iteration process, and the training efficiency is improved.

Description

Technical Field

[0001] The present invention discloses a speech emotion recognition model training method and system, and relates to the field of model training. Background Art

[0002] The existing speech emotion recognition model training methods and systems have the following deficiencies:

[0003] Difficulty in data collection and processing: Voice emotion recognition requires a large amount of emotional voice data support, but the differences in emotional expression in different cultures, regions, and languages ​​make the construction of data sets particularly complex; in addition, voice information often has various noise interferences, such as environmental noise, distortion of voice information, etc. These noises will affect the accuracy of emotion recognition.

[0004] Poor generalization ability: The existing models extract fewer feature values ​​of speech information. The analysis results of the existing models for the same sentence and speech information with different tones are mostly the same, which is not targeted. In addition, the speech habits between different cultures vary greatly. The existing models have a single training set selection and are difficult to cope with speech emotion recognition tasks in different cultures.

[0005] Lack of competitiveness: There are currently a variety of speech emotion recognition models and systems on the market, but their performance varies greatly; most of the existing models are obsessed with completing the given tasks and are not compared with popular models on the market, resulting in companies having little understanding of the advantages and disadvantages of the models, being unable to determine their market positioning, and lacking competitiveness. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention aims to provide a speech emotion recognition model training method and system, aiming to solve the problem of low model training efficiency.

[0007] In order to achieve the above object, the present invention is implemented by the following technical scheme: A speech emotion recognition model training method comprises:

[0008] Step S1: Acquire positive voice information and negative voice information; back up the positive voice information and negative voice information, and perform a primary process of pre-emphasis, windowing and Fourier transform on each voice information, and then perform a secondary process of Mel transform and discrete cosine transform to obtain the MFCC coefficient of each voice information;

[0009] Step S2: Perform homomorphic and cepstrum processing on each voice information to obtain the LPCC coefficient of each voice information; obtain the original model; convert the voice information into text information, and divide the positive voice information and the negative voice information into sample set α and sample set β in the format of text information-MFCC coefficient-LPCC coefficient;

[0010] Take sample set α and sample set β as independent inputs to perform supervised training on the original model; take sample set α and sample set β as mixed inputs to perform unsupervised training on the original model; until the output results of supervised training and unsupervised training of the original model are the same, the undetermined model is obtained;

[0011] Inputting positive speech information and negative speech information into the model to be tested to obtain a pending output;

[0012] Step S3: taking the positive speech information and the negative speech information as inputs of the hot model and obtaining a standard output;

[0013] Compare the performance of the proposed model and the popular model based on the proposed output and the standard output;

[0014] If the performance of the candidate model is weaker than that of the popular model, the outputs of the convolutional layer, pooling layer, and fully connected layer of the candidate model are obtained in sequence, and substituted into the convolutional layer, pooling layer, and fully connected layer corresponding to the popular model to analyze the weaknesses of the candidate model and obtain an improvement report;

[0015] If the performance of the pending model is stronger than that of the popular model, it will not be processed;

[0016] Step S4: Summarize the improvement report and provide feedback; obtain new positive voice information and new negative voice information, and perform the training and analysis process of the original model again until the performance of the original model is close to or stronger than that of the popular model.

[0017] Furthermore, the specific steps of step S1 are as follows:

[0018] Step S11: obtaining positive voice information and negative voice information, and counting the total amount of voice information, recorded as vn;

[0019] Step S12: Count the duration of each voice message, recorded as vd1, vd2~vd vn ;

[0020] Extract vd1~vd vn The maximum value in is denoted as vd max , the minimum value is recorded as vd min , the majority is denoted as vd mode ;

[0021] Step S13: Define the relational expression a1: lim(vd i )→vd max ;

[0022] Define the relationship a2: lim(vd i )→vd min ;

[0023] The i in the relational expressions a1 and a2 is an integer, and the value range of i is: 1~vn; vd i Indicates the duration of the i-th voice message;

[0024] Put vd1~vd vn Substitute into the relational expressions a1 and a2, count the number of numbers that satisfy the relational expression a1, record it as vn1; count the number of numbers that satisfy the relational expression a2, record it as vn2;

[0025] Calculate the maximum duration ratio, denoted as va1, va1 = vn1 / vn;

[0026] Calculate the minimum duration ratio, denoted as va2, va2 = vn2 / vn;

[0027] Step S14: Calculate the frame duration of the voice information;

[0028] Calculate the weighted duration vd of the speech information we ;vd we The calculation formula is as follows:

[0029] vd we =(va1×vd max +va2×vd min +vd mode ) / 3;

[0030] Calculate the rated number of samples of speech information, denoted as sq; the calculation formula of sq is as follows:

[0031] sq = [log 60 (vd we )]×1024;

[0032] Calculate the frame duration of a single voice message, recorded as fd; the calculation formula of fd is as follows:

[0033] fd=vd we / sq;

[0034] The frame shift duration of a single voice message is recorded as fsd, fsd = fd / 2;

[0035] Step S15: Count the number of positive voice information, recorded as va; count the number of negative voice information, recorded as vb; va, vb and vn satisfy: va+vb=vn;

[0036] Back up positive voice information and negative voice information, and process the positive voice information:

[0037] The first active voice information is recorded as voice information V, the voice information V is enhanced by pre-emphasis, windowing and Fourier transform, and then the MFCC coefficient of the first voice information is obtained by using Mel filter and discrete cosine transform;

[0038] Step S16: Repeat the same steps of processing the first voice information, process the second to vath active voice information, and obtain the MFCC coefficients of the second to vbth active voice information;

[0039] Step S17: Repeat the same steps of processing the positive voice information to process the negative voice information, obtain the MFCC coefficients of the negative voice information, and enter step S2.

[0040] Furthermore, the specific steps of step S15 are as follows:

[0041] Step S151: Use a high-pass filter to remove low-frequency noise in the voice information to obtain voice information V1; perform frame processing on the voice information V1 according to the frame length fd and the frame shift length fsd to obtain voice information V2;

[0042] The amount of voice information V2 is denoted as F, F = vd1 / fd; F is a positive number and is rounded up;

[0043] Step S152: Using a Hamming window to perform windowing processing on the voice information V2 to obtain voice information V3; the amount of voice information V3 is the same as the amount of voice information V2;

[0044] The function expression w(n) of the Hamming window is as follows:

[0045] w(n)=0.54-0.46cos[(2πn) / (F-1)]; wherein n represents the index of the (n+1)th voice information V2, and the value range of n is: 0-(F-1);

[0046] Step S153: Using Fourier transform, the voice information V3 of each time stamp is converted into the frequency domain information of the corresponding time stamp to obtain information V4; the function expression of Fourier transform is as follows:

[0047]

[0048] Wherein, k represents the index of information V4, n represents the index of voice information V3, and the value range of k and n is: 0~(F-1);

[0049] X(k) represents the information V4 of the kth timestamp, and x(n) represents the voice information V3 of the nth timestamp;

[0050] e represents a natural constant, j represents an imaginary unit (i.e. j 2=-1), π represents the ratio of a circle to a circle.

[0051] Furthermore, the subsequent steps of step S153 are as follows:

[0052] Step S154: Obtain the frequency of each information V4, recorded as f1, f2~f F ;

[0053] Using a Mel-scale triangular filter, the received information V4 simulates the nonlinear frequency perception of human hearing and obtains information V5;

[0054] The number of triangular filters is F;

[0055] Set the frequency scanning range of the triangle filter to: 0 Hz ~ (2 / F) Hz;

[0056] Convert the V4 frequency of the information corresponding to each timestamp to the Mel frequency. The conversion formula is as follows:

[0057] F mel (f i )=2595×lg(1+f i / 700); where i represents the number of information V4, f i represents the frequency of the i-th information V4; F mel (f i ) indicates f in Mel i The perceived frequency;

[0058] Summary F mel (f1)~F mel (f F ) as information V5;

[0059] Step S155: performing discrete cosine transform on the perceived frequency of the information V4 to obtain the MFCC coefficient of the first voice information;

[0060] The formula for discrete cosine transform is as follows:

[0061]

[0062] Among them, MFCC(i) represents the MFCC coefficient corresponding to the i-th information V5;

[0063] Sum up MFCC(1) to MFCC(F) as the MFCC coefficients of the first active speech information.

[0064] Furthermore, the specific steps of step S2 are as follows:

[0065] Step S21: performing homomorphic and cepstrum processing on the active speech to obtain the LPCC coefficient of the active speech information;

[0066] Step S22: Repeat the same steps of performing homomorphic and cepstrum processing on the positive speech to process the negative speech information and obtain the LPCC coefficient of the negative speech information;

[0067] Step S23: Obtain the model provided by the user as the original model;

[0068] Converting positive voice information and negative voice information into corresponding positive voice text and negative voice text;

[0069] In the format of active speech text—MFCC coefficient of active speech information—LPCC coefficient of active speech information, the active speech information is summarized as a sample set α;

[0070] In the format of negative speech text-MFCC coefficient of negative speech information-LPCC coefficient of negative speech information, the negative speech information is summarized as the sample set β;

[0071] Step S24: Define the training process for supervised training of the original model:

[0072] Process a: Extract the MFCC coefficients corresponding to all active speech information to construct MFCC sequence I, denoted as CmfⅠ, CmfⅠ={MFCC1, MFCC2~MFCC va};

[0073] Extract the LPCC coefficients corresponding to all active speech information to construct LPCC sequence I, denoted as ClpⅠ, ClpⅠ={LPCC1, LPCC2~LPCC va};

[0074] Process b: Extract the MFCC coefficients corresponding to all negative speech information to construct MFCC sequence II, denoted as CmfⅡ, CmfⅡ={MFCC1`, MFCC2`~MFCC vb `};

[0075] Extract the LPCC coefficients corresponding to all negative speech information to construct LPCC sequence II, denoted as ClpⅡ, ClpⅡ={LPCC1`, LPCC2`~LPCC vb `};

[0076] Process c: Fuse CmfⅠ and ClpⅠ to obtain the positive voiceprint feature sequence, denoted as CⅠ, CⅠ={(MFCC1, LPCC1), (MFCC2, LPCC2)~(MFCC va ,LPCC va )};

[0077] The active voiceprint feature sequence is obtained by fusing CmfⅡ and ClpⅡ, which is recorded as CⅡ, where CⅡ = {(MFCC1`, LPCC1`), (MFCC2`, LPCC2`) ~ (MFCC vb `,LPCC vb `)};

[0078] Process d: Take the positive speech text—CⅠ as the feature set of sample set α, recorded as feature set α1; take the negative speech text—CⅡ as the feature set of sample set β, recorded as feature set β1;

[0079] Process e: Set the number of states in the original model to 4 and the number of Gaussian elements to 3;

[0080] Bring the feature set α1 and the feature set β1 into the original model respectively, and iterate until the speech information corresponding to the feature set α1 and the feature set β1 are output by the original model;

[0081] Process f: Set the judgment condition for model convergence: the difference between the two adjacent output probabilities is less than 1×10 -4 ;

[0082] Set the cross-validation algorithm as the testing algorithm (to prevent the original model from overfitting).

[0083] Furthermore, the subsequent steps of step S24 are as follows:

[0084] Step S25: Define the training process for unsupervised training of the original model:

[0085] Process 1: Repeat process a to process c to obtain positive voiceprint feature sequence CⅠ and positive voiceprint feature sequence CⅡ;

[0086] Process 2: Using CI and CⅡ as training set 1, and positive speech text and negative speech text as training set 2, the original model is iterated using the K-means clustering algorithm until the elements in training set 1 and the elements in training set 2 are output simultaneously and the output elements correspond;

[0087] Process 3: Set the judgment condition for model convergence: the difference between the two adjacent output probabilities is less than 1×10 -4 ;

[0088] Set the cross-validation algorithm as the test algorithm (to prevent overfitting of the original model);

[0089] Step S26: Perform supervised training and unsupervised training on the original model until the output results of the supervised training and the unsupervised training of the original model are the same, thereby obtaining a pending model;

[0090] Step S27: Input the positive voice information and the negative voice information into the model to be tested to obtain the pending output, and record the response time of the pending output as the pending time, and enter step S3.

[0091] Furthermore, the specific steps of step S21 are as follows:

[0092] Step S211: performing homomorphic and cepstrum processing on the first active voice information; performing framing, windowing and Fourier transformation on the first active voice information with the frame length fd and the frame shift length fsd to obtain discrete time information of the first active voice information, recorded as information E; the number of information E is F;

[0093] Get the frequency corresponding to each piece of information E, recorded as x1, x2~x F ;

[0094] Step S212: performing Z transformation, logarithmic transformation and inverse Z transformation on the 1st to Fth information E;

[0095] The function expression of Z transform is as follows:

[0096] Among them, i represents the number of information E, x i represents the frequency of the i-th information E; X(z) represents the output result of Z transforming the 1st to the i-th information E;

[0097] z i The complex variable representing the i-th information E, z i The calculation formula is as follows:

[0098] z i =(vc×x i )×e(j×ω) -1 ; where vc represents the speed of sound, e represents the natural constant, and j represents the imaginary unit j 2 =-1; ω represents the phase of the i-th information E, and the calculation formula of ω is as follows:

[0099] ω=arctan[(vc×x i ) / fd]; where arctan represents the inverse tangent function;

[0100] The function expression of logarithmic transformation is as follows:

[0101] Wherein, lnX(z) represents the output result of logarithmic transformation of X(z);

[0102] Use the iztrans function to perform inverse Z transform on lnX(z) to convert the information E from frequency domain information to time domain information, denoted as ts1, ts2~ts F ;

[0103] Step S213: ts1 to ts F Perform cepstrum processing to obtain the LPC coefficients of the 1st to Fth information E, denoted as ce1~ce F ;

[0104] The function expression of cepstrum processing is as follows:

[0105] Wherein, i and k both represent the number of information E, and the value range of i and k is: 1~F; x i represents the frequency of the i-th information E, ts i represents the time domain information corresponding to the i-th information E, ce i represents the LPC coefficient of the i-th information E;

[0106] Using the discrete cosine transform algorithm, ce1~ce F Convert the time domain information to the frequency domain information to obtain the LPCC coefficient of the first active speech information;

[0107] Step S214: Repeat the same steps of performing homomorphic and cepstrum processing on the first active voice information, process the second to the vath active voice information, and obtain the LPCC coefficients of the second to the vath active voice information.

[0108] Furthermore, the specific steps of step S3 are as follows:

[0109] Step S31: taking the positive voice information and the negative voice information as inputs of the hot model, obtaining a standard output, and recording the response time of the hot model as the standard time;

[0110] Step S32: using the difflib module and the fuzzywuzzy library, calculating the text matching degree of the pending output relative to the standard output, denoted as tmd; denoting the pending time as ta, and the standard time as tb;

[0111] Step S33: define a relational expression C: [lim(tmd)→1]∩(ta≤tb);

[0112] Determine whether relation C holds true;

[0113] If it is true, it means that the performance of the undetermined model is stronger than that of the popular model, and it will not be processed;

[0114] If not, the performance of the undetermined model is weaker than that of the popular model, and the outputs of the convolutional layer, pooling layer, and fully connected layer of the undetermined model are obtained in sequence, and the process goes to step S34;

[0115] Step S34: Substitute the outputs of the convolutional layer, pooling layer, and fully connected layer of the undetermined model into the convolutional layer, pooling layer, and fully connected layer corresponding to the popular model to determine whether the output of the popular model is the same as the standard output;

[0116] Step S341: Substitute the output of the convolution layer of the undetermined model into the convolution layer of the popular model, execute the popular model to obtain the output a1; determine whether the output a1 is the same as the standard output;

[0117] If the output a1 is the same as the standard output, it means that the convolutional layer of the original model is good;

[0118] If the output a1 is different from the standard output, it means that the convolution layer of the original model is abnormal;

[0119] Step S342: Substitute the output of the pooling layer of the undetermined model into the pooling layer of the popular model, execute the popular model to obtain output a2; determine whether the output a2 is the same as the standard output;

[0120] If the output a2 is the same as the standard output, it means that the pooling layer of the original model is good;

[0121] If the output a2 is different from the standard output, it means that the pooling layer of the original model is abnormal;

[0122] Step S343: Substitute the output of the fully connected layer of the undetermined model into the fully connected layer of the popular model, execute the popular model to obtain output a3; determine whether the output a3 is the same as the standard output;

[0123] If the output a3 is the same as the standard output, it means that the fully connected layer of the original model is good;

[0124] If the output a3 is different from the standard output, it means that the fully connected layer of the original model is abnormal;

[0125] Step S35: Summarize the analysis results of step S34 as an improvement report of the original model.

[0126] A speech emotion recognition model training system includes:

[0127] Speech acquisition module: used to obtain positive and negative speech information, and perform pre-emphasis, windowing and Fourier transform on each piece of speech information, and then perform Mel transform and discrete cosine transform on each piece of speech information to obtain the MFCC coefficient of each piece of speech information;

[0128] Model training module: used to perform homomorphic and cepstrum processing on each voice information to obtain the LPCC coefficient of each voice information; obtain the original model; convert the voice information into text information, and divide the positive voice information and the negative voice information into sample set α and sample set β in the format of text information-MFCC coefficient-LPCC coefficient;

[0129] Take sample set α and sample set β as independent inputs to perform supervised training on the original model; take sample set α and sample set β as mixed inputs to perform unsupervised training on the original model; until the output results of supervised training and unsupervised training of the original model are the same, the undetermined model is obtained;

[0130] Inputting positive speech information and negative speech information into the model to be tested to obtain a pending output;

[0131] Comparative analysis module: used to take positive and negative speech information as inputs of popular models and obtain standard outputs;

[0132] Compare the performance of the proposed model and the popular model based on the proposed output and the standard output;

[0133] If the performance of the candidate model is weaker than that of the popular model, the outputs of the convolutional layer, pooling layer, and fully connected layer of the candidate model are obtained in sequence, and substituted into the convolutional layer, pooling layer, and fully connected layer corresponding to the popular model to analyze the weaknesses of the candidate model and obtain an improvement report;

[0134] If the performance of the pending model is stronger than that of the popular model, it will not be processed;

[0135] Voice update module: used to summarize the improvement report and provide feedback; obtain new positive voice information and new negative voice information, and re-execute the training and analysis process of the original model until the performance of the original model approaches or exceeds that of the popular model.

[0136] Compared with the prior art, the present invention has the following beneficial effects:

[0137] Improve service quality: The present invention processes voice information and trains machine models based on multiple data sources, multiple angles, and multiple aspects, so that the model can analyze the customer's voice emotions in real time; when the machine model trained based on the present invention detects that the customer expresses dissatisfaction or anger, the model can quickly identify and remind relevant personnel to prevent unnecessary conflicts.

[0138] The training effect is significant: before model training, the present invention extracts voice information from multiple angles, adopts multiple training modes during the training process, and compares the trained model with the popular model for optimization training, so that the model can continuously learn and optimize during the training process, thereby improving the accuracy and stability of emotion recognition.

[0139] Reduce labor costs: The present invention can automatically identify the emotional state of customers, reduce manual intervention, reduce labor costs, and is suitable for large-scale processing of voice data. BRIEF DESCRIPTION OF THE DRAWINGS

[0140] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0141] Figure 1 It is a schematic diagram of the method of the present invention;

[0142] Figure 2 It is a schematic diagram of the system of the present invention;

[0143] Figure 3 It is a schematic diagram of language information extraction of the present invention. DETAILED DESCRIPTION

[0144] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0145] Embodiment 1

[0146] See also Figure 1 and Figure 3 , a speech emotion recognition model training method includes:

[0147] Step S1: Acquire positive voice information and negative voice information, and perform a primary process of pre-emphasis, windowing and Fourier transform on each voice information, and then perform a secondary process of Mel transform and discrete cosine transform to obtain the MFCC coefficient of each voice information;

[0148] The specific steps of step S1 are as follows:

[0149] Step S11: obtaining positive voice information and negative voice information, and counting the total amount of voice information (i.e. the total amount of positive voice information and negative voice information), recorded as vn;

[0150] It should be noted that the "positive voice information" in the present invention refers to voices related to positive emotions (such as happiness, joy, optimism, etc.); "negative voice information" refers to voices related to negative emotions (such as frustration, loss, anger, etc.);

[0151] The information sources of the positive and negative voice information in the present invention are: open voice libraries, and common open voice libraries include LibriSpeech, VoxForge, TIMIT, etc.

[0152] Step S12: Count the duration of each voice message, recorded as vd1, vd2~vd vn ; (where vd1 represents the duration of the first voice message; vd2 represents the duration of the second voice message; and so on, vd vn Indicates the duration of the vnth voice message;)

[0153] Extract vd1~vd vn The maximum value in is denoted as vd max , the minimum value is recorded as vd min , the majority is denoted as vd mode ;

[0154] Step S13: define relational expressions a1 and a2;

[0155] Relational formula a1: lim(vd i )→vd max ;

[0156] Relationship a2: lim(vd i )→vd min ;

[0157] The i in the relational expressions a1 and a2 is an integer, and the value range of i is: 1~vn; vd i Indicates the duration of the i-th voice message;

[0158] Put vd1~vd vn Substitute into the relational expressions a1 and a2, count the number of numbers that satisfy the relational expression a1, record it as vn1; count the number of numbers that satisfy the relational expression a2, record it as vn2;

[0159] Calculate the maximum duration ratio, denoted as va1, va1 = vn1 / vn;

[0160] Calculate the minimum duration ratio, denoted as va2, va2 = vn2 / vn;

[0161] Step S14: Calculate the frame duration of the voice information;

[0162] Calculate the weighted duration vd of the speech information we ;vd we The calculation formula is as follows:

[0163] vd we =(va1×vd max +va2×vd min +vd mode ) / 3;

[0164] Calculate the rated number of samples of speech information, denoted as sq; the calculation formula of sq is as follows:

[0165] sq = [log 60 (vd we )]×1024;

[0166] Calculate the frame duration of a single voice message, recorded as fd; the calculation formula of fd is as follows:

[0167] fd=vd we / sq; (fd unit: milliseconds)

[0168] The frame shift duration of a single voice message is recorded as fsd, fsd = fd / 2;

[0169] Step S15: Count the number of positive voice information, recorded as va; count the number of negative voice information, recorded as vb; va, vb and vn satisfy: va+vb=vn;

[0170] Back up positive voice information and negative voice information, and process the positive voice information:

[0171] The first active voice information is recorded as voice information V, the voice information V is enhanced by pre-emphasis, windowing and Fourier transform, and then the MFCC coefficient of the first voice information is obtained by using Mel filter and discrete cosine transform;

[0172] Step S151: Use a high-pass filter to remove low-frequency noise in the voice information to obtain voice information V1; perform frame processing on the voice information V1 according to the frame length fd and the frame shift length fsd to obtain voice information V2;

[0173] The number of speech information V2 (i.e., the number of samples) is denoted as F, F = vd1 / fd; F is a positive number, rounded up;

[0174] Step S152: Using a Hamming window to perform windowing processing on the voice information V2 to obtain voice information V3; the amount of voice information V3 is the same as the amount of voice information V2;

[0175] The function expression w(n) of the Hamming window is as follows:

[0176] w(n)=0.54-0.46cos[(2πn) / (F-1)]; where n represents the index of the (n+1)th voice information V2, and the value range of n is: 0-(F-1); (the reason why n starts counting from zero is to match the default counting method inside the computer, which starts counting from zero by default)

[0177] Step S153: Using Fourier transform, the voice information V3 (time domain information) of each time stamp is converted into the frequency domain information of the corresponding time stamp to obtain information V4; the function expression of Fourier transform is as follows:

[0178]

[0179] Wherein, k represents the index of information V4 (i.e., frequency domain information), n represents the index of voice information V3 (i.e., time domain information), and the value ranges of k and n are both: 0~(F-1);

[0180] X(k) represents the information V4 (i.e., frequency domain information) of the kth timestamp, and x(n) represents the speech information V3 (i.e., time domain information) of the nth timestamp;

[0181] e represents a natural constant, j represents an imaginary unit (i.e. j 2 =-1), π represents the ratio of circumference to diameter;

[0182] Step S154: Obtain the frequency of each information V4, recorded as f1, f2~f F ; (where f1 represents the frequency of information V4 corresponding to the first timestamp; f2 represents the frequency of information V4 corresponding to the second timestamp; and so on, f F Indicates the frequency of information V4 corresponding to the Fth timestamp;)

[0183] Using a Mel-scale triangular filter, the received information V4 (i.e., frequency domain information) simulates the nonlinear frequency perception of human hearing and obtains information V5;

[0184] The number of triangular filters is F; (the reason for setting the number of triangular filters to F here is to prevent interference between two pieces of information V4 of adjacent timestamps when performing Mel frequency conversion;

[0185] Users or relevant technicians can adjust the number of triangle filters according to actual needs)

[0186] Set the frequency scanning range of the triangle filter to: 0 Hz ~ (2 / F) Hz;

[0187] Convert the V4 frequency of the information corresponding to each timestamp to the Mel frequency. The conversion formula is as follows:

[0188] F mel (f i )=2595×lg(1+f i / 700); where i represents the number of information V4, f i represents the frequency of the i-th information V4; F mel (f i ) indicates f in Mel i The perceived frequency (i.e. Mel frequency);

[0189] Summary F mel (f1)~F mel (f F ) as information V5;

[0190] Step S155: performing discrete cosine transform on the perceived frequency of the information V4 to obtain the MFCC coefficient of the first voice information;

[0191] The formula for discrete cosine transform is as follows:

[0192]

[0193] Among them, MFCC(i) represents the MFCC coefficient corresponding to the i-th information V5;

[0194] Summarize MFCC(1) to MFCC(F) as the MFCC coefficients of the first active speech information;

[0195] Step S16: Repeat the same steps of processing the first voice information (i.e., steps S151 to S155), process the second to vath active voice information, and obtain the MFCC coefficients of the second to vbth active voice information;

[0196] Step S17: Repeat the same steps of processing the positive voice information (ie, steps S15 to S16) to process the negative voice information and obtain the MFCC coefficients of the negative voice information.

[0197] Step S2: Perform homomorphic and cepstrum processing on each voice information to obtain the LPCC coefficient of each voice information; obtain the model provided by the user as the original model; convert the voice information into text information, and divide the positive voice information and the negative voice information into sample set α and sample set β in the format of text information-MFCC coefficient-LPCC coefficient;

[0198] Take sample set α and sample set β as independent inputs and perform supervised training on the original model; take sample set α and sample set β as mixed inputs and perform unsupervised training on the original model; until the output results of supervised training and unsupervised training of the original model are the same (model training is completed), the pending model is obtained;

[0199] Input the positive speech information and the negative speech information into the model to be tested, and obtain the output to be determined (i.e., the output of the model to be tested);

[0200] The specific steps of step S2 are as follows:

[0201] Step S21: performing homomorphic and cepstrum processing on the active speech to obtain the LPCC coefficient of the active speech information;

[0202] Perform homomorphic and cepstrum processing on the first positive voice information;

[0203] Step S211: framing, windowing and Fourier transforming the first active voice information with the frame duration fd and the frame shift duration fsd (i.e., steps S151 to S153) to obtain discrete time information of the first active voice information, recorded as information E; the number of information E is F (the same as the number of voice information V2);

[0204] Get the frequency corresponding to each piece of information E, recorded as x1, x2~x F ; (where x1 represents the frequency of the first information E; x2 represents the frequency of the second information E; and so on, x F represents the frequency of the Fth information E)

[0205] Step S212: performing Z transformation, logarithmic transformation and inverse Z transformation on the 1st to Fth information E;

[0206] The function expression of Z transform is as follows:

[0207] Among them, i represents the number of information E, x i represents the frequency of the i-th information E; X(z) represents the output result of Z transforming the 1st to the i-th information E;

[0208] z i The complex variable representing the i-th information E, z i The calculation formula is as follows:

[0209] z i =(vc×x i )×e(j×ω) -1 ; where vc represents the speed of sound, e represents the natural constant, and j represents the imaginary unit j 2 =-1; ω represents the phase of the i-th information E, and the calculation formula of ω is as follows:

[0210] ω=arctan[(vc×x i ) / fd]; where arctan represents the inverse tangent function;

[0211] The function expression of logarithmic transformation is as follows:

[0212] Wherein, lnX(z) represents the output result of logarithmic transformation of X(z);

[0213] Use the iztrans function to perform inverse Z transform on lnX(z) to convert the information E from frequency domain information to time domain information, denoted as ts1, ts2~ts F ; (where ts1 represents the time domain information corresponding to the first information E; ts2 represents the time domain information corresponding to the second information E; and so on, ts F Indicates the time domain information corresponding to the Fth information E;)

[0214] Step S213: ts1 to ts F Perform cepstrum processing to obtain the LPC coefficients of the 1st to Fth information E, denoted as ce1~ce F ;

[0215] The function expression of cepstrum processing is as follows:

[0216] Wherein, i and k both represent the number of information E, and the value range of i and k is: 1~F; x i represents the frequency of the i-th information E, ts i represents the time domain information corresponding to the i-th information E, ce i represents the LPC coefficient of the i-th information E;

[0217] Using the discrete cosine transform algorithm, ce1~ce F Convert the time domain information to the frequency domain information to obtain the LPCC coefficient of the first active speech information;

[0218] Step S214: Repeat the same steps of performing homomorphic and cepstrum processing on the first active voice information (steps S211 to S213), process the second to vath active voice information, and obtain the LPCC coefficients of the second to vath active voice information;

[0219] Step S22: Repeat the same steps of performing homomorphic and cepstrum processing on the positive speech to process the negative speech information and obtain the LPCC coefficient of the negative speech information;

[0220] Step S23: Obtain the model provided by the user as the original model;

[0221] Use the recognize_google function and recognize_sphinx function in the SpeechRecognition library to convert positive speech information and negative speech information into corresponding positive speech text and negative speech text;

[0222] In the format of active speech text—MFCC coefficient of active speech information—LPCC coefficient of active speech information, the active speech information is summarized as a sample set α;

[0223] In the format of negative speech text-MFCC coefficient of negative speech information-LPCC coefficient of negative speech information, the negative speech information is summarized as the sample set β;

[0224] Step S24: Define the training process for supervised training of the original model:

[0225] Process a: Extract the MFCC coefficients corresponding to all active speech information to construct MFCC sequence I, denoted as CmfⅠ, CmfⅠ={MFCC1, MFCC2~MFCC va}; (where MFCC1 represents the MFCC coefficient of the first active voice information; MFCC2 represents the MFCC coefficient of the second active voice information; and so on. va MFCC coefficients representing the vath positive speech information)

[0226] Extract the LPCC coefficients corresponding to all active speech information to construct LPCC sequence I, denoted as ClpⅠ, ClpⅠ={LPCC1, LPCC2~LPCC va}; (where LPCC1 represents the LPCC coefficient of the first positive voice information; LPCC2 represents the LPCC coefficient of the second positive voice information; and so on. va LPCC coefficient representing the vath positive speech information)

[0227] Process b: Extract the MFCC coefficients corresponding to all negative speech information to construct MFCC sequence II, denoted as CmfⅡ, CmfⅡ={MFCC1`, MFCC2`~MFCC vb `}; (where MFCC1` represents the MFCC coefficient of the first negative voice information; MFCC2` represents the MFCC coefficient of the second negative voice information; and so on, MFCC vb `MFCC coefficients representing the negative speech information of the vbth item)

[0228] Extract the LPCC coefficients corresponding to all negative speech information to construct LPCC sequence II, denoted as ClpⅡ, ClpⅡ={LPCC1`, LPCC2`~LPCC vb `}; (where LPCC1` represents the LPCC coefficient of the first negative voice information; LPCC2` represents the LPCC coefficient of the second negative voice information; and so on, LPCC vb `LPCC coefficient representing the negative speech information of item vb)

[0229] Process c: Fuse CmfⅠ and ClpⅠ to obtain the positive voiceprint feature sequence, denoted as CⅠ, CⅠ={(MFCC1, LPCC1), (MFCC2, LPCC2)~(MFCC va ,LPCC va )};

[0230] The active voiceprint feature sequence is obtained by fusing CmfⅡ and ClpⅡ, which is recorded as CⅡ, where CⅡ = {(MFCC1`, LPCC1`), (MFCC2`, LPCC2`) ~ (MFCC vb `,LPCC vb `)};

[0231] Process d: Take the positive speech text—CⅠ as the feature set of sample set α, recorded as feature set α1; take the negative speech text—CⅡ as the feature set of sample set β, recorded as feature set β1;

[0232] Process e: Set the number of states in the original model to 4 and the number of Gaussian elements to 3;

[0233] Bring the feature set α1 and the feature set β1 into the original model respectively, and iterate until the speech information corresponding to the feature set α1 and the feature set β1 are output by the original model;

[0234] Process f: Set the judgment condition for model convergence: the difference between the two adjacent output probabilities (positive) is less than 1×10 -4 ;

[0235] Set the cross-validation algorithm as the test algorithm (to prevent overfitting of the original model);

[0236] Step S25: Define the training process for unsupervised training of the original model:

[0237] Process 1: Repeat process a to process c to obtain positive voiceprint feature sequence CⅠ and positive voiceprint feature sequence CⅡ;

[0238] Process 2: Use CI and CⅡ as training set 1, and positive voice text and negative voice text as training set 2, and use the K-means clustering algorithm to iterate the original model until the elements in training set 1 and the elements in training set 2 are output at the same time and the output elements correspond (that is, the (positive or negative) voice text output by training set 1 corresponds to the (positive or negative) voiceprint feature sequence output by training set 2);

[0239] Process 3: Set the judgment condition for model convergence: the difference between the two adjacent output probabilities (positive) is less than 1×10 -4 ;

[0240] Set the cross-validation algorithm as the test algorithm (to prevent overfitting of the original model);

[0241] Step S26: Perform supervised training and unsupervised training on the original model until the output results of the supervised training and unsupervised training of the original model are the same (model training is completed), and a pending model is obtained;

[0242] Step S27: input the positive voice information and the negative voice information into the model to be tested, obtain the pending output (ie, the output of the model to be tested), and record the response time of the pending output, which is recorded as the pending time.

[0243] Step S3: (directly) taking the positive speech information and the negative speech information as inputs of the hot model to obtain a standard output (i.e., the output of the hot model);

[0244] Compare the performance of the proposed model and the popular model based on the proposed output and the standard output;

[0245] If the performance of the candidate model is weaker than that of the popular model, the outputs of the convolutional layer, pooling layer, and fully connected layer of the candidate model are obtained in sequence, and substituted into the convolutional layer, pooling layer, and fully connected layer corresponding to the popular model to analyze the weaknesses of the candidate model and obtain an improvement report;

[0246] If the performance of the pending model is stronger than that of the popular model, it will not be processed;

[0247] The specific steps of step S3 are as follows:

[0248] Step S31: (directly) taking the positive voice information and the negative voice information as the input of the hot model, obtaining the standard output (i.e. the output of the hot model), and recording the response time of the hot model as the standard time;

[0249] It should be noted that the “popular models” in the present invention refer to language emotion recognition models that are widely used on the market. Common popular models include: GPT, BERT, RoBERTa, etc.

[0250] Step S32: using the difflib module and the fuzzywuzzy library, calculating the text matching degree of the pending output relative to the standard output, denoted as tmd; denoting the pending time as ta, and the standard time as tb;

[0251] Step S33: define a relational expression C: [lim(tmd)→1]∩(ta≤tb);

[0252] Determine whether relation C holds true;

[0253] If it is true, it means that the performance of the undetermined model is stronger than that of the popular model, and it will not be processed;

[0254] If not, the performance of the undetermined model is weaker than that of the popular model, and the outputs of the convolutional layer, pooling layer, and fully connected layer of the undetermined model are obtained in sequence, and the process goes to step S34;

[0255] Step S34: Substitute the outputs of the convolutional layer, pooling layer, and fully connected layer of the undetermined model into the convolutional layer, pooling layer, and fully connected layer corresponding to the popular model to determine whether the output of the popular model is the same as the standard output;

[0256] Step S341: Substitute the output of the convolution layer of the undetermined model into the convolution layer of the popular model, execute the popular model to obtain the output a1; determine whether the output a1 is the same as the standard output;

[0257] If the output a1 is the same as the standard output, it means that the convolutional layer of the original model is good;

[0258] If the output a1 is different from the standard output, it means that the convolution layer of the original model is abnormal;

[0259] Step S342: Substitute the output of the pooling layer of the undetermined model into the pooling layer of the popular model, execute the popular model to obtain output a2; determine whether the output a2 is the same as the standard output;

[0260] If the output a2 is the same as the standard output, it means that the pooling layer of the original model is good;

[0261] If the output a2 is different from the standard output, it means that the pooling layer of the original model is abnormal;

[0262] Step S343: Substitute the output of the fully connected layer of the undetermined model into the fully connected layer of the popular model, execute the popular model to obtain output a3; determine whether the output a3 is the same as the standard output;

[0263] If the output a3 is the same as the standard output, it means that the fully connected layer of the original model is good;

[0264] If the output a3 is different from the standard output, it means that the fully connected layer of the original model is abnormal;

[0265] Step S35: Summarize the analysis results of step S34 as an improvement report of the original model.

[0266] Step S4: Summarize the improvement report and provide feedback; obtain new positive voice information and new negative voice information, and perform the training and analysis process of the original model again (i.e., steps S1 to S3) until the performance of the original model approaches or exceeds that of the popular model;

[0267] It should be noted that the information source of the “new positive voice information and new negative voice information” in step S4 must be different from the information source of the “positive voice information and negative voice information” in step S1.

[0268] Embodiment 2

[0269] See also Figure 2 A speech emotion recognition model training system includes: a speech acquisition module, a model training module, a comparison and analysis module, a speech update module, a database and a server; wherein the speech acquisition module, the model training module, the comparison and analysis module and the speech update module are connected to the database and the server respectively.

[0270] Speech acquisition module: used to obtain positive and negative speech information, and perform pre-emphasis, windowing and Fourier transform on each piece of speech information, and then perform Mel transform and discrete cosine transform on each piece of speech information to obtain the MFCC coefficient of each piece of speech information;

[0271] Model training module: used to perform homomorphic and cepstrum processing on each voice information to obtain the LPCC coefficient of each voice information; obtain the model provided by the user as the original model; convert the voice information into text information, and divide the positive voice information and the negative voice information into sample set α and sample set β in the format of text information-MFCC coefficient-LPCC coefficient;

[0272] Take sample set α and sample set β as independent inputs and perform supervised training on the original model; take sample set α and sample set β as mixed inputs and perform unsupervised training on the original model; until the output results of supervised training and unsupervised training of the original model are the same (model training is completed), the pending model is obtained;

[0273] Input the positive speech information and the negative speech information into the model to be tested, and obtain the output to be determined (i.e., the output of the model to be tested);

[0274] Comparative analysis module: used to (directly) use positive speech information and negative speech information as input of the popular model to obtain the standard output (i.e. the output of the popular model);

[0275] Compare the performance of the proposed model and the popular model based on the proposed output and the standard output;

[0276] If the performance of the candidate model is weaker than that of the popular model, the outputs of the convolutional layer, pooling layer, and fully connected layer of the candidate model are obtained in sequence, and substituted into the convolutional layer, pooling layer, and fully connected layer corresponding to the popular model to analyze the weaknesses of the candidate model and obtain an improvement report;

[0277] If the performance of the pending model is stronger than that of the popular model, it will not be processed;

[0278] Voice update module: used to summarize the improvement report and provide feedback; obtain new positive voice information and new negative voice information, and re-execute the training and analysis process of the original model (i.e., steps S1 to S3) until the performance of the original model approaches or exceeds that of the popular model.

[0279] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technical personnel in this field according to actual conditions. For example, if there are weight coefficients and proportional coefficients, their set sizes are to quantify each parameter to obtain a specific value, which is convenient for subsequent comparison. Regarding the size of the weight coefficient and the proportional coefficient, it is sufficient as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0280] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A speech emotion recognition model training method, characterized in that: Training methods include: Step S1: Acquire positive voice information and negative voice information; back up the positive voice information and negative voice information, and perform a primary process of pre-emphasis, windowing and Fourier transform on each voice information, and then perform a secondary process of Mel transform and discrete cosine transform to obtain the MFCC coefficient of each voice information; Step S2: Perform homomorphic and cepstrum processing on each voice information to obtain the LPCC coefficient of each voice information; obtain the original model; convert the voice information into text information, and divide the positive voice information and the negative voice information into sample set α and sample set β in the format of text information-MFCC coefficient-LPCC coefficient; Take sample set α and sample set β as independent inputs to perform supervised training on the original model; take sample set α and sample set β as mixed inputs to perform unsupervised training on the original model; until the output results of supervised training and unsupervised training of the original model are the same, the undetermined model is obtained; Inputting positive speech information and negative speech information into the model to be tested to obtain a pending output; Step S3: taking the positive speech information and the negative speech information as inputs of the hot model and obtaining a standard output; Compare the performance of the proposed model and the popular model based on the proposed output and the standard output; If the performance of the candidate model is weaker than that of the popular model, the outputs of the convolutional layer, pooling layer, and fully connected layer of the candidate model are obtained in sequence, and substituted into the convolutional layer, pooling layer, and fully connected layer corresponding to the popular model to analyze the weaknesses of the candidate model and obtain an improvement report; If the performance of the pending model is stronger than that of the popular model, it will not be processed; Step S4: Summarize the improvement report and provide feedback; obtain new positive voice information and new negative voice information, and perform the training and analysis process of the original model again until the performance of the original model is close to or stronger than that of the popular model.

2. A speech emotion recognition model training method according to claim 1, characterized in that: The specific steps of step S1 are as follows: Step S11: obtaining positive voice information and negative voice information, and counting the total amount of voice information, recorded as vn; Step S12: Count the duration of each voice message, recorded as vd1, vd2~vd vn ; Extract vd1~vd vn The maximum value in is denoted as vd max , the minimum value is recorded as vd min , the majority is denoted as vd mode ; Step S13: Define the relational expression a1: lim(vd i )→vd max ; Define the relationship a2: lim(vd i )→vd min ; The i in the relational expressions a1 and a2 is an integer, and the value range of i is: 1~vn; vd i Indicates the duration of the i-th voice message; Put vd1~vd vn Substitute into the relational expressions a1 and a2, count the number of numbers that satisfy the relational expression a1, record it as vn1; count the number of numbers that satisfy the relational expression a2, record it as vn2; Calculate the maximum duration ratio, denoted as va1, va1 = vn1 / vn; Calculate the minimum duration ratio, denoted as va2, va2 = vn2 / vn; Step S14: Calculate the frame duration of the voice information; Calculate the weighted duration vd of the speech information we ;vd we The calculation formula is as follows: vd we =(va1×vd max +va2×vd min +vd mode ) / 3; Calculate the rated number of samples of speech information, denoted as sq; the calculation formula of sq is as follows: sq=[log 60 (vd we )]×1024; Calculate the frame duration of a single voice message, recorded as fd; the calculation formula of fd is as follows: fd=vd we / sq; The frame shift duration of a single voice message is recorded as fsd, fsd = fd / 2; Step S15: Count the number of positive voice information, recorded as va; count the number of negative voice information, recorded as vb; va, vb and vn satisfy: va+vb=vn; Back up positive voice information and negative voice information, and process the positive voice information: The first active voice information is recorded as voice information V, the voice information V is enhanced by pre-emphasis, windowing and Fourier transform, and then the MFCC coefficient of the first voice information is obtained by using Mel filter and discrete cosine transform; Step S16: Repeat the same steps of processing the first voice information, process the second to vath active voice information, and obtain the MFCC coefficients of the second to vbth active voice information; Step S17: Repeat the same steps of processing the positive voice information to process the negative voice information, obtain the MFCC coefficients of the negative voice information, and enter step S2.

3. A speech emotion recognition model training method according to claim 2, characterized in that: The specific steps of step S15 are as follows: Step S151: Use a high-pass filter to remove low-frequency noise in the voice information to obtain voice information V1; perform frame processing on the voice information V1 according to the frame length fd and the frame shift length fsd to obtain voice information V2; The amount of voice information V2 is denoted as F, F = vd1 / fd; F is a positive number and is rounded up; Step S152: Using a Hamming window to perform windowing processing on the voice information V2 to obtain voice information V3; The function expression w(n) of the Hamming window is as follows: w(n)=0.54-0.46cos[(2πn) / (F-1)]; wherein n represents the index of the (n+1)th voice information V2, and the value range of n is: 0-(F-1); Step S153: Using Fourier transform, the voice information V3 of each time stamp is converted into the frequency domain information of the corresponding time stamp to obtain information V4; the function expression of Fourier transform is as follows: Wherein, k represents the index of information V4, n represents the index of voice information V3, and the value range of k and n is: 0~(F-1); X(k) represents the information V4 of the kth timestamp, and x(n) represents the voice information V3 of the nth timestamp; e represents a natural constant and j represents an imaginary unit.

4. A speech emotion recognition model training method according to claim 3, characterized in that: The subsequent steps of step S153 are as follows: Step S154: Obtain the frequency of each information V4, recorded as f1, f2~f F ; Use the Mel-scale triangle filter to receive information V4 and obtain information V5; The number of triangular filters is F; Set the frequency scanning range of the triangle filter to: 0 Hz ~ (2 / F) Hz; Convert the V4 frequency of the information corresponding to each timestamp to the Mel frequency. The conversion formula is as follows: F mel (f i )=2595×lg(1+f i / 700); where i represents the number of information V4, f i represents the frequency of the i-th information V4; F mel (f i ) indicates f in Mel i The perceived frequency; Summary F mel (f1)~F mel (f F ) as information V5; Step S155: performing discrete cosine transform on the perceived frequency of the information V4 to obtain the MFCC coefficient of the first voice information; The formula for discrete cosine transform is as follows: Among them, MFCC(i) represents the MFCC coefficient corresponding to the i-th information V5; Sum up MFCC(1) to MFCC(F) as the MFCC coefficients of the first active speech information.

5. A speech emotion recognition model training method according to claim 2, characterized in that: The specific steps of step S2 are as follows: Step S21: performing homomorphic and cepstrum processing on the active speech to obtain the LPCC coefficient of the active speech information; Step S22: Repeat the same steps of performing homomorphic and cepstrum processing on the positive speech to process the negative speech information and obtain the LPCC coefficient of the negative speech information; Step S23: Obtain the model provided by the user as the original model; Converting positive voice information and negative voice information into corresponding positive voice text and negative voice text; In the format of active speech text—MFCC coefficient of active speech information—LPCC coefficient of active speech information, the active speech information is summarized as a sample set α; In the format of negative speech text-MFCC coefficient of negative speech information-LPCC coefficient of negative speech information, the negative speech information is summarized as the sample set β; Step S24: Define the training process for supervised training of the original model: Process a: Extract the MFCC coefficients corresponding to all active speech information to construct MFCC sequence I, denoted as CmfⅠ, CmfⅠ={MFCC1, MFCC2~MFCC va }; Among them, MFCC1 represents the MFCC coefficient of the first positive voice information; MFCC2 represents the MFCC coefficient of the second positive voice information; and so on. va The MFCC coefficients representing the va-th positive speech information; Extract the LPCC coefficients corresponding to all active speech information to construct LPCC sequence I, denoted as ClpⅠ, ClpⅠ={LPCC1, LPCC2~LPCC va }; Among them, LPCC1 represents the LPCC coefficient of the first positive voice information; LPCC2 represents the LPCC coefficient of the second positive voice information; and so on. va The LPCC coefficient representing the va-th positive speech information; Process b: Extract the MFCC coefficients corresponding to all negative speech information to construct MFCC sequence II, denoted as CmfⅡ, CmfⅡ={MFCC1`, MFCC2`~MFCC vb `}; Among them, MFCC1` represents the MFCC coefficient of the first negative voice information; MFCC2` represents the MFCC coefficient of the second negative voice information; and so on. vb `MFCC coefficients representing the vbth negative speech information; Extract the LPCC coefficients corresponding to all negative speech information to construct LPCC sequence II, denoted as ClpⅡ, ClpⅡ={LPCC1`, LPCC2`~LPCC vb `}; Among them, LPCC1` represents the LPCC coefficient of the first negative voice information; LPCC2` represents the LPCC coefficient of the second negative voice information; and so on. vb `LPCC coefficient representing the negative speech information of item vb; Process c: Fuse CmfⅠ and ClpⅠ to obtain the positive voiceprint feature sequence, denoted as CⅠ, CⅠ={(MFCC1, LPCC1), (MFCC2, LPCC2)~(MFCC va ,LPCC va )}; The active voiceprint feature sequence is obtained by fusing CmfⅡ and ClpⅡ, which is recorded as CⅡ, where CⅡ = {(MFCC1`, LPCC1`), (MFCC2`, LPCC2`) ~ (MFCC vb `,LPCC vb `)}; Process d: Take the positive speech text—CⅠ as the feature set of sample set α, recorded as feature set α1; take the negative speech text—CⅡ as the feature set of sample set β, recorded as feature set β1; Process e: Set the number of states in the original model to 4 and the number of Gaussian elements to 3; Bring the feature set α1 and the feature set β1 into the original model respectively, and iterate until the speech information corresponding to the feature set α1 and the feature set β1 are output by the original model; Process f: Set the judgment condition for model convergence: the difference between the two adjacent output probabilities is less than 1×10 -4 ; Set the cross validation algorithm as the testing algorithm.

6. A speech emotion recognition model training method according to claim 5, characterized in that: The subsequent steps of step S24 are as follows: Step S25: Define the training process for unsupervised training of the original model: Process 1: Repeat process a to process c to obtain positive voiceprint feature sequence CⅠ and positive voiceprint feature sequence CⅡ; Process 2: Using CI and CⅡ as training set 1, and positive speech text and negative speech text as training set 2, the original model is iterated using the K-means clustering algorithm until the elements in training set 1 and the elements in training set 2 are output simultaneously and the output elements correspond; Process 3: Set the judgment condition for model convergence: the difference between the two adjacent output probabilities is less than 1×10 -4 ; Set the cross validation algorithm as the testing algorithm; Step S26: Perform supervised training and unsupervised training on the original model until the output results of the supervised training and the unsupervised training of the original model are the same, thereby obtaining a pending model; Step S27: Input the positive voice information and the negative voice information into the model to be tested to obtain the pending output, and record the response time of the pending output as the pending time, and enter step S3.

7. A speech emotion recognition model training method according to claim 5, characterized in that: The specific steps of step S21 are as follows: Step S211: performing homomorphic and cepstrum processing on the first active voice information; performing framing, windowing and Fourier transformation on the first active voice information with the frame length fd and the frame shift length fsd to obtain discrete time information of the first active voice information, recorded as information E; the number of information E is F; Get the frequency corresponding to each piece of information E, recorded as x1, x2~x F ; Step S212: performing Z transformation, logarithmic transformation and inverse Z transformation on the 1st to Fth information E; The function expression of Z transform is as follows: Among them, i represents the number of information E, x i represents the frequency of the i-th information E; X(z) represents the output result of Z-transformation of the 1st to i-th information E; z i The complex variable representing the i-th information E, z i The calculation formula is as follows: z i =(vc×x i )×e(j×ω) -1 ; Where vc represents the speed of sound, e represents the natural constant; j represents the imaginary unit j 2 =-1; ω represents the phase of the i-th information E, and the calculation formula of ω is as follows: ω=arctan[(vc×x i ) / fd]; where arctan represents the inverse tangent function; The function expression of logarithmic transformation is as follows: Wherein, lnX(z) represents the output result of logarithmic transformation of X(z); Use the iztrans function to perform inverse Z transform on lnX(z) to convert the information E from frequency domain information to time domain information, denoted as ts1, ts2~ts F ; Step S213: ts1 to ts F Perform cepstrum processing to obtain the LPC coefficients of the 1st to Fth information E, denoted as ce1~ce F ; The function expression of cepstrum processing is as follows: Wherein, i and k both represent the number of information E, and the value range of i and k is: 1~F; x i represents the frequency of the i-th information E, ts i represents the time domain information corresponding to the i-th information E, ce i represents the LPC coefficient of the i-th information E; Using the discrete cosine transform algorithm, ce1~ce F Convert the time domain information to the frequency domain information to obtain the LPCC coefficient of the first active speech information; Step S214: Repeat the same steps of performing homomorphic and cepstrum processing on the first active voice information, process the second to the vath active voice information, and obtain the LPCC coefficients of the second to the vath active voice information.

8. A speech emotion recognition model training method according to claim 6, characterized in that: The specific steps of step S3 are as follows: Step S31: taking the positive voice information and the negative voice information as inputs of the hot model, obtaining a standard output, and recording the response time of the hot model as the standard time; Step S32: using the difflib module and the fuzzywuzzy library, calculating the text matching degree of the pending output relative to the standard output, denoted as tmd; denoting the pending time as ta, and the standard time as tb; Step S33: define a relational expression C: [lim(tmd)→1]∩(ta≤tb); Determine whether relation C holds true; If it is true, it means that the performance of the undetermined model is stronger than that of the popular model, and it will not be processed; If not, the performance of the undetermined model is weaker than that of the popular model, and the outputs of the convolutional layer, pooling layer, and fully connected layer of the undetermined model are obtained in sequence, and the process goes to step S34; Step S34: Substitute the outputs of the convolutional layer, pooling layer, and fully connected layer of the undetermined model into the convolutional layer, pooling layer, and fully connected layer corresponding to the popular model to determine whether the output of the popular model is the same as the standard output; Step S341: Substitute the output of the convolution layer of the undetermined model into the convolution layer of the popular model, execute the popular model to obtain the output a1; determine whether the output a1 is the same as the standard output; If the output a1 is the same as the standard output, it means that the convolutional layer of the original model is good; If the output a1 is different from the standard output, it means that the convolution layer of the original model is abnormal; Step S342: Substitute the output of the pooling layer of the undetermined model into the pooling layer of the popular model, execute the popular model to obtain output a2; determine whether the output a2 is the same as the standard output; If the output a2 is the same as the standard output, it means that the pooling layer of the original model is good; If the output a2 is different from the standard output, it means that the pooling layer of the original model is abnormal; Step S343: Substitute the output of the fully connected layer of the undetermined model into the fully connected layer of the popular model, execute the popular model to obtain output a3; determine whether the output a3 is the same as the standard output; If the output a3 is the same as the standard output, it means that the fully connected layer of the original model is good; If the output a3 is different from the standard output, it means that the fully connected layer of the original model is abnormal; Step S35: Summarize the analysis results of step S34 as an improvement report of the original model.

9. A speech emotion recognition model training system, applicable to a speech emotion recognition model training method according to any one of claims 1 to 8, characterized in that: The training system comprises: Speech acquisition module: used to obtain positive and negative speech information, and perform pre-emphasis, windowing and Fourier transform on each piece of speech information, and then perform Mel transform and discrete cosine transform on each piece of speech information to obtain the MFCC coefficient of each piece of speech information; Model training module: used to perform homomorphic and cepstrum processing on each voice information to obtain the LPCC coefficient of each voice information; obtain the original model; convert the voice information into text information, and divide the positive voice information and the negative voice information into sample set α and sample set β in the format of text information-MFCC coefficient-LPCC coefficient; Take sample set α and sample set β as independent inputs to perform supervised training on the original model; take sample set α and sample set β as mixed inputs to perform unsupervised training on the original model; until the output results of supervised training and unsupervised training of the original model are the same, the undetermined model is obtained; Inputting positive speech information and negative speech information into the model to be tested to obtain a pending output; Comparative analysis module: used to take positive and negative speech information as inputs of popular models and obtain standard outputs; Compare the performance of the proposed model and the popular model based on the proposed output and the standard output; If the performance of the candidate model is weaker than that of the popular model, the outputs of the convolutional layer, pooling layer, and fully connected layer of the candidate model are obtained in sequence, and substituted into the convolutional layer, pooling layer, and fully connected layer corresponding to the popular model to analyze the weaknesses of the candidate model and obtain an improvement report; If the performance of the pending model is stronger than that of the popular model, it will not be processed; Voice update module: used to summarize the improvement report and provide feedback; obtain new positive voice information and new negative voice information, and re-execute the training and analysis process of the original model until the performance of the original model approaches or exceeds that of the popular model.

Images