Children autism tendency detection method based on voice MFCC characteristics

Abstract

The invention discloses a children autism tendency detection method based on voice MFCC features. The method comprises the following steps: constructing a voice data set; the method comprises the following steps: extracting MFCC features in voice emotion data of normal and sick children, performing standardization processing on voice data, and finally performing feature data enhancement to obtain an MFCC feature enhanced data set, and dividing the MFCC feature enhanced data set into a training set and a test set according to a proportion; constructing a network model of a mixed local area type attention mechanism; training a network model mixed with a local area type attention mechanism; and testing the network model mixed with the local area type attention mechanism. The voice analysis is based on objective acoustic data, the influence of subjective judgment is reduced, compared with a detection method needing expensive equipment and a professional environment in the prior art, voice data collection is relatively easy, the collection speed is high, voice data can be completed through daily dialogues or specially designed voice tasks, the data processing cost is low, and the method is suitable for popularization and application. The method has the advantage of high detection accuracy.

Description

Technical Field

[0001] The invention belongs to the technical field of speech data processing, and in particular relates to a method for detecting autism tendency in children based on speech MFCC features. Background Art

[0002] Autism, also known as autism, is classified as a developmental disorder caused by a disorder of the nervous system, and its symptoms include incomplete social skills, communication skills, interests and behavior patterns. Autism is a pervasive developmental disorder, a mental illness characterized by severe and extensive impairment of social interaction and communication skills, as well as stereotyped behaviors, interests and activities. Early detection of autism tendencies can help patients get professional help early, promote the development of language, social and cognitive abilities, and help parents and educators correctly understand autism and provide appropriate support and guidance. At present, a large number of studies have adopted various methods to detect autism. At present, common auxiliary diagnostic methods include eye tracking, brain CT scans, eye tracking, behavioral analysis, and multi-scale comprehensive evaluation. Most doctors still make diagnoses based on long-term clinical observations supplemented by multiple scales. Due to the lack of effective objective evaluation methods, the diagnosis cycle is too long, and it is easy to miss the best intervention period. In addition, the above research methods require expensive specific equipment. Due to cost and equipment limitations, they are usually only applicable to specialized clinical environments and difficult to promote on a large scale. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method for detecting autism tendency in children based on speech MFCC features with low cost, high efficiency and high accuracy.

[0004] The technical solution adopted to solve the above technical problems is: a method for detecting autism tendency in children based on speech MFCC features, comprising the following steps:

[0005] Step 1. Build a speech dataset

[0006] Based on a specific VR game scenario designed to screen autistic children's tendencies, autistic children and normal children are asked to imitate the behaviors of doctors and patients in VR animation scenarios and say specific emotional sentences, thereby obtaining the voice emotion data of autistic children and the voice data of normal children;

[0007] Step 2. Extract MFCC features from the speech emotion data of normal and diseased children, standardize the speech data, and finally enhance the feature data to obtain the MFCC feature enhanced data set, which is divided into a training set and a test set according to the ratio;

[0008] Step 3. Build a network model with hybrid local attention mechanism

[0009] The network model of the hybrid local attention mechanism is composed of a bidirectional LSTM layer, a global attention mechanism layer, a GRU layer, a self-attention mechanism layer, and a classification unit connected in sequence;

[0010] The bidirectional LSTM layer includes a forward LSTM layer and a backward LSTM layer, the forward LSTM layer and the backward LSTM layer respectively process the MFCC feature enhancement data set, generate a forward hidden state and a backward hidden state of each time step, and fuse them to form a hidden state of the bidirectional LSTM layer at each time step and output it;

[0011] The global attention mechanism layer obtains the attention weight of each hidden state output by the bidirectional LSTM layer according to the following formula and performs weighted aggregation;

[0012] e t =v T tanh(W h h t +b h )

[0013]

[0014] In the formula, e t is the score of attention weight, W h and b h is a trainable parameter of the attention mechanism, h t is the hidden state of the tth time step output by the bidirectional LSTM layer, e t′ is the score of all time steps t′ in the sequence, where t′ can be any time step, including t itself, and α t is the attention weight normalized by the softmax function, and c is the context vector after weighted aggregation;

[0015] The GRU layer learns and models the characteristics of the speech signal according to the output of the global attention mechanism layer;

[0016] The self-attention mechanism layer is used to calculate the similarity between all positions in the sequence according to the following formula for the output of the GRU layer, so as to capture and maintain long-term dependencies;

[0017] Q=H GRU ·W Q ,K=H GRU ·W K ,V=H GRU ·W V

[0018]

[0019] H SA =α·V

[0020] Where W Q , W K , W V They are the weight matrices of Query, Key, and Value, respectively. GRU is the output of the GRU layer. Q, K, and V are the Query, Key, and Value matrices, respectively. Query is used to find the information that the current input element should pay attention to. Key is the feature of the input element, which is used to match the Query. Value contains the actual information and is used to generate the final attention output. is the scaling factor, α is the attention weight matrix obtained by softmax, and H SA is the final output of the self-attention mechanism;

[0021] The classification unit is composed of a fully connected layer and a Softmax layer connected in sequence, and is used to output the probability value of whether the speech belongs to autistic people or normal people;

[0022] Step 4. Train the network model of the hybrid local attention mechanism

[0023] The model is trained using the model.fit method. The specified parameters include epochs of 32 and batch_size of 32. The training is completed after reaching the training number.

[0024] Step 5. Test the network model of the hybrid local attention mechanism

[0025] The test set is input into the trained network model of the hybrid local attention mechanism, and the output is the probability value of the speech belonging to autistic or normal people.

[0026] Preferably, the extracting MFCC features in the speech emotion data comprises the following steps:

[0027] Step A1. Pre-processing the original speech emotion data by pre-emphasis, framing, and windowing;

[0028] Step A2. Perform Fourier transform on the windowed signal x(n) according to the following formula to convert the time domain signal into a frequency domain signal;

[0029]

[0030] Where X[k] is the discrete Fourier transform coefficient of the kth frequency component of the signal x(n), j is the imaginary unit, and N is the total length of the signal x(n), that is, the number of sampling points of the signal;

[0031] Step A3. Square the amplitude of each frequency domain signal to obtain the power spectrum of each frame;

[0032] Step A4. Filter the power spectrum using a Mel filter bank and take the logarithm of the result;

[0033] Step A5: Apply discrete cosine transform to the logarithmic Mel spectrum to obtain MFCC coefficients, that is, extract corresponding MFCC features.

[0034] Preferably, the number of mel filters in the mel filter bank in step A4 is 20 to 40.

[0035] Preferably, in step 2, the method of standardizing the voice data is: using Python library and Librosa software to process and analyze the voice signal.

[0036] Preferably, in step 2, the feature data enhancement operation includes:

[0037] 1) Add noise: add random values ​​to the data with a noise coefficient of 0.03;

[0038] 2) Volume gain: randomly changes the pitch of the signal with a pitch factor of 1.2;

[0039] 3) Time shift: Randomly shift the speech signal to the left or right, with a maximum shift value of 0.2 seconds.

[0040] The beneficial effects of the present invention are as follows:

[0041] The speech analysis of the present invention is based on objective acoustic data, which reduces the influence of subjective judgment. Compared with the detection methods in the prior art that require expensive equipment and professional environments, speech data collection is relatively easy and fast, and can be completed through daily conversations or specially designed speech tasks. The cost of data processing is also low.

[0042] By hybridizing the local attention mechanism, the model of the present invention can more accurately capture and represent the key features in the speech signal, thereby improving the accuracy of autism tendency detection. The application of data augmentation technology improves the model's generalization ability for autism features under different speech conditions.

[0043] The present invention uses virtual reality animation scenes to collect voice data, which can improve the participation and interest of autistic children and make the detection process more friendly. Since autism tendency detection can be performed quickly and accurately, it helps to identify autistic children earlier, thereby seizing the best intervention period.

[0044] The present invention does not require any specific equipment, is easier to implement in different environments, and is conducive to large-scale promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1It is a flow chart of a method for detecting autism tendency in children based on speech MFCC features of the present invention.

[0046] Figure 2 It is a schematic diagram of the processing flow of the network model of the hybrid local attention mechanism of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples, but the present invention is not limited to the following embodiments.

[0048] exist Figure 1 In this embodiment, a method for detecting autism tendency in children based on speech MFCC features comprises the following steps:

[0049] Step 1. Build a speech dataset

[0050] Let autistic children and normal children imitate the behaviors of doctors and patients in a specific virtual reality game scene designed based on the screening of autistic children's tendencies, and say specific emotional sentences, so as to obtain the voice emotion data of autistic children and the voice data of normal children; this method can not only enhance their participation and interest, but also help improve their ability to express emotions;

[0051] Step 2. Extract MFCC features (MFCC is Mel-Frequency Cepstral Coefficients) from the speech emotion data of normal and diseased children, and use Python library and Librosa software to standardize the speech data. Finally, perform feature data enhancement to obtain the MFCC feature enhanced dataset, and divide the MFCC feature enhanced dataset into a training set and a test set in a ratio of 8:2.

[0052] The extraction of MFCC features in speech emotion data includes the following steps:

[0053] Step A1. Pre-processing the original speech emotion data by pre-emphasis, framing, and windowing;

[0054] Step A2. Perform Fourier transform on the windowed signal x(n) according to the following formula to convert the time domain signal into a frequency domain signal;

[0055]

[0056] Where X[k] is the discrete Fourier transform coefficient of the kth frequency component of the signal x(n), j is the imaginary unit, and N is the total length of the signal x(n), that is, the number of sampling points of the signal;

[0057] Step A3. Square the amplitude of each frequency domain signal to obtain the power spectrum of each frame;

[0058] Step A4. Filter the power spectrum using a Mel filter bank, where the number of Mel filters in the Mel filter bank is 20 to 40, and take the logarithm of the result;

[0059] Step A5. Apply discrete cosine transform to the Mel spectrum after logarithmization to obtain MFCC coefficients, that is, extract corresponding MFCC features;

[0060] Among them, the operations of feature data enhancement include:

[0061] 1) Add noise: add random values ​​to the data with a noise coefficient of 0.03;

[0062] 2) Volume gain: randomly changes the pitch of the signal with a pitch factor of 1.2;

[0063] 3) Time shift: Randomly shift the speech signal to the left or right, with a maximum shift of 0.2 seconds;

[0064] Step 3. Build a network model with hybrid local attention mechanism

[0065] The network model of the hybrid local attention mechanism is composed of a bidirectional LSTM layer, a global attention mechanism layer, a GRU layer, a self-attention mechanism layer, and a classification unit connected in sequence, such as Figure 2 ;

[0066] The bidirectional LSTM layer includes a forward LSTM layer and a backward LSTM layer. The forward LSTM layer and the backward LSTM layer process the MFCC feature enhancement data set respectively, generate the forward hidden state and the backward hidden state of each time step and fuse them to form the hidden state of the bidirectional LSTM layer at each time step and output it;

[0067] The global attention mechanism layer obtains the attention weight of each hidden state output by the bidirectional LSTM layer according to the following formula and performs weighted aggregation;

[0068] e t =v T tanh(W h h t +b h )

[0069]

[0070] In the formula, e t is the score of attention weight, W h and b h is a trainable parameter of the attention mechanism, h tis the hidden state of the tth time step output by the bidirectional LSTM layer, e t′ is the score of all time steps t′ in the sequence, where t′ can be any time step, including t itself, and α t is the attention weight normalized by the softmax function, and c is the context vector after weighted aggregation;

[0071] The GRU layer learns and models the characteristics of the speech signal based on the output of the global attention mechanism layer;

[0072] The self-attention mechanism layer is used to calculate the similarity between all positions in the sequence according to the following formula for the output of the GRU layer, capturing and maintaining long-term dependencies;

[0073] Q=H GRU ·W Q ,K=H GRU ·W K ,V=H GRU ·W V

[0074]

[0075] H SA =α·V

[0076] Where W Q , W K , W V They are the weight matrices of Query, Key, and Value, respectively. GRU is the output of the GRU layer. Q, K, and V are the Query, Key, and Value matrices, respectively. Query is used to find the information that the current input element should pay attention to. Key is the feature of the input element, which is used to match the Query. Value contains the actual information and is used to generate the final attention output. is the scaling factor, α is the attention weight matrix obtained by softmax, and H SA is the final output of the self-attention mechanism;

[0077] The classification unit is composed of a fully connected layer and a Softmax layer connected in sequence, and is used to output the probability value of whether the speech belongs to autistic people or normal people;

[0078] Step 4. Train the network model of the hybrid local attention mechanism

[0079] The model was trained using the model.fit method. The specified parameters included epochs of 32 and batch_size of 32. The training was completed after reaching the training number. The accuracy of the trained model reached 89.75%, which can effectively and accurately distinguish whether the person being tested has autism.

[0080] Step 5. Test the network model of the hybrid local attention mechanism

[0081] The test set is input into the trained network model of the hybrid local attention mechanism, and the output is the probability value of the speech belonging to autistic or normal people.

Claims

1. A method for detecting autism tendency in children based on speech MFCC features, characterized in that: The following steps are involved: Step 1. Build a speech dataset Based on a specific VR game scenario designed to screen autistic children's tendencies, autistic children and normal children are asked to imitate the behaviors of doctors and patients in VR animation scenarios and say specific emotional sentences, thereby obtaining the voice emotion data of autistic children and the voice data of normal children; Step 2. Extract MFCC features from the speech emotion data of normal and diseased children, standardize the speech data, and finally enhance the feature data to obtain the MFCC feature enhanced data set, which is divided into a training set and a test set according to the ratio; Step 3. Build a network model with hybrid local attention mechanism The network model of the hybrid local attention mechanism is composed of a bidirectional LSTM layer, a global attention mechanism layer, a GRU layer, a self-attention mechanism layer, and a classification unit connected in sequence; The bidirectional LSTM layer includes a forward LSTM layer and a backward LSTM layer, the forward LSTM layer and the backward LSTM layer respectively process the MFCC feature enhancement data set, generate a forward hidden state and a backward hidden state of each time step, and fuse them to form a hidden state of the bidirectional LSTM layer at each time step and output it; The global attention mechanism layer obtains the attention weight of each hidden state output by the bidirectional LSTM layer according to the following formula and performs weighted aggregation; e t =v T tanh(W h h t +b h ) In the formula, e t is the score of attention weight, W h and b h is a trainable parameter of the attention mechanism, h t is the hidden state of the tth time step output by the bidirectional LSTM layer, e t′ is the score of all time steps t′ in the sequence, where t′ can be any time step, including t itself, and α t is the attention weight normalized by the softmax function, and c is the context vector after weighted aggregation; The GRU layer learns and models the characteristics of the speech signal according to the output of the global attention mechanism layer; The self-attention mechanism layer is used to calculate the similarity between all positions in the sequence according to the following formula for the output of the GRU layer, so as to capture and maintain long-term dependencies; Q=H GRU ·W Q ,K=H GRU ·W K ,V=H GRU ·W V H SA =α·V Where W Q , W K , W V They are the weight matrices of Query, Key, and Value, respectively. GRU is the output of the GRU layer. Q, K, and V are the Query, Key, and Value matrices, respectively. Query is used to find the information that the current input element should pay attention to. Key is the feature of the input element, which is used to match the Query. Value contains the actual information and is used to generate the final attention output. is the scaling factor, α is the attention weight matrix obtained by softmax, and H SA is the final output of the self-attention mechanism; The classification unit is composed of a fully connected layer and a Softmax layer connected in sequence, and is used to output the probability value of whether the speech belongs to autistic people or normal people; Step 4. Train the network model of the hybrid local attention mechanism The model is trained using the model.fit method. The specified parameters include epochs of 32 and batch_size of 32. The training is completed after reaching the training number. Step 5. Test the network model of the hybrid local attention mechanism The test set is input into the trained network model of the hybrid local attention mechanism, and the output is the probability value of the speech belonging to autistic or normal people.

2. The method for detecting autism tendency in children based on speech MFCC features according to claim 1, characterized in that: In step 2, extracting MFCC features from speech emotion data comprises the following steps: Step A1. Pre-processing the original speech emotion data by pre-emphasis, framing, and windowing; Step A2. Perform Fourier transform on the windowed signal x(n) according to the following formula to convert the time domain signal into a frequency domain signal; Where X[k] is the discrete Fourier transform coefficient of the kth frequency component of the signal x[n], j is the imaginary unit, and N is the total length of the signal x[n], that is, the number of sampling points of the signal; Step A3. Square the amplitude of each frequency domain signal to obtain the power spectrum of each frame; Step A4. Filter the power spectrum using a Mel filter bank and take the logarithm of the result; Step A5: Apply discrete cosine transform to the logarithmic Mel spectrum to obtain MFCC coefficients, that is, extract corresponding MFCC features.

3. The method for detecting autism tendency in children based on speech MFCC features according to claim 2, characterized in that: The number of mel filters in the mel filter bank in step A4 is 20 to 40.

4. The method for detecting autism tendency in children based on speech MFCC features according to claim 1, characterized in that: In step 2, the method of standardizing the speech data is to use the Python library and Librosa software to process and analyze the speech signal.

5. The method for detecting autism tendency in children based on speech MFCC features according to claim 1, characterized in that: In step 2, the feature data enhancement operation includes: 1) Add noise: add random values ​​to the data with a noise coefficient of 0.03; 2) Volume gain: randomly changes the pitch of the signal with a pitch factor of 1.2; 3) Time shift: Randomly shift the speech signal to the left or right, with a maximum shift value of 0.2 seconds.

Images