Post-stroke dysarthria pathological speech recognition method and device based on SNN

Abstract

The invention discloses a post-stroke dysarthria pathological speech recognition method and device based on SNN, and is used for solving the technical problem that the existing dysarthria pathological speech recognition method based on a deep learning algorithm causes excessive power consumption in a model operation process. The method comprises the following steps: acquiring a plurality of initial audio data of a patient to be detected, preprocessing each initial audio data, and outputting a Mel-cepstrum coefficient feature corresponding to each initial audio data; inputting each Mel-cepstrum coefficient feature into a pulse neural network fusing word attention and channel attention for identification, and outputting a plurality of syllable category disease probabilities; and according to the multiple syllable category disease probabilities, generating a post-stroke dysarthria pathological speech recognition result corresponding to the patient to be detected.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a SNN-based post-stroke dysarthria pathological speech recognition method and device. Background Art

[0002] In modern society, the incidence of stroke is increasing year by year, and dysarthria (PSD) is a common sequela of stroke, which can cause patients to have poor voice quality or even be unable to communicate normally, seriously affecting their quality of life. Traditionally, the diagnosis of dysarthria is mainly done through a series of processes, including patient interviews, invasive surgical examinations with instruments such as laryngoscopes and endoscopes, acoustic analysis, and subjective voice evaluation by doctors, which are time-consuming and cumbersome.

[0003] In recent years, with the development of deep learning algorithms, the performance of image, speech and other classification algorithms has become increasingly superior. Therefore, some people have proposed exploring the use of these algorithms to assist doctors in identifying pathological speech such as dysarthria. This method can achieve non-invasive, economical, and unbiased detection, and can quickly evaluate speech pathology.

[0004] Most of the existing dysarthria pathological speech recognition methods based on deep learning algorithms use deep CNN models or large Transformer models to complete the recognition of dysarthria pathological speech, but these models all belong to artificial neural networks. Whether in the training stage or the inference stage, artificial neural networks require a large amount of computing resources to support their complex neuronal connections and calculation operations, resulting in excessive power consumption in the model calculation process. Summary of the invention

[0005] The present invention provides a method and device for post-stroke dysarthria pathological speech recognition based on SNN, which are used to solve the technical problem that the existing dysarthria pathological speech recognition method based on deep learning algorithm leads to excessive power consumption in the model calculation process.

[0006] The first aspect of the present invention provides a method for recognizing pathological speech of post-stroke dysarthria based on SNN, comprising:

[0007] Acquire multiple initial audio data of the patient to be tested, pre-process each of the initial audio data, and output Mel-frequency cepstral coefficient features corresponding to each of the initial audio data;

[0008] Inputting each of the Mel-frequency cepstral coefficient features into a spiking neural network that integrates word attention and channel attention for recognition, and outputting disease probabilities for multiple syllable categories;

[0009] According to the disease probabilities of the multiple syllable categories, a pathological speech recognition result of post-stroke dysarthria corresponding to the patient to be detected is generated.

[0010] Optionally, the preprocessing of each of the initial audio data respectively and outputting Mel-frequency cepstral coefficient features corresponding to each of the initial audio data includes:

[0011] Pre-emphasize each of the initial audio data respectively, and output the pre-emphasized audio data corresponding to each of the initial audio data;

[0012] Respectively dividing each of the pre-emphasized audio data into frames to generate a plurality of audio frame data;

[0013] Performing windowing on each of the audio frame data, and outputting window audio data corresponding to each of the audio frame data;

[0014] Performing fast Fourier transform on the audio data in each window respectively, and outputting the target spectrum corresponding to the audio data in each window;

[0015] Performing filtering operations on each of the target spectrums respectively to generate Mel spectrums corresponding to each of the target spectrums;

[0016] Perform logarithmic operation and discrete cosine transform on each of the Mel spectrums, and output Mel cepstral coefficient features corresponding to each of the Mel spectrums.

[0017] Optionally, the spiking neural network integrating word attention and channel attention includes an embedding module, a spiking self-attention module, and a linear output layer; the inputting each of the Mel-frequency cepstral coefficient features into the spiking neural network integrating word attention and channel attention for recognition, and outputting disease probabilities of multiple syllable categories, includes:

[0018] Using an embedding module to perform enhanced feature extraction on each of the Mel-frequency cepstral coefficient features, and generating a target pulse nonlinear feature corresponding to each of the Mel-frequency cepstral coefficient features;

[0019] Using each of the target pulse nonlinear features as the input of the pulse self-attention module, and outputting the target attention matrix corresponding to each of the target pulse nonlinear features;

[0020] Each of the target attention matrices is used as the input of the linear output layer, and the disease probability of the syllable category corresponding to each of the target attention matrices is output.

[0021] Optionally, the embedding module includes a first convolutional layer, a first batch of normalization layers, a first leakage pulse neuron, a second convolutional layer, a second batch of normalization layers, and a second leakage pulse neuron; the embedding module is used to perform enhanced feature extraction on each of the Mel-cephalogram coefficient features to generate a target pulse nonlinear feature corresponding to each of the Mel-cephalogram coefficient features, including:

[0022] Using a first convolutional layer to perform convolution operations on each of the Mel-frequency cepstral coefficient features, and outputting first convolution features corresponding to each of the Mel-frequency cepstral coefficient features;

[0023] Normalizing each of the first convolutional features respectively through a first batch of normalization layers to generate a first normalized feature corresponding to each of the first convolutional features;

[0024] Using each of the first normalized features as an input of a second leakage pulse neuron, and outputting an initial pulse nonlinear feature corresponding to each of the first normalized features;

[0025] Using a second convolutional layer to perform convolution operations on each of the initial pulse nonlinear features, and outputting second convolutional features corresponding to each of the initial pulse nonlinear features;

[0026] Normalizing each of the second convolutional features respectively through a second batch of normalization layers to generate second normalized features corresponding to each of the second convolutional features;

[0027] Each of the second normalized features is used as an input of a second leakage pulse neuron, and a target pulse nonlinear feature corresponding to each of the second normalized features is output.

[0028] Optionally, the pulse self-attention module includes a word attention layer, a channel attention layer and a multi-layer perceptron layer; the step of taking each of the target pulse nonlinear features as an input of the pulse self-attention module and outputting a target attention matrix corresponding to each of the target pulse nonlinear features includes:

[0029] Using each of the target pulse nonlinear features as the input of the word attention layer, and outputting the word attention matrix corresponding to each of the pulse nonlinear features;

[0030] Using each of the target pulse nonlinear features as the input of the channel attention layer, and outputting the channel attention matrix corresponding to each of the pulse nonlinear features;

[0031] Respectively summing and superimposing each of the target pulse nonlinear features and the word attention matrix and the channel attention matrix corresponding to each of the target pulse nonlinear features, and outputting the attention residual matrix corresponding to each of the target pulse nonlinear features;

[0032] Using each of the attention residual matrices as the input of a multi-layer perceptron layer, and outputting the perceptual pulse nonlinear features corresponding to each of the attention residual matrices;

[0033] The attention residual matrices and the perceptual pulse nonlinear features corresponding to the attention residual matrices are summed and superimposed respectively, and the target attention matrices corresponding to the target pulse nonlinear features are output.

[0034] Optionally, generating the post-stroke dysarthria pathological speech recognition result corresponding to the patient to be detected according to the disease probabilities of the plurality of syllable categories includes:

[0035] Performing a mean operation on the disease probabilities of the plurality of syllable categories, and outputting a mean disease probability;

[0036] Based on the comparison result of the disease probability mean and the preset probability threshold, the post-stroke dysarthria pathological speech recognition result corresponding to the patient to be detected is generated.

[0037] Optionally, the model training process of the spiking neural network integrating word attention and channel attention includes:

[0038] Acquire an audio data training set, preprocess the audio data training set, and output a plurality of Mel-frequency cepstral coefficient features to be trained;

[0039] Inputting each of the Mel-frequency cepstral coefficient features to be trained into an initial spiking neural network that integrates word attention and channel attention for recognition, and outputting disease probabilities of multiple syllable categories to be trained;

[0040] Calculating the mean of the disease probabilities of the plurality of syllable categories to be trained to determine the mean of the disease probabilities to be trained;

[0041] Calculate the target loss value according to the mean disease probability to be trained;

[0042] Based on the target loss value, the model parameters of the initial spiking neural network that integrates the word attention and the channel attention are updated, an intermediate spiking neural network that integrates the word attention and the channel attention is determined, and the number of iterations is counted in real time;

[0043] Determine whether the number of iterations reaches a preset iteration threshold;

[0044] If so, the intermediate pulse neural network that integrates word attention and channel attention is used as the trained pulse neural network that integrates word attention and channel attention.

[0045] The second aspect of the present invention provides a post-stroke dysarthria pathological speech recognition device based on SNN, comprising:

[0046] An acquisition module, used for acquiring a plurality of initial audio data of a patient to be detected, preprocessing each of the initial audio data, and outputting a Mel-frequency cepstral coefficient feature corresponding to each of the initial audio data;

[0047] A recognition module, used for inputting each of the Mel-frequency cepstral coefficient features into a pulse neural network integrating word attention and channel attention for recognition, and outputting disease probabilities of multiple syllable categories;

[0048] The prediction module is used to generate a pathological speech recognition result of post-stroke dysarthria corresponding to the patient to be detected according to the disease probabilities of the multiple syllable categories.

[0049] A third aspect of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the SNN-based post-stroke dysarthria pathological speech recognition method as described in any one of the above items.

[0050] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the steps of the SNN-based post-stroke dysarthria pathological speech recognition method as described in any one of the above.

[0051] It can be seen from the above technical solutions that the present invention has the following advantages:

[0052] The above technical scheme of the present invention provides a post-stroke dysarthria pathological speech recognition method based on SNN, first, a plurality of initial audio data of a patient to be detected are obtained, and each initial audio data is preprocessed respectively, and the Mel-cepstral coefficient feature corresponding to each initial audio data is output; then, each Mel-cepstral coefficient feature is input into a pulse neural network that integrates word attention and channel attention for recognition, and the disease probabilities of multiple syllable categories are output; finally, according to the disease probabilities of multiple syllable categories, the post-stroke dysarthria pathological speech recognition result corresponding to the patient to be detected is generated; based on the above scheme, after preprocessing the obtained multiple initial audio data, the pulse neural network that integrates word attention and channel attention is used to recognize the obtained multiple Mel-cepstral coefficient features, and the post-stroke dysarthria pathological speech recognition result is output. The present invention combines word attention and channel attention on the basis of the pulse neural network, which can reduce the computational complexity and parameters of the model, thereby reducing the power consumption of the model calculation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0054] Figure 1 A flowchart of a method for recognizing post-stroke dysarthria pathological speech based on SNN provided in Example 1 of the present invention;

[0055] Figure 2 A data schematic diagram of syllables corresponding to the audio data provided in the first embodiment of the present invention;

[0056] Figure 3 A framework diagram of the SNN-based TCAFormer network provided in Example 1 of the present invention;

[0057] Figure 4 A flowchart of the steps of model training of a spiking neural network integrating word attention and channel attention provided in the second embodiment of the present invention;

[0058] Figure 5 A framework diagram of the model training of a spiking neural network integrating word attention and channel attention provided in the second embodiment of the present invention;

[0059] Figure 6 A structural block diagram of a post-stroke dysarthria pathological speech recognition device based on SNN is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The embodiment of the present invention provides a method and device for post-stroke dysarthria pathological speech recognition based on SNN, which is used to solve the technical problem that the existing dysarthria pathological speech recognition method based on deep learning algorithm causes excessive power consumption in the model calculation process.

[0061] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] Terminology explanation:

[0063] PSD: post-stroke dysarthria.

[0064] SNN:Spiking Neural Network.

[0065] LIF: Leaky Integrate - and - Fire.

[0066] MFCC: Mel-Frequency Cepstral Coefficients.

[0067] See also Figure 1 , Figure 1 A flowchart of the steps of a method for post-stroke dysarthria pathological speech recognition based on SNN is provided in Example 1 of the present invention.

[0068] The present invention provides a SNN-based post-stroke dysarthria pathological speech recognition method, comprising:

[0069] Step 101: Acquire multiple initial audio data of a patient to be tested, pre-process each initial audio data, and output Mel-frequency cepstral coefficient features corresponding to each initial audio data.

[0070] Please note that Figure 2 , the patient to be detected has multiple syllables, and each syllable corresponds to an initial audio data. The present invention obtains multiple initial audio data of the patient to be detected from the Chinese Syllable PSD pathological speech database collected by the First Affiliated Hospital of Jinan University, that is, the pathological speech data (audio data) is collected by the Speech Therapy Center of the Affiliated Hospital of Jinan University. All participants are adults who speak Mandarin, including 17 ordinary males, 23 ordinary females, 38 male patients, and 22 female patients, as shown in Table 1. The age range of the people participating in the collection is between 25 and 83 years old [mean ± standard deviation: 56.74 ± 16.40 years old]. They all received physical examinations, Frenche dysarthria assessments and other auxiliary examinations (such as brain CT, MRI).

[0071] Before developing dysarthria, all patients were able to communicate fluently in Mandarin, without visual or severe auditory comprehension impairment. All audio data were collected at a sampling rate of 16K and saved in .wav format. The collected pronunciation syllables covered all Mandarin vowel syllables, consonant syllables, and intonation changes. Figure 2As shown, some testers have missing syllables. Table 1 shows the proportion of participants in the data set. Taking into account the gender of the patients, the data set is divided into a training set and a test set at a ratio of 4:1. The test set contains all the syllables of 20 people for subsequent experiments. The training set (audio data training set) is used to train the pulse neural network that integrates word attention and channel attention proposed in the present invention, and then the audio data in the test set (multiple initial audio data of the patient to be tested) is input into the trained pulse neural network that integrates word attention and channel attention to obtain the probability that all syllable categories of the patient to be tested belong to pathological speech. The probability that the speaker is ill (mean probability of illness) is finally obtained by calculating the average of all syllable probabilities.

[0072] Table 1: Information of participants

[0073]

[0074] Furthermore, the process of preprocessing each of the initial audio data and outputting the Mel-frequency cepstral coefficient features corresponding to each of the initial audio data can be achieved by executing the following sub-steps S11 to S16:

[0075] Step S11, pre-emphasize each initial audio data, and output the pre-emphasized audio data corresponding to each initial audio data;

[0076] Step S12, dividing each pre-emphasized audio data into frames to generate a plurality of audio frame data;

[0077] Step S13, windowing each audio frame data, and outputting the window audio data corresponding to each audio frame data;

[0078] Step S14, performing fast Fourier transform on the audio data in each window respectively, and outputting the target spectrum corresponding to the audio data in each window;

[0079] Step S15, performing filtering operations on each target spectrum respectively to generate a Mel spectrum corresponding to each target spectrum;

[0080] Step S16: perform logarithmic operation and discrete cosine transform on each Mel spectrum, and output Mel cepstral coefficient features corresponding to each Mel spectrum.

[0081] It should be noted that the present invention obtains multiple window audio data by pre-emphasis, framing, and windowing the speech signal (initial audio data) in sequence, and then extracts 24-dimensional MFCCs features (Mel-frequency cepstral coefficient features) from the obtained window audio data. The present invention sets the window length to 400 and the frame shift to 160. In order to keep the time length of each audio consistent, the present invention sets the number of time windows to 120, fills 0 for time windows less than 120, and truncates for time windows exceeding 120.

[0082] Furthermore, the present invention performs a fast Fourier transform on each time window (window audio data) to extract a spectrum (target spectrum), further extracts the spectrum using a Mel filter array to obtain a Mel spectrum, and then performs logarithmic operations and discrete cosine transforms on the Mel spectrum to ultimately obtain Mel cepstral coefficient features (MFCCs). The dimension of the Mel cepstral coefficient is the number of Mel filters. The Mel frequency is a nonlinear frequency scale determined based on the human ear's sensory judgment of equally spaced pitch changes, and has a good effect on pathological speech classification.

[0083] In this embodiment, a plurality of initial audio data of the patient to be detected are obtained, and each of the initial audio data is preprocessed respectively, and the Mel-frequency cepstral coefficient features corresponding to each of the initial audio data are output.

[0084] Step 102: Input each Mel-frequency cepstral coefficient feature into a pulse neural network that integrates word attention and channel attention for recognition, and output disease probabilities of multiple syllable categories.

[0085] The spiking neural network integrating word attention and channel attention includes an embedding module, a spiking self-attention module and a linear output layer.

[0086] Please note that Figure 3, the pulse neural network that integrates word attention and channel attention is a TCAFormer (Transformer-based Cross-Attention Former) network based on SNN, which can complete the pathological speech recognition task. Since SNN is an event-driven neural network, the floating-point data is discretely binarized, which will lead to the loss of effective features. Therefore, for the audio data set used in the present invention, it is necessary to repeat the feature map for T time steps to ensure the acquisition of model information. The present invention sets T to 4, that is, the input feature map is repeated 4 times; wherein, the pulse neural network that integrates word attention and channel attention is composed of an embedding module, a pulse self-attention module and a linear output layer; the embedding module is composed of two convolutional submodules (convolutional submodule 1 and convolutional submodule 2), and the convolutional submodule is composed of a convolutional layer, a batch normalization layer (BN layer, Batch Normalization layer), and a LIF neuron; the pulse self-attention module is composed of a word attention layer, a channel attention layer and a multi-layer perceptron layer (MLP layer, Multilayer Perceptron).

[0087] Specifically, step 102 may include the following sub-steps S21-S23:

[0088] Step S21, using an embedding module to perform enhanced feature extraction on each Mel-frequency cepstral coefficient feature, and generating a target pulse nonlinear feature corresponding to each Mel-frequency cepstral coefficient feature;

[0089] The embedding module includes a first convolutional layer, a first batch of normalization layers, a first leakage spike neuron, a second convolutional layer, a second batch of normalization layers, and a second leakage spike neuron.

[0090] Further, step S31 may include the following sub-steps S211-S216:

[0091] Step S211: Use the first convolution layer to perform convolution operations on each Mel-frequency cepstral coefficient feature, and output the first convolution features corresponding to each Mel-frequency cepstral coefficient feature;

[0092] Step S212: normalize each first convolutional feature through the first batch of normalization layers to generate a first normalized feature corresponding to each first convolutional feature;

[0093] Step S213: using each first normalized feature as an input of a second leakage pulse neuron, and outputting an initial pulse nonlinear feature corresponding to each first normalized feature;

[0094] Step S214: using the second convolution layer to perform convolution operations on each initial pulse nonlinear feature, and outputting a second convolution feature corresponding to each initial pulse nonlinear feature;

[0095] Step S215: normalize each second convolutional feature respectively through a second batch of normalization layers to generate a second normalized feature corresponding to each second convolutional feature;

[0096] Step S216: Use each second normalized feature as an input of a second leakage pulse neuron, and output a target pulse nonlinear feature corresponding to each second normalized feature.

[0097] It should be noted that for the embedding module, since the data of the present invention is speech data with time sequence, in the convolution submodule 1, the present invention modifies the size of the input convolution kernel to (24, 3), and the width is the number of dimensions of the MFCC. While keeping the number of time windows unchanged, the features of each time window are extracted by dimension upgrade, and then the features are discretized and encoded through the LIF neurons. In the convolution submodule 2, the convolution kernel is modified to one dimension to adapt to the shape of the data and further extract local features. The data processing process of the two convolution submodules of the embedding module can be expressed as:

[0098] ;

[0099] Among them, P is the output of the convolution submodule; X is the input of the convolution submodule; LIF stands for LIF neuron, BN stands for batch normalization, and Conv stands for convolution layer.

[0100] Step S22, using each target pulse nonlinear feature as the input of the pulse self-attention module, and outputting the target attention matrix corresponding to each target pulse nonlinear feature;

[0101] The pulse self-attention module includes a word attention layer, a channel attention layer, and a multi-layer perceptron layer.

[0102] It should be noted that for the pulse self-attention module, the present invention designs a word attention layer and a channel attention layer to better grasp the global features, and inputs them into the MLP layer through the residual connection to fully extract the global and local features.

[0103] It is worth mentioning that compared with the self-attention layer of the existing Transformer model, whose computational complexity is proportional to the square of the input length, the computational complexity of the pulse self-attention module proposed in the present invention is proportional to the length of the input and the embedding dimension, which greatly reduces the number of parameters, helps to speed up the calculation speed and reduce the hardware load.

[0104] Specifically, step S22 may include the following sub-steps S221-S22:

[0105] Step S221, taking each target pulse nonlinear feature as the input of the word attention layer, and outputting the word attention matrix corresponding to each pulse nonlinear feature;

[0106] Specifically, in the word attention layer, first, the linear matrix learned by the network generates two matrices Q and K. That is, in the word attention layer, according to each target pulse nonlinear feature, the query matrix Q and key matrix K corresponding to each target pulse nonlinear feature are generated. The sizes of the two matrices Q and K are consistent with the input P. Among them, the processing process of the query matrix and the key matrix can be expressed as:

[0107] ;

[0108] Among them, Q is the query matrix; P is the output of the convolution submodule; K is the key matrix; is the weight corresponding to the nonlinear feature of the target pulse in the query matrix; is the weight corresponding to the nonlinear feature of the target pulse in the key matrix; is the LIF neuron corresponding to the query matrix; is the LIF neuron corresponding to the key matrix.

[0109] Furthermore, in the word attention layer, the present invention adds the matrix Q row by row to form a row of features, and after LIF neuron processing, multiplies it with the matrix K to obtain the word attention matrix TA. The processing process can be expressed as:

[0110] ;

[0111] Among them, TA is the word attention matrix; D is the number of embedding dimensions; N is the number of words; is the value of the i-th row and j-th column in the query matrix; K is the key matrix; is matrix multiplication.

[0112] Step S222: taking each target pulse nonlinear feature as the input of the channel attention layer, and outputting the channel attention matrix corresponding to each pulse nonlinear feature;

[0113] It should be noted that since word attention and channel attention are similar, both generate two matrices Q and K through linear matrices learned by the network, that is, in the channel attention layer, according to each target pulse nonlinear feature, the query matrix Q and key matrix K corresponding to each target pulse nonlinear feature are generated, wherein the principle of the steps of generating the query matrix Q and key matrix K corresponding to each target pulse nonlinear feature in the channel attention layer is consistent with the principle of the steps of generating the query matrix Q and key matrix K corresponding to each target pulse nonlinear feature in the word attention layer, and the present invention will not go into details.

[0114] It is worth mentioning that the query matrix Q and key matrix K generated in the channel attention layer are different from those generated in the word attention layer.

[0115] Furthermore, in the channel attention, the present invention adds the matrix Q column by column to form a column of features, and multiplies it with the matrix K after LIF neuron processing to obtain the channel attention matrix CA; wherein, the processing process of the word attention matrix and the channel attention matrix can be expressed as:

[0116] ;

[0117] Among them, TA is the word attention matrix; D is the number of embedding dimensions; N is the number of words; is the value of the i-th row and j-th column in the query matrix; K is the key matrix; is matrix multiplication.

[0118] Step S223, respectively summing and superimposing each target pulse nonlinear feature and the word attention matrix and channel attention matrix corresponding to each target pulse nonlinear feature, and outputting the attention residual matrix corresponding to each target pulse nonlinear feature;

[0119] Step S224, using each attention residual matrix as the input of the multi-layer perceptron layer, and outputting the perceptual pulse nonlinear features corresponding to each attention residual matrix;

[0120] Step S225, sum and superimpose each attention residual matrix and the perceptual pulse nonlinear features corresponding to each attention residual matrix, and output the target attention matrix corresponding to each target pulse nonlinear feature.

[0121] In this embodiment, each Mel-frequency cepstral coefficient feature is input into a pulse neural network that integrates word attention and channel attention for recognition, and the disease probabilities of multiple syllable categories are output.

[0122] Step 103: Generate a post-stroke dysarthria pathological speech recognition result corresponding to the patient to be tested according to the disease probabilities of multiple syllable categories.

[0123] Specifically, step 103 may include the following sub-steps S31-S32:

[0124] S31, performing mean calculation on the disease probabilities of multiple syllable categories, and outputting the mean disease probability;

[0125] S32. Based on the comparison result between the mean disease probability and the preset probability threshold, generate the post-stroke dysarthria pathological speech recognition result corresponding to the patient to be detected.

[0126] It should be noted that the present invention averages the time step T, and operates the attention matrix through the linear output layer to obtain the output of the model, and uses it as the probability that the syllable is uttered by the patient. By performing a mean operation on the disease probabilities of all syllable categories, the mean disease probability is output, and the mean disease probability is compared with the preset probability threshold. If the mean disease probability is greater than or equal to the preset probability threshold, the pathological speech recognition result of post-stroke dysarthria is that the patient to be detected is ill. If the mean disease probability is less than the preset probability threshold, the pathological speech recognition result of post-stroke dysarthria is that the patient to be detected is not ill. Among them, the preset probability threshold is set as needed, and the present invention is not limited to this.

[0127] In this embodiment, a post-stroke dysarthria pathological speech recognition result corresponding to the patient to be detected is generated according to the disease probabilities of multiple syllable categories.

[0128] As a comparison of technical effects, we can refer to the existing technology. Speech signal is a one-dimensional time series signal, which contains a lot of information (some of which have a great impact on the results, and some are redundant interference signals). The efficiency of direct classification is not high. Generally, the speech signal is preprocessed to extract information before classification. For preprocessing and extracting features, most of them start from the frequency domain. Mel frequency cepstral coefficients (MFCC) and spectrograms are more commonly used. In addition, some researchers use nonlinear features, multimedia audio description (MPEG-7) features, and MFCC features combined with fundamental frequency, spectrum centroid, rhythm features, etc. With the development of technology, some end-to-end large model algorithms for speech have emerged, but they require a huge data set, and the parameters of their models are also extremely large. Therefore, especially in the field of pathological speech, due to the small number of experimental subjects and the privacy issues involving experimental subjects, the general data sets are relatively small. Most of the current research is to preprocess the speech signal first, extract features useful for the speech classification system, and then process the extracted features by using deep learning algorithms.

[0129] With the development of computer technology, the acceleration of computing speed and the increase of dedicated high-speed computing memory have prompted neural network models to continuously develop in the direction of deepening the number of layers and complex structures, from the initial RNN and CNN models to deep CNN models such as Resnet, and then to the Transformer large model proposed in 2017. Its application range has expanded from natural language processing to various fields such as images, and these well-known algorithms belong to artificial neural networks. However, the calculation of model parameters in artificial neural networks is basically floating-point calculation, which will cause the power consumption of network training and reasoning to be quite large. The brain-like stimulated pulse neural network SNN, which is considered to be the third generation of neural networks, is a potential competitor of ANN. Its event-driven characteristics make it have high biological rationality and quite high energy efficiency. The SNN network rebuilt based on the characteristics of SNN and combined with the advanced ANN network structure provides great potential for the development of high-performance and low-energy algorithms.

[0130] The core of SNN is that the pulse neuron receives the input signal and accumulates the membrane potential at each time step. When the membrane potential reaches the emission threshold, the neuron sends a pulse signal. After sending the pulse signal, its membrane potential returns to the resting potential. If the membrane potential does not reach the threshold, the neuron will not send a pulse signal but continue to accumulate, and the membrane potential will leak over time. Therefore, the output of the pulse neuron can be regarded as a binary output. Compared with the floating-point multiplication operation of the ANN network, the multiplication operation of the pulse neuron is equivalent to a logical AND operation, which is much more efficient.

[0131] At present, the main research subjects of international research on articulation disorders are people who speak English, French, German, Swedish and other languages. In comparison, there are relatively few studies on articulation disorders in Mandarin Chinese.

[0132] Based on the above foundation, most existing technologies still use shallow CNN models, without further studying the role of deep models and different network topologies, and cannot handle the inherent dependencies of features at different time steps well, resulting in poor practical application effects. At the same time, most existing technologies still belong to ANN, which consumes a lot of power and is not suitable for deployment on edge devices for reasoning.

[0133] In view of the above problems, the present invention proposes a pathological speech recognition method for post-stroke dysarthria based on SNN. Specifically, firstly, by converting the two-dimensional feature map into a one-dimensional feature map in the embedding layer, it helps to reduce the number of model parameters, and makes the shapes of most subsequent convolution kernels consistent, which is helpful for hardware level implementation. Secondly, the combination of word attention and channel attention in the network replaces the self-attention in the Transformer model to reduce the computational complexity and parameters of the model. Moreover, a variety of attention mechanisms are used to capture global and local features and fuse them, effectively improving the accuracy of the model. The present invention builds a TCAFormer SNN model (a pulse neural network that integrates word attention and channel attention). Thanks to the low power consumption characteristics of the SNN model framework, the network helps to deploy applications to edge devices to expand application scenarios. In addition, the pulse neural network that integrates word attention and channel attention proposed in the present invention is a deep network that uses multiple self-attentions for fusion to fully extract the features of different time periods on the speech signal, thereby better improving the performance of pathological speech classification tasks.

[0134] Compared with traditional methods, the present invention not only has higher accuracy but also greatly reduces power consumption. This is mainly due to the fusion of multiple attention mechanisms of the TCAFormer network, which can comprehensively capture the characteristics of speech. In addition, thanks to the SNN network framework, the power consumption of the network is greatly reduced compared to the traditional ANN network, and it has a wider range of application scenarios. Moreover, experiments have found that this method also has very good effects on the classification of spoken commands with certain background noise in the SpeechCommands dataset. This method can be applied to more tasks related to speech classification, and can be deployed on edge devices for application in a wider range of scenarios.

[0135] In summary, the pulse neural network that integrates word attention and channel attention proposed in the present invention can be extended to different speech classification tasks. In addition, the low power consumption characteristics of the pulse neural network help to deploy the model to embedded devices and apply it to a wider range of fields; at the same time, the present invention starts from the 24-dimensional MFCCs time-frequency characteristics of the speech signal, and captures the dependency of the features of the speech signal in different time windows by integrating the pulse neural network of word attention and channel attention. The core of the pulse neural network that integrates word attention and channel attention is to introduce word attention and channel attention to extract features together. Experiments have shown that the model can fully extract the features of the audio and achieve a high accuracy rate in classification tasks.

[0136] In an embodiment of the present invention, the present invention provides a post-stroke dysarthria pathological speech recognition method based on SNN, first, multiple initial audio data of a patient to be detected are obtained, and each initial audio data is preprocessed respectively, and the Mel-cephalogram cepstral coefficient feature corresponding to each initial audio data is output; then, each Mel-cephalogram cepstral coefficient feature is input into a pulse neural network that integrates word attention and channel attention for recognition, and multiple syllable category disease probabilities are output; finally, according to the multiple syllable category disease probabilities, the post-stroke dysarthria pathological speech recognition result corresponding to the patient to be detected is generated; based on the above scheme, after preprocessing the multiple initial audio data obtained, the multiple Mel-cephalogram cepstral coefficient features obtained are recognized by combining the pulse neural network that integrates word attention and channel attention, and the post-stroke dysarthria pathological speech recognition result is output. The present invention combines word attention and channel attention on the basis of the pulse neural network, which can reduce the computational complexity and parameters of the model, thereby reducing the power consumption of the model calculation process.

[0137] For better explanation, refer to Figure 4 , shows a flowchart of the steps of model training of a pulse neural network integrating word attention and channel attention provided by Embodiment 2 of the present invention, and the process may include the following steps:

[0138] Step 401: Obtain an audio data training set, pre-process the audio data training set, and output a plurality of Mel-frequency cepstral coefficient features to be trained;

[0139] Step 402: Input each Mel-frequency cepstral coefficient feature to be trained into an initial spiking neural network that integrates word attention and channel attention for recognition, and output disease probabilities of multiple syllable categories to be trained;

[0140] Step 403, calculating the mean of the disease probabilities of multiple syllable categories to be trained, and determining the mean of the disease probabilities to be trained;

[0141] Step 404, calculating the target loss value according to the mean value of the disease probability to be trained;

[0142] Step 405: based on the target loss value, the model parameters of the initial spiking neural network integrating word attention and channel attention are updated, an intermediate spiking neural network integrating word attention and channel attention is determined, and the number of iterations is counted in real time;

[0143] Step 406: determine whether the number of iterations reaches a preset iteration threshold;

[0144] Step 407: If yes, the intermediate pulse neural network integrating word attention and channel attention is used as the trained pulse neural network integrating word attention and channel attention.

[0145] It should be noted that the present invention extracts 24-dimensional MFCCs features from the collected audio data, that is, the Chinese syllable PSD pathological speech database collected by the First Affiliated Hospital of Jinan University is divided to obtain an audio data training set, and multiple MFCCs features for model training are extracted based on the audio data training set, and the number of time windows is unified. Then the SNN-based TCAFormer model (initial pulse neural network based on word attention and channel attention) is classified and trained.

[0146] Furthermore, if the number of iterations does not reach a preset iteration threshold, the intermediate pulse neural network that fuses the word attention and the channel attention is used as the initial pulse neural network that fuses the word attention and the channel attention, and the process jumps to step 402 until the number of iterations reaches the preset iteration threshold, and the intermediate pulse neural network that fuses the word attention and the channel attention determined when the number of iterations reaches the preset iteration threshold is used as the trained pulse neural network that fuses the word attention and the channel attention; wherein the preset iteration threshold can be set as needed, and the present invention is not limited to this.

[0147] In an embodiment of the present invention, the present invention extracts 24-dimensional MFCCs features from the collected audio data and unifies the number of time windows. Then, the MFCC features extracted from the syllable audio data are classified and trained by the SNN-based TCAFormer model to obtain the accuracy of the syllable level. Finally, the audio data in the test set is input into the trained model to obtain the probability that all syllable categories of a person belong to pathological speech, and the probability that the speaker is ill is finally obtained by calculating the average of all syllable probabilities.

[0148] See also Figure 6 , Figure 6 A structural block diagram of a post-stroke dysarthria pathological speech recognition device based on SNN is provided for an embodiment of the present invention.

[0149] The present invention provides a post-stroke dysarthria pathological speech recognition device based on SNN, comprising:

[0150] The acquisition module 601 is used to acquire a plurality of initial audio data of the patient to be detected, and pre-process each initial audio data, and output the Mel-frequency cepstral coefficient feature corresponding to each initial audio data;

[0151] Recognition module 602, used for inputting each Mel-frequency cepstral coefficient feature into a spiking neural network integrating word attention and channel attention for recognition, and outputting disease probabilities of multiple syllable categories;

[0152] The prediction module 603 is used to generate a pathological speech recognition result of post-stroke dysarthria corresponding to the patient to be detected according to the disease probabilities of multiple syllable categories.

[0153] Furthermore, the acquisition module 601 is specifically used for:

[0154] Pre-emphasize each initial audio data respectively, and output the pre-emphasized audio data corresponding to each initial audio data;

[0155] Respectively dividing each pre-emphasized audio data into frames to generate a plurality of audio frame data;

[0156] Windowing each audio frame data respectively, and outputting the window audio data corresponding to each audio frame data;

[0157] Perform fast Fourier transform on the audio data of each window respectively, and output the target spectrum corresponding to the audio data of each window;

[0158] Perform filtering operations on each target spectrum respectively to generate the Mel spectrum corresponding to each target spectrum;

[0159] Perform logarithmic operation and discrete cosine transform on each Mel spectrum respectively, and output the Mel cepstral coefficient features corresponding to each Mel spectrum.

[0160] Furthermore, the spiking neural network integrating word attention and channel attention includes an embedding module, a spiking self-attention module, and a linear output layer; the recognition module 602 includes:

[0161] The first submodule is used to use an embedding module to perform enhanced feature extraction on each Mel-frequency cepstral coefficient feature, and generate a target pulse nonlinear feature corresponding to each Mel-frequency cepstral coefficient feature;

[0162] The second submodule is used to use each target pulse nonlinear feature as the input of the pulse self-attention module, and output the target attention matrix corresponding to each target pulse nonlinear feature;

[0163] The third submodule is used to use each target attention matrix as the input of the linear output layer, and output the disease probability of the syllable category corresponding to each target attention matrix.

[0164] Furthermore, the embedding module includes a first convolutional layer, a first batch of normalization layers, a first leakage pulse neuron, a second convolutional layer, a second batch of normalization layers, and a second leakage pulse neuron; a first submodule is specifically used for:

[0165] The first convolution layer is used to perform convolution operations on each Mel-frequency cepstral coefficient feature, and the first convolution features corresponding to each Mel-frequency cepstral coefficient feature are output;

[0166] Normalizing each first convolution feature through the first batch of normalization layers to generate a first normalized feature corresponding to each first convolution feature;

[0167] Using each first normalized feature as an input of a second leakage pulse neuron, and outputting an initial pulse nonlinear feature corresponding to each first normalized feature;

[0168] The second convolution layer is used to perform convolution operations on each initial pulse nonlinear feature, and the second convolution features corresponding to each initial pulse nonlinear feature are output;

[0169] Normalizing each second convolutional feature through a second batch of normalization layers to generate a second normalized feature corresponding to each second convolutional feature;

[0170] Each second normalized feature is used as the input of the second leakage pulse neuron, and the target pulse nonlinear feature corresponding to each second normalized feature is output.

[0171] Furthermore, the pulse self-attention module includes a word attention layer, a channel attention layer and a multi-layer perceptron layer; the second submodule is specifically used for:

[0172] Each target pulse nonlinear feature is used as the input of the word attention layer, and the word attention matrix corresponding to each pulse nonlinear feature is output;

[0173] The nonlinear features of each target pulse are used as the input of the channel attention layer, and the channel attention matrix corresponding to each pulse nonlinear feature is output;

[0174] The word attention matrix and channel attention matrix corresponding to each target pulse nonlinear feature and each target pulse nonlinear feature are summed and superimposed respectively, and the attention residual matrix corresponding to each target pulse nonlinear feature is output;

[0175] Each attention residual matrix is ​​used as the input of the multi-layer perceptron layer, and the perceptual pulse nonlinear features corresponding to each attention residual matrix are output;

[0176] Each attention residual matrix and the perceptual pulse nonlinear features corresponding to each attention residual matrix are summed and superimposed respectively, and the target attention matrix corresponding to each target pulse nonlinear feature is output.

[0177] Furthermore, the prediction module 603 is specifically used for:

[0178] Perform mean calculation on the disease probabilities of multiple syllable categories and output the mean disease probability;

[0179] Based on the comparison result between the mean probability of illness and the preset probability threshold, the pathological speech recognition result of post-stroke dysarthria corresponding to the patient to be detected is generated.

[0180] In an optional device embodiment, it also includes:

[0181] The first module is used to obtain an audio data training set, pre-process the audio data training set, and output a plurality of Mel-frequency cepstral coefficient features to be trained;

[0182] The second module is used to input the Mel-frequency cepstral coefficient features to be trained into the initial pulse neural network that integrates word attention and channel attention for recognition, and output the disease probabilities of multiple syllable categories to be trained;

[0183] The third module is used to calculate the mean of the disease probabilities of multiple syllable categories to be trained, and determine the mean of the disease probabilities to be trained;

[0184] The fourth module is used to calculate the target loss value according to the mean value of the disease probability to be trained;

[0185] The fifth module is used to update the model parameters of the initial spiking neural network that integrates word attention and channel attention based on the target loss value, determine the intermediate spiking neural network that integrates word attention and channel attention, and count the number of iterations in real time;

[0186] The sixth module is used to determine whether the number of iterations reaches a preset iteration threshold;

[0187] The seventh module is used for, if yes, using the intermediate pulse neural network that integrates word attention and channel attention as the trained pulse neural network that integrates word attention and channel attention.

[0188] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and sub-modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0189] An embodiment of the present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory; when the computer program is executed by the processor, the processor executes the steps of the SNN-based post-stroke dysarthria pathological speech recognition method as described in the first embodiment above.

[0190] An embodiment of the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the SNN-based post-stroke dysarthria pathological speech recognition method as in any of the above embodiments are implemented.

[0191] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0192] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A SNN-based post-stroke dysarthria pathological speech recognition method, characterized in that: include: Acquire multiple initial audio data of the patient to be tested, pre-process each of the initial audio data, and output Mel-frequency cepstral coefficient features corresponding to each of the initial audio data; Inputting each of the Mel-frequency cepstral coefficient features into a spiking neural network that integrates word attention and channel attention for recognition, and outputting disease probabilities for multiple syllable categories; According to the disease probabilities of the multiple syllable categories, a pathological speech recognition result of post-stroke dysarthria corresponding to the patient to be detected is generated.

2. The SNN-based post-stroke dysarthria pathological speech recognition method according to claim 1, characterized in that: The preprocessing of each of the initial audio data respectively and outputting the Mel-frequency cepstral coefficient features corresponding to each of the initial audio data comprises: Pre-emphasize each of the initial audio data respectively, and output the pre-emphasized audio data corresponding to each of the initial audio data; Respectively dividing each of the pre-emphasized audio data into frames to generate a plurality of audio frame data; Performing windowing on each of the audio frame data, and outputting window audio data corresponding to each of the audio frame data; Performing fast Fourier transform on the audio data in each window respectively, and outputting the target spectrum corresponding to the audio data in each window; Performing filtering operations on each of the target spectrums respectively to generate Mel spectrums corresponding to each of the target spectrums; Perform logarithmic operation and discrete cosine transform on each of the Mel spectrums, and output Mel cepstral coefficient features corresponding to each of the Mel spectrums.

3. The SNN-based post-stroke dysarthria pathological speech recognition method according to claim 1, characterized in that: The spiking neural network integrating word attention and channel attention includes an embedding module, a spiking self-attention module, and a linear output layer; the Mel-frequency cepstral coefficient features are input into the spiking neural network integrating word attention and channel attention for recognition, and the disease probabilities of multiple syllable categories are output, including: Using an embedding module to perform enhanced feature extraction on each of the Mel-frequency cepstral coefficient features, and generating a target pulse nonlinear feature corresponding to each of the Mel-frequency cepstral coefficient features; Using each of the target pulse nonlinear features as the input of the pulse self-attention module, and outputting the target attention matrix corresponding to each of the target pulse nonlinear features; Each of the target attention matrices is used as the input of the linear output layer, and the disease probability of the syllable category corresponding to each of the target attention matrices is output.

4. The SNN-based post-stroke dysarthria pathological speech recognition method according to claim 3, characterized in that: The embedding module includes a first convolution layer, a first batch of normalization layers, a first leakage pulse neuron, a second convolution layer, a second batch of normalization layers, and a second leakage pulse neuron; the embedding module is used to perform enhanced feature extraction on each of the Mel-cephalogram coefficient features to generate a target pulse nonlinear feature corresponding to each of the Mel-cephalogram coefficient features, including: Using a first convolutional layer to perform convolution operations on each of the Mel-frequency cepstral coefficient features, and outputting first convolution features corresponding to each of the Mel-frequency cepstral coefficient features; Normalizing each of the first convolutional features respectively through a first batch of normalization layers to generate a first normalized feature corresponding to each of the first convolutional features; Using each of the first normalized features as an input of a second leakage pulse neuron, and outputting an initial pulse nonlinear feature corresponding to each of the first normalized features; Using a second convolutional layer to perform convolution operations on each of the initial pulse nonlinear features, and outputting second convolutional features corresponding to each of the initial pulse nonlinear features; Normalizing each of the second convolutional features respectively through a second batch of normalization layers to generate second normalized features corresponding to each of the second convolutional features; Each of the second normalized features is used as an input of a second leakage pulse neuron, and a target pulse nonlinear feature corresponding to each of the second normalized features is output.

5. The SNN-based post-stroke dysarthria pathological speech recognition method according to claim 3, characterized in that: The pulse self-attention module includes a word attention layer, a channel attention layer and a multi-layer perceptron layer; the target pulse nonlinear features are respectively used as inputs of the pulse self-attention module, and the target attention matrix corresponding to each target pulse nonlinear feature is output, including: Using each of the target pulse nonlinear features as the input of the word attention layer, and outputting the word attention matrix corresponding to each of the pulse nonlinear features; Using each of the target pulse nonlinear features as the input of the channel attention layer, and outputting the channel attention matrix corresponding to each of the pulse nonlinear features; Respectively summing and superimposing each of the target pulse nonlinear features and the word attention matrix and the channel attention matrix corresponding to each of the target pulse nonlinear features, and outputting the attention residual matrix corresponding to each of the target pulse nonlinear features; Using each of the attention residual matrices as the input of a multi-layer perceptron layer, and outputting the perceptual pulse nonlinear features corresponding to each of the attention residual matrices; The attention residual matrices and the perceptual pulse nonlinear features corresponding to the attention residual matrices are summed and superimposed respectively, and the target attention matrices corresponding to the target pulse nonlinear features are output.

6. The SNN-based post-stroke dysarthria pathological speech recognition method according to claim 1, characterized in that: The generating of the post-stroke dysarthria pathological speech recognition result corresponding to the patient to be detected according to the disease probabilities of the plurality of syllable categories comprises: Performing a mean operation on the disease probabilities of the plurality of syllable categories, and outputting a mean disease probability; Based on the comparison result of the disease probability mean and the preset probability threshold, the post-stroke dysarthria pathological speech recognition result corresponding to the patient to be detected is generated.

7. The SNN-based post-stroke dysarthria pathological speech recognition method according to claim 1, characterized in that: The model training process of the pulse neural network integrating word attention and channel attention includes: Acquire an audio data training set, preprocess the audio data training set, and output a plurality of Mel-frequency cepstral coefficient features to be trained; Inputting each of the Mel-frequency cepstral coefficient features to be trained into an initial spiking neural network that integrates word attention and channel attention for recognition, and outputting disease probabilities of multiple syllable categories to be trained; Calculating the mean of the disease probabilities of the plurality of syllable categories to be trained to determine the mean of the disease probabilities to be trained; Calculate the target loss value according to the mean disease probability to be trained; Based on the target loss value, the model parameters of the initial spiking neural network that integrates the word attention and the channel attention are updated, an intermediate spiking neural network that integrates the word attention and the channel attention is determined, and the number of iterations is counted in real time; Determine whether the number of iterations reaches a preset iteration threshold; If so, the intermediate pulse neural network that integrates word attention and channel attention is used as the trained pulse neural network that integrates word attention and channel attention.

8. A post-stroke dysarthria pathological speech recognition device based on SNN, characterized in that: include: An acquisition module, used for acquiring a plurality of initial audio data of a patient to be detected, preprocessing each of the initial audio data, and outputting a Mel-frequency cepstral coefficient feature corresponding to each of the initial audio data; A recognition module, used for inputting each of the Mel-frequency cepstral coefficient features into a pulse neural network integrating word attention and channel attention for recognition, and outputting disease probabilities of multiple syllable categories; The prediction module is used to generate a pathological speech recognition result of post-stroke dysarthria corresponding to the patient to be detected according to the disease probabilities of the multiple syllable categories.

9. A computer device, characterized in that: It includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the SNN-based post-stroke dysarthria pathological speech recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the SNN-based post-stroke dysarthria pathological speech recognition method is implemented as described in any one of claims 1-7.

Images