A method and device for speech recognition of stroke dysarthria

Abstract

The present application discloses a method and device for speech recognition of dysarthria caused by stroke. The technical solution of the present application classifies the acquired speech sample data according to the syllable category represented by the audio and the normal and patient categories, and obtains the audio spectrogram through transformation. Then, in the stage of building the network model, the front-end network uses the core processing module designed by plant morphology and physiology, and forms the rhizome with continuous Downsample after the STEM module to quickly transport the computing nodes to a higher receptive field area. The back-end network is based on the alternating configuration of the deep separable convolution and attention mechanism based on the Xception module to form a vine cross structure. The attention mechanism is selectively placed in the alternating convolution modules to improve the recognition ability and accuracy of key speech features, thereby being able to capture the global receptive field at multiple scales, accurately learn and discriminate the significant characteristic information of stroke pathology, and solve the technical problem of low accuracy in the existing articulation analysis of stroke patients.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for speech recognition of stroke dysarthria. Background Art

[0002] Computer technology, particularly machine learning and deep learning, has become a core technology in the field of medical pathology speech analysis. These advanced computational models are capable of processing and analyzing large amounts of speech data, identifying subtle acoustic changes that may be associated with specific diseases. For example, convolutional neural networks (CNNs) and recurrent neural networks (RNNs) can automatically extract and identify key acoustic features associated with various stroke conditions by learning from large datasets of speech samples.

[0003] Key steps in processing pathological speech data include the accurate extraction and effective representation of acoustic features. Signal processing techniques such as spectrograms and Mel-Frequency Cepstral Coefficients (MFCCs) are used within a machine learning framework to extract diagnostically valuable acoustic attributes. These features are then used as training data to train deep learning models to identify pathological conditions in new speech samples.

[0004] However, in practical applications, the quality of speech samples from stroke patients is generally low, making analysis difficult and requiring reliance on larger models with stronger computing power. This leads to the technical problem of low accuracy in articulation analysis of stroke patients using conventional lightweight neural network models. Summary of the Invention

[0005] The present application provides a method and device for speech recognition of stroke dysarthria, which is used to solve the technical problem of low accuracy of existing articulation analysis of stroke patients based on lightweight neural network models.

[0006] To solve the above technical problems, the first aspect of the present application provides a method for speech recognition of stroke dysarthria, comprising:

[0007] Acquiring preset speech sample data, wherein the speech sample data includes: paired dysarthria speech sample data and normal speech sample data;

[0008] Performing syllable classification processing on the speech sample data to obtain a plurality of speech syllable data;

[0009] Converting each of the speech syllable data into corresponding spectrogram data;

[0010] Based on a preset lightweight neural network framework, a dysarthric speech graph recognition model is constructed, and the dysarthric speech graph recognition model is trained using the spectrogram data, wherein the front-end network of the lightweight neural network framework specifically includes: a root processing block, a lightweight network block, and multiple consecutive downsampling blocks; the back-end network of the lightweight neural network framework includes: multiple lightweight network blocks arranged in sequence and connected in a vine cross structure that alternates depthwise separable convolutions and attention mechanisms, as well as a pooling layer and a fully connected layer;

[0011] The dysarthric speech data to be recognized is obtained and converted into spectrogram data as model input. The trained dysarthric speech spectrogram recognition model is operated to obtain the first syllable recognition probability result.

[0012] Preferably, the types of the speech syllable data include: monophthong speech syllable data, diphthong speech syllable data, consonant speech syllable data and nasal vowel speech syllable data.

[0013] Preferably, converting each of the speech syllable data into corresponding spectrogram data specifically includes:

[0014] Performing window processing on each of the speech syllable data;

[0015] The power spectrum density of the speech syllable data is determined by Fourier transform, and the corresponding spectrogram data is obtained according to the power spectrum density using logarithmically transformed color mapping.

[0016] Preferably, it also includes:

[0017] According to the first syllable recognition probability result, the second syllable recognition probability result is calculated using a soft voting method statistical formula.

[0018] Preferably, after converting each of the speech syllable data into corresponding spectrogram data, the method further includes:

[0019] Performing standardization processing on the spectrogram data;

[0020] The spectrogram data is converted into a tensor form, the data robustness is enhanced by noise injection, and the data dynamic range is adjusted using logarithmic compression. Then, the mean and standard deviation are normalized and the processed spectrogram data is output.

[0021] At the same time, the second aspect of the present application provides a speech recognition device for stroke dysarthria, comprising:

[0022] A sample data acquisition unit, configured to acquire preset speech sample data, wherein the speech sample data includes: paired dysarthria speech sample data and normal speech sample data;

[0023] A sample processing unit, configured to perform syllable classification processing on the speech sample data to obtain a plurality of speech syllable data;

[0024] a spectrogram conversion unit, configured to convert each of the speech syllable data into corresponding spectrogram data;

[0025] a network model construction and training unit, configured to construct a dysarthric speech spectrogram recognition model based on a preset lightweight neural network framework, and perform model training on the dysarthric speech spectrogram recognition model using the spectrogram data, wherein the front-end network of the lightweight neural network framework specifically comprises: a root processing block, a lightweight network block, and a plurality of consecutive downsampling blocks; and the back-end network of the lightweight neural network framework comprises: a plurality of lightweight network blocks sequentially arranged and connected in a vine cross structure alternating between depthwise separable convolution and attention mechanisms, as well as a pooling layer and a fully connected layer;

[0026] The dysarthric syllable recognition unit is used to obtain the dysarthric speech data to be recognized and convert it into spectrogram data as model input, and obtain the first syllable recognition probability result through the operation of the trained dysarthric speech spectrogram recognition model.

[0027] Preferably, the types of the speech syllable data include: monophthong speech syllable data, diphthong speech syllable data, consonant speech syllable data and nasal vowel speech syllable data.

[0028] Preferably, the spectrogram conversion unit is specifically used for:

[0029] Performing window processing on each of the speech syllable data;

[0030] The power spectrum density of the speech syllable data is determined by Fourier transform, and the corresponding spectrogram data is obtained according to the power spectrum density using logarithmically transformed color mapping.

[0031] Preferably, it further comprises: a secondary identification unit, configured to:

[0032] According to the first syllable recognition probability result, the second syllable recognition probability result is calculated using a soft voting method statistical formula.

[0033] Preferably, the method further comprises: a spectrogram preprocessing unit, configured to:

[0034] Performing standardization processing on the spectrogram data;

[0035] The spectrogram data is converted into a tensor form, the data robustness is enhanced by noise injection, and the data dynamic range is adjusted using logarithmic compression. Then, the mean and standard deviation are normalized and the processed spectrogram data is output.

[0036] It can be seen from the above technical solutions that this application has the following advantages:

[0037] The technical solution of this application involves classifying the collected speech sample data according to the syllable category represented by the audio, as well as normal and patient, and generating corresponding audio spectrograms. The network model optimizes the front-end and back-end structures to optimize information flow and improve the coverage of the receptive field. In the front-end network, the model imitates the morphological and physiological characteristics of plant roots, using the STEM module together with the continuous downsampling process to form a rhizome-like structure, rapidly expanding the transmission path of the computing nodes, so that the model can perform effective calculations in a wider receptive field. The back-end network is based on the alternating configuration of depthwise separable convolution and attention mechanism involved in the Xception model, forming a "vine cross" structure. In this structure, the attention mechanism is selectively placed in alternating convolution modules to improve the recognition ability and accuracy of key speech features. Through this design, the insertion of each attention module is intended to enhance the model's capture of subtle differences in speech, especially when analyzing the speech of stroke patients. Combined with adaptive maximum pooling technology, this application can accurately extract and utilize key feature information in the global receptive field at multiple scales, significantly improving the accuracy of speech recognition. This technical solution solves the problem of low accuracy of articulation analysis for stroke patients in existing technologies and has important practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0039] Figure 1 This is a flow chart of an embodiment of a method for speech recognition of stroke dysarthria provided in this application.

[0040] Figure 2 This is a schematic diagram of the neural network model architecture in an embodiment of a stroke dysarthria speech recognition method provided in this application.

[0041] Figure 3 This is a structural diagram of an embodiment of a speech recognition device for stroke dysarthria provided in this application. DETAILED DESCRIPTION

[0042] The embodiments of the present application provide a method and device for speech recognition of stroke dysarthria, which are used to address the technical problem of low accuracy in existing articulation analysis of stroke patients based on lightweight neural network models.

[0043] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0044] First, the present application provides a method for speech recognition of stroke dysarthria, comprising:

[0045] See also Figure 1 This embodiment provides a method for speech recognition of stroke dysarthria, including:

[0046] Step 101: Obtain preset voice sample data;

[0047] The speech sample data includes: paired dysarthria speech sample data and normal speech sample data;

[0048] Step 102: performing syllable classification processing on the speech sample data to obtain a plurality of speech syllable data;

[0049] Step 103: convert each speech syllable data into corresponding spectrogram data;

[0050] Step 104: constructing a dysarthric speech spectrum recognition model based on a preset lightweight neural network framework, and training the dysarthric speech spectrum recognition model using the spectrogram data;

[0051] The front-end network of the lightweight neural network framework specifically includes: a root processing block, a lightweight network block, and multiple consecutive downsampling blocks. The back-end network of the lightweight neural network framework includes: multiple lightweight network blocks arranged in sequence according to a vine cross structure that alternates depthwise separable convolution and attention mechanisms, as well as a pooling layer and a fully connected layer.

[0052] Step 105: Acquire the dysarthric speech data to be recognized and convert it into spectrogram data as model input, and obtain the first syllable recognition probability result by operating the trained dysarthric speech spectrogram recognition model.

[0053] It should be noted that according to the technical solution provided in this embodiment, speech sample data of dysarthria speech sample data and normal speech sample data are first obtained, and these speech sample data are used to generate speech syllable data for training the model. The specific implementation method is: in an environment without background noise, the speech data of normal people and dysarthria patients (who have undergone a comprehensive medical assessment) are recorded in sequence, and then the speech sample data are syllable classified to obtain multiple speech syllable data, including vowels, compound vowels, consonants and nasal vowels. The obtained speech syllable data can be selected as audio files that meet the sound quality standards for storage, and each speech syllable data can be further uniformly cropped to one second in length to facilitate model learning.

[0054] Next, based on the speech syllable data obtained in the previous step and the resulting dataset, the sound signal is preprocessed and windowed. This process divides the signal into short time frames and applies a window function to reduce edge effects. Subsequently, a Fast Fourier Transform (FFT) is used to analyze the frequency components of each frame and calculate its power spectral density. The power values ​​are converted to a logarithmic scale to accommodate the human ear's nonlinear perception of sound intensity. Finally, color mapping is used to convert the power spectrum values ​​into color images, forming a clear spectrogram.

[0055] Next, based on a preset lightweight neural network framework, a speech pattern recognition model for articulation disorders is constructed. The speech pattern recognition model for articulation disorders provided in this embodiment is specifically constructed based on the lightweight neural network RepViT framework. The front-end network portion of the lightweight neural network framework includes: a root processing block, a lightweight network block, and multiple consecutive downsampling blocks. By simplifying the network front-end, the Stem part remains unchanged. After the Stem part, a lightweight module and an attention module including a spatial and channel attention mechanism are added, and then multiple consecutive downsamplings are performed. The back-end network includes: multiple lightweight network blocks arranged in sequence according to a vine cross structure with alternating configurations of depthwise separable convolution and attention mechanism, as well as a pooling layer and a fully connected layer. Overlap calculation (set to 11 times for testing with reference to the Xception network and vine cross structure) and depthwise separable convolution technology are used, and residual connections are used to enhance the performance of the model. In the classification output stage, adaptive maximum pooling technology is used to extract key information, and the diagnosis results are output through the fully connected layer.

[0056] This vine crossover enhances selective feature processing by alternately inserting attention mechanisms within the depthwise separable modules. For example, among the 11 repeated modules, the odd-numbered modules do not include the attention mechanism (ATT), while the even-numbered modules do. This optimizes performance and helps the network focus more on important features during processing, thereby improving the accuracy and efficiency of the model.

[0057] More specifically, the lightweight neural network framework architecture mentioned in this embodiment is as follows Figure 2 As shown in the figure, Stem: root processing block, contains three parts Conv2d_BN, GeluAct, and Conv2d_BN.

[0058] Conv2d_BN: This is a custom class that combines two-dimensional convolution (Conv2d) and batch normalization (BatchNorm2d) operations. The main function of this class is to combine convolution and batch normalization together for easy use when building neural networks; is the size of the convolution kernel; Stride2 refers to the stride of the convolution.

[0059] GeluAct: refers to the use of Gelu function as the activation function.

[0060] RepvitBlock: It is a lightweight network block customized based on the Repvit network framework, including DW_Conv, OptionalCBAM, and FFN.

[0061] DW_Conv: refers to the depth of separable convolution; Indicates that operations are only performed within the channel, and each channel uses an independent 3x3 convolution kernel to extract spatial features without changing the number of channels; This convolution, also known as point-by-point convolution, operates on each pixel using a 1x1 convolution kernel. Unlike depthwise convolution, this 1x1 convolution kernel operates across all input channels. It blends information between channels and is typically used after depthwise convolution to combine features from each channel. Therefore, the two constitute depthwise separable convolution, which is simply two submodules of the underlying implementation.

[0062] OptionalCBAM: refers to the CBAM module that can be executed after performing the above operations. It is also called the optional convolutional attention module.

[0063] ResidualBlock, a residual block, uses two 1x1 convolutional layers to adjust the number of channels and adds a GELU activation function to introduce nonlinearity. Finally, the residual module implements a residual connection (implemented in the code), allowing the input to be directly added to the output. This channel mixing approach combines the classic convolution operation in deep learning with modern residual connection techniques, efficiently processing and mixing information from different channels while maintaining model training stability and ease of optimization.

[0064] Pooling: pooling; fc: fully connected.

[0065] Downsample: Downsampling. The modules used include Conv2d_BN and FFN, which have been described above and will not be elaborated on.

[0066] The number of downsampling steps is not fixed and unchangeable; it is determined by the model configuration. Downsampling aims to balance feature representation and computational efficiency. The preferred four-fold downsampling in this embodiment generally reduces the input image resolution to a reasonable size, allowing subsequent fully connected layers or other high-level operations to effectively process the feature maps. For example, if the initial input is a 224x224 image, after four downsampling steps, the feature map size will be 14x14 (assuming a 2:1 downsampling ratio each time). This preserves sufficient feature information while significantly reducing computational effort. It also expands the effective receptive field, allowing the network to capture global information within a smaller feature map, thereby better capturing the overall structure and context of the image. While further downsampling can further reduce the feature map size, this can also result in information loss, particularly of fine detail. Therefore, four-fold downsampling generally strikes a balance between reducing the feature map size and minimizing the loss of detail.

[0067] In addition, after building the initial network model, the lightweight convolutional neural network is trained and adjusted using the spectrogram data obtained in the previous steps to optimize its performance. Before model training, the spectrogram data can be preprocessed and then the lightweight convolutional neural network RepViT can be adjusted using the enhanced spectrogram dataset. The preprocessing implementation example includes: comparing the spectrograms of normal people and patients, uniformly adjusting the data to a size of 224x224 pixels, and performing data augmentation processing, including random horizontal and vertical flipping and random image rotation, to increase the diversity of the dataset. After that, the image is converted into a tensor, and the data robustness is enhanced through noise injection. Then, logarithmic compression is used to adjust the data dynamic range. After that, the mean and standard deviation are normalized to standardize the data scale, and the processed spectrogram data is finally output to ensure that all data is at the same scale.

[0068] Finally, based on the trained dysarthria speech graph recognition model, in actual application, the speech data to be recognized of the target patient can be collected as the model input of the dysarthria speech graph recognition model, so as to obtain the syllable recognition probability result output by the model through the prior knowledge contained in the dysarthria speech graph recognition model.

[0069] Furthermore, after step 105, the following steps may be further included:

[0070] Step 106: Based on the first syllable recognition probability result, the second syllable recognition probability result is calculated using a soft voting statistical formula.

[0071] It should be noted that, according to a standardized process, each syllable of the patient's data is converted into a spectrogram input. The model then analyzes and tests its recognition probability. The recognition probability of each syllable's corresponding image is then printed, and a soft voting method is used to calculate the patient's pathology probability. Through dual verification of the model probability and the calculated probability, a comprehensive probability value is ultimately output.

[0072] The soft voting method of this embodiment is expressed as: , where, given a speaker with m syllables, let σ k Refers to the logistic regression output of the k-th syllable of the speaker. Soft majority This is the speaker-level probability score calculated by the soft majority method; m: represents the total number of syllables of the test speaker; σ k : represents the logistic regression output of the k-th syllable, usually in the range of [0,1], indicating the probability that the syllable belongs to a certain category; : represents the sum of the logistic regression outputs of all syllables. This formula basically calculates the arithmetic mean of the logistic regression outputs of all syllables to get the overall speaker probability.

[0073] The above is a detailed description of an embodiment of a method for speech recognition of dysarthria caused by stroke provided by the present application. The following is a detailed description of an embodiment of a speech recognition device for dysarthria caused by stroke provided by the present application.

[0074] See also Figure 3 This embodiment provides a speech recognition device for stroke dysarthria, comprising:

[0075] The sample data acquisition unit 201 is configured to acquire preset speech sample data, wherein the speech sample data includes: paired dysarthria speech sample data and normal speech sample data;

[0076] The sample processing unit 202 is used to perform syllable classification processing on the speech sample data to obtain a plurality of speech syllable data;

[0077] The spectrogram conversion unit 203 is used to convert each speech syllable data into corresponding spectrogram data;

[0078] The network model construction and training unit 204 is configured to construct a dysarthric speech graph recognition model based on a preset lightweight neural network framework, and to perform model training on the dysarthric speech graph recognition model using spectrogram data, wherein the front-end network of the lightweight neural network framework specifically includes: a root processing block, a lightweight network block, and multiple consecutive downsampling blocks; and the back-end network of the lightweight neural network framework includes: multiple lightweight network blocks sequentially arranged and connected in a vine cross structure that alternates depthwise separable convolution and attention mechanisms, as well as a pooling layer and a fully connected layer;

[0079] The dysarthric syllable recognition unit 205 is used to obtain the dysarthric speech data to be recognized and convert it into spectrogram data as a model input, and obtain the first syllable recognition probability result by operating the trained dysarthric speech spectrogram recognition model.

[0080] Furthermore, the types of speech syllable data include: monophthong speech syllable data, diphthong speech syllable data, consonant speech syllable data and nasal vowel speech syllable data.

[0081] Furthermore, the spectrogram conversion unit 203 is specifically used to:

[0082] Perform window processing on each speech syllable data;

[0083] The power spectrum density of the speech syllable data is determined by Fourier transform, so as to obtain the corresponding spectrogram data according to the power spectrum density.

[0084] Furthermore, the system further includes a secondary recognition unit 206 configured to:

[0085] According to the first syllable recognition probability result, the second syllable recognition probability result is calculated using a soft voting method statistical formula.

[0086] Furthermore, the system further comprises a spectrogram preprocessing unit 2031, configured to:

[0087] Unify the spectrogram data into standard spectrogram specifications;

[0088] The spectrogram data is converted into a tensor form, the data robustness is enhanced by noise injection, and the data dynamic range is adjusted using logarithmic compression. Then, the mean and standard deviation are normalized and the processed spectrogram data is output.

[0089] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the terminals, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0090] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0091] The terms "first," "second," "third," "fourth," and the like (if any) in the specification of the present application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present application described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0092] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0093] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0094] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0095] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0096] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for speech recognition of stroke dysarthria, characterized in that: include: Acquiring preset speech sample data, wherein the speech sample data includes: paired dysarthria speech sample data and normal speech sample data; Performing syllable classification processing on the speech sample data to obtain a plurality of speech syllable data; Converting each of the speech syllable data into corresponding spectrogram data; Based on a preset lightweight neural network framework, a dysarthric speech graph recognition model is constructed, and the dysarthric speech graph recognition model is trained using the spectrogram data, wherein the front-end network of the lightweight neural network framework specifically includes: a root processing block, a lightweight network block, and multiple consecutive downsampling blocks; the back-end network of the lightweight neural network framework includes: multiple lightweight network blocks arranged in sequence and connected in a vine cross structure that alternates depthwise separable convolutions and attention mechanisms, as well as a pooling layer and a fully connected layer; The dysarthric speech data to be recognized is obtained and converted into spectrogram data as model input. The trained dysarthric speech spectrogram recognition model is operated to obtain the first syllable recognition probability result.

2. A method for speech recognition of stroke dysarthria according to claim 1, characterized in that: The types of the speech syllable data include: monophthong speech syllable data, diphthong speech syllable data, consonant speech syllable data and nasal vowel speech syllable data.

3. A method for speech recognition of stroke dysarthria according to claim 1, characterized in that: The converting of each of the speech syllable data into corresponding spectrogram data specifically includes: Performing window processing on each of the speech syllable data; The power spectrum density of the speech syllable data is determined by Fourier transform, and the corresponding spectrogram data is obtained according to the power spectrum density using logarithmically transformed color mapping.

4. The method for speech recognition of stroke dysarthria according to claim 1, characterized in that: Also includes: According to the first syllable recognition probability result, the second syllable recognition probability result is calculated using a soft voting method statistical formula.

5. The method for speech recognition of stroke dysarthria according to claim 1, characterized in that: After converting each of the speech syllable data into corresponding spectrogram data, the method further includes: Performing standardization processing on the spectrogram data; The spectrogram data is converted into a tensor form, the data robustness is enhanced by noise injection, and the data dynamic range is adjusted using logarithmic compression. Then, the mean and standard deviation are normalized and the processed spectrogram data is output.

6. A speech recognition device for stroke dysarthria, characterized in that: include: A sample data acquisition unit, configured to acquire preset speech sample data, wherein the speech sample data includes: paired dysarthria speech sample data and normal speech sample data; A sample processing unit, configured to perform syllable classification processing on the speech sample data to obtain a plurality of speech syllable data; a spectrogram conversion unit, configured to convert each of the speech syllable data into corresponding spectrogram data; a network model construction and training unit, configured to construct a dysarthric speech spectrogram recognition model based on a preset lightweight neural network framework, and perform model training on the dysarthric speech spectrogram recognition model using the spectrogram data, wherein the front-end network of the lightweight neural network framework specifically comprises: a root processing block, a lightweight network block, and a plurality of consecutive downsampling blocks; and the back-end network of the lightweight neural network framework comprises: a plurality of lightweight network blocks sequentially arranged and connected in a vine cross structure alternating between depthwise separable convolution and attention mechanisms, as well as a pooling layer and a fully connected layer; The dysarthric syllable recognition unit is used to obtain the dysarthric speech data to be recognized and convert it into spectrogram data as model input, and obtain the first syllable recognition probability result through the operation of the trained dysarthric speech spectrogram recognition model.

7. The speech recognition device for stroke dysarthria according to claim 6, characterized in that: The types of the speech syllable data include: monophthong speech syllable data, diphthong speech syllable data, consonant speech syllable data and nasal vowel speech syllable data.

8. The speech recognition device for stroke dysarthria according to claim 6, characterized in that: The spectrogram conversion unit is specifically used for: Performing window processing on each of the speech syllable data; The power spectrum density of the speech syllable data is determined by Fourier transform, and the corresponding spectrogram data is obtained according to the power spectrum density using logarithmically transformed color mapping.

9. The speech recognition device for stroke dysarthria according to claim 6, characterized in that: Also includes: Secondary identification unit, used for: According to the first syllable recognition probability result, the second syllable recognition probability result is calculated using a soft voting method statistical formula.

10. The speech recognition device for stroke dysarthria according to claim 6, characterized in that: Also includes: Spectrogram preprocessing unit, used to: Performing standardization processing on the spectrogram data; The spectrogram data is converted into a tensor form, the data robustness is enhanced by noise injection, and the data dynamic range is adjusted using logarithmic compression. Then, the mean and standard deviation are normalized and the processed spectrogram data is output.

Images