Speech recognition method and system for dysarthria group

Abstract

The application relates to a speech recognition method and system for a dysarthria group, the method comprising: collecting dysarthria speech data, preprocessing the dysarthria speech data, and obtaining effective speech segments; inputting the effective speech segments into a dysarthria speech recognition model to obtain phoneme-level or Chinese character-level recognition results; the dysarthria speech recognition model is obtained by training a Conformer model using a first training set, the first training set comprising: fake dysarthria audio data; the fake dysarthria audio data is obtained by speech conversion based on a CycleGAN-VC speech conversion model; in the process of training the model in the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized through a whale optimization algorithm. The application can improve the accuracy and robustness of dysarthria speech recognition.

Description

Technical Field

[0001] This invention relates to the fields of speech recognition and artificial intelligence, and in particular to a speech recognition method and system for people with speech disorders. Background Technology

[0002] Speech recognition technology has made significant progress in recent years and is widely used in human-computer interaction, intelligent voice assistants, and voice control. However, for people with speech disorders, existing speech recognition systems still face many challenges. Speech disorders are a common motor speech disorder, usually caused by neurological diseases such as cerebral palsy, Parkinson's disease, amyotrophic lateral sclerosis (ALS), and stroke. These diseases cause patients to have inaccurate, disfluent, slow, and indistinct speech, making their speech difficult to recognize accurately by existing speech recognition systems. In addition, most people with speech disorders also have motor impairments, which makes it difficult for them to use traditional interaction methods such as mice, keyboards, and touch interfaces. Therefore, voice control has become an ideal assistive interaction solution.

[0003] Despite significant progress in general speech recognition, speech recognition for articulation disorders remains a highly challenging task. First, data on articulation disorders is extremely limited. Due to the relatively small number of people with articulation disorders and the complex process of data collection and annotation, suitable training data is scarce. This makes it difficult for existing deep learning models to obtain sufficient data for effective training, thus limiting model performance. Second, articulation disorders exhibit high variability. Speech by individuals with articulation disorders shows significant individual differences and irregularities in pitch, intensity, speech rate, and pauses. This variability makes it difficult for existing speech recognition models to accurately match and recognize the speech. Furthermore, the significant differences in spectral-temporal features between articulation disorder speech and normal speech further increase the difficulty of recognition. Finally, due to the large differences in speech features among different individuals with articulation disorders, existing speech recognition models often require adaptive adjustments to improve accuracy and robustness. However, due to data sparsity and high speech variability, existing adaptive methods perform poorly on articulation disorder speech recognition tasks.

[0004] In summary, existing speech recognition technologies suffer from problems such as data sparsity, high speech variability, and difficulty in adaptation for individuals with articulation disorders when processing their speech. These issues limit the development and application of speech recognition technology for articulation disorders. Therefore, developing a speech recognition method for the articulation disorder community that can effectively address these problems is of significant theoretical and practical importance. Summary of the Invention

[0005] To address the problems existing in the prior art, the present invention aims to provide a speech recognition method and system for people with articulation disorders, in order to address the issues of data sparsity, strong speech variability, and speaker adaptation in speech recognition for people with articulation disorders, and to improve the accuracy and robustness of speech recognition for people with articulation disorders.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A speech recognition method for people with articulation disorders, comprising:

[0008] Collect speech data of articulation disorders, preprocess the speech data of articulation disorders, and obtain valid speech segments;

[0009] The effective speech segment is input into the speech recognition model for articulation disorders to obtain phoneme-level or character-level recognition results; the speech recognition model for articulation disorders is obtained by training a Conformer model using a first training set, the first training set including: fake articulation disorder audio data; the fake articulation disorder audio data is obtained by speech conversion based on the CycleGAN-VC speech conversion model;

[0010] During the training of the model on the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized using the whale optimization algorithm.

[0011] Optionally, preprocessing the speech data of the articulation disorder includes:

[0012] The speech data with articulation disorders is pre-emphasized, framed, and windowed. The processed speech data with articulation disorders is then automatically detected and cropped to retain the effective speech segments.

[0013] Optionally, pre-emphasizing the speech data of the articulation disorder includes:

[0014] y[n] = x[n] - αx[n-1];

[0015] Where α is a weighting parameter used to control the degree of pre-emphasis, y[n] represents the value of the preprocessed speech signal at the nth sampling point, and x[n] represents the value of the original speech signal at the nth sampling point.

[0016] Optionally, windowing processing of the speech data with articulation disorders includes:

[0017] w[n]=0.54-0.46cos(2πn / (N-1));

[0018] Where w[n] represents the window function value of the nth sampling point after windowing, n represents the current sampling point index, and N represents the length of the window function, i.e. the number of sampling points.

[0019] Optionally, obtaining the fake speech disorder audio data includes:

[0020] Multiple speech datasets of articulation disorders are acquired, and the speech datasets of articulation disorders are processed. The processing includes: organizing annotation information; the annotation information includes: ID, gender, age, disorder type, and recording conditions.

[0021] The processed dataset is randomly sampled and segmented, and long audio files are segmented according to predefined time windows to obtain short audio segments;

[0022] For each short speech segment, silence endpoint detection is performed, and silent segments before and after are removed to obtain valid speech segments, and the sampling rate of the valid speech segments is unified.

[0023] To acquire healthy speech data, the effective speech segments and healthy speech data after uniform sampling rate are respectively input into the CycleGAN-VC speech conversion model to obtain synthesized data. The CycleGAN-VC speech conversion model is obtained by training using a second training set and adversarial loss and cycle consistency loss. The second training set includes: the original effective speech segments and their corresponding healthy speech data.

[0024] Optionally, obtaining the fake speech disorder audio data further includes:

[0025] The synthesized data is preprocessed, and multi-channel spectral features are extracted from the preprocessed synthesized data to obtain multi-channel spectral features;

[0026] The multi-channel spectral features are weighted and spliced ​​or weighted and summed to obtain a comprehensive vector. The comprehensive vector is then subjected to layer normalization to obtain the fake articulation disorder audio data.

[0027] Optionally, obtaining the multi-channel spectral features includes:

[0028] Mel frequency calculations are performed on the preprocessed synthetic data to obtain the multidimensional Mel frequency cepstral coefficients of each frame and their corresponding first-order and second-order difference features.

[0029] Bark frequency scale extraction was performed on the preprocessed synthetic data to obtain multidimensional perceptual linear prediction coefficients and dynamic features;

[0030] Fbank extraction is performed on the preprocessed synthetic data to obtain the energy characteristics of the multidimensional filter bank;

[0031] The multidimensional Mel frequency cepstral coefficients and their corresponding first-order and second-order difference features, as well as the multidimensional sensing linear prediction coefficients and dynamic features and the multidimensional filter bank energy features, are preprocessed respectively. Specifically, the multidimensional Mel frequency cepstral coefficients and their corresponding first-order and second-order difference features and the multidimensional filter bank energy features are logarithmically compressed to reflect the perceptual characteristics of the human ear. All processed features are then normalized to obtain the multi-channel spectral features.

[0032] To achieve the above objectives, the present invention provides a speech recognition system for people with articulation disorders, comprising:

[0033] The effective speech acquisition module is used to collect speech data of speech disorders, preprocess the speech data of speech disorders, and obtain effective speech segments;

[0034] An effective speech recognition module is used to input the effective speech segment into a speech recognition model for articulation disorders to obtain phoneme-level or character-level recognition results; the speech recognition model for articulation disorders is obtained by training a Conformer model using a first training set, the first training set including: fake articulation disorder audio data; the fake articulation disorder audio data is obtained by speech conversion based on the CycleGAN-VC speech conversion model;

[0035] During the training of the model on the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized using the whale optimization algorithm.

[0036] To achieve the above objectives, the present invention provides a memory, comprising: a storage unit for storing executable instructions; and a processing unit for connecting to the memory to execute the executable instructions.

[0037] The beneficial effects of this invention are as follows:

[0038] This invention uses the CycleGAN-VC framework to achieve timbre transfer without pairwise data. Through adversarial training and cycle consistency loss, it generates high-quality synthetic data while maintaining pronunciation / intonation, thus alleviating the problem of scarce Chinese articulation disorder speech data.

[0039] This invention employs a three-level enhancement system, which involves constructing a spectrum transformation and feature enhancement, followed by fusion of variable features, to improve the generalization and robustness of the model.

[0040] This invention uses an optimized Conformer model, which, compared with conventional speech recognition models, has a stronger ability to capture local features while focusing on global features, and can achieve better recognition performance with a smaller number of model parameters. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of a speech recognition method for people with speech disorders according to an embodiment of the present invention;

[0043] Figure 2 This is a flowchart of the method for generating fake speech disorder audio data according to an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the CycleGan-VC network structure according to an embodiment of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] like Figure 1 As shown, this embodiment discloses a speech recognition method for people with articulation disorders, including: collecting speech data with articulation disorders; preprocessing the speech data with articulation disorders to obtain effective speech segments; inputting the effective speech segments into a speech recognition model with articulation disorders to obtain phoneme-level or character-level recognition results; the speech recognition model with articulation disorders is obtained by training a Conformer model using a first training set, the first training set including: fake audio data with articulation disorders; the fake audio data with articulation disorders is obtained by speech conversion based on the CycleGAN-VC speech conversion model; during the training process of the model on the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized using the whale optimization algorithm.

[0048] Specifically, this embodiment discloses a speech recognition method for people with articulation disorders, including:

[0049] S1: Voice data acquisition. When the user clicks the "Start Recognition" button, the system automatically starts sampling at a sampling rate of 16kHz and 16bit PCM. The system records and displays the voice waveform in real time.

[0050] S2: Speech preprocessing: S21: After acquisition, the system performs the following in sequence: pre-emphasis (α=0.97) to enhance high-frequency components; framing (25ms frame length, 10ms frame shift) and adding a Hamming window; spectral subtraction noise reduction based on noise estimation; S22: Automatic detection and cropping of silent segments to retain effective speech segments; S23: Mean and variance normalization of the speech signal to prepare it for input into the model.

[0051] S3: Model Inference: S31: The system feeds the preprocessed frame features into the trained Conformer model; S32: The model outputs phoneme-level or Chinese character-level recognition results based on CTC decoding; S33: The system extracts and records pronunciation deviation features (such as phoneme error type, duration distortion, formant shift, etc.).

[0052] S4: The system frequently determines whether the user has stopped speaking based on the current silence period.

[0053] S5: Display the recognized text results.

[0054] As a preferred choice, the Conformer model was selected as the base model. This model combines the advantages of Transformer and Convolutional Neural Networks (CNNs), effectively handling global dependencies and local features of speech signals. The structure of the Conformer model was optimized, including adjusting the parameters of the multi-head self-attention module, convolutional module, and feedforward module to improve model performance. Finally, the whale optimization algorithm was used to optimize the kernel size, number of filters, and number of neurons in the CNN, enhancing the model's ability to learn the speech variability features of articulation disorders.

[0055] Furthermore, the preprocessing of speech data with articulation disorders includes: pre-emphasis, framing, and windowing of the speech data with articulation disorders, automatic detection and cropping of the processed speech data with articulation disorders, and retention of valid speech segments.

[0056] Furthermore, obtaining forged articulation disorder audio data includes: acquiring multiple articulation disorder speech datasets; processing these datasets, including: organizing annotation information; annotation information includes: ID, gender, age, disorder type, and recording conditions; randomly sampling and segmenting the organized datasets, dividing long speech files into short speech segments according to predefined time windows; performing silence endpoint detection on each short speech segment, removing preceding and following silent segments to obtain valid speech segments, and unifying the sampling rate of valid speech segments; obtaining healthy speech data, and inputting the valid speech segments and healthy speech data after unifying the sampling rate into the CycleGAN-VC speech conversion model to obtain synthesized data; the CycleGAN-VC speech conversion model is trained using a second training set and adversarial loss and cycle consistency loss; the second training set includes: the original valid speech segments and their corresponding healthy speech data. The synthesized data is preprocessed, and multi-channel spectral features are extracted from the preprocessed synthesized data to obtain multi-channel spectral features. The multi-channel spectral features are then weighted and concatenated or weighted and summed to obtain a comprehensive vector. The comprehensive vector is then subjected to layer normalization to obtain fake speech disorder audio data.

[0057] The acquisition of multi-channel spectral features includes: performing Mel frequency calculation on the preprocessed synthetic data to obtain the multidimensional Mel frequency cepstral coefficients of each frame and their corresponding first-order and second-order difference features; and extracting the Bark frequency scale from the preprocessed synthetic data to obtain multidimensional perceptual linear prediction coefficients and dynamic features.

[0058] Fbank extraction was performed on the preprocessed synthetic data to obtain the energy features of the multidimensional filter bank. The multidimensional Mel frequency cepstral coefficients and their corresponding first-order and second-order difference features, as well as the multidimensional sensing linear prediction coefficients, dynamic features, and energy features of the multidimensional filter bank were preprocessed respectively. Among them, the multidimensional Mel frequency cepstral coefficients and their corresponding first-order and second-order difference features and the energy features of the multidimensional filter bank were logarithmically compressed to reflect the perceptual characteristics of the human ear. All processed features were normalized to obtain multi-channel spectral features.

[0059] Specifically, obtaining fake speech disorder audio data includes:

[0060] S1: Obtain speech datasets for articulation disorders. Download or obtain multiple publicly available speech datasets for articulation disorders (such as UASpeech, Laryngograph, CDSD, etc.).

[0061] S2: Preprocessing the dataset: S21: Organize annotation information, including speaker ID, gender, age, disability type, and recording conditions. S22: Randomly extract and segment the original speech file, dividing long speech files into short speech segments according to predefined time windows (e.g., 5s). S23: Perform silence endpoint detection on each segment, removing silent segments before and after, to obtain the trimmed effective speech segments. S24: Standardize the sampling rate of the trimmed speech to 16kHz, 16bit PCM format.

[0062] S3: As Figure 2 As shown, the data generation based on CycleGAN-VC is as follows: S31: Construct a CycleGAN-VC speech conversion model, including two generators G(H→D) (healthy → disordered) and G(D→H) (disordered → healthy), and two discriminators D(D) and D(H), trained using adversarial loss and cycle consistency loss. S32: Input the speech of a healthy person with articulation disorder and the speech with articulation disorder obtained in step S2 into G(H→D) and G(D→H) respectively to generate synthetic data similar to real speech with articulation disorder; at the same time, retain the original real data to expand the training set.

[0063] like Figure 3 As shown, the CycleGAN-VC model is a deep learning model for speech conversion that can convert source speech into target speech while preserving important features such as the speaker's timbre. The core structure of this model includes two generators and two discriminators.

[0064] The generator is the core component of CycleGAN-VC, used for speech conversion. The model has two generators:

[0065] First generator (G(X→Y)): Converts the input source speech X into the target speech Y.

[0066] The second generator (G(Y→X)) converts the input target speech Y back to the source speech X.

[0067] The generator's architecture is based on a 1D convolutional neural network (CNN), which effectively captures relationships between features while preserving the temporal structure of speech. The generator's architecture includes an encoder, downsampling layers, residual layers, and upsampling layers. To simulate a forgetting gate mechanism similar to LSTM, the generator employs a gated convolutional neural network (GCNN) with gated linear units (GLUs) as activation functions, allowing selective propagation of information based on the state of previous layers. Furthermore, the generator uses instance normalization to stabilize the training process and improve model performance.

[0068] The discriminator's role is to evaluate whether the generated speech is authentic. CycleGAN-VC contains two discriminators:

[0069] First discriminator (D(Y)): Used to determine whether the target speech Y is real.

[0070] Second discriminator (D(X)): used to determine whether the source speech X is real.

[0071] Discriminators typically employ a 2D CNN structure, making judgments based on 2D spectral textures. To better capture local and global information, the discriminator uses a PatchGAN structure, outputting a multi-dimensional matrix that takes into account differences in global receptive field information.

[0072] The interaction between the generator and discriminator is as follows: Source speech X is converted into target speech Y by the first generator (G(X→Y)). Target speech Y is converted back into source speech X by the second generator (G(Y→X)). Source speech X is discriminated by the second discriminator (D(X)). Target speech Y is discriminated by the first discriminator (D(Y)). With this structure, CycleGAN-VC can achieve efficient conversion from source speech to target speech while preserving some key speech features, such as the speaker's timbre. This design not only improves the naturalness of speech conversion but also enhances the model's robustness and generalization ability.

[0073] S4: Speech signal pre-emphasis, framing, and windowing:

[0074] S41: Pre-emphasis - Apply a first-order high-pass filter to each speech segment;

[0075] y[n]=x[n]-αx[n-1],α=0.97;

[0076] S42: Framing - Divide the pre-emphasized signal into frames according to a 25ms window and a 10ms step size;

[0077] S43: Windowing – Apply a Hamming window to each frame to reduce signal truncation effects;

[0078] w[n]=0.54-0.46cos(2πn / (N-1));

[0079] Where w[n] represents the window function value of the nth sampling point after windowing, n represents the current sampling point index (from 0 to N-1), and N represents the length of the window function, i.e. the number of sampling points.

[0080] S5: Multi-channel spectral feature extraction: S51: MFCC - Calculate the 13-dimensional Mel frequency cepstral coefficients and their first and second order differences for each frame, totaling 39 dimensions; S52: PLP - Extract 12-dimensional perceptual linear prediction coefficients and dynamic features based on the Bark frequency scale; S53: Fbank - Extract 40-dimensional filter bank energy; S54: Perform logarithmic compression and dimensionality normalization on the above features respectively.

[0081] S6: Feature Fusion and Normalization: S61: Fuse MFCC, PLP, and Fbank features into a comprehensive vector by weighted concatenation or weighted summation (e.g., the dimensions after concatenation are 39+13+40=92); S62: Perform layer normalization on the fused vector to ensure consistent distribution of each dimension, which is beneficial for deep network learning.

[0082] S7: Obtain the final fake speech disorder audio data.

[0083] As a preferred choice, the Conformer model was selected as the base model. This model combines the advantages of Transformer and Convolutional Neural Networks (CNNs), effectively handling global dependencies and local features of speech signals. The structure of the Conformer model was optimized, including adjusting the parameters of the multi-head self-attention module, convolutional module, and feedforward module to improve model performance. Finally, the whale optimization algorithm was used to optimize the kernel size, number of filters, and number of neurons in the CNN, enhancing the model's ability to learn the speech variability features of articulation disorders.

[0084] This embodiment also provides a speech recognition system for people with articulation disorders, including: an effective speech acquisition module for collecting speech data with articulation disorders, preprocessing the speech data with articulation disorders, and acquiring effective speech segments; an effective speech recognition module for inputting the effective speech segments into a speech recognition model with articulation disorders to obtain phoneme-level or character-level recognition results; the speech recognition model with articulation disorders is obtained by training a Conformer model using a first training set, the first training set including: fake articulation disorder audio data; the fake articulation disorder audio data is obtained by speech conversion based on the CycleGAN-VC speech conversion model; during the training of the model using the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized using the whale optimization algorithm.

[0085] Specifically, this embodiment also provides a speech recognition system for people with articulation disorders, including:

[0086] The first processing module is used to obtain publicly available Chinese speech datasets with articulation disorders.

[0087] The second processing module is used to generate fake speech data with speech disorders based on the Chinese speech dataset with the CycleGAN-VC model.

[0088] The third processing module is used to pre-emphasize, frame, and window real and fake speech data to reduce redundant information in the speech signal.

[0089] The fourth processing module is used to extract various speech features, including Mel-frequency cepstral coefficients (MFCC), perceptual linear prediction coefficients (PLP), and filter banks (Fbank), to comprehensively capture the spectral information of the speech and perform spectral enhancement processing on the extracted features.

[0090] The fifth processing module is used to input the extracted speech features into the improved Conformer model for training, using the CTC loss function as the training objective to obtain the trained model weights.

[0091] The sixth processing module extracts speech variability features as auxiliary features and combines them with adaptive techniques for speech disorders to adjust the acoustic model. This improves the model's adaptability and recognition performance for speech with different degrees of speech disorder.

[0092] The seventh processing module is used to validate the trained model weights on multiple public datasets;

[0093] The eighth processing module is used to perform inference on the trained model to achieve the recognition of speech disorders.

[0094] This embodiment also provides a memory, including: a storage unit for storing executable instructions; and a processing unit for connecting to the memory to execute the executable instructions.

[0095] Specifically, this embodiment also provides a memory, including a memory and a processor. The memory stores a computer program that is run by the processor. When the computer program is run by the processor, it executes a Chinese speech recognition method for speech disorders.

[0096] This embodiment also provides a storage medium on which a computer program is stored. When the computer program is run, it executes a Chinese speech recognition method for speech disorders.

[0097] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A speech recognition method for people with articulation disorders, characterized in that, include: Collect speech data of articulation disorders, preprocess the speech data of articulation disorders, and obtain valid speech segments; The effective speech segment is input into the speech recognition model for speech disorders to obtain phoneme-level or Chinese character-level recognition results; The speech recognition model for articulation disorders is obtained by training a Conformer model using a first training set, which includes: fake articulation disorder audio data; the fake articulation disorder audio data is obtained by speech conversion based on the CycleGAN-VC speech conversion model. Obtaining the fake speech disorder audio data includes: Multiple speech datasets of articulation disorders are acquired, and the speech datasets of articulation disorders are processed. The processing includes: organizing annotation information; the annotation information includes: ID, gender, age, disorder type, and recording conditions. The processed dataset is randomly sampled and segmented, and long audio files are segmented according to predefined time windows to obtain short audio segments; For each short speech segment, silence endpoint detection is performed, and silent segments before and after are removed to obtain valid speech segments, and the sampling rate of the valid speech segments is unified. To acquire healthy speech data, effective speech segments and healthy speech data after uniform sampling rate are input into the CycleGAN-VC speech conversion model to obtain synthesized data. The CycleGAN-VC speech conversion model is trained using a second training set and adversarial loss and cycle consistency loss. The second training set includes: original effective speech segments and their corresponding healthy speech data. Obtaining the fake speech disorder audio data also includes: The synthesized data is preprocessed, and multi-channel spectral features are extracted from the preprocessed synthesized data to obtain multi-channel spectral features; The multi-channel spectral features are weighted and spliced ​​or weighted and summed to obtain a comprehensive vector. The comprehensive vector is then subjected to layer normalization to obtain the fake articulation disorder audio data. During the training of the model on the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized using the whale optimization algorithm.

2. The speech recognition method for people with articulation disorders according to claim 1, characterized in that, Preprocessing of the speech data with articulation disorders includes: The speech data with articulation disorders is pre-emphasized, framed, and windowed. The processed speech data with articulation disorders is then automatically detected and cropped to retain the effective speech segments.

3. The speech recognition method for people with speech disorders according to claim 2, characterized in that, Pre-emphasis on the speech data of the articulation disorder includes: ； in, This is a weighting parameter used to control the degree of pre-emphasis. This represents the value of the preprocessed speech signal at the nth sampling point. This represents the value of the original speech signal at the nth sampling point.

4. The speech recognition method for people with articulation disorders according to claim 2, characterized in that, Windowing processing of the speech data with articulation disorders includes: ； in, This represents the window function value at the nth sampling point after windowing. Indicates the current sampling point index. This indicates the length of the window function, i.e., the number of sampling points.

5. The speech recognition method for people with articulation disorders according to claim 1, characterized in that, Obtaining the multi-channel spectral features includes: Mel frequency calculations are performed on the preprocessed synthetic data to obtain the multidimensional Mel frequency cepstral coefficients of each frame and their corresponding first-order and second-order difference features. Bark frequency scale extraction was performed on the preprocessed synthetic data to obtain multidimensional perceptual linear prediction coefficients and dynamic features; Fbank extraction is performed on the preprocessed synthetic data to obtain the energy characteristics of the multidimensional filter bank; The multidimensional Mel frequency cepstral coefficients and their corresponding first-order and second-order difference features, as well as the multidimensional sensing linear prediction coefficients and dynamic features and the multidimensional filter bank energy features, are preprocessed respectively. Specifically, the multidimensional Mel frequency cepstral coefficients and their corresponding first-order and second-order difference features and the multidimensional filter bank energy features are logarithmically compressed to reflect the perceptual characteristics of the human ear. All processed features are then normalized to obtain the multi-channel spectral features.

6. A speech recognition system applied to the speech recognition method for speech disorders as described in any one of claims 1-5, characterized in that, include: The effective speech acquisition module is used to collect speech data of speech disorders, preprocess the speech data of speech disorders, and obtain effective speech segments; An effective speech recognition module is used to input the effective speech segment into a speech recognition model for articulation disorders to obtain phoneme-level or character-level recognition results; the speech recognition model for articulation disorders is obtained by training a Conformer model using a first training set, the first training set including: fake articulation disorder audio data; the fake articulation disorder audio data is obtained by speech conversion based on the CycleGAN-VC speech conversion model; Obtaining the fake speech disorder audio data includes: Multiple speech datasets of articulation disorders are acquired, and the speech datasets of articulation disorders are processed. The processing includes: organizing annotation information; the annotation information includes: ID, gender, age, disorder type, and recording conditions. The processed dataset is randomly sampled and segmented, and long audio files are segmented according to predefined time windows to obtain short audio segments; For each short speech segment, silence endpoint detection is performed, and silent segments before and after are removed to obtain valid speech segments, and the sampling rate of the valid speech segments is unified. To acquire healthy speech data, effective speech segments and healthy speech data after uniform sampling rate are input into the CycleGAN-VC speech conversion model to obtain synthesized data. The CycleGAN-VC speech conversion model is trained using a second training set and adversarial loss and cycle consistency loss. The second training set includes: original effective speech segments and their corresponding healthy speech data. Obtaining the fake speech disorder audio data also includes: The synthesized data is preprocessed, and multi-channel spectral features are extracted from the preprocessed synthesized data to obtain multi-channel spectral features; The multi-channel spectral features are weighted and spliced ​​or weighted and summed to obtain a comprehensive vector. The comprehensive vector is then subjected to layer normalization to obtain the fake articulation disorder audio data. During the training of the model on the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized using the whale optimization algorithm.

7. A memory, characterized in that, include: Storage unit, used to store executable instructions; And a processing unit, configured to be connected to a memory to execute executable instructions to perform the method as described in any one of claims 1-5.

Images