Speech recognition method and system for dysarthria group

Abstract

The invention relates to a dysarthria group-oriented speech recognition method and system, and the method comprises the steps: collecting dysarthria speech data, carrying out the preprocessing of the dysarthria speech data, and obtaining an effective speech segment; inputting the effective speech segments into a dysarthria speech recognition model to obtain a phoneme-level or Chinese character-level recognition result; the dysarthria speech recognition model is obtained by training a Conformer model by using a first training set, and the first training set comprises forged dysarthria audio data; the forged dysarthria audio data are obtained through voice conversion based on a CyclGAN-VC voice conversion model; and in the process of training the model by the first training set, model parameters of the Conformer model are adjusted, and the Conformer model is optimized through a whale optimization algorithm. According to the invention, the accuracy and robustness of dysarthria speech recognition can be improved.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition and artificial intelligence technology, and in particular to a speech recognition method and system for a group of people with articulation disorders. Background Art

[0002] Speech recognition technology has made significant progress in recent years and is widely used in human-computer interaction, intelligent voice assistants, voice control and other fields. However, for people with dysarthria, existing speech recognition systems still face many challenges. Dysarthria is a common motor speech disorder, usually caused by neurological diseases such as cerebral palsy, Parkinson's disease, amyotrophic lateral sclerosis, and stroke. These diseases cause patients to have problems such as inaccurate, unfluent, slow speech speed and low voice clarity when pronouncing, making their speech difficult to be accurately recognized by existing speech recognition systems. In addition, most people with dysarthria also have movement disorders, which makes it difficult for them to use traditional interaction methods such as mouse, keyboard and touch interface. Therefore, voice control has become an ideal auxiliary interaction solution.

[0003] While speech recognition technology has made tremendous progress in general speech recognition, recognizing dysarthric speech remains an extremely challenging task. First, dysarthric speech data is extremely limited. Due to the relatively small number of dysarthric individuals and the complex process of collecting and annotating speech data, available dysarthric speech data for training is extremely scarce. This makes it difficult for existing deep learning models to obtain sufficient data for effective training, thus limiting model performance. Second, dysarthric speech is highly variable. Dysarthric speech exhibits significant individual differences and irregularities in pitch, intensity, speaking rate, and pauses. This variability makes it difficult for existing speech recognition models to accurately match and recognize it. Furthermore, dysarthric speech differs significantly from normal speech in its spectro-temporal characteristics, further increasing the difficulty of recognition. Finally, due to the significant differences in speech characteristics between different dysarthric individuals, existing speech recognition models often require adaptive adjustments to improve recognition accuracy and robustness when processing the speech of different dysarthric individuals. However, due to data sparsity and high speech variability, existing adaptive methods have been ineffective in dysarthric speech recognition.

[0004] In summary, existing speech recognition technologies face challenges in processing dysarthric speech, including data sparsity, high speech variability, and difficulty adapting to dysarthric individuals. These issues have limited the development and application of dysarthric speech recognition technology. Therefore, developing a speech recognition method for dysarthric individuals that can effectively address these issues is of great theoretical and practical significance. Summary of the Invention

[0005] In order to solve the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a speech recognition method and system for the articulation-dysphonia group, so as to address the problems of data sparsity, strong voice variability and speaker adaptation in articulation-dysphonia speech recognition, and improve the accuracy and robustness of articulation-dysphonia speech recognition.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A speech recognition method for a group of people with articulation disorders, comprising:

[0008] Collecting dysarthric speech data, preprocessing the dysarthric speech data, and obtaining valid speech segments;

[0009] Inputting the valid speech segment into a dysarthric speech recognition model to obtain a recognition result at the phoneme level or the Chinese character level; the dysarthric speech recognition model is obtained by training a Conformer model using a first training set, the first training set including: forged dysarthric audio data; the forged dysarthric audio data is obtained by performing speech conversion based on a CycleGAN-VC speech conversion model;

[0010] During the model training process of the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized using the whale optimization algorithm.

[0011] Optionally, preprocessing the dysarthric speech data includes:

[0012] The dysarthric speech data is pre-emphasized, framed and windowed, and the processed dysarthric speech data is automatically detected and cropped to retain the valid speech segments.

[0013] Optionally, pre-emphasizing the dysarthric speech data includes:

[0014] y[n]=x[n]-αx[n-1];

[0015] Where α is a weight parameter used to control the degree of pre-emphasis, y[n] represents the value of the preprocessed speech signal at the nth sampling point, and x[n] represents the value of the original speech signal at the nth sampling point.

[0016] Optionally, performing windowing processing on the dysarthric speech data includes:

[0017] w[n]=0.54-0.46cos(2πn / (N-1));

[0018] Where w[n] represents the window function value of the nth sampling point after windowing, n represents the current sampling point index, and N represents the length of the window function, that is, the number of sampling points.

[0019] Optionally, obtaining the forged dysarthria audio data comprises:

[0020] Acquire multiple dysarthric speech data sets, and process the dysarthric speech data sets, wherein the processing includes: arranging annotation information; the annotation information includes: ID, gender, age, disorder type, and recording conditions;

[0021] Randomly extract and split the sorted data set, split the long voice file into predefined time windows to obtain short voice segments;

[0022] Performing silence endpoint detection on each short speech segment, removing the leading and trailing silent segments, obtaining valid speech segments, and unifying the sampling rate of the valid speech segments;

[0023] Healthy speech data is obtained, and the valid speech segments and the healthy speech data after unified sampling rate are respectively input into the CycleGAN-VC speech conversion model to obtain synthesized data; the CycleGAN-VC speech conversion model is trained using a second training set and an adversarial loss and a cycle consistency loss; the second training set includes: the original valid speech segments and their corresponding healthy speech data.

[0024] Optionally, obtaining the forged dysarthria audio data further comprises:

[0025] Preprocessing the synthesized data, and extracting multi-channel spectrum features from the preprocessed synthesized data to obtain multi-channel spectrum features;

[0026] The multi-channel spectral features are weightedly concatenated or weighted summed to obtain a comprehensive vector, and the comprehensive vector is further layer-normalized to obtain the forged dysarthria audio data.

[0027] Optionally, obtaining the multi-channel spectrum characteristics includes:

[0028] Perform Mel frequency calculation on the preprocessed synthetic data to obtain the multidimensional Mel frequency cepstral coefficients of each frame and their corresponding first-order and second-order differential features;

[0029] Perform Bark frequency scale extraction on the preprocessed synthetic data to obtain multidimensional perceptual linear prediction coefficients and dynamic features;

[0030] Perform Fbank extraction on the preprocessed synthetic data to obtain the multidimensional filter bank energy features;

[0031] The multidimensional Mel-frequency cepstral coefficients and their corresponding first-order and second-order differential features, the multidimensional perceptual linear prediction coefficients and dynamic features, and the multidimensional filter group energy features are preprocessed respectively; wherein, the multidimensional Mel-frequency cepstral coefficients and their corresponding first-order and second-order differential features and the multidimensional filter group energy features are logarithmically compressed to reflect the perceptual characteristics of the human ear, and all processed features are dimensionally normalized to obtain the multi-path spectral features.

[0032] To achieve the above-mentioned object, the present invention provides a speech recognition system for people with dysarthria, comprising:

[0033] An effective speech acquisition module, configured to collect dysarthric speech data, pre-process the dysarthric speech data, and obtain effective speech segments;

[0034] an effective speech recognition module, configured to input the effective speech segment into a dysarthric speech recognition model to obtain a recognition result at the phoneme level or the Chinese character level; the dysarthric speech recognition model is obtained by training a Conformer model using a first training set, the first training set including: forged dysarthric audio data; the forged dysarthric audio data is obtained by performing speech conversion based on a CycleGAN-VC speech conversion model;

[0035] During the model training process of the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized using the whale optimization algorithm.

[0036] To achieve the above objectives, the present invention provides a memory, comprising: a storage unit for storing executable instructions; and a processing unit for connecting to the memory to execute the executable instructions.

[0037] The beneficial effects of the present invention are:

[0038] This paper adopts the CycleGAN-VC framework to achieve timbre migration without paired data. Through adversarial training and cycle consistency loss, it generates high-quality synthetic data while keeping pronunciation / intonation unchanged, alleviating the scarcity of Chinese articulation disorder speech data.

[0039] The present invention adopts a three-level enhancement system that constructs spectrum conversion and feature enhancement and further performs variability feature fusion to improve the generalization and robustness of the model.

[0040] The present invention uses an optimized Conformer model. Compared with conventional speech recognition models, it has a stronger ability to capture local features while focusing on global features, and can achieve better recognition performance with a smaller number of model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 This is a flow chart of a speech recognition method for dysarthria patients according to an embodiment of the present invention;

[0043] Figure 2 This is a flow chart of a method for generating forged dysarthria audio data according to an embodiment of the present invention;

[0044] Figure 3 Schematic diagram of the CycleGan-VC network structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] like Figure 1 As shown, this embodiment discloses a speech recognition method for dysarthria groups, including: collecting dysarthria speech data, preprocessing the dysarthria speech data, and obtaining valid speech segments; inputting the valid speech segments into a dysarthria speech recognition model to obtain recognition results at the phoneme level or the Chinese character level; the dysarthria speech recognition model is obtained by training a Conformer model using a first training set, and the first training set includes: forged dysarthria audio data; the forged dysarthria audio data is obtained by performing speech conversion based on a CycleGAN-VC speech conversion model; during the model training process of the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized by the whale optimization algorithm.

[0048] Specifically, this embodiment discloses a speech recognition method for a group of people with dysarthria, including:

[0049] S1: Voice data collection. When the user clicks the "Start Recognition" button, the system automatically starts sampling with a sampling rate of 16kHz and 16-bit PCM. The system records and displays the voice waveform in real time.

[0050] S2: Speech preprocessing: S21: After the acquisition is completed, the system executes the following in sequence: pre-emphasis (α=0.97) to enhance the high-frequency components; frame division (25ms frame length, 10ms frame shift) and adding Hamming window; spectral subtraction based on noise estimation for noise reduction; S22: automatic detection and cropping of silent segments, retaining valid speech segments; S23: mean-variance normalization of the speech signal, preparing for input into the model.

[0051] S3: Model inference: S31: The system feeds the preprocessed frame features into the trained Conformer model; S32: The model outputs phoneme-level or Chinese character-level recognition results based on CTC decoding; S33: The system extracts and records pronunciation deviation features (such as phoneme error type, duration distortion, resonance peak shift, etc.).

[0052] S4: The system determines whether the user has stopped speaking based on the current silent period.

[0053] S5: Display the recognized text results.

[0054] The Conformer model was selected as the preferred base model. Combining the advantages of the Transformer and convolutional neural networks (CNNs), it effectively processes both global dependencies and local features of speech signals. The Conformer model's structure was optimized, including adjusting the parameters of the multi-head self-attention module, convolution module, and feedforward module to improve model performance. Finally, the whale optimization algorithm was used to optimize the CNN's convolution kernel size, number of filters, and number of neurons, enhancing the model's ability to learn the variability characteristics of dysarthric speech.

[0055] Furthermore, preprocessing the dysarthric speech data includes: pre-emphasizing, framing and windowing the dysarthric speech data, automatically detecting and cropping the processed dysarthric speech data, and retaining valid speech segments.

[0056] Furthermore, obtaining forged dysarthria audio data includes: obtaining multiple dysarthria speech data sets, processing the dysarthria speech data sets, the processing including: organizing annotation information; the annotation information includes: ID, gender, age, disorder type and recording conditions; randomly extracting and segmenting the sorted data sets, segmenting the long speech files according to predefined time windows to obtain short speech segments; performing silence endpoint detection on each short speech segment, removing the front and back silent segments, obtaining valid speech segments, and unifying the sampling rates of the valid speech segments; obtaining healthy speech data, inputting the valid speech segments and the healthy speech data after the unified sampling rate into the CycleGAN-VC speech conversion model respectively to obtain synthesized data; the CycleGAN-VC speech conversion model is trained using the second training set and the adversarial loss and the cycle consistency loss; the second training set includes: the original valid speech segments and their corresponding healthy speech data. The synthesized data is preprocessed, and multi-channel spectral feature extraction is performed on the preprocessed synthesized data to obtain multi-channel spectral features; weighted splicing or weighted summation is performed on the multi-channel spectral features to obtain a comprehensive vector, and layer normalization is further performed on the comprehensive vector to obtain forged articulation disorder audio data.

[0057] Acquiring multi-channel spectrum features includes: performing Mel frequency calculation on the pre-processed synthetic data to obtain the multi-dimensional Mel frequency cepstral coefficients of each frame and their corresponding first-order and second-order differential features; performing Bark frequency scale extraction on the pre-processed synthetic data to obtain multi-dimensional perceptual linear prediction coefficients and dynamic features;

[0058] Fbank extraction is performed on the preprocessed synthetic data to obtain the multidimensional filter bank energy features; the multidimensional Mel-frequency cepstral coefficients and their corresponding first-order and second-order differential features, as well as the multidimensional perceptual linear prediction coefficients and dynamic features and the multidimensional filter bank energy features are preprocessed respectively; among them, the multidimensional Mel-frequency cepstral coefficients and their corresponding first-order and second-order differential features and the multidimensional filter bank energy features are logarithmically compressed to reflect the perceptual characteristics of the human ear, and all processed features are dimensionally normalized to obtain multi-channel spectral features.

[0059] Specifically, obtaining the fake dysarthria audio data includes:

[0060] S1: Obtain dysarthric speech datasets. Download or obtain multiple public dysarthric speech datasets (such as UASpeech, Laryngograph, CDSD, etc.).

[0061] S2: Preprocess the dataset: S21: Organize the annotation information, including speaker ID, gender, age, impairment type, and recording conditions. S22: Randomly extract and segment the original speech files, breaking long speech files into short speech segments according to predefined time windows (e.g., 5s). S23: Perform silence endpoint detection on each segment, removing the leading and trailing silences to obtain the cropped valid speech segments. S24: Unify the cropped speech to a uniform sampling rate of 16kHz, 16-bit PCM format.

[0062] S3: If Figure 2 As shown, data generation based on CycleGAN-VC: S31: Construct a CycleGAN-VC speech conversion model, including two generators G(H→D) (healthy→disabled) and G(D→H) (disabled→healthy), and two discriminators D(D) and D(H), trained jointly using adversarial loss and cycle consistency loss. S32: The speech of a healthy dysarthric patient and the dysarthric speech obtained in step S2 are input into G(H→D) and G(D→H), respectively, to generate synthetic data similar to the real dysarthric speech. At the same time, the original real data is retained to expand the training set.

[0063] like Figure 3 As shown in Figure 1, the CycleGAN-VC model is a deep learning model for speech conversion that can convert source speech into target speech while preserving important features such as the speaker's timbre. The core structure of the model consists of two generators and two discriminators.

[0064] The generator is the core component of CycleGAN-VC, which is used to realize speech conversion. There are two generators in the model:

[0065] The first generator (G(X→Y)): converts the input source speech X into the target speech Y.

[0066] The second generator (G(Y→X)): converts the input target speech Y back to the source speech X.

[0067] The generator's structure is based on a 1D convolutional neural network (CNN), which effectively captures relationships between features while preserving the temporal structure of speech. The generator's architecture consists of an encoder, downsampling layers, residual layers, and upsampling layers. To simulate the forget gate mechanism similar to LSTM, the generator employs a gated convolutional neural network (Gated CNN) and uses a gated linear unit (GLU) as the activation function, allowing for selective information propagation based on the state of the previous layer. Furthermore, the generator utilizes instance normalization to stabilize the training process and improve model performance.

[0068] The role of the discriminator is to evaluate whether the generated speech is realistic. CycleGAN-VC contains two discriminators:

[0069] The first discriminator (D(Y)): is used to determine whether the target speech Y is real.

[0070] The second discriminator (D(X)): is used to determine whether the source speech X is real.

[0071] The discriminator typically uses a 2D CNN structure and performs discrimination based on 2D spectral textures. To better capture local and global information, the discriminator adopts a PatchGAN structure, outputting a multidimensional matrix that takes into account differences in global receptive field information.

[0072] The interaction between the generator and the discriminator is as follows: The source speech X is converted to the target speech Y by the first generator (G(X→Y)). The target speech Y is converted back to the source speech X by the second generator (G(Y→X)). The source speech X is discriminated by the second discriminator (D(X)). The target speech Y is discriminated by the first discriminator (D(Y)). With this structure, CycleGAN-VC can achieve efficient conversion from source speech to target speech while preserving key speech features, such as the speaker's timbre. This design not only improves the naturalness of the speech conversion but also enhances the model's robustness and generalization capabilities.

[0073] S4: Speech signal pre-emphasis, framing and windowing:

[0074] S41: Pre-emphasis - applies a first-order high-pass filter to each speech segment;

[0075] y[n]=x[n]-αx[n-1],α=0.97;

[0076] S42: Framing - the pre-emphasized signal is divided into frames with a 25ms window and a 10ms step size;

[0077] S43: Windowing - applies a Hamming window to each frame to reduce signal truncation effects;

[0078] w[n]=0.54-0.46cos(2πn / (N-1));

[0079] Where w[n] represents the window function value of the nth sampling point after windowing, n represents the current sampling point index (from 0 to N-1), and N represents the length of the window function, that is, the number of sampling points.

[0080] S5: Multi-channel spectral feature extraction: S51: MFCC - calculate the 13-dimensional Mel-frequency cepstral coefficients and their first-order and second-order differences of each frame, a total of 39 dimensions; S52: PLP - extract 12-dimensional perceptual linear prediction coefficients and dynamic features based on the Bark frequency scale; S53: Fbank - extract 40-dimensional filter bank energy; S54: perform logarithmic compression and dimension normalization on the above features respectively.

[0081] S6: Feature fusion and normalization: S61: Fuse MFCC, PLP, and Fbank features into a comprehensive vector by weighted splicing or weighted summation (e.g., the dimension after splicing is 39+13+40=92); S62: Perform layer normalization on the fused vector to ensure consistent distribution of each dimension, which is conducive to deep network learning.

[0082] S7: Obtain the final fake dysarthria audio data.

[0083] The Conformer model was selected as the preferred base model. Combining the advantages of the Transformer and convolutional neural networks (CNNs), it effectively processes both global dependencies and local features of speech signals. The Conformer model's structure was optimized, including adjusting the parameters of the multi-head self-attention module, convolution module, and feedforward module to improve model performance. Finally, the whale optimization algorithm was used to optimize the CNN's convolution kernel size, number of filters, and number of neurons, enhancing the model's ability to learn the variability characteristics of dysarthric speech.

[0084] This embodiment also provides a speech recognition system for the articulation disorder group, including: an effective speech acquisition module, used to collect articulation disorder speech data, pre-process the articulation disorder speech data, and obtain effective speech segments; an effective speech recognition module, used to input the effective speech segments into the articulation disorder speech recognition model, and obtain recognition results at the phoneme level or the Chinese character level; the articulation disorder speech recognition model is obtained by training a Conformer model using a first training set, and the first training set includes: forged articulation disorder audio data; the forged articulation disorder audio data is obtained by speech conversion based on the CycleGAN-VC speech conversion model; during the model training process of the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized using the whale optimization algorithm.

[0085] Specifically, this embodiment further provides a speech recognition system for people with dysarthria, including:

[0086] The first processing module is used to obtain a public Chinese dysarthria speech dataset;

[0087] The second processing module is used to generate fake dysarthric speech data using the CycleGAN-VC model based on the Chinese dysarthric speech dataset;

[0088] a third processing module, configured to perform pre-emphasis, framing, and windowing processing on the real voice data and the forged voice data to reduce redundant information in the voice signals;

[0089] The fourth processing module is used to extract multiple speech features, including Mel-frequency cepstral coefficients (MFCCs), perceptual linear prediction coefficients (PLPs), and filter banks (Fbanks), to fully capture the spectral information of the speech and perform spectral enhancement processing on the extracted features;

[0090] The fifth processing module is used to input the extracted speech features into the improved Conformer model for training, using the CTC loss function as the training target to obtain the trained model weights;

[0091] The sixth processing module is used to extract speech variability features as auxiliary features and adjust the acoustic model in combination with dysarthria adaptation technology to improve the model's adaptability and recognition performance for speech with different degrees of dysarthria.

[0092] The seventh processing module is used to verify the trained model weights on multiple public datasets;

[0093] The eighth processing module is used to perform inference on the trained model to realize the recognition of dysarthric speech.

[0094] This embodiment further provides a memory, comprising: a storage unit for storing executable instructions; and a processing unit for connecting to the memory to execute the executable instructions.

[0095] Specifically, this embodiment further provides a memory, including: a memory and a processor, wherein a computer program executed by the processor is stored in the memory, and the computer program executes the Chinese dysarthria speech recognition method when executed by the processor.

[0096] This embodiment further provides a storage medium having a computer program stored thereon. The computer program executes the Chinese dysarthric speech recognition method when running.

[0097] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A speech recognition method for dysarthria groups, characterized in that: include: Collecting dysarthric speech data, preprocessing the dysarthric speech data, and obtaining valid speech segments; Inputting the valid speech segment into a dysarthric speech recognition model to obtain a recognition result at the phoneme level or the Chinese character level; The dysarthric speech recognition model is obtained by training a Conformer model using a first training set, wherein the first training set includes: forged dysarthric audio data; the forged dysarthric audio data is obtained by voice conversion based on a CycleGAN-VC voice conversion model; During the model training process of the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized using the whale optimization algorithm.

2. The speech recognition method for dysarthria-related groups according to claim 1, characterized in that: Preprocessing the dysarthric speech data includes: The dysarthric speech data is pre-emphasized, framed and windowed, and the processed dysarthric speech data is automatically detected and cropped to retain the valid speech segments.

3. The speech recognition method for dysarthria-oriented people according to claim 2, characterized in that: Pre-emphasizing the dysarthric speech data includes: y[n]=x[n]-αx[n-1]; Where α is a weight parameter used to control the degree of pre-emphasis, y[n] represents the value of the preprocessed speech signal at the nth sampling point, and x[n] represents the value of the original speech signal at the nth sampling point.

4. The speech recognition method for dysarthria-related groups according to claim 2, characterized in that: Performing windowing processing on the dysarthric speech data includes: w[n]=0.54-0.46cos(2πn / (N-1)); Where w[n] represents the window function value of the nth sampling point after windowing, n represents the current sampling point index, and N represents the length of the window function, that is, the number of sampling points.

5. The speech recognition method for dysarthria-related groups according to claim 1, characterized in that: Obtaining the forged dysarthria audio data comprises: Acquire multiple dysarthric speech data sets, and process the dysarthric speech data sets, wherein the processing includes: arranging annotation information; the annotation information includes: ID, gender, age, disorder type, and recording conditions; Randomly extract and split the sorted data set, split the long voice file into predefined time windows to obtain short voice segments; Performing silence endpoint detection on each short speech segment, removing the leading and trailing silent segments, obtaining valid speech segments, and unifying the sampling rate of the valid speech segments; Healthy speech data is obtained, and the valid speech segments and the healthy speech data after unified sampling rate are respectively input into the CycleGAN-VC speech conversion model to obtain synthesized data; the CycleGAN-VC speech conversion model is trained using a second training set and an adversarial loss and a cycle consistency loss; the second training set includes: the original valid speech segments and their corresponding healthy speech data.

6. The speech recognition method for dysarthria-related groups according to claim 5, characterized in that: Obtaining the forged dysarthria audio data further includes: Preprocessing the synthesized data, and extracting multi-channel spectrum features from the preprocessed synthesized data to obtain multi-channel spectrum features; The multi-channel spectral features are weightedly concatenated or weighted summed to obtain a comprehensive vector, and the comprehensive vector is further layer-normalized to obtain the forged dysarthria audio data.

7. The speech recognition method for dysarthria-oriented people according to claim 6, characterized in that: Acquiring the multi-channel spectrum characteristics includes: Perform Mel frequency calculation on the preprocessed synthetic data to obtain the multidimensional Mel frequency cepstral coefficients of each frame and their corresponding first-order and second-order differential features; Perform Bark frequency scale extraction on the preprocessed synthetic data to obtain multidimensional perceptual linear prediction coefficients and dynamic features; Perform Fbank extraction on the preprocessed synthetic data to obtain the multidimensional filter bank energy features; The multidimensional Mel-frequency cepstral coefficients and their corresponding first-order and second-order differential features, the multidimensional perceptual linear prediction coefficients and dynamic features, and the multidimensional filter group energy features are preprocessed respectively; wherein, the multidimensional Mel-frequency cepstral coefficients and their corresponding first-order and second-order differential features and the multidimensional filter group energy features are logarithmically compressed to reflect the perceptual characteristics of the human ear, and all processed features are dimensionally normalized to obtain the multi-path spectral features.

8. A speech recognition system applied to the speech recognition method for dysarthria-oriented groups according to any one of claims 1 to 7, characterized in that: include: An effective speech acquisition module, configured to collect dysarthric speech data, pre-process the dysarthric speech data, and obtain effective speech segments; an effective speech recognition module, configured to input the effective speech segment into a dysarthric speech recognition model to obtain a recognition result at the phoneme level or the Chinese character level; the dysarthric speech recognition model is obtained by training a Conformer model using a first training set, the first training set including: forged dysarthric audio data; the forged dysarthric audio data is obtained by performing speech conversion based on a CycleGAN-VC speech conversion model; During the model training process of the first training set, the model parameters of the Conformer model are adjusted, and the Conformer model is optimized using the whale optimization algorithm.

9. A memory, characterized in that: include: a storage unit for storing executable instructions; and a processing unit, configured to be connected to the memory to execute the executable instructions to complete the method according to any one of claims 1 to 7.

Images