A speech recognition method for Parkinson's patients

Abstract

The present invention discloses a speech recognition method for Parkinson's patients, and belongs to the field of speech recognition technology. The present invention first removes the tremor frequency of the speech signal of the Parkinson's patient to obtain a detremorized speech spectrum set; secondly, based on the difference of the detremorized speech spectrum at adjacent moments, the spectrum set is divided into multiple pronunciation subsets; then, the peak frequency, overtone centroid value and overtone deviation value are extracted in each pronunciation subset to construct a characteristic pronunciation vector; finally, a multi-layer pronunciation vector processing model is used to finally parse the semantic content of the Parkinson's patient. The present invention significantly improves the accuracy of speech recognition of Parkinson's patients.

Description

Technical Field

[0001] The present invention relates to the technical field of speech recognition, and in particular to a speech recognition method for Parkinson's disease patients. Background Art

[0002] Parkinson's disease is a common neurodegenerative disease that mainly affects the patient's motor ability and speech expression. As the disease progresses, the patient's voice characteristics will change significantly, including the pitch, volume, and clarity of the voice. These changes not only affect the patient's ability to communicate, but also have a negative impact on their mental health and quality of life. Therefore, developing effective speech recognition technology to help Parkinson's patients communicate has become an important research direction.

[0003] Existing technologies usually use traditional spectrum analysis and acoustic feature extraction methods, such as Mel-Frequency Cepstral Coefficients (MFCC), Linear Predictive Coding (LPC), etc. However, these methods have significant limitations when processing the speech of Parkinson's patients. They cannot separate the same pronunciation signal segments, and feature extraction must be performed on the entire speech signal, which greatly reduces the accuracy and effectiveness of signal processing and leads to low speech recognition accuracy. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a speech recognition method for Parkinson's disease patients, which solves the problem of low speech recognition accuracy in the prior art.

[0005] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a speech recognition method for Parkinson's patients, comprising the following steps:

[0006] S1. Eliminate the tremor frequency from the speech signal of Parkinson's patients to obtain a set of detremored speech spectra;

[0007] S2. In the detremorized speech spectrum set, the detremorized speech spectrum set is divided according to the difference between the detremorized speech spectra at adjacent moments to obtain detremorized speech spectrum subsets of multiple pronunciations;

[0008] S3, extracting the peak frequency, the overtone centroid value and the overtone deviation value from the frequency in the detremorized speech spectrum subset of each pronunciation, and constructing a pronunciation vector;

[0009] S4. Use a multi-layer pronunciation vector processing model to process multiple pronunciation vectors to obtain the semantics of Parkinson's patients.

[0010] Furthermore, S1 comprises the following sub-steps:

[0011] S11, slide the sliding window on the voice signal, advancing one time length each time;

[0012] S12, at each sliding, performing Fourier transform on the signal segment under the sliding window to obtain the speech spectrum at each moment;

[0013] S13, in the speech spectrum at each moment, calculating the average value of each amplitude, screening out the frequency values ​​corresponding to the amplitudes lower than the average value, and forming a candidate tremor frequency set;

[0014] S14, taking the intersection of the candidate tremor frequency sets at each moment to obtain the tremor frequency;

[0015] S15. Eliminate the tremor frequency from the speech spectrum at each moment to obtain a detremored speech spectrum at each moment, and arrange the detremored speech spectrum at each moment in chronological order to form a detremored speech spectrum set.

[0016] Further, S2 includes the following sub-steps:

[0017] S21, in the de-trembled speech spectrum set, taking the union of the frequencies in the de-trembled speech spectrum at time t+1 and the frequencies in the de-trembled speech spectrum at time t;

[0018] S22, removing the frequencies in the detrembled speech spectrum at time t from the union, and obtaining the newly added frequencies at time t+1;

[0019] S23, calculating the frequency change value according to the newly added frequency at time t+1;

[0020] S24, when the frequency change value is greater than the frequency change threshold, marking time t+1 as a pronunciation transition point;

[0021] S25. Divide the detremorized speech spectrum set according to the positions of the pronunciation conversion points to obtain detremorized speech spectrum subsets of multiple pronunciations.

[0022] Furthermore, the formula for calculating the frequency change value in S23 is: , , where μ t+1 is the frequency change value at time t+1, f t+1,z,i is the i-th new frequency at time t+1, f t,s,j is the jth frequency in the detremized speech spectrum at time t, | | is the absolute value, M is the number of frequencies in the detremized speech spectrum at time t, and min is the number of M frequencies. The minimum value in , i and j are positive integers, N is the number of new frequencies added at time t+1, d t+1,z,i It is the minimum difference between the i-th newly added frequency at time t+1 and the frequency in the detrembled speech spectrum at time t.

[0023] Furthermore, S3 includes the following sub-steps:

[0024] S31, in each detremorized speech spectrum of a pronounced detremorized speech spectrum subset, find the frequency corresponding to the maximum amplitude as the main frequency, and average each main frequency to obtain the peak frequency;

[0025] S32. In a subset of detremorized speech spectra of a pronunciation, taking other frequencies except the main frequency in each detremorized speech spectrum as overtone frequencies;

[0026] S33, calculating the overtone centroid value of the overtone frequency;

[0027] S34, calculating the overtone deviation value according to the difference between each overtone frequency and the peak frequency;

[0028] S35. Use the peak frequency, overtone centroid value and overtone deviation value as elements to construct a pronunciation vector.

[0029] Furthermore, the calculation formula of the overtone centroid value in S33 is: , where f o,c is the overtone center of mass value, f o,i is the i-th harmonic frequency in the spectrum subset of the pronounced detremized speech, A o,i is the i-th overtone frequency f o,i , L is the number of overtone frequencies in the spectrum subset of the pronounced detrembled speech, and i is a positive integer.

[0030] Furthermore, the formula for calculating the harmonic deviation value in S34 is:

[0031] , where f o,d is the harmonic deviation value, f o,i is the i-th harmonic frequency in the spectrum subset of the pronounced detremized speech, i is a positive integer, and f peak is the peak frequency, and L is the number of overtone frequencies in the subset of the detremized speech spectrum of the articulated speech.

[0032] Further, the multi-layer pronunciation vector processing model in S4 includes: a plurality of first pronunciation processing layers, a plurality of second pronunciation processing layers, a first BiLSTM layer, a second BiLSTM layer, a feature integration layer and a CRF unit;

[0033] The input end of each first pronunciation processing layer is used to input a pronunciation vector; the input end of each second pronunciation processing layer is used to input a pronunciation vector; the input end of the first BiLSTM layer is respectively connected to the output ends of multiple first pronunciation processing layers; the input end of the second BiLSTM layer is respectively connected to the output ends of multiple second pronunciation processing layers; the input end of the feature integration layer is respectively connected to the output end of the first BiLSTM layer and the output end of the second BiLSTM layer, and its output end is connected to the input end of the CRF unit; the output end of the CRF unit serves as the output end of the multi-layer pronunciation vector processing model.

[0034] Furthermore, the expression of the first pronunciation processing layer is: , where y1 is the output of the first pronunciation processing layer, Tanh is the hyperbolic tangent activation function, and x n is the nth element in the pronunciation vector, ω 1,n is the nth weight in the first pronunciation processing layer, b 1,n is the nth bias in the first pronunciation processing layer, and the value of n is a positive integer ranging from 1 to 3;

[0035] The expression of the second pronunciation processing layer is: , where y2 is the output of the second pronunciation processing layer, Sigmoid is the S-type activation function, ω 2,n is the nth weight in the second pronunciation processing layer, b 2,n is the nth bias in the second pronunciation processing layer.

[0036] Furthermore, the expression of the feature integration layer is: , where H m is the mth integrated feature output by the feature integration layer, h 1,m is the mth feature output by the first BiLSTM layer, h 2,m is the mth feature output by the second BiLSTM layer, ω 1,m is the mth first weight in the feature integration layer, ω 2,m is the mth second weight in the feature integration layer.

[0037] The beneficial effects of the present invention are:

[0038] 1. The present invention first removes the tremor frequency from the speech signal to avoid the tremor frequency affecting speech recognition and improve the accuracy of speech recognition.

[0039] 2. In the detremorized speech spectrum set, the present invention finds the segmentation point of each pronunciation according to the difference between the detremorized speech spectra at adjacent moments, and divides the detremorized speech spectrum set, which can more accurately distinguish different pronunciation segments and avoid the problem of inaccurate feature extraction caused by processing the entire speech signal.

[0040] 3. The present invention reflects the speech characteristics of each utterance by extracting the peak frequency, the overtone centroid value and the overtone deviation value from the detremorized speech spectrum subset of each utterance.

[0041] 4. The present invention adopts a multi-layer pronunciation vector processing model to process multiple pronunciation vectors, integrates the pronunciation vectors corresponding to the speech signal, recognizes the semantics of Parkinson's patients, and improves the accuracy of speech recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flow chart of a method for speech recognition of Parkinson's disease patients.

[0043] Figure 2 Schematic diagram of the structure of the multi-layer pronunciation vector processing model. DETAILED DESCRIPTION

[0044] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0045] like Figure 1 As shown, a speech recognition method for Parkinson's patients comprises the following steps:

[0046] S1. Eliminate the tremor frequency from the speech signal of Parkinson's patients to obtain a set of detremored speech spectra;

[0047] S2. In the detremorized speech spectrum set, the detremorized speech spectrum set is divided according to the difference between the detremorized speech spectra at adjacent moments to obtain detremorized speech spectrum subsets of multiple pronunciations;

[0048] S3, extracting the peak frequency, the overtone centroid value and the overtone deviation value from the frequency in the detremorized speech spectrum subset of each pronunciation, and constructing a pronunciation vector;

[0049] S4. Use a multi-layer pronunciation vector processing model to process multiple pronunciation vectors to obtain the semantics of Parkinson's patients.

[0050] In this embodiment, S1 includes the following sub-steps:

[0051] S11, slide the sliding window on the voice signal, advancing one time length each time;

[0052] S12, at each sliding, performing Fourier transform on the signal segment under the sliding window to obtain the speech spectrum at each moment;

[0053] S13, in the speech spectrum at each moment, calculating the average value of each amplitude, screening out the frequency values ​​corresponding to the amplitudes lower than the average value, and forming a candidate tremor frequency set;

[0054] S14, taking the intersection of the candidate tremor frequency sets at each moment to obtain the tremor frequency;

[0055] S15. Eliminate the tremor frequency from the speech spectrum at each moment to obtain a detremored speech spectrum at each moment, and arrange the detremored speech spectrum at each moment in chronological order to form a detremored speech spectrum set.

[0056] The present invention uses a sliding window to slide on the voice signal, advances one moment each time, and performs Fourier transform on the signal segment under the sliding window after each sliding, so as to obtain the voice spectrum at each moment, realize dynamic and real-time spectrum analysis of the voice signal, and can capture subtle changes in the voice signal. Since the motor control of the nervous system of Parkinson's patients is impaired, the muscle control ability is reduced, and the coordination of the vocal cords and pronunciation muscles is weakened, Parkinson's patients will have voice tremor when speaking. Tremor is manifested as low-amplitude, continuous abnormal frequency, which is often covered in normal voice signals. Therefore, the present invention screens out the frequency corresponding to the low amplitude in the voice spectrum at each moment, takes the intersection of the candidate tremor frequency set at each moment, realizes the extraction of the frequency that has always existed in the voice signal, and the frequency is distributed in the entire stage of the voice signal, effectively extracts the frequency that persists in the entire voice signal stage, and can accurately locate the stable tremor frequency in the voice of Parkinson's patients, rather than sporadic noise.

[0057] In this embodiment, the sliding window is a one-dimensional window, and its length is K, where K is a positive integer. It can cover K moments each time, that is, a signal segment with a length of K can be obtained.

[0058] In this embodiment, the fundamental frequency range of human speech is usually 50-250Hz, and the frequency of voice tremor of Parkinson's patients is generally 4-7Hz. The sliding window can be set to 20-40 milliseconds, which can cover the fundamental frequency period (usually 10-20 milliseconds) and the typical frequency range of voice tremor of Parkinson's patients, ensuring the balance between time resolution and frequency resolution. For example, when the sampling rate is 16kHz, K can be 32 milliseconds, and the corresponding number of sampling points is approximately: 16kHz×0.032s=512 sampling points. The range of the length of one forward moment is 5-20 milliseconds. For example, when the sampling rate is 16kHz, 15 milliseconds is taken as the length of one moment, and the corresponding number of sampling points is: 16kHz×0.015=240 sampling points.

[0059] In this embodiment, the length of the sliding window and a moment can be adjusted. When the sliding window is too small, the frequency resolution is low and the tremor cannot be accurately captured. When the sliding window is too large, the time resolution decreases and the detailed features are lost. When the length of a moment is too small, the amount of calculation is large, which increases the burden on the system. When it is too large, important speech features may be lost.

[0060] In this embodiment, S2 includes the following sub-steps:

[0061] S21, in the de-trembled speech spectrum set, taking the union of the frequencies in the de-trembled speech spectrum at time t+1 and the frequencies in the de-trembled speech spectrum at time t;

[0062] S22, removing the frequencies in the detrembled speech spectrum at time t from the union, and obtaining the newly added frequencies at time t+1;

[0063] S23, calculating the frequency change value according to the newly added frequency at time t+1;

[0064] S24, when the frequency change value is greater than the frequency change threshold, marking time t+1 as a pronunciation transition point;

[0065] S25. Divide the detremorized speech spectrum set according to the positions of the pronunciation conversion points to obtain detremorized speech spectrum subsets of multiple pronunciations.

[0066] The present invention marks the pronunciation conversion points in the detremorized speech spectrum set, and divides the spectrum according to the pronunciation conversion points. Each segment of the detremorized speech spectrum constitutes a subset.

[0067] Speech is a dynamically changing signal. There is a conversion of frequency characteristics between different phonemes and syllables. The pronunciation process is a continuous but discrete language expression. Therefore, the present invention compares the frequencies in the detremorized speech spectrum at adjacent moments, finds the newly added frequency in the next moment, and calculates the frequency change value based on the newly added frequency at time t+1, which can reflect the change in pronunciation. If the frequency change value is greater than the frequency change threshold, it means that there is a change in pronunciation between time t and time t+1. Pronunciation involves the coordination of organs such as the vocal cords, oral cavity, and tongue. The production of each phoneme requires a specific organ position and muscle state, and this conversion will cause a significant change in frequency. The present invention marks the pronunciation conversion point so that each pronunciation has a subset of the detremorized speech spectrum.

[0068] In this embodiment, if a detrembled speech spectrum subset has a small number of detrembled speech spectra, it can be discarded. For example, there is only one detrembled speech spectrum, that is, the frequency difference between the detrembled speech spectrum at this location and the frequencies at the left and right moments is large. Therefore, there is no continuous pronunciation in this subset and no valuable information can be provided. By discarding these unstable spectra, the overall quality of the data set can be improved.

[0069] Speech signal is a continuous process, and pronunciation is usually smooth with natural transitions between phonemes. The characteristics of a pronunciation are similar at multiple moments. During normal pronunciation, some frequency features remain relatively stable for a short period of time. The spectrum of detremized speech at multiple moments can reflect this stability, forming a subset containing multiple moments.

[0070] When dividing the detrembled speech spectrum set, the present invention, when the frequency change value is greater than the frequency change threshold, time t belongs to the previous detrembled speech spectrum subset, and time t+1 belongs to the next detrembled speech spectrum subset.

[0071] In this embodiment, the formula for calculating the frequency change value in S23 is: , , where μ t+1 is the frequency change value at time t+1, f t+1,z,i is the i-th new frequency at time t+1, f t,s,j is the jth frequency in the detremized speech spectrum at time t, | | is the absolute value, M is the number of frequencies in the detremized speech spectrum at time t, and min is the number of M frequencies. The minimum value in , i and j are positive integers, N is the number of new frequencies added at time t+1, d t+1,z,i It is the minimum difference between the i-th newly added frequency at time t+1 and the frequency in the detrembled speech spectrum at time t.

[0072] The present invention calculates the difference between each newly added frequency and each frequency in the detrembled speech spectrum at time t, finds out the minimum difference, determines the frequency offset, and then combines the number of newly added frequencies to obtain the frequency change value. When the frequency change value is larger, the pronunciation at time t+1 has changed significantly, so the time t+1 may be the pronunciation of the next word.

[0073] In the present invention, the frequency change threshold is a threshold set for the frequency change value. In this embodiment, the frequency change threshold is set to 0.5.

[0074] In this embodiment, S3 includes the following sub-steps:

[0075] S31, in each detremorized speech spectrum of a pronounced detremorized speech spectrum subset, find the frequency corresponding to the maximum amplitude as the main frequency, and average each main frequency to obtain the peak frequency;

[0076] S32. In a subset of detremorized speech spectra of a pronunciation, taking other frequencies except the main frequency in each detremorized speech spectrum as overtone frequencies;

[0077] S33, calculating the overtone centroid value of the overtone frequency;

[0078] S34, calculating the overtone deviation value according to the difference between each overtone frequency and the peak frequency;

[0079] S35. Use the peak frequency, overtone centroid value and overtone deviation value as elements to construct a pronunciation vector.

[0080] In the sound signal, the frequency point with the highest amplitude corresponds to the fundamental frequency, which is the main feature of the sound. The fundamental frequency determines the pitch of the sound and is the key factor in identifying the pitch and timbre. In each detremized speech spectrum, there is a main frequency corresponding to the maximum amplitude. Therefore, the average of the main frequencies of the pronunciation segment is taken to obtain the peak frequency, which reflects the main feature of the pronunciation segment.

[0081] In this embodiment, the calculation formula of the overtone centroid value in S33 is: , where f o,c is the overtone center of mass value, f o,i is the i-th harmonic frequency in the spectrum subset of the pronounced detremized speech, A o,i is the i-th overtone frequency f o,i , L is the number of overtone frequencies in the spectrum subset of the pronounced detrembled speech, and i is a positive integer.

[0082] In this embodiment, the formula for calculating the harmonic deviation value in S34 is: , where f o,d is the harmonic deviation value, f o,i is the i-th harmonic frequency in the spectrum subset of the pronounced detremized speech, i is a positive integer, and f peak is the peak frequency, and L is the number of overtone frequencies in the subset of the detremized speech spectrum of the articulated speech.

[0083] The present invention forms a subset of detremorized speech spectra belonging to the same pronunciation, finds out the frequency corresponding to the maximum amplitude in each detremorized speech spectrum in the subset as the main frequency, takes the average of each main frequency to obtain the peak frequency, and the main frequency reflects the basic pronunciation characteristics. Therefore, extracting the peak frequency of this section of pronunciation can reflect the main characteristics of the pronunciation, and the overtones reflect the subtle state of the pronunciation organs. The overtone centroid value of the overtone frequency is extracted to reflect the main frequency range of the overtone, and then the overtone deviation value is calculated to reflect the gap between each overtone frequency and the peak frequency, so as to further explore the distribution of the overtone frequency and the peak frequency during this pronunciation.

[0084] like Figure 2 As shown, the multi-layer pronunciation vector processing model in S4 includes: multiple first pronunciation processing layers, multiple second pronunciation processing layers, a first BiLSTM layer, a second BiLSTM layer, a feature integration layer and a CRF unit;

[0085] The input end of each first pronunciation processing layer is used to input a pronunciation vector; the input end of each second pronunciation processing layer is used to input a pronunciation vector; the input end of the first BiLSTM layer is respectively connected to the output ends of multiple first pronunciation processing layers; the input end of the second BiLSTM layer is respectively connected to the output ends of multiple second pronunciation processing layers; the input end of the feature integration layer is respectively connected to the output end of the first BiLSTM layer and the output end of the second BiLSTM layer, and its output end is connected to the input end of the CRF unit; the output end of the CRF unit serves as the output end of the multi-layer pronunciation vector processing model.

[0086] In the present invention, the number of the first pronunciation processing layer and the number of the second pronunciation processing layer are both equal to the number of pronunciation vectors.

[0087] In this embodiment, the expression of the first pronunciation processing layer is: , where y1 is the output of the first pronunciation processing layer, Tanh is the hyperbolic tangent activation function, and x n is the nth element in the pronunciation vector, ω 1,n is the nth weight in the first pronunciation processing layer, b 1,n is the nth bias in the first pronunciation processing layer, and the value of n ranges from 1 to 3.

[0088] In this embodiment, the expression of the second pronunciation processing layer is: , where y2 is the output of the second pronunciation processing layer, Sigmoid is the S-type activation function, ω 2,n is the nth weight in the second pronunciation processing layer, b 2,n is the nth bias in the second pronunciation processing layer.

[0089] In this embodiment, the expression of the feature integration layer is: , where H m is the mth integrated feature output by the feature integration layer, h 1,m is the mth feature output by the first BiLSTM layer, h 2,m is the mth feature output by the second BiLSTM layer, ω 1,m is the mth first weight in the feature integration layer, ω 2,m is the mth second weight in the feature integration layer.

[0090] The present invention arranges two pronunciation processing layers to process the same pronunciation vector, extracts features of speech signals from different levels, enhances the model's ability to understand complex speech signals, obtains two pronunciation features of the same pronunciation vector, and then uses a first BiLSTM layer to process one pronunciation feature of each pronunciation vector, and uses a second BiLSTM layer to process another pronunciation feature of each pronunciation vector, so that the BiLSTM layer can better capture contextual relationships, and then uses a feature integration layer to integrate features of the first BiLSTM layer and the second BiLSTM layer, so as to achieve dual-channel feature fusion and improve the accuracy of semantic recognition.

[0091] In this embodiment, the CRF unit may also be replaced by a Softmax layer or a CTC layer.

[0092] In the present invention, all pronunciation vectors corresponding to the speech signal are converted into readable text through a multi-layer pronunciation vector processing model.

[0093] In this embodiment, the weights and biases in the multi-layer pronunciation vector processing model are trained using the existing gradient descent method.

[0094] The present invention first removes the tremor frequency from the speech signal to avoid the tremor frequency affecting speech recognition and improve the accuracy of speech recognition.

[0095] The present invention finds the segmentation point of each pronunciation in the detremorized speech spectrum set according to the difference between the detremorized speech spectra at adjacent moments, divides the detremorized speech spectrum set, and can more accurately distinguish different pronunciation segments, thus avoiding the problem of inaccurate feature extraction caused by processing the entire speech signal.

[0096] The present invention reflects the speech characteristics of each utterance by extracting the peak frequency, the overtone centroid value and the overtone deviation value from the detrembled speech spectrum subset of each utterance.

[0097] The present invention adopts a multi-layer pronunciation vector processing model to process multiple pronunciation vectors, integrates the pronunciation vectors corresponding to the speech signal, recognizes the semantics of Parkinson's patients, and improves the accuracy of speech recognition.

[0098] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A speech recognition method for Parkinson's disease patients, characterized in that: The following steps are involved: S1. Eliminate the tremor frequency from the speech signal of Parkinson's patients to obtain a set of detremored speech spectra; S2. In the detremorized speech spectrum set, the detremorized speech spectrum set is divided according to the difference between the detremorized speech spectra at adjacent moments to obtain detremorized speech spectrum subsets of multiple pronunciations; S3, extracting the peak frequency, the overtone centroid value and the overtone deviation value from the frequency in the detremorized speech spectrum subset of each pronunciation, and constructing a pronunciation vector; S4. Use a multi-layer pronunciation vector processing model to process multiple pronunciation vectors to obtain the semantics of Parkinson's patients.

2. The speech recognition method for Parkinson's disease patients according to claim 1, characterized in that: The S1 comprises the following sub-steps: S11, slide the sliding window on the voice signal, advancing one time length each time; S12, at each sliding, performing Fourier transform on the signal segment under the sliding window to obtain the speech spectrum at each moment; S13, in the speech spectrum at each moment, calculating the average value of each amplitude, screening out the frequency values ​​corresponding to the amplitudes lower than the average value, and forming a candidate tremor frequency set; S14, taking the intersection of the candidate tremor frequency sets at each moment to obtain the tremor frequency; S15. Eliminate the tremor frequency from the speech spectrum at each moment to obtain a detremored speech spectrum at each moment, and arrange the detremored speech spectrum at each moment in chronological order to form a detremored speech spectrum set.

3. The speech recognition method for Parkinson's disease patients according to claim 1, characterized in that: The S2 comprises the following sub-steps: S21, in the de-trembled speech spectrum set, taking the union of the frequencies in the de-trembled speech spectrum at time t+1 and the frequencies in the de-trembled speech spectrum at time t; S22, removing the frequencies in the detrembled speech spectrum at time t from the union, and obtaining the newly added frequencies at time t+1; S23, calculating the frequency change value according to the newly added frequency at time t+1; S24, when the frequency change value is greater than the frequency change threshold, marking time t+1 as a pronunciation transition point; S25. Divide the detremorized speech spectrum set according to the positions of the pronunciation conversion points to obtain detremorized speech spectrum subsets of multiple pronunciations.

4. The speech recognition method for Parkinson's disease patients according to claim 3, characterized in that: The formula for calculating the frequency change value in S23 is: , , where μ t+1 is the frequency change value at time t+1, f t+1,z,i is the i-th new frequency at time t+1, f t,s,j is the jth frequency in the detremorized speech spectrum at time t, | | is the absolute value, M is the number of frequencies in the detremorized speech spectrum at time t, and min is the number of M frequencies. The minimum value in , i and j are positive integers, N is the number of new frequencies added at time t+1, d t+1,z,i It is the minimum difference between the i-th newly added frequency at time t+1 and the frequency in the detrembled speech spectrum at time t.

5. The speech recognition method for Parkinson's disease patients according to claim 1, characterized in that: The S3 comprises the following sub-steps: S31, in each detremorized speech spectrum of a pronounced detremorized speech spectrum subset, find the frequency corresponding to the maximum amplitude as the main frequency, and average each main frequency to obtain the peak frequency; S32. In a subset of detremorized speech spectra of a pronunciation, taking other frequencies except the main frequency in each detremorized speech spectrum as overtone frequencies; S33, calculating the overtone centroid value of the overtone frequency; S34, calculating the overtone deviation value according to the difference between each overtone frequency and the peak frequency; S35. Use the peak frequency, overtone centroid value and overtone deviation value as elements to construct a pronunciation vector.

6. The speech recognition method for Parkinson's disease patients according to claim 5, characterized in that: The calculation formula of the overtone centroid value in S33 is: , where f o,c is the overtone center of mass value, f o,i is the i-th harmonic frequency in the spectrum subset of the pronounced detremized speech, A o,i is the i-th overtone frequency f o,i , L is the number of overtone frequencies in the spectrum subset of the pronounced detrembled speech, and i is a positive integer.

7. The speech recognition method for Parkinson's disease patients according to claim 5, characterized in that: The formula for calculating the harmonic deviation value in S34 is: , where f o,d is the harmonic deviation value, f o,i is the i-th harmonic frequency in the spectrum subset of the pronounced detremized speech, i is a positive integer, and f peak is the peak frequency, and L is the number of overtone frequencies in the subset of the detremized speech spectrum of the articulated speech.

8. The speech recognition method for Parkinson's disease patients according to claim 1, characterized in that: The multi-layer pronunciation vector processing model in S4 includes: a plurality of first pronunciation processing layers, a plurality of second pronunciation processing layers, a first BiLSTM layer, a second BiLSTM layer, a feature integration layer and a CRF unit; The input end of each of the first pronunciation processing layers is used to input a pronunciation vector; the input end of each of the second pronunciation processing layers is used to input a pronunciation vector; the input end of the first BiLSTM layer is respectively connected to the output ends of multiple first pronunciation processing layers; the input end of the second BiLSTM layer is respectively connected to the output ends of multiple second pronunciation processing layers; the input end of the feature integration layer is respectively connected to the output end of the first BiLSTM layer and the output end of the second BiLSTM layer, and its output end is connected to the input end of the CRF unit; the output end of the CRF unit serves as the output end of the multi-layer pronunciation vector processing model.

9. The speech recognition method for Parkinson's disease patients according to claim 8, characterized in that: The expression of the first pronunciation processing layer is: , where y1 is the output of the first pronunciation processing layer, Tanh is the hyperbolic tangent activation function, and x n is the nth element in the pronunciation vector, ω 1,n is the nth weight in the first pronunciation processing layer, b 1,n is the nth bias in the first pronunciation processing layer, and the value of n is a positive integer ranging from 1 to 3; The expression of the second pronunciation processing layer is: , where y2 is the output of the second pronunciation processing layer, Sigmoid is the S-type activation function, ω 2,n is the nth weight in the second pronunciation processing layer, b 2,n is the nth bias in the second pronunciation processing layer.

10. The speech recognition method for Parkinson's disease patients according to claim 8, characterized in that: The expression of the feature integration layer is: , where H m is the mth integrated feature output by the feature integration layer, h 1,m is the mth feature output by the first BiLSTM layer, h 2,m is the mth feature output by the second BiLSTM layer, ω 1,m is the mth first weight in the feature integration layer, ω 2,m is the mth second weight in the feature integration layer.

Images