Early warning system for progressive dysarthria in parkinson's patients based on speech features

Abstract

The present application relates to the technical field of speech analysis, and particularly relates to a Parkinson's patient progressive dysarthria early warning system based on speech characteristics. According to the time characteristics of adjacent voiced segments of each pause segment, the present application obtains the pause rationality of each pause segment, and screens out the reasonable pause segment in each audio data. According to the time characteristics and the pause rationality of different reasonable pause segments in each audio data, the present application obtains the speech continuity of each audio data. According to the energy value distribution of different frequencies in the frequency domain of each audio data, the present application obtains the pronunciation ambiguity degree of each audio data. Then, the speech quality score of the speech sample at each moment is obtained. According to the speech quality score distribution of the speech sample at different historical moments corresponding to the current moment, the potential dysarthria risk factor of the current moment is obtained, and the dysarthria risk is warned. The present application can accurately obtain the potential dysarthria risk factor of the current moment, and improve the accuracy of dysarthria warning.

Description

Technical Field

[0001] This invention relates to the field of speech analysis technology, and specifically to an early warning system for progressive dysarthria in Parkinson's patients based on speech features. Background Technology

[0002] Parkinson's disease is a serious neurodegenerative disease that commonly affects the elderly. It is a progressive and irreversible neurodegenerative disease. By the time patients show typical symptoms, neuronal damage is already quite severe, resulting in the missed window for optimal intervention. Therefore, it is necessary to provide early warning of dysarthria and promptly identify the early symptoms of the disease.

[0003] Existing technologies can only perform single-shot detection of speech signals, and cannot effectively utilize historical speech data to reflect the gradual changes in articulation function. It is difficult to distinguish whether the abnormal pronunciation is a random fluctuation or a real progression of the disease. Furthermore, it cannot accurately quantify subtle pathological features such as high and low frequency energy imbalance and spectral entropy changes. It cannot objectively quantify and assess the rate of progression and potential risk of deterioration of articulation disorders, thus reducing the accuracy of early warning of Parkinson's articulation disorders. Summary of the Invention

[0004] To address the technical problem that relying solely on single-shot speech signal detection fails to effectively utilize historical speech data to reflect the progressive changes in articulation function, resulting in low accuracy in early warning of Parkinson's disease dysarthria, this invention aims to provide an early warning system for progressive dysarthria in Parkinson's patients based on speech features. The specific technical solution adopted is as follows: This invention proposes an early warning system for progressive dysarthria in Parkinson's patients based on speech features, the system comprising: The data acquisition module is used to acquire speech samples containing multiple audio data from Parkinson's patients at each historical moment; The speech continuity quantization module is used to obtain the pause segments and spoken segments of each audio data; based on the time characteristics of the adjacent spoken segments of each pause segment, the rationality of the pause segment is obtained, and reasonable pause segments in each audio data are selected; based on the time characteristics and rationality of different reasonable pause segments in each audio data, the speech continuity of each audio data is obtained. The speech quality assessment module is used to obtain the degree of pronunciation ambiguity of each audio data based on the energy value distribution of each frequency in the frequency domain; and to obtain the speech quality score of each speech sample at each time step based on the speech continuity and pronunciation ambiguity of different audio data in the speech sample at each time step. The articulation disorder early warning module is used to obtain the potential articulation risk factor at the current moment based on the distribution of speech quality scores of speech samples at different historical moments corresponding to the current moment; and to provide risk warnings for articulation disorders based on the potential articulation risk factor at the current moment.

[0005] Furthermore, the method for obtaining the pause segments and audio segments includes: Obtain the binary sequence of each audio data, and take the range of consecutive binary 1s in the binary sequence as the audio segment; The range of consecutive binary 0s in a binary sequence is taken as a pause segment.

[0006] Furthermore, the method for obtaining the rationality of the pause includes: For each pause, the adjacent vocal segments before and after it are analyzed to obtain the difference between the duration of any adjacent vocal segment and the preset minimum pronunciation duration threshold, and then normalized to serve as the pronunciation integrity of any adjacent vocal segment. The rationality of each pause segment is determined by the pause duration and the pronunciation integrity of all adjacent phonological segments. The pause duration is negatively correlated with the rationality of the pause, while the pronunciation integrity is positively correlated with the rationality of the pause.

[0007] Furthermore, the method for obtaining the reasonable pause segment includes: If the reasonableness of a pause segment is greater than or equal to a preset reasonable threshold, the corresponding pause segment will be regarded as a reasonable pause segment.

[0008] Furthermore, the method for obtaining speech continuity includes: Obtain the time interval between the start time of each pause segment and the start time of the corresponding speech sample, calculate the ratio of the time interval to the total duration of the speech sample, and if the ratio is within a preset importance threshold range, set the positional importance of each pause segment to a preset first importance value; if the ratio is not within the preset importance threshold range, set the positional importance of each pause segment to a preset second importance value; the preset first importance value is greater than the preset second importance value. If the total number of pauses is not 0, the speech continuity of each audio data is obtained based on the number of reasonable pauses in each audio data, as well as the rationality and positional importance of the reasonable pauses. The number of reasonable pauses, the rationality of the pauses, and the positional importance are all positively correlated with speech continuity. If the total number of pauses is 0, the speech continuity of each audio data is set to a positive integer 1.

[0009] Furthermore, the method for obtaining the degree of pronunciation ambiguity includes: For any audio data, a first target frequency with a frequency less than a preset first frequency threshold is obtained in the frequency domain, and the cumulative value of the energy values ​​of all first target frequencies is obtained as the low-frequency energy value; a second target frequency with a frequency greater than a preset second frequency threshold is obtained in the frequency domain, and the cumulative value of the energy values ​​of all second target frequencies is obtained as the high-frequency energy value, wherein the preset first frequency threshold is less than the preset second frequency threshold. The degree of pronunciation ambiguity of the corresponding audio data is obtained based on the low-frequency energy value, high-frequency energy value, and the degree of energy value disorder at different frequencies. The low-frequency energy value and the degree of energy value disorder are positively correlated with the degree of pronunciation ambiguity, while the high-frequency energy value is negatively correlated with the degree of pronunciation ambiguity.

[0010] Furthermore, the method for obtaining the degree of pronunciation ambiguity includes: Obtain the energy ratio between low-frequency and high-frequency energy values; obtain the entropy values ​​of energy values ​​at different frequencies as the degree of energy disorder. The product of the energy ratio and the degree of energy confusion is used as the degree of pronunciation ambiguity.

[0011] Furthermore, the method for obtaining the speech quality score includes: A negative correlation mapping is performed on the pronunciation ambiguity of each audio data, and the product between the negative correlation mapping result and the speech continuity is obtained as the speech fluency of each audio data. The average speech fluency of all audio data in the speech sample at each time step is obtained as the speech quality score of the speech sample at each time step.

[0012] Furthermore, the method for obtaining the potential articulation risk factors includes: Obtain the fitted straight line that linearly fits the speech quality scores of speech samples at all historical moments; The speech quality scores of the speech samples at the latest historical moment are negatively correlated and normalized, which is used as the first risk coefficient. The slope of the fitted line is negatively correlated to obtain the product between the first risk coefficient and the negative correlation mapping value of the slope, and then normalized to serve as the potential articulation risk factor at the current moment.

[0013] Furthermore, the risk warning for articulation disorders includes: If the potential articulation risk factor at the current moment is less than or equal to the preset first threshold, the articulation disorder at the current moment is judged to be at the first risk warning level; If the potential articulation risk factor at the current moment is greater than the preset first threshold and less than the preset second threshold, the articulation disorder at the current moment is judged to be at the second risk warning level. If the potential articulation risk factor at the current moment is greater than or equal to a preset second threshold, the articulation disorder at the current moment is determined to be at the third risk warning level; the preset second threshold is greater than a preset first threshold.

[0014] The present invention has the following beneficial effects: This invention obtains the pause segments and vocal segments of each audio data point, providing precise data slices for extracting speech rhythm features and articulation duration features; based on the temporal features of adjacent vocal segments of each pause segment, the rationality of the pause in each pause segment is obtained, and reasonable pause segments in each audio data point are selected, accurately identifying normal physiological pauses; based on the temporal features and rationality of different reasonable pause segments in each audio data point, the speech continuity of each audio data point is obtained, reflecting the motor coordination of the articulatory organs during the transition between the start and end of speech; based on the energy value distribution of different frequencies in the frequency domain of each audio data point, the pronunciation ambiguity of each audio data point is obtained. This invention reveals the loss of high-frequency consonants due to articulatory muscle weakness through frequency domain energy distribution; it obtains the speech quality score of each speech sample at each time point based on the speech continuity and pronunciation ambiguity of different audio data in the speech sample, providing an objective, comprehensive, and standardized quantitative assessment index for single-instance articulation function; it obtains the potential articulation risk factors at the current time based on the distribution of speech quality scores of speech samples at different historical time points, filtering out accidental emotional or environmental noise interference in a single detection, and uncovering the degree of hidden articulation risk; based on the potential articulation risk factors at the current time, it provides risk warnings for articulation disorders. This invention improves the accuracy of articulation disorder warnings by accurately obtaining the potential articulation risk factors at the current time. Attached Figure Description

[0015] Figure 1 A structural block diagram of an early warning system for progressive dysarthria in Parkinson's patients based on speech features, provided as an embodiment of the present invention; Figure 2 A flowchart illustrating a method for obtaining the degree of pronunciation ambiguity according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating a method for obtaining potential articulation risk factors according to an embodiment of the present invention. Detailed Implementation

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0017] The following description, in conjunction with the accompanying drawings, details the specific scheme of an early warning system for progressive dysarthria in Parkinson's patients based on speech features, provided by the present invention.

[0018] Please see Figure 1The diagram illustrates a structural block diagram of an early warning system for progressive dysarthria in Parkinson's disease patients based on speech features, according to an embodiment of the present invention. The system specifically includes: a data acquisition module 101, a speech continuous quantization module 102, a speech quality assessment module 103, and a dysarthria early warning module 104. The data acquisition module 101 is used to acquire speech samples containing multiple audio data at each historical moment of a Parkinson's patient.

[0019] In embodiments of the present invention, in order to effectively distinguish between occasional fluctuations in condition and the actual progression of the disease, thereby accurately capturing subtle early abnormal changes and achieving dynamic tracking and timely early warning of the risk of dysarthria progression, it is necessary to establish a regular speech collection plan, such as collecting patients' speech samples weekly or monthly to continuously monitor their speech characteristics. First, a quiet indoor environment with no obvious echo and no multiple people talking is selected, and speech is recorded using fixed equipment such as mobile phones or microphones at a uniform sampling rate. The collection program sequentially plays or displays fixed, simple, and uniform Chinese guiding statements, and the patient responds naturally and clearly verbally according to the questions. The system records every audio of the patient throughout the process, thereby completing one speech sample collection. Speech samples containing multiple audio data of Parkinson's patients at each historical moment are obtained.

[0020] It should be noted that, in the embodiments of the present invention, in order to facilitate continuous monitoring of its articulation characteristics, the patient's speech samples are collected once a week, that is, the time interval is 1 week, and the speech is recorded at a sampling rate of 16kHz.

[0021] The speech continuity quantization module 102 is used to obtain the pause segments and the spoken segments of each audio data; based on the time characteristics of the adjacent spoken segments of each pause segment, the rationality of the pause segment is obtained, and reasonable pause segments in each audio data are selected; based on the time characteristics and rationality of different reasonable pause segments in each audio data, the speech continuity of each audio data is obtained.

[0022] Normal speech communication requires appropriate pauses, while excessively long pauses may indicate obstacles in the speech process. Therefore, obtaining the pause segments and audible segments of each audio data point helps to analyze the rationality of the pause segments.

[0023] Preferably, in one embodiment of the present invention, the method for obtaining pause segments and audio segments includes: Obtain the binary sequence of each audio data, and take the range of consecutive binary 1s in the binary sequence as the audio segment; The range of consecutive binary 0s in a binary sequence is taken as a pause segment.

[0024] It should be noted that the audio data is input into the rVADfast model, and the output is a binary sequence containing 0 and 1. rVADfast is an efficient speech activity detection model that divides the audio data input by the patient into several small frames of fixed duration, analyzes the audio features frame by frame, and outputs the audible frames marked as 1 and the silent frames marked as 0, which can accurately detect pauses in the speech data. The specific methods are well known to those skilled in the art and will not be described in detail here.

[0025] In normal speech, the audio segments before and after a pause are usually quite long, corresponding to complete words or sentence components. The longer the adjacent audio segments are and the more complete the pronunciation units are, the more reasonable the pause is. The reasonableness of each pause can be obtained by considering the time characteristics of the adjacent audio segments of each pause.

[0026] Preferably, in one embodiment of the present invention, the method for obtaining the rationality of the pause includes: The first step is to analyze the adjacent vocal segments before and after each pause, obtain the difference between the duration of any adjacent vocal segment and the preset minimum pronunciation duration threshold, and normalize it as the pronunciation integrity of any adjacent vocal segment. It should be noted that, in the embodiments of the present invention, 100ms is the lower limit of the physiological duration of a stable articulation unit in human language. Speech segments below this threshold usually correspond to momentary articulation caused by Parkinson's disease-induced articulation obstruction or muscle tremors. Therefore, the preset minimum articulation duration threshold is set to 100ms. The larger the difference, the longer the duration of adjacent speech segments exceeds the preset minimum articulation duration threshold, and the more complete the pronunciation.

[0027] It should be noted that, in the embodiments of the present invention, normalization is performed by maximum and minimum value normalization, that is, the maximum and minimum values ​​of the difference are obtained from a large amount of historical experimental data and normalized. If there is a difference less than the minimum value, the pronunciation integrity is set to 0; if there is a difference greater than the maximum value, the pronunciation integrity is set to 1, so that the value range of pronunciation integrity is within 0-1. The specific means are well known to those skilled in the art and will not be described in detail here.

[0028] The second step is to determine the rationality of each pause based on the pause duration and the pronunciation integrity of all adjacent phonological segments. The pause duration is negatively correlated with the rationality of the pause, while the pronunciation integrity is positively correlated with the rationality of the pause.

[0029] It should be noted that pause duration reflects the interval of articulation; the larger the interval, the slower the articulation. Pronunciation integrity reflects the robustness of the speech motor sequence and the organizational ability of articulation units; the greater the pronunciation integrity, the more reasonable the pause. Therefore, pause duration is negatively correlated with pause rationality, while pronunciation integrity is positively correlated with pause rationality.

[0030] In one embodiment of the present invention, a negative correlation mapping is performed on the pause duration to obtain the product between the negative correlation mapping result and the pronunciation integrity, and then normalized as the pause rationality of each pause segment.

[0031] It should be noted that, in the embodiments of the present invention, the pause duration of the obtained pause segment is not 0. Therefore, negative correlation mapping is performed by taking the reciprocal. The longer the pause duration, the less reasonable the pause is. Normalization is performed by using maximum and minimum value normalization, that is, the maximum and minimum values ​​of the product are selected from a large amount of historical experimental data and normalized to the range of 0-1. The specific means are well known to those skilled in the art and will not be described in detail here.

[0032] The greater the reasonableness of the pause, the greater the probability of correct pronunciation. Filter out reasonable pause segments in each audio data.

[0033] Preferably, in one embodiment of the present invention, the method for obtaining a reasonable pause segment includes: If the reasonableness of a pause segment is greater than or equal to a preset reasonable threshold, the corresponding pause segment will be regarded as a reasonable pause segment.

[0034] It should be noted that, in one embodiment of the present invention, the greater the reasonableness of the pause, the greater the probability of a reasonable pause. Therefore, the preset reasonable threshold is a slightly larger value to filter out pause segments with higher credibility. The preset reasonable threshold is set to 0.7. In other embodiments of the present invention, the preset reasonable threshold can be set according to specific circumstances, and will not be limited or elaborated here.

[0035] Because the core and keywords of Chinese sentences are mostly located in the middle, the beginning of a sentence is often a function word or an initiating tone, and the end of a sentence is often a concluding tone, frequent abnormal pauses in the middle of a sentence in Parkinson's patients can directly disrupt the expression of core information. These pauses are more indicative of the severity of articulation impairment than pauses at the beginning or end of a sentence, and therefore have a higher positional importance and cause more severe disruption to speech continuity. Therefore, by analyzing the temporal characteristics and rationality of reasonable pause segments, we can reflect the speech continuity of the articulatory organs during the transition between the beginning and end of speech flow. Based on the temporal characteristics and rationality of different reasonable pause segments in each audio data, we can obtain the speech continuity of each audio data.

[0036] Preferably, in one embodiment of the present invention, the method for obtaining speech continuity includes: The first step is to obtain the time interval between the start time of each pause and the start time of the corresponding speech sample, and calculate the ratio of the time interval to the total duration of the speech sample. If the ratio is within a preset importance threshold range, the positional importance of each pause is set to a preset first importance value; if the ratio is not within the preset importance threshold range, the positional importance of each pause is set to a preset second importance value; the preset first importance value is greater than the preset second importance value. It should be noted that the time interval between the start time of each pause segment and the start time of the corresponding speech sample represents the difference between the start time of each pause segment and the start time of the corresponding speech sample. This reflects the relative time offset of the pause in the speech sample. The larger the time offset, the longer the corresponding pause segment appears in the speech sample.

[0037] In one embodiment of the present invention, since the main body and keywords of Chinese sentences are mostly distributed in the middle of the sentence, the closer the pause is to the middle of the speech sample (i.e., the proportion of the pause to the total duration of the speech sample is in the middle range), the greater its impact on the continuity of speech, and the higher the importance of the pause position. Based on relevant historical experience, the preset importance threshold range is set to 0.2-0.8. If the proportion of each pause is in the range of 0.2-0.8, the closer it is to the middle of the sentence, the greater its positional importance, and the preset first importance value is set to 1.5. If the proportion of each pause is not in the range of 0.2-0.8, the closer it is to the beginning or end of the sentence, the smaller its positional importance, and the preset second importance value is set to 0.5. In other embodiments of the present invention, the preset importance threshold range, the preset first importance value, and the preset second importance value can be specifically set according to specific circumstances, and are not limited or elaborated here.

[0038] The second step is to obtain the speech continuity of each audio data based on the number of reasonable pauses in each audio data, as well as the rationality and positional importance of the reasonable pauses. The number of reasonable pauses, the rationality of the pauses, and the positional importance are all positively correlated with speech continuity. If the total number of pauses is 0, the speech continuity of each audio data is set to a positive integer of 1.

[0039] It should be noted that, due to the smoothing window control in the rVADfast model, normal, small gaps between words are smoothed out by the model and marked as spoken frames. Therefore, there may be cases where the number of pauses is zero. The fewer the total number of pauses, the less significant the intervals, the more coherent the tone, and the better the speech continuity. The more pauses, the more frequent the unexpected interruptions caused by insufficient airflow support, stiff vocal muscles, or slow movements when expressing short sentences, resulting in poorer speech continuity. However, the more reasonable pauses there are, the greater the proportion of pauses in normal speech, and the better the speech continuity. The higher the reasonableness of a reasonable pause, the more it belongs to normal physiological pauses or language habits, and the greater its positional importance, the greater the contribution of reasonable pauses to speech continuity, and the better the speech continuity. Therefore, the number of pauses is negatively correlated with speech continuity, while the reasonableness and positional importance of pauses are both positively correlated with speech continuity.

[0040] It should be noted that, in one embodiment of the present invention, if the total number of pause segments is not zero, the cumulative value of the product between the pause rationality and positional importance of all reasonable pause segments is obtained as the first continuity; the ratio of the number of reasonable pause segments to the total number of pause segments is calculated, and the product between the ratio result and the first continuity is obtained and normalized, which is used as the speech continuity of each audio data. The formula for speech continuity is expressed as: ;in, Indicates the first Speech continuity of audio data; Indicates the first The number of reasonable pauses in each audio data segment; Indicates the first The total number of pauses in each audio data item; Indicates the first The first audio data The rationality of a pause in a reasonable pause segment; Indicates the first The first audio data The importance of the location of a reasonable pause segment; This represents the normalization function.

[0041] It should be noted that, in the embodiments of the present invention, the normalization function adopts maximum and minimum value normalization, that is, all audio data are analyzed, and the maximum and minimum values ​​of the product under the dimension to be normalized are selected and normalized to the range of 0-1; the specific means are well known to those skilled in the art and will not be described in detail here.

[0042] The speech quality assessment module 103 is used to obtain the degree of pronunciation ambiguity of each audio data based on the energy value distribution of each audio data at different frequencies in the frequency domain; and to obtain the speech quality score of each speech sample at each time step based on the speech continuity and pronunciation ambiguity of different audio data in the speech sample at each time step.

[0043] Parkinson's patients experience stiffness and weakness in the throat and lip muscles, leading to unstable airflow control and vocal cord vibration during pronunciation. This results in severe disruption of the speech spectrum structure, with weakened high-frequency consonant energy and relatively enhanced low-frequency components. The energy is no longer concentrated at specific frequencies but is diffused and evenly distributed across the entire frequency domain. Therefore, the degree of pronunciation ambiguity for each audio data can be obtained by analyzing the energy distribution of each audio data at different frequencies in the frequency domain.

[0044] Preferably, in one embodiment of the present invention, the method for obtaining the degree of pronunciation ambiguity is described in [reference needed]. Figure 2 It shows a flowchart of a method for obtaining the degree of pronunciation ambiguity, including: Step S201: For any audio data, obtain a first target frequency in the frequency domain whose frequency is less than a preset first frequency threshold, and obtain the cumulative value of the energy values ​​of all first target frequencies as the low-frequency energy value; obtain a second target frequency in the frequency domain whose frequency is greater than a preset second frequency threshold, and obtain the cumulative value of the energy values ​​of all second target frequencies as the high-frequency energy value, wherein the preset first frequency threshold is less than the preset second frequency threshold.

[0045] It should be noted that, in one embodiment of the present invention, the complex audio signal can be decomposed into its different frequency components by Fourier transform, that is, the time-domain signal is converted into a frequency-domain signal to obtain the frequency-domain energy spectrum, including the energy values ​​corresponding to different frequencies; in other embodiments of the present invention, the audio signal can also be decomposed into multiple resolutions by wavelet transform and wavelet basis functions to obtain the energy distribution at different frequency band scales; the specific means are well known to those skilled in the art and will not be described in detail here.

[0046] It should be noted that, in one embodiment of the present invention, in order to quantify the typical pathological characteristics of consonant weakening in Parkinson's patients, namely, weakened high-frequency consonant energy and relatively enhanced low-frequency components, the frequency bands containing the fundamental frequency and basic vowel energy of vocal cord vibration, as well as the frequency bands mainly containing high-frequency consonant energy such as fricatives and plosives, are analyzed. Based on relevant historical experience, a first frequency threshold is preset to 500Hz and a second frequency threshold is preset to 2kHz. In other embodiments of the present invention, the values ​​of the preset first frequency threshold and the preset second frequency threshold can be specifically set according to the specific circumstances, and are not limited or elaborated here.

[0047] Step S202: Based on the low-frequency energy value, high-frequency energy value, and the degree of confusion of energy values ​​at different frequencies, obtain the degree of pronunciation ambiguity of the corresponding audio data. The low-frequency energy value and the degree of confusion of energy values ​​are positively correlated with the degree of pronunciation ambiguity, while the high-frequency energy value is negatively correlated with the degree of pronunciation ambiguity.

[0048] It should be noted that healthy individuals pronounce vowels and consonants clearly, with a clear and coordinated high- and low-frequency energy structure. Parkinson's patients, due to stiff and slowed oral muscles, experience insufficient obstruction of airflow for consonants, leading to a sharp decrease in high-frequency energy. This forces energy to concentrate in the low frequencies, resulting in an imbalance in the high- and low-frequency energy ratio and a greater degree of ambiguity in pronunciation. The degree of energy disorder at different frequencies reflects the concentration and stability of the frequency energy distribution. Healthy sound energy is regularly concentrated on specific resonant peaks; the less disorder, the less ambiguity. In patients, airflow control and vocal cord vibration are unstable, causing energy to diffuse irregularly across the entire frequency range; the greater the disorder, the greater the ambiguity. Therefore, both low-frequency energy values ​​and the degree of energy disorder are positively correlated with the degree of ambiguity, while high-frequency energy values ​​are negatively correlated.

[0049] In one embodiment of the present invention, considering that environmental noise and device background noise always exist in actual recorded speech, the high-frequency energy will not be absolutely zero. The energy ratio of low-frequency energy value and high-frequency energy value is obtained; the entropy value of energy value of different frequencies is obtained as the degree of energy value disorder; the product of energy ratio and degree of energy value disorder is obtained as the degree of pronunciation ambiguity.

[0050] It should be noted that in real-world scenarios, human voices contain fundamental frequency, harmonics, and ambient noise. The spectral energy is distributed, so the entropy of energy values ​​at different frequencies will not be zero. The greater the entropy, the more energy is scattered across many frequencies, resulting in a uniform, chaotic, and irregular distribution. The greater the disorder of energy values, the more indistinct the pronunciation.

[0051] Speech continuity reflects the control stability of the articulatory organs and the coordination of respiratory airflow. The greater the speech continuity, the greater the coordination, and the greater the speech quality. The degree of articulation ambiguity reflects the strength and precision of the articulatory muscles and the regularity of the spectral energy distribution. The greater the ambiguity, the worse the regularity, and the worse the speech quality. The speech quality score is quantified by combining the two features. The speech quality score of each speech sample is obtained based on the speech continuity and articulation ambiguity of different audio data in the speech sample at each time step.

[0052] Preferably, in one embodiment of the present invention, the method for obtaining the speech quality score includes: A negative correlation mapping is performed on the pronunciation ambiguity of each audio data, and the product between the negative correlation mapping result and the speech continuity is obtained as the speech fluency of each audio data. It should be noted that, in one embodiment of the present invention, negative correlation mapping is performed by taking the reciprocal; in other embodiments of the present invention, it is performed using an exponential function with the natural constant as the base. Negative correlation mapping is performed; the less blurred the pronunciation, the clearer the pronunciation and the more fluent the speech. The specific methods are well known to those skilled in the art and will not be elaborated here.

[0053] The average speech fluency of all audio data in the speech sample at each time step is obtained as the speech quality score of the speech sample at each time step.

[0054] Therefore, the greater the fluency of the speech, the less the speech is affected by abnormal pauses, and the higher the speech quality score.

[0055] The articulation disorder early warning module 104 is used to obtain the potential articulation risk factor at the current moment based on the distribution of speech quality scores of speech samples at different historical moments corresponding to the current moment; and to provide risk warning for articulation disorders based on the potential articulation risk factor at the current moment.

[0056] The speech continuity of a single audio data point may be affected by a variety of factors, and its performance may be random. However, the comprehensive analysis of multiple audio data points can reduce the error caused by randomness. Therefore, based on the speech quality score distribution of speech samples at different historical times corresponding to the current time, the potential articulation risk factor at the current time can be obtained.

[0057] Preferably, in one embodiment of the present invention, the method for obtaining potential articulation risk factors is described in [reference needed]. Figure 3 It illustrates a flowchart of a method for obtaining potential articulation risk factors, including: Step S301: Obtain the fitted straight line for linearly fitting the speech quality scores of speech samples at all historical time points.

[0058] It should be noted that in some embodiments of the present invention, the order of historical moments is used as the horizontal axis and the speech quality score is used as the vertical axis. Linear fitting is performed by least squares method or polynomial fitting. The specific means are well known to those skilled in the art and will not be described in detail here.

[0059] Step S302: Perform negative correlation normalization mapping on the speech quality scores of the speech samples at the latest historical moment, and use it as the first risk coefficient.

[0060] It should be noted that the historical moment refers to the historical moment before the current moment, and the latest historical moment is the last one obtained among all historical moments, that is, the historical moment adjacent to the current moment.

[0061] It should be noted that, in one embodiment of the present invention, the maximum value of the speech quality scores of speech samples at all historical time points is obtained; if the maximum value is 0, that is, the speech quality scores of speech samples at all historical time points are all 0, there is no effective coherent speech, and the first risk coefficient is set to 1; if the maximum value is not 0, the ratio of the speech quality score of the speech sample at the latest historical time point to the maximum value is obtained, and the difference between the positive integer 1 and the ratio result is obtained, that is, negative correlation normalization mapping is performed to obtain the first risk coefficient; the larger the speech quality score of the speech sample at the latest historical time point, the closer it is to the maximum value of the speech quality score, the better the speech ability at the latest historical time point, the smaller the articulation risk, and the smaller the first risk coefficient.

[0062] Step S303: Perform a negative correlation mapping on the slope of the fitted straight line to obtain the product between the first risk coefficient and the slope negative correlation mapping value, and normalize it as the potential articulation risk factor at the current moment.

[0063] It should be noted that, in one embodiment of the present invention, the slope of the fitted line is extracted by differentiating the fitted line using an exponential function with the natural constant as the base. By performing a negative correlation mapping on the slope, calculating the product between the first risk coefficient and the negative correlation mapping value of the slope, and then performing a normalized mapping using maximum and minimum value normalization, the potential articulation risk factor at the current moment is obtained. The formula for the potential articulation risk factor is expressed as: ;in, This indicates potential articulation risk factors at the current moment; This represents the slope of the fitted line; This represents the speech quality score of the speech sample at the latest historical moment. This represents the maximum speech quality score among all historical speech samples. Represents an exponential function with the natural constant as its base; This represents the normalization function.

[0064] It should be noted that the slope of the fitted line for historical speech quality scores reflects the trend of disease progression. When the overall speech quality score tends to rise, the slope is positive; the larger the positive slope, the lower the relative risk of articulation. When the overall speech quality score tends to fall, the slope is negative; the larger the negative slope, the higher the relative risk of articulation. Therefore, an exponential function with a natural constant is used. By negatively mapping the slope, the larger the slope, the smaller the articulation risk; therefore, the smaller the first risk coefficient, the smaller the negative correlation mapping value of the slope, and the smaller the potential articulation risk factor.

[0065] It should be noted that, in the embodiments of the present invention, the method for obtaining the maximum and minimum normalization values ​​is as follows: the maximum and minimum values ​​of the product are selected from a large amount of historical experimental data and normalized. If there is a product at the current moment that is greater than the maximum value, the potential articulation risk factor at the current moment is set to 1; if there is a product at the current moment that is less than the minimum value, the potential articulation risk factor at the current moment is set to 0, so that the potential articulation risk factor takes a value within the range of 0-1. The specific means are well known to those skilled in the art and will not be described in detail here.

[0066] Potential articulation risk factors reflect the degree of risk in the articulation process. The higher the potential articulation risk factor, the greater the potential articulation disorder may be, and the greater the need for early warning. Based on the potential articulation risk factor at the current moment, risk warnings for articulation disorders are issued.

[0067] Preferably, in one embodiment of the present invention, risk warning for articulation disorders includes: If the potential articulation risk factor at the current moment is less than or equal to the preset first threshold, the articulation disorder at the current moment is judged to be at the first risk warning level; If the potential articulation risk factor at the current moment is greater than the preset first threshold and less than the preset second threshold, the articulation disorder at the current moment is judged to be at the second risk warning level. If the potential articulation risk factor at the current moment is greater than or equal to the preset second threshold, the articulation disorder at the current moment is judged to be at the third risk warning level; the preset second threshold is greater than the preset first threshold.

[0068] It should be noted that the greater the potential articulation risk factor, the greater the possibility of articulation disorder. Therefore, the preset second threshold is greater than the preset first threshold, the first risk warning level is lower than the second risk warning level, and the second risk warning level is lower than the third risk warning level. In one embodiment of the present invention, the preset first threshold is set to 0.3 and the preset second threshold is set to 0.7. In other embodiments of the present invention, the values ​​of the preset first threshold and the preset second threshold can be set according to specific circumstances, and are not limited or elaborated here.

[0069] It should be noted that for the first risk warning level, the patient's overall articulation function is stable, with no obvious trend of speech quality deterioration. The greater the recent improvement in speech quality and the lower the risk of articulation problems, the lower the corresponding warning level. For the second risk warning level, the patient's articulation function has shown a noticeable trend of deterioration, and there is a possibility of continued worsening of the articulation problem. It is necessary to raise the warning level, increase the monitoring frequency, and promptly carry out speech rehabilitation training. For the third risk warning level, the patient's problems such as stiffness of the throat and lip muscles and weakness caused by Parkinson's disease continue to worsen, and the motor control ability of the articulatory organs is rapidly declining. It is necessary to activate the warning immediately and recommend to carry out clinical evaluation and strengthen speech rehabilitation training as soon as possible.

[0070] In summary, this invention obtains the rationality of each pause segment based on the temporal characteristics of adjacent vocal segments, thus filtering out reasonable pause segments in each audio data. Based on the temporal characteristics and rationality of different reasonable pause segments in each audio data, it obtains the speech continuity of each audio data. Based on the energy value distribution of different frequencies in the frequency domain, it obtains the degree of pronunciation ambiguity of each audio data. Furthermore, it obtains the speech quality score of the speech sample at each moment. Based on the distribution of speech quality scores of speech samples at different historical moments corresponding to the current moment, it obtains the potential articulation risk factors for the current moment, providing a risk warning for articulation disorders. This invention improves the accuracy of articulation disorder warnings by accurately obtaining the potential articulation risk factors at the current moment.

[0071] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A voice feature-based early warning system for progressive dysarthria in Parkinson's patients, characterized in that, The system includes: The data acquisition module is used to acquire speech samples containing multiple audio data from Parkinson's patients at each historical moment; The speech continuity quantization module is used to obtain the pause segments and spoken segments of each audio data; based on the time characteristics of the adjacent spoken segments of each pause segment, the rationality of the pause segment is obtained, and reasonable pause segments in each audio data are selected; based on the time characteristics and rationality of different reasonable pause segments in each audio data, the speech continuity of each audio data is obtained. The methods for obtaining the rationality of the pause include: The rationality of each pause segment is obtained based on the pause duration of each pause segment and the pronunciation integrity of all adjacent phonological segments. The pause duration is negatively correlated with the rationality of the pause, while the pronunciation integrity is positively correlated with the rationality of the pause. The method for obtaining speech continuity includes: If the total number of pauses is not 0, the speech continuity of each audio data is obtained based on the number of reasonable pauses in each audio data, the reasonableness of the reasonable pauses, and the positional importance of the reasonable pauses. The number of reasonable pauses, the reasonableness of the pauses, and the positional importance are all positively correlated with the speech continuity. If the total number of pauses is 0, the speech continuity of each audio data is set to a positive integer 1. The speech quality assessment module is used to obtain the degree of pronunciation ambiguity of each audio data based on the energy value distribution of each frequency in the frequency domain; and to obtain the speech quality score of each speech sample at each time step based on the speech continuity and pronunciation ambiguity of different audio data in the speech sample at each time step. The articulation disorder early warning module is used to obtain the potential articulation risk factor at the current moment based on the distribution of speech quality scores of speech samples at different historical moments corresponding to the current moment; and to provide risk warnings for articulation disorders based on the potential articulation risk factor at the current moment.

2. The speech feature based early warning system for progressive dysarthria in Parkinson's patients according to claim 1, wherein, The methods for obtaining the pause segments and audio segments include: Obtain the binary sequence of each audio data, and take the range of consecutive binary 1s in the binary sequence as the audio segment; The range of consecutive binary 0s in a binary sequence is taken as a pause segment.

3. The speech feature based early warning system for progressive dysarthria in Parkinson's patients according to claim 1, wherein, The method for obtaining pronunciation integrity includes: For each pause, the adjacent vocal segments before and after it are analyzed to obtain the difference between the duration of any adjacent vocal segment and the preset minimum pronunciation duration threshold. This difference is then normalized and used as the pronunciation integrity of any adjacent vocal segment.

4. The speech feature based early warning system for progressive dysarthria in Parkinson's patients according to claim 1, wherein, The method for obtaining the reasonable pause segment includes: If the reasonableness of a pause segment is greater than or equal to a preset reasonable threshold, the corresponding pause segment will be regarded as a reasonable pause segment.

5. The speech feature based early warning system for progressive dysarthria in Parkinson's patients according to claim 1, wherein, The method for obtaining the importance of the location includes: Obtain the time interval between the start time of each pause segment and the start time of the corresponding speech sample, calculate the ratio of the time interval to the total duration of the speech sample, and if the ratio is within a preset importance threshold range, set the positional importance of each pause segment to a preset first importance value; if the ratio is not within the preset importance threshold range, set the positional importance of each pause segment to a preset second importance value; the preset first importance value is greater than the preset second importance value.

6. The early warning system for progressive dysarthria in Parkinson's patients based on speech features according to claim 1, characterized in that, The method for obtaining the degree of pronunciation ambiguity includes: For any audio data, a first target frequency with a frequency less than a preset first frequency threshold is obtained in the frequency domain, and the cumulative value of the energy values ​​of all first target frequencies is obtained as the low-frequency energy value; a second target frequency with a frequency greater than a preset second frequency threshold is obtained in the frequency domain, and the cumulative value of the energy values ​​of all second target frequencies is obtained as the high-frequency energy value, wherein the preset first frequency threshold is less than the preset second frequency threshold. The degree of pronunciation ambiguity of the corresponding audio data is obtained based on the low-frequency energy value, high-frequency energy value, and the degree of energy value disorder at different frequencies. The low-frequency energy value and the degree of energy value disorder are positively correlated with the degree of pronunciation ambiguity, while the high-frequency energy value is negatively correlated with the degree of pronunciation ambiguity.

7. The early warning system for progressive dysarthria in Parkinson's patients based on speech features according to claim 6, characterized in that, The method for obtaining the degree of pronunciation ambiguity includes: Obtain the energy ratio between low-frequency and high-frequency energy values; obtain the entropy values ​​of energy values ​​at different frequencies as the degree of energy disorder. The product of the energy ratio and the degree of energy confusion is used as the degree of pronunciation ambiguity.

8. The early warning system for progressive dysarthria in Parkinson's patients based on speech features according to claim 1, characterized in that, The method for obtaining the speech quality score includes: A negative correlation mapping is performed on the pronunciation ambiguity of each audio data, and the product between the negative correlation mapping result and the speech continuity is obtained as the speech fluency of each audio data. The average speech fluency of all audio data in the speech sample at each time step is obtained as the speech quality score of the speech sample at each time step.

9. The early warning system for progressive dysarthria in Parkinson's patients based on speech features according to claim 1, characterized in that, The methods for obtaining the potential articulation risk factors include: Obtain the fitted straight line that linearly fits the speech quality scores of speech samples at all historical moments; The speech quality scores of the speech samples at the latest historical moment are negatively correlated and normalized, which is used as the first risk coefficient. The slope of the fitted line is negatively correlated to obtain the product between the first risk coefficient and the negative correlation mapping value of the slope, and then normalized to serve as the potential articulation risk factor at the current moment.

10. The early warning system for progressive dysarthria in Parkinson's patients based on speech features according to claim 1, characterized in that, The risk warning for articulation disorders includes: If the potential articulation risk factor at the current moment is less than or equal to the preset first threshold, the articulation disorder at the current moment is judged to be at the first risk warning level; If the potential articulation risk factor at the current moment is greater than the preset first threshold and less than the preset second threshold, the articulation disorder at the current moment is judged to be at the second risk warning level. If the potential articulation risk factor at the current moment is greater than or equal to a preset second threshold, the articulation disorder at the current moment is determined to be at the third risk warning level; the preset second threshold is greater than a preset first threshold.

Images