Information processing method and system for early recognition of dementia based on speech features

Abstract

The application relates to the technical field of speech signal processing, and particularly discloses an information processing method and system for early recognition of dementia based on speech characteristics, which collects continuous speech signals of a subject in a word production task, extracts a word time label and a text sequence, divides a time window according to the time label, counts the number of words to obtain a production rate sequence, performs multi-scale coarse-grained decomposition on the sequence, calculates sample entropy of each scale to form a multi-scale entropy spectrum, and calculates single-scale sample entropy, detects local extreme peaks in the entropy spectrum, combines the decay slope of the production rate, judges a silent paragraph as a strategic reorganization pause or a word retrieval failure pause, accumulates the total duration and the earliest position of the word retrieval failure pause, and generates a cognitive risk level in combination with the total word quantity. The application can distinguish between pauses caused by educational background and pauses caused by cognitive impairment, reduces the misjudgment rate of the low-education population, and improves the specificity and fairness of early screening.

Description

Technical Field

[0001] This invention relates to the field of speech signal processing technology, specifically to a method and system for early identification of dementia based on speech features. Background Technology

[0002] In recent years, speech feature-based methods for early dementia identification have attracted widespread attention due to their advantages such as being non-invasive, low-cost, and remotely operable. Existing technologies often employ word fluency tasks (e.g., requiring subjects to name as many animals as possible within one minute) to collect speech samples. Then, acoustic or linguistic features such as speech rate, pause duration, vocabulary richness, and average output interval are extracted from the speech signals to construct classification models that distinguish cognitively normal individuals from those with mild cognitive impairment or dementia.

[0003] Current technologies lack the ability to distinguish between strategic reorganization pauses caused by educational background and lexical retrieval exhaustion pauses caused by cognitive decline in speech signals, in order to eliminate the confounding effect of educational background differences on early dementia screening results.

[0004] This invention constructs a multi-scale entropy spectrum of vocabulary production rate and detects local extreme peaks. By combining the adaptive segmented decay slope of the production rate sequence, silent segments are quantitatively classified into strategic recombination pauses or vocabulary retrieval exhaustion pauses. This decouples the confounding effects of educational background at the feature level, making subsequent assessments based on vocabulary retrieval exhaustion pauses more purely reflect the state of cognitive impairment and reducing the false positive rate in low-education populations. At the same time, it improves the sensitivity to early cognitive risks by using time-series indicators such as total duration and earliest occurrence position. Summary of the Invention

[0005] The purpose of this invention is to provide an information processing method and system based on speech features for early identification of dementia, in order to solve the problems mentioned above.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A speech feature-based information processing method for early dementia identification includes the following steps:

[0008] S1: Collect the continuous speech signals emitted by the subjects when they perform a word production task within a limited time, and extract the time marker of each word and the corresponding word text sequence from the continuous speech signals;

[0009] S2: Divide the time frame into multiple consecutive and non-overlapping time windows based on the time stamp, count the number of words appearing in each time window, and obtain the distribution sequence of word production rate over time.

[0010] S3: Perform multi-scale coarse-grained decomposition on the vocabulary production rate distribution sequence, calculate the sample entropy value at each coarse-grained scale to form a multi-scale entropy spectrum, and at the same time calculate the single-scale sample entropy of the vocabulary production rate distribution sequence in the entire time domain.

[0011] S4: Detect whether there are local extreme peaks in the medium-interval scale interval of the multi-scale entropy spectrum, and combine the decay slope of the single-scale sample entropy with the task progress to determine the silent segment in the time window as a strategic recombination pause or a vocabulary retrieval exhaustion pause.

[0012] S5: Accumulate the total duration and earliest occurrence time position of the vocabulary retrieval exhaustion pause, and combine them with the total vocabulary size of the vocabulary text sequence to generate a level output to indicate early cognitive risk.

[0013] As a further aspect of the present invention: S1 specifically includes:

[0014] The Hilbert-Huang transform is performed on the continuous speech signal to obtain the instantaneous frequency curve. Local abrupt change points on the instantaneous frequency curve are detected as syllable boundary points, and the continuous speech is divided into several syllable segments.

[0015] For each syllable segment, extract linear prediction coefficients, match the linear prediction coefficient vector with the reference coefficient vector stored in the pre-built acoustic prototype library using Euclidean distance, and assign the reference phoneme corresponding to the minimum distance to the corresponding syllable segment.

[0016] The sequentially arranged phoneme sequences are combined into a word text sequence, and the time of the first sampling point in the syllable segment corresponding to each word is recorded as the time stamp of the corresponding word.

[0017] As a further aspect of the present invention: S2 specifically includes:

[0018] The instantaneous sequence of word production intervals is calculated based on the difference in time stamps of adjacent words, and the mode of the instantaneous sequence is extracted as the dynamic window width base.

[0019] Using the dynamic window width base as the unit step size, slide forward along the time axis starting from the time mark of the first word. After each slide, extract a time period of fixed length equal to the dynamic window width base as a time window until the entire limited time is covered.

[0020] The total number of word timestamps falling within the corresponding time period is counted within each time window, and this number is taken as the word count for the time window.

[0021] Arrange all time windows in chronological order of their start times, and output the number of words in each window sequentially to form a sequence of word production rate distribution over time.

[0022] As a further aspect of the present invention: S3 specifically includes:

[0023] Using the scale parameter as the step size, a non-overlapping sliding window is used to truncate the vocabulary output rate distribution sequence, and the weighted median of the sequence values ​​within each window is taken as the coarse-grained representative value of the corresponding scale.

[0024] For each coarse-grained scale, the embedding dimension is fixed at twice the scale parameter, and the similarity tolerance is 5% of the dynamic fluctuation range of the sequence value. The sample entropy value at the corresponding scale is then calculated.

[0025] Arrange the sample entropy values ​​calculated at all scales in ascending order of scale to form a multi-scale entropy spectrum;

[0026] Using the original, uncoarse-grained vocabulary production rate distribution sequence as input, the single-scale sample entropy is calculated using an embedding dimension of 3 and a similarity tolerance of 10% of the sequence standard deviation.

[0027] As a further aspect of the present invention: the calculation of the sample entropy value at the corresponding scale specifically includes:

[0028] The representative value sequence at each coarse-grained scale is reorganized into an embedding vector in chronological order. The length of each embedding vector is equal to twice the current scale parameter, and each element in the vector is taken from the value at a consecutive position in the representative value sequence.

[0029] For each embedding vector, the Chebyshev distance is used to measure its spatial distance with the other embedding vectors in the sequence. The number of vectors whose distance is less than the similarity tolerance is counted, and the vectors that match themselves are excluded.

[0030] The natural logarithm of the ratio of the number of vectors counted to the total number of vectors is taken, and the arithmetic mean of the logarithmic results of all embedded vectors is then taken as the negative value, which is used as the sample entropy value at the coarse-grained scale.

[0031] As a further aspect of the present invention: S4 specifically includes:

[0032] The entropy value sequence within the third to sixth scale interval is extracted from the multi-scale entropy spectrum. The second-order central difference is calculated on the entropy value sequence. If there is a position where the difference value changes from negative to positive, it is determined that there is a local extreme peak at the corresponding position.

[0033] The vocabulary production rate distribution sequence is divided into a front segment and a back segment according to the time axis. The linear tendency of the two segments is fitted respectively. The fitting slope of the back segment is subtracted from the fitting slope of the front segment to obtain the decay slope of the sample entropy at a single scale.

[0034] When a local extreme peak exists and the decay slope is negative, the silent segment within the time window is classified as a strategic reorganization pause; when a local extreme peak does not exist and the decay slope is negative, the silent segment is classified as a vocabulary retrieval exhaustion pause.

[0035] As a further aspect of the present invention: the calculation process of the attenuation slope is as follows:

[0036] Calculate the local standard deviation sequence of the vocabulary production rate distribution sequence, detect the position of the first extreme point in the local standard deviation sequence that exceeds the global standard deviation mean, and use the extreme point position as an adaptive dividing point to divide the sequence into the front segment and the back segment;

[0037] Weighted least squares fitting is applied to the front and rear segments respectively, where the weight of each data point is inversely proportional to the rate difference of its neighboring points;

[0038] Extract the slope values ​​of the fitted straight line in the latter segment and the fitted straight line in the former segment, and calculate the difference between the two as the attenuation slope.

[0039] As a further aspect of the present invention: S5 specifically includes:

[0040] The duration of each consecutive silent segment that is determined to be a word retrieval exhaustion pause is summed to obtain the total duration, and the start time of the first silent segment in all such pauses is recorded as the earliest occurrence time position.

[0041] The total vocabulary size in the text sequence is statistically analyzed. The difference between the total vocabulary size and the upper limit of the vocabulary size of healthy subjects in the preset age norm is calculated, and the difference is normalized to the range of 0 to 1.

[0042] The total duration normalized value, the earliest occurrence time normalized value, and the difference normalized value are weighted and harmonic averaged, and the average value is matched to one of the three preset risk level intervals for output.

[0043] Information processing systems based on speech features for early dementia identification include:

[0044] The speech acquisition and vocabulary tagging module is used to acquire the continuous speech signals emitted by the subject when performing a word production task within a limited time, and to extract the time marker of each word and the corresponding word text sequence from the continuous speech signals;

[0045] The time window division and rate sequence construction module divides the limited time into multiple continuous and non-overlapping time windows based on time markers, counts the number of words appearing in each time window, and obtains the distribution sequence of word production rate over time.

[0046] The entropy spectrum and sample entropy calculation module is used to perform multi-scale coarse-grained decomposition of the vocabulary production rate distribution sequence, calculate the sample entropy value at each coarse-grained scale to form a multi-scale entropy spectrum, and at the same time calculate the single-scale sample entropy of the vocabulary production rate distribution sequence in the entire time domain.

[0047] The pause type discrimination module is used to detect whether there are local extreme peaks in the medium-interval scale interval of the multi-scale entropy spectrum, and combined with the decay slope of the single-scale sample entropy as the task progresses, to determine the silent segments within the time window as strategic recombination pauses or vocabulary retrieval exhaustion pauses.

[0048] The cognitive risk level output module is used to accumulate the total duration and earliest occurrence time position of the vocabulary retrieval exhaustion pause, and in conjunction with the total vocabulary size of the vocabulary text sequence, generate a level output to indicate early cognitive risk.

[0049] The beneficial effects of this invention are:

[0050] (1) By extracting the multi-scale entropy spectrum of word production rate from speech signals and detecting local extreme peaks, and combining the attenuation slope to distinguish between strategic reorganization pauses and word retrieval exhaustion pauses in silent segments, the interference of differences in the educational background of subjects on the screening results is effectively reduced, making it less likely that elderly people with low educational backgrounds will be misjudged as high-risk in rapid screening, thus improving the applicability and fairness of this method in different populations.

[0051] (2) Based on the combined assessment of the total duration of vocabulary retrieval exhaustion pause, the earliest occurrence time location, and the total vocabulary size, it is possible to capture early signs of vocabulary retrieval difficulties that are difficult to detect by traditional methods. Thus, risk level prompts can be output at the stage of mild cognitive impairment, providing an earlier intervention opportunity for further clinical examination. Attached Figure Description

[0052] The invention will now be further described with reference to the accompanying drawings.

[0053] Figure 1 This is a flowchart of the method of the present invention;

[0054] Figure 2 This is a system block diagram of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Please see Figure 1 As shown, this invention is an information processing method for early dementia identification based on speech features, comprising the following steps:

[0057] S1: Collect the continuous speech signals emitted by the subjects when they perform a word production task within a limited time, and extract the time marker of each word and the corresponding word text sequence from the continuous speech signals;

[0058] S2: Divide the time frame into multiple consecutive and non-overlapping time windows based on the time stamp, count the number of words appearing in each time window, and obtain the distribution sequence of word production rate over time.

[0059] S3: Perform multi-scale coarse-grained decomposition on the vocabulary production rate distribution sequence, calculate the sample entropy value at each coarse-grained scale to form a multi-scale entropy spectrum, and at the same time calculate the single-scale sample entropy of the vocabulary production rate distribution sequence in the entire time domain.

[0060] S4: Detect whether there are local extreme peaks in the medium-interval scale interval of the multi-scale entropy spectrum, and combine the decay slope of the single-scale sample entropy with the task progress to determine the silent segment in the time window as a strategic recombination pause or a vocabulary retrieval exhaustion pause.

[0061] S5: Accumulate the total duration and earliest occurrence time position of the vocabulary retrieval exhaustion pause, and combine them with the total vocabulary size of the vocabulary text sequence to generate a level output to indicate early cognitive risk.

[0062] In S1, continuous speech signals emitted by subjects during a word production task within a limited time are collected, and the time stamp of each word and the corresponding word text sequence are extracted from the continuous speech signals, specifically including:

[0063] Continuous speech signals emitted by subjects during a word production task within a limited time were collected, specifically using a directional condenser microphone and a digital recorder. The time limit was set at 60 seconds, and the word production task required subjects to name as many animals as possible. Continuous recording was performed from the moment the start instruction was given to obtain continuous speech signals for the entire task duration.

[0064] For the acquired continuous speech signal, a Hilbert-Huang transform is first performed to obtain the instantaneous frequency curve. Specifically, empirical mode decomposition (EMD) is performed on the continuous speech signal. All intrinsic mode functions (IMFs) are extracted through iterative screening, and the top three IMFs with an energy percentage exceeding 90% are selected as principal components. A Hilbert transform is then performed on each principal component, forming an analytic signal from each IMF and its own Hilbert transform result. The phase derivative of this analytic signal is calculated to obtain the instantaneous frequency curve for each component. The instantaneous frequency curves of each component are then summed according to energy weighting to synthesize the final instantaneous frequency curve. Local abrupt changes on the instantaneous frequency curve are detected as syllable boundary points. The method for determining local abrupt changes is as follows: the first-order difference sequence of the instantaneous frequency curve is calculated, and sampling points whose absolute difference value exceeds a preset threshold (0.5 times the arithmetic mean of the instantaneous frequency sequence) are found. All detected abrupt changes are arranged in ascending time order, and the speech segment between two adjacent abrupt changes constitutes a syllable segment, thus dividing the continuous speech signal into several continuous and non-overlapping syllable segments.

[0065] For each segmented syllable, its linear prediction coefficients are extracted. The extraction process employs 10th-order linear prediction analysis, specifically: the syllable segment signal is multiplied by a 30-millisecond Hamming window for windowing, and the autocorrelation function of the windowed signal is calculated. The order of the autocorrelation function ranges from 0 to 10. Then, the Levinson-Dubin recursive algorithm is used to solve the autocorrelation matrix, yielding 10 linear prediction coefficients, forming a 10-dimensional coefficient vector. An acoustic prototype library is pre-constructed. This library is constructed by collecting over 1000 standard syllable samples read aloud by healthy individuals from different genders, ages, and dialect backgrounds. For each sample, 10th-order linear prediction coefficients are also extracted. Then, the mean clustering algorithm is used to cluster all coefficient vectors into several categories equal to the number of phonemes in the local language. The center vector of each category serves as the reference coefficient vector for that phoneme and is stored in the prototype library. Subsequently, the Euclidean distance is calculated between the linear prediction coefficient vector of the syllable segment to be matched and each reference coefficient vector in the prototype library. This is done by calculating the sum of squares of the differences between the corresponding components of the two vectors and then taking the square root of the sum. The phoneme corresponding to the reference coefficient vector with the smallest Euclidean distance is selected as the matching phoneme for that syllable segment.

[0066] The sequentially arranged phoneme sequences are combined into a word text sequence, and the time stamp of each word is recorded. The Viterbi backtracking algorithm is used for combination: a dictionary is pre-built, where each word entry contains the Chinese character or word form of the word and its corresponding standard phoneme sequence; the phoneme sequence obtained in the previous steps is used as the observation sequence, and all phoneme sequences in the dictionary are used as hidden state paths. The cumulative matching score between the observation sequence and each path is calculated, and the score is obtained by summing the inverse of the Euclidean distance of the phoneme matching. Finally, the path with the highest score is backtracked, thus dividing the continuous phoneme sequence into word boundaries, and the Chinese characters or words within each boundary are read out sequentially to form the word text sequence. Simultaneously, for each identified word, the sampling time of the first sampling point in all syllable segments corresponding to that word in the original continuous speech signal is found. The time difference (in seconds) between this sampling time and the recording start time is the time stamp of that word.

[0067] In the Hilbert-Huang transform, the stopping criterion for empirical mode decomposition is a standard deviation of less than 0.2. The mean instantaneous frequency in the local mutation point detection threshold is obtained by calculating the arithmetic mean of the instantaneous frequency sequence over the entire task duration. The determination of the linear prediction coefficient order of 10 is based on the residual energy analysis of 50 test samples. When the order increases from 8 to 10, the residual energy decreases significantly, but after exceeding 10, the decrease is less than 5%, so order 10 is chosen. The number of categories in the acoustic prototype library clustering is equal to the number of standard phonemes in the local language, for example, 32 in Mandarin Chinese. The specific values ​​of these parameters are all fixed in advance through experiments and do not change with the subjects.

[0068] After the above processing, two sets of data are finally output: one set is a sequence of words in chronological order, and the other set is a time stamp in seconds for each word.

[0069] In S2, the time frame is divided into multiple consecutive, non-overlapping time windows based on time stamps. The number of words appearing within each time window is counted to obtain the distribution sequence of the word production rate over time, specifically including:

[0070] Obtain the timestamp of each word in the output, along with the start and end times of the entire time limit (60 seconds). Arrange the words in ascending order of their timestamps. Calculate the difference between the timestamps of adjacent words (i.e., subtract the timestamp of the preceding word from the timestamp of the following word), resulting in a series of differences, each representing the output interval between adjacent words. Calculate the differences for all adjacent word pairs sequentially, forming an instantaneous sequence of word output intervals. The length of this sequence is equal to the total vocabulary size minus one.

[0071] The mode value is extracted from the instantaneous sequence of vocabulary production intervals and used as the dynamic window width base. The specific extraction method for the mode value is as follows: all differences in the instantaneous sequence are grouped according to their numerical value, with each distinct difference forming a group. The frequency of each difference within each group is counted. The difference with the highest frequency is identified. If multiple differences have the same frequency and are all the highest, the minimum of these differences is taken as the dynamic window width base. If the instantaneous sequence is empty (i.e., the total vocabulary size is less than 2), the dynamic window width base is set to 1 second.

[0072] Using the dynamic window width as the unit step size, the system slides forward along the time axis, starting from the time marker of the first word. Specifically: the time marker of the first word is used as the start time of the first time window; the dynamic window width base value is added to this start time to obtain the end time of the first time window, thus extracting the first fixed-length time period. Then, the start time of the first time window is added to the dynamic window width base value to obtain the start time of the second time window, and the dynamic window width base value is added again to obtain the end time of the second time window, and so on, with each sliding step size equal to the dynamic window width base value.

[0073] Repeat the sliding capture operation described above until the entire time limit is covered. If the end time of the last time window is less than the end time of the time limit, but exceeds that end time after adding the dynamic window width base, then the end time of the time limit is directly used as the end time of the last time window. In this case, the actual length of the last time window is less than the dynamic window width base. All time windows do not overlap, and the start and end times of each time window are recorded in seconds, maintaining consistency with the precision of the time stamp.

[0074] Within each time window, the total number of word timestamps falling within that time period is counted. Specifically, for each word's timestamp, if the timestamp is greater than or equal to the start time of the current time window and less than the end time of that time window, it is counted as a word in that window. Boundary handling: Word timestamps exactly equal to the end time are moved to the next time window. After the statistics are completed, each time window corresponds to a non-negative integer representing the number of words produced within that time period.

[0075] Arrange all time windows in chronological order of their start times, and output the number of words corresponding to each time window in sequence to form a numerical sequence. This sequence is the distribution sequence of word production rate over time. The position of each value in the sequence corresponds to which time window from the start time, and the value reflects the speed of word production within that time period.

[0076] In S3, the vocabulary production rate distribution sequence is decomposed into multi-scale coarse-grained components, and the sample entropy value at each coarse-grained scale is calculated to form a multi-scale entropy spectrum. Simultaneously, the single-scale sample entropy of the vocabulary production rate distribution sequence over the entire time domain is calculated, specifically including:

[0077] A multi-scale coarse-grained decomposition is performed on the vocabulary production rate distribution sequence. The scale parameter is set to increment sequentially from 1 to 10. At each scale, a non-overlapping sliding window is used to truncate the original sequence with the current scale parameter as the step size. That is, the first window contains values ​​from the 1st to the th scale parameter, the second window contains values ​​from the th scale parameter plus 1 to the 2nd scale parameter, and so on. Within each window, the weighted median of the sequence values ​​is calculated as the coarse-grained representative value for that scale. The weighted median is calculated as follows: the sequence values ​​within the window are arranged in ascending order of numerical value, and the weight of each value is set as the ratio of its position index within the window to the window length. Then, the weights are accumulated starting from the smallest value. When the accumulated weight first reaches or exceeds 0.5, that value is the weighted median. If the last part of the window is less than a full window length, the remaining data is discarded. A representative value sequence is obtained at each scale, and the length of this sequence is equal to the original sequence length divided by the current scale parameter and rounded down.

[0078] For each coarse-grained scale, the sample entropy value for that scale is calculated. First, the dynamic fluctuation range of the sequence's values ​​is determined, specifically the difference between the maximum and minimum values ​​in the sequence. The similarity tolerance is set to 5% of this dynamic fluctuation range. The embedding dimension is fixed at twice the current scale parameter.

[0079] For the representative value sequence at the current scale, it is reorganized into an embedding vector sequence in chronological order. Specifically, starting from the first value in the sequence, two consecutive values ​​that are twice the scale parameter are taken to form the first embedding vector; then, moving one position forward, the second to the next two times the scale parameter plus one value are taken to form the second embedding vector; this process is repeated until no embedding vector of complete length can be formed at the end of the sequence. The total number of embedding vectors is equal to the length of the representative value sequence minus the two-times-scale parameter plus one.

[0080] For each embedding vector, the Chebyshev distance is used to measure its spatial distance to the remaining embedding vectors in the sequence. The Chebyshev distance is calculated by comparing the differences between the components of two vectors at the same position, and taking the maximum absolute value of the differences among all components as the distance between the two vectors. Then, the number of all other embedding vectors whose distance to this vector is less than the aforementioned similarity tolerance is counted, excluding matching the vector itself. This statistical process is repeated for each embedding vector.

[0081] After obtaining the statistical count of each embedding vector, divide this count by the total number of embedding vectors (the total number after subtracting one is usually approximated; here, we directly use the total number of embedding vectors as the denominator, but the actual denominator should be subtracted by one when excluding self-matches; for simplicity, we use the total number of embedding vectors subtracted by one) to obtain a ratio. Take the natural logarithm of this ratio, then calculate the arithmetic mean of the natural logarithms of all embedding vectors, and then take the negative value of this mean. The result is the sample entropy value at the current coarse-grained scale. Repeat the above steps for all scale parameters from 1 to 10 to obtain 10 sample entropy values, which are arranged in ascending order of scale to form a multi-scale entropy spectrum.

[0082] Simultaneously, using the original, uncoarse-grained vocabulary production rate distribution sequence as input, the single-scale sample entropy is calculated. The specific calculation method is the same as above, but the embedding dimension is fixed at 3, and the similarity tolerance is set to 10% of the standard deviation of the original sequence. The standard deviation is calculated as follows: first, calculate the arithmetic mean of all values ​​in the sequence; then, calculate the sum of squares of the differences between each value and the mean, divide by the sequence length minus one, and then take the square root. Using an embedding dimension of 3, and following the steps of embedding vector recombination, Chebyshev distance statistics, and natural logarithmic averaging with the negative, the single-scale sample entropy value is obtained.

[0083] In S4, the presence of local extreme peaks within the equal-interval scale intervals of the multi-scale entropy spectrum is detected. Combined with the decay slope of the single-scale sample entropy as the task progresses, silent segments within the time window are identified as strategic recombination pauses or vocabulary retrieval exhaustion pauses. Specifically, this includes:

[0084] An entropy value sequence from the third to the sixth scale interval is extracted from the multi-scale entropy spectrum, containing a total of four entropy values. The second-order central difference is calculated for this entropy value sequence, using the following method: for the first entropy value in the sequence... The first-order central difference of the n positions is divided into the nth position. Add 1 entropy value and subtract the first one. After subtracting one entropy value and dividing by 2, the second-order central difference is obtained by performing the same central difference operation on the first-order central difference result again. Specifically, let the truncated entropy value sequence be denoted as E3, E4, E5, and E6 in ascending order of scale. First, calculate the first-order central difference: D3 equals the difference between E4 and E2 divided by 2 (E2 does not exist here, so the boundary point is replaced by forward difference), D4 equals the difference between E5 and E3 divided by 2, and D5 equals the difference between E6 and E4 divided by 2. Then calculate the second-order central difference: S4 equals the difference between D5 and D3 divided by 2, and S5 equals the difference between D6 and D4 divided by 2 (backward difference is used when D6 does not exist). Detect adjacent points where the second-order central difference value changes from negative to positive: if there exists a point where the previous second-order central difference is negative and the next is positive, then it is determined that a local extremum peak exists at that point.

[0085] Calculate the decay slope of the single-scale sample entropy as the task progresses. This decay slope is not directly calculated from the slope of the sample entropy sequence, but is obtained through adaptive piecewise linear fitting of the vocabulary production rate distribution sequence. The specific process is as follows: Obtain the constructed vocabulary production rate distribution sequence, denoted as R sequence, with a length of L (i.e., the total number of time windows). Calculate the local standard deviation sequence of R sequence: for each position in the sequence... Take Calculate the standard deviation of all values ​​within a neighborhood centered at a value with a window width of 5 (the window width decreases as the value approaches the boundary). The standard deviation is calculated as follows: first, calculate the arithmetic mean of the values ​​within the neighborhood; then, calculate the sum of the squares of the differences between each value and the mean; divide this sum by the number of values ​​in the neighborhood minus one; finally, take the square root. Repeat this process for each position. The corresponding local standard deviations form a local standard deviation sequence. .

[0086] This task involves detecting the location of the first extreme point in a local standard deviation sequence that exceeds the global standard deviation mean. The global standard deviation mean is the sum of all... The result is the sum of all values ​​in the sequence divided by L. The method for determining extreme points is: scan from left to right. Sequence, when encountering a position ,satisfy It is also greater than its previous value. and the next value (Only one side is compared at the boundary), and If the value is greater than the aforementioned global standard deviation mean, the scan is stopped and the location is changed. As an adaptive dividing point. If no extreme point that meets the conditions is found after traversing the entire sequence, then take 1 / 2 of the sequence length (rounded to the nearest integer) as the dividing point. Using this dividing point as the boundary, the vocabulary production rate distribution sequence R is divided into a front segment (the 1st to the 1st dividing point values) and a back segment (the 1st dividing point plus 1st to the Lth values).

[0087] Weighted least squares fitting is applied to the first and second segments of the sequence. In the weighted least squares fitting, the weight of each data point is inversely proportional to the rate difference between its neighbors. Specifically, for the first segment of the sequence... Number of values Calculate the rate difference between its neighboring points: take its left adjacent point The absolute value of the difference (if it exists), plus the value of the point adjacent to its right. The absolute values ​​of the differences (if they exist) are used as the variance measure. The weight is defined as the reciprocal of the variance measure (1 weight when the variance measure is 0). Then, for each segment, the fitted straight line is formed as follows: the predicted rate is equal to the fitted slope multiplied by the time index plus the intercept. The goal of the fit is to minimize the sum of the squares of the differences between the actual and predicted rates at each data point, multiplied by the weight of that point. The expression for the fitted slope can be obtained analytically as follows: ;in, Indicates the first The time index of each time window (integer starting from 1). The weight of that point. For the weighted average time index, the calculation formula is all Multiply The sum of all weights divided by the sum of all weights; The weighted average rate is calculated using the formula for all... Multiply The sum of all weights is divided by the sum of all weights. The fitting slope of the first segment is then calculated using this formula. The fitted slope of the latter segment .

[0088] After obtaining the two slopes, the attenuation slope is defined as the fitted slope of the latter segment minus the fitted slope of the former segment, that is, the attenuation slope equals... minus The decay slope reflects the trend of word production rate over time: a negative value indicates that the decline in production rate in the later section is steeper than that in the earlier section; a positive value indicates that the rate is accelerating.

[0089] The pause type is determined. The presence of local extreme peaks in the multi-scale entropy spectrum, along with the sign of the decay slope, jointly determine the classification of silent segments within the current time window. It should be noted that the time window here is synchronized with each time window in the second step, and a silent segment refers to a period between adjacent words that is not occupied by any other word. The determination rules are as follows: when a local extreme peak exists and the decay slope is negative, the silent segment within that time window is classified as a strategic reorganization pause; when a local extreme peak does not exist and the decay slope is negative, the silent segment within that time window is classified as a word retrieval exhaustion pause. For cases where the decay slope is non-negative (i.e., zero or positive), regardless of the presence of a local extreme peak, it is not classified into either of the above two types of pauses, but is considered normal speech fluctuation. The above determination results will output a type label for the silent segment within each time window.

[0090] In S5, the total duration and earliest occurrence time position of the cumulative vocabulary retrieval exhaustion pause are combined with the total vocabulary size of the vocabulary text sequence to generate a level output indicating early cognitive risk, specifically including:

[0091] Obtain the type label of the silent segment within each time window, as well as the recorded word timestamps and word text sequences. From all silent segments identified as word retrieval exhaustion pauses, filter out consecutive silent periods. Each silent segment is defined by its start and end times; the start time is the timestamp of the preceding word, and the end time is the timestamp of the following word. Sum the duration of each such segment (end time minus start time) to obtain the total duration of the word retrieval exhaustion pause, in seconds. Simultaneously, record the start time of the first silent segment among all word retrieval exhaustion pauses, using the offset (in seconds) of this start time relative to the start time of the entire word production task as the earliest occurrence time position. If no word retrieval exhaustion pause is detected in the entire task, set the total duration to 0 seconds and the earliest occurrence time position to the total task duration (i.e., 60 seconds) as the default value.

[0092] The total vocabulary size in the text sequence was calculated, i.e., the number of all valid words spoken by the subject within 60 seconds, excluding filler words such as "uh," "that," and repeated words. A pre-defined vocabulary size norm for healthy subjects, stratified by age, was established. This norm was constructed by selecting at least 500 clinically diagnosed healthy elderly individuals without cognitive impairment, aged 60 to 90 years, divided into 5-year age groups. Vocabulary size data was collected for each group under the same word production task (speaking animal names within 60 seconds). The 85th percentile of vocabulary size for each age group was taken as the upper limit of vocabulary size for that age group. For the current subject, the upper limit of vocabulary size for their age group was determined based on their actual age. The difference between the subject's total vocabulary size and the upper limit of vocabulary size for their corresponding age group was calculated, i.e., the upper limit minus the total vocabulary size. If the difference was negative (i.e., the total vocabulary size exceeded the upper limit), the difference was set to 0.

[0093] The three indicators (total duration, earliest occurrence time position, and vocabulary size difference) are normalized to the range of 0 to 1. The normalization method is as follows: For the total duration, 30 seconds is used as the maximum reference value, and the normalized value is equal to the actual total duration divided by 30. If it exceeds 1, it is taken as 1. For the earliest occurrence time position, the normalized value is equal to the actual earliest occurrence time position divided by 60 (total task duration). The smaller this value, the earlier the abnormal pause occurs. Therefore, in subsequent calculations, its complementary value (1 minus the normalized value) is used so that the larger the value, the higher the risk. For the vocabulary size difference, which is already in the range of 0 to (the maximum possible difference for the same age group), the maximum difference observed in the history of this age group is used as the denominator for normalization. If the denominator is 0, it is directly taken as 0. Specifically, the normalized value of the vocabulary size difference is equal to the actual difference divided by the preset maximum difference for this age group (the difference between the upper limit of vocabulary size for this age group and the lowest observed value). If it exceeds 1, it is taken as 1.

[0094] The three normalized values ​​are then weighted and harmonic averaged. The weighted harmonic average is calculated as follows: three weight coefficients are set, corresponding to the normalized value of total duration, the complementary normalized value of the earliest occurrence time position, and the normalized value of vocabulary difference, respectively. The sum of the three weight coefficients is 1. The default values ​​for the weights are 0.4, 0.3, and 0.3, which can be adjusted based on clinical data in subsequent applications. The formula for calculating the weighted harmonic average is: divide each of the three normalized values ​​by the reciprocal of its corresponding weight, sum the results, and then take the reciprocal. Specifically, first calculate the first normalized value divided by (1 divided by the first weight), add the second normalized value divided by (1 divided by the second weight), and add the third normalized value divided by (1 divided by the third weight) to obtain an intermediate sum; then divide 1 by this intermediate sum to obtain the weighted harmonic average. The value ranges from 0 to 1.

[0095] Three risk level ranges are preset: low risk (weighted harmonic mean greater than or equal to 0 and less than 0.3), medium risk (greater than or equal to 0.3 and less than 0.6), and high risk (greater than or equal to 0.6 and less than or equal to 1). The calculated weighted harmonic mean is matched with the corresponding risk level, and this level is output. Simultaneously, for ease of clinical recording, the raw values ​​and weight configuration parameters of the three normalized indicators can also be output.

[0096] After the above processing, a risk level indicator (choose one from low, medium, or high) is output, which reflects the cognitive impairment tendency shown by the subject in the vocabulary production task.

[0097] Please see Figure 2 As shown, an information processing system based on speech features for early identification of dementia includes:

[0098] The speech acquisition and vocabulary tagging module is used to acquire the continuous speech signals emitted by the subject when performing a word production task within a limited time, and to extract the time marker of each word and the corresponding word text sequence from the continuous speech signals;

[0099] The time window division and rate sequence construction module divides the limited time into multiple continuous and non-overlapping time windows based on time markers, counts the number of words appearing in each time window, and obtains the distribution sequence of word production rate over time.

[0100] The entropy spectrum and sample entropy calculation module is used to perform multi-scale coarse-grained decomposition of the vocabulary production rate distribution sequence, calculate the sample entropy value at each coarse-grained scale to form a multi-scale entropy spectrum, and at the same time calculate the single-scale sample entropy of the vocabulary production rate distribution sequence in the entire time domain.

[0101] The pause type discrimination module is used to detect whether there are local extreme peaks in the medium-interval scale interval of the multi-scale entropy spectrum, and combined with the decay slope of the single-scale sample entropy as the task progresses, to determine the silent segments within the time window as strategic recombination pauses or vocabulary retrieval exhaustion pauses.

[0102] The cognitive risk level output module is used to accumulate the total duration and earliest occurrence time position of the vocabulary retrieval exhaustion pause, and in conjunction with the total vocabulary size of the vocabulary text sequence, generate a level output to indicate early cognitive risk.

[0103] The working principle of this invention is as follows: Continuous speech signals emitted by subjects during a word production task within a limited time are collected, and the time stamp and word text sequence for each word are extracted from the signal. Based on the time stamps, the limited time is divided into multiple continuous and non-overlapping time windows. The number of words within each time window is counted to obtain a distribution sequence of the word production rate over time. The distribution sequence is then subjected to multi-scale coarse-grained decomposition, and the sample entropy value at each coarse-grained scale is calculated to form a multi-scale entropy spectrum. Simultaneously, the single-scale sample entropy of the distribution sequence over the entire time domain is calculated. The presence of local extreme peaks within the equally spaced scale intervals of the multi-scale entropy spectrum is detected, and combined with the decay slope of the single-scale sample entropy as the task progresses, silent segments within the time window are identified as strategic recombination pauses or word retrieval exhaustion pauses. The total duration and earliest occurrence time position of word retrieval exhaustion pauses are accumulated, and combined with the total vocabulary size of the word text sequence, a level output indicating early cognitive risk is generated.

[0104] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. An information processing method for early identification of dementia based on speech features, characterized in that, Includes the following steps: S1: Collect the continuous speech signals emitted by the subjects when they perform a word production task within a limited time, and extract the time marker of each word and the corresponding word text sequence from the continuous speech signals; S2: Divide the time frame into multiple consecutive and non-overlapping time windows based on the time stamp, count the number of words appearing in each time window, and obtain the distribution sequence of word production rate over time. S3: Perform multi-scale coarse-grained decomposition on the vocabulary production rate distribution sequence, calculate the sample entropy value at each coarse-grained scale to form a multi-scale entropy spectrum, and at the same time calculate the single-scale sample entropy of the vocabulary production rate distribution sequence in the entire time domain. S4: Detect whether there are local extreme peaks in the medium-interval scale intervals of the multi-scale entropy spectrum, and combine the decay slope of the single-scale sample entropy with the progress of the task to determine the silent segments in the time window as strategic recombination pauses or vocabulary retrieval exhaustion pauses. When a local extreme peak is detected and the decay slope is negative, it is determined to be a strategic recombination pause; when a local extreme peak is not detected and the decay slope is negative, it is determined to be a vocabulary retrieval exhaustion pause. S5: Accumulate the total duration of all silent segments that are judged as vocabulary retrieval exhaustion pauses, and record the time offset of the start time of the first silent segment relative to the start time of the task, as the earliest time position. The total duration, the earliest occurrence time position, and the total vocabulary size of the word text sequence are normalized and then weighted and fused. Based on the fusion result, a preset risk level range is matched to generate a level output to indicate early cognitive risk.

2. The information processing method for early dementia identification based on speech features according to claim 1, characterized in that, S1 specifically includes: The Hilbert-Huang transform is performed on the continuous speech signal to obtain the instantaneous frequency curve. Local abrupt change points on the instantaneous frequency curve are detected as syllable boundary points, and the continuous speech is divided into several syllable segments. For each syllable segment, extract linear prediction coefficients, match the linear prediction coefficient vector with the reference coefficient vector stored in the pre-built acoustic prototype library using Euclidean distance, and assign the reference phoneme corresponding to the minimum distance to the corresponding syllable segment. The sequentially arranged phoneme sequences are combined into a word text sequence, and the time of the first sampling point in the syllable segment corresponding to each word is recorded as the time stamp of the corresponding word.

3. The information processing method for early dementia identification based on speech features according to claim 1, characterized in that, S2 specifically includes: The instantaneous sequence of word production intervals is calculated based on the difference in time stamps of adjacent words, and the mode of the instantaneous sequence is extracted as the dynamic window width base. Using the dynamic window width base as the unit step size, slide forward along the time axis starting from the time mark of the first word. After each slide, extract a time period of fixed length equal to the dynamic window width base as a time window until the entire limited time is covered. The total number of word timestamps falling within the corresponding time period is counted within each time window, and this number is taken as the word count for the time window. Arrange all time windows in chronological order of their start times, and output the number of words in each window sequentially to form a sequence of word production rate distribution over time.

4. The information processing method for early dementia identification based on speech features according to claim 1, characterized in that, S3 specifically includes: Using the scale parameter as the step size, a non-overlapping sliding window is used to truncate the vocabulary output rate distribution sequence, and the weighted median of the sequence values ​​within each window is taken as the coarse-grained representative value of the corresponding scale. For each coarse-grained scale, the embedding dimension is fixed at twice the scale parameter, and the similarity tolerance is 5% of the dynamic fluctuation range of the sequence value. The sample entropy value at the corresponding scale is calculated. Arrange the sample entropy values ​​calculated at all scales in ascending order of scale to form a multi-scale entropy spectrum; Using the original, uncoarse-grained vocabulary production rate distribution sequence as input, the single-scale sample entropy is calculated using an embedding dimension of 3 and a similarity tolerance of 10% of the sequence standard deviation.

5. The information processing method for early dementia identification based on speech features according to claim 4, characterized in that, The calculation of the sample entropy value at the corresponding scale specifically includes: The representative value sequence at each coarse-grained scale is reorganized into an embedding vector in chronological order. The length of each embedding vector is equal to twice the current scale parameter, and each element in the vector is taken from the value at a consecutive position in the representative value sequence. For each embedding vector, the Chebyshev distance is used to measure its spatial distance with the other embedding vectors in the sequence. The number of vectors whose distance is less than the similarity tolerance is counted, and the vectors that match themselves are excluded. The natural logarithm of the ratio of the number of vectors counted to the total number of vectors is taken, and the arithmetic mean of the logarithmic results of all embedded vectors is then taken as the negative value, which is used as the sample entropy value at the coarse-grained scale.

6. The information processing method for early dementia identification based on speech features according to claim 1, characterized in that, S4 specifically includes: The entropy value sequence within the third to sixth scale interval is extracted from the multi-scale entropy spectrum. The second-order central difference is calculated on the entropy value sequence. If there is a position where the difference value changes from negative to positive, it is determined that there is a local extreme peak at the corresponding position. The vocabulary production rate distribution sequence is divided into a front segment and a back segment according to the time axis. The linear tendency of the two segments is fitted respectively. The fitting slope of the back segment is subtracted from the fitting slope of the front segment to obtain the decay slope of the single-scale sample entropy. When a local extreme peak exists and the decay slope is negative, the silent segment within the time window is classified as a strategic reorganization pause; when a local extreme peak does not exist and the decay slope is negative, the silent segment is classified as a vocabulary retrieval exhaustion pause.

7. The information processing method for early dementia identification based on speech features according to claim 6, characterized in that, The calculation process for the attenuation slope is as follows: Calculate the local standard deviation sequence of the vocabulary production rate distribution sequence, detect the position of the first extreme point in the local standard deviation sequence that exceeds the global standard deviation mean, and use the extreme point position as an adaptive dividing point to divide the sequence into the front segment and the back segment; Weighted least squares fitting is applied to the front and rear segments respectively, where the weight of each data point is inversely proportional to the rate difference of its neighboring points; Extract the slope values ​​of the fitted straight line in the latter segment and the fitted straight line in the former segment, and calculate the difference between the two as the attenuation slope.

8. The information processing method for early dementia identification based on speech features according to claim 1, characterized in that, S5 specifically includes: The duration of each consecutive silent segment that is determined to be a word retrieval exhaustion pause is summed to obtain the total duration, and the start time of the first silent segment in all such pauses is recorded as the earliest occurrence time position. The total vocabulary size in the text sequence is statistically analyzed. The difference between the total vocabulary size and the upper limit of the vocabulary size of healthy subjects in the preset age norm is calculated, and the difference is normalized to the range of 0 to 1. The total duration normalized value, the earliest occurrence time normalized value, and the difference normalized value are weighted and harmonic averaged, and the average value is matched to one of the three preset risk level intervals for output.

9. An information processing system for early identification of dementia based on speech features, characterized in that, The information processing method for early dementia identification based on speech features as described in any one of claims 1-8 includes: The speech acquisition and vocabulary tagging module is used to acquire the continuous speech signals emitted by the subject when performing a word production task within a limited time, and to extract the time marker of each word and the corresponding word text sequence from the continuous speech signals; The time window division and rate sequence construction module divides the limited time into multiple continuous and non-overlapping time windows based on time markers, counts the number of words appearing in each time window, and obtains the distribution sequence of word production rate over time. The entropy spectrum and sample entropy calculation module is used to perform multi-scale coarse-grained decomposition of the vocabulary production rate distribution sequence, calculate the sample entropy value at each coarse-grained scale to form a multi-scale entropy spectrum, and at the same time calculate the single-scale sample entropy of the vocabulary production rate distribution sequence in the entire time domain. The pause type discrimination module is used to detect whether there are local extreme peaks in the medium-interval scale interval of the multi-scale entropy spectrum, and combined with the decay slope of the single-scale sample entropy as the task progresses, to determine the silent segments within the time window as strategic recombination pauses or vocabulary retrieval exhaustion pauses. The cognitive risk level output module is used to accumulate the total duration and earliest occurrence time position of the vocabulary retrieval exhaustion pause, and in conjunction with the total vocabulary size of the vocabulary text sequence, generate a level output to indicate early cognitive risk.

Images