An intelligent neuropsychological scale voice collection and analysis method and system based on a multi-factor adaptive time window regulation

Abstract

The application discloses an intelligent neuropsychological scale voice collection and analysis method and system based on multi-factor adaptive time window regulation. The method first collects information of a subject, records a current remaining time pool, calculates a previous performance dynamic adjustment factor, and calculates an initial time window of the subject in an MMSE test system. Then, the initial time window is adjusted to obtain an adaptive time window, the question is presented, and the voice recording and timing are started. Then, the adaptive time window is dynamically extended or compensated by using a time window dynamic extension mechanism based on real-time voice activity detection and a time window additional compensation mechanism based on real-time cognitive load indicators. Finally, the final actual recording time window is recorded, the voice quality is evaluated, and all questions are completed. The system comprises a front-end interactive collection module, a multi-factor time window dynamic calculation engine module, a real-time voice activity detection module, a back-end voice analysis service module and a global state management and process control module.

Description

Technical Field

[0001] This invention relates to the field of speech signal processing, specifically to an intelligent neuropsychological scale speech acquisition and analysis method and system based on multi-factor adaptive time window regulation. Background Technology

[0002] The Mini-Mental State Examination (MMSE) is one of the most widely used cognitive function screening tools in clinical practice. It was first proposed by Folstein et al. in 1975 and covers 11 cognitive subdomains, including time orientation, place orientation, immediate memory, attention and calculation, delayed recall, language ability, and visuospatial construction. The full score is 30 points and the test is usually completed in 5-10 minutes.

[0003] However, the following core technical challenges exist in traditional MMSE administration:

[0004] (1) Time window missing problem: Neither the original MMSE scale nor its standardized version (SMMSE) set a clear upper limit for answering each question, which means that the test takers need to rely on their personal experience to judge when to stop waiting and move on to the next question. This subjective judgment seriously affects the standardization of the test and cross-test taker consistency.

[0005] (2) Difficulty in adapting to individual differences: Different subjects have significantly different reasonable time requirements for answering each question due to factors such as age, education level, degree of cognitive impairment, native language background and sensorimotor function degeneration (such as hearing loss and speech delay). A fixed and uniform time setting will inevitably lead to some subjects having insufficient time and producing a floor effect, or excessive time and increasing ineffective waiting.

[0006] (3) The challenge of matching the timing of speech acquisition and analysis: When automatic speech recognition (ASR) technology is introduced to assist in the administration of MMSE, the precise control of the start and end time of recording becomes a key technical bottleneck. If the recording time is too short, the effective response speech of the subject may be truncated; if the recording time is too long, a large number of silent segments and background noise will be introduced, which will seriously affect the quality of downstream speech analysis and the efficiency of system operation.

[0007] (4) Insufficient ease of operation: The common subjects of MMSE are the elderly, many of whom have difficulty operating smart devices. Existing technical solutions usually require the test administrator or subject to manually control the start and end of each question, which poses a significant operational obstacle for elderly patients with cognitive impairment and non-professional caregivers.

[0008] Currently, a small number of literatures and patents have involved the digitization and automation of cognitive assessment, including tablet-based cognitive testing interfaces, speech recognition-assisted cognitive assessment, etc. However, none of the existing technical solutions propose a multi-factor adaptive dynamic adjustment method for the answer time window of each item in the MMSE, let alone establish a unified time window calculation framework that integrates multi-dimensional factors such as the subject's real-time voice activity status, cognitive load estimation, inherent difficulty characteristics of items and individual demographic attributes. Summary of the Invention

[0009] The object of the present invention is to provide an intelligent neuropsychological scale voice collection and analysis method based on multi-factor adaptive time window adjustment, comprising the following steps:

[0010] Step 1) Collect subject information, and calculate a static adjustment factor according to the subject information;

[0011] Step 2) Record the current remaining time pool , and calculate the pre-performance dynamic adjustment factor for the i-th item; the initial value of i is 1; i=1,2,...,N;

[0012] Step 3) Calculate the initial time window of the i-th item for the subject in the MMSE testing system according to the static adjustment factor and the pre-performance dynamic adjustment factor;

[0013] Step 4) Based on the current remaining time pool , adjust the initial time window to obtain an adaptive time window;

[0014] Step 5) Present the item and start recording and timing;

[0015] Step 6) Dynamically extend or compensate the adaptive time window by using a time window dynamic extension mechanism based on real-time voice activity detection and a time window additional compensation mechanism based on real-time cognitive load indicators;

[0016] Step 7) Stop recording after the termination condition is met, and record the final actual recording time window;

[0017] Step 8) Evaluate the current voice quality, determine whether a retry is triggered, if yes, return to step 5), otherwise, proceed to step 9);

[0018] Step 9) Determine whether i<N holds, if yes, let i=i+1, return to step 2), otherwise, end the recording.

[0019] Further, in step 1), the subject information includes the subject's age, years of education, and native language / dialect background;

[0020] The static adjustment factors include an age adjustment factor, an education level adjustment factor, a language background adjustment factor and an item cognitive load factor.

[0021] Furthermore, the age adjustment factors are as follows:

[0022] (1)

[0023] In the formula, This is the maximum age compensation coefficient. This refers to the steepness parameter of the Sigmoid curve. The median turning point age. The subject's age;

[0024] The moderating factors for education level are shown below:

[0025] (2)

[0026] In the formula, The maximum education compensation coefficient, The number of years of education of the subjects, This is the decay rate parameter;

[0027] The language background adjustment factors are as follows:

[0028] (3)

[0029] In the formula, For non-native language test indicator functions, For non-native language compensation coefficient, For dialect compensation coefficient, Dialectal distance index;

[0030] The cognitive load factors for the questions are shown below:

[0031] (4)

[0032] In the formula, , , , These are the standardized scores for the memory load, attention load, executive function load, and language output load of the i-th question, respectively. , , , All are weighting coefficients.

[0033] Furthermore, in step 2), the preceding performance dynamic adjustment factor is as follows:

[0034] (5)

[0035] In the formula, For the maximum dynamic adjustment range, For sensitivity parameters, It is the hyperbolic tangent function. Let be the cumulative performance deviation index for the i-th question;

[0036] The cumulative performance bias index for question i is shown below:

[0037] (6)

[0038] In the formula, i is the index of the current question; The actual time spent answering question j The time window is allocated for the j-th question.

[0039] Furthermore, in step 3), the initial time window for the i-th question is as follows:

[0040] (7)

[0041] In the formula, The baseline answering time for the i-th question As an age-regulating factor, As a moderating factor for education level, As a language background modifier, For the cognitive load factor of the topic, For the preceding dynamic adjustment factor;

[0042] When the i-th question has multiple sub-steps for answering, , The time window for the k-th response sub-step is:

[0043] (8)

[0044] In the formula, For the time window of the first sub-step, The attenuation rate, For the first The random perturbation term of the step.

[0045] Furthermore, in step 4), the adaptive time window is as follows:

[0046] (9)

[0047] In the formula, For redistribution sensitivity coefficient, For the remaining time pool;

[0048] The remaining time pool is shown below:

[0049] (10)

[0050] In the formula, For the final actual recording time window of question j, Let j be the initial time window for the j-th question. For global time budget;

[0051] The overall time budget is as follows:

[0052] (11)

[0053] In the formula, This is the global reserve coefficient.

[0054] Furthermore, in step 6), the time window dynamic extension mechanism based on real-time voice activity detection is as follows:

[0055] Step 6.1) The current time t reaches the initial time window. At this time, the system checks the real-time voice activity density. If the real-time voice activity density is greater than the voice activity threshold, the extension mode is activated, and the time window is extended by a preset time step each time. Seconds, and continuous monitoring;

[0056] In extended mode, whenever a new extended checkpoint is reached, the real-time voice activity density is reassessed. If it is still greater than the voice activity threshold, extended mode is activated again; otherwise, a mute countdown begins.

[0057] Step 6.2) The mute countdown continues for a preset mute time step. If there is no new voice activity during the mute countdown, the recording of the current question is terminated and the process automatically moves to the next question; if new voice activity is detected during the mute countdown, proceed to step 6.3).

[0058] Step 6.3) Reset the countdown and return to step 6.2).

[0059] The real-time voice activity density calculation is shown below:

[0060] (12)

[0061] In the formula, For the width of the sliding window, For a moment The frame-by-frame speech activity detection binary output, where t is the current time. The time variable is within the integration interval;

[0062] The time-window-based supplementary compensation mechanism based on real-time cognitive load indicators is as follows:

[0063] When real-time cognitive load estimation metrics Exceeding the preset threshold At that time, the system will add compensation to the remaining time window of the current question, that is:

[0064] (13)

[0065] In the formula, This is the cognitive load compensation gain coefficient;

[0066] The real-time cognitive load estimation metrics are shown below:

[0067] (14)

[0068] In the formula, This represents the percentage of time spent speaking and remaining silent. Due to speech rate deviation, The fundamental frequency variation coefficient, , , These are the weighting coefficients for the proportion of speech and silence time, speech rate deviation, and fundamental frequency variation coefficient, respectively.

[0069] Furthermore, in step 7), the final actual recording time window is shown below:

[0070] (15)

[0071] In the formula, The actual number of extensions that occurred. The time consumed for the final silent determination Preset time step;

[0072] The conditions for stopping recording are as follows:

[0073] (16)

[0074] In the formula, For maximum elongation ratio, This is the maximum time limit for a single question globally.

[0075] In step 8), the current speech quality is evaluated using the speech clarity index. When the speech clarity index is lower than the quality threshold, the system prompts the subject to answer the current question again or supplement the answer.

[0076] The speech clarity metrics are as follows:

[0077] (17)

[0078] In the formula, Let be the signal-to-noise ratio estimation function. For estimating the overlap of multiple people speaking simultaneously, For speech integrity estimation.

[0079] The system using any of the intelligent neuropsychological scale speech acquisition and analysis methods based on multi-factor adaptive time window regulation includes a front-end interactive acquisition module, a multi-factor time window dynamic calculation engine module, a real-time speech activity detection module, a back-end speech analysis service module, and a global state management and process control module.

[0080] The front-end interactive acquisition module is responsible for the presentation of questions, real-time voice acquisition, and user interface interaction of the MMSE administration system.

[0081] The multi-factor time window dynamic calculation engine module generates adaptive answer time window and recording time window parameters for each question in real time.

[0082] The real-time speech activity detection module is based on streaming speech signal analysis of the subject's speech activity state to trigger the dynamic expansion or contraction of the time window.

[0083] The backend speech analysis service module performs transcription, semantic scoring, and cognitive feature extraction on the collected speech segments;

[0084] The global state management and process control module is used to coordinate the data flow and control logic between various modules.

[0085] A computer-readable storage medium having a computer program stored thereon;

[0086] When the computer program is invoked, it executes any of the steps of the intelligent neuropsychological scale speech acquisition and analysis method based on multi-factor adaptive time window regulation.

[0087] The technical effects of this invention are undeniable. In the speech recognition-based MMSE intelligent testing system, this invention automatically generates personalized optimal answering and recording time windows for each question. This allows for multi-factor adaptive adjustment based on the subject's demographic characteristics, real-time speech activity, previous answering performance, and the inherent cognitive load requirements of the question. This enables precise control of recording start and end times and automatic progression of the testing process without manual intervention, ultimately improving the standardization, ease of operation, and quality of speech data acquisition in the MMSE test. The main beneficial effects of this invention are as follows:

[0088] (1) Standardization improvement: By replacing subjective human judgment with a multi-factor mathematical model, the time control differences among testers are eliminated, which significantly improves the standardization and repeatability of MMSE administration.

[0089] (2) Personalized adaptation: The integration of five-dimensional moderating factors (age, education, language, cognitive load, dynamic performance) enables the time window to accurately match the individual characteristics and real-time status of each subject, effectively reducing the floor effect and ineffective waiting.

[0090] (3) Zero threshold for operation: The fully automated time window control and recording start and end management make the testing process free from manual intervention in timing, greatly reducing the professional requirements of the testers, and is particularly suitable for community screening, primary healthcare and remote testing scenarios.

[0091] (4) Data quality optimization: The intelligent recording strategy based on RT-VAD and the voice quality adaptive enhancement mechanism ensure the effectiveness and clarity of the collected voice, and improve the accuracy of downstream AI analysis.

[0092] (5) System self-evolution: The time window parameter write-back mechanism enables the system to continuously optimize the baseline parameters as the data used accumulates, forming a positive feedback self-evolution closed loop. Attached Figure Description

[0093] Figure 1 This is a schematic diagram of the overall system architecture of the present invention;

[0094] Figure 2 This is a flowchart of the multi-factor time window calculation of the present invention;

[0095] Figure 3 This is a sequence diagram of the single-item test administration of the present invention;

[0096] Figure 4 This is a schematic diagram illustrating the global time budget constraint and inter-question time reallocation of the present invention. Detailed Implementation

[0097] The present invention will be further described below with reference to embodiments, but it should not be construed that the scope of the present invention is limited to the following embodiments. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the above-described technical concept of the present invention should be included within the scope of protection of the present invention.

[0098] Example 1:

[0099] See Figures 1 to 4 A method for intelligent neuropsychological scale speech acquisition and analysis based on multi-factor adaptive time window regulation includes the following steps:

[0100] Step 1) Collect subject information and calculate the static regulation factor based on the subject information;

[0101] Step 2) Record the current remaining time pool and calculating a pre-order performance dynamic regulation factor of the i-th question; the initial value of i is 1; i=1,2,...,N;

[0102] Step 3) calculating an initial time window of the i-th question of the subject in the MMSE test administration system according to the static regulation factor and the pre-order performance dynamic regulation factor;

[0103] Step 4) based on the current remaining time pool , adjusting the initial time window to obtain an adaptive time window;

[0104] Step 5) presenting the question and starting recording and timing;

[0105] Step 6) dynamically extending or compensating the adaptive time window by using a time window dynamic extension mechanism based on real-time voice activity detection and a time window additional compensation mechanism based on real-time cognitive load indicators;

[0106] Step 7) stopping recording after a termination condition is reached, and recording a final actual recording time window;

[0107] Step 8) evaluating current voice quality, judging whether retesting is triggered, if yes, returning to step 5), otherwise, entering step 9);

[0108] Step 9) judging whether i<N is true, if yes, setting i=i+1 and returning to step 2), otherwise, ending recording.

[0109] Example 2:

[0110] The main structure of this example is the same as that of Example 1. Further, in step 1), the subject information includes the subject's age, years of education, and native language / dialect background;

[0111] The static regulation factors include an age regulation factor, an education level regulation factor, a language background regulation factor and a question cognitive load factor.

[0112] Example 3:

[0113] The main structure of this example is the same as any one of Examples 1 to 2. Further, the age regulation factor is as follows:

[0114] ;(1)

[0115] In the formula, is the maximum age compensation coefficient, is the steepness parameter of the Sigmoid curve, is the median turning age, is the age of the subject;

[0116] The education level regulation factor is as follows:

[0117] (2)

[0118] In the formula, The maximum education compensation coefficient, The number of years of education of the subjects, This is the decay rate parameter;

[0119] The language background adjustment factors are as follows:

[0120] (3)

[0121] In the formula, For non-native language test indicator functions, For non-native language compensation coefficient, For dialect compensation coefficient, Dialectal distance index;

[0122] The cognitive load factors for the questions are shown below:

[0123] (4)

[0124] In the formula, , , , These are the standardized scores for the memory load, attention load, executive function load, and language output load of the i-th question, respectively. , , , All are weighting coefficients.

[0125] Example 4:

[0126] The main structure of this embodiment is the same as any one of embodiments 1-3. Further, in step 2), the preceding dynamic adjustment factor is as follows:

[0127] (5)

[0128] In the formula, For the maximum dynamic adjustment range, For sensitivity parameters, It is the hyperbolic tangent function. Let be the cumulative performance deviation index for the i-th question;

[0129] The cumulative performance bias index for question i is shown below:

[0130] (6)

[0131] In the formula, i is the index of the current question; The actual time spent answering question j The time window is allocated for the j-th question.

[0132] Example 5:

[0133] The main structure of this embodiment is the same as any one of embodiments 1 to 4. Further, in step 3), the initial time window of the i-th question is as follows:

[0134] (7)

[0135] In the formula, The baseline answering time for the i-th question As an age-regulating factor, As a moderating factor for education level, As a language background modifier, For the cognitive load factor of the topic, For the preceding dynamic adjustment factor;

[0136] When the i-th question has multiple sub-steps for answering, , The time window for the k-th response sub-step is:

[0137] (8)

[0138] In the formula, For the time window of the first sub-step, The attenuation rate, For the first The random perturbation term of the step.

[0139] Example 6:

[0140] The main structure of this embodiment is the same as any one of embodiments 1-5. Further, in step 4), the adaptive time window is as follows:

[0141] (9)

[0142] In the formula, For redistribution sensitivity coefficient, For the remaining time pool;

[0143] The remaining time pool is shown below:

[0144] (10)

[0145] In the formula, For the final actual recording time window of question j, Let j be the initial time window for the j-th question. For global time budget;

[0146] The overall time budget is as follows:

[0147] (11)

[0148] In the formula, This is the global reserve coefficient.

[0149] Example 7:

[0150] The main structure of this embodiment is the same as any one of embodiments 1-6. Further, in step 6), the dynamic extension mechanism of the time window based on real-time voice activity detection is as follows:

[0151] Step 6.1) The current time t reaches the initial time window. At this time, the system checks the real-time voice activity density. If the real-time voice activity density is greater than the voice activity threshold, the extension mode is activated, and the time window is extended by a preset time step each time. Seconds, and continuous monitoring;

[0152] In extended mode, whenever a new extended checkpoint is reached, the real-time voice activity density is reassessed. If it is still greater than the voice activity threshold, extended mode is activated again; otherwise, a mute countdown begins.

[0153] Step 6.2) The mute countdown continues for a preset mute time step. If there is no new voice activity during the mute countdown, the recording of the current question is terminated and the process automatically moves to the next question; if new voice activity is detected during the mute countdown, proceed to step 6.3).

[0154] Step 6.3) Reset the countdown and return to step 6.2).

[0155] The real-time voice activity density calculation is shown below:

[0156] (12)

[0157] In the formula, The width of the sliding window; For a moment The frame-by-frame speech activity detection binary output is set to 1 for speech activity and 0 for silence. For a moment Real-time speech activity density, used to characterize recent speech activity density. The average proportion of voice activity within a time window; At the current assessment point, It is the time variable within the integration interval.

[0158] The time-window-based supplementary compensation mechanism based on real-time cognitive load indicators is as follows:

[0159] When real-time cognitive load estimation metrics Exceeding the preset threshold At that time, the system will add compensation to the remaining time window of the current question, that is:

[0160] (13)

[0161] In the formula, This is the cognitive load compensation gain coefficient;

[0162] The real-time cognitive load estimation metrics are shown below:

[0163] (14)

[0164] In the formula, This represents the percentage of time spent speaking and remaining silent. Due to speech rate deviation, The fundamental frequency variation coefficient, , , These are the weighting coefficients for the proportion of speech and silence time, speech rate deviation, and fundamental frequency variation coefficient, respectively.

[0165] Example 8:

[0166] The main structure of this embodiment is the same as any one of embodiments 1 to 7. Further, in step 7), the final actual recording time window is as follows:

[0167] (15)

[0168] In the formula, The actual number of extensions that occurred. The time consumed for the final silent determination Preset time step;

[0169] The conditions for stopping recording are as follows:

[0170] (16)

[0171] In the formula, For maximum elongation ratio, This is the maximum time limit for a single question globally.

[0172] In step 8), the current speech quality is evaluated using the speech clarity index. When the speech clarity index is lower than the quality threshold (default value: 0.60; implementation range: [0.55, 0.70]), the system prompts the subject to answer the current question again or supplement the answer.

[0173] The speech clarity metrics are as follows:

[0174] (17)

[0175] In the formula, Let be the signal-to-noise ratio estimation function. For estimating the overlap of multiple people speaking simultaneously, For speech integrity estimation.

[0176] Example 9:

[0177] The system using any one of the embodiments 1 to 8, which is based on the intelligent neuropsychological scale speech acquisition and analysis method with multi-factor adaptive time window regulation, includes a front-end interactive acquisition module, a multi-factor time window dynamic calculation engine module, a real-time speech activity detection module, a back-end speech analysis service module, and a global state management and process control module.

[0178] The front-end interactive acquisition module is responsible for the presentation of questions, real-time voice acquisition, and user interface interaction of the MMSE administration system.

[0179] The multi-factor time window dynamic calculation engine module generates adaptive answer time window and recording time window parameters for each question in real time.

[0180] The real-time speech activity detection module is based on streaming speech signal analysis of the subject's speech activity state to trigger the dynamic expansion or contraction of the time window.

[0181] The backend speech analysis service module performs transcription, semantic scoring, and cognitive feature extraction on the collected speech segments;

[0182] The global state management and process control module is used to coordinate the data flow and control logic between various modules.

[0183] Example 10:

[0184] A computer-readable storage medium having a computer program stored thereon;

[0185] When the computer program is invoked, it executes the steps of the method described in any one of Embodiments 1-8.

[0186] Example 11:

[0187] 1 System Overall Architecture

[0188] The system proposed in this invention comprises the following core modules: (a) a front-end interactive acquisition module, responsible for MMSE question presentation, real-time voice acquisition, and user interface interaction; (b) a multi-factor time window dynamic calculation engine (MFTW-Engine), which generates adaptive answer time window and recording time window parameters for each question in real time; (c) a real-time speech activity detection module (RT-VAD), which analyzes the subject's speech activity state based on streaming speech signals to trigger the dynamic expansion or contraction of the time window; (d) a back-end speech analysis service module, which performs transcription, semantic scoring, and cognitive feature extraction on the acquired speech segments; and (e) a global state management and flow control module, which coordinates the data flow and control logic between the modules. A schematic diagram of the overall system architecture of this invention is shown below. Figure 1 As shown.

[0189] 2. Core Algorithm of Multi-Factor Time Window Dynamic Calculation Engine (MFTW-Engine)

[0190] The core innovation of this invention lies in the multi-factor fusion time window calculation framework adopted by MFTW-Engine, as shown in the flowchart below. Figure 2 As shown. For the i-th question in MMSE (i = 1, 2, ..., N, where N is the total number of questions), its initial adaptive time window... Calculated using the following formula:

[0191] ……(1)

[0192] The factors are defined as follows:

[0193] 2.1 Baseline Time for the Question

[0194] The baseline response time for the i-th question is determined by the MMSE question type and cognitive domain. This invention sets baseline time parameters for each of the 11 cognitive subdomains of the MMSE. For example: 8 seconds per question for time orientation questions, 12 seconds per step for calculation questions (continuous subtraction of 7), 6 seconds per question for naming questions, 10 seconds for sentence retelling questions, and 30 seconds for pentagon copying questions. This baseline time was obtained through statistical analysis of test data from a large-scale healthy elderly population, with the 75th percentile response time used as the baseline.

[0195] 2.2 Age-modifying factors

[0196] The age moderating factor is modeled using a piecewise sigmoid function to capture the non-linear decay characteristics of cognitive processing speed with age.

[0197] …(2)

[0198] in This is the maximum age compensation coefficient (default value is 0.8). This is the steepness parameter for the Sigmoid curve (default value is 0.12). The median transition age is set to 72 years (default). This function ensures that: the age adjustment factor for subjects under 60 years of age is close to 1.0 (with almost no additional time compensation); time compensation begins to increase significantly around age 72; and subjects over 85 years of age receive near-maximum compensation. ≈ 1.8).

[0199] 2.3 Moderating Factors of Education Level

[0200] The impact of education level on MMSE performance has been widely confirmed. This invention uses an inverse logarithmic function for modeling:

[0201] ……(3)

[0202] in The number of years of education of the subjects, This is the maximum education compensation coefficient (default value is 0.6). This is the decay rate parameter (default value is 0.15). This function results in: subjects with very low years of education (such as illiterate individuals or those with only 1-2 years of education) receiving greater time compensation; subjects with more than 12 years of education have a compensation coefficient approaching zero.

[0203] 2.4 Language Background Moderating Factor γ (lang)

[0204] When the test language is not the subject's native language, or when the subject's dialect differs from the standard test language, additional language processing time is required to compensate for the language.

[0205] ……(4)

[0206] in Apply test indicator functions for non-native languages ​​(yes = 1, no = 0). This is the non-native language compensation coefficient (default value is 0.3). This is the dialect distance index (value 0-1). This is the dialect compensation coefficient (default value is 0.2). The dialect distance index is based on the pre-defined phonetic distance between the administered language and the subject's everyday language or obtained from a pre-administration questionnaire.

[0207] 2.5 Cognitive Load Factor of Items

[0208] Different MMSE questions require different types and depths of cognitive processing. This invention proposes a cognitive load factor quantification model:

[0209] ……(5)

[0210] in , , , These are the standardized scores (range [0.5, 2.0]) for the memory load, attention load, executive function load, and language output load of the i-th question. , , , These are weighting coefficients (satisfying ∑w_j = 1). The workload score for each question is calibrated by cognitive psychology experts and fixed in the system parameter table. For example, the attention load of the "continuous subtraction of 7" calculation question... and execution function load Set higher values ​​(such as 1.8 and 1.6), while the language output load of naming problems... Set a higher value (e.g., 1.5).

[0211] 2.6 Precursor Performance Dynamic Regulation Factors

[0212] A key innovation of this invention is the introduction of the subject's real-time performance on preceding items in the current test as the basis for dynamic adjustment of the time window for subsequent items. The Cumulative Performance Index (CPI) is defined as follows:

[0213] ……(6)

[0214] in The actual time spent answering question j The time window is allocated for the j-th question. A value >0 indicates that the overall tendency of the subjects to complete the task beyond the time limit. < 0 indicates that the subjects generally tend to complete the task ahead of schedule.

[0215] The preceding performance moderating factor is calculated using the following formula:

[0216] ……(7)

[0217] in This is the maximum dynamic adjustment range (default value is 0.4). This is the sensitivity parameter (default value is 2.5). The function ensures that the adjustment range is bounded and has saturation characteristics, avoiding unreasonable expansion or compression of the time window due to extreme deviations.

[0218] 3. Real-time voice activity detection-driven dynamic time window extension mechanism

[0219] In the initial time window Building upon this foundation, the present invention designs a dynamic time window extension mechanism based on real-time speech activity detection (RT-VAD). The core idea of ​​this mechanism is that as long as the subject continues to produce meaningful speech activity, the system should automatically extend the recording and waiting time, rather than mechanically truncating it when the initial time window expires.

[0220] Define the real-time speech activity density function :

[0221] ……(8)

[0222] in For a moment The frame-by-frame speech activity detection output is a binary value (1 = speech activity, 0 = silence). The width of the sliding window (default value is 2.0 seconds).

[0223] The decision-making rules for extending the time window are as follows:

[0224] When the current time t reaches the initial time window At that time, the system checks :

[0225] - like (If the voice activity threshold is set to 0.3 by default), the extended mode will be activated, and the time window will be extended by Δext seconds each time (3.0 seconds by default), and monitoring will continue.

[0226] - In extended mode, reassess whenever a new extended checkpoint is reached. If it is still greater than If it continues to extend, it will enter a silent countdown.

[0227] - Continuous mute countdown The countdown timer is set to 3.0 seconds (default value is 3.0 seconds). If no new audio activity occurs during the countdown, the recording of the current question will be terminated and the process will automatically proceed to the next question. If a new audio activity is detected during the countdown, the countdown will be reset.

[0228] The final actual recording time window T_i^(final) is described by the following formula:

[0229] ……(9)

[0230] in The actual number of extensions that occurred. The time consumed for the final silent determination.

[0231] To prevent the time window from extending indefinitely under abnormal circumstances, this invention sets a global hard upper limit:

[0232] ……(10)

[0233] in This is the maximum stretch ratio (default is 3.0). This is the maximum time limit for a single question globally (default is 120 seconds). Figure 3 This is a timing diagram for administering a single test item.

[0234] 4. Real-time estimation of cognitive load and fine-tuning of time window

[0235] This invention further proposes a real-time cognitive load estimation method based on speech signals, serving as an auxiliary basis for dynamic fine-tuning of the time window. A real-time cognitive load index is defined. (Real-time Cognitive Load):

[0236] …(11)

[0237] in:

[0238] - The speech-to-silence ratio is the percentage of time spent in speech and silence during a response. Under high cognitive load, subjects tend to pause more frequently. reduce;

[0239] - Speech rate bias: The ratio of the current speech rate to the individual baseline speech rate established by the subject during the registration memory phase (MMSE Question 1 "Please remember three words"). Speech rate decreases significantly under high cognitive load.

[0240] - The fundamental frequency variation coefficient reflects the degree of monotonicity in intonation; under high cognitive load, intonation tends to be flatter. reduce.

[0241] when Exceeding the preset threshold At that time, the system will add compensation to the remaining time window for the current question:

[0242] ……(12)

[0243] in This is the cognitive load compensation gain factor (default value is 0.15). This compensation is superimposed on the VAD-driven extension to ensure that subjects under high cognitive load conditions have additional response time.

[0244] 5. Multi-stage time window decay scheduling strategy

[0245] For questions with multiple sub-steps (such as "subtracting 7 consecutively" which requires 5 subtraction operations), this invention proposes a stepped time window decay scheduling strategy. The time window for the k-th sub-step in the i-th question is defined as follows:

[0246] ……(13)

[0247] in For the time window of the first sub-step, The decay rate (default value is 0.85, reflecting the acceleration as subjects gradually enter the calculation state after the "priming effect" fades). For the k-th step, there is a random perturbation term used to prevent mechanized rhythm. It follows a truncated normal distribution with a mean of 0 and a standard deviation of 0.5 seconds.

[0248] 6. Global test time budget constraints and inter-question time reallocation

[0249] To ensure the entire MMSE test is completed within a reasonable total time, this invention introduces a global time budget constraint mechanism. A diagram illustrating the global time budget constraint and the time reallocation between questions is shown below. Figure 4 As shown. Define the global time budget T_budget:

[0250] ……(14)

[0251] in The global reserve coefficient (default value is 0.25) means that 25% of the elastic time is reserved above the sum of all initial time windows.

[0252] During the test, if the cumulative actual time taken for previous questions is significantly less than the allocated time, the saved time can be redistributed to subsequent questions. Define a remaining time pool. (Remaining Time Pool):

[0253] ……(15)

[0254] when When the time limit is > 0, subsequent questions can obtain additional time compensation from the time pool; when When the time window is less than 0, the time window for subsequent questions needs to be compressed proportionally to ensure that the total time remains controllable:

[0255] ……(16)

[0256] in The redistribution sensitivity coefficient (default value is 0.6) is set to ensure that the redistribution intensity is moderate and will not cause drastic fluctuations in the time window of subsequent questions.

[0257] 7 Adaptive Recording Enhancement Strategies for Voice Quality

[0258] To improve the data quality of backend speech analysis, this invention proposes an adaptive speech quality enhancement strategy within the recording time window. The signal-to-noise ratio estimation function SNR_est(t) and the speech clarity index Clarity(t) are defined as follows:

[0259] (17)

[0260] in For estimating the overlap of multiple people speaking simultaneously, For speech integrity estimation (based on ASR intermediate decoding confidence). When Below the quality threshold During the transition between questions, the system automatically inserts a retry prompt, inviting the test taker to answer the current question again or supplement the answer. However, this retry does not affect the scoring logic (it is only used to obtain higher quality voice samples for analysis).

[0261] 8 System Workflow

[0262] Step S1: Test initialization - Collect the subject's demographic information (age, years of education, native language / dialect background) and calculate the static moderating factor using formulas (2)-(4).

[0263] Step S2: Test-by-question loop - For the i-th question (i=1,2,...,N), execute:

[0264] S2.1: Calculate the initial time window using formula (1) (First question) =1.0; Subsequent questions will be dynamically updated using formulas (6) and (7). ).

[0265] S2.2: Present the question and start recording and timing.

[0266] S2.3: Run the RT-VAD module (Equation (8)) and RCL estimation module (Equation (11)) in real time, and dynamically extend or compensate the time window according to Equations (9) and (12).

[0267] S2.4: After the termination condition (VAD mute judgment or hard limit) is met, stop recording and record. .

[0268] S2.5: Perform a speech quality assessment (Formula (17)) to determine whether to trigger a retry.

[0269] S2.6: Update (Formula (6)) and (Formula (15)) prepares the parameters for the next question.

[0270] Step S3: Test Complete - Send all collected speech segments to the backend analysis service for ASR transcription, semantic scoring, and cognitive feature extraction. Generate MMSE total score and subdomain score reports.

[0271] Step S4: Time Window Parameter Write-back - Incorporate the actual time taken in this test into the system parameter database for use as a reference time. Continuous calibration and optimization.

[0272] Example 12:

[0273] Community-based cognitive screening for the elderly. The system of this invention was deployed in a community health service center. The subject was a 78-year-old woman with 3 years of education, whose native language was Sichuan dialect, and the testing language was Mandarin. The system automatically calculated various moderating factors: (Formula 2), (Formula 3), = 1.18 (Formula 4, non-native language administration + dialect distance 0.4). The initial value is 1.0. For questions like "What is today's date?" (time-oriented questions),... = 8s, δ = 0.9), system calculation = 8 × 1.56 × 1.38 × 1.18 × 0.9 × 1.0 = 18.3 seconds. The subject took 16.2 seconds to answer, and the recording captured the entire process of the subject saying "Um... today is..." followed by a 5-second pause before continuing the answer. The system's RT-VAD module correctly identified the pause as "thinking silence" (…). lower than However, it was still above the absolute silence threshold of 0.05, and did not trigger early termination.

[0274] Example 13:

[0275] A multi-step question involving "continuous subtraction of 7". The subject was a 68-year-old male with 16 years of education. For the question set "100 minus 7 equals what, then subtract 7 again", the system calculated the first step's time window. = 14.2 seconds (including a baseline of 12 seconds and adjustments for various factors), subsequent steps decay by ρ = 0.85: = 12.1 seconds, = 10.3 seconds, = 8.7 seconds, = 7.4 seconds (Formula 13). The subject experienced a longer pause (8 seconds) in step 3, and the RCL estimation module detected a significant decrease in speech rate. )and Decrease (0.31), Exceeding the threshold The system automatically added a 2.1-second compensation (Formula 12), and the subject ultimately completed step 3 within 12.4 seconds. The CPI was +0.15 after the update in step 4. Adjusted to version 1.06, slightly increasing the time allocation for subsequent steps.

[0276] Example 14:

[0277] Remote home testing scenario. The subject takes the MMSE test at home via a tablet app, with a family member assisting in operating the device. The system detects background television sound and occasional conversations from the family member. The voice quality monitoring module (Formula 17) detects this during question 6, "Please name this item." (below the threshold) ), the reason is (Family members are also being prompted). During the transition between questions, the system automatically prompts, "Please answer this question again in a quiet environment," and marks the retry recording as supplementary sampling, without affecting the original scoring process. When processing the audio for this question, the backend analysis service prioritizes using the clearer retry recording for ASR transcription.

Claims

1. A method for intelligent neuropsychological scale speech acquisition and analysis based on multi-factor adaptive time window regulation, characterized in that, Comprising the following steps: Step 1) collecting subject information, and calculating a static adjustment factor according to the subject information; Step 2) Record the current remaining time pool And calculate the dynamic adjustment factor of the preceding performance of the i-th question; the initial value of i is 1; i = 1, 2, ..., N; Step 3) calculating an initial time window of the i-th question of a subject in an MMSE test system according to the static adjustment factor and a pre-performance dynamic adjustment factor; Step 4) Based on the current remaining time pool The initial time window is adjusted to obtain an adaptive time window; Step 5) presenting a question and starting recording and timing; Step 6) dynamically extending or compensating an adaptive time window by using a time window dynamic extension mechanism based on real-time voice activity detection and a time window additional compensation mechanism based on a real-time cognitive load index; Step 7) stopping recording after a termination condition is met, and recording a final actual recording time window; Step 8) evaluating current voice quality, judging whether a retry is triggered, returning to step 5) if the retry is triggered, and otherwise entering step 9); Step 9) judging whether i<N is true, setting i=i+1 and returning to step 2) if i<N is true, and otherwise ending recording.

2. The intelligent neuropsychological scale speech acquisition and analysis method based on multi-factor adaptive time window regulation according to claim 1, characterized in that, In step 1), the subject information comprises the age, education years, and native language / dialect background of the subject; The static adjustment factors comprise an age adjustment factor, an education level adjustment factor, a language background adjustment factor and a question cognitive load factor.

3. The intelligent neuropsychological scale speech acquisition and analysis method based on multi-factor adaptive time window regulation according to claim 2, characterized in that, The age adjustment factor is as shown below: ;(1) In the formula, This is the maximum age compensation coefficient. This refers to the steepness parameter of the Sigmoid curve. The median turning point age. The subject's age; The education level adjustment factor is as shown below: ;(2) In the formula, The maximum education compensation coefficient, The number of years of education of the subjects, This is the decay rate parameter; The language background adjustment factor is as shown below: ;(3) In the formula, For non-native language test indicator functions, For non-native language compensation coefficient, This is the dialect compensation coefficient. Dialectal distance index; The question cognitive load factor is as shown below: ;(4) In the formula, , , , These are the standardized scores for the memory load, attention load, executive function load, and language output load of the i-th question, respectively. , , , All are weighting coefficients.

4. The intelligent neuropsychological scale speech acquisition and analysis method based on multi-factor adaptive time window regulation according to claim 3, characterized in that, In step 2), the pre-performance dynamic adjustment factor is as shown below: ;(5) In the formula, For the maximum dynamic adjustment range, For sensitivity parameters, It is the hyperbolic tangent function. Let be the cumulative performance deviation index for the i-th question; The cumulative performance deviation index of the i-th question is as shown below: ;(6) In the formula, i is the index of the current question; The actual time spent answering question j The time window is allocated for the j-th question.

5. The intelligent neuropsychological scale speech acquisition and analysis method based on multi-factor adaptive time window regulation according to claim 4, characterized in that, In step 3), the initial time window of the i-th question is as shown below: ;(7) In the formula, The baseline answering time for the i-th question As an age-regulating factor, As a moderating factor for education level, As a language background modifier, For the cognitive load factor of the topic, This serves as a dynamic adjustment factor for the preceding performance. When the i-th question has multiple sub-steps for answering, , The time window for the k-th response sub-step is: ;(8) In the formula, For the time window of the first sub-step, The attenuation rate, For the first The random perturbation term of the step.

6. The intelligent neuropsychological scale speech acquisition and analysis method based on multi-factor adaptive time window regulation according to claim 5, characterized in that, In step 4), the adaptive time window is as shown below: ;(9) In the formula, For redistribution sensitivity coefficients, For the remaining time pool; The remaining time pool is as shown below: ;(10) In the formula, For the final actual recording time window of question j, Let j be the initial time window for the j-th question. For global time budget; The global time budget is as shown below: ;(11) In the formula, This is the global reserve coefficient.

7. The intelligent neuropsychological scale speech acquisition and analysis method based on multi-factor adaptive time window regulation according to claim 6, characterized in that, In step 6), the time window dynamic extension mechanism based on real-time voice activity detection is as shown below: Step 6.1) The current time t reaches the initial time window. At this time, the system checks the real-time voice activity density. If the real-time voice activity density is greater than the voice activity threshold, the extension mode is activated, and the time window is extended by a preset time step each time. Seconds, and continuous monitoring; In an extension mode, whenever a new extension checkpoint is reached, the real-time voice activity density is re-evaluated; if the real-time voice activity density is still greater than a voice activity threshold, the extension mode is continuously started, otherwise, mute countdown is entered; Step 6.2) the mute countdown lasts for a preset mute time step; if no new voice activity occurs during the mute countdown, recording of the current question is terminated and the process automatically advances to the next question; if new voice activity is detected during the mute countdown, the process enters step 6.3); Step 6.3) resetting the countdown, and returning to step 6.2); The calculation of real-time voice activity density is as shown below: ;(12) In the formula, For the width of the sliding window, For a moment The frame-by-frame speech activity detection binary output, where t is the current time. The time variable is within the integration interval; The time window additional compensation mechanism based on the real-time cognitive load index is as shown below: When real-time cognitive load estimation metrics Exceeding the preset threshold At that time, the system will add compensation to the remaining time window of the current question, that is: ;(13) In the formula, This is the cognitive load compensation gain coefficient; The real-time cognitive load estimation index is as shown below: ;(14) In the formula, This represents the percentage of time spent speaking and remaining silent. Due to speech rate deviation, The fundamental frequency variation coefficient, , , These are the weighting coefficients for the proportion of speech and silence time, speech rate deviation, and fundamental frequency variation coefficient, respectively.

8. The intelligent neuropsychological scale speech acquisition and analysis method based on multi-factor adaptive time window regulation according to claim 7, characterized in that, In step 7), the final actual recording time window is as shown below: ;(15) In the formula, The actual number of extensions that occurred. The time consumed for the final silent determination Preset time step; The termination condition for stopping recording is as shown below: ;(16) In the formula, For maximum elongation ratio, This is the maximum time limit for a single question globally. In step 8), the current voice quality is evaluated by a voice clarity index; when the voice clarity index is lower than a quality threshold, the system prompts to invite the subject to re-answer or supplement the answer to the current question; The voice clarity index is as shown below: ;(17) In the formula, Let be the signal-to-noise ratio estimation function. Estimating the overlap of multiple people speaking simultaneously. For speech integrity estimation.

9. A system using the intelligent neuropsychological scale speech acquisition and analysis method based on multi-factor adaptive time window regulation as described in any one of claims 1-8, characterized in that, Comprising a front-end interactive acquisition module, a multi-factor time window dynamic calculation engine module, a real-time voice activity detection module, a back-end voice analysis service module and a global state management and process control module; The front-end interactive acquisition module is responsible for question presentation, real-time voice acquisition and user interface interaction of the MMSE test system; The multi-factor time window dynamic calculation engine module generates adaptive answer time window and recording time window parameters for each question in real time. The real-time speech activity detection module is based on streaming speech signal analysis of the subject's speech activity state to trigger the dynamic expansion or contraction of the time window. The backend speech analysis service module performs transcription, semantic scoring, and cognitive feature extraction on the collected speech segments. The global state management and process control module is used to coordinate the data flow and control logic between various modules.

10. A computer-readable storage medium, characterized in that, It contains computer programs; When the computer program is invoked, it performs the steps of the method according to any one of claims 1-8.

Images