Dynamic cognitive training method and system based on large language model and multi-dimensional evaluation

Abstract

The invention relates to the technical field of data processing, and discloses a dynamic cognitive training method and system based on a large language model and multi-dimensional evaluation, which combines multi-dimensional input data such as standardized cognitive evaluation, demographic statistics, historical behaviors and real-time performance to construct a user cognitive portrait. The dynamic prompting engineering module and the content generation process of the large language model are integrated, so that the training content, the prompting mode, the interaction style and the like can deeply meet the individual differences and specific requirements of users, the fine-grained and deep personalized training process is provided, and through the dynamic prompting engineering module, the user experience is improved. According to the method, the fine cognitive state change of the user in each interaction and task can be captured in real time, the change is immediately converted into accurate control over large language model behaviors, training content and difficulty which are accurately matched with the current level of the user are generated, highly-dynamic and real-time adaptive cognitive training can be achieved, and the user experience is improved. The problems that a traditional cognitive training system is lagged in adjustment and insufficient in roughness are solved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a dynamic cognitive training method and system based on large language models and multi-dimensional evaluation. Background Technology

[0002] Cognitive training, as one of the main non-pharmacological interventions for the early intervention of neurodegenerative diseases such as Alzheimer's disease, is beneficial in delaying disease progression and improving patients' quality of life. However, existing cognitive training methods and systems have the following shortcomings:

[0003] Training content and difficulty are static or adjusted based on simple rules: Existing cognitive training methods and systems only provide preset training task sets or make rough adjustments to difficulty based on users' simple scores on a few tasks. This makes it difficult to adapt to users' rapidly changing cognitive states and specific difficulties in real time, resulting in limited training effectiveness. Users may find the tasks too easy or too difficult, affecting compliance.

[0004] Insufficient personalization: Existing cognitive training methods and systems mainly rely on the overall scores of a few standardized assessments (such as MoCA) to set the initial training plan. They fail to make full use of richer user data, such as detailed cognitive report sub-item scores, demographic information, past medical history, and subtle performance in real-time interactions, to build a comprehensive user cognitive profile. As a result, it is difficult to provide training content and strategies that truly match individual characteristics and needs.

[0005] Limited system flexibility and content generation capabilities: Existing cognitive training methods and systems typically rely on pre-set question banks or content templates, resulting in poor scalability. New training tasks or content require manual design and addition, which is costly, inefficient, and makes it difficult to quickly respond to different training needs and user situations.

[0006] This invention provides a dynamic cognitive training method and system based on large language models and multi-dimensional evaluation to solve the above-mentioned problems. Summary of the Invention

[0007] The purpose of this invention is to provide a dynamic cognitive training method and system based on a large language model and multi-dimensional evaluation, so as to solve the problems that traditional cognitive training methods and systems are not easy to adjust in terms of training content and difficulty, and have insufficient personalization.

[0008] This invention is achieved through the following technical solution:

[0009] The dynamic cognitive training method based on large language models and multi-dimensional evaluation includes the following steps:

[0010] Step A1: After the user logs in, collect the user's multidimensional raw data and process it to obtain heterogeneous standardized multidimensional data;

[0011] Step A2: Perform data fusion and feature extraction on the heterogeneous standardized multidimensional data to obtain multidimensional input data and its feature parameters;

[0012] Step A3: Process the results of data fusion and feature extraction to obtain fusion features, determine the user's cognitive profile based on the fusion features, and at the same time perform cognitive evaluation to obtain the user's current instantaneous cognitive state, and obtain the user cognitive state vector composed of the cognitive profile and cognitive state.

[0013] Step A4: Using the user's cognitive state vector, feature parameters in the multidimensional input data, and the type and objective of the training task as input, construct a policy library for reasoning and decision-making on the input to select and execute the optimal Prompt string generation strategy. Based on the constructed policy library, dynamically prepare the Prompt string instructions to be sent to the large language model.

[0014] Step A5: Receive dynamic Prompt string instructions and input them into the large language model to generate training task content or natural language responses to user input;

[0015] Step A6: Transform the training task content into a user-operable training task interface, receive various user interactions until the user completes the task, evaluate the task completion status to obtain real-time performance data, and provide immediate feedback.

[0016] Step A7: Iterate through steps A1-A6 to continuously and dynamically generate training tasks until the training duration is reached. Transfer the user's behavior data from the previous task to the multidimensional raw data of the next task, and integrate the real-time performance data into the multidimensional input data during the next task.

[0017] Furthermore, in step A3, the user's long-term profile includes the baseline ability and long-term trend of cognitive ability levels corresponding to each cognitive domain.

[0018] Furthermore, in step A3, the instantaneous cognitive state is obtained by analyzing the user's real-time behavioral interaction data in the multidimensional input data according to the following rules:

[0019] The specific mapping rule based on condition-state is as follows:

[0020] Transient cognitive state classification rules:

[0021] By analyzing and judging the real-time interaction data of users through the correspondence between preset conditions and cognitive states, the corresponding cognitive state judgment will be triggered when the following situations are detected:

[0022] Cognitive fatigue: If a user's reaction time continues to increase across multiple consecutive task points, while the task accuracy rate shows a downward trend, then the user may be in a state of "cognitive fatigue".

[0023] Lack of confidence: If a user requests prompts on a certain knowledge point more than the set number of times within the first preset time, or does not take any action at a key decision point within the second preset time, the user is determined to be in a state of "lack of confidence".

[0024] Knowledge Barrier: If a user makes the same type of error on the same task type or similar knowledge points within a third preset time, it is determined that the user has encountered a "knowledge barrier".

[0025] Decreased focus: If a user interrupts multiple times, pauses for a long time, or inputs irrelevant information within the fourth preset time during the interaction, it is determined that the user may be in a state of "distraction".

[0026] Transient Cognitive State Classification Rule: The transient cognitive state classification rule determines the user's transient cognitive state through a machine learning model and feature analysis. This rule is implemented using a dynamic machine learning model (e.g., a pre-trained classifier) ​​deployed in the system. This model uses the user's historical interaction data as a training set and can learn and identify more complex behavioral patterns than a single rule. It uses real-time multi-dimensional inputs (such as accuracy, reaction time, error type, dependence on prompts, and even acoustic features extracted from speech, such as pitch and speech rate) as a feature vector, comprehensively analyzing the dynamic changes and complex combinations of these indicators. In this way, the model can directly output labels for one or more transient cognitive states (e.g., "highly focused," "distracted," "knowledge mastery," "exploration," etc.) along with corresponding confidence scores.

[0027] Furthermore, in step A4, the established strategy library includes:

[0028] The rule base establishes a difficulty reduction trigger threshold and a difficulty reduction strategy. When a user's real-time performance data reaches the difficulty reduction trigger threshold, the Prompt string instruction is modified through the difficulty reduction strategy command Prompt generator.

[0029] The model library contains lightweight supervised or reinforcement learning models that predict the optimal combination of prompt features in real time based on the current user performance and historical trajectory, and output policy weights for the prompt policy module to call, for predicting and classifying user task performance and instantaneous cognitive state.

[0030] The algorithm library is equipped with an adaptive optimization algorithm, which is used to learn and iteratively optimize the combination of long-term Prompt string instruction generation strategies for specific users.

[0031] Furthermore, in step A4, a prompt type strategy is also established to accurately determine and decide what type of prompt to provide based on the user's current real-time status.

[0032] Furthermore, in step A5, the dynamic Prompt string instruction is combined with multidimensional input data as context information and input into the large language model.

[0033] Furthermore, the implementation of step A6 specifically includes the following steps:

[0034] Step B1: Transform the training task content generated by the large language model into a user-operable training task interface;

[0035] Step B2: Capture various user interactions on the training task interface;

[0036] Step B3: Establish different evaluation mechanisms based on the nature of the task and conduct evaluations to obtain user evaluation results on task completion.

[0037] Step B4: Provide real-time feedback to users based on the evaluation results.

[0038] Furthermore, during the implementation of step B3, the specific evaluation mechanism includes:

[0039] For tasks with definite answers, the user's input is directly compared with the preset standard answer. If the comparison is consistent, it is considered a complete match; otherwise, it is considered a mismatch.

[0040] For open-ended or text generation tasks, the following methods are used for evaluation:

[0041] The text similarity-based evaluation method represents the user-input answer text and the pre-stored reference answer text as numerical vectors, calculates the similarity score between the answer text vector and the reference answer text vector, and uses the similarity score to evaluate the semantic closeness between the answer text and the reference answer text.

[0042] The evaluation method based on a large language model involves sending the user's complete answer, the corresponding original task instructions, and the evaluation prompt string to the large language model. The evaluation prompt string instructs the large language model to analyze from the required dimensions and requests it to return the evaluation results in structured data. The evaluation results include quantitative scores and qualitative comments.

[0043] The basic solution of this invention also provides a dynamic cognitive training system based on a large language model and multi-dimensional evaluation, used to implement the dynamic cognitive training method based on a large language model and multi-dimensional evaluation described above, including the following modules:

[0044] The data acquisition and preprocessing module is used to collect multidimensional raw data from users and preprocess the multidimensional raw data to obtain multidimensional input data.

[0045] The multidimensional data integration module is used to perform data fusion and feature extraction on heterogeneous standardized multidimensional data to obtain multidimensional input data, and extract feature parameters for driving the dynamic prompting engineering module or large language model for personalized optimization.

[0046] The cognitive profiling and assessment module is used to build and maintain a user's cognitive profile through multi-dimensional input data, while performing cognitive assessment to obtain the user's current cognitive state and obtain a user cognitive state vector composed of the cognitive profile and cognitive state.

[0047] The dynamic prompting engineering module is used to receive the user's cognitive state vector, feature parameters in multi-dimensional input data, and the current training task type and target to dynamically prepare and send the Prompt string instruction to the large language model;

[0048] The large language model interaction and content generation module is used to receive dynamic prompt string instructions generated by the dynamic prompt engineering module, send them to the large language model, and receive training task content or natural language responses to user input generated by the large language model, and format them.

[0049] The training task execution and feedback module is used to receive the training task content generated by the formatted large language model, transform it into a user-operable training task interface, receive various user interactions, obtain user evaluation results on task completion, and provide real-time feedback to the data acquisition and preprocessing module.

[0050] The bimodal interaction module provides a user interface that allows users to input via voice and text, and to receive text and voice output from a large language model.

[0051] Data storage module: Used for persistent storage of all types of user data.

[0052] Furthermore, the cognitive profiling and evaluation module is equipped with an instantaneous cognitive state evaluation engine. The instantaneous cognitive state evaluation engine has a fast evaluation model based on self-heuristic rules and a dynamic machine learning model. The fast evaluation model has several pre-set condition-to-state mapping rules, and the dynamic machine learning model has instantaneous cognitive state classification rules.

[0053] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0054] 1. This invention can combine multi-dimensional input data such as MoCA sub-items, ADNI class reports, demographics, historical behavior, and real-time performance to construct a comprehensive user cognitive profile. Moreover, the aforementioned multi-dimensional input data is not only used for initial evaluation, but is also technically integrated into the dynamic prompting engineering module and the content generation process of the large language model. This allows the training content, prompting methods, and interaction styles to deeply match the individual differences and specific needs of users, rather than simply classifying them based on a total score. This provides a fine-grained and in-depth personalized training process.

[0055] 2. This invention, through a dynamic prompting engineering module, can capture subtle changes in the user's cognitive state in every interaction and task in real time, and immediately transform these changes into precise control over the behavior of the large language model, generating training content and difficulty that are precisely matched to the user's current level. This enables highly dynamic and real-time adaptive cognitive training, overcoming the problems of lagging adjustment and insufficient coarseness in traditional cognitive training systems.

[0056] 3. This invention leverages the powerful generative capabilities of a large language model to generate entirely new, non-preset training task content in real time. Simultaneously, the dynamic prompting engineering module's precise control over the large language model ensures that the generated tasks meet specific cognitive training duration and difficulty requirements, significantly improving the diversity and flexibility of training content. This greatly enriches the training content, prevents users from becoming bored with repetitive content, and allows the system to quickly adapt to new training theories or needs. Attached Figure Description

[0057] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0058] Figure 1 This is a flowchart of the dynamic cognitive training method based on a large language model and multi-dimensional evaluation proposed in an embodiment of this application;

[0059] Figure 2 This is a schematic diagram of the overall modular architecture of the dynamic cognitive training system based on a large language model and multi-dimensional evaluation proposed in the embodiments of this application;

[0060] Figure 3 This is a flowchart illustrating the cognitive profile construction and maintenance process proposed in this application embodiment;

[0061] Figure 4The flowcharts for the processing input and output of the bimodal interaction module proposed in the embodiments of this application are shown, wherein (a) is a flowchart for the processing output of the bimodal interaction module, and (b) is a flowchart for the processing input of the bimodal interaction module. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0063] Dynamic cognitive training methods based on large language models and multi-dimensional evaluation are implemented, for example... Figure 1 As shown, it includes the following steps:

[0064] Step A1: After user login, collect the user's raw multidimensional data, and obtain heterogeneous standardized multidimensional data after preprocessing. The difficulty and content of the new user's first task are adjusted once by DPEF based on the multidimensional assessment data provided during registration, rather than using a uniform standard difficulty directly.

[0065] The user's multidimensional raw data was collected from various data channels, including assessment report data and background data such as demographic data. The collected user multidimensional raw data includes:

[0066] Standardized cognitive assessment data includes scores for each sub-item of the MoCA scale manually entered by assessors, and neuropsychological assessment scores extracted from scanned external assessment reports via OCR. The MoCA scale sub-item scores include naming, memory, attention, language, abstraction, orientation, and visuospatial reasoning, while the neuropsychological assessment scores include scores for each cognitive domain obtained from external assessment reports: simulated ADNI reports.

[0067] Demographic and background data, including age, education level, occupation, language preference, hobbies, medical history, etc. provided by the user during registration or setup.

[0068] Furthermore, during subsequent user tasks, multidimensional raw data can be collected from user interaction data to obtain user behavior data, including:

[0069] User's historical cognitive training data includes the user's previous cognitive training records, which include the task type, difficulty, score, time taken, and error patterns of the tasks the user has completed.

[0070] Real-time user interaction data includes detailed user actions captured and recorded in real time during the execution of the current cognitive training task. These detailed user actions include the speed of text input, the number of modifications, the use of specific words, or the duration, speech rate, pauses, volume, and emotion of voice input (preliminarily analyzed through voice emotion recognition technology), or the total time taken to complete a subtask, or the degree of dependence on prompts or repetitions, or the error patterns of the mistakes made (whether they are conceptual errors, execution errors, or memory errors, etc.).

[0071] The processing steps for the aforementioned multidimensional raw data are specifically preprocessing steps, including but not limited to: data cleaning, including removing outliers, handling missing values, format conversion, and basic standardization. For example, for the collected user voice data, after data cleaning, the voice input needs to be transcribed using ASR and the non-linguistic features of the voice need to be extracted. For example, after speech-to-text conversion, in addition to the text content itself, the original waveform or acoustic features of the speech also need to be recorded.

[0072] Step A2: Perform data fusion and feature extraction on heterogeneous standardized multidimensional data to obtain multidimensional input data and its feature parameters, which are the parameters in the multidimensional input data that can be personalized for optimization.

[0073] Step A3: Process the results of data fusion and feature extraction to obtain fused features. The fused features refer to a comprehensive set of parameters extracted from standardized multidimensional data and real-time task performance data, which can simultaneously reflect the user's long-term / static characteristics and dynamic / behavioral characteristics. Based on this, a user's cognitive profile is determined based on the fused features, and a cognitive assessment is performed to obtain the user's current instantaneous cognitive state. The cognitive profile and the instantaneous cognitive state together constitute the user's cognitive state vector.

[0074] Furthermore, in subsequent user tasks, after receiving real-time performance data from the previous task, this data will be added to the multi-dimensional input data to help determine the user's cognitive profile.

[0075] Specifically, such as Figure 3 As shown, the user's cognitive profile is determined based on the analysis results of standardized cognitive assessment data, demographic and background data, and the user's historical cognitive training data. The final result is a cognitive profile that reflects the user's cognitive ability level in each cognitive domain, including the baseline ability and long-term trend of the cognitive ability level in each cognitive domain.

[0076] In this embodiment, the instantaneous state of a new user upon initial login is obtained by using static data provided during registration and a task of basic difficulty to acquire the initial state (which can be understood as the instantaneous cognitive state at the initial moment). Subsequently, the instantaneous cognitive state is obtained by analyzing the user's real-time behavioral interaction data in the multi-dimensional input data according to the following rules:

[0077] Condition-state mapping rules: The system analyzes and judges users' real-time interaction data by establishing a correspondence between preset conditions and cognitive states. When the following conditions are detected, a corresponding cognitive state judgment will be triggered:

[0078] 1. Cognitive fatigue: If a user's reaction time continues to increase across multiple consecutive task points, while the task accuracy rate shows a downward trend, then the user may be in a state of "cognitive fatigue".

[0079] 2. Lack of confidence: If a user frequently requests hints on a certain knowledge point, or does not take any action for a long time at a key decision point, the user is judged to be in a state of "lack of confidence".

[0080] 3. Knowledge Barrier: If a user consistently makes the same type of error (such as a conceptual error or an execution error) on the same task type or similar knowledge points (several knowledge points are considered similar if their similarity exceeds a set threshold), then the user is determined to have encountered a "knowledge barrier".

[0081] 4. Decreased focus: If users frequently interrupt, pause for long periods, or input irrelevant information during the interaction, it is determined that the user may be in a state of "distraction".

[0082] Based on the above rules, the system can quickly perform preliminary analysis and state classification of the user's current interaction performance without complex calculations, providing a basis for subsequent dynamic cognitive state assessment and task adjustment.

[0083] Transient cognitive state classification rules: The system uses users' historical interaction data as a training set, combined with real-time collected multi-dimensional input data, and determines users' transient cognitive states through machine learning models and feature analysis methods. Specifically, it includes the following steps:

[0084] Input data acquisition: Collect multidimensional feature data of users during task execution, including task accuracy, reaction time, error type, degree of reliance on prompts, as well as the pitch, speech rate, pauses, and emotional features of voice input.

[0085] Feature processing and analysis: The collected data is standardized and vectorized to form feature vectors that can comprehensively represent the user's current behavior.

[0086] Model determination: The feature vector is input into a pre-trained classification model (such as support vector machine, random forest or deep neural network). The model predicts the user's instantaneous cognitive state based on the patterns learned from historical training samples.

[0087] Label generation and confidence score output: The model outputs one or more instantaneous cognitive state labels and provides corresponding confidence scores to characterize the reliability of the prediction results. For example: "Highly focused" (confidence score 0.92).

[0088] State Application: Through the above analysis and tag generation process, the system can transform abstract behavioral manifestations into explicit cognitive state tags, which can then be used as inputs for subsequent cognitive assessment and task dynamic adjustment modules based on the user's current instantaneous cognitive state.

[0089] Step A4: Using the user's cognitive state vector, feature parameters in the multidimensional input data, and the type and objective of the training task as input, construct a policy library for reasoning and decision-making on the input to select and execute the optimal Prompt string generation strategy, and dynamically prepare the Prompt string instruction to be sent to the large language model based on the constructed policy library.

[0090] In this step, the user's cognitive state vector, feature parameters from the multidimensional input data, and the type and objective of the current training task are taken as input. The output is one or a set of optimized Prompt strings used to drive the large language model to generate specific training task content or perform specific interactions. During this process, a state encoding mechanism (state encoding module) is established. This mechanism, designed in-house, mainly includes the following steps: receiving the user's cognitive state vector and multidimensional feature parameters → encoding the above data into a standardized parameter set → generating state instructions that can be recognized by the large language model (e.g., 'short-term memory is declining, the current task is memory-related' is encoded as a set of specific parameters or flags), and dynamically adjusting the Prompt string instructions in subsequent tasks.

[0091] In this step, a policy library is built for reasoning and decision-making on the input to select and execute the optimal Prompt string generation strategy. Based on this policy library, the Prompt string instructions are dynamically prepared. This policy library includes:

[0092] The rule base establishes thresholds and strategies for reducing difficulty. When a user's real-time performance data reaches the threshold for reducing difficulty, the Prompt string is modified by the Prompt generator using methods such as "simplifying task instructions," "reducing information density," or "providing explicit cues." For example, if a user's task reaction time is greater than a preset time threshold and their accuracy is less than a preset accuracy threshold, the difficulty reduction strategy is triggered, and the Prompt string is modified by the Prompt generator using methods such as "simplifying task instructions," "reducing information density," or "providing explicit cues." Simultaneously, thresholds and strategies for increasing difficulty are also established. When a user's real-time performance data reaches the corresponding threshold, the system dynamically adjusts the Prompt instructions using appropriate strategies, such as "increasing task complexity" or "extending task steps" when increasing difficulty.

[0093] A model library integrating machine learning or deep learning models for predicting and classifying user task performance and cognitive states. Example: A pre-trained classification model (such as a random forest or convolutional neural network) can be used to comprehensively determine the most effective training strategy based on the user's historical performance, task sequences, and physiological indicators, and then pass the strategy label to the Prompt generator.

[0094] This algorithm library incorporates adaptive optimization algorithms such as reinforcement learning, enabling users to autonomously learn and iteratively optimize long-term Prompt string instruction generation strategy combinations for specific users, aiming to maximize long-term training effectiveness. Example: A reinforcement learning algorithm based on Deep Q-Learning (DQN) can be used, employing user task completion rate, engagement, and cognitive improvement as reward signals to continuously explore and utilize these factors, gradually optimizing long-term Prompt instruction generation strategy combinations for specific users.

[0095] In this embodiment, a prompt type strategy is also established. The prompt type strategy can accurately judge and decide what type of prompt to provide based on the user's current instantaneous cognitive state.

[0096] Specifically, the decision-making logic for the prompt type strategy includes the following steps:

[0097] First, determine the root cause of the user's difficulty. For example, if a user has a high error rate on a specific concept question, they are considered to be in a "difficulty in understanding" state. In this case, a "concept explanation" type prompt will be activated, and the strength of the prompt will be adjusted according to the complexity of the preceding content. Specifically, for a complex concept, the Prompt string instruction will instruct the large language model to "explain using a concrete example or a simple analogy"; while for simpler content, it will instruct the large language model to "restate the problem in another way".

[0098] When users' reaction times gradually increase and the number of common errors increases, it is determined that they have entered a state of "cognitive fatigue". At this time, a "de-load" prompt will be selected, and instructions will be generated requiring the large language model to "summarize the task requirements into a sentence" or "break down the next step into smaller steps".

[0099] By establishing a prompt type strategy, we ensure that prompts are not sent blindly, but rather directly respond to the judgment results of the user's real-time status, thereby providing the most effective support.

[0100] In this embodiment, an interaction style strategy is also established. This strategy is implemented by adding language style constraint parameters to the Prompt string, such as setting the parameter instruction to 'tone=encouraging' or 'style=conversational', to adjust the language output style of the large language model. In this embodiment, a task variant generation strategy is also established. This strategy requires the large language model to generate different content variants based on the same task logic by setting 'task variant parameters' in the Prompt string. For example, changing the memory material in a memory task or adjusting the question angle in a reasoning task. In this embodiment, specific difficulty control parameters, such as 'difficulty level=3', can also be embedded in the Prompt string instruction to guide the large language model to generate content that meets difficulty constraints, thereby ensuring that the training task progresses gradually, avoiding both excessive difficulty and excessive ease.

[0101] Step A5: Receive the dynamic Prompt string instruction and input it into the large language model to generate training task content or a natural language response to user input. For new users, the system will generate a basic task as a starting point based on the static data and initial evaluation results provided during registration, so that the real-time data collected after executing the task can be used for difficulty adjustment; for users with existing task records, the system will directly generate matching task content based on their historical performance and current status.

[0102] The dynamic Prompt string instructions include instructions with detailed constraints on the generated content, format, style, length, and difficulty. After being input into the large language model, the generated training task content includes a story, a set of questions, and a scene description. The generated natural language responses to user input include evaluation feedback, encouraging words, and explanations.

[0103] In step A5, the dynamic Prompt string instruction can also be combined with contextual information (such as user interests and background) from multidimensional input data and inputted into the large language model. By combining multidimensional input data as contextual information into the large language model, personalization can be enhanced. For example, adding "This user is [age] years old, has [educational background], and is interested in [interests]" to the dynamic Prompt string instruction requires the large language model to incorporate these elements when generating training task content or natural language responses to user input, making it more relevant to the user's life and increasing engagement.

[0104] Step A6: Transform the training task content into a user-operable training task interface, receive user input and operations until the user completes the task, evaluate the task completion status to obtain real-time performance data (including task accuracy, reaction time, error type, prompt dependence, interaction behavior data such as click / input count, and voice features such as speech rate, pauses, volume and emotion), and provide instant feedback.

[0105] Specifically, it includes the following steps:

[0106] Step B1: Based on the training task content generated by the large language model, dynamically generate or call corresponding user interface components, such as text display boxes, voice players, input boxes, multiple choice questions, drag-and-drop areas, etc., to transform the training task content generated by the large language model into a user-operable training task interface, such as displaying text, playing voice, and presenting interactive elements.

[0107] Step B2: Input acquisition, capturing various user interactions on the training task interface, such as text input, voice input, clicks, selections, gestures, etc.

[0108] Step B3: Task evaluation. Establish different evaluation mechanisms based on the nature of the task to obtain user evaluation results on task completion.

[0109] The specific evaluation mechanisms include:

[0110] For tasks with deterministic answers, such as numerical calculation or sequence memory, the user's input is directly compared with the preset standard answer. If the comparison is consistent, it is considered a complete match; otherwise, it is considered a mismatch.

[0111] For open-ended or text generation tasks, such as text paraphrasing and opinion presentation, the following methods are used for evaluation:

[0112] The text similarity-based evaluation method first converts the user-input answer text and one or more pre-stored reference answer texts into high-dimensional numerical vector representations using word embedding technology. Then, it calculates the angle or distance between the answer text vector and the reference answer text vector to obtain a quantitative similarity score (e.g., between 0 and 1). The similarity score is used to evaluate how semantically close the answer text is to the reference answer text.

[0113] The evaluation method based on a large language model works as a structured invocation process. The user's complete answer, the corresponding original task instructions, and a pre-defined evaluation prompt string optimized for evaluation needs are sent to the large language model. This evaluation prompt string is predefined by the system during the design phase based on task characteristics and includes dimensions that the large language model needs to assess (such as 'logical coherence', 'factual accuracy', 'whether key concepts A and B are included', etc.). The evaluation results are returned in a structured format (such as JSON). The evaluation results include quantitative scores and qualitative comments, such as "Correct answer," "Please try again," and "You remembered 3 key pieces of information correctly," to more accurately judge the quality of the user's answer. Especially for a text paraphrasing task, this method can accurately assess whether the user has omitted key information, rather than just superficial textual similarity.

[0114] Step B4: Real-time feedback. Based on the evaluation results, provide real-time feedback to the user, such as "Correct answer", "Please try again", "You remembered 3 key pieces of information correctly". This real-time feedback can also be generated by the large language model based on the current performance and prompting strategy.

[0115] Step A7: Iterate through steps A1-A6 to continuously and dynamically generate training tasks until the training duration is reached. Transfer the user's behavior data from the previous task to the multidimensional raw data of the next task, and integrate the real-time performance data into the multidimensional input data during the next task.

[0116] This invention also provides a dynamic cognitive training system based on a large language model and multi-dimensional evaluation, used to implement the aforementioned dynamic cognitive training method based on a large language model and multi-dimensional evaluation, such as... Figure 2 As shown, it includes:

[0117] The data acquisition and preprocessing module is used to collect multidimensional raw data from users and perform preprocessing such as cleaning, noise reduction, format conversion, and standardization on the multidimensional raw data to obtain multidimensional input data, providing standardized multidimensional input data for subsequent modules.

[0118] The multidimensional data integration module is used to process heterogeneous multidimensional raw data, such as MoCA sub-items, ADNI-type reports, demographic data, historical behavior data, etc. The processing includes:

[0119] Data cleaning and standardization: This involves processing missing and outlier values ​​in multidimensional raw data, standardizing multidimensional raw data of different dimensions, and obtaining standardized multidimensional data.

[0120] It is also used for deep integration and feature extraction of standardized multidimensional data from different sources and formats, extracting feature parameters for personalized optimization of dynamic prompting engineering modules or large language models. In this embodiment, the feature parameters include:

[0121] (a) Training difficulty bias parameter, indicating the recommended starting or adjusting level of cognitive training for the user across the whole or a specific cognitive domain, derived from sub-item scores of standardized cognitive assessment data, user's historical cognitive training data, demographic and background data. After being input into the dynamic prompting engineering module, it influences the content complexity and information density requirements imposed on the large language model when selecting the difficulty of the basic task or generating Prompt string instructions.

[0122] (b) Content topic preference parameter, indicating the training content topics, scenarios, or domains that the user is interested in or prefers, derived from interests, occupations, cultures, and the frequency of performance or interaction in the user's historical cognitive training data from demographic and background data. After being input as a Prompt string instruction or API parameter into the input large language model interaction and content generation module, it guides the large language model to generate stories, examples, dialogues, or scenarios related to the user's interests.

[0123] (c) Interaction style preference parameters, indicating the user's preferred system interaction tone, expression, or level of detail, are derived from demographic and background data, user responses to feedback in historical cognitive training data, and the user's own language style and speech characteristics in real-time interaction behavior data. After being input into the input dynamic prompting engineering module or the large language model interaction and content generation module, it affects the language style, tone, and level of detail (such as "encouraging," "detailed explanation," and "concise and direct") when the large language model generates a response.

[0124] (d) The prompt frequency / detail bias parameter indicates the system's tendency to provide task prompts to the user, such as frequency and detail. This is derived from specific weaknesses in the user's cognitive profile, learning speed, the user's historical cognitive training data showing reliance on prompts, and hesitation and error patterns in the user's real-time interaction data. After being input into the dynamic prompting engineering module, it influences whether the system requests prompts from the large language model in the Prompt string instruction, as well as the frequency, timing, and level of detail of the prompts.

[0125] (e) Modality priority / preference parameter, indicating the user's preferred information receiving or input modality, including voice, text or bimodality, derived from visual and hearing data in demographic and background data, and performance in different modalities in the user's historical cognitive training data. After being input into the input dynamic prompting engineering module or task execution module, it affects the form of the generated content (e.g., whether it is more suitable for hearing) or the way the content is presented (whether to display text and play voice at the same time, or to prioritize a certain modality).

[0126] (f) Cognitive domain emphasis / compensation parameters indicate which cognitive domains the user needs to strengthen training in, or what abilities need to be compensated when performing specific tasks. These parameters are derived from the user's cognitive profile, which categorizes the strengths and weaknesses of each cognitive domain, and from the sub-item scores in the standardized cognitive assessment data. After being input into the input dynamic prompting engineering module, these parameters influence its priority selection of task types for specific cognitive domains, or require the large language model to focus on training or compensating for certain cognitive abilities (such as using simple vocabulary to compensate for weaknesses in language skills) when generating prompt string instructions.

[0127] (g) Real-time feedback sensitivity parameters indicate the system's sensitivity to changes in real-time user performance (such as error rate and reaction time fluctuations). These parameters are derived from the state variability in the user's cognitive profile, the performance stability in the user's historical cognitive training data, the sudden changes in performance in the user's real-time interactive behavior data, and voice emotion. After inputting these parameters into the dynamic prompting engineering module, the threshold or reaction speed of the dynamic prompting engineering module's trigger strategy adjustment (such as reducing difficulty or providing prompts) is adjusted. The higher the sensitivity, the faster the response to the user's instantaneous difficulties.

[0128] It is also used in the data fusion process: it fuses standardized and feature-extracted multidimensional input data into a unified representation, which can be represented as a high-dimensional vector or as a knowledge graph containing multiple information structures.

[0129] The optimization effect on the dynamic prompting engineering module / large language model: These integrated multidimensional input data are mainly used for:

[0130] Initialize cognitive profile: Set more accurate initial training difficulty and content preferences for new users or users who have not used the service for a long time.

[0131] Long-term trend assessment: This is used to distinguish whether a user is experiencing short-term poor performance or long-term cognitive impairment, thereby adjusting the overall training strategy accordingly.

[0132] Constraining or biasing the behavior of the large language model: Incorporating stronger personalized constraints into the Prompt string instructions (e.g., "use simpler vocabulary, avoid overly complex sentence structures, as users have lower levels of education"), or guiding the large language model to generate content that better aligns with the user's cultural background and interests. This optimization is relatively stable, unlike the frequent changes in real-time performance; it serves as a supplement and macro-level guide to real-time dynamic adjustments.

[0133] The cognitive profiling and assessment module is used to construct and maintain a user's cognitive profile by using multi-dimensional input data and technologies such as statistical models and machine learning algorithms. The cognitive profile is updated in real time, and cognitive assessment is performed to obtain the user's current cognitive state. The module also obtains a user cognitive state vector composed of the cognitive profile and the instantaneous cognitive state.

[0134] The cognitive profiling and evaluation module includes a transient cognitive state evaluation engine, which incorporates a fast evaluation model based on heuristic rules and a dynamic machine learning model. The fast evaluation model pre-configures several condition-to-state mapping rules, while the dynamic machine learning model features a pre-trained classifier and constructs transient cognitive state classification rules.

[0135] The dynamic prompting engineering module is used to receive the user's cognitive state vector, feature parameters from multi-dimensional input data, and the current training task type and objective to dynamically prepare and send the Prompt string instruction to the large language model.

[0136] The Large Language Model Interaction and Content Generation Module, also known as the LLM Interaction and Content Generation Module, serves as an interface with one or more large language models. It receives dynamic Prompt strings generated by the Dynamic Prompt Engineering Module and sends them to the large language models. It also receives training task content or natural language responses to user input generated by the large language models and formats them. Furthermore, the Large Language Model Interaction and Content Generation Module can provide personalized contextual information (such as user interests and background) from the Multidimensional Data Integration Module as parameters for API calls or as part of the Prompt strings to the large language models.

[0137] The training task execution and feedback module is used to receive the formatted training task content generated by the large language model, transform it into a user-operable training task interface, receive user input and operations, evaluate the task completion status, and provide real-time feedback to the data acquisition model to form a real-time data stream.

[0138] The training task execution and feedback module is equipped with preset evaluation logic and establishes different evaluation mechanisms based on the nature of the task. These include automated evaluation methods based on clear objective rules, which are suitable for tasks with deterministic answers or standardized processes, and evaluation methods based on large language models, which are suitable for tasks that are more open-ended, creative, or subjective (such as open-ended question answering and opinion expression).

[0139] The bimodal interaction module provides a user interface that allows users to input via voice and text, and to receive text and voice output from a large language model.

[0140] Specifically, the bimodal interaction module includes the following functions:

[0141] like Figure 4 As shown in Figure (a), it includes voice output function, integrates text-to-speech (TTS) technology, converts the text feedback or task instructions output by the system into voice output, and can adjust the speech rate, tone and timbre of the voice according to the user's cognitive profile to make it more friendly or clearer;

[0142] Text output function: Standard text rendering to display text;

[0143] like Figure 4 As shown in Figure (b), it includes voice input processing functionality, integrating Automatic Speech (ASR) technology to convert user speech into text. Simultaneously, it analyzes non-verbal characteristics of the speech, such as speech rate, pauses, volume, pitch, and emotion. These characteristics can be used as additional real-time data reflecting the user's cognitive state, input into the data acquisition module, and the processed input data is collected.

[0144] The text input processing function performs standard text input box processing, analyzes the features of user text and inputs it into the data acquisition module, and collects the processed input data.

[0145] The voice / text modality switching and coordination function is used to coordinate or use voice / text modalities simultaneously. For example, complex instructions or content that needs to be accurately remembered may be provided with both text and voice; voice input is mainly used when the user gives a verbal response; and the user is guided to use text input in situations that require precise input (such as numerical calculations).

[0146] Data storage module: Used to persistently store all types of user data, including multidimensional raw data and multidimensional input data, cognitive profiles, training history, system configuration and other information, and supports data retrieval, updating and analysis.

[0147] All of the above modules exchange data and work together through internal APIs or message queues.

[0148] The following content provides an application example of the dynamic cognitive training system based on a large language model and multi-dimensional evaluation provided by this invention. This system is equipped with the dynamic cognitive training method based on a large language model and multi-dimensional evaluation proposed in this invention.

[0149] Specifically, an application example is a dynamic difficulty adjustment process based on a listening memory task, as follows:

[0150] If the user's cognitive profile shows a slight decline in short-term memory ability, and the user's current condition is judged to be good, then an auditory memory training task is initiated.

[0151] Initial state: Cognitive profile vector V0, task type = "listening memory", multi-dimensional parameters P0 (including education level, age, etc.).

[0152] Dynamic prompts for the engineering module input: V0,"Hearing Memory",P0.

[0153] The dynamic prompting engineering module processes information based on the information in V0 (low short-term memory score but good state) and P0 (educational level information), generating a Prompt string instruction. This Prompt string instruction may require the large language model to generate a story containing four key information points, with relatively simple language but a clear structure, and to include a prompt before the story begins: "Listen carefully, there are several important pieces of information in the story."

[0154] Example of a Prompt string instruction: "Please generate a short story about a garden, approximately 150 words long, containing the following 4 key information points: [Information Point 1], [Information Point 2], [Information Point 3], [Information Point 4]. Please use a style of language that is easy for users of [educational level] to understand. Please add a prompt at the beginning of the story: 'Listen carefully, there are several important pieces of information in the story.'"

[0155] Large Language Model Generation: The large language model generates story text and corresponding speech based on the Prompt string instructions.

[0156] Task execution: The story is played via voice, and the user retells it via voice.

[0157] Data Acquisition and Evaluation: After the user finishes retelling the story, the bimodal interaction module records the speech waveform and retelling duration. The training task execution and feedback module analyzes the matching degree between the retelling text and the original story text and corresponding speech key information points, records the number of correct information points, incorrect information points, and missing information points, and analyzes speech features such as pause frequency and duration, ultimately generating the result of this task.

[0158] Cognitive profile update: Based on the results of this task (e.g., only 2 out of 4 information points were repeated, and there were many pauses), update the short-term memory transient state score V1 in the cognitive profile.

[0159] The dynamic prompting engineering module dynamically adjusts its approach: For the next listening memory task, the module receives the updated cognitive state V1. Based on the change from V0 to V1, the module determines that the user's performance is below expectations and dynamically adjusts the Prompt string generation strategy. The new Prompt string can request the large language model to generate a simpler story, such as containing only 3 key information points, or to repeat key information points within the story, or to generate a text version of the story that explicitly marks key information points for the user's reference (although it's a listening task, it provides supplementary text). Example Prompt string: "Based on the user's recent performance in memory tasks, please generate a simpler listening memory story. The story is about [user interests], approximately 100 words long, and contains 3 key information points. Please try to appropriately emphasize or repeat the key information points in the story."

[0160] Continue with the next adjusted task until the training duration is reached.

[0161] By continuously repeating the above process, we can ensure that the difficulty and format of cognitive training always meet the ever-changing needs of users.

[0162] This invention, through a dynamic prompting engineering module, can capture subtle changes in the user's cognitive state in every interaction and task in real time, and immediately transform these changes into precise control over the behavior of the large language model. It generates training content and difficulty that are precisely matched to the user's current level, enabling highly dynamic and real-time adaptive cognitive training, overcoming the problems of lagging adjustment and insufficient coarsening in traditional cognitive training systems.

[0163] This invention combines multi-dimensional data such as MoCA sub-items, ADNI class reports, demographics, historical behavior, and real-time performance to construct a comprehensive user cognitive profile. Moreover, the aforementioned multi-dimensional data is not only used for initial evaluation, but is also technically integrated into the dynamic prompting engineering module and the content generation process of the large language model. This allows the training content, prompting methods, and interaction styles to deeply match the individual differences and specific needs of users, rather than simply classifying them based on a total score. This provides a fine-grained and in-depth personalized training process.

[0164] This invention leverages the powerful generative capabilities of large language models to generate entirely new, non-preset training task content in real time. Simultaneously, the dynamic prompting engineering module's precise control over the large language model ensures that the generated tasks meet specific cognitive training duration and difficulty requirements, significantly improving the diversity and flexibility of training content. This greatly enriches the training content, prevents user boredom with repetitive content, and allows the system to quickly adapt to new training theories or needs.

[0165] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A dynamic cognitive training method based on a large language model and multi-dimensional evaluation, characterized in that, Includes the following steps: Step A1: After the user logs in, collect the user's multidimensional raw data and process it to obtain heterogeneous standardized multidimensional data; Step A2: Perform data fusion and feature extraction on heterogeneous standardized multidimensional data to obtain multidimensional input data and its feature parameters; Step A3: Process the results of data fusion and feature extraction to obtain fusion features, determine the user's cognitive profile based on the fusion features, and at the same time perform cognitive evaluation to obtain the user's current instantaneous cognitive state, and obtain the user cognitive state vector composed of the cognitive profile and cognitive state. Step A4: Using the user's cognitive state vector, feature parameters in the multidimensional input data, and the type and objective of the training task as input, construct a policy library for reasoning and decision-making on the input to select and execute the optimal Prompt string generation strategy. Based on the constructed policy library, dynamically prepare the Prompt string instructions to be sent to the large language model. Step A5: Receive dynamic Prompt string instructions and input them into the large language model to generate training task content or natural language responses to user input; Step A6: Transform the training task content into a user-operable training task interface, receive various user interactions until the user completes the task, evaluate the task completion status to obtain real-time performance data, and provide immediate feedback. Step A7: Iterate through steps A1-A6 to continuously and dynamically generate training tasks until the training duration is reached. Transfer the user's behavior data from the previous task to the multidimensional raw data of the next task, and integrate the real-time performance data into the multidimensional input data during the next task.

2. The dynamic cognitive training method based on a large language model and multi-dimensional evaluation according to claim 1, characterized in that, In step A3, the user's long-term profile includes the baseline ability and long-term trend of cognitive ability levels corresponding to each cognitive domain.

3. The dynamic cognitive training method based on a large language model and multi-dimensional evaluation according to claim 1, characterized in that, In step A3, the transient cognitive state is obtained by analyzing the user's real-time behavioral interaction data in the multidimensional input data according to the following rules: The specific mapping rule based on condition-state is as follows: By analyzing and judging the real-time interaction data of users through the correspondence between preset conditions and cognitive states, the corresponding cognitive state judgment will be triggered when the following situations are detected: Cognitive fatigue: If a user's reaction time continues to increase across multiple consecutive task points, while the task accuracy rate shows a downward trend, then the user may be in a state of "cognitive fatigue". Lack of confidence: If a user requests prompts on a certain knowledge point more than the set number of times within the first preset time, or does not take any action at a key decision point within the second preset time, the user is determined to be in a state of "lack of confidence". Knowledge Barrier: If a user makes the same type of error on the same task type or similar knowledge points within a third preset time, it is determined that the user has encountered a "knowledge barrier". Decreased focus: If a user interrupts multiple times, pauses for a long time, or inputs irrelevant information during the interaction within the fourth preset time, it is determined that the user may be in a state of "distraction". Instantaneous cognitive state classification rules: The instantaneous cognitive state classification rules are used to determine the user's instantaneous job status through machine learning models and feature analysis methods.

4. The dynamic cognitive training method based on a large language model and multi-dimensional evaluation according to claim 1, characterized in that, In step A4, the established policy library includes: The rule base establishes a difficulty reduction trigger threshold and a difficulty reduction strategy. When a user's real-time performance data reaches the difficulty reduction trigger threshold, the Prompt string instruction is modified through the difficulty reduction strategy command Prompt generator. The model library contains machine learning or deep learning models for predicting and classifying user task performance and transient cognitive states. The algorithm library is equipped with an adaptive optimization algorithm, which is used to learn and iteratively optimize the combination of long-term Prompt string instruction generation strategies for specific users.

5. The dynamic cognitive training method based on a large language model and multi-dimensional evaluation according to claim 1, characterized in that, In step A4, a prompt type strategy is also established to accurately determine and decide what type of prompt to provide based on the user's current real-time status.

6. The dynamic cognitive training method based on a large language model and multi-dimensional evaluation according to claim 1, characterized in that, In step A5, the dynamic Prompt string instruction is combined with multidimensional input data as context information and input into the large language model.

7. The dynamic cognitive training method based on a large language model and multi-dimensional evaluation according to claim 1, characterized in that, The implementation of step A6 specifically includes the following steps: Step B1: Transform the training task content generated by the large language model into a user-operable training task interface; Step B2: Capture various user interactions on the training task interface; Step B3: Establish different evaluation mechanisms based on the nature of the task and conduct evaluations to obtain user evaluation results on task completion. Step B4: Provide real-time feedback to users based on the evaluation results.

8. The dynamic cognitive training method based on a large language model and multi-dimensional evaluation according to claim 7, characterized in that, During the implementation of step B3, the specific evaluation mechanisms include: For tasks with definite answers, the user's input is directly compared with the preset standard answer. If the comparison is consistent, it is considered a complete match; otherwise, it is considered a mismatch. For open-ended or text generation tasks, the following methods are used for evaluation: The text similarity-based evaluation method represents the user-input answer text and the pre-stored reference answer text as numerical vectors, calculates the similarity score between the answer text vector and the reference answer text vector, and uses the similarity score to evaluate the semantic closeness between the answer text and the reference answer text. The evaluation method based on a large language model involves sending the user's complete answer, the corresponding original task instructions, and the evaluation prompt string to the large language model. The evaluation prompt string instructs the large language model to analyze from the required dimensions and requests it to return the evaluation results in structured data. The evaluation results include quantitative scores and qualitative comments.

9. A dynamic cognitive training system based on a large language model and multi-dimensional evaluation, used to implement the dynamic cognitive training method based on a large language model and multi-dimensional evaluation according to any one of claims 1 to 7, characterized in that, Includes the following modules: The data acquisition and preprocessing module is used to collect multidimensional raw data from users and preprocess the multidimensional raw data to obtain multidimensional input data. The multidimensional data integration module is used to perform data fusion and feature extraction on heterogeneous standardized multidimensional data to obtain multidimensional input data, and extract feature parameters for driving the dynamic prompting engineering module or large language model for personalized optimization. The cognitive profiling and assessment module is used to build and maintain a user's cognitive profile through multi-dimensional input data, while performing cognitive assessment to obtain the user's current cognitive state and obtain a user cognitive state vector composed of the cognitive profile and cognitive state. The dynamic prompting engineering module is used to receive the user's cognitive state vector, feature parameters in multi-dimensional input data, and the current training task type and target to dynamically prepare and send the Prompt string instruction to the large language model; The large language model interaction and content generation module is used to receive dynamic prompt string instructions generated by the dynamic prompt engineering module, send them to the large language model, and receive training task content or natural language responses to user input generated by the large language model, and format them. The training task execution and feedback module is used to receive the training task content generated by the formatted large language model, transform it into a user-operable training task interface, receive various user interactions, obtain user evaluation results on task completion, and provide real-time feedback to the data acquisition and preprocessing module. The bimodal interaction module provides a user interface that allows users to input via voice and text, and to receive text and voice output from a large language model. Data storage module: Used for persistent storage of all types of user data.

10. The dynamic cognitive training system based on a large language model and multi-dimensional evaluation according to claim 9, characterized in that, The cognitive profiling and evaluation module is equipped with an instantaneous cognitive state evaluation engine. The instantaneous cognitive state evaluation engine has a fast evaluation model based on self-heuristic rules and a dynamic machine learning model. The fast evaluation model has several pre-set condition-to-state mapping rules, and the dynamic machine learning model has instantaneous cognitive state classification rules.

Images