A few-sample-based learning assistant cognitive load adaptive method and related device

Abstract

The application discloses a learning assistant cognitive load adaptive method based on few samples and related equipment, and relates to the field of artificial intelligence. The method comprises the following steps: providing a candidate explanation group composed of multiple candidate explanations to a user, recording the selection behavior of the user in multiple rounds of graphic-text question and answer, extracting cognitive preference features of multiple style dimensions, and generating a style preference vector; selecting a basic multi-modal model with frozen parameters and adjustable LoRA parameters, mapping the style preference vector to a gating coefficient, training only the LoRA parameters and keeping the basic model parameters frozen, obtaining a low-rank increment and forming a user personalized parameter set; analyzing the input features before task input, generating a task complexity score, and generating a temporary adjustment coefficient based on the difference between the task complexity and the user's expected information density, correcting the gating coefficient and calculating the effective low-rank increment in the reasoning process; discarding the temporary correction parameters after reasoning is completed, so as to maintain the stability of the model structure.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and specifically to a cognitive load adaptation method and related equipment for a learning assistant based on few samples. Background Technology

[0002] In recent years, general-purpose intelligent systems, represented by Multimodal Large Language Models (MLLMs), have made significant progress in cross-modal tasks such as image-text, speech-text, and video-text, driving the implementation of applications in intelligent question answering, assisted creation, and educational tutoring. Meanwhile, Human Preference Alignment (HPA) has gradually become a key direction for improving model usability and security. Common approaches include Reinforcement Learning from Human Feedback (RLHF), Direct Preference Optimization (DPO), and rule / tool ​​enhancement methods. In complex interactions with real users, models not only need to understand the semantics of multimodal inputs but also need to align the granularity, structure, and tone of the output with individual preferences and the task context to achieve stable, controllable, and interpretable human-computer collaboration.

[0003] Cognitive science research indicates that individuals are subject to various load constraints described by Cognitive Load Theory (CLT) when processing information, including intrinsic load, extrinsic load, and facilitatory load. These loads collectively influence learning efficiency and depth of understanding. Integrating CLT into the human preference alignment process of a multimodal large-scale model means that the system should be able to dynamically adjust the granularity of explanation (fine vs. coarse), structural organization (step-by-step vs. summary), and tone style (rigorous academic vs. popular guidance) according to the user's cognitive threshold while maintaining factual accuracy. It should also perform consistency alignment and conflict resolution for cross-modal evidence. Existing approaches, primarily based on Prompt Engineering (PE) or static templates, often struggle to maintain consistent fit and stability in scenarios involving changes in user state, fluctuations in task difficulty, and modal transitions (such as switching from diagrammatic explanations to formula derivations).

[0004] At the engineering implementation level, parameter-efficient fine-tuning (PEFT) methods (such as low-rank adaptation (LoRA) and its gated variants) provide a feasible path for personalization and scenario-based applications. However, challenges remain, including consistency constraints in cross-modal alignment, interpretability and transferability of preference modeling, and low-overhead, controllable decoding during the inference phase. Simply relying on offline fine-tuning or fixed strategies makes it difficult to simultaneously meet the real-time adaptive needs of multiple users, multiple tasks, and multiple modalities. Based on this, research on "aligning multimodal large models with human preferences" is gradually shifting from static preference matching to dynamic adaptation oriented towards cognitive load: focusing on both the effectiveness of preference representation and alignment mechanisms, and emphasizing the controllable adjustment and interpretable feedback of output style and structure without compromising the capabilities of the main model, providing a more robust human-computer collaboration foundation for scenarios such as educational tutoring, professional retrieval, and enterprise knowledge question answering. Summary of the Invention

[0005] The technical problem this invention aims to solve is that existing multimodal large models, when used in learning assistant scenarios, struggle to accurately extract users' personalized preferences under limited sample conditions and dynamically adjust output styles based on task complexity. This often leads to excessive cognitive load or insufficient information, and lacks the adaptive capability to simultaneously consider both long-term user preferences and current task requirements. The goal is to provide a cognitive load adaptation method for learning assistants based on limited sample data. This method can extract multi-dimensional user preference features with a limited number of interaction samples, generate a style preference vector, and combine LoRA parameter injection and task complexity adjustment to achieve personalized and dynamic adaptation of the output style. This resolves the difficulties in personalized modeling under limited sample conditions and the contradiction that multimodal large models cannot simultaneously consider both long-term user preferences and immediate task complexity during inference, thereby improving the intelligence level and cognitive load adjustment capability of learning assistants.

[0006] This invention is achieved through the following technical solution:

[0007] A method for adaptive cognitive load in a learning assistant based on few samples, comprising:

[0008] The image and text answers provided to users are set up with candidate explanation groups consisting of multiple candidate explanations. The user's selection behavior in multiple rounds of image and text Q&A is recorded. Cognitive preference features of multiple style dimensions are extracted from the behavioral data to generate style preference vectors.

[0009] Select a base multimodal model with frozen parameters and adjustable LoRA parameters, map style preference vectors to gating coefficients, train only LoRA parameters and keep the base multimodal model parameters frozen, use the low-rank increments obtained from training LoRA parameters as a personalized parameter set, and inject them into the low-rank adaptation layer of the base multimodal model decoding layer in a read-only manner.

[0010] Before inputting the task into the basic multimodal model, feature analysis is performed on the task input to generate a task complexity score. A temporary adjustment coefficient is generated based on the difference between the complexity score and the user's expected information density. The gating coefficient is temporarily corrected based on the temporary adjustment coefficient to generate an effective gating coefficient for the current task. During the inference process, the effective gating coefficient is used to calculate the effective low-rank increment. After the inference is completed, the effective gating coefficient and the effective low-rank increment are no longer retained.

[0011] Furthermore, the text and image answers provided to the user include candidate explanation groups consisting of multiple candidate explanations, recording the user's selection behavior in multiple rounds of text and image Q&A, including:

[0012] For multiple style dimensions, corresponding candidate explanation groups are designed. Each candidate explanation in the candidate explanation group differs from other candidate explanations in a single style dimension, while maintaining consistency in other style dimensions. Each group of candidate explanations and the user's selection behavior constitute an explanation sample. Multiple explanation sample sets form a comparison explanation sample set.

[0013] For each explanation sample, features are extracted in multiple style dimensions to generate multi-dimensional feature vectors, which form a set of dimensional feature vectors.

[0014] Furthermore, the extraction of the style preference vector involves establishing a Bradley-Terry pairwise comparison model for each style dimension, specifically including:

[0015] Dimensional labels are generated for each style dimension based on the dimensional feature vectors, and weights are assigned to different style dimensions to characterize the degree of influence of the style dimension on the overall preference.

[0016] The user's selection behavior in the candidate explanation group is weighted to obtain weighted preference data;

[0017] For each style dimension, parameter estimation is performed on the weighted preference data to obtain the preference intensity of the candidate explanation under that style dimension;

[0018] The preference intensity of each style dimension is combined according to weights to form a style preference vector that describes the overall preference distribution of users across multiple style dimensions.

[0019] Furthermore, the decoding layer of the basic multimodal model is equipped with an injectable low-rank adaptation layer, and the generation of gating coefficients includes:

[0020] Input the style preference vector into the gating to generate a subnetwork;

[0021] In each low-rank adaptation layer, a linear mapping is performed based on the style preference vector and the preset weight matrix and bias vector to obtain the corresponding gating coefficients.

[0022] The activation function is used to map each gate coefficient so that its value is within a preset range;

[0023] Each gating coefficient is applied to the corresponding low-rank adaptation layer.

[0024] Furthermore, the decoding layer of the basic multimodal model is equipped with an injectable LoRA low-rank adaptation layer, and the training LoRA parameters include:

[0025] A low-rank decomposition matrix is ​​introduced into the target layer to form an increment, resulting in a low-rank matrix in the input direction and a low-rank matrix in the output direction.

[0026] The gating coefficients generated from the style preference vector are applied as scaling factors to the low-rank decomposition matrix, and the low-rank increment is calculated from this.

[0027] The low-rank increment is superimposed on the frozen original weight matrix, and forward computation is performed;

[0028] Backpropagation is performed based on training error;

[0029] Iterate until convergence, and use the low-rank increments obtained from training as a personalized parameter set. The personalized parameter set is injected into the decoding layer in read-only mode during the inference period of the basic multimodal model.

[0030] Furthermore, the generation of the temporary adjustment coefficient includes:

[0031] Perform feature analysis on the task input to form a task feature vector, and calculate the task complexity score;

[0032] The detail preference in the style preference vector is mapped to the user's expected information density benchmark. The difference between the task complexity score and the benchmark is calculated, and a temporary adjustment coefficient is generated based on the difference.

[0033] The temporary adjustment coefficient is used to correct the gating coefficient during the inference process of the basic multimodal model.

[0034] Furthermore, the application of the temporary adjustment coefficient includes:

[0035] In the inference process of the basic multimodal model, the gating coefficients generated by the style preference vector are combined with the temporary adjustment coefficients of the current task to construct effective gating coefficients;

[0036] The effective gating coefficients are used to replace the original gating coefficients and applied to the calculation process of the low-rank increment to obtain the effective low-rank increment for task correction.

[0037] The effective low-rank increment is combined with the original weight matrix to form the effective weight for the current task, and the effective weight drives the decoder to perform forward autoregressive inference to generate an output sequence and output a graphic and textual answer.

[0038] After inference is complete, release the effective gating coefficients and effective low-rank increments.

[0039] This invention also provides a few-shot-based learning assistant cognitive load adaptation system, based on the aforementioned few-shot-based learning assistant cognitive load adaptation method, characterized in that it includes:

[0040] Preference collection module: It is used to set up candidate explanation groups consisting of multiple candidate explanations in the text and image answers provided to users, record the user's selection behavior in multiple rounds of text and image Q&A, extract cognitive preference features of multiple style dimensions from the behavioral data, and generate style preference vectors;

[0041] LoRA training module: used to select a base multimodal model with frozen parameters and adjustable LoRA parameters, map style preference vectors to gating coefficients, train only LoRA parameters while keeping the base multimodal model parameters frozen, and use the low-rank increments obtained from training LoRA parameters as a personalized parameter set to be injected into the low-rank adaptation layer of the base multimodal model decoding layer in a read-only manner.

[0042] Task adjustment module: Before inputting the task into the basic multimodal model, it performs feature analysis on the task input, generates a task complexity score, generates a temporary adjustment coefficient based on the difference between the complexity score and the user's expected information density, temporarily corrects the gating coefficient based on the temporary adjustment coefficient, generates an effective gating coefficient for the current task, calculates the effective low-rank increment using the effective gating coefficient during inference, and no longer retains the effective gating coefficient and effective low-rank increment after inference.

[0043] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned cognitive load adaptation method for a learning assistant based on few samples.

[0044] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for adaptive cognitive load of a learning assistant based on few samples.

[0045] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0046] This invention, by designing candidate explanation groups and comparative explanation sample sets, can extract users' cognitive preferences across multiple style dimensions under limited interaction conditions. It then utilizes the Bradley-Terry pairwise comparison model for parameter estimation, generating accurate style preference vectors, thus overcoming the dependence on large amounts of user data in existing technologies. By training only LoRA parameters and keeping the backbone parameters frozen, the style preference vector is mapped to the generation and injection of low-rank increments driven by gating coefficients, achieving user-level personalized modeling and avoiding the high computational and storage overhead of large-scale model full-parameter fine-tuning.

[0047] This invention introduces a task complexity identification mechanism. By combining the difference between the task feature vector and the user's expected information density, it dynamically generates temporary adjustment coefficients, corrects the gating coefficients, and constructs effective gating coefficients during inference. This achieves adaptive adjustment of the output style under different task complexities, solving the problem of mismatch between personalized output and cognitive load in existing methods. During inference, it uses persistent user preference parameters and the temporary adjustment coefficients of the current task to jointly calculate effective low-rank increments. This ensures the inheritance of long-term preferences while allowing for adjustments based on immediate task requirements, improving the flexibility and reliability of the learning assistant in complex interactive scenarios.

[0048] The method of this invention can be applied to various basic multimodal models without increasing the additional hardware burden. It has good scalability and transferability, and is easy to promote and use in different learning scenarios and cognitive tasks. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0050] Figure 1 This is a flowchart of the cognitive load adaptive method for learning assistants based on few samples in Embodiment 1 of the present invention;

[0051] Figure 2 This is a structural diagram of the cognitive load adaptive learning assistant system based on few samples in Embodiment 2 of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0053] Example 1

[0054] A method for adaptive cognitive load in a learning assistant based on few samples, such as Figure 1 As shown, it includes:

[0055] Preference collection: The image and text answers provided to users are set up with candidate explanation groups consisting of multiple candidate explanations. The user's selection behavior in multiple rounds of image and text Q&A is recorded. Cognitive preference features of multiple style dimensions are extracted from the behavioral data to generate style preference vectors.

[0056] LoRA training: Select a base multimodal model with frozen parameters and adjustable LoRA parameters, map the style preference vector to gating coefficients, train only the LoRA parameters and keep the base multimodal model parameters frozen, and use the low-rank increment obtained from training the LoRA parameters as a personalized parameter set, which is then injected into the low-rank adaptation layer of the base multimodal model decoding layer in a read-only manner.

[0057] Task adjustment: Before inputting the task into the basic multimodal model, feature analysis is performed on the task input to generate a task complexity score. Based on the difference between the complexity score and the user's expected information density, a temporary adjustment coefficient is generated. The gating coefficient is temporarily corrected according to the temporary adjustment coefficient to generate an effective gating coefficient for the current task. During the inference process, the effective gating coefficient is used to calculate the effective low-rank increment. After the inference is completed, the effective gating coefficient and the effective low-rank increment are no longer retained.

[0058] In this embodiment, as Figure 1 As shown, the text and image answers provided to the user include candidate explanation groups consisting of multiple candidate explanations, recording the user's selection behavior in multiple rounds of text and image Q&A, including:

[0059] For multiple style dimensions, corresponding candidate explanation groups are designed. Each candidate explanation in the candidate explanation group differs from other candidate explanations in a single style dimension, while maintaining consistency in other style dimensions. Each group of candidate explanations and the user's selection behavior constitute an explanation sample. Multiple explanation sample sets form a comparison explanation sample set.

[0060] For each explanation sample, features are extracted in multiple style dimensions to generate multi-dimensional feature vectors, which form a set of dimensional feature vectors.

[0061] In this embodiment, two candidate explanations constitute a set of candidate explanations. This invention designs comparative explanation sample sets for each of the D key style dimensions. Each set of comparisons aims to highlight the differences in a single dimension while keeping other dimensions as similar as possible. These include the following style dimensions:

[0062] Structural Comparison: "Conclusion First" vs. "Derivation Process First"

[0063] Detailed comparison: "Concise and Streamlined" vs. "Detailed Steps"

[0064] A comparison of emotional dimensions: "Purely technical" vs. "Emotionally rich"

[0065] A Comparison of Abstract Dimensions: "Concrete Instances" vs. "Abstract Concepts"

[0066] Interaction Dimension Comparison: "One-Way Statement" vs. "Interactive Guidance"

[0067] Joint design In a style-based comparison, users express their preferences by choosing between two options, which are then denoted as preference pairs. , This represents the local index of the two candidate explanations in the pair. Where the first... In the round, user preferences Better than ,total .

[0068] For each sample, perform full-dimensional feature extraction to generate a D-dimensional feature vector. Even when primarily controlling for a single variable in the comparative design, feature values ​​across all dimensions are still calculated to accurately characterize the overall style of the samples.

[0069] 1) Structural feature extraction ( ):

[0070]

[0071] in,

[0072] The relative position of the conclusion in the text (0 = beginning, 1 = end).

[0073] Logical flow score calculated through dependency parsing.

[0074] Information entropy of paragraph structure;

[0075] 2) Detailed feature extraction ( ):

[0076]

[0077] in,

[0078] Average sentence length

[0079] Density of step markers (such as "first", "then", "finally")

[0080] The proportion of explanatory content in the total content;

[0081] 3) Sentiment feature extraction ( ):

[0082]

[0083] in,

[0084] Frequency of sentiment words

[0085] The combined score of personal pronouns, modal particles, and interjections.

[0086] Formalism Index (negative correlation);

[0087] 4) Abstract level feature extraction ( ):

[0088]

[0089] in,

[0090] Abstract concept density

[0091] Specificity of the instance;

[0092] 5) Interactive feature extraction ( ):

[0093]

[0094] in,

[0095] Frequency of interrogative and rhetorical questions

[0096] Density of interactive marker words (such as "let us", "you will find")

[0097] In the above formula These are the weight coefficients for each dimension of the feature, used to adjust the relative contribution of each dimension of the feature to the corresponding score.

[0098] In this embodiment, as Figure 1 As shown, the extraction of the style preference vector involves establishing a Bradley-Terry pairwise comparison model for each style dimension, specifically including:

[0099] a) Label each style dimension according to the dimensional feature vector and assign weights to different style dimensions to characterize the influence of the style dimension on the overall preference;

[0100] For each group of comparisons Mark its main comparison dimensions :

[0101]

[0102] in, This represents the weight coefficient of the d-th style dimension;

[0103] b) Weight the user's selection behavior in the candidate explanation group to obtain weighted preference data;

[0104] Calculate the discrimination weights of this set of comparisons in each dimension. , is represented as:

[0105]

[0106] They represent the current comparison pairs. The feature representation objects of the two candidate answers.

[0107] Based on this, the probability that a user chooses one candidate over another is modeled as follows:

[0108] User selection Superior probability Represented as:

[0109]

[0110] in This represents the strength of a user's preference in the d-th dimension. This is the sigmoid function.

[0111] c) For each style dimension, perform parameter estimation on the weighted preference data to obtain the preference intensity of the candidate explanation under that style dimension;

[0112] By maximizing the weighted log-likelihood function:

[0113]

[0114] Optimize the parameters of each dimension using the gradient ascent method. .in This represents the regularization strength coefficient, used to suppress overfitting.

[0115] d) Combine the preference intensities of each style dimension according to their weights to form a style preference vector that describes the overall preference distribution of users across multiple style dimensions;

[0116] Based on the learned dimensional preference parameters, construct a style preference vector:

[0117]

[0118] The square root transform is used to preserve directional information while compressing extrema. Finally, the preference intensity is normalized.

[0119]

[0120] Dimensional Explanation:

[0121] Preference derivation process first; Preferential conclusions first;

[0122] : Prefer detailed steps; Prefers concise expression;

[0123] Prefers emotionally rich individuals; : Prefers technological rationality;

[0124] Prefers abstract concepts; Prefer specific examples;

[0125] : Prefers interactive guidance; Prefers one-way statements.

[0126] This invention employs the LoRA structure from Parameter-Efficient Fine-Tuning (PEFT). Low-rank adapters are inserted into several selected layers at the decoding end of the basic multimodal model, while the backbone parameters of the basic multimodal model are frozen, training only the low-rank decomposition parameters and a small number of conditional mappings. Unlike conventional LoRA, we do not use a uniform low-rank update for all samples; instead, we use the user-explained style preference vector... Mapped to rank-gated coefficients, achieving "rank-interpretable control"—that is, by... This determines the effective strength of each rank channel in each layer, thereby solidifying the user's long-term stable style selection into a persistent set of personalized parameters. This parameter set remains unchanged between user sessions.

[0127] This invention constructs candidate explanation groups and forms a comparative explanation sample set to collect users' cognitive preferences across multiple style dimensions under limited interaction conditions; based on the Bradley-Terry pairwise comparison model, it estimates the parameters of selection behavior to obtain a user preference vector representing comprehensive preferences. This reduces reliance on large-scale historical data.

[0128] Among them, such as Figure 1 As shown, the decoding layer of the basic multimodal model is equipped with an injectable low-rank adaptation layer, and the generation of gating coefficients includes:

[0129] Input the style preference vector into the gating to generate a subnetwork;

[0130] In each low-rank adaptation layer, a linear mapping is performed based on the style preference vector and the preset weight matrix and bias vector to obtain the corresponding gating coefficients.

[0131] The activation function is used to map each gate coefficient so that its value is within a preset range;

[0132] Each gating coefficient is applied to the corresponding low-rank adaptation layer.

[0133] To avoid mixing individual task states into long-term profiles, gating is only used by... For input, for the th Each adapted layer (LoRA rank is ) ), defined as:

[0134]

[0135] in Let be the sigmoid activation function, and let the gating coefficients of the k rank channels be... In The interval is used to "enhance / suppress" the effective strength of the channel. For the preset weight matrix, This is a preset bias vector. For numerical stability and interpretability, adjustments can be made during the training period. Add slight sparsity regularization or temperature scaling; do not introduce any forward preprocessing or dimensionality reduction modules.

[0136] In this embodiment, as Figure 1 As shown, the decoding layer of the basic multimodal model is equipped with an injectable LoRA low-rank adaptation layer, and the training LoRA parameters include:

[0137] A low-rank decomposition matrix is ​​introduced into the target layer to form an increment, resulting in a low-rank matrix in the input direction. and the low-rank matrix in the output direction ;

[0138] The gating coefficients generated from the style preference vector are applied as scaling factors to the low-rank decomposition matrix, and the low-rank increment is calculated from this.

[0139] The low-rank increment is superimposed on the frozen original weight matrix, and forward computation is performed;

[0140] Backpropagation is performed based on training error;

[0141] The process iterates until convergence, using the low-rank increments obtained during training as a personalized parameter set. This personalized parameter set is injected into the decoding layer in read-only mode during the inference phase of the basic multimodal model. The backbone parameters include the original weight matrix. .

[0142] Record No. Original weight matrix of layer Its LoRA low-rank decomposition is:

[0143]

[0144] Inject the gated vector as a rank-scaled low-rank increment:

[0145]

[0146] in Inject coefficients into the hierarchy. Freeze the original weight matrix during the training phase. Update only It can be used in conjunction with weight decay and gradient clipping to ensure small, stable style transfer. After training, Persistence as a set of user-personalized parameters It is used in conjunction with the basic multimodal model during reasoning to achieve a consistent explanation style for the user over a long period of time.

[0147] Configure a LoRA low-rank adaptation layer in the decoding layer, keep the backbone parameters frozen, and train only the LoRA parameters; The mapping is used to generate and inject low-rank increments in an additive manner to complete user-level personalized modeling, avoiding the computing power and storage overhead caused by full parameter fine-tuning.

[0148] The basic multimodal model (frozen) is a pre-trained backbone network whose backbone parameters are frozen (and do not participate in reverse updates). The basic multimodal model is a visual encoder / text encoder-decoder structure used to encode multimodal inputs and generate outputs by the decoder.

[0149] The decoding layer of the basic multimodal model decoder is set to an injectable low-rank adaptation layer. The injection layer (the injectable low-rank adaptation layer) includes the upprojection weights of the feedforward network (FFN). With downward projection weights Optionally includes output projection weights for self-attention. For any weight matrix injected Introducing a low-rank matrix in the input direction Low-rank matrix with output direction To form a low-rank increment.

[0150] After completing the LoRA parameter control injection, the basic multimodal model can now be based on style preference vectors. The intermediate computational paths of multimodal large models are customized and controlled. However, relying solely on personalized preferences may still result in "excessive" or "insufficient" information presentation under different task types and complexities. For example, for extremely simple and intuitive tasks, strictly adhering to the user's preference for "detailed step-by-step explanation" may lead to information redundancy and excessive cognitive load; while for tasks with complex structures or deep reasoning, a preference for "concise expression" may result in information loss and difficulty in understanding. To address this, this module introduces a task complexity-driven dynamic adjustment mechanism. Based on personalized control, it further performs temporary backtracking / convergence based on the current task complexity using temporary adjustment coefficients, achieving optimal interpretation generation under the dual constraints of task and user.

[0151] The generation of the temporary adjustment coefficient includes:

[0152] Perform feature analysis on the task input (i.e., the question entered by the user) to form a task feature vector, and calculate the task complexity score;

[0153] The detail preference in the style preference vector is mapped to the user's expected information density benchmark. The difference between the task complexity score and the benchmark is calculated, and a temporary adjustment coefficient is generated based on the difference.

[0154] The temporary adjustment coefficient is used to correct the gating coefficient during the inference process of the basic multimodal model.

[0155] First, lightweight feature analysis is performed on the task input to extract core indicators that represent the overall difficulty of the task. For text-based tasks, average sentence length, the proportion of subordinate clauses, and the density of key reasoning prompts (such as "therefore," "infer," and "hypothesize") are selected as the main representations of structural and reasoning complexity. For information content measurement, the Type-Token Ratio (TTR) is calculated to reflect the diversity and compactness of the content. For tasks involving images, a weighted sum of the number of main objects and the number of text boxes output by the object detection model is used as an approximate estimate of visual complexity.

[0156] The above features are combined into a low-dimensional task feature vector. The initial complexity score is obtained through linear transformation and the Sigmoid function:

[0157]

[0158] in This input serves only as temporary adjustment for this task, passing to subsequent steps to generate valid parameters, and will not be written back to persistent personalized parameters. A larger value indicates a more complex task.

[0159] Obtaining a task complexity score Then, this invention compares the degree of matching between user preferences and the current task, and determines the direction and magnitude of the rollback accordingly, so as to control the level of cognitive load while ensuring information sufficiency. Let the preference vector be... The "detail" component is Map it to the information density (complexity) scale expected by the user (the larger the scale, the more detailed the information):

[0160]

[0161] Define preference-task difference (temporary adjustment factor):

[0162]

[0163] The temporary adjustment range for this task is determined by the absolute value of the preference-task difference:

[0164]

[0165] in Controlling sensitivity, For the upper limit of the project, This is the sensitivity coefficient.

[0166] The rollback direction is determined by sign ( )Decide:

[0167] like (Preferring simplicity over current complexity), the execution reverts to more complex (refined) tasks;

[0168] like (Preferring more complex tasks while the current task is simpler), the execution reverts to simplicity;

[0169] like (The two are basically matched) and no temporary adjustments will be made.

[0170] These are temporary variables within the current session, used to construct effective gating coefficients / effective low-rank increments. They are cleared upon completion of inference and do not rewrite any persistent weights or gating maps.

[0171] like Figure 1 As shown, the application of the temporary adjustment coefficient includes:

[0172] In the inference process of the basic multimodal model, the gating coefficients generated by the style preference vector are combined with the temporary adjustment coefficients of the current task to construct effective gating coefficients;

[0173] Replace the original gate coefficient with the effective gate coefficient. The calculation process of the low-rank increment is applied to obtain the effective low-rank increment for task correction;

[0174] The effective low-rank increment is combined with the original weight matrix to form the effective weight for the current task, and the effective weight drives the decoder to perform forward autoregressive inference to generate the output sequence (answer), outputting a graphic and textual answer;

[0175] After inference is completed, the effective gating coefficients and effective low-rank increments are released. The effective weights are synthesized online additively based on the above temporary quantities (effective gating coefficients and effective low-rank increments) and the frozen original weight matrix. Therefore, the effective weights are not persisted and are discarded immediately.

[0176] To avoid modifying persistent personalization parameters Under the premise of completing the bidirectional backtracking of the current task, the current task only constructs effective gating coefficients and obtains effective low-rank increments from them during the inference phase. Both only participate in the inference of the current task and are discarded after the inference is completed.

[0177] remember For the first Layer by The generated and persistent rank-based gating, As a neutral gating mechanism, the effective gating for this inference is defined as a piecewise form:

[0178]

[0179] Will Substitute the first The low-rank increment of the layer is constructed only in the forward pass of this task:

[0180]

[0181] in, Nothing changes; the only difference is the use of persistent gating. Replace with the current task After the reasoning was completed, and Discarded and not included in the basic multimodal model of this invention, the basic multimodal model naturally reverts to the original model. and Personalized control style.

[0182] This invention calculates a task complexity score using task feature vectors and compares it with the user's expected information density to obtain a temporary adjustment coefficient, which is used to correct the basic gating coefficient and construct an effective gating coefficient for the current task. During inference, a persistent user preference parameter set is used in conjunction with the temporary adjustment coefficient to generate an effective low-rank increment, thereby preserving long-term personalized preferences while making real-time corrections according to the task. This achieves adaptive output style under tasks of different complexity, solves the problem of mismatch between personalized output and cognitive load in existing methods, and improves the flexibility and reliability of the learning assistant in complex interactive scenarios.

[0183] The method of this invention can be applied to various basic multimodal models without increasing the additional hardware burden. It has good scalability and transferability, and is easy to promote and use in different learning scenarios and cognitive tasks.

[0184] Example 2

[0185] A few-shot-based learning assistant cognitive load adaptive system, used to implement the method of Example 1, such as... Figure 2 As shown, it includes:

[0186] Preference collection module: It is used to set up candidate explanation groups consisting of multiple candidate explanations in the text and image answers provided to users, record the user's selection behavior in multiple rounds of text and image Q&A, extract cognitive preference features of multiple style dimensions from the behavioral data, and generate style preference vectors;

[0187] LoRA training module: used to select a base multimodal model with frozen parameters and adjustable LoRA parameters, map style preference vectors to gating coefficients, train only LoRA parameters while keeping the base multimodal model parameters frozen, and use the low-rank increments obtained from training LoRA parameters as a personalized parameter set to be injected into the low-rank adaptation layer of the base multimodal model decoding layer in a read-only manner.

[0188] Task adjustment module: Before inputting the task into the basic multimodal model, it performs feature analysis on the task input, generates a task complexity score, generates a temporary adjustment coefficient based on the difference between the complexity score and the user's expected information density, temporarily corrects the gating coefficient based on the temporary adjustment coefficient, generates an effective gating coefficient for the current task, calculates the effective low-rank increment using the effective gating coefficient during inference, and no longer retains the effective gating coefficient and effective low-rank increment after inference.

[0189] Example 3

[0190] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method of embodiment 1.

[0191] Example 4

[0192] This application provides a computer-readable storage medium (non-transitory) that stores computer instructions that cause a computer to execute the method of embodiment 1.

[0193] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0194] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0195] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0196] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A cognitive load adaptive method for learning assistants based on few samples, characterized in that, include: The image and text answers provided to users are set up with candidate explanation groups consisting of multiple candidate explanations. The user's selection behavior in multiple rounds of image and text Q&A is recorded. Cognitive preference features of multiple style dimensions are extracted from the behavioral data to generate style preference vectors. Select a base multimodal model with frozen parameters and adjustable LoRA parameters, map style preference vectors to gating coefficients, train only LoRA parameters and keep the base multimodal model parameters frozen, use the low-rank increments obtained from training LoRA parameters as a personalized parameter set, and inject them into the low-rank adaptation layer of the base multimodal model decoding layer in a read-only manner. Before inputting the task into the basic multimodal model, feature analysis is performed on the task input to generate a task complexity score. A temporary adjustment coefficient is generated based on the difference between the complexity score and the user's expected information density. The gating coefficient is temporarily corrected based on the temporary adjustment coefficient to generate an effective gating coefficient for the current task. During the inference process, the effective gating coefficient is used to calculate the effective low-rank increment. After the inference is completed, the effective gating coefficient and the effective low-rank increment are no longer retained.

2. The cognitive load adaptation method for learning assistants based on few samples according to claim 1, characterized in that, The text and image answers provided to the user include candidate explanation groups consisting of multiple candidate explanations, and the user's selection behavior in multiple rounds of text and image Q&A is recorded, including: For multiple style dimensions, corresponding candidate explanation groups are designed. Each candidate explanation in the candidate explanation group differs from other candidate explanations in a single style dimension, while maintaining consistency in other style dimensions. Each group of candidate explanations and the user's selection behavior constitute an explanation sample. Multiple explanation sample sets form a comparison explanation sample set. For each explanation sample, features are extracted in multiple style dimensions to generate multi-dimensional feature vectors, which form a set of dimensional feature vectors.

3. The cognitive load adaptation method for learning assistants based on few samples according to claim 2, characterized in that, The extraction of the style preference vector involves establishing a Bradley-Terry pairwise comparison model for each style dimension, specifically including: Dimensional labels are generated for each style dimension based on the dimensional feature vectors, and weights are assigned to different style dimensions to characterize the degree of influence of the style dimension on the overall preference. The user's selection behavior in the candidate explanation group is weighted to obtain weighted preference data; For each style dimension, parameter estimation is performed on the weighted preference data to obtain the preference intensity of the candidate explanation under that style dimension; The preference intensity of each style dimension is combined according to weights to form a style preference vector that describes the overall preference distribution of users across multiple style dimensions.

4. The cognitive load adaptation method for learning assistants based on few samples according to claim 1, characterized in that, The decoding layer of the basic multimodal model is equipped with an injectable low-rank adaptation layer, and the generation of the gating coefficients includes: Input the style preference vector into the gating to generate a subnetwork; In each low-rank adaptation layer, a linear mapping is performed based on the style preference vector and the preset weight matrix and bias vector to obtain the corresponding gating coefficients. The activation function is used to map each gate coefficient so that its value is within a preset range; Each gating coefficient is applied to the corresponding low-rank adaptation layer.

5. The cognitive load adaptation method for learning assistants based on few samples according to claim 4, characterized in that, The training LoRA parameters include: A low-rank decomposition matrix is ​​introduced into the target layer to form an increment, resulting in a low-rank matrix in the input direction and a low-rank matrix in the output direction. The gating coefficients generated from the style preference vector are applied as scaling factors to the low-rank decomposition matrix, and the low-rank increment is calculated from this. The low-rank increment is superimposed on the frozen original weight matrix, and forward computation is performed; Backpropagation is performed based on training error; Iterate until convergence, and use the low-rank increments obtained from training as a personalized parameter set. The personalized parameter set is injected into the decoding layer in read-only mode during the inference period of the basic multimodal model.

6. The cognitive load adaptation method for learning assistants based on few samples according to claim 1, characterized in that, The generation of the temporary adjustment coefficient includes: Perform feature analysis on the task input to form a task feature vector, and calculate the task complexity score; The detail preference in the style preference vector is mapped to the user's expected information density benchmark. The difference between the task complexity score and the benchmark is calculated, and a temporary adjustment coefficient is generated based on the difference. The temporary adjustment coefficient is used to correct the gating coefficient during the inference process of the basic multimodal model.

7. The cognitive load adaptation method for learning assistants based on few samples according to claim 6, characterized in that, The application of the temporary adjustment factor includes: In the inference process of the basic multimodal model, the gating coefficients generated by the style preference vector are combined with the temporary adjustment coefficients of the current task to construct effective gating coefficients; The effective gating coefficients are used to replace the original gating coefficients and applied to the calculation process of the low-rank increment to obtain the effective low-rank increment for task correction. The effective low-rank increment is combined with the original weight matrix to form the effective weight for the current task, and the effective weight drives the decoder to perform forward autoregressive inference to generate an output sequence and output a graphic and textual answer. After inference is complete, release the effective gating coefficients and effective low-rank increments.

8. A few-shot-based learning assistant cognitive load adaptation system, used to execute the few-shot-based learning assistant cognitive load adaptation method according to any one of claims 1-7, characterized in that, include: Preference collection module: It is used to set up candidate explanation groups consisting of multiple candidate explanations in the text and image answers provided to users, record the user's selection behavior in multiple rounds of text and image Q&A, extract cognitive preference features of multiple style dimensions from the behavioral data, and generate style preference vectors; LoRA training module: used to select a base multimodal model with frozen parameters and adjustable LoRA parameters, map style preference vectors to gating coefficients, train only LoRA parameters while keeping the base multimodal model parameters frozen, and use the low-rank increments obtained from training LoRA parameters as a personalized parameter set to be injected into the low-rank adaptation layer of the base multimodal model decoding layer in a read-only manner. Task adjustment module: Before inputting the task into the basic multimodal model, it performs feature analysis on the task input, generates a task complexity score, generates a temporary adjustment coefficient based on the difference between the complexity score and the user's expected information density, temporarily corrects the gating coefficient based on the temporary adjustment coefficient, generates an effective gating coefficient for the current task, calculates the effective low-rank increment using the effective gating coefficient during inference, and no longer retains the effective gating coefficient and effective low-rank increment after inference.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the cognitive load adaptation method for a learning assistant based on few samples as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the cognitive load adaptation method for a learning assistant based on few samples as described in any one of claims 1 to 7.

Images