Parameter region positioning and null space projection-based large language model learning method and device, electronic equipment, storage medium and program product

Abstract

This disclosure provides a method, apparatus, electronic device, storage medium, and program product for learning a large language model based on parametric region localization and null projection. The method includes: acquiring a calibration dataset and a training dataset for the task to be learned, and acquiring the target projection matrix of the learned task; performing importance analysis on the parameters of the pre-trained large language model based on the calibration dataset, and determining the important parameters of the pre-trained large language model based on the analysis results; calculating the original parameter gradients of the pre-trained large language model based on the training dataset; correcting the original parameter gradients based on the important parameters and the target projection matrix to obtain the corrected parameter gradients; and adjusting the parameters of the pre-trained large language model based on the corrected parameter gradients to obtain the trained large language model. This method balances stability, flexibility, and sustainability.

Description

Technical Field

[0001] This disclosure generally relates to the field of artificial intelligence, and more specifically, to a method, apparatus, electronic device, storage medium, and program product for learning large language models based on parametric region localization and null spatial projection. Background Technology

[0002] In recent years, large language models (LLMs), represented by Llama and Qwen, have made significant progress in the field of natural language processing, becoming a key technology for dealing with constantly evolving real-world scenarios, continuously updating domain knowledge, and meeting diverse user needs. To enable large models to adapt to streaming tasks and achieve long-term knowledge iteration, continuous learning (CL) capabilities are particularly important.

[0003] An ideal continuous learning method for large models not only needs to efficiently handle current streaming tasks, but also should maintain cognitive coherence and optimal resource allocation throughout long-term knowledge evolution. However, the following existing major continuous learning schemes all have shortcomings.

[0004] (1) Full Fine-Tuning: This method prioritizes the adaptability of the current task by globally updating the parameters, which means it has good plasticity. However, this not only leads to catastrophic forgetting of past knowledge, which sacrifices stability, but also prematurely exhausts the representational power of the parameter space, seriously damaging the model's potential for future expansion.

[0005] (2) Regularization-based methods: This approach attempts to protect old knowledge (emphasizing stability) by imposing rigid constraints on parameter changes or selectively freezing weights. However, this "defensive" rigid mechanism severely limits the model's plasticity, making it difficult for the model to fully adapt to current new tasks or effectively cope with future changes in requirements.

[0006] (3) Architecture-based methods: This approach relies on adding dedicated modules or adapters for each new task to mitigate inter-task interference, such as the LoRA (Low-Rank Adaptation) module. While this physical isolation mechanism alleviates forgetting to some extent, it leads to a linear increase in the number of model parameters (i.e., unbounded parameter expansion) as the number of tasks increases. This inefficient approach to architectural expansion is unsustainable in long-term learning scenarios.

[0007] In summary, the core technical bottleneck currently facing continuous learning of large language models lies in how to effectively reserve parameter capacity for future adaptability while balancing the immediate plasticity of the current task with the long-term stability of past knowledge. Summary of the Invention

[0008] This disclosure provides a method, apparatus, electronic device, storage medium, and program product for learning large language models based on parametric region localization and null spatial projection, for solving at least one of the above-mentioned problems.

[0009] According to a first aspect of the present disclosure, a method for learning a large language model based on parametric region localization and null projection is provided, comprising: acquiring a calibration dataset and a training dataset of a task to be learned, and acquiring a target projection matrix of a learned task, wherein the target projection matrix is ​​determined based on the null projection matrix of the non-centered covariance matrix of the input activation feature information of the learned task; performing importance analysis on the parameters of a pre-trained large language model according to the calibration dataset, and determining the important parameters of the pre-trained large language model according to the analysis results; calculating the original parameter gradient of the pre-trained large language model according to the training dataset; correcting the original parameter gradient according to the important parameters and the target projection matrix to obtain a corrected parameter gradient; and adjusting the parameters of the pre-trained large language model according to the corrected parameter gradient to obtain a trained large language model.

[0010] Optionally, the pre-trained large language model includes multiple neurons and an output layer, wherein the importance analysis of the parameters of the pre-trained large language model based on the calibration dataset includes: determining the importance score of a specified neuron among the multiple neurons based on the calibration dataset, wherein the importance score of the specified neuron is calculated using the following formula: in, Represents the specified neuron, This represents the importance score of the specified neuron. This refers to the calibration dataset. This represents a calibration data point. This represents the input data in the calibration data. This indicates the target output corresponding to the input data. Indicates the vector The predicted probability of the target output is obtained from the output layer. This represents the input vector of the specified neuron. This represents the output vector of the specified neuron.

[0011] Optionally, the plurality of neurons are divided into a plurality of model components, wherein determining the importance score of a specified neuron among the plurality of neurons based on the calibration dataset includes: determining the importance score of a specified model component among the plurality of model components based on the calibration dataset; determining a preset number of model components with the highest importance scores among the plurality of model components as candidate model components; determining the importance score of each neuron in each candidate model component based on the calibration dataset; wherein determining the important parameters of the pre-trained large language model based on the analysis results includes: determining a plurality of important neurons from the neurons of all the candidate model components based on the importance scores of each neuron, and using the parameters of the plurality of important neurons as the important parameters of the pre-trained large language model.

[0012] Optionally, the target projection matrix is ​​obtained through the following steps: constructing a globally accumulated non-centralized covariance matrix with all initial elements being zero; in response to completing a learned task, calculating the non-centralized covariance matrix of the input activation feature information corresponding to the training dataset of the learned task, and accumulating the calculated non-centralized covariance matrix into the globally accumulated non-centralized covariance matrix; in response to the instruction to obtain the target projection matrix, using the null space projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix.

[0013] Optionally, the step of using the null-space projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix in response to the instruction to obtain the target projection matrix includes: accumulating the non-centralized covariance matrix of the input activation feature information corresponding to the anchor dataset into the globally accumulated non-centralized covariance matrix in response to the instruction to obtain the target projection matrix, wherein the anchor dataset contains general domain knowledge data; and using the null-space projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix.

[0014] Optionally, the step of correcting the original parameter gradient based on the important parameters and the target projection matrix to obtain the corrected parameter gradient includes: correcting the original parameter gradient using the following formula to obtain the corrected parameter gradient: ;in, This represents the gradient of the correction parameter. This represents the gradient of the original parameters. It is a binary mask used to represent the important parameters. This represents the target projection matrix.

[0015] According to a second aspect of the present disclosure, a large language model learning apparatus based on parametric region localization and null projection is provided, comprising: an acquisition unit configured to acquire a calibration dataset and a training dataset of a task to be learned, and to acquire a target projection matrix of a learned task, wherein the target projection matrix is ​​determined based on the null projection matrix of the non-centered covariance matrix of the input activation feature information of the learned task; an analysis unit configured to perform importance analysis on the parameters of a pre-trained large language model based on the calibration dataset, and to determine the important parameters of the pre-trained large language model based on the analysis results; a calculation unit configured to calculate the original parameter gradient of the pre-trained large language model based on the training dataset; a correction unit configured to correct the original parameter gradient based on the important parameters and the target projection matrix to obtain a corrected parameter gradient; and a parameter tuning unit configured to adjust the parameters of the pre-trained large language model based on the corrected parameter gradient to obtain a trained large language model.

[0016] Optionally, the pre-trained large language model includes multiple neurons and an output layer, and the analysis unit is further configured to determine the importance score of a specified neuron among the multiple neurons based on the calibration dataset, wherein the importance score of the specified neuron is calculated using the following formula: in, Represents the specified neuron, This represents the importance score of the specified neuron. This refers to the calibration dataset. This represents a calibration data point. This represents the input data in the calibration data. This indicates the target output corresponding to the input data. Indicates the vector The predicted probability of the target output is obtained from the output layer. This represents the input vector of the specified neuron. This represents the output vector of the specified neuron.

[0017] Optionally, the plurality of neurons are divided into a plurality of model components, and the analysis unit is further configured to: determine the importance score of a specified model component among the plurality of model components based on the calibration dataset; determine a preset number of model components with the highest importance scores among the plurality of model components as candidate model components; determine the importance score of each neuron in each candidate model component based on the calibration dataset; determine a plurality of important neurons from the neurons of all candidate model components based on the importance scores of each neuron, and use the parameters of the plurality of important neurons as important parameters of the pre-trained large language model.

[0018] Optionally, the target projection matrix is ​​obtained through the following steps: constructing a globally accumulated non-centralized covariance matrix with all initial elements being zero; in response to completing a learned task, calculating the non-centralized covariance matrix of the input activation feature information corresponding to the training dataset of the learned task, and accumulating the calculated non-centralized covariance matrix into the globally accumulated non-centralized covariance matrix; in response to the instruction to obtain the target projection matrix, using the null space projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix.

[0019] Optionally, the step of using the null-space projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix in response to the instruction to obtain the target projection matrix includes: accumulating the non-centralized covariance matrix of the input activation feature information corresponding to the anchor dataset into the globally accumulated non-centralized covariance matrix in response to the instruction to obtain the target projection matrix, wherein the anchor dataset contains general domain knowledge data; and using the null-space projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix.

[0020] Optionally, the correction unit is further configured to correct the original parameter gradient using the following formula to obtain the corrected parameter gradient: ;in, This represents the gradient of the correction parameter. This represents the gradient of the original parameters. It is a binary mask used to represent the important parameters. This represents the target projection matrix.

[0021] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform a large language model learning method based on parametric region localization and null spatial projection according to exemplary embodiments of the present disclosure.

[0022] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein instructions in the computer-readable storage medium, when executed by at least one processor, cause the at least one processor to perform a large language model learning method based on parametric region localization and null-spatial projection according to an exemplary embodiment of the present disclosure.

[0023] According to a fifth aspect of the present disclosure, a computer program product is provided, including computer instructions that, when executed by at least one processor, cause the at least one processor to perform a large language model learning method based on parametric region localization and null-spatial projection according to an exemplary embodiment of the present disclosure.

[0024] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects: According to the large language model learning method, device, electronic device, storage medium and program product based on parameter region localization and null spatial projection of this disclosure, a unified solution that takes into account the past (stability), the present (plasticity) and the future (sustainability) is provided. By simulating the "functional partitioning" mechanism of the biological brain, on the one hand, by accurately locating sparse specific neurons, the model is endowed with efficient plasticity for the current learning task, which can quickly adapt to new tasks and efficiently acquire new knowledge, and can reserve sufficient parameter capacity for long-term adaptation in the future, ensuring that the model still has learning efficiency when facing subsequent challenges; on the other hand, through null spatial projection technology, the parameter update is mathematically strictly guaranteed to be orthogonal to the old knowledge representation learned from the learned task, thereby achieving excellent stability of past knowledge, effectively consolidating previously learned knowledge and avoiding catastrophic forgetting.

[0025] Meanwhile, unlike traditional architecture-based methods, the embodiments disclosed herein are optimized within a fixed parameter space, eliminating the need to add additional adapter modules for each new task and avoiding parameter expansion. This avoids the problem of linear expansion of the model size with the number of tasks, resulting in extremely high architectural efficiency.

[0026] Furthermore, the embodiments of this disclosure utilize the statistical properties of a decentralized covariance matrix to characterize prior knowledge, eliminating the need to store any original text data of previously learned tasks. This completely avoids the risk of data privacy leakage and storage overhead, making it a strictly rehearsal-free method.

[0027] Therefore, the embodiments of this disclosure can achieve efficient, stable and sustainable continuous learning of large models without increasing the model size or replaying the original data.

[0028] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0030] Figure 1This is a flowchart of a large language model learning method based on parametric region localization and null-spatial projection, according to an exemplary embodiment of the present disclosure.

[0031] Figure 2 This is a schematic diagram of the overall architecture of a large language model learning method based on parametric region localization and null-spatial projection according to a specific embodiment of the present disclosure.

[0032] Figure 3 This is a schematic diagram comparing the characteristics of a large language model learning method based on parametric region localization and null spatial projection, according to a specific embodiment of the present disclosure, with related technologies.

[0033] Figure 4 This is a comparison of ablation experiment results for a large language model learning method based on parametric region localization and null spatial projection according to exemplary embodiments of the present disclosure.

[0034] Figure 5 This is a block diagram of a large language model learning apparatus based on parametric region localization and null-spatial projection according to an exemplary embodiment of the present disclosure.

[0035] Figure 6 This is a block diagram of an electronic device according to exemplary embodiments of the present disclosure. Detailed Implementation

[0036] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0037] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0038] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0039] Hereinafter, with reference to the accompanying drawings, a method, apparatus, electronic device, storage medium, and program product for learning large language models based on parametric region localization and null-spatial projection according to exemplary embodiments of the present disclosure will be described in detail.

[0040] Figure 1 This is a flowchart of a large language model learning method based on parametric region localization and null-spatial projection, according to an exemplary embodiment of the present disclosure. Figure 2 This is a schematic diagram of the overall architecture of a large language model learning method based on parametric region localization and null-spatial projection, according to an exemplary embodiment of this disclosure. This method can be executed on an electronic device with sufficient computing power.

[0041] Reference Figure 1 In step S101, the calibration dataset and training dataset of the task to be learned are obtained, and the target projection matrix of the learned task is obtained.

[0042] It should be understood that the task to be learned is the task currently to be learned in the streaming task, and the learned task is the task that has already been learned in the streaming task. The target projection matrix is ​​determined based on the null space projection matrix of the non-centered covariance matrix of the input activation feature information of the learned task (i.e., the non-centered covariance matrix of the input activation feature information of the training dataset of the learned task). This step can obtain the input activation feature information of the previously learned task, construct a non-centered covariance matrix representing the knowledge of the old task, and perform singular value decomposition based on the non-centered covariance matrix to construct the null space projection matrix, thereby obtaining the target projection matrix. As an example, the null space projection matrix of the non-centered covariance matrix of the input activation feature information of the learned task can be directly used as the target projection matrix, or further processing can be performed, which is not limited in this disclosure.

[0043] In step S102, the importance of the parameters of the pre-trained large language model is analyzed based on the calibration dataset, and the important parameters of the pre-trained large language model are determined based on the analysis results.

[0044] It should be understood that a pre-trained large language model is a model that has already completed learning for the aforementioned learned tasks. This step is based on the training dataset of the current task to be learned. For example, a neuron-level attribution strategy can be used to analyze the importance of model parameters to the task to be learned, to identify important parameters, and accordingly, the model parameter space is topologically divided into sparse activation regions (regions where important parameters are located) and frozen regions. Only parameters in the activation regions (i.e., important parameters) are allowed to be updated, while the vast majority of parameters remain frozen to reserve capacity for future use. As an example, a binary mask corresponding to the important parameters can be generated, and this binary mask can be used to represent the important parameters.

[0045] In step S103, the gradient of the original parameters of the pre-trained large language model is calculated based on the training dataset.

[0046] This step corresponds to the forward propagation process in conventional model training.

[0047] In step S104, the original parameter gradient is corrected based on the important parameters and the target projection matrix to obtain the corrected parameter gradient.

[0048] This step uses important parameters to spatially filter the gradient of the original parameters to achieve topological isolation, and uses the target projection matrix to orthogonally correct the gradient direction to achieve geometric constraints, thereby correcting the parameter gradient.

[0049] Optionally, step S104 includes: correcting the original parameter gradient using the following formula to obtain the corrected parameter gradient: .

[0050] In the above formula, This indicates the gradient of the corrected parameters. Represents the gradient of the original parameters. It is a binary mask used to represent important parameters. Represents the target projection matrix. This represents the Hadamard product. Specifically, the Hadamard product of the original parameter gradient and the binary mask is first calculated to achieve spatial filtering. The resulting matrix is ​​then multiplied with the target projection matrix to achieve geometric constraints. The result is the corrected parameter gradient.

[0051] In step S105, the parameters of the pre-trained large language model are adjusted according to the correction parameter gradient to obtain the trained large language model.

[0052] This step corresponds to the backpropagation process in regular model training, except that the parameter gradients used are replaced with modified parameter gradients instead of the original parameter gradients.

[0053] The large language model learning method based on parametric region localization and null-spatial projection, as described in the exemplary embodiments of this disclosure, provides a unified solution that balances past (stability), present (plasticity), and future (sustainability). By simulating the "functional partitioning" mechanism of the biological brain, it endows the model with highly efficient plasticity for the current learning task by precisely locating sparse, specific neurons. This allows the model to quickly adapt to new tasks and efficiently acquire new knowledge, while reserving sufficient parameter capacity for long-term adaptation in the future, ensuring that the model still has learning efficiency when facing subsequent challenges. On the other hand, through null-spatial projection technology, it mathematically guarantees that parameter updates are orthogonal to the representations of old knowledge learned from previously learned tasks, thereby achieving excellent stability of past knowledge, effectively consolidating previously acquired knowledge, and avoiding catastrophic forgetting.

[0054] Meanwhile, unlike traditional architecture-based methods, the exemplary embodiments disclosed herein are optimized within a fixed parameter space, eliminating the need to add additional adapter modules for each new task and avoiding parameter expansion. This avoids the problem of linear expansion of the model size with the number of tasks, resulting in extremely high architectural efficiency.

[0055] Furthermore, the exemplary embodiments of this disclosure utilize the statistical properties of a decentralized covariance matrix to characterize prior knowledge, eliminating the need to store any original text data of previously learned tasks, thus completely avoiding the risk of data privacy leakage and storage overhead, and representing a strictly no-replay method.

[0056] Therefore, the exemplary embodiments of this disclosure enable efficient, stable, and sustainable continuous learning of large models without increasing the model size or requiring replay of the original data.

[0057] The following section further describes a large language model learning method based on parametric region localization and null-spatial projection according to exemplary embodiments of the present disclosure.

[0058] Regarding the determination of important parameters, optionally, the pre-trained large language model includes multiple neurons and an output layer. Step S102, which involves performing an importance analysis on the parameters of the pre-trained large language model based on a calibration dataset, includes: determining the importance score of a specified neuron among the multiple neurons based on the calibration dataset. The importance score of the specified neuron is calculated using the following formula:

[0059] In the above formula, Indicates a specified neuron, This represents the importance score of a specified neuron. This represents the calibration dataset. This indicates the number of calibration data points in the calibration dataset. This represents a calibration data point. This represents the input data in the calibration data. This indicates the target output corresponding to the input data. Indicates the vector The predicted probabilities of the target output obtained from the input-output layer. This represents the input vector of a specified neuron. This represents the output vector of a specified neuron. A neuron's output vector is computed using the Logit Lens technique. The output vector Add residual flow Subsequently, the log-probability gain of the predicted probability of the target output (specifically, the target token) of the learned task is used as the importance score of the neuron, enabling quantitative analysis of the importance of a specified neuron. Of course, other processing can be performed on the predicted probabilities corresponding to the neuron's input and output vectors to obtain the neuron's importance score, or other reasonable methods can be used to calculate the importance score; this disclosure does not impose any limitations on this.

[0060] Optionally, the aforementioned multiple neurons are divided into multiple model components. Determining the importance score of a specific neuron among the multiple neurons based on the calibration dataset includes: determining the importance score of a specific model component among the multiple model components based on the calibration dataset; identifying a predetermined number of model components with the highest importance scores among the multiple model components as candidate model components; determining the importance score of each neuron in each candidate model component based on the calibration dataset; and determining the important parameters of the pre-trained large language model based on the analysis results in step S10, including: identifying multiple important neurons from the neurons of all candidate model components based on the importance scores of each neuron, and using the parameters of these multiple important neurons as the important parameters of the pre-trained large language model. By employing a hierarchical screening strategy from hierarchical filtering to neuron-level locking, the specific neuron clusters most sensitive to the current learning task can be accurately located, improving the efficiency of important parameter determination. Specifically, in the hierarchical filtering stage, the overall importance score of each model component (including but not limited to multi-head self-attention modules and feedforward network modules in each layer) is calculated, and the Top-K layers with the highest importance scores are selected as candidate model components. Then, in the neuron-level locking stage, within these Top-K layers, the importance score of each neuron or attention head is further calculated, and the set of neurons with relatively high importance scores is selected as activation regions. For example, in the neuron-level locking stage, the set of neurons with importance scores higher than a preset threshold can be selected, or neurons with a set number of parameters can be selected in descending order of importance scores. For the latter, the set number of parameters can be a fixed value or a preset proportion of the total number of model parameters. The preset proportion can be flexibly configured according to the complexity of the current learning task, for example, but not limited to 5%.

[0061] Optionally, the target projection matrix can be obtained through the following steps: constructing a globally accumulated non-centralized covariance matrix with all initial elements being zero; in response to completing a learned task, calculating the non-centralized covariance matrix of the input activation feature information corresponding to the training dataset of the learned task, and accumulating the calculated non-centralized covariance matrix into the globally accumulated non-centralized covariance matrix; in response to the instruction to obtain the target projection matrix, using the null space projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix. By employing a streaming recursive accumulation strategy to maintain the non-centralized covariance matrices of historical learned tasks, that is, summing element-wise these non-centralized covariance matrices with identical dimensions, it is possible to utilize the properties of linear algebra (i.e., the null space of the input activation matrix is ​​equivalent to the null space of its non-centralized covariance matrix, and the null space of the sum of multiple positive semi-definite matrices is equivalent to the intersection of the null spaces of each matrix) to calculate the common null space of all previously learned tasks without storing the original historical data.

[0062] Optionally, the above-described response to the instruction to obtain the target projection matrix, using the null-space projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix, includes: in response to the instruction to obtain the target projection matrix, accumulating the non-centralized covariance matrix of the input activation feature information corresponding to the anchor dataset into the globally accumulated non-centralized covariance matrix, wherein the anchor dataset contains general domain knowledge data; and using the null-space projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix. By introducing a general knowledge anchoring mechanism, the exemplary embodiments of this disclosure effectively reduce the risk of degradation of general reasoning ability (such as logical and mathematical ability) caused by parameter drift of the model while learning new knowledge in a specific domain, thus helping to protect general intelligence. It should be understood that the calculation method of the non-centralized covariance matrix of the input activation feature information corresponding to the anchor dataset is the same as the calculation method of the non-centralized covariance matrix of the input activation feature information of the previously learned task.

[0063] Figure 2 This is a schematic diagram of the overall architecture of a large language model learning method based on parametric region localization and null-spatial projection according to a specific embodiment of the present disclosure.

[0064] Reference Figure 2 This specific embodiment includes three core steps.

[0065] 1. Activation of task-specific parameter areas (corresponding to) Figure 2 Part A).

[0066] The goal of this step is to identify and activate the clusters of neurons most sensitive to the current new task, while freezing other parameters.

[0067] Specifically, when faced with a new downstream task Tk, the first step is to use a neuron-level knowledge attribution strategy for localization.

[0068] (1) Importance score calculation: The importance score of the model components is calculated using the Logit Lens technique. v (e.g., a specific layer or neuron) for the target token y The logarithmic probability gain. The calculation formula is as follows:

[0069] in, Represents the input representation. Representation Component v The output vector, This is the calibration dataset for the current task.

[0070] (2) Hierarchical filtering: To improve efficiency, a coarse-to-fine filtering method is adopted. First, hierarchical filtering is performed to retain the most important Top- Layers; then, neuron-level targeting is performed within these layers to select high-scoring neurons.

[0071] (3) Generate mask: Generate a binary mask based on the filtering results. ,in Indicates activation. This indicates a freeze. This topology isolation mechanism ensures that only about 5% of the core functional parameters are updated, thus reserving about 95% of the parameter space for future tasks.

[0072] 2. Construction of the null projection matrix (corresponding to) Figure 2 Part B).

[0073] The goal of this step is to construct a mathematical constraint that ensures parameter updates do not interfere with the old task.

[0074] (1) Calculation of non-centralized covariance matrix.

[0075] To preserve knowledge from previous tasks, this specific implementation does not store the original data, but instead maintains a decentralized covariance matrix Σ. This is achieved using properties of linear algebra. Σ is updated through streaming recursive accumulation.

[0076] (2) Construction of the null projection matrix.

[0077] For the accumulated non-centralized covariance matrix Perform singular value decomposition (SVD) to obtain Select the eigenvector corresponding to the largest eigenvalue. It represents the feature space of old knowledge.

[0078] (3) Calculate the projector.

[0079] Constructing the null projection matrix The purpose of this matrix is ​​to project any vector onto a direction orthogonal to prior knowledge.

[0080] 3. Dual-constraint optimization (corresponding to) Figure 2 Part C).

[0081] During training, the gradient is adjusted in real time to satisfy both constraints mentioned above.

[0082] set up Given the gradients of the original parameters for the current task, the final parameter update rule is:

[0083] in, It's the learning rate. It's a spatial constraint that ensures updates only occur in the active region; right multiplication. It's a directional constraint, ensuring that the update direction is orthogonal to the old knowledge. For example... Figure 2 As shown in section (C), the obtained parameter update ΔW is both sparse and undisturbed.

[0084] It should be noted that, Figure 2 The layer mask obtained after layer filtering of part A in the middle Used to activate a specific layer and points to the backpropagation of part C, not referring to layer mask-based propagation. Instead of changing the fundamental computational logic of backpropagation, this refers to the real-time interception and correction of the original parameter gradients generated by backpropagation. The specific logic is as follows: 1. The admission function of hierarchical filtering: Before backpropagation begins, hierarchical filtering determines the "active layer" and the "frozen layer." For the frozen layer, its gradient update is blocked, and no weight bias is generated. 2. Real-time gradient correction: For the active layer, after the original parameter gradient G is generated by backpropagation, a binary mask is used... M (Achieving topological isolation in spatial dimensions) and null projection matrix P By implementing orthogonal constraints in the algebraic dimension, G is corrected to obtain the gradient of the corrected parameters. This logic of "first determining the point (hierarchical admission), then correcting the deviation (gradient correction)" ensures that PaRSP can accurately complete the learning task within a specific region.

[0085] like Figure 3 As shown, the large language model learning method based on parameter region localization and null-space projection according to a specific embodiment of this disclosure addresses the shortcomings of related technologies such as full-parameter fine-tuning, regularization methods, and architecture-based methods in achieving a balance between stability (past), plasticity (present), and sustainability (future). It achieves plasticity through sparse activation, stability through null-space orthogonality, and reserves a large amount of parameter space for future tasks. This method can be named the Null-Space Constrained Parameter RegionSpecificity Method (PaRSP).

[0086] To verify the effectiveness of this specific embodiment, extensive experiments were conducted on the Llama-3.1-8B and Qwen-2.5-7B models.

[0087] 1. Ablation experiment analysis.

[0088] like Figure 4 As shown, ablation experiments were performed on the standard continuous learning benchmark (SC Order 1).

[0089] (1) When the null-space constraint is removed, the model’s resistance to forgetting (stability) drops sharply and the forgetting rate increases significantly (Figure 3, right bar chart), which proves that null-space projection is the key to protecting old knowledge.

[0090] (2) When important parameter localization is removed (without Attribution) and random masking is used for random parameter updates, both the model's average performance (AP) and final performance (FP) decrease. Figure 4 (Left and middle bar charts) This demonstrates the importance of precise positioning for learning new tasks (plasticity).

[0091] (3) This specific embodiment (PaRSP) achieves the best balance in all aspects.

[0092] 2. Overall performance comparison.

[0093] Table 1: Main Experiment Results Data Table

[0094] (1) Main Experiment Results: Table 1 shows the main experiment results of this specific embodiment on the Standard CL, Long Sequence, and TRACE benchmarks (a comprehensive benchmark designed specifically for evaluating the continuous learning ability of large language models). As shown in Table 1, on the three benchmarks of Standard CL, Long Sequence, and TRACE, this specific embodiment (PaRSP) significantly outperforms baseline methods such as SeqLoRA and O-LoRA in terms of final performance (FP) and forgetting resistance (Forget), and does not require data replay.

[0095] Table 2: Cross-task generalization ability test results data table

[0096] (2) Generalization ability: Table 2 shows the test results of the cross-task generalization ability of this specific embodiment on unseen tasks, demonstrating the results of using different methods to handle unseen tasks. Zero-Shot means zero-shot learning, which means that when the model faces a new task, it has not received any special training for that task (i.e., it has not seen any examples of that task), but directly completes the task based on existing knowledge or general abilities. As shown in Table 2, on unseen tasks (such as MMLU, GSM8K, etc. in Table 2, which are benchmark datasets used to evaluate artificial intelligence models), this specific embodiment effectively maintains the model's general reasoning ability and avoids the general intelligence degradation caused by catastrophic forgetting common in traditional methods such as SeqLoRA.

[0097] In summary, this specific embodiment successfully solves the three major challenges of stability, plasticity, and sustainability in the continuous learning of large language models through its unique parameter region localization and null projection mechanism.

[0098] Figure 5 This is a block diagram of a large language model learning apparatus based on parametric region localization and null-spatial projection, according to an exemplary embodiment of the present disclosure. (Refer to...) Figure 5 The large language model learning device 500 based on parametric region localization and null projection includes an acquisition unit 501, an analysis unit 502, a calculation unit 503, a correction unit 504, and a parameter tuning unit 505.

[0099] The acquisition unit 501 can acquire the calibration dataset and training dataset of the task to be learned, and acquire the target projection matrix of the learned task. The target projection matrix is ​​determined based on the null projection matrix of the non-centered covariance matrix of the input activation feature information of the learned task.

[0100] Analysis unit 502 can perform importance analysis on the parameters of the pre-trained large language model based on the calibration dataset, and determine the important parameters of the pre-trained large language model based on the analysis results.

[0101] The computing unit 503 can calculate the gradient of the original parameters of the pre-trained large language model based on the training dataset.

[0102] The correction unit 504 can correct the original parameter gradient based on the important parameters and the target projection matrix to obtain the corrected parameter gradient.

[0103] The parameter tuning unit 505 can adjust the parameters of the pre-trained large language model according to the correction parameter gradient to obtain the trained large language model.

[0104] Optionally, the pre-trained large language model includes multiple neurons and an output layer. The analysis unit 502 can also determine the importance score of a specific neuron among the multiple neurons based on the calibration dataset. The importance score of the specific neuron is calculated using the following formula: in, Indicates a specified neuron, This represents the importance score of a specified neuron. This represents the calibration dataset. This represents a calibration data point. This represents the input data in the calibration data. This indicates the target output corresponding to the input data. Indicates the vector The predicted probabilities of the target output obtained from the input-output layer. This represents the input vector of a specified neuron. This represents the output vector of a specified neuron.

[0105] Optionally, multiple neurons are divided into multiple model components, and the analysis unit 502 may further: determine the importance score of a specified model component among the multiple model components based on the calibration dataset; determine a preset number of model components with the highest importance scores among the multiple model components as candidate model components; determine the importance score of each neuron in each candidate model component based on the calibration dataset; determine multiple important neurons from the neurons of all candidate model components based on the importance scores of each neuron, and use the parameters of the multiple important neurons as important parameters of the pre-trained large language model.

[0106] Optionally, the target projection matrix is ​​obtained through the following steps: constructing a globally accumulated non-centralized covariance matrix with all initial elements being zero; in response to completing a learned task, calculating the non-centralized covariance matrix of the input activation feature information corresponding to the training dataset of the learned task, and accumulating the calculated non-centralized covariance matrix into the globally accumulated non-centralized covariance matrix; in response to the instruction to obtain the target projection matrix, using the null space projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix.

[0107] Optionally, in response to the instruction to obtain the target projection matrix, the null projection matrix of the globally accumulated non-centralized covariance matrix is ​​used as the target projection matrix, including: in response to the instruction to obtain the target projection matrix, accumulating the non-centralized covariance matrix of the input activation feature information corresponding to the anchor dataset into the globally accumulated non-centralized covariance matrix, wherein the anchor dataset contains general domain knowledge data; and using the null projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix.

[0108] Optionally, the correction unit 504 can also correct the original parameter gradient using the following formula to obtain the corrected parameter gradient: ;in, This indicates the gradient of the corrected parameters. Represents the gradient of the original parameters. It is a binary mask used to represent important parameters. This represents the target projection matrix.

[0109] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0110] Figure 6 A structural block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown.

[0111] Reference Figure 6 The electronic device 600 includes at least one memory 601 and at least one processor 602. The at least one memory 601 stores computer-executable instructions that, when executed by the at least one processor 602, cause the at least one processor to perform the target correspondence method as described in the exemplary embodiments above.

[0112] As an example, electronic device 600 may be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, electronic device 600 is not necessarily a single electronic device 600, but may be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 600 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device 600 locally or remotely (e.g., via wireless transmission) through an interface.

[0113] In electronic device 600, processor 602 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor 602 may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.

[0114] The processor 602 can execute instructions or code stored in the memory 601, which can also store data. Instructions and data can also be sent and received via a network through a network interface device, which can employ any known transmission protocol.

[0115] The memory 601 may be integrated with the processor 602, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 601 may include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory 601 and the processor 602 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 602 to read files stored in the memory.

[0116] In addition, the electronic device 600 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device 600 can be interconnected via a bus and / or network.

[0117] According to exemplary embodiments of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein the instructions, when executed by at least one processor, cause at least one processor to perform the target-correspondence method as described in the exemplary embodiments above. Examples of computer-readable storage media herein include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0118] According to exemplary embodiments of the present disclosure, a computer program product may also be provided, including computer instructions that, when executed by at least one processor, perform the target correspondence method as described in the exemplary embodiments above.

[0119] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0120] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for learning large language models based on parametric region localization and null projection, characterized in that, include: Obtain the calibration dataset and training dataset of the task to be learned, and obtain the target projection matrix of the learned task, wherein the target projection matrix is ​​determined based on the null space projection matrix of the non-centered covariance matrix of the input activation feature information of the learned task. Based on the calibration dataset, the importance of the parameters of the pre-trained large language model is analyzed, and the important parameters of the pre-trained large language model are determined based on the analysis results. Based on the training dataset, calculate the gradient of the original parameters of the pre-trained large language model; Based on the important parameters and the target projection matrix, the original parameter gradient is corrected to obtain the corrected parameter gradient; Based on the correction parameter gradient, the parameters of the pre-trained large language model are adjusted to obtain the trained large language model.

2. The large language model learning method as described in claim 1, characterized in that, The pre-trained large language model includes multiple neurons and an output layer, wherein the importance analysis of the parameters of the pre-trained large language model based on the calibration dataset includes: Based on the calibration dataset, the importance score of a specified neuron among the plurality of neurons is determined, wherein the importance score of the specified neuron is calculated using the following formula: in, Represents the specified neuron, This represents the importance score of the specified neuron. This refers to the calibration dataset. This represents a calibration data point. This represents the input data in the calibration data. This indicates the target output corresponding to the input data. Indicates the vector The predicted probability of the target output is obtained from the output layer. This represents the input vector of the specified neuron. This represents the output vector of the specified neuron.

3. The large language model learning method as described in claim 2, characterized in that, The plurality of neurons are divided into a plurality of model components, wherein determining the importance score of a specific neuron among the plurality of neurons based on the calibration dataset includes: Based on the calibration dataset, determine the importance score of a specified model component among the plurality of model components; The model components with the highest importance scores among the multiple model components are determined as candidate model components; Based on the calibration dataset, determine the importance score of each neuron in each of the candidate model components; The determination of key parameters of the pre-trained large language model based on the analysis results includes: From the neurons of all the candidate model components, multiple important neurons are determined based on the importance scores of each neuron, and the parameters of the multiple important neurons are used as important parameters of the pre-trained large language model.

4. The large language model learning method as described in claim 1, characterized in that, The target projection matrix is ​​obtained through the following steps: Construct a globally summative, non-centralized covariance matrix with all initial elements being zero; In response to completing a learned task, the non-centralized covariance matrix of the input activation feature information corresponding to the training dataset of the learned task is calculated, and the calculated non-centralized covariance matrix is ​​accumulated into the global accumulated non-centralized covariance matrix. In response to the instruction to obtain the target projection matrix, the null projection matrix of the globally accumulated non-centralized covariance matrix is ​​used as the target projection matrix.

5. The large language model learning method as described in claim 4, characterized in that, The step of responding to the instruction to obtain the target projection matrix by using the null projection matrix of the globally accumulated non-centralized covariance matrix as the target projection matrix includes: In response to the instruction to obtain the target projection matrix, the non-centralized covariance matrix of the input activation feature information corresponding to the anchor dataset is accumulated into the global accumulated non-centralized covariance matrix, wherein the anchor dataset contains general domain knowledge data; The null projection matrix of the globally accumulated non-centralized covariance matrix is ​​used as the target projection matrix.

6. The large language model learning method as described in claim 1, characterized in that, The step of correcting the original parameter gradient based on the important parameters and the target projection matrix to obtain the corrected parameter gradient includes: The corrected parameter gradient is obtained by correcting the original parameter gradient using the following formula: ； in, This represents the gradient of the correction parameter. This represents the gradient of the original parameters. It is a binary mask used to represent the important parameters. This represents the target projection matrix.

7. A large language model learning device based on parametric region localization and null projection, characterized in that, include: The acquisition unit is configured to acquire a calibration dataset and a training dataset for the task to be learned, and to acquire a target projection matrix for the learned task, wherein the target projection matrix is ​​determined based on the null projection matrix of the non-centered covariance matrix of the input activation feature information of the learned task. The analysis unit is configured to perform importance analysis on the parameters of the pre-trained large language model based on the calibration dataset, and determine the important parameters of the pre-trained large language model based on the analysis results. The computing unit is configured to compute the gradient of the original parameters of the pre-trained large language model based on the training dataset. The correction unit is configured to correct the original parameter gradient based on the important parameters and the target projection matrix to obtain the corrected parameter gradient; The parameter tuning unit is configured to adjust the parameters of the pre-trained large language model according to the gradient of the correction parameters, so as to obtain the trained large language model.

8. An electronic device, characterized in that, include: At least one processor; At least one memory that stores computer-executable instructions. Wherein, when the computer-executable instructions are executed by the at least one processor, the at least one processor causes the at least one processor to execute the large language model learning method based on parametric region localization and null spatial projection as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor causes the at least one processor to perform the large language model learning method based on parametric region localization and null spatial projection as described in any one of claims 1 to 6.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by at least one processor, the at least one processor causes the processor to perform the large language model learning method based on parametric region localization and null spatial projection as described in any one of claims 1 to 6.

Images