LoRA and continuous learning-based supply chain decision method and system, and medium
By introducing orthogonal decoupling and L2 amplitude suppression terms into the loss function, the parameter updates of newly added LoRA adapters are restricted, thus solving the problem of subspace interference in continuous learning of LoRA and realizing efficient decision-making for rapid adaptation to new tasks in complex supply chain environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-16
- Publication Date
- 2026-04-03
AI Technical Summary
In existing technologies, LoRA is prone to subspace interference during continuous learning, leading to decreased model performance and unstable decision-making, making it difficult to quickly adapt to new tasks and rules in complex supply chain environments.
By introducing an orthogonal decoupling term and an L2 amplitude suppression term into the loss function, the parameter updates of new LoRA adapters are restricted to the orthogonal subspace of existing LoRA adapters. By combining the parameters of the frozen backbone network and historical LoRA adapters, only the new LoRA adapters are updated, thus enabling the reuse of old knowledge and efficient training of new tasks.
It improves learning efficiency, avoids subspace interference, ensures that the model can quickly adapt to new tasks in complex supply chain environments, maintains the stability and accuracy of decision-making, and reduces catastrophic forgetting and small-sample overfitting.
Smart Images
Figure CN121787514A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of supply chain decision technology, specifically relating to a supply chain decision method and system, and medium based on LoRA and continuous learning. Background Technology
[0002] In complex supply chains, judgments in subdomains such as transportation operations, customer orders, inventory replenishment, procurement bidding, compliance quality control, and delivery coordination can often be abstracted into rapid binary judgments of verifiable propositions (e.g., whether there is a delay, whether bidding is required, or whether standards are met), and mapped to business actions (early warning / initiation / release, etc.). Real-world scenarios exhibit characteristics of non-stationary data flows and continuously arriving tasks, accompanied by cross-domain heterogeneity (a mixture of structured indicators, semi-structured text, and rule templates), small sample / cold start (new suppliers, new product categories), and distribution drift (seasons, holidays, policies, weather events). Against this backdrop, general models or routine fine-tuning face challenges in terms of stability-adaptability trade-offs, parameter and computational costs, and evaluation reproducibility: they need to stably retain existing knowledge to avoid catastrophic forgetting while also possessing the ability to quickly adapt to new tasks and rules.
[0003] Faced with the above constraints, existing continuous learning paths include replay / memory (limited by privacy and data non-returnability), regularization to protect old knowledge (easily disconnected from engineering evaluation of long-sequence multi-task scenarios), and parameter isolation / structural expansion (increasing inference overhead and system complexity). Although LoRA (Low Rank Adaptation) has the advantages of low parameter increment and easy deployment, when directly used for continuous learning, it is prone to subspace interference (different tasks crowd each other out in the shared low-rank subspace, causing the decision boundary of old tasks to be disturbed, the performance of new and old tasks to decline alternately, and unstable decisions) and small-sample overfitting. Moreover, it lacks an online selection and solidification mechanism aligned with business evaluation indicators, making it difficult to achieve auditable "immediate solidification of optimality" and cross-stage hot start. Summary of the Invention
[0004] The purpose of this invention is to provide a supply chain decision-making method, system, and medium based on LoRA and continuous learning to solve the technical problem of subspace interference that easily occurs in the prior art.
[0005] To address the aforementioned technical problems, this invention provides a supply chain decision-making method based on LoRA and continuous learning, comprising: a supply chain decision-making method based on LoRA and continuous learning, the method comprising:
[0006] (1) Obtain relevant descriptive information based on the type of supply chain task required for decision-making;
[0007] (2) Input the description information into a pre-trained decision model corresponding to the task type to obtain the decision action;
[0008] The decision model includes a skeleton network for outputting binary decision actions and LoRA adapters corresponding to different task types. The decision model for a certain task type is trained as follows: the parameters of the skeleton network and existing LoRA adapters of the decision model are frozen; the parameters of newly added LoRA adapters corresponding to the task type are updated only based on the training data of the task type to train the decision model; the loss function during training includes an orthogonal decoupling term of the dimensionality reduction matrix used to decouple the newly added LoRA adapters from the existing LoRA adapters.
[0009] (3) Make supply chain decisions based on the aforementioned decision-making actions.
[0010] The beneficial effects of the above technical solution are as follows: When training a decision model for a new task, this invention trains the new task based on existing tasks, improving learning efficiency by reusing the latent knowledge of old tasks. Therefore, an orthogonal decoupling term is introduced into the loss function to decouple the network parameters of the new LoRA adapter from those of the existing LoRA adapter. This orthogonal decoupling term restricts the parameter updates of the new LoRA adapter to the orthogonal subspace of the existing LoRA adapter, avoiding mutual interference between different subspaces and the resulting degradation in model performance. This invention solves the technical problem of subspace interference that easily occurs in existing technologies.
[0011] Furthermore, the loss function also includes an L2 amplitude suppression term to suppress excessive updates of the newly added LoRA adapter parameters.
[0012] Furthermore, the orthogonal decoupling term for:
[0013] ; in, For existing LoRA adapters The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension reduction matrix of the layer; for transpose; It is an L1 norm.
[0014] Furthermore, the L2 amplitude suppression term for:
[0015] ; in, For the newly added LoRA adapter The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension-upgrading matrix of the layer; It is the Frobenius norm.
[0016] Furthermore, the loss function for:
[0017] ; ; in, These are orthogonal decoupling terms; This is the L2 amplitude suppression term; λ1 and λ2 are both tradeoff coefficients greater than 0. It is the negative log-likelihood function of the generative formula; The number of training samples for the current task type; θ represents the model parameters, including frozen parameters and learnable parameters. The model parameters are: When the conditional probability is given input Predicting the first Elements at each position The probability of; For the i-th sample; Let be the label of the i-th sample.
[0018] Furthermore, the final decision model is the decision model that achieves the best evaluation metric on the validation set within a set number of training iterations.
[0019] Furthermore, the initial values of the network parameters of the newly added LoRA adapter are the optimal parameters for training the LoRA corresponding to the previous task type.
[0020] Furthermore, the evaluation metric is accuracy.
[0021] This invention also provides a technical solution for a supply chain decision-making system based on LoRA and continuous learning: a supply chain decision-making system based on LoRA and continuous learning, comprising a processor, the processor being used to execute a computer program to implement the steps of the supply chain decision-making method based on LoRA and continuous learning as described below:
[0022] (1) Obtain relevant descriptive information based on the type of supply chain task required for decision-making;
[0023] (2) Input the description information into a pre-trained decision model corresponding to the task type to obtain the decision action;
[0024] The decision model includes a skeleton network for outputting binary decision actions and LoRA adapters corresponding to different task types. The decision model for a certain task type is trained as follows: the parameters of the skeleton network and existing LoRA adapters of the decision model are frozen; the parameters of newly added LoRA adapters corresponding to the task type are updated only based on the training data of the task type to train the decision model; the loss function during training includes an orthogonal decoupling term of the dimensionality reduction matrix used to decouple the newly added LoRA adapters from the existing LoRA adapters.
[0025] (3) Make supply chain decisions based on the aforementioned decision-making actions.
[0026] The beneficial effects of the above technical solution are as follows: When training a decision model for a new task, this invention trains the new task based on existing tasks, improving learning efficiency by reusing the latent knowledge of old tasks. Therefore, an orthogonal decoupling term is introduced into the loss function to decouple the network parameters of the new LoRA adapter from those of the existing LoRA adapter. This orthogonal decoupling term restricts the parameter updates of the new LoRA adapter to the orthogonal subspace of the existing LoRA adapter, avoiding mutual interference between different subspaces and the resulting degradation in model performance. This invention solves the technical problem of subspace interference that easily occurs in existing technologies.
[0027] Furthermore, the loss function also includes an L2 amplitude suppression term to suppress excessive updates of the newly added LoRA adapter parameters.
[0028] Furthermore, the orthogonal decoupling term for:
[0029] ; in, For existing LoRA adapters The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension reduction matrix of the layer; for transpose; It is an L1 norm.
[0030] Furthermore, the L2 amplitude suppression term for:
[0031] ; in, For the newly added LoRA adapter The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension-upgrading matrix of the layer; It is the Frobenius norm.
[0032] Furthermore, the loss function for:
[0033] ; ; in, These are orthogonal decoupling terms; This is the L2 amplitude suppression term; λ1 and λ2 are both tradeoff coefficients greater than 0. It is the negative log-likelihood function of the generative formula; The number of training samples for the current task type; θ represents the model parameters, including frozen parameters and learnable parameters. The model parameters are: When the conditional probability is given input Predicting the first Elements at each position The probability of; For the i-th sample; Let be the label of the i-th sample.
[0034] Furthermore, the final decision model is the decision model that achieves the best evaluation metric on the validation set within a set number of training iterations.
[0035] Furthermore, the initial values of the network parameters of the newly added LoRA adapter are the optimal parameters for training the LoRA corresponding to the previous task type.
[0036] Furthermore, the evaluation metric is accuracy.
[0037] The present invention also provides a technical solution for a computer-readable storage medium: a computer-readable storage medium internally storing a computer program, the computer program being processed to execute the steps of the supply chain decision-making method based on LoRA and continuous learning as described below:
[0038] (1) Obtain relevant descriptive information based on the type of supply chain task required for decision-making;
[0039] (2) Input the description information into a pre-trained decision model corresponding to the task type to obtain the decision action;
[0040] The decision model includes a skeleton network for outputting binary decision actions and LoRA adapters corresponding to different task types. The decision model for a certain task type is trained as follows: the parameters of the skeleton network and existing LoRA adapters of the decision model are frozen; the parameters of newly added LoRA adapters corresponding to the task type are updated only based on the training data of the task type to train the decision model; the loss function during training includes an orthogonal decoupling term of the dimensionality reduction matrix used to decouple the newly added LoRA adapters from the existing LoRA adapters.
[0041] (3) Make supply chain decisions based on the aforementioned decision-making actions.
[0042] The beneficial effects of the above technical solution are as follows: When training a decision model for a new task, this invention trains the new task based on existing tasks, improving learning efficiency by reusing the latent knowledge of old tasks. Therefore, an orthogonal decoupling term is introduced into the loss function to decouple the network parameters of the new LoRA adapter from those of the existing LoRA adapter. This orthogonal decoupling term restricts the parameter updates of the new LoRA adapter to the orthogonal subspace of the existing LoRA adapter, avoiding mutual interference between different subspaces and the resulting degradation in model performance. This invention solves the technical problem of subspace interference that easily occurs in existing technologies.
[0043] Furthermore, the loss function also includes an L2 amplitude suppression term to suppress excessive updates of the newly added LoRA adapter parameters.
[0044] Furthermore, the orthogonal decoupling term for:
[0045] ; in, For existing LoRA adapters The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension reduction matrix of the layer; for transpose; It is an L1 norm.
[0046] Furthermore, the L2 amplitude suppression term for:
[0047] ; in, For the newly added LoRA adapter The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension-upgrading matrix of the layer; It is the Frobenius norm.
[0048] Furthermore, the loss function for:
[0049] ; ; in, These are orthogonal decoupling terms; This is the L2 amplitude suppression term; λ1 and λ2 are both tradeoff coefficients greater than 0. It is the negative log-likelihood function of the generative formula; The number of training samples for the current task type; θ represents the model parameters, including frozen parameters and learnable parameters. The model parameters are: When the conditional probability is given input Predicting the first Elements at each position The probability of; For the i-th sample; Let be the label of the i-th sample.
[0050] Furthermore, the final decision model is the decision model that achieves the best evaluation metric on the validation set within a set number of training iterations.
[0051] Furthermore, the initial values of the network parameters of the newly added LoRA adapter are the optimal parameters for training the LoRA corresponding to the previous task type.
[0052] Furthermore, the evaluation metric is accuracy. Attached Figure Description
[0053] Figure 1 This is a flowchart illustrating the implementation method of the supply chain decision-making method based on LoRA and continuous learning of the present invention.
[0054] Figure 2 This is a joint workflow diagram of the implementation method of the supply chain decision-making method based on LoRA and continuous learning of the present invention;
[0055] Figure 3 This is a schematic diagram illustrating the components of the supply chain decision-making system based on LoRA and continuous learning according to the present invention. Detailed Implementation
[0056] This invention improves learning efficiency by training a decision model for a new task based on existing tasks and reusing the latent knowledge from older tasks. Therefore, an orthogonal decoupling term is introduced into the loss function to decouple the network parameters of the new LoRA adapter from those of the existing LoRA adapters. This term restricts the parameter updates of the new LoRA adapter to the orthogonal subspace of the existing LoRA adapters, avoiding interference between different subspaces and thus preventing performance degradation. This invention solves the technical problem of subspace interference in existing technologies.
[0057] Implementation of Supply Chain Decision-Making Methods Based on LoRA and Continuous Learning:
[0058] This implementation abstracts the decision-making content of the entire supply chain into different binary judgment tasks (such as whether bidding is needed, whether an early warning is needed, etc.), and maps the judgment results to different business actions (i.e., decision results): for example: bidding is needed → trigger the bidding process; bidding is not needed → release without processing; an early warning is needed → trigger the early warning process; an early warning is not needed → pass. By executing different binary judgment tasks through different types of decision models, the decision-making process of the supply chain is realized.
[0059] Specifically, in this embodiment, the decision model consists of two parts: a backbone network and a LoRA adapter. The backbone network performs binary decision-making based on the input business description information related to different task types, using its basic classification / decision head to output the corresponding decision result. The "backbone network" refers to the pre-trained model backbone and its shared parameters, which maintain generality and stability except for the LoRA adapters for each task. The second part is the LoRA adapter, where different LoRA adapters correspond to different task types. The LoRA adapter is connected to the backbone network in a bypass manner, enabling the backbone network to make correct decisions by incorporating the knowledge learned within the LoRA adapter. How the LoRA adapter specifically connects to the backbone network can be referred to in existing LoRA fine-tuning techniques, which will not be described in detail in this embodiment. The LoRA adapter is a low-rank incremental parameter module added to the weight matrix corresponding to the "backbone network," used for lightweight adaptation to different tasks without changing the shared weights of the backbone network.
[0060] Specifically, the LoRA adapter is trained in the following manner to obtain a decision model corresponding to a specific task type, such as... Figure 1 and Figure 2 As shown:
[0061] Initialization: When the system starts, the pre-trained backbone network is loaded, the LoRA adapter manager is set up, and the basic structure for real-time incremental learning is prepared.
[0062] Define the supply chain decision-making process as a multi-task sequence This task sequence covers various aspects of the supply chain decision-making process, such as orders, transportation, warehousing, compliance, and suppliers. The t-th task T... t The input to the corresponding decision model is a scenario description x based on supply chain business entities and rules related to the task. t (Composed of structured or unstructured features of different elements), the output is a binary decision result. The judgment results are then mapped to different decision actions.
[0063] For example: T t Let x represent the bidding decision. t This section describes the supply chain business entities and rules that influence the bidding results. This indicates that a bidding process is needed, and the corresponding decision-making action is to trigger the bidding process. This indicates that no bidding is required, and the corresponding decision-making action is to release the application without any further processing.
[0064] Train each of the above tasks sequentially, for each task T. t Training requires a training / development dataset for the task. x i For the sample, y i For x i Tags; also includes historical tasks (i.e., T1~T) t－1 Knowledge storage S that has been learned t－1 This definition is used to drive subsequent sequential learning and evaluation.
[0065] S01, Fixed Feature Strategy: When the new task T t Upon arrival, instantiate the LoRA adapter corresponding to the new task. (i.e., adding a new LoRA adapter), which includes all parameters of the skeleton network and the corresponding LoRA adapters for historical tasks (i.e., Corresponding to historical potential knowledge storage S t－1 The parameters of A are frozen only. t The parameters are optimized using gradient descent.
[0066] Input data: Current task training / development set ,in For samples, For metrics, business rules and evaluation metric configuration, historical potential knowledge storage With the already solidified set of optimal LoRA adapters for the current stage.
[0067] Model representation: Pre-trained LoRA adapter parameters (historical) With the newly introduced ΔW in this stage t ).
[0068] The fixed-feature strategy, in addition to preserving the intermediate representations before the output layer, also solidifies the LoRA low-rank subspace summary of the previously learned adapters into latent knowledge for each learning task. This subspace is used to characterize the update direction of the old tasks. It can be approximated by the row space of the historical adapter matrix, characterizing the gradient / update subspace of past tasks, and thus serves as a reference basis for interference suppression in subsequent learning. The fixed-feature strategy preserves knowledge from historical learning tasks, enabling the model to adapt quickly to new tasks while retaining performance on historical tasks.
[0069] S02, Historical Knowledge Reuse Strategy: Utilize the backbone network and the LoR adapters corresponding to historical tasks, and freeze the parameters of the backbone network and historical LoR adapters (i.e., the weight increment ΔW corresponding to historical tasks). t－1 Only the parameters of the newly added LoRA adapter are updated using gradient descent (only for new task T). t The corresponding weight increment ΔW t ).
[0070] By using historical LoRA adapters without updating them, we can reuse the latent knowledge of old tasks and stabilize old representations without replaying historical data, while improving the training efficiency and convergence stability of new tasks. At the same time, we only need to update the lightweight parameters (the newly introduced LoRA adapters), thus improving learning efficiency.
[0071] By introducing a fixed feature and historical knowledge reuse strategy, the backbone network and the feature extraction layers and adapter parameters of historical tasks are frozen, and only the adapter parameters of the current task are adjusted. This approach retains the learned representations while introducing new knowledge, thus achieving a balance between old and new knowledge.
[0072] Throughout the continuous learning process, there is no need to store or replay data samples from historical tasks. Instead, old knowledge is maintained and new knowledge is injected by fixing the historical adapter and orthogonal constraints. This improves learning efficiency by reusing the potential knowledge from old tasks while training new tasks.
[0073] In Task T t During the training period, the generative negative log-likelihood loss is used as the main loss, and a structured regularization term (orthogonal decoupling and amplitude suppression of LoRA subspace) is superimposed according to the weights to minimize the overall objective function and gradually optimize the model parameters.
[0074] Specifically, the loss function during training consists of three parts: the main training loss, an orthogonal decoupling term, and an L2 amplitude suppression term. Structured regularization constraints are applied to the old and new adapters within the LoRA parameter subspace. These constraints include at least orthogonal decoupling to reduce parameter interference and amplitude suppression to limit the update scale. The learned knowledge represented by the LoRA adapters of historical tasks is structurally and regularly fused with the feature representations of the current task. By introducing regularization terms aligned with / distilled from historical knowledge into the adapter parameter update objective, the current task learns new knowledge while reusing latent knowledge from old tasks, thereby improving training efficiency and suppressing catastrophic forgetting.
[0075] The main training loss employs generative negative log-likelihood (NLL) based on teacher forcing, and unified inference hyperparameters (including decoding length, etc.) are used to ensure comparability across stages. The generative negative log-likelihood function is:
[0076] ; in, It is the negative log-likelihood function of the generative formula; θ represents the number of training samples for the current task type; θ represents the model parameters, including the trainable parameters of the LoRA adapter for the current task, while the parameters of the backbone network and historical tasks are frozen. For the i-th sample; Let be the label of the i-th sample; The model parameters are: When the conditional probability is given input (For example, "order information, historical logistics status" in the supply chain) predict "the first Elements at each position The probability of "".
[0077] Orthogonal decoupling term: Orthogonal constraint on the LoRA subspaces of different tasks; makes the historical tasks... The subspace is composed of Row vector spanning, current task The subspace is composed of The span of row vectors satisfies This reduces the correlation and interference between cross-task updates.
[0078] Among them, the orthogonal decoupling term for:
[0079] ; in, For existing LoRA adapters The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension reduction matrix of the layer; for transpose; It is an L1 norm.
[0080] LoRA adapters typically have a built-in dimension-up matrix B and a dimension-down matrix A. The weight increments generated by the LoRA adapter satisfy... .
[0081] By orthogonally decoupling, the LoRA update constraints of the new task are performed in the orthogonal subspace of the historical subspace, so as to reduce mutual interference with the loss surface of the old task.
[0082] L2 amplitude suppression term: Apply L2 amplitude suppression to the current adapter to alleviate overfitting and excessive updates. The L2 amplitude suppression term is:
[0083] ; in, For the newly added LoRA adapter The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension-upgrading matrix of the layer; It is the Frobenius norm.
[0084] In summary, the final loss function of this implementation method for:
[0085] ; in, These are orthogonal decoupling terms; This is the L2 amplitude suppression term; λ1 and λ2 are both tradeoff coefficients greater than 0; they can be set according to the supply chain task type and business evaluation indicators.
[0086] S03. Adaptive learning strategy: When the development / validation set performance index strictly exceeds the historical best of the current stage in any training step, the current adapter is immediately fixed as the "stage best", and a warm start initialization is performed in the next stage with the "stage best" to reduce the number of effective training steps and reduce cross-stage fluctuations.
[0087] The strategy rule for selecting the optimal LoRA adapter in a given phase is as follows:
[0088]
[0089] In the formula, For the t-th task T t Parameters of the LoRA adapter network after training in step s; For T t The validation set; Represents the verification set exist The corresponding performance metrics of the decision-making model This represents the optimal LoRA adapter parameters for the t-th task, as determined in the final calculation. For the (t+1)th task T t+1 The initial value of the corresponding LoRA adapter during a warm start.
[0090] Based on an adaptive learning strategy and combined with business evaluation metrics, adapter parameters are selected and optimized to improve training efficiency and ensure the reliability of output decisions.
[0091] That is, using the training set Training the LoRA adapter During training, the validation set is evaluated at a predetermined frequency (e.g., at the end of each round or every certain number of training steps) over several rounds. The current network parameters are validated to obtain the corresponding performance metrics, and the "stage-optimal" network parameters are updated and saved as the validation metrics improve. After training, the network parameters corresponding to the best performance metrics on the validation set are used as the final network parameters determined for this stage (i.e., "stage-optimal"), and the LoRA adapter parameters of this "stage-optimal" are saved as an adapter model, which is used as the initial parameters for loading the corresponding LoRA adapter when training the next task.
[0092] Specifically, in this embodiment, accuracy is used as the performance metric. In other embodiments, other performance metrics may also be selected.
[0093] Once all tasks have been learned, the system obtains a continuous learning model adapted to multiple tasks in the supply chain. This model outputs a unified True / False caliber and maps it to business actions such as bidding, early warning, and release by subdomain. This model can execute in real-time supply chain scenarios, improving learning speed and performance, while also enabling rapid adaptation to new tasks in changing supply chain data environments, thereby enhancing its resistance to forgetting and its adaptability.
[0094] As new tasks or rules are introduced, the above training steps are repeated to continuously update the model parameters and the optimal adapter set, maintaining a stable, malleable equilibrium and engineering feasibility in scenarios where supply chain tasks arrive in sequence.
[0095] Implementation of a supply chain decision-making system based on LoRA and continuous learning:
[0096] A supply chain decision-making system based on LoRA and continuous learning includes a processor for executing a computer program to implement the steps of the LoRA and continuous learning-based supply chain decision-making method described below. The specific LoRA and continuous learning-based supply chain decision-making method has been described in sufficient detail in the above-described implementations and will not be repeated here.
[0097] This implementation method can adopt, for example... Figure 3 The above-described supply chain decision-making method based on LoRA and continuous learning includes: a fixed feature strategy module, a historical knowledge reuse strategy module, an adaptive learning strategy module, a model training module, and a model output module.
[0098] Fixed Feature Strategy Module: When a new task arrives, the parameters of the pre-trained skeleton network and the parameters of the feature extraction layer and LoRA adapter corresponding to the historical task are set to a non-updated state. Only the LoRA adapter added for the current task is allowed to participate in gradient update, so as to maintain the learned ability and skeleton stability while taking into account the plasticity of the new task.
[0099] The historical knowledge reuse strategy module is configured to optimize only the current task LoRA adapter during the training process of a new task, and apply structured regularization (including orthogonal decoupling and amplitude suppression) to the current and historical adapters in the LoRA subspace. It then performs regularized fusion of the learned knowledge represented by the historical adapters with the features of the current task, and uses this fusion as a constraint for updating the parameters of the current task adapter, so as to improve learning efficiency and suppress interference.
[0100] The adaptive learning strategy module is configured to immediately solidify the current adapter as the "stage best" when the performance metrics of the development / validation set strictly exceed the historical best of the current stage, and perform a warm-start initialization with the "stage best" in the next stage to reduce the number of effective training steps and reduce cross-stage fluctuations.
[0101] Model training module: It is configured to perform incremental continuous training and switching on multiple tasks based on the above fixed feature strategy module, historical knowledge reuse strategy module and adaptive learning and evaluation module.
[0102] Model Output Module: Configured to output the supply chain decision model trained with multi-task adaptation, and support incremental continuous learning and inference decisions for new tasks in supply chain scenarios. The model output module includes a rule triggering submodule, which is used to trigger corresponding actions based on preset rule templates for different business subdomains; for example, in the inventory management subdomain, when the rule "the actual inventory level of materials is lower than the preset safety stock threshold" is found to be true, True is output and replenishment application and procurement process are triggered; otherwise, False is output to maintain the existing inventory strategy without initiating additional procurement processes.
[0103] For details on each of the above modules, please refer to the implementation method of the supply chain decision-making method based on LoRA and continuous learning.
[0104] Specifically, the processor can be a CPU, or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor can also be a processor supporting the Advanced Reduced Instruction Set Machine (ARM) architecture.
[0105] Implementation of computer-readable storage media:
[0106] A computer-readable storage medium storing a computer program internally, the computer program being processed to perform the steps of the LoRA-based and continuous learning-based supply chain decision-making method described below. The specific LoRA-based and continuous learning-based supply chain decision-making method has been described in sufficient detail in the above-described embodiments and will not be repeated here. The method includes:
[0107] (1) Obtain the corresponding descriptive information for the supply chain task type required for decision-making.
[0108] (2) Input the description information into a pre-trained decision model corresponding to the task type to obtain the decision action.
[0109] The decision model includes a skeleton network for outputting binary decision actions and LoRA adapters corresponding to different task types. The decision model for a certain task type is trained as follows: the parameters of the skeleton network and existing LoRA adapters of the decision model are frozen; the parameters of newly added LoRA adapters corresponding to the task type are updated only based on the training data of the task type to train the decision model; the loss function during training includes an orthogonal decoupling term of the dimensionality reduction matrix used to decouple the newly added LoRA adapters from the existing LoRA adapters.
[0110] Furthermore, the loss function also includes an L2 amplitude suppression term to suppress excessive updates of the newly added LoRA adapter parameters.
[0111] Furthermore, the orthogonal decoupling term for:
[0112] ; in, For existing LoRA adapters The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension reduction matrix of the layer; for transpose; It is an L1 norm.
[0113] Furthermore, the L2 amplitude suppression term for:
[0114] ; in, For the newly added LoRA adapter The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension-upgrading matrix of the layer; It is the Frobenius norm.
[0115] Furthermore, the loss function for:
[0116] ; ; in, These are orthogonal decoupling terms; This is the L2 amplitude suppression term; λ1 and λ2 are both tradeoff coefficients greater than 0. It is the negative log-likelihood function of the generative formula; The number of training samples for the current task type; θ represents the model parameters, including frozen parameters and learnable parameters. The model parameters are: When the conditional probability is given input Predicting the first Elements at each position The probability of; For the i-th sample; Let be the label of the i-th sample.
[0117] Furthermore, the final decision model is the decision model that achieves the best evaluation metric on the validation set within a set number of training iterations.
[0118] Furthermore, the initial values of the network parameters of the newly added LoRA adapter are the optimal parameters for training the LoRA corresponding to the previous task type.
[0119] Furthermore, the evaluation metric is accuracy.
[0120] (3) Make supply chain decisions based on the aforementioned decision-making actions.
[0121] Specifically, the computer-readable storage medium can be volatile memory or non-volatile memory, or may include both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which serves as an external cache. For example, Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), SynchLink DRAM (SLDRAM), or Direct Rambus RAM (DRRAM).
[0122] The present invention has the following beneficial effects and advantages:
[0123] 1. This invention provides a supply chain decision-making method based on LoRA and continuous learning. Its fixed feature strategy freezes the skeleton and learned adapters in each task learning round and retains the potential knowledge of old tasks, so that the model can still robustly call existing representations when learning new tasks, thereby significantly reducing catastrophic forgetting and cross-stage performance degradation.
[0124] 2. Its historical knowledge reuse strategy optimizes only the LoRA adapter for the current task during the training of new tasks, and applies structured regularization (including orthogonal decoupling and amplitude suppression) to the current and historical adapters in the LoRA subspace. It only updates lightweight parameters (such as LoRA adapters), avoids repeated training of feature extractors, reduces computation and memory usage, shortens training time and improves online / near real-time adaptation efficiency.
[0125] 3. Its adaptive learning strategy uses the development set index to drive the selection and solidification of the optimal adapter for each stage. In the next stage, it performs a hot start initialization with the "stage optimal" adapter, achieving stable convergence and smaller cross-stage fluctuations without replaying historical data, thus balancing stability and adaptability.
[0126] 4. By comprehensively applying fixed feature strategies, historical knowledge reuse strategies, and adaptive learning strategies, this invention achieves a stable-plastic equilibrium in continuous supply chain task arrival scenarios: compared to existing technologies, it effectively reduces the forgetting of old knowledge and suppresses overfitting to new tasks with small samples. The model can quickly adapt to new tasks in changing supply chain data environments, thereby improving its resistance to forgetting and adaptability.
[0127] 5. This invention is engineering-feasible and scalable: It adopts a unified True / False generative decision and action mapping, which is adapted to subdomains such as transportation, orders, inventory, procurement, compliance and delivery in the supply chain; new tasks and rules can be incrementally expanded without saving historical samples, thus meeting privacy and compliance requirements.
[0128] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still make modifications to the technical solutions described in the foregoing embodiments without creative effort, or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A supply chain decision-making method based on LoRA and continuous learning, characterized in that, The method includes: (1) Obtain relevant descriptive information based on the type of supply chain task required for decision-making; (2) Input the description information into a pre-trained decision model corresponding to the task type to obtain the decision action; The decision model includes a skeleton network for outputting binary decision actions and LoRA adapters corresponding to different task types. The decision model for a certain task type is trained as follows: the parameters of the skeleton network and existing LoRA adapters of the decision model are frozen; the parameters of newly added LoRA adapters corresponding to the task type are updated only based on the training data of the task type to train the decision model; the loss function during training includes an orthogonal decoupling term of the dimensionality reduction matrix used to decouple the newly added LoRA adapters from the existing LoRA adapters. (3) Make supply chain decisions based on the aforementioned decision-making actions.
2. The supply chain decision-making method based on LoRA and continuous learning according to claim 1, characterized in that, The loss function also includes an L2 amplitude suppression term to suppress excessive updates of the newly added LoRA adapter parameters.
3. The supply chain decision-making method based on LoRA and continuous learning according to claim 1 or 2, characterized in that, The orthogonal decoupling term for: ； in, For existing LoRA adapters The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension reduction matrix of the layer; for Transpose of; It is an L1 norm.
4. The supply chain decision-making method based on LoRA and continuous learning according to claim 2, characterized in that, The L2 amplitude suppression term for: ； in, For the newly added LoRA adapter The dimension reduction matrix of the layer; For the newly added LoRA adapter The dimension-upgrading matrix of the layer; It is the Frobenius norm.
5. The supply chain decision-making method based on LoRA and continuous learning according to claim 2, characterized in that, The loss function for: ； ； in, These are orthogonal decoupling terms; This is the L2 amplitude suppression term; λ1 and λ2 are both tradeoff coefficients greater than 0. It is the negative log-likelihood function of the generative formula; The number of training samples for the current task type; θ represents the model parameters, including frozen parameters and learnable parameters. The model parameters are: When the conditional probability is given input Predicting the first Elements at each position The probability of; For the i-th sample; Let be the label of the i-th sample.
6. The supply chain decision-making method based on LoRA and continuous learning according to claim 1, 2, or 5, characterized in that, The final decision model is the one that achieves the best evaluation metric on the validation set within a set number of training iterations.
7. The supply chain decision-making method based on LoRA and continuous learning according to claim 1, characterized in that, The initial values of the network parameters of the newly added LoRA adapter are the optimal parameters for training the LoRA corresponding to the previous task type.
8. The supply chain decision-making method based on LoRA and continuous learning according to claim 6, characterized in that, The evaluation metric is accuracy.
9. A supply chain decision-making system based on LoRA and continuous learning, comprising a processor, characterized in that, The processor is used to execute a computer program to implement the steps of the supply chain decision-making method based on LoRA and continuous learning as described in any one of claims 1 to 8.
10. A computer-readable storage medium, wherein a computer program is stored internally, characterized in that, The computer program is used to be processed to perform the steps of the supply chain decision-making method based on LoRA and continuous learning as described in any one of claims 1 to 8.