Special large language model construction method based on efficient fine tuning of parameters

Abstract

The invention discloses a special large language model construction method based on efficient parameter fine tuning. The method comprises the following steps: collecting a scheduling specification text, analyzing conditions, actions and constraint elements, and constructing a scheduling procedure topological structure; traversing the procedure topological structure, generating a procedure execution path, and extracting a topological path identifier and hierarchical information; constructing a topological regulation and control memory, and establishing a parameter memory slot position relationship; the scheduling task instruction is analyzed, a semantic energy value is generated, and the activation and updating amplitude of the corresponding parameter slot position is regulated and controlled; freezing main model parameters, only updating an activated slot position, executing parameter fine tuning, and constructing a special scheduling language model; and accessing a new procedure text, executing hierarchical distillation and playback operation, and updating the model to keep the scheduling logic consistency. According to the method, long-term consistency, controllable updating and stable reasoning of the proprietary large language model in a power dispatching normative business scene are realized by introducing a scheduling procedure topological structure perceived parameter efficient fine tuning mechanism.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, and in particular to a proprietary method for constructing large language models based on efficient parameter fine-tuning. Background Technology

[0002] With the development of artificial intelligence technology, deep learning-based large language models have been widely used in text understanding, knowledge-based question answering, and decision support. In highly standardized business scenarios such as power dispatching, dispatching standards, procedures, and emergency plans constitute important bases for dispatching decisions. These texts are characterized by clear hierarchical clauses, strict logical constraints, and explicit causal relationships. Current technologies typically employ a general-purpose large language model combined with industry-specific text for domain-adaptive training to enhance the model's ability to understand specialized semantics. Common methods include full-parameter fine-tuning or efficient parameter fine-tuning, enabling the model to maintain general language capabilities while possessing a certain level of industry-specific knowledge expression ability.

[0003] However, most existing industry-specific large language models based on efficient parameter fine-tuning treat scheduling specification texts as ordinary linear corpora, lacking the ability to model the hierarchical relationships and causal structures among the internal conditions, actions, and constraints of scheduling procedures. During training, these models typically employ static parameter mapping, directly writing new industry knowledge into a fixed parameter subspace, failing to distinguish the structural differences between execution paths of different procedures. When scheduling procedures are updated or different scheduling tasks frequently switch, model parameter updates can easily interfere with each other, leading to problems such as forgetting professional knowledge, inconsistent inference results, and confusion between old and new procedures, making it difficult to meet the consistency and reliability requirements of power dispatching operations.

[0004] Existing technologies rely heavily on overall retraining or simple sample replay during model iteration and continuous learning, lacking a mechanism for selective parameter replay and alignment based on scheduling rules. This not only increases the cost of model training but also makes it difficult to effectively control the parameter evolution process, easily leading to uncontrollable changes in the model's inference behavior.

[0005] Therefore, how to provide a proprietary large language model construction method based on efficient parameter fine-tuning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a proprietary large language model construction method based on efficient parameter fine-tuning. This invention combines scheduling procedure topology modeling, parameter memory slot management, and semantic energy flow regulation with hierarchical distillation to perform low-cost, controllable domain-adaptive training on a general-purpose large language model. This enables structured modeling and continuous learning of complex hierarchical relationships and causal constraints in power dispatching regulations. This method effectively avoids the problems of professional knowledge forgetting and inconsistencies in reasoning during scheduling procedure updates and task switching. It possesses advantages such as traceable parameter updates, controllable model evolution, and high long-term operational stability, making it suitable for standardized business scenarios such as power dispatching where consistency and reliability are paramount.

[0007] A proprietary large language model construction method based on efficient parameter fine-tuning according to an embodiment of the present invention includes: Collect dispatching specification texts related to power dispatching operations, process the dispatching specification texts, and construct a dispatching procedure topology structure consisting of condition nodes, action nodes, and constraint nodes; Based on the scheduling procedure topology, a topology path identifier corresponding to each procedure execution path is generated, and a corresponding topology path identifier and hierarchical order information are generated for each procedure execution path. Construct a topology control memory, allocate a corresponding parameter memory slot to each topology path identifier, and establish a binding relationship between the parameter increments generated by the general large language model during the efficient fine-tuning of parameters and the corresponding parameter memory slots. A semantic energy flow regulator is introduced to parse the input scheduling task instructions, generate semantic energy values ​​and allocate them to the parameter memory slots corresponding to the topology path identifiers, and control the activation state of the parameter memory slots and the parameter update magnitude. While keeping the main parameters of the general large language model frozen, based on the parameter memory slots, activation states and parameter update magnitudes, efficient parameter fine-tuning training is performed only on the parameter increments corresponding to the activated parameter memory slots to obtain a scheduling knowledge-specific large language model. When iteratively updating the scheduling knowledge-specific large language model or integrating new scheduling specification texts, hierarchical distillation and topology coupling replay operations are performed based on the scheduling procedure topology structure, topology path identifiers, and parameter memory slots. Selective replay and alignment updates are performed on historical parameter memory slots to obtain the updated scheduling knowledge-specific large language model.

[0008] Optionally, the dispatching specification text includes power dispatching standard documents, power dispatching procedure documents, and power dispatching emergency plan documents.

[0009] Optionally, the processing of the scheduling specification text to construct a scheduling procedure topology structure consisting of condition nodes, action nodes, and constraint nodes includes: The scheduling specification text is parsed hierarchically to identify the chapter structure, clause numbers, and clause hierarchical relationships, generating the hierarchical structure information of the scheduling specification text. Based on the hierarchical structure information, condition elements representing the scheduling triggering conditions are extracted from the content of each clause, and the condition elements are mapped to condition nodes. Extract action elements representing scheduling operation behaviors from the content of each clause, and map the action elements as action nodes; Extract constraint elements representing scheduling restriction rules, execution order, or prohibition relationships from each clause, and map these constraint elements as constraint nodes; Based on the hierarchical relationship between clauses and the logical relationship between condition elements, action elements and constraint elements, a directed connection relationship is constructed between condition nodes, action nodes and constraint nodes to form a scheduling procedure topology.

[0010] Optionally, generating a corresponding topology path identifier and hierarchical order information for each procedure execution path includes: Based on the scheduling procedure topology, three types of nodes are identified as condition nodes, action nodes, and constraint nodes. Directed relationships between nodes are established to represent the procedure execution relationship from condition to action that is constrained. The directed relation is traversed in an ordered manner from the condition node to the action node. The execution paths that satisfy the condition from the start to the target action and are restricted by the corresponding constraint nodes are enumerated. The node order, node type sequence and relation type sequence of each execution path are recorded. For each procedure execution path, a topology path identifier corresponding to the procedure execution path is generated. The topology path identifier is obtained by combining the node identifier sequence and relation type sequence of the procedure execution path according to fixed rules and then encoding it with fixed length. Based on the chapters, clauses, and sub-clauses in the scheduling specification text, the level to which each node belongs in the procedure execution path is determined, forming hierarchical sequence information consistent with the order of the path nodes, and synchronously recording the sequential constraints determined by the hierarchical relationship; Establish a bidirectional index between topology path identifiers and procedure execution paths, complete the deduplication, uniqueness verification and number management of all procedure execution paths, and output the set of topology path identifiers and the corresponding hierarchical order information.

[0011] Optionally, the step of establishing a binding relationship between the parameter increments generated during the efficient fine-tuning of the general large language model and the corresponding parameter memory slots includes: Based on the topology path identifier set and hierarchical order information, the data structure of the topology control memory is initialized. The topology control memory includes a parameter memory slot area, a slot index area, a topology adjacency area, and a version evolution area. In the parameter memory slot area, a unique parameter memory slot record is created for each topology path identifier. The parameter memory slot record includes the slot identifier, the bound topology path identifier, the hierarchical order information reference, the slot status identifier, the slot version identifier, and the parameter incremental storage area, which is divided into a write area and a freeze area. In the slot index area, a forward index table from topology path identifier to parameter memory slot and a reverse index table from parameter memory slot to topology path identifier are constructed. Uniqueness checks are performed on all indexes to ensure that each topology path identifier corresponds to only one parameter memory slot and each parameter memory slot corresponds to only one topology path identifier. In the topological adjacency area, based on the scheduling procedure topology structure, the adjacency relationship between each topological path identifier is determined, a topological path adjacency table is generated and written into the adjacency reference field associated with the corresponding parameter memory slot, so that a searchable adjacency reference chain is established between the parameter memory slots corresponding to adjacent topological path identifiers. In the version evolution area, a version chain record is established for each parameter memory slot. When writing parameter increments, the parameter increment in the writing area is first solidified into a new version and written into the version chain record. Then, the parameter increment of the previous version is transferred to the frozen area and the status flag of the corresponding slot is updated to a replayable state, thus completing the construction of the topology control memory.

[0012] Optionally, the step of generating semantic energy values ​​and allocating them to parameter memory slots corresponding to topological path identifiers, and controlling the activation state and parameter update magnitude of the parameter memory slots, includes: A semantic power flow controller is constructed, which consists of an instruction element extraction unit, a topology power flow allocation unit, and a slot control unit. The semantic power flow controller is associated with the topology path identifier set and the parameter memory slot. The system receives scheduling task instructions, uses the instruction element extraction unit to structurally decompose the scheduling task instructions, and obtains task category elements, urgency elements and instruction keyword sets. Based on the topology path identifier set, the system performs matching and retrieval on the instruction keyword set to obtain the target topology path identifier set. The instruction element extraction unit generates semantic energy values ​​based on task category elements, urgency elements, and the size of the instruction keyword set. The semantic energy values ​​are then limited to the total energy constraint range of the semantic energy flow controller and used as the input for this energy flow allocation. The semantic energy value is allocated to the parameter memory slots corresponding to the target topology path identifier set using the topology energy flow allocation unit. The allocation includes determining the matching degree of each target topology path identifier based on the matching degree between the instruction keyword set and the procedure execution path keyword set associated with the target topology path identifier, allocating the semantic energy value as slot allocation energy of each parameter memory slot according to the matching degree, and transferring the slot allocation energy to adjacent parameter memory slots in a decreasing topology distance manner based on the adjacency reference chain, forming an adjacency energy flow between adjacent parameter memory slots. The slot control unit generates slot control parameters for each parameter memory slot based on the slot energy allocation and adjacent energy flow. The slot control parameters include the slot activation state and parameter update magnitude.

[0013] Optionally, the obtained scheduling knowledge-specific large language model includes: Obtain the main parameters of the general large language model and set the main parameters to a frozen state. Read the parameter increments stored in the parameter memory slots, read the slot activation state and parameter update magnitude, and form the trainable parameter range for this efficient fine-tuning training of parameters. A training sample set is constructed based on the scheduling specification text, and a target topology path identifier is associated with each training sample. The correspondence between the topology path identifier and the procedure execution path is used to map the target topology path identifier to the target parameter memory slot, so that the training sample and the parameter memory slot are associated. For each training sample, the parameter increment corresponding to the target parameter memory slot is called, and the combined parameters are generated according to the slot activation state and parameter update magnitude. The combined parameters are formed by superimposing the frozen main parameters and the activated parameter increments, and the parameter increments corresponding to the inactive parameter memory slots do not participate in the generation of combined parameters. The model output is obtained by performing forward computation on the training samples based on the combined parameters, and the training loss is calculated based on the target output of the training samples and the model output. Perform backpropagation update on the training loss, update only the parameter increments corresponding to the activated parameter memory slots and keep the main parameters of the general large language model unchanged. Write the updated parameter increments back to the corresponding parameter memory slots and update the slot version identifier to obtain the scheduling knowledge-specific large language model.

[0014] Optionally, the obtained updated scheduling knowledge-specific large language model includes: When iteratively updating the proprietary language model of scheduling knowledge or integrating new scheduling specification texts, the newly added or changed scheduling specification texts are parsed hierarchically and the scheduling condition elements, scheduling action elements and constraint elements are extracted. The scheduling procedure topology is incrementally updated and the procedure execution path and topology path identifier corresponding to the newly added or changed content are generated. Based on the topology path identifier, the target parameter memory slot corresponding to the topology path identifier is retrieved from the topology control memory, and the historical version record of the target parameter memory slot is retrieved to form a set of historical parameter memory slots. Construct the set of models required for hierarchical distillation, set the current scheduling knowledge-specific large language model to be updated as the student model, set the scheduling knowledge-specific large language model of the previous iteration as the upper-level teacher model, and set the general large language model as the basic teacher model. Performing a topology coupling replay operation, specifically: Based on the scheduling procedure topology, the topological adjacency relationship and topological distance between the topological path identifier and each topological path identifier in the historical parameter memory slot set are determined. A replay priority sequence is generated based on topological distance. Based on the replay priority sequence, parameter increments and associated samples are read from the historical parameter memory slot set to form a replay sample set. Performing a tiered distillation operation, specifically: Input the replay sample set and the sample set corresponding to the newly added or changed content into the basic teacher model, the upper-level teacher model and the student model respectively to obtain the corresponding output; Using the teacher model output as the alignment target for the student model output, the sample-level distillation loss is calculated, and the distillation loss is updated with weights based on the replay priority sequence. While keeping the main parameters of the general large language model frozen, only the parameter increments corresponding to the target parameter memory slot and the historical parameter memory slot set are updated, and the updated parameter increments are written back to the topology control memory to obtain the updated scheduling knowledge-specific large language model.

[0015] The beneficial effects of this invention are: This invention introduces scheduling procedure topology modeling and parameter memory management mechanisms within a parameter-efficient fine-tuning framework. This enables the proprietary large language model to explicitly perceive the hierarchical relationships and causal structures among conditions, actions, and constraints in the scheduling specification text during training and updating, thereby avoiding the structural information loss problem caused by treating scheduling procedures as linear corpora in existing technologies. By binding different procedure execution paths to corresponding parameter memory slots, this invention achieves a one-to-one correspondence between parameter updates and procedure structures. This allows the model to perform reasoning based on explicit procedure paths when facing complex scheduling tasks, improving the professional consistency and logical reliability of the model's output results.

[0016] This invention introduces a semantic energy flow regulation mechanism to dynamically adjust parameter update behavior under different scheduling task scenarios. This enables the model to prioritize the activation of relevant parameter memory slots in high-urgency or critical scheduling tasks, while suppressing unnecessary parameter disturbances in regular queries or low-risk tasks. This effectively avoids parameter drift caused by frequent updates during efficient parameter fine-tuning, improves the stability and controllability of the model in multi-task switching scenarios, and reduces the cost of model maintenance and continuous training.

[0017] This invention introduces a hierarchical distillation and topology-coupled replay mechanism during the model iteration phase. This allows the model to selectively replay and align historical parameter memories based on the scheduling rule topology when absorbing new scheduling rules, thereby maintaining rapid adaptation to new rules while effectively preventing the forgetting of existing scheduling knowledge. Through this technical solution, this invention achieves knowledge continuity and reasoning consistency in the long-term operation of a proprietary large language model, demonstrating high engineering practical value and promising prospects for widespread application. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a proprietary large language model construction method based on efficient parameter fine-tuning proposed in this invention; Figure 2 This is a schematic diagram of the semantic energy flow regulator of a proprietary large language model construction method based on efficient parameter fine-tuning proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figure 1 and Figure 2 A proprietary large language model construction method based on efficient parameter fine-tuning includes: Collect dispatching specification texts related to power dispatching operations, process the dispatching specification texts, and construct a dispatching procedure topology structure consisting of condition nodes, action nodes, and constraint nodes; Based on the scheduling procedure topology, a topology path identifier corresponding to each procedure execution path is generated, and a corresponding topology path identifier and hierarchical order information are generated for each procedure execution path. Construct a topology control memory, allocate a corresponding parameter memory slot to each topology path identifier, and establish a binding relationship between the parameter increments generated by the general large language model during the efficient fine-tuning of parameters and the corresponding parameter memory slots. A semantic energy flow regulator is introduced to parse the input scheduling task instructions, generate semantic energy values ​​and allocate them to the parameter memory slots corresponding to the topology path identifiers, and control the activation state of the parameter memory slots and the parameter update magnitude. While keeping the main parameters of the general large language model frozen, based on the parameter memory slots, activation states and parameter update magnitudes, efficient parameter fine-tuning training is performed only on the parameter increments corresponding to the activated parameter memory slots to obtain a scheduling knowledge-specific large language model. When iteratively updating the scheduling knowledge-specific large language model or integrating new scheduling specification texts, hierarchical distillation and topology coupling replay operations are performed based on the scheduling procedure topology structure, topology path identifiers, and parameter memory slots. Selective replay and alignment updates are performed on historical parameter memory slots to obtain the updated scheduling knowledge-specific large language model.

[0021] In this embodiment, the dispatching specification text includes power dispatching standard documents, power dispatching procedure documents, and power dispatching emergency plan documents.

[0022] In this embodiment, the process of processing the scheduling specification text and constructing a scheduling procedure topology structure composed of condition nodes, action nodes, and constraint nodes includes: The scheduling specification text is parsed hierarchically to identify the chapter structure, clause numbers, and clause hierarchical relationships, generating the hierarchical structure information of the scheduling specification text. Based on the hierarchical structure information, condition elements representing the scheduling triggering conditions are extracted from the content of each clause, and the condition elements are mapped to condition nodes. Extract action elements representing scheduling operation behaviors from the content of each clause, and map the action elements as action nodes; Extract constraint elements representing scheduling restriction rules, execution order, or prohibition relationships from each clause, and map these constraint elements as constraint nodes; Based on the hierarchical relationship between clauses and the logical relationship between condition elements, action elements and constraint elements, a directed connection relationship is constructed between condition nodes, action nodes and constraint nodes to form a scheduling procedure topology.

[0023] In this embodiment, generating a corresponding topology path identifier and hierarchical order information for each procedure execution path includes: Based on the scheduling procedure topology, three types of nodes are identified as condition nodes, action nodes, and constraint nodes. Directed relationships between nodes are established to represent the procedure execution relationship from condition to action that is constrained. The directed relation is traversed in an ordered manner from the condition node to the action node. The execution paths that satisfy the condition from the start to the target action and are restricted by the corresponding constraint nodes are enumerated. The node order, node type sequence and relation type sequence of each execution path are recorded. For each procedure execution path, a topology path identifier corresponding to each procedure execution path is generated. This topology path identifier is obtained by combining the node identifier sequence and relation type sequence of the procedure execution path according to a fixed rule and then encoding it with a fixed length. The fixed rule is as follows: According to the order of nodes in the execution path, the node type identifier and node unique identifier of each node are concatenated in sequence, and the corresponding relationship type identifier is inserted between adjacent nodes to form a structured path encoding string. Then, fixed-length encoding processing is performed on the structured path encoding string to generate a topology path identifier. Based on the chapters, clauses, and sub-clauses in the scheduling specification text, the level to which each node belongs in the procedure execution path is determined, forming hierarchical sequence information consistent with the order of the path nodes, and synchronously recording the sequential constraints determined by the hierarchical relationship; Establish a bidirectional index between topology path identifiers and procedure execution paths, complete the deduplication, uniqueness verification and number management of all procedure execution paths, and output the set of topology path identifiers and the corresponding hierarchical order information.

[0024] In this embodiment, the step of establishing a binding relationship between the parameter increments generated during the efficient fine-tuning of the general large language model and the corresponding parameter memory slots includes: Based on the topology path identifier set and hierarchical order information, the data structure of the topology control memory is initialized. The topology control memory includes a parameter memory slot area, a slot index area, a topology adjacency area, and a version evolution area. In the parameter memory slot area, a unique parameter memory slot record is created for each topology path identifier. The parameter memory slot record includes a slot identifier, a bound topology path identifier, a hierarchy order information reference, a slot status identifier, a slot version identifier, and a parameter incremental storage area. This parameter incremental storage area is divided into a write area and a freeze area, specifically as follows: The parameter increment storage area is logically divided according to the parameter generation time order and slot version identifier. The parameter increments generated in the current training cycle are written to the write area and marked as updatable. When the training period corresponding to the parameter increment ends or the slot version identifier changes, the parameter increment in the writing area is transferred to the frozen area, and the parameter increment in the frozen area is marked as non-updateable, allowing only reading and playback. In the slot index area, a forward index table from topology path identifier to parameter memory slot and a reverse index table from parameter memory slot to topology path identifier are constructed. Uniqueness checks are performed on all indexes to ensure that each topology path identifier corresponds to only one parameter memory slot and each parameter memory slot corresponds to only one topology path identifier. In the topological adjacency area, based on the scheduling procedure topology structure, the adjacency relationship between each topological path identifier is determined, a topological path adjacency table is generated and written into the adjacency reference field associated with the corresponding parameter memory slot, so that a searchable adjacency reference chain is established between the parameter memory slots corresponding to adjacent topological path identifiers. In the version evolution area, a version chain record is established for each parameter memory slot. When writing parameter increments, the parameter increment in the writing area is first solidified into a new version and written into the version chain record. Then, the parameter increment of the previous version is transferred to the frozen area and the status flag of the corresponding slot is updated to a replayable state, thus completing the construction of the topology control memory.

[0025] In this embodiment, the step of generating semantic energy values ​​and allocating them to parameter memory slots corresponding to topological path identifiers, and controlling the activation state and parameter update magnitude of the parameter memory slots, includes: A semantic power flow controller is constructed, which consists of an instruction element extraction unit, a topology power flow allocation unit, and a slot control unit. The semantic power flow controller is associated with the topology path identifier set and the parameter memory slot. The system receives scheduling task instructions, uses the instruction element extraction unit to structurally decompose the scheduling task instructions, and obtains task category elements, urgency elements and instruction keyword sets. Based on the topology path identifier set, the system performs matching and retrieval on the instruction keyword set to obtain the target topology path identifier set. The instruction element extraction unit generates semantic energy values ​​based on task category elements, urgency elements, and the size of the instruction keyword set. These semantic energy values ​​are then constrained within the total energy limit of the semantic energy flow regulator and used as input for this energy flow allocation. Specifically, the generation of the semantic energy values ​​involves: Based on the task category to which the scheduling task belongs, select the corresponding basic energy value within the preset task category energy range; Based on the urgency factor of the scheduling task, an urgency correction energy is applied to the base energy value; The additional energy corresponding to the keyword scale is determined based on the number of keywords in the instruction keyword set; The basic energy value, the urgency-based energy adjustment, and the additional energy corresponding to the keyword size are combined to form the initial semantic energy value; When the initial semantic energy value exceeds the total energy limit allowed by the semantic energy flow regulator, the initial semantic energy value is limited to the total energy limit and used as the final semantic energy value. The semantic energy value is allocated to the parameter memory slots corresponding to the target topology path identifier set using the topology energy flow allocation unit. The allocation includes determining the matching degree of each target topology path identifier based on the matching degree between the instruction keyword set and the procedure execution path keyword set associated with the target topology path identifier, allocating the semantic energy value as slot allocation energy of each parameter memory slot according to the matching degree, and transferring the slot allocation energy to adjacent parameter memory slots in a decreasing topology distance manner based on the adjacency reference chain, forming an adjacency energy flow between adjacent parameter memory slots. The slot control unit generates slot control parameters for each parameter memory slot based on the slot energy allocation and adjacent energy flow. The slot control parameters include the slot activation state and parameter update magnitude.

[0026] In this embodiment, obtaining the scheduling knowledge-specific large language model includes: Obtain the main parameters of the general large language model and set the main parameters to a frozen state. Read the parameter increments stored in the parameter memory slots, read the slot activation state and parameter update magnitude, and form the trainable parameter range for this efficient fine-tuning training of parameters. A training sample set is constructed based on the scheduling specification text, and a target topology path identifier is associated with each training sample. The correspondence between the topology path identifier and the procedure execution path is used to map the target topology path identifier to the target parameter memory slot, so that the training sample and the parameter memory slot are associated. For each training sample, the parameter increment corresponding to the target parameter memory slot is called, and the combined parameters are generated according to the slot activation state and parameter update magnitude. The combined parameters are formed by superimposing the frozen main parameters and the activated parameter increments, and the parameter increments corresponding to the inactive parameter memory slots do not participate in the generation of combined parameters. The model output is obtained by performing forward computation on the training samples based on the combined parameters. The training loss is then calculated based on the target output and the model output of the training samples. Specifically, the forward computation on the training samples based on the combined parameters is as follows: Load the main parameters of the general large language model at the start of training and keep them from being updated; Based on the topological path identifier associated with the training samples, determine the corresponding parameter memory slot and read the activated parameter increment; The parameter increment is scaled according to the parameter update magnitude and superimposed with the main parameter to form a combined parameter; The combined parameters are used to perform the forward inference process inside the model on the training samples, and the prediction results corresponding to the training samples are output. Perform backpropagation update on the training loss, update only the parameter increments corresponding to the activated parameter memory slots and keep the main parameters of the general large language model unchanged. Write the updated parameter increments back to the corresponding parameter memory slots and update the slot version identifier to obtain the scheduling knowledge-specific large language model.

[0027] In this embodiment, the obtained updated scheduling knowledge-specific large language model includes: When iteratively updating the proprietary language model of scheduling knowledge or integrating new scheduling specification texts, the newly added or changed scheduling specification texts are parsed hierarchically and the scheduling condition elements, scheduling action elements and constraint elements are extracted. The scheduling procedure topology is incrementally updated and the procedure execution path and topology path identifier corresponding to the newly added or changed content are generated. Based on the topology path identifier, the target parameter memory slot corresponding to the topology path identifier is retrieved from the topology control memory, and the historical version record of the target parameter memory slot is retrieved to form a set of historical parameter memory slots. Construct the set of models required for hierarchical distillation, set the current scheduling knowledge-specific large language model to be updated as the student model, set the scheduling knowledge-specific large language model of the previous iteration as the upper-level teacher model, and set the general large language model as the basic teacher model. Performing a topology coupling replay operation, specifically: Based on the scheduling procedure topology, the topological adjacency relationship and topological distance between the topological path identifier and each topological path identifier in the historical parameter memory slot set are determined. A replay priority sequence is generated based on topological distance. Then, a replay sample set is formed by reading parameter increments and associated samples from the historical parameter memory slot set based on this priority sequence. Specifically, the generation of the replay priority sequence based on topological distance is as follows: Determine the execution path of the procedure corresponding to the newly added or changed scheduling specification text as the target path; In the scheduling procedure topology, the number of nodes traversed or the path level difference between the target path and the historical procedure execution path are calculated to determine the topological distance; Arrange the parameter memory slots corresponding to the historical procedure execution paths with smaller topological distances at the beginning of the replay priority sequence; When multiple parameter memory slots have the same topological distance, the parameter memory slot that is closer to the target path in hierarchical order and has been updated the most recently will be selected for playback. Performing a tiered distillation operation, specifically: Input the replay sample set and the sample set corresponding to the newly added or changed content into the basic teacher model, the upper-level teacher model and the student model respectively to obtain the corresponding output; Using the teacher model output as the alignment target for the student model output, the sample-level distillation loss is calculated, and the distillation loss is updated with weights based on the replay priority sequence. Specifically, the sample-level distillation loss is calculated as follows: Input the replay sample into the teacher model and obtain the target output representation generated by the teacher model for the sample; Input the same replay sample into the student model and obtain the predicted output representation generated by the student model for the sample. Compare the differences between the teacher model output representation and the student model output representation in terms of semantic content, output probability distribution, or representation vector; Based on the degree of difference, a distillation loss corresponding to the replay sample is generated to measure the alignment between the student model output and the teacher model output. While keeping the main parameters of the general large language model frozen, only the parameter increments corresponding to the target parameter memory slot and the historical parameter memory slot set are updated, and the updated parameter increments are written back to the topology control memory to obtain the updated scheduling knowledge-specific large language model.

[0028] Example 1:

[0029] To verify the feasibility of this invention in practice, it was applied to the dispatch control center of a power grid company. This dispatch center is responsible for the operation monitoring, mode adjustment, and emergency response of the regional main grid and important substations. Its daily work heavily relies on dispatch specification texts, including dispatch standards, dispatch procedures, and emergency plans. These dispatch specification texts are characterized by a large number of clauses, a complex hierarchical structure, strict conditions and constraints, and a high update frequency. Furthermore, they have mandatory enforcement attributes in dispatch operations.

[0030] In actual operations, dispatchers frequently need to query regulatory clauses, verify operational conditions, and generate dispatch instruction suggestions through information systems. As the amount of dispatch specification text increases and is updated, traditional manual retrieval and rule-based systems gradually become insufficient to meet response efficiency requirements. While the introduction of a general large language model has improved text understanding speed, several problems have gradually emerged during continuous operation. These include the model still referencing old clauses after specification updates, logical inconsistencies in suggestions given under different dispatch tasks, and deviations in understanding earlier specifications after multiple parameter fine-tunings. These issues affect the security and reliability of dispatch operations.

[0031] In this embodiment, the scheduling specification text currently used by the scheduling center is first incorporated into the model construction process. The system processes the scheduling specification text uniformly, extracting scheduling condition elements, scheduling action elements, and constraint elements, and constructs a scheduling procedure topology structure accordingly, so that each scheduling specification has a clear execution logic in its structure.

[0032] Subsequently, the execution paths in the scheduling procedure topology are traversed, generating topology path identifiers for different execution paths and recording the corresponding hierarchical order information to distinguish different scheduling logic paths. These topology path identifiers serve as structural indexes for subsequent parameter management and updates, preventing the mixing of different procedure logics at the parameter level.

[0033] During the model training phase, a topology control memory is constructed, allocating an independent parameter memory slot to each topology path identifier. Parameter increments generated during efficient parameter fine-tuning are written into the corresponding slots, achieving a structured binding between scheduling procedure paths and model parameters. A semantic energy flow control mechanism is introduced to dynamically control the activation state of relevant parameter memory slots and the parameter update magnitude based on the content of the scheduling task instructions, enabling the model to focus on strengthening relevant path parameters in critical scheduling tasks.

[0034] During model inference and continuous training, updates are performed only on the parameter increments corresponding to the activated parameter memory slots, while the main parameters of the general large language model remain frozen. When scheduling procedures are updated or new contingency plans are added, new procedure execution paths and topology path identifiers are generated for the added or changed clauses. Through hierarchical distillation and topology coupling replay, historical parameter memory slots are selectively replayed and aligned for updates, thereby absorbing new procedure content while maintaining the model's stable understanding of existing scheduling logic.

[0035] Three consecutive months of actual operational data were selected from the scheduling center to conduct comparative tests on the scheduling knowledge-specific large language model constructed using the method of this invention and the model constructed using the traditional static parameter efficient fine-tuning method. The test tasks covered indicators such as scheduling specification clause query, multi-condition procedure matching, stability after procedure updates, and consistency of daily model operation.

[0036] Table 1 Comparison of Model Application Effects in Scheduling Business Scenarios

[0037] As shown in Table 1, in core business scenarios such as scheduling specification clause query and multi-condition procedure matching, the model of this invention exhibits a stable and reasonable performance improvement compared to the static parameter efficient fine-tuning model. The accuracy rate of scheduling specification clause query increased from 87.9% to 92.3%, and the accuracy rate of multi-condition procedure matching increased from 81.6% to 88.7%. This indicates that by introducing the scheduling procedure topology structure and topology path identifier, the model can more accurately distinguish different procedure execution paths, reduce confusion between similar clauses, and thus improve the accuracy of scheduling specification understanding.

[0038] Regarding adaptability after protocol updates, the number of misreferences in the model of this invention decreased significantly within one week after the update, from 12 times to 5 times. This indicates that through topology-controlled memory and hierarchical distillation and topology-coupled replay mechanisms, the model can selectively replay and align relevant historical parameters when absorbing new protocol content, effectively alleviating the confusion between old and new protocols caused by overall parameter updates in existing technologies. Furthermore, the model of this invention maintains high scores in emergency response recommendation consistency and continuous operation stability indicators, demonstrating output consistency during multi-task switching and long-term operation.

[0039] In terms of model maintenance and update efficiency, the method of this invention significantly reduces the adjustment time after a single procedure update from 8.6 hours to 3.9 hours. This indicates that the method can complete model adaptation by updating local parameter memory slots without the need for overall retraining, thereby reducing operation and maintenance costs and improving the availability and engineering feasibility of the model in actual scheduling operations.

[0040] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A proprietary large language model construction method based on efficient parameter fine-tuning, characterized in that, include: Collect dispatching specification texts related to power dispatching operations, process the dispatching specification texts, and construct a dispatching procedure topology structure consisting of condition nodes, action nodes, and constraint nodes; Based on the scheduling procedure topology, a topology path identifier corresponding to each procedure execution path is generated, and a corresponding topology path identifier and hierarchical order information are generated for each procedure execution path. Construct a topology control memory, allocate a corresponding parameter memory slot to each topology path identifier, and establish a binding relationship between the parameter increments generated by the general large language model during the efficient fine-tuning of parameters and the corresponding parameter memory slots. A semantic energy flow regulator is introduced to parse the input scheduling task instructions, generate semantic energy values ​​and allocate them to the parameter memory slots corresponding to the topology path identifiers, and control the activation state of the parameter memory slots and the parameter update magnitude. While keeping the main parameters of the general large language model frozen, based on the parameter memory slots, activation states and parameter update magnitudes, efficient parameter fine-tuning training is performed only on the parameter increments corresponding to the activated parameter memory slots to obtain a scheduling knowledge-specific large language model. When iteratively updating the scheduling knowledge-specific large language model or integrating new scheduling specification texts, hierarchical distillation and topology coupling replay operations are performed based on the scheduling procedure topology structure, topology path identifiers, and parameter memory slots. Selective replay and alignment updates are performed on historical parameter memory slots to obtain the updated scheduling knowledge-specific large language model.

2. The proprietary large language model construction method based on efficient parameter fine-tuning according to claim 1, characterized in that, The dispatching specifications include power dispatching standard documents, power dispatching procedure documents, and power dispatching emergency response plan documents.

3. The proprietary large language model construction method based on efficient parameter fine-tuning according to claim 1, characterized in that, The process of processing the scheduling specification text and constructing a scheduling procedure topology structure consisting of condition nodes, action nodes, and constraint nodes includes: The scheduling specification text is parsed hierarchically to identify the chapter structure, clause numbers, and clause hierarchical relationships, generating the hierarchical structure information of the scheduling specification text. Based on the hierarchical structure information, condition elements representing the scheduling triggering conditions are extracted from the content of each clause, and the condition elements are mapped to condition nodes. Extract action elements representing scheduling operation behaviors from the content of each clause, and map the action elements as action nodes; Extract constraint elements representing scheduling restriction rules, execution order, or prohibition relationships from each clause, and map these constraint elements as constraint nodes; Based on the hierarchical relationship between clauses and the logical relationship between condition elements, action elements and constraint elements, a directed connection relationship is constructed between condition nodes, action nodes and constraint nodes to form a scheduling procedure topology.

4. The proprietary large language model construction method based on efficient parameter fine-tuning according to claim 1, characterized in that, The process of generating a corresponding topology path identifier and hierarchical order information for each procedure execution path includes: Based on the scheduling procedure topology, three types of nodes are identified as condition nodes, action nodes, and constraint nodes. Directed relationships between nodes are established to represent the procedure execution relationship from condition to action that is constrained. The directed relation is traversed in an ordered manner from the condition node to the action node. The execution paths that satisfy the condition from the start to the target action and are restricted by the corresponding constraint nodes are enumerated. The node order, node type sequence and relation type sequence of each execution path are recorded. For each procedure execution path, a topology path identifier corresponding to the procedure execution path is generated. The topology path identifier is obtained by combining the node identifier sequence and relation type sequence of the procedure execution path according to fixed rules and then encoding it with fixed length. Based on the chapters, clauses, and sub-clauses in the scheduling specification text, the level to which each node belongs in the procedure execution path is determined, forming hierarchical sequence information consistent with the order of the path nodes, and synchronously recording the sequential constraints determined by the hierarchical relationship; Establish a bidirectional index between topology path identifiers and procedure execution paths, complete the deduplication, uniqueness verification and number management of all procedure execution paths, and output the set of topology path identifiers and the corresponding hierarchical order information.

5. The proprietary large language model construction method based on efficient parameter fine-tuning according to claim 1, characterized in that, The process of binding parameter increments generated during efficient parameter fine-tuning of the general-purpose large language model with corresponding parameter memory slots includes: Based on the topology path identifier set and hierarchical order information, the data structure of the topology control memory is initialized. The topology control memory includes a parameter memory slot area, a slot index area, a topology adjacency area, and a version evolution area. In the parameter memory slot area, a unique parameter memory slot record is created for each topology path identifier. The parameter memory slot record includes the slot identifier, the bound topology path identifier, the hierarchical order information reference, the slot status identifier, the slot version identifier, and the parameter incremental storage area, which is divided into a write area and a freeze area. In the slot index area, a forward index table from topology path identifier to parameter memory slot and a reverse index table from parameter memory slot to topology path identifier are constructed. Uniqueness checks are performed on all indexes to ensure that each topology path identifier corresponds to only one parameter memory slot and each parameter memory slot corresponds to only one topology path identifier. In the topological adjacency area, based on the scheduling procedure topology structure, the adjacency relationship between each topological path identifier is determined, a topological path adjacency table is generated and written into the adjacency reference field associated with the corresponding parameter memory slot, so that a searchable adjacency reference chain is established between the parameter memory slots corresponding to adjacent topological path identifiers. In the version evolution area, a version chain record is established for each parameter memory slot. When writing parameter increments, the parameter increment in the writing area is first solidified into a new version and written into the version chain record. Then, the parameter increment of the previous version is transferred to the frozen area and the status flag of the corresponding slot is updated to a replayable state, thus completing the construction of the topology control memory.

6. The proprietary large language model construction method based on efficient parameter fine-tuning according to claim 1, characterized in that, The process of generating semantic energy values ​​and allocating them to parameter memory slots corresponding to topological path identifiers, and controlling the activation state and parameter update magnitude of these parameter memory slots, includes: A semantic power flow controller is constructed, which consists of an instruction element extraction unit, a topology power flow allocation unit, and a slot control unit. The semantic power flow controller is associated with the topology path identifier set and the parameter memory slot. The system receives scheduling task instructions, uses the instruction element extraction unit to structurally decompose the scheduling task instructions, and obtains task category elements, urgency elements and instruction keyword sets. Based on the topology path identifier set, the system performs matching and retrieval on the instruction keyword set to obtain the target topology path identifier set. The instruction element extraction unit generates semantic energy values ​​based on task category elements, urgency elements, and the size of the instruction keyword set. The semantic energy values ​​are then limited to the total energy constraint range of the semantic energy flow controller and used as the input for this energy flow allocation. The semantic energy value is allocated to the parameter memory slots corresponding to the target topology path identifier set using the topology energy flow allocation unit. The allocation includes determining the matching degree of each target topology path identifier based on the matching degree between the instruction keyword set and the procedure execution path keyword set associated with the target topology path identifier, allocating the semantic energy value as slot allocation energy of each parameter memory slot according to the matching degree, and transferring the slot allocation energy to adjacent parameter memory slots in a decreasing topology distance manner based on the adjacency reference chain, forming an adjacency energy flow between adjacent parameter memory slots. The slot control unit generates slot control parameters for each parameter memory slot based on the slot energy allocation and adjacent energy flow. The slot control parameters include the slot activation state and parameter update magnitude.

7. The proprietary large language model construction method based on efficient parameter fine-tuning according to claim 1, characterized in that, The obtained scheduling knowledge-specific large language model includes: Obtain the main parameters of the general large language model and set the main parameters to a frozen state. Read the parameter increments stored in the parameter memory slots, read the slot activation state and parameter update magnitude, and form the trainable parameter range for this efficient fine-tuning training of parameters. A training sample set is constructed based on the scheduling specification text, and a target topology path identifier is associated with each training sample. The correspondence between the topology path identifier and the procedure execution path is used to map the target topology path identifier to the target parameter memory slot, so that the training sample and the parameter memory slot are associated. For each training sample, the parameter increment corresponding to the target parameter memory slot is called, and the combined parameters are generated according to the slot activation state and parameter update magnitude. The combined parameters are formed by superimposing the frozen main parameters and the activated parameter increments, and the parameter increments corresponding to the inactive parameter memory slots do not participate in the generation of combined parameters. The model output is obtained by performing forward computation on the training samples based on the combined parameters, and the training loss is calculated based on the target output of the training samples and the model output. Perform backpropagation update on the training loss, update only the parameter increments corresponding to the activated parameter memory slots and keep the main parameters of the general large language model unchanged. Write the updated parameter increments back to the corresponding parameter memory slots and update the slot version identifier to obtain the scheduling knowledge-specific large language model.

8. The proprietary large language model construction method based on efficient parameter fine-tuning according to claim 1, characterized in that, The updated scheduling knowledge-specific large language model includes: When iteratively updating the proprietary language model of scheduling knowledge or integrating new scheduling specification texts, the newly added or changed scheduling specification texts are parsed hierarchically and the scheduling condition elements, scheduling action elements and constraint elements are extracted. The scheduling procedure topology is incrementally updated and the procedure execution path and topology path identifier corresponding to the newly added or changed content are generated. Based on the topology path identifier, the target parameter memory slot corresponding to the topology path identifier is retrieved from the topology control memory, and the historical version record of the target parameter memory slot is retrieved to form a set of historical parameter memory slots. Construct the set of models required for hierarchical distillation, set the current scheduling knowledge-specific large language model to be updated as the student model, set the scheduling knowledge-specific large language model of the previous iteration as the upper-level teacher model, and set the general large language model as the basic teacher model. Performing a topology coupling replay operation, specifically: Based on the scheduling procedure topology, the topological adjacency relationship and topological distance between the topological path identifier and each topological path identifier in the historical parameter memory slot set are determined. A replay priority sequence is generated based on topological distance. Based on the replay priority sequence, parameter increments and associated samples are read from the historical parameter memory slot set to form a replay sample set. Performing a tiered distillation operation, specifically: Input the replay sample set and the sample set corresponding to the newly added or changed content into the basic teacher model, the upper-level teacher model and the student model respectively to obtain the corresponding output; Using the teacher model output as the alignment target for the student model output, the sample-level distillation loss is calculated, and the distillation loss is updated with weights based on the replay priority sequence. While keeping the main parameters of the general large language model frozen, only the parameter increments corresponding to the target parameter memory slot and the historical parameter memory slot set are updated, and the updated parameter increments are written back to the topology control memory to obtain the updated scheduling knowledge-specific large language model.

Images