A progressive fine-tuning method and device for power scenario model migration

Abstract

The application relates to a progressive fine-tuning method and device for power scene model migration. The method comprises the following steps: obtaining a plurality of model parameters of a target model to be trained, determining the attention weights of each model parameter to each power knowledge entity in a pre-constructed power knowledge graph; mapping the attention weights corresponding to each model parameter into the activation scores of each model parameter; classifying each model parameter into a corresponding parameter layer according to the activation scores; obtaining a plurality of power corpora, and determining the first information entropy of each power corpus; classifying each power corpus into a corresponding corpus gradient according to the first information entropy; activating the parameter layer matched with each corpus gradient in the ascending order of the corpus gradient, training the activated parameter layer by using the power corpus of the corresponding corpus gradient, and obtaining the trained target model until the training stopping condition is reached. The method can improve the training efficiency and performance of model migration learning in the power scene.

Description

Technical Field

[0001] This application relates to the field of power data processing technology, and in particular to a progressive fine-tuning method, apparatus, computer equipment, computer-readable storage medium, and computer program product for power scenario model migration. Background Technology

[0002] With the development of artificial intelligence technology, multimodal large models are gradually being applied in business scenarios such as production, safety supervision, and infrastructure construction in the power industry to process multi-source data such as images, videos, and text, and to assist in achieving intelligent recognition and decision support.

[0003] One key technical approach for applying large-scale models is to transfer learning from general pre-trained models to specific power scenarios. Since general pre-trained models lack power domain expertise, it is necessary to retrain and fine-tune the model parameters using training data relevant to power scenarios to adapt them to specific power scenarios.

[0004] In related technologies, training is performed on the entire power corpus at a single level. However, due to the highly specialized nature of power corpus, its wide range of professional fields, and its high degree of complexity, this training method may lead to overfitting due to redundancy of simple parameters and underfitting due to insufficient complex parameters, ultimately resulting in slow model convergence and poor performance. Summary of the Invention

[0005] Based on this, it is necessary to provide a progressive fine-tuning method, apparatus, computer equipment, computer-readable storage medium, and computer program product for power scenario model transfer that can improve the model training efficiency and performance under power scenario transfer learning, thereby addressing the aforementioned technical problems.

[0006] Firstly, this application provides a progressive fine-tuning method for power scenario model transfer, including:

[0007] Obtain multiple model parameters of the target model to be trained, and determine the attention weight of each model parameter to each power knowledge entity in the pre-constructed power knowledge graph;

[0008] The attention weights corresponding to each of the model parameters are mapped to the activation scores of each of the model parameters;

[0009] Each of the model parameters is categorized into a corresponding parameter layer according to its activation score; wherein, different parameter layers correspond to different numerical ranges of the activation score.

[0010] Acquire multiple power-related data sets and determine the first information entropy of each data set.

[0011] Each of the aforementioned power-related corpora is categorized into a corresponding corpus gradient according to its respective first information entropy; wherein, different corpus gradients correspond to different numerical ranges of the first information entropy;

[0012] According to the increasing order of the corpus gradient, the parameter layer that matches each corpus gradient is activated, and the activated parameter layer is trained using the electrical corpus of the corresponding corpus gradient until the training stopping condition is reached, so as to obtain the trained target model.

[0013] In one embodiment, the parameter layer includes a basic parameter layer, a physical feature encoding layer, and a task-related parameter layer that increase according to the numerical range of the activation scores; the corpus gradient includes a basic corpus gradient, a business logic corpus gradient, and an expert decision corpus gradient that increase according to the numerical range of the first information entropy; and the parameter layer that matches the activation with each of the corpus gradients includes:

[0014] At any of the corpus gradients, the basic parameter layer remains frozen.

[0015] When the gradient of the corpus is the same as the gradient of the base corpus, the physical feature encoding layer is activated;

[0016] When the corpus gradient is the same as the business logic corpus gradient, the task-related parameter layer is activated;

[0017] When the gradient of the corpus is the same as the gradient of the expert decision corpus, the physical feature encoding layer and the task-related parameter layer are activated.

[0018] In one embodiment, the target model structure includes a multi-head attention layer, and the model parameters in the task-related parameter layer include the query matrix and key matrix of the multi-head attention layer. Training the activated parameter layer using electrical corpus corresponding to the corpus gradient includes:

[0019] When the corpus gradient is the business logic corpus gradient, the power corpus of the business logic corpus gradient is divided into power corpus corresponding to multiple training batches;

[0020] The second information entropy of the power corpus corresponding to each training batch is determined, and the rank value of the preset low-rank matrix is ​​adjusted according to each second information entropy to obtain the target low-rank matrix corresponding to each training batch; wherein, the second information entropy is positively correlated with the rank value;

[0021] The query matrix and key matrix corresponding to each training batch are respectively concatenated in parallel with the target low-rank matrix to train the query matrix and key matrix corresponding to each training batch.

[0022] In one embodiment, training the activated parameter layer using electrical corpus corresponding to the corpus gradient further includes:

[0023] When the gradient of the corpus is the gradient of the expert decision corpus, and gradient updates need to be performed on the parameter layer, the output state variable of the target model is obtained.

[0024] Determine the Jacobian matrix of the pre-constructed power physics constraint function relative to the output state variable, and obtain the zero-space basis matrix from the Jacobian matrix;

[0025] Obtain the first updated gradient vector of the parameter layer, and project the first updated gradient vector onto the null space corresponding to the null space basis matrix to obtain the first projection vector of the first updated gradient vector.

[0026] The first projection vector is used to perform gradient updates on the parameter layer.

[0027] In one embodiment, the trained target model has a corresponding initial power scenario, and the method further includes:

[0028] Determine the Fisher information matrix corresponding to the model parameters of the trained target model;

[0029] Based on the activation scores corresponding to the physical feature coding layers, adjust the diagonal elements in the Fischer information matrix;

[0030] The adjusted Fischer information matrix is ​​injected into a pre-constructed loss function, and the loss function is used to update the parameters of the trained target model for the target power scenario, so as to transfer the trained target model from the initial power scenario to the target power scenario.

[0031] In one embodiment, updating the parameters of the trained target model using the loss function for the target power scenario includes:

[0032] Obtain the second updated gradient vector of the model parameters of the trained target model, and project the second updated gradient vector onto the null space to obtain the second projected vector of the second updated gradient vector;

[0033] If the similarity between the second updated gradient vector and the second projected vector does not meet the preset conditions, the decay coefficient is determined based on the similarity.

[0034] The parameter update amount for the target power scenario is determined based on the attenuation coefficient and the second projection vector, and the parameter update amount is used to update the parameters of the trained target model.

[0035] Secondly, this application also provides a progressive fine-tuning device for power scenario model transfer, comprising:

[0036] The acquisition module is used to acquire multiple model parameters of the target model to be trained and determine the attention weight of each model parameter to each power knowledge entity in the pre-built power knowledge graph.

[0037] The mapping module is used to map the attention weights corresponding to each of the model parameters to the activation scores of each of the model parameters;

[0038] The first classification module is used to classify each of the model parameters into a corresponding parameter layer according to each of the activation scores; wherein, different parameter layers correspond to different numerical ranges of the activation scores.

[0039] A determination module is used to acquire multiple power-related data sets and determine the first information entropy of each power-related data set.

[0040] The second classification module is used to classify each of the power-related corpora into a corresponding corpus gradient according to each of the first information entropy; wherein, different corpus gradients correspond to different numerical ranges of the first information entropy.

[0041] The training module is used to activate parameter layers that match each of the corpus gradients in ascending order of the corpus gradients, and train the activated parameter layers using electrical corpus corresponding to the corpus gradients until the training stopping condition is met, so as to obtain the trained target model.

[0042] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described progressive fine-tuning method for power scenario model migration.

[0043] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described progressive fine-tuning method for power scenario model migration.

[0044] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described progressive fine-tuning method for power scenario model migration.

[0045] The aforementioned progressive fine-tuning method, apparatus, computer equipment, computer-readable storage medium, and computer program product for power scenario model transfer, maps the attention weights corresponding to each model parameter to the activation scores of each model parameter. These activation scores reflect the sensitivity and correlation of each model parameter to power-related professional knowledge. The model parameters are then categorized into different parameter layers based on their activation scores. The first information entropy reflects the information uncertainty of each power-related corpus, and the parameters are categorized into different corpus gradients. Training is performed according to the increasing order of the corpus gradients, i.e., from low to high training task complexity, activating matching parameter layers. The model parameter configuration is dynamically adjusted, achieving phased, progressive fine-tuning of model parameters. This balances training efficiency and expressive power of model parameters as much as possible, while also accelerating training convergence and improving the target model's reasoning ability for complex power scenario tasks. It is particularly suitable for scenarios involving transfer learning of models for power scenarios. For example, learning transfer on small-scale power data can reduce overfitting and ensure the trained target model has good adaptability to the corresponding power scenario. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of the application environment provided for an embodiment of this application.

[0048] Figure 2 A flowchart illustrating the steps of a progressive fine-tuning method for power scenario model migration provided in an embodiment of this application.

[0049] Figure 3 This is a simplified flowchart of a power scenario transfer learning process provided in one embodiment of this application.

[0050] Figure 4 This is a system architecture diagram of a transfer learning system applied to a power scenario, provided as an embodiment of this application.

[0051] Figure 5 This is a structural block diagram of a progressive fine-tuning device for power scenario model migration provided in an embodiment of this application.

[0052] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0055] The progressive fine-tuning method for power scenario model transfer provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server. Terminal 102 sends a request to server 104 to train the target model. Server 104 responds to the request by acquiring multiple model parameters of the target model to be trained, determining the attention weights of each model parameter to each power knowledge entity in the pre-constructed power knowledge graph, mapping the attention weights of each model parameter to the activation score of each model parameter, and classifying each model parameter into its corresponding parameter layer according to its activation score. Different parameter layers correspond to different numerical ranges of activation scores. Server 104 acquires multiple power-related corpora and determines the first information entropy of each corpus. Server 104 classifies each power-related corpus into its corresponding corpus gradient according to its first information entropy. Different corpus gradients correspond to different numerical ranges of the first information entropy. Server 104 activates the parameter layers matching each corpus gradient in ascending order of the corpus gradients, trains the activated parameter layers using power-related corpus data with the corresponding gradients until the training stops, and obtains the trained target model. Server 104 then feeds back the training results of the target model to terminal 102. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0056] In one exemplary embodiment, such as Figure 2 As shown, a progressive fine-tuning method for power scenario model transfer is provided, which is then applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 202 to 212. Wherein:

[0057] Step 202: Obtain multiple model parameters of the target model to be trained, and determine the attention weight of each model parameter to each power knowledge entity in the pre-constructed power knowledge graph;

[0058] The target model can be a general model that needs to be transferred to a specific power scenario, such as a model that can perform data processing, text processing, image processing, etc.

[0059] Among them, the power knowledge graph can be pre-built based on power system professional knowledge. Each power knowledge entity in the power knowledge graph can have a corresponding power business, such as power transmission and distribution, safety, planning and other power businesses. Specifically, power knowledge entities can be related knowledge entities such as circuit breakers, buses, power flow state quantities, relay protection logic and so on.

[0060] In practical implementation, the relationships between various power knowledge entities in the pre-built power knowledge graph can be extracted. The model parameters and relationships are then input into a pre-trained weight recognition model based on an attention mechanism. The model parameters are output as attention weights for each power knowledge entity. These attention weights reflect the sensitivity of the model parameters to individual power knowledge entities related to power business logic.

[0061] Step 204: Map the attention weights corresponding to each model parameter to the activation scores of each model parameter;

[0062] In practical implementation, each attention weight corresponding to each model parameter can be mapped to a corresponding activation score according to a preset algorithm or preset mapping relationship. For example, the activation score of a model parameter can be obtained by weighting and averaging the multiple attention weights corresponding to it. The activation score can reflect the degree of adaptation of the model parameter to the overall power business knowledge. The higher the activation score, the better the adaptation of the model parameter to the power business knowledge.

[0063] Step 206: Classify each model parameter into a corresponding parameter layer according to each activation score; wherein, different parameter layers correspond to different numerical ranges of the activation scores;

[0064] The parameter layer is a collection of model parameters after classification. The higher the maximum activation score of the numerical range corresponding to the parameter layer, the higher the correlation between each model parameter in the parameter layer and power business knowledge.

[0065] In some examples, parameter layers can be divided according to the quantiles of the activation scores. Specifically, the 25th quantile of the activation score is set as the lower threshold, and the 75th quantile is set as the upper threshold. Model parameters with activation scores below the lower threshold are classified into the first parameter layer; model parameters with activation scores between the lower and upper thresholds are classified into the second parameter layer; and model parameters with activation scores above the upper threshold are classified into the third parameter layer.

[0066] Step 208: Obtain multiple power-related data sets and determine the first information entropy of each data set;

[0067] Among them, the first information entropy is used to reflect the information uncertainty of the power corpus. The higher the entropy value of the first information entropy, the greater the information uncertainty and the higher the logical complexity of the power corpus.

[0068] In a practical implementation, Shannon entropy calculation can be performed on each power corpus sequence. The probability of each token appearing in the power corpus sequence is counted, and the negative value of the sum of the products of the probabilities of all tokens and their logarithms is calculated to obtain the Shannon entropy value of the power corpus sequence. The Shannon entropy value is used as the first information entropy.

[0069] For example, suppose the input sequence of electricity-related corpus contains There are 3 different tokens, the 1st The probability of each marker appearing is Then the Shannon entropy of the sequence The calculation formula is:

[0070] (1)

[0071] Step 210: Classify each of the power-related corpora into a corresponding corpus gradient according to each of the first information entropies; wherein, different corpus gradients correspond to different numerical ranges of the first information entropy;

[0072] In practical implementation, the quantile method can be used to determine the partition boundaries of the gradient of each corpus.

[0073] For example, the 33rd quantile of the entropy value of the first information entropy is set as the dividing point between low complexity and medium complexity, and the 67th quantile is set as the dividing point between medium complexity and high complexity, thus obtaining the corpus gradients representing low complexity, medium complexity, and high complexity, respectively.

[0074] Step 212: Activate the parameter layer that matches each of the corpus gradients in the increasing order of the corpus gradients, and train the activated parameter layer using the electrical corpus of the corresponding corpus gradient until the training stopping condition is met, so as to obtain the trained target model.

[0075] In the implementation, a mapping relationship between each corpus gradient and each parameter layer can be predefined. The corpus gradients and the parameter layers to be activated are then matched according to this mapping relationship. Activated parameter layers participate in training, while inactive parameter layers remain frozen and do not participate in gradient updates. After training each parameter layer in ascending order of corpus gradients (e.g., from low complexity to medium complexity to high complexity), the trained target model is obtained.

[0076] In some embodiments, the parameter layer includes a basic parameter layer, a physical feature encoding layer, and a task-related parameter layer that increase according to the numerical range of the activation scores; the corpus gradient includes a basic corpus gradient, a business logic corpus gradient, and an expert decision corpus gradient that increase according to the numerical range of the first information entropy; and the parameter layer whose activation matches each of the corpus gradients includes:

[0077] At any of the corpus gradients, the basic parameter layer remains frozen.

[0078] When the gradient of the corpus is the same as the gradient of the base corpus, the physical feature encoding layer is activated;

[0079] When the corpus gradient is the same as the business logic corpus gradient, the task-related parameter layer is activated;

[0080] When the gradient of the corpus is the same as the gradient of the expert decision corpus, the physical feature encoding layer and the task-related parameter layer are activated.

[0081] The basic parameter layer contains the model parameters responsible for basic language representation functions. These parameters remain frozen throughout the fine-tuning process and do not participate in gradient updates. For example, the basic parameter layer can be categorized from model parameters with activation scores below a lower threshold, such as the weight matrix and positional encoding parameters of the word embedding layer.

[0082] The physical feature encoding layer contains model parameters for specific neurons that directly process related continuous numerical features.

[0083] The task-related parameter layer contains core model parameters for processing power industry terminology and multi-business scenario procedural logic, such as the query (projection) matrix in the multi-head attention projection matrix. Key (projection) matrix Sum (Value) (Projection) Matrix In one possible scenario, when the target model deals with electrical engineering terms such as 220kV busbar undervoltage, the activation scores of each model parameter in the task-related parameter layer are higher than those of each model parameter in the basic parameter layer.

[0084] The basic corpus gradient is a low-complexity corpus gradient. This gradient primarily covers highly structured texts such as power industry terminology tables, equipment ledger parameters, and national standards for the power industry. Examples include standardized expressions in the "Preventive Testing Regulations for Power Equipment," transformer nameplate parameter tables, and circuit breaker model specifications. The physical feature encoding layer is activated, and the power corpus from the basic corpus gradient is used for training. The key objective is to correct the weights of the basic embedding layer of the target model and establish preliminary vector representations of power entities.

[0085] The business logic corpus gradient is of medium complexity. This gradient primarily covers texts containing operational timing and causal relationships, such as multi-service coordination procedures, substation switching operation procedures, and fault inspection records. For example, texts containing conditional judgments and operational sequences include phrases like, "When the 220kV bus voltage is below 198kV, the dispatcher should immediately check the reactive power compensation device's activation status and, if necessary, activate the capacitor bank." The task-related parameter layer is activated, and the power corpus of the business logic corpus is used for training. The key objective is to enable the target model to fit the topological connections and operational timing logic between power equipment.

[0086] Among them, the expert decision-making corpus gradient belongs to the highly complex corpus gradient category. The expert decision-making corpus gradient can include power grid accident analysis reports, relay protection setting calculation instructions, complex power flow calculations and adjustment commands, and other power corpus texts requiring deep reasoning. For example, in a 220kV busbar undervoltage accident at a 500kV substation, the system operation mode before the fault was during the Spring Festival off-peak load period, with the 220kV I section busbar load at 32MW and the II section busbar load at 28MW. Analysis determined the fault to be caused by the blown secondary fuse of the 220kV I busbar PT, leading to protection malfunction, and other complex analyses involving multiple causal chains. Activating the physical feature encoding layer and the task-related parameter layer, and training with power corpus data from the expert decision-making corpus gradient, the key objective is to enhance the target model's causal reasoning ability under unsteady-state conditions.

[0087] In some embodiments, the classification of the basic parameter layer, the physical feature encoding layer, and the task-related parameter layer can be achieved by constructing a Boolean mask based on activation scores.

[0088] Specifically, let the total parameter set of the target model be... Each parameter The corresponding normalized activation score is ,but:

[0089] Normalized activation scores below the lower quantile threshold The model parameters are categorized into the basic parameter layer and a freeze operation is performed, i.e., when Set Boolean mask at this time. The gradients of the model parameters in the basic parameter layer are set to zero during training.

[0090] Model parameters whose activation scores are in the middle range and are directly connected to the output of the numerical encoder are classified as physical feature encoding layers, i.e., when... and When connected to a numerical encoder, set a Boolean mask. ;

[0091] For scores higher than the upper quantile threshold And the parameters located in the attention projection matrix are labeled as task-related parameter layers, i.e., when and Set Boolean mask at this time. .

[0092] In some embodiments, where the corpus gradient is the base corpus gradient, the model parameters of the base parameter layer are kept frozen during the training phase of this base corpus gradient, and only the model parameters in the physical feature encoding layer are updated. All parameter updates in this training phase directly affect the original weight matrix.

[0093] Specifically, the model output distribution of the target model can be minimized. Probability distribution of electricity corpus The KL divergence between the two features is used as the training objective, and the model parameters in the physical feature encoding layer are trained using a pre-constructed loss function. The loss function is specifically expressed as follows:

[0094] (2)

[0095] in, This represents the set of all possible output labels. This optimization objective forces the model to map electrical physical quantities (such as voltage, current, active power, reactive power, etc.) to a specific vector space, reducing semantic biases introduced by general corpora. For example, through training in this stage, the model can correctly identify that 10kV represents the distribution voltage level rather than a distance unit, and that bus segment I represents a specific bus segment rather than a sequence number.

[0096] In practical applications, when the decrease in KL divergence on the validation set is less than a preset convergence threshold for five consecutive training epochs... (For example When the training phase of the basic corpus gradient has converged, the training can be terminated and the next corpus gradient training phase can begin.

[0097] In some embodiments, the target model structure includes a multi-head attention layer, and the model parameters in the task-related parameter layer include the query matrix and key matrix of the multi-head attention layer. Training the activated parameter layer using electrical corpus corresponding to the corpus gradient includes:

[0098] When the corpus gradient is the business logic corpus gradient, the power corpus of the business logic corpus gradient is divided into power corpus corresponding to multiple training batches;

[0099] The second information entropy of the power corpus corresponding to each training batch is determined, and the rank value of the preset low-rank matrix is ​​adjusted according to each second information entropy to obtain the target low-rank matrix corresponding to each training batch; wherein, the second information entropy is positively correlated with the rank value;

[0100] The query matrix and key matrix corresponding to each training batch are respectively concatenated in parallel with the target low-rank matrix to train the query matrix and key matrix corresponding to each training batch.

[0101] A low-rank matrix is ​​a matrix whose rank is much smaller than the number of rows and columns, and can be represented by the product of two small-dimensional matrices. Specifically, low-rank matrices can be implemented using the LoRA (Low-Rank Adaptive) architecture, which allows for efficient parameter fine-tuning.

[0102] In the specific implementation, when the corpus gradient is the same as the business logic corpus gradient, the basic parameter layer is kept frozen, and the task-related parameter layer is activated for training. The model parameters in the task-related parameter layer include the query matrix of the multi-head attention layer. Bond matrix ;

[0103] According to the preset partitioning logic, the power corpus of the business logic corpus gradient can be divided into power corpus corresponding to multiple training batches. The average Shannon entropy of all power corpus in each training batch is used as the second information entropy corresponding to each training batch to reflect the information uncertainty of each training batch.

[0104] Specifically, during the training iteration process, the average Shannon entropy of the power corpus for each training batch can be calculated in real time using Equation (1).

[0105] Furthermore, based on a preset entropy-rank mapping function, the rank value of a preset low-rank matrix is ​​adjusted according to the entropy of each second information. The entropy-rank mapping function is a piecewise mapping function between the entropy value interval and the rank value of the low-rank adaptation matrix, and it is based on the low-end quantile of the entropy value distribution of historical power multi-business domain tasks. With high-end quantiles The boundary is mapped to a preset minimum rank. With the maximum rank Interval.

[0106] The entropy-rank mapping function is shown below:

[0107] (3)

[0108] in, Let be the rank of the low-rank matrix. The second information entropy is the input. This indicates taking values ​​downwards, which ensures the discreteness of the rank value.

[0109] For example, when bits bits At that time, if the second information entropy of the current input... bits, then the calculated rank value is .

[0110] Thus, the target low-rank matrix after rank adjustment is obtained. When the second information entropy is in the high-order range, the higher-dimensional (i.e., higher-rank) target low-rank matrix is ​​automatically activated to capture complex cross-domain business logic; when the second information entropy is in the low-order range, it switches to the lower-dimensional target low-rank matrix to prevent overfitting. This dynamic rank adjustment mechanism is applied to the weight update process in the task-related parameter layer.

[0111] Furthermore, the query matrix and key matrix corresponding to each training batch are concatenated in parallel with the target low-rank matrix.

[0112] Specifically, for the query matrix Bond matrix The original weight matrix of the association Do not modify directly Instead, it adds a low-rank decomposition form of the target low-rank matrix in the bypass path. ,in, rank value That is, in the query matrix Bond matrix The target low-rank matrix is ​​mounted on the side to achieve parallel connection. During forward propagation, the output is:

[0113] (4)

[0114] in, For the input vector, This is the output vector. During training, the original weights... Keep frozen, update only and .

[0115] Thus, when the second information entropy increases, the rank value is automatically increased through equation (3). To allocate more parameter dimensions to capture the complex characteristics of general power business logic; when the second information entropy decreases, such as when processing highly standardized corpora, the rank value is automatically reduced by equation (3). This reduces parameter redundancy and prevents overfitting.

[0116] For example, when processing standardized switching operation tickets (with low second information entropy, e.g.) When bits are used, the rank value may be set to When processing global business instructions containing multiple conditional branches (with high second information entropy, for example...), When (bits), the rank may be automatically boosted to .

[0117] In practical applications, when the L2 norm of the gradient of the task-related parameter layer is lower than a preset threshold for three consecutive training epochs... (For example When the training phase of the business logic corpus gradient is determined, i.e. the current training phase has converged and training is terminated, and the next training phase of the corpus gradient is entered.

[0118] Specifically, the gradient L2 norm can be calculated using the following formula:

[0119] (5)

[0120] in, This is the parameter set for the task-related parameter layer. This is the loss function.

[0121] In this embodiment, dynamic rank adjustment of the low-rank matrix is ​​performed based on the second information entropy, which achieves precise matching between parameter configuration and task complexity, improves fine-tuning efficiency, and reduces the risk of overfitting.

[0122] In some embodiments, training the activated parameter layer using electrical corpus corresponding to the corpus gradient further includes:

[0123] When the gradient of the corpus is the gradient of the expert decision corpus, and gradient updates need to be performed on the parameter layer, the output state variable of the target model is obtained.

[0124] Determine the Jacobian matrix of the pre-constructed power physics constraint function relative to the output state variable, and obtain the zero-space basis matrix from the Jacobian matrix;

[0125] Obtain the first updated gradient vector of the parameter layer, and project the first updated gradient vector onto the null space corresponding to the null space basis matrix to obtain the first projection vector of the first updated gradient vector.

[0126] The first projection vector is used to perform gradient updates on the parameter layer.

[0127] The power physical constraint functions for system state variables can be pre-constructed as follows: extract a set of general logical decision rules from power system safety monitoring and production processes; assume that this set of general logical decision rules includes... Each business node, node The relationship between multimodal input features and output decision variables is described by a cross-modal joint representation nonlinear equation. Linearizing the relevant image and text features in the current vector space yields the linearized constraint equation. ,in This is the decision state vector output by the model. Simultaneously, it extracts the power grid's safe and stable operation limits, such as voltage upper and lower limits. Line transmission power constraints etc. These nonlinear equations at the current running point... Linearization is performed to obtain the final power physics constraint function. .

[0128] In the specific implementation, when the corpus gradient is the gradient of the expert decision corpus, the basic parameter layer is kept frozen, the task-related parameter layer and the physical feature encoding layer are activated, and a low learning rate is used for fine-tuning.

[0129] In practical applications, during the training phase corresponding to the gradient of the expert decision-making corpus, the learning rate can be configured to be 0.1 times the learning rate of the previous training phase (i.e., the training phase corresponding to the gradient of the business logic corpus). For example, the learning rate of the previous training phase was... Then the learning rate for the training phase corresponding to the gradient of the expert decision-making corpus can be configured as follows: .

[0130] During the training phase of the expert decision-making corpus gradient, and when gradient updates to the parameter layer are required, a gradient projection mechanism needs to be executed. This involves projecting the initial updated gradient vector onto a null space that conforms to electrical physics constraints to ensure that the update of model parameters does not cause the target model's output to deviate from the physical rules. Specifically, the gradient projection mechanism is executed as follows:

[0131] Obtain the output state variables of the target model According to the power physics constraint function Determine the power Compared to Jacobian matrix .

[0132] Jacobian matrix The element is represented as ,right Perform singular value decomposition:

[0133] (6)

[0134] in, It is a diagonal matrix, and the diagonal elements are singular values. .

[0135] Furthermore, from the singular value decomposition In the process, obtain the null basis matrix. :

[0136] From equation (6) The column vectors corresponding to zero singular values ​​are identified by the set of singular vectors whose eigenvalues ​​are zero, forming the null space basis matrix. That is, when (in For numerical tolerance, for example When ), the corresponding The column vector Constructing the zero-space basis matrix A set of bases.

[0137] Furthermore, obtain the initial first update gradient vector of the parameter layer. It can be obtained through the null basis matrix orthogonal projection operator ,right Perform a transformation to... Projecting onto the null space corresponding to N, we obtain the first projection vector. :

[0138] (7)

[0139] in, The component in the normal direction of the electrical physics constraint function is always zero, that is, it satisfies... .

[0140] This ensures that during parameter iteration, the model's output always strictly adheres to physical principles such as Kirchhoff's current law, and the generated multi-scenario business decisions will not violate the physical operating limits of the power grid. For example, the model will not output results that violate power balance, such as a node injecting +50MW of active power, but the total active power outflow from all connected lines being only 30MW.

[0141] During the training phase corresponding to the gradient of the expert decision-making corpus, a preset training cycle can be executed to fine-tune the parameters of the target model. For example, 10 to 20 training cycles can be configured.

[0142] In practical applications, the node injection power algebra output by the target model can also be detected. If the node injection power algebra meets the judgment condition, the gradient projection mechanism described above will be executed; otherwise, the gradient projection mechanism will not be executed.

[0143] Specifically, the judgment condition can be configured as the algebra of the node injection power output by the target model exceeding a preset per-unit deviation threshold. (For example pu), as shown in the following formula:

[0144] (8)

[0145] in, The first output of the model The active power injection value of each node. For system baseline capacity (e.g.) MVA). When the judgment condition of equation (8) is met, the above gradient projection mechanism is executed; otherwise, the above gradient projection mechanism is skipped to improve computational efficiency.

[0146] In this embodiment, when the target model attempts to adjust the parameters related to the physical state quantities, the above projection mechanism can be used to forcibly filter gradient components that may cause the output to violate Kirchhoff's current law or the voltage safety range, thus ensuring the rationality of the target model's output.

[0147] In some embodiments, at the transition point of the training phase corresponding to each corpus gradient, that is, when one training phase ends and the next training phase begins, the attention weight score of the target model to the core power knowledge entities in the power knowledge graph can be calculated, and the need to trigger the rollback mechanism can be determined based on the attention weight score.

[0148] For example, the attention weight scores of the target model for key power knowledge entities in the power knowledge graph are compared with the standard knowledge graph distribution in the power knowledge graph.

[0149] Specifically, the average attention weights of the target model on the validation set for core power knowledge entities (such as circuit breakers, busbars, transformers, etc.) are extracted to form the current attention distribution vector. This vector is compared with the standard distribution vector obtained from the power knowledge graph. Calculate the Euclidean distance to obtain the offset:

[0150] (9)

[0151] If offset Exceeding the preset standard deviation threshold (For example If the learning rate of the current training phase is reduced to 0.5 times its original value, the resampling ratio of the previous phase's basic corpus will be increased according to the sampling weight proportional to the offset.

[0152] Resampling Proportion Meter The formula is:

[0153] (10)

[0154] In the current training phase, by proportion Samples are drawn from the corpus of the previous stage and mixed with samples from the current training stage for training until the attention distribution regresses to an acceptable range (i.e., ...). ).

[0155] In some embodiments, the trained target model has a corresponding initial power scenario, and the method further includes:

[0156] Determine the Fisher information matrix corresponding to the model parameters of the trained target model;

[0157] Based on the activation scores corresponding to the physical feature coding layers, adjust the diagonal elements in the Fischer information matrix;

[0158] The adjusted Fischer information matrix is ​​injected into a pre-constructed loss function, and the loss function is used to update the parameters of the trained target model for the target power scenario, so as to transfer the trained target model from the initial power scenario to the target power scenario.

[0159] The initial power scenario refers to the source power business or grid operation scenario in which the target model has been trained and has stable adaptation capabilities before incremental migration.

[0160] Among them, the target power scenario refers to the new scenario to be migrated that needs to be newly adapted, has not been sufficiently trained before, or has undergone topology or business changes, which is the target of transfer learning.

[0161] In the specific implementation, the diagonal elements of the Fisher information matrix As shown in the following formula:

[0162] (11)

[0163] in, The data distribution for the old task (i.e., the task of training for the initial power scenario). For the target model's first Each model parameter Output probabilities for the model.

[0164] The activation scores corresponding to the physical feature coding layer are Then, the diagonal elements in the Fischer information matrix are adjusted using the following formula:

[0165] (12)

[0166] in, The physical feature weighting coefficient can be set to... .

[0167] Equation (12) allows the model parameters (corresponding to Kirchhoff's laws and power flow equations) that are highly correlated with electrical physics constraints such as power flow equations to be optimized. Larger (larger) values ​​gain higher regularization protection.

[0168] Furthermore, during the parameter update stage of migrating the target model to the target power scenario, the Fischer information matrix with adjusted diagonal elements will be... Inject a loss function that includes the original task loss. With regularization terms:

[0169] (13)

[0170] in, These are the model parameters after training for the old task. The regularization strength coefficient can be configured as follows: .

[0171] The loss function is used to perform parameter updates in order to transfer the target model to the target power scenario.

[0172] In this embodiment, by integrating elastic weights and using a knowledge preservation mechanism for physical constraints, the model parameters (i.e., the diagonal elements of the Fischer information matrix) that are highly relevant to the physical constraints of the power system are better protected, thereby improving the rationality of the target model output after migration to the target power scenario.

[0173] In some embodiments, updating the parameters of the trained target model using the loss function for the target power scenario includes:

[0174] Obtain the second updated gradient vector of the model parameters of the trained target model, and project the second updated gradient vector onto the null space to obtain the second projected vector of the second updated gradient vector;

[0175] If the similarity between the second updated gradient vector and the second projected vector does not meet the preset conditions, the decay coefficient is determined based on the similarity.

[0176] The parameter update amount for the target power scenario is determined based on the attenuation coefficient and the second projection vector, and the parameter update amount is used to update the parameters of the trained target model.

[0177] In the specific implementation, the second update gradient vector is: The transformation will be performed according to equation (7), and the second updated gradient vector will be projected onto the null space to obtain... The second projection vector in null space ;

[0178] Sure and Similarity:

[0179] (14)

[0180] If the similarity is within the preset threshold (For example If the similarity between the second updated gradient vector and the second projected vector does not meet the preset conditions, and the update direction of the model parameters deviates from the physical rules, then the decay coefficient is determined based on the following formula. :

[0181] (15)

[0182] in, The preset attenuation rate can be configured as follows: .

[0183] The parameter update amount for the target power scenario is:

[0184] (16)

[0185] in, For learning rate, These are the model parameters for the target model in the initial power scenario.

[0186] In this embodiment, a physical consistency-based incremental learning mechanism is established to address the catastrophic forgetting problem when the model is migrated to a new power scenario.

[0187] In some embodiments, when migrating the target model to a target power scenario, the new nodes generated by the topology change can be adapted to address the expansion of input dimensions caused by the change in power grid topology. This can be achieved using an initialization method based on knowledge graph semantic mapping.

[0188] Specifically, when a new substation or line is added to the power grid, the electrical attribute vector of the newly connected node is extracted using a pre-built power knowledge graph. The electrical attribute vector... Includes the following dimensions: impedance parameters (positive sequence impedance) Zero-sequence impedance Rated voltage Number of connecting branches Node type (PQ node, PV node, or balancing node), etc. For example, the attribute vector of a certain outgoing line of a newly added 220kV substation might be: .

[0189] By calculating the new node vector With existing node set The Euclidean distances of the vectors in the matrix are used to determine several candidate nodes whose topological features are closest. The distance calculation formula is:

[0190] (17)

[0191] Choose the one with the smallest distance. Nodes (e.g.) Candidate nodes are selected and fused using a weighted average of their inverse distances to become the baseline nodes. Baseline node vector. The calculation formula is:

[0192] (18)

[0193] Extract the input weight matrix corresponding to the reference node in the physical feature encoding layer in step S100. Establish a linear transformation matrix based on the ratio of electrical parameters. The diagonal scaling factor of this matrix is ​​determined by the ratio of the rated capacity of the new node to that of the base node. Let the rated capacity of the new node be... The rated capacity of the reference node is The scaling factor is then:

[0194] (19)

[0195] offset The result is calculated by multiplying the difference in topological connectivity by a preset influence coefficient:

[0196] (20)

[0197] in, The influence coefficient is preferably set to . A vector consisting entirely of 1s. Linear transformation matrix. ,in It is an identity matrix.

[0198] The initial weight matrix of the new node is then:

[0199] (twenty one)

[0200] In this embodiment, the core physical knowledge is effectively preserved during cross-scene migration by integrating the elastic weights of the physical feature encoding layer and initializing the topological nodes based on the knowledge graph.

[0201] In some embodiments, when migrating the target model to a target power scenario, joint training can be performed based on a sample selection strategy that considers information density and physical boundaries in order to achieve lightweight storage and maintain model performance.

[0202] Specifically, all historical training data are scored using the Shannon entropy calculated by equation (1) and the electrophysical constraint function constructed in the aforementioned example;

[0203] The frequency of triggering physical constraint projection when the statistical samples are fine-tuned during the training phase of the expert decision-making corpus gradient. Deviation Amplitude The deviation amplitude is defined as:

[0204] (twenty two)

[0205] The larger the deviation, the more likely the sample is at the power grid operating boundary (such as heavy load condition or fault condition). The Shannon entropy value of the sample is extracted using equation (1). A high entropy value indicates that the sample has a higher degree of logical coupling.

[0206] Define sample importance score for:

[0207] (twenty three)

[0208] Importance scores based on all historical samples Perform statistical analysis to determine the high quantile threshold. .

[0209] For example, the 90th quantile can be set as the threshold, i.e. Only samples with importance scores above a preset high quartile are retained, thus satisfying the requirement. The sample.

[0210] When the model undergoes version iteration or topology transfer, these boundary samples are mixed into the new task data stream at a preset ratio to perform joint training based on knowledge preservation.

[0211] For example, the boundary sample mixing ratio can be set to 20%, that is, in each training batch, 80% of the samples come from new task data and 20% of the samples come from the filtered boundary samples.

[0212] In this embodiment, by fusing information density and physical boundary data, only high-entropy values ​​and boundary condition samples are retained for empirical replay, thus reducing storage overhead while maintaining model performance.

[0213] In some embodiments, a workflow for learning and transferring models for power scenarios is also provided, specifically including the following S1 to S3:

[0214] S1: Partitioning and structured constraints of power sector model parameters.

[0215] By leveraging entity relationships in the power system professional knowledge graph, the sensitivity of model internal parameters to power business logic is determined. Core entities (such as equipment names, defect types, violations, and drawing symbols) and their relationships from power production, safety supervision, and infrastructure operations are extracted. These entities are input into a pre-trained model, and attention weight scores in the attention mechanism are calculated to identify the neuron paths and weight parameters with the highest activation intensity when processing specific power business operations. By statistically ranking the activation scores of all parameters and determining partition thresholds based on activation contribution rate quantiles, model parameters are divided into three independent regions: a basic parameter layer (parameters with activation scores below a preset lower threshold are considered the basic language representation layer and are frozen during fine-tuning); a task-related parameter layer (the core parameter layer processing power professional terminology and business rules, mainly concentrated in the Transformer's multi-head attention projection matrix); and a physical feature encoding layer (a specific set of neurons directly processing image features, video sequences, and text semantics, providing targets for subsequent physical constraint projections). Parameter partitioning is achieved by constructing a Boolean mask based on activation scores. Parameters with normalized activation scores below the lower quantile threshold are frozen. Parameters with scores above the upper quantile threshold and located in the attention projection matrix are labeled as task-related parameter layers. Parameters with scores in the middle range and directly connected to the output of the multimodal encoder are labeled as physical feature encoding layers.

[0216] An input entropy calculation module is set up at the model encoder entry point to calculate the Shannon entropy of the input power data sequence. The magnitude of the entropy value represents the amount of information and logical coupling contained in the task. A piecewise mapping function is established between the entropy value interval and the rank value of the low-rank adaptation (LoRA) matrix. When the task entropy value is in the high-order interval, the higher-dimensional low-rank adaptation matrix is ​​automatically activated; when the task entropy value is in the low-order interval, it switches to the lower-dimensional low-rank adaptation matrix. The rank dimension is calculated using a linear interpolation function, which maps to the preset minimum and maximum rank value intervals using the low-end and high-end quantiles of the historical power business task entropy value distribution as boundaries. The calculation result is rounded down to ensure the discreteness of the rank value. This is only applied to the weight update process in the task-related parameter layer.

[0217] The steady-state power balance equations (nodal power flow equations) and safe and stable operation limits of the power system are extracted. These nonlinear equations are linearized at the current operating point to construct the Jacobian matrix of power physical constraints. The null space basis matrix of this Jacobian matrix is ​​calculated through singular value decomposition. During the model parameter update phase, the calculated original gradient vector is projected into this null space. The direction of the parameter update must be perpendicular to the normal vector of the physical constraint equations. Specifically, when the model attempts to adjust parameters related to physical state quantities, the orthogonal projection mechanism will forcibly filter out gradient components that may cause the output to violate Kirchhoff's current law or the voltage safety range.

[0218] S2: Fine-tuning strategy.

[0219] Before fine-tuning, information entropy needs to be calculated on the entire power-related text corpus to determine the distribution of information density, serving as the input for subsequent phased training. Using the computation module built in phase S1, Shannon entropy calculation is performed on each sequence in the corpus. Specifically, the probability of each token appearing in the sequence is calculated, and the negative of the sum of the products of the probabilities of all tokens and their logarithms is calculated. A higher entropy value indicates greater uncertainty in the information contained in the corpus and higher logical complexity. By statistically analyzing the entropy values ​​of the entire corpus, the quantile method was used to determine the boundaries of each gradient. The corpus was divided into three gradients according to the entropy quantiles: basic corpus (low complexity), mainly covering power industry terminology, equipment name parameters, and national standards for the power industry, with the goal of correcting the basic embedding layer weights of the model and establishing preliminary vector representations of power entities; business logic corpus (medium complexity), mainly covering safety supervision regulations, work operation tickets, inspection records, and construction plan documents, with the goal of enabling the model to fit the power business process and operational sequence logic; and expert decision-making corpus (high complexity), mainly covering accident analysis reports, defect handling instructions, and complex drawing analysis tasks, with the goal of enhancing the model's causal reasoning ability in complex scenarios.

[0220] In the first stage, the input data is the basic corpus. Using the Boolean mask defined in S1, the parameters of the multi-head attention layer and feedforward network layer in the Transformer model are completely frozen, and only the parameters in the physical feature encoding layer are updated. A low-rank adaptation matrix is ​​not used in this stage. By minimizing the KL divergence between the model's output distribution and the probability distribution of the power industry corpus, the model is forced to map power entity features (such as equipment type, defect category, and violation type) to a specific vector space, reducing semantic bias caused by the general corpus. Training in this stage terminates when the decrease in KL divergence on the validation set falls below a preset convergence threshold for several consecutive training epochs.

[0221] In the second stage, the input data is business logic corpus. The basic parameter layer is kept frozen, and the projection parameters of the query and key matrices of the multi-head attention layers in the Transformer architecture are activated. A low-rank fitting matrix is ​​attached to these matrices. During training iterations, the average Shannon entropy of the sequences within each training batch is calculated in real time. An entropy-rank mapping function fitted based on historical training data is established. When the corpus entropy increases, the rank of the low-rank fitting matrix is ​​automatically increased to allocate more parameter dimensions to capture complex business logic features; when processing highly standardized corpora, the rank is decreased to reduce parameter redundancy and prevent overfitting. Training in this stage terminates when the L2 norm of the gradient of the task-related parameter layer falls below a preset threshold for multiple consecutive training epochs.

[0222] In the third stage, the input data is expert decision-making corpus. The full parameter freeze is lifted, and fine-tuning of all parameters is performed using a low learning rate. The steady-state power balance equation of the power grid is introduced to construct constraint functions for the system state variables. Before each gradient update, the Jacobian matrix of the constraint functions with respect to the current model output state is calculated. Singular value decomposition is performed on the Jacobian matrix to identify the set of singular vectors corresponding to eigenvalues ​​of zero. This set constitutes the null basis matrix of the constraint equation. The original gradient vector is transformed using the orthogonal projection operator of the null basis matrix. The projection operator is the product of the null basis matrix and its transpose. The projected gradient vector is the effective update gradient. After projection, the component of the gradient update in the normal direction of the physical constraint equation is always zero. When the algebraic sum of the node injection power output of the model is detected to exceed a preset per-unit deviation threshold, the physical constraint orthogonal projection operation is triggered. This ensures that during parameter iteration, the model output always strictly follows physical principles such as Kirchhoff's current law, and the generated decisions will not violate the physical operating limits of the power grid. This phase involves performing a preset number of training cycles to complete full parameter fine-tuning.

[0223] At each fine-tuning stage transition point, the attention weight score of the model for the core power entity needs to be calculated, and this score is compared with the standard knowledge graph distribution determined in stage S1. If the offset exceeds the preset standard deviation threshold, an automatic rollback mechanism will be triggered: the learning rate of the current stage will be reduced, and the resampling ratio of the basic corpus of the previous stage will be increased according to the sampling weight proportional to the offset, until the attention distribution returns to the allowable range.

[0224] S3: Incremental evolution and knowledge retention mechanism for cross-business scenario feature changes.

[0225] To address the catastrophic forgetting caused by parameter updates during model migration to new scenarios (such as different regional power grids or new business types), an incremental learning mechanism based on physical consistency is established. When calculating the Fisher information matrix of the Elastic Weighted Integration (EWC), the activation intensity of the physical feature encoding layer defined in S1 is introduced as a regulating factor. For parameters identified in S1 that are highly correlated with Kirchhoff's laws and power flow equations, their regularization terms are assigned higher regularization coefficients. The regularization coefficient is determined by the weighted product of the diagonal elements of the Fisher information matrix and the activation intensity of the physical feature encoding layer. During the parameter update phase, the original gradient vector generated by the new task is projected and compared with the null-space basis matrix of the physical constraint Jacobian matrix constructed in S1, and the cosine similarity between the original gradient vector and the null-space projection vector is calculated. If the similarity is lower than a preset threshold (indicating that the update direction deviates from the physical rules), the update step size in that direction is reduced by a decay coefficient. The decay coefficient adopts a negative exponential decay function, the exponent of which is the product of the cosine similarity deviation and the preset decay rate. The final parameter update is equal to the product of the learning rate, the decay coefficient and the null projection gradient.

[0226] To address the expansion of input dimensions caused by changes in power grid topology (such as the addition of new substations or lines), an initialization method based on knowledge graph semantic mapping is designed. Utilizing the power knowledge graph in S1, electrical attribute vectors (including impedance parameters, rated voltage, number of connected branches, etc.) of newly connected nodes are extracted. By calculating the Euclidean distance between the new node vector and each vector in the existing node set, several candidate nodes with the closest topological features are determined, and these are weighted and fused according to the inverse of the distance to serve as the benchmark node. The input weight matrix corresponding to the benchmark node in the S1 physical feature encoding layer is extracted. A linear transformation matrix based on the ratio of electrical parameters is established. The diagonal scaling factor of this matrix is ​​determined by the ratio of the rated capacity of the new node and the benchmark node, and the offset is calculated by multiplying the topology connectivity difference by a preset influence coefficient. The initial weight of the new node is obtained by multiplying the benchmark node weight matrix by the linear transformation matrix.

[0227] To address the expansion of input dimensions caused by new subcategories of power grid services (such as new substations and lines), an initialization method based on knowledge graph semantic mapping is designed. By calculating the Euclidean distance between the new service entity vector and each vector in the existing entity set, several candidate entity nodes with the closest service features are determined.

[0228] To achieve lightweight storage while maintaining model performance, a sample selection strategy based on information density and physical boundaries is designed to replace the full experience replay buffer. The Shannon entropy calculation module in S1 and the physical constraint function in S2 are combined to score all historical training data. The frequency and deviation magnitude of physical constraint projection triggered by samples during the third stage fine-tuning in S2 are statistically analyzed. A larger deviation magnitude indicates that the sample is at the power grid operating boundary (e.g., heavy load condition, fault condition). The Shannon entropy value calculated in S1 is extracted; a high entropy value indicates a higher logical coupling degree for the sample. A high quantile threshold is determined through statistical analysis of the product of physical deviation frequency and Shannon entropy. Only samples with this product index above a preset high quantile are retained. During model version iteration or topology migration, these boundary samples are mixed into the new task data stream at a preset ratio for joint training based on knowledge preservation.

[0229] The above process can be simplified as follows: Figure 3 The workflow diagram shown involves inputting power-related corpora, partitioning parameters, performing progressive training across three training phases (basic, business logic, and expert), applying physical constraints to correct the gradient updates of model parameters, and finally implementing scenario transfer and incremental evolution to output the transferred model.

[0230] The aforementioned transfer learning process offers the following advantages: By using activation intensity-based parameter partitioning and input entropy-driven dynamic rank adaptation, precise matching of parameter configuration and task complexity is achieved, improving fine-tuning efficiency and reducing overfitting risk. Through the null-space orthogonal projection of the physically constrained Jacobian matrix, the gradient update direction is forced to follow the laws of electrical physics, minimizing the risk of output violating power flow balance equations, and ensuring that the generated recognition and analysis results conform to physical rules. A three-stage progressive fine-tuning strategy, training layer by layer from basic terminology to business logic and then to expert decision-making, accelerates convergence and enhances the model's reasoning ability for complex power scenario tasks. Through the integration of elastic weights weighted by the physical feature encoding layer and the initialization of topological nodes based on knowledge graphs, effective preservation of core physical knowledge is achieved during cross-scenario transfer. By fusing information density and physical boundaries in sample selection, only high-entropy values ​​and boundary condition samples are retained for experience replay, reducing storage overhead while maintaining model performance.

[0231] In some embodiments, a transfer learning system for power scenarios is also provided, such as... Figure 4 As shown, the transfer learning system includes: a (semantic-physical) parameter partitioning module, a complexity evaluation and adaptive adaptation module, a physical constraint correction module, and an incremental evolution control module. These modules work together to achieve safe transfer and fine-tuning of the power model.

[0232] The parameter partitioning module is used for structural reorganization during the model initialization phase. This module connects the power knowledge graph with the pre-trained model and, by calculating entity activation weights, divides the model parameters into a basic parameter layer (frozen), a task-related parameter layer (LoRA fine-tuning objects), and a physical feature encoding layer (multimodal processing). The module outputs a structured model with a Boolean mask to the fine-tuning training stage, ensuring that subsequent training only applies to specific business logic or physical feature regions.

[0233] The entropy-driven complexity assessment and adaptive adaptation module is used for data distribution and dynamic network adjustment during training. Located at the data input end, this module calculates Shannon entropy on the power training corpus. On one hand, it categorizes the data into three types based on entropy values: basic, business, and expert, and delivers them to the model in stages. On the other hand, it establishes a mapping between entropy values ​​and network rank, sending rank adjustment instructions to task-related parameter layers in real time to ensure automatic expansion of parameter dimensions when handling high-complexity business logic.

[0234] The physical constraint correction module is used for gradient correction during the parameter update phase. This module connects the power grid's physical constraint library (such as power flow equations) with the model optimizer. During backpropagation, it receives the model's original gradient vector, performs orthogonal projection using the null basis of the physical constraint Jacobian matrix, filters out components that violate physical rules such as Kirchhoff's laws, and then feeds the corrected safety gradient back to the model for parameter updates, forming a closed-loop physical constraint.

[0235] The incremental evolution control module is used for knowledge preservation and topology expansion during scenario migration. When facing a new power grid scenario, this module uses semantic mapping to calculate the initial weights of new nodes and injects them into the model; simultaneously, it combines physical layer activation intensity and high-entropy boundary samples to construct a weighted regularization term and a replay data stream. It mixes the filtered high-risk historical samples into the current training data stream, ensuring that the model adapts to the new scenario without forgetting the old physical rules and core logic.

[0236] The embodiments of this application have the following advantages: by mapping the attention weights corresponding to each model parameter to the activation scores of each model parameter, the activation scores can reflect the sensitivity and correlation of each model parameter to power professional knowledge. Then, each model parameter is classified into different levels of parameter layers according to the activation scores. The first information entropy reflects the information uncertainty of each power corpus and is classified into different levels of corpus gradients. According to the increasing order of the corpus gradient, that is, according to the complexity of the training task from low to high, the matching parameter layers are activated for training. The model parameter configuration is dynamically adjusted to achieve phased and gradual fine-tuning of model parameters. While balancing the training efficiency and expressive power of model parameters as much as possible, it also accelerates the training convergence speed and improves the reasoning ability of the target model for complex power scenario tasks. It is especially suitable for scenarios where the model is transferred to power scenarios. For example, learning transfer for small-scale power data can reduce the occurrence of overfitting and other situations, so that the trained target model has good adaptability to the corresponding power scenarios.

[0237] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0238] Based on the same inventive concept, this application also provides a progressive fine-tuning device for power scenario model migration, which implements the progressive fine-tuning method for power scenario model migration described above. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the progressive fine-tuning device for power scenario model migration provided below can be found in the limitations of the progressive fine-tuning method for power scenario model migration described above, and will not be repeated here.

[0239] In one exemplary embodiment, such as Figure 5 As shown, a progressive fine-tuning device 50 for power scenario model transfer is provided, comprising:

[0240] The acquisition module 502 is used to acquire multiple model parameters of the target model to be trained and determine the attention weight of each model parameter to each power knowledge entity in the pre-constructed power knowledge graph.

[0241] The mapping module 504 is used to map the attention weights corresponding to each of the model parameters to the activation scores of each of the model parameters;

[0242] The first classification module 506 is used to classify each of the model parameters into a corresponding parameter layer according to each of the activation scores; wherein, different parameter layers correspond to different numerical ranges of the activation scores.

[0243] The determination module 508 is used to acquire multiple power-related data and determine the first information entropy of each power-related data.

[0244] The second classification module 510 is used to classify each of the power corpora into a corresponding corpus gradient according to each of the first information entropy; wherein, different corpus gradients correspond to different numerical ranges of the first information entropy.

[0245] The training module 512 is used to activate the parameter layer that matches each of the corpus gradients in the increasing order of the corpus gradients, and train the activated parameter layer using the electrical corpus of the corresponding corpus gradient until the training stopping condition is reached, so as to obtain the trained target model.

[0246] In one embodiment, the parameter layer includes a basic parameter layer, a physical feature encoding layer, and a task-related parameter layer that increase according to the numerical range of the activation scores; the corpus gradient includes a basic corpus gradient, a business logic corpus gradient, and an expert decision corpus gradient that increase according to the numerical range of the first information entropy; and the parameter layer that matches the activation with each of the corpus gradients includes:

[0247] At any of the corpus gradients, the basic parameter layer remains frozen.

[0248] When the gradient of the corpus is the same as the gradient of the base corpus, the physical feature encoding layer is activated;

[0249] When the corpus gradient is the same as the business logic corpus gradient, the task-related parameter layer is activated;

[0250] When the gradient of the corpus is the same as the gradient of the expert decision corpus, the physical feature encoding layer and the task-related parameter layer are activated.

[0251] In one embodiment, the target model structure includes a multi-head attention layer, and the model parameters in the task-related parameter layer include the query matrix and key matrix of the multi-head attention layer. Training the activated parameter layer using electrical corpus corresponding to the corpus gradient includes:

[0252] When the corpus gradient is the business logic corpus gradient, the power corpus of the business logic corpus gradient is divided into power corpus corresponding to multiple training batches;

[0253] The second information entropy of the power corpus corresponding to each training batch is determined, and the rank value of the preset low-rank matrix is ​​adjusted according to each second information entropy to obtain the target low-rank matrix corresponding to each training batch; wherein, the second information entropy is positively correlated with the rank value;

[0254] The query matrix and key matrix corresponding to each training batch are respectively concatenated in parallel with the target low-rank matrix to train the query matrix and key matrix corresponding to each training batch.

[0255] In one embodiment, training the activated parameter layer using electrical corpus corresponding to the corpus gradient further includes:

[0256] When the gradient of the corpus is the gradient of the expert decision corpus, and gradient updates need to be performed on the parameter layer, the output state variable of the target model is obtained.

[0257] Determine the Jacobian matrix of the pre-constructed power physics constraint function relative to the output state variable, and obtain the zero-space basis matrix from the Jacobian matrix;

[0258] Obtain the first updated gradient vector of the parameter layer, and project the first updated gradient vector onto the null space corresponding to the null space basis matrix to obtain the first projection vector of the first updated gradient vector.

[0259] The first projection vector is used to perform gradient updates on the parameter layer.

[0260] In one embodiment, the trained target model has a corresponding initial power scenario, and the apparatus is further configured to:

[0261] Determine the Fisher information matrix corresponding to the model parameters of the trained target model;

[0262] Based on the activation scores corresponding to the physical feature coding layers, adjust the diagonal elements in the Fischer information matrix;

[0263] The adjusted Fischer information matrix is ​​injected into a pre-constructed loss function, and the loss function is used to update the parameters of the trained target model for the target power scenario, so as to transfer the trained target model from the initial power scenario to the target power scenario.

[0264] In one embodiment, updating the parameters of the trained target model using the loss function for the target power scenario includes:

[0265] Obtain the second updated gradient vector of the model parameters of the trained target model, and project the second updated gradient vector onto the null space to obtain the second projected vector of the second updated gradient vector;

[0266] If the similarity between the second updated gradient vector and the second projected vector does not meet the preset conditions, the decay coefficient is determined based on the similarity.

[0267] The parameter update amount for the target power scenario is determined based on the attenuation coefficient and the second projection vector, and the parameter update amount is used to update the parameters of the trained target model.

[0268] Each module in the aforementioned progressive fine-tuning device for power scenario model migration can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0269] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to implementing the aforementioned incremental fine-tuning method for power scenario model migration. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an incremental fine-tuning method for power scenario model migration.

[0270] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0271] In one exemplary embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program that, when executed by the processor, implements the above-described progressive fine-tuning method for power scenario model migration.

[0272] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described progressive fine-tuning method for the migration of the power scenario model.

[0273] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described progressive fine-tuning method for power scenario model migration.

[0274] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0275] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0276] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0277] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A progressive fine-tuning method for power scenario model transfer, characterized in that, The method includes: Obtain multiple model parameters of the target model to be trained, and determine the attention weight of each model parameter to each power knowledge entity in the pre-constructed power knowledge graph; The attention weights corresponding to each of the model parameters are mapped to the activation scores of each of the model parameters; Each of the model parameters is categorized into a corresponding parameter layer according to its activation score; wherein, different parameter layers correspond to different numerical ranges of the activation score. Acquire multiple power-related data sets and determine the first information entropy of each data set. Each of the aforementioned power-related corpora is categorized into a corresponding corpus gradient according to its respective first information entropy; wherein, different corpus gradients correspond to different numerical ranges of the first information entropy; According to the increasing order of the corpus gradient, the parameter layer that matches each corpus gradient is activated, and the activated parameter layer is trained using the electrical corpus of the corresponding corpus gradient until the training stopping condition is reached, so as to obtain the trained target model.

2. The method according to claim 1, characterized in that, The parameter layer includes a basic parameter layer, a physical feature encoding layer, and a task-related parameter layer that increase according to the numerical range of the activation scores. The corpus gradient includes a basic corpus gradient, a business logic corpus gradient, and an expert decision corpus gradient that increase according to the numerical range of the first information entropy. The parameter layer that matches the activation with each of the corpus gradients includes: At any of the corpus gradients, the basic parameter layer remains frozen. When the gradient of the corpus is the same as the gradient of the base corpus, the physical feature encoding layer is activated; When the corpus gradient is the same as the business logic corpus gradient, the task-related parameter layer is activated; When the gradient of the corpus is the same as the gradient of the expert decision corpus, the physical feature encoding layer and the task-related parameter layer are activated.

3. The method according to claim 2, characterized in that, The target model structure includes a multi-head attention layer, and the model parameters in the task-related parameter layer include the query matrix and key matrix of the multi-head attention layer. The step of training the activated parameter layer using electrical corpus corresponding to the corpus gradient includes: When the corpus gradient is the business logic corpus gradient, the power corpus of the business logic corpus gradient is divided into power corpus corresponding to multiple training batches; The second information entropy of the power corpus corresponding to each training batch is determined, and the rank value of the preset low-rank matrix is ​​adjusted according to each second information entropy to obtain the target low-rank matrix corresponding to each training batch; wherein, the second information entropy is positively correlated with the rank value; The query matrix and key matrix corresponding to each training batch are respectively concatenated in parallel with the target low-rank matrix to train the query matrix and key matrix corresponding to each training batch.

4. The method according to claim 2, characterized in that, The step of training the activated parameter layer using the electrical corpus with the corresponding corpus gradient further includes: When the gradient of the corpus is the gradient of the expert decision corpus, and gradient updates need to be performed on the parameter layer, the output state variable of the target model is obtained. Determine the Jacobian matrix of the pre-constructed power physics constraint function relative to the output state variable, and obtain the zero-space basis matrix from the Jacobian matrix; Obtain the first updated gradient vector of the parameter layer, and project the first updated gradient vector onto the null space corresponding to the null space basis matrix to obtain the first projection vector of the first updated gradient vector. The first projection vector is used to perform gradient updates on the parameter layer.

5. The method according to claim 4, characterized in that, The trained target model has a corresponding initial power scenario, and the method further includes: Determine the Fisher information matrix corresponding to the model parameters of the trained target model; Based on the activation scores corresponding to the physical feature coding layers, adjust the diagonal elements in the Fischer information matrix; The adjusted Fischer information matrix is ​​injected into a pre-constructed loss function, and the loss function is used to update the parameters of the trained target model for the target power scenario, so as to transfer the trained target model from the initial power scenario to the target power scenario.

6. The method according to claim 5, characterized in that, The step of updating the parameters of the trained target model using the loss function for the target power scenario includes: Obtain the second updated gradient vector of the model parameters of the trained target model, and project the second updated gradient vector onto the null space to obtain the second projected vector of the second updated gradient vector; If the similarity between the second updated gradient vector and the second projected vector does not meet the preset conditions, the decay coefficient is determined based on the similarity. The parameter update amount for the target power scenario is determined based on the attenuation coefficient and the second projection vector, and the parameter update amount is used to update the parameters of the trained target model.

7. A progressive fine-tuning device for power scenario model transfer, characterized in that, The device includes: The acquisition module is used to acquire multiple model parameters of the target model to be trained and determine the attention weight of each model parameter to each power knowledge entity in the pre-built power knowledge graph. The mapping module is used to map the attention weights corresponding to each of the model parameters to the activation scores of each of the model parameters; The first classification module is used to classify each of the model parameters into a corresponding parameter layer according to each of the activation scores; wherein, different parameter layers correspond to different numerical ranges of the activation scores. A determination module is used to acquire multiple power-related data sets and determine the first information entropy of each power-related data set. The second classification module is used to classify each of the power-related corpora into a corresponding corpus gradient according to each of the first information entropy; wherein, different corpus gradients correspond to different numerical ranges of the first information entropy. The training module is used to activate parameter layers that match each of the corpus gradients in ascending order of the corpus gradients, and train the activated parameter layers using electrical corpus corresponding to the corpus gradients until the training stopping condition is met, so as to obtain the trained target model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images