A method, system, and platform for predicting transmission line relay protection settings based on large language models.

Abstract

This invention discloses a method, system, and platform for predicting transmission line relay protection settings based on a large language model, relating to the technical field of relay protection. By determining whether the input text contains prompts for transmission line relay protection setting prediction, the invention separates input text tasks that do not contain such prompts from those that do. Then, the pre-trained large language model for transmission line relay protection setting prediction is fine-tuned to improve the efficiency of subsequent data analysis. This model possesses both broad language generation capabilities and accurate relay protection setting prediction calculation capabilities, improving the accuracy of transmission line relay protection setting prediction. This can reduce untimely fault clearing or malfunctions caused by improper setting settings and provide relay protection setting adjustment suggestions, answering questions from users. The system and platform proposed in this invention can automatically predict transmission line relay protection settings, significantly improving the automation and intelligence level of relay protection systems and promoting the further development of secondary operation and maintenance of relay protection.

Description

Technical Field

[0001] This invention relates to the technical field of relay protection, and more specifically, to a method, system, and platform for predicting transmission line relay protection settings based on a large language model. Background Technology

[0002] With the development and changes in power systems, the parameters and operating modes of transmission lines also change. The correct operation of transmission line relay protection depends on accurate setting values. Reasonable relay protection setting values ​​ensure that protection devices can adapt to these changes and meet the operational needs of the power system. As a key parameter, the protection setting value directly determines the response speed and accuracy of the relay protection when a fault occurs. Setting the value too high may prevent timely fault clearing, increasing the risk of accidents, while setting it too low may cause malfunctions, affecting the normal operation of the power system. Therefore, how to scientifically and accurately calculate and set the transmission line relay protection settings is an important research topic for ensuring power system safety.

[0003] Traditional relay protection setting calculations primarily rely on dedicated software, with annual verification completed through manual comparison of settings, consuming significant manpower. Currently, relay protection setting calculations are evolving towards automation and integration, improving system efficiency, reliability, and security by integrating various protection and monitoring functions. For example, some advanced relay protection setting calculation platforms have achieved process-guided automatic setting and verification, connecting all stages of setting order generation, including parameter modeling, principle coordination, setting templates, and setting order output, significantly improving the efficiency and accuracy of setting calculations. However, when dealing with complex and ever-changing power system operating environments, the level of intelligence still needs improvement.

[0004] The rapid development of artificial intelligence (AI) models in numerical prediction and their ability to understand and process language have significant advantages in power system fault diagnosis and system optimization. Existing methods for relay protection setting calculation and verification aim to achieve setting comparison and verification during protection operation and maintenance, but they do not address the complexity and tediousness of relay protection setting calculation. With the development of AI technology, especially the advancements in machine learning and deep learning, conditions have been created for optimizing these processes. The difficulty and key point of using AI models to assist in relay protection setting calculation lies in how to prepare and process data and train models to adapt to the relay protection setting calculation task, thereby improving the accuracy and effectiveness of relay protection setting adjustments, given the numerous parameters and complex actual operating conditions of transmission lines. As the requirements for the accuracy of relay protection setting calculations in power systems continue to increase, traditional data processing and model training methods are insufficient to support functional completeness and adapt to high-precision requirements, resulting in limited accuracy and effectiveness of transmission line setting prediction and adjustment results. Summary of the Invention

[0005] To address the issue of low prediction accuracy in current AI-based relay protection setting prediction methods, this invention proposes a transmission line relay protection setting prediction method, system, and platform based on a large language model. Based on an improved input data processing approach, it utilizes a pre-trained and fine-tuned large language model, making it better suited for predicting relay protection settings in complex and diverse transmission lines. Furthermore, it can respond to setting-related questions from staff, improving the accuracy and effectiveness of relay protection setting prediction, reducing manual intervention, operational errors, and labor costs, and enhancing the automation and intelligence level of relay protection.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] Firstly, this application proposes a method for predicting transmission line relay protection settings based on a large language model, comprising the following steps:

[0008] Get the input text;

[0009] Determine whether the input text contains a prediction prompt for the setting of the transmission line relay protection. If so, convert the transmission line parameters following the prompt into input value tokens by encoding the numerical range deviation. Otherwise, perform word segmentation on the input text and convert the input text into input text tokens.

[0010] Input numerical tokens or text tokens into the finely tuned pre-trained large language model for transmission line relay protection setting prediction, and generate output numerical tokens or output text tokens.

[0011] The transmission line relay protection setting prediction setting sheet is generated based on the output numerical tokens, and the transmission line relay protection prediction response dialogue content is generated based on the output text tokens.

[0012] Preferably, the input text includes a question dialogue text, transmission line parameters, and relay protection control words; the transmission line parameters and relay protection control words are converted into tensor forms respectively to obtain transmission line parameter tensors and relay protection control word tensors, where the elements in the transmission line parameter tensor correspond to different transmission line parameter types, and the elements in the relay protection control word tensor correspond to different relay protection control word types; the transmission line relay protection setting prediction prompt is followed by the transmission line parameter tensor.

[0013] By adopting the above technical solution, the transmission line parameters and relay protection control words are converted into tensor form, which can unify the data representation method and facilitate the representation of various complex data structures. This allows different types of data to be processed and analyzed in the subsequent pre-trained large language model for transmission line relay protection setting prediction, thereby improving processing efficiency.

[0014] Preferably, the process of converting the transmission line parameters following the prompt term into input values ​​(Tokens) through numerical range deviation encoding includes:

[0015] Based on the distribution range of transmission line parameter values, discrete intervals are defined and numbered. Each transmission line parameter value is mapped to a discrete interval to obtain the interval deviation value.

[0016] The transmission line parameter values ​​are converted into tensors composed of Tokens, resulting in the input values ​​Tokens.

[0017] The interval number and interval deviation value are combined to form the ID of the input value Tokens, thereby realizing the encoding of the transmission line parameter values ​​into the input value Tokens.

[0018] By adopting the above technical solution, the transmission line parameters are discretized, simplifying the data representation and facilitating subsequent data processing and analysis by large language models. This improves computational efficiency, model performance and compatibility, and enhances the accuracy and effectiveness of transmission line relay protection setting prediction.

[0019] Preferably, the input text is segmented into words, and the input text is converted into a tensor composed of word tokens to form input text tokens.

[0020] Preferably, the fine-tuning process of the pre-trained large language model for transmission line relay protection setting prediction includes:

[0021] The system acquires and preprocesses the transmission line relay protection setting sheet data, transmission line parameters, and relay protection control words to obtain the transmission line relay protection setting sheet data tensor, transmission line parameter tensor, and relay protection control word tensor.

[0022] The transmission line parameter tensor and the relay protection control word tensor are used as input samples, and the transmission line relay protection setting value sheet data tensor is used as input target samples. One input sample and one target sample constitute a training sample.

[0023] The training samples are divided into several mutually exclusive subsets, which are used as the training set to train a pre-trained large language model for transmission line relay protection setting prediction. The model is then validated using cross-validation to obtain the trained pre-trained large language model for transmission line relay protection setting prediction, thus enabling fine-tuning of the pre-trained large language model for transmission line relay protection setting prediction.

[0024] By adopting the above technical solution, based on the pre-trained large language model for transmission line relay protection setting prediction, and targeting the specific task of transmission line relay protection setting prediction, the pre-trained large language model is further trained by combining the single data tensor of transmission line relay protection setting, the tensor of transmission line parameters, and the tensor of relay protection control words, so that it has both extensive language generation capabilities and more accurate relay protection setting prediction calculation capabilities.

[0025] Preferably, during the training of the pre-trained large language model for transmission line relay protection setting prediction, the mean squared error function MSE(X) of the training set and the mean squared error function MSE(Y) of cross-validation are obtained. The mean squared error functions MSE(X) and MSE(Y) are then combined to form a new loss function, Loss, expressed as:

[0026] Loss = σMSE(X) + (1-σ)MSE(Y)

[0027] Where σ represents the weight to be optimized, and the value of σ is between 0 and 1;

[0028] The pre-trained large language model for transmission line relay protection setting prediction is trained using a model optimizer. After several training rounds, when the loss function is minimized or the mean square error function MSE(X) of the training set is minimized, the trained pre-trained large language model for transmission line relay protection setting prediction is obtained, and the model parameters of the pre-trained large language model for transmission line relay protection setting prediction at this time are saved.

[0029] Preferably, the process of generating the transmission line relay protection prediction setting sheet based on the output numerical tokens is as follows: convert the output numerical tokens into a numerical tensor, and multiply it element-wise with the relay protection control word tensor, as shown in the expression:

[0030]

[0031] Where i represents the numerical tensor order, x i Let y represent the i-th constant in the numerical tensor. i x represents i The corresponding transmission line relay protection predicted settings are collected, and the predicted settings of relay protection for all transmission lines are combined to obtain the transmission line relay protection predicted setting list; x max x representsi Corresponding protection setting maximum value or recommended maximum value; C i This represents a relay protection control word tensor, which is a binary tensor.

[0032] The process of generating the relay protection prediction response dialogue content of the transmission line based on the output text tokens is as follows: convert the output text tokens into text format; perform post-processing on the text, including removing redundant information and adjusting the text structure.

[0033] By adopting the above technical solution, the method proposed in this application not only has extensive language generation capabilities but also more accurate relay protection setting prediction calculation capabilities, provides relay protection setting suggestions, answers questions from questioners, and improves the accuracy and effectiveness of transmission line relay protection setting prediction.

[0034] Secondly, this application proposes a transmission line relay protection setting prediction system based on a large language model. The system is used to implement the aforementioned transmission line relay protection setting prediction method based on a large language model, comprising:

[0035] The text acquisition unit is used to acquire the input text;

[0036] The prompt word recognition and processing unit is used to determine whether the input text contains prompt words for transmission line relay protection setting prediction. If so, the transmission line parameters following the prompt word are converted into input value tokens through numerical range deviation encoding; otherwise, the input text is segmented and converted into input text tokens.

[0037] The large language model prediction unit inputs the input numerical tokens or input text tokens into the finely tuned pre-trained large language model for transmission line relay protection setting prediction, and generates output numerical tokens or output text tokens.

[0038] The prediction result generation unit generates a transmission line relay protection setting prediction set based on the output numerical tokens, and generates the transmission line relay protection prediction response dialogue content based on the output text tokens.

[0039] By adopting the above system, the automation and intelligence level of relay protection will be improved, and the secondary operation and maintenance of relay protection will be further developed.

[0040] Preferably, the prompt word recognition processing unit includes: a prompt word recognition module, a range deviation encoding module, and a word segmentation module;

[0041] The prompt term recognition module is used to determine whether the input text contains prompt terms for transmission line relay protection setting prediction; when the input text contains prompt terms for transmission line relay protection setting prediction, the interval deviation encoding module converts the transmission line parameters following the prompt term into input value Tokens through numerical interval deviation encoding; when the input text does not contain prompt terms for transmission line relay protection setting prediction, the word segmentation module performs word segmentation on the input text and converts the input text into input text Tokens.

[0042] Thirdly, this application proposes a platform equipped with the aforementioned transmission line relay protection setting prediction system based on a large language model.

[0043] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0044] This invention proposes a method, system, and platform for predicting transmission line relay protection settings based on a large language model. In this method, the input text is separated into tasks that do not contain such prompts and tasks that do. If no prompts are found, the input text is segmented; if prompts are found, the transmission line parameter values ​​are encoded using interval deviation encoding. Then, the pre-trained large language model for predicting transmission line relay protection settings is fine-tuned to improve the efficiency of subsequent data analysis. This model possesses both broad language generation capabilities and accurate relay protection setting prediction calculation capabilities, improving the accuracy of transmission line relay protection setting prediction. This can reduce untimely fault clearing or malfunctions caused by improper setting settings, and provide relay protection setting adjustment suggestions to answer questions from users.

[0045] The system and platform proposed in this invention can automatically predict the settings of transmission line relay protection, reducing reliance on manual calculation and verification, thereby reducing labor costs and operational errors; it can process and analyze large amounts of power system data, extract useful information from it, support intelligent setting of settings, significantly improve the automation and intelligence level of relay protection systems, and promote the further development of secondary operation and maintenance of relay protection. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the transmission line relay protection setting prediction method based on a large language model proposed in an embodiment of the present invention.

[0047] Figure 2 This diagram illustrates the basic structure of the pre-trained large language model for predicting transmission line relay protection settings used in this embodiment of the invention.

[0048] Figure 3 This diagram illustrates how the root mean square error of the training set changes with the number of training iterations during the fine-tuning process of the large language model proposed in this embodiment of the invention.

[0049] Figure 4 This diagram illustrates how the root mean square error of the large language model proposed in this embodiment of the invention changes with the number of training iterations during the fine-tuning process using cross-validation and without cross-validation.

[0050] Figure 5 This diagram illustrates the structural composition of the transmission line relay protection setting prediction system based on a large language model proposed in this embodiment of the invention. Detailed Implementation

[0051] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0052] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;

[0053] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0054] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0055] Example 1 This example proposes a method for predicting transmission line relay protection settings based on a large language model. A schematic diagram of the implementation process of this method can be found here. Figure 1 ,like Figure 1 As shown, the method includes the following steps:

[0056] S1: Get the input text;

[0057] S2: Determine whether the input text contains a prediction prompt for the setting of the transmission line relay protection. If so, convert the transmission line parameters following the prompt into input value Tokens by encoding the numerical range deviation. Otherwise, perform word segmentation on the input text and convert the input text into input text Tokens.

[0058] S3: Input the input numerical tokens or input text tokens into the finely tuned pre-trained large language model for transmission line relay protection setting prediction, and generate output numerical tokens or output text tokens.

[0059] S4: Generate a transmission line relay protection setting prediction setting sheet based on the output numerical tokens, and generate the transmission line relay protection prediction response dialogue content based on the output text tokens.

[0060] In this embodiment, the prompt terms for transmission line relay protection setting prediction refer to keywords or phrases that can directly or indirectly indicate or imply the topic of "transmission line relay protection setting prediction." These prompt terms may include "transmission line," "relay protection," "setting," "prediction," etc., but may also include more specific terms or word combinations. In natural language processing, "tokens" are the basic units of text analysis, representing a character, word, phrase, or sentence, and are a format or representation more suitable for numerical processing of large language models.

[0061] In the method proposed in this embodiment, by determining whether the input text contains prompts for transmission line relay protection setting prediction, the task of input text tasks without prompts for transmission line relay protection setting prediction is separated from the task of input text tasks containing prompts for transmission line relay protection setting prediction. Specifically: if no prompts are contained, the input text is segmented; if prompts are contained, the transmission line parameter values ​​are encoded using interval deviation encoding. The two tasks are separated, and the input text tokens are used to perform language generation. Based on the given input text tokens, a new natural language text is generated using a pre-trained and fine-tuned large language model for transmission line relay protection setting prediction. This includes tasks such as completing sentences, generating paragraphs, answering questions, or performing other similar tasks, generating question-and-answer dialogue content, thus improving the level of intelligence; input text tokens are used to perform setting value calculation functions, utilizing a pre-trained and fine-tuned large language model for transmission line relay protection setting value prediction to perform setting value prediction. The idea of ​​task separation improves data analysis efficiency, enabling the large language model to have both extensive language generation capabilities and accurate relay protection setting value prediction calculation capabilities, thereby improving the accuracy of transmission line relay protection setting value prediction. This can reduce the untimely fault clearing or malfunctions caused by improper setting value settings, and provide relay protection setting value adjustment suggestions, answer questions from questioners, and improve the level of intelligence.

[0062] Example 2

[0063] The input text includes question dialogue text, transmission line parameters, and relay protection control words. In this embodiment, to unify the data representation method and facilitate the representation of various complex data structures, making it easier for different types of data to be processed and analyzed in the subsequent pre-trained large language model for transmission line relay protection setting prediction, the transmission line parameters and relay protection control words are converted into tensor forms respectively, resulting in transmission line parameter tensors and relay protection control word tensors.

[0064] In this context, the elements in the transmission line parameter tensor correspond to different transmission line parameter types, and the elements in the relay protection control word tensor correspond to different relay protection control word types. The transmission line parameter tensor follows the prompt term for transmission line relay protection setting prediction.

[0065] Transmission line parameter types include resistive transmission lines and reactive transmission lines, etc. Relay protection control words are a set of binary codes used to control various protection functions and operating modes of relay protection devices. By modifying the control words, the functions of relay protection devices can be enabled, disabled, or adjusted to adapt to different power system protection needs. Relay protection control word types include relay protection control words, monitoring control words, and regulating control words.

[0066] In this embodiment, the process of converting the transmission line parameters following the prompt term into input values ​​(Tokens) through numerical range deviation encoding includes:

[0067] Based on the distribution range of transmission line parameter values, discrete intervals are defined and numbered. Each transmission line parameter value is mapped to a discrete interval to obtain the interval deviation value.

[0068] The transmission line parameter values ​​are converted into tensors composed of Tokens, resulting in the input values ​​Tokens.

[0069] The interval number and interval deviation value are combined to form the ID of the input value Tokens, thereby realizing the encoding of the transmission line parameter values ​​into the input value Tokens.

[0070] Assuming a set of transmission line parameter numerical data, we first observe the distribution range of the data and reasonably define discrete intervals. In this embodiment, these intervals are evenly divided according to the maximum and minimum values ​​of the data. Based on the number of intervals and the width of each interval, the discrete intervals are obtained. Then, based on the specific values, the values ​​are mapped to different intervals, and the interval deviation value is calculated. This can be the deviation between the value within the interval and the midpoint of the interval, or the deviation between the value within the interval and the mean of the interval. The deviation value of each value from the midpoint of its interval is calculated, which simplifies the data representation and facilitates subsequent data processing and analysis by large language models. This improves computational efficiency, model performance and compatibility, and enhances the accuracy and effectiveness of transmission line relay protection setting prediction.

[0071] In this embodiment, the input text is segmented into words, and the input text is converted into a tensor composed of word tokens to form input text tokens.

[0072] In this embodiment, the pre-trained large language model for transmission line relay protection setting prediction is the LLaMa2-7B large language model, such as... Figure 2As shown, after initial word segmentation and interval bias encoding, the language enters the LLaMa2-7B large language model. The LLaMa2-7B large language model includes an embedding layer, several Transformer encoder layers, a root mean square normalization layer, a linear layer, and a softmax layer. These layers are sequentially connected. In this embodiment, there are 32 Transformer encoder layers. The output of the linear layer, in addition to being connected to the softmax layer, can also be directly output. Each Transformer encoder layer includes a multi-head attention mechanism unit and a feedforward network unit. The specific structural connections of the multi-head attention mechanism unit and the feedforward network unit can be found in [reference needed]. Figure 2 .

[0073] In this embodiment, the fine-tuning process of the pre-trained large language model for transmission line relay protection setting prediction includes:

[0074] The process involves acquiring and preprocessing the transmission line relay protection setting sheet data, transmission line parameters, and relay protection control words to obtain tensors for the transmission line relay protection setting sheet data, transmission line parameters, and relay protection control words. This step transforms the transmission line relay protection setting sheet data, transmission line parameters, and relay protection control words into a format suitable for input to the LLaMa2-7B large language model.

[0075] The transmission line parameter tensor and the relay protection control word tensor are used as input samples, and the transmission line relay protection setting value sheet data tensor is used as the input target sample. One input sample and one target sample constitute a training sample. In this step, each element corresponds to a different setting value type. Several training samples form a training sample library.

[0076] The training samples are divided into several mutually exclusive subsets, which are used as the training set to train a pre-trained large language model for transmission line relay protection setting prediction. This pre-trained model is then validated using cross-validation to obtain a fine-tuned version. In this embodiment, the training samples can be divided into k mutually exclusive subsets. Each time, the union of k-1 subsets is used as the training set, and the remaining subset is used as the validation set. This process is repeated k times, and the mean of the k test results is returned. This method utilizes the training samples more comprehensively. In the process of training the pre-trained large language model for transmission line relay protection setting prediction, the mean squared error function (MSE(X)) of the training set and the mean squared error function (MSE(Y)) of the cross-validation are obtained. The MSE(X) and MSE(Y) are combined to form a new loss function, Loss, expressed as:

[0077] Loss = σMSE(X) + (1-σ)MSE(Y)

[0078] Where σ represents the weight to be optimized, and the value of σ is between 0 and 1;

[0079] The pre-trained large language model for transmission line relay protection setting prediction is trained using a model optimizer. After several training rounds, when the loss function is minimized or the mean square error function (MSE(X)) of the training set is minimized, the pre-trained large language model for transmission line relay protection setting prediction is obtained. The model parameters of the pre-trained large language model for transmission line relay protection setting prediction at this time are saved. In this embodiment, the model parameters include weight parameters, such as linear layer parameters, attention layer weights, and embedding layer weights; bias parameters, such as linear layer bias and normalization layer bias; and special parameters, such as position encoding parameters and layer normalization scale parameters.

[0080] The following example illustrates this. In a certain regional power grid, 252 sets of relay protection setting notification sheets for 110kV to 220kV lines were collected and compiled. The data includes line parameters, primary relay protection values, protection parameter settings, protection control words, and function soft switches.

[0081] Transmission line parameters, primary values ​​of relay protection, and protection settings are extracted as data samples and preprocessed into a format suitable for input to a pre-trained large language model for transmission line relay protection setting prediction. Each data sample contains an input sample and a target sample. Taking a 220kV line protection setting sheet as an example, the input line parameters are shown in Table 1, and the transformed input format is "[Value Predict] Line Parameters:", followed by a tensor concatenated with the parameter values ​​from Table 1.

[0082] Example: Input: "You are a senior professional relay protection technician. Based on the transmission line parameters I gave you, please calculate the relay protection settings for this line. [Value Predict] Line parameters:" followed by the line parameters, as shown in Table 1.

[0083] Table 1

[0084]

[0085] The transmission line parameter format output by the pre-trained transmission line relay protection setting prediction large language model is shown in Table 2. The converted target format is "Prediction successful! The predicted relay protection setting of this line is:", followed by a tensor concatenated with the parameter values ​​in Table 2.

[0086] Table 2

[0087]

[0088]

[0089] In this embodiment, the model is fine-tuned using the Trainer library from the Transformer library within the PyTorch framework. The learning rate is set to 2 × 10⁻⁶. -5 The batch size for each training session was set to 8, and the number of training epochs was set to 10. The model optimizer chosen was AdamW (Adam with Weight Decay), which, compared to Adam, incorporates a weight decay mechanism to help mitigate overfitting and performs well in training large-scale parameter models.

[0090] Figure 3 This diagram illustrates how the root mean square error of the training set changes with the number of training iterations during the fine-tuning process of the large language model proposed in this embodiment of the invention. Figure 3 It can be seen that during the fine-tuning training of the large language model, the root mean square error of the training set changes with the number of training iterations (epochs). The root mean square error of the training set tends to stabilize in the 9th and 10th epochs. Figure 4 This invention presents a schematic diagram illustrating the variation of the root mean square error (RMSE) of the large language model during fine-tuning with and without cross-validation, over Epochs. Typically, model parameters that minimize either the loss function or the training set RMSE are saved. However, from... Figure 3 It can be observed that when the epoch is 8, the root mean square error of cross-validation decreases to the lowest level and then increases. The training set loss function, on the other hand, decreases steadily.

[0091] As mentioned earlier, the goal of this experiment is to predict protection setpoints that the model has not learned before; therefore, the focus should be on the model's cross-validation evaluation performance. Cross-validation performed best at Epoch 8, with the overall training set mean squared error decreasing to a low level and tending to stabilize. Therefore, retaining the model parameters at Epoch=8 allows for optimal model performance on the test set. Model training took 34 minutes, and single-sample prediction took 0.1 seconds.

[0092] The process of generating a transmission line relay protection prediction setting sheet based on the output numerical tokens is as follows: The output numerical tokens are converted into a numerical tensor, and then multiplied element-wise with the relay protection control word tensor. The expression is:

[0093]

[0094] Where i represents the numerical tensor order, x i Let y represent the i-th constant in the numerical tensor.i x represents i The corresponding transmission line relay protection predicted settings are collected, and the predicted settings of relay protection for all transmission lines are combined to obtain the transmission line relay protection predicted setting list; x max x represents i Corresponding protection setting maximum value or recommended maximum value; C i This represents a relay protection control word tensor, which is a binary tensor.

[0095] The process of generating the relay protection prediction response dialogue content of the transmission line based on the output text tokens is as follows: convert the output text tokens into text format; perform post-processing on the text, including removing redundant information and adjusting the text structure.

[0096] Example 3

[0097] like Figure 5 As shown, this embodiment proposes a transmission line relay protection setting prediction system based on a large language model. The system is used to implement the aforementioned transmission line relay protection setting prediction method based on a large language model, including:

[0098] The text acquisition unit is used to acquire the input text;

[0099] The prompt word recognition and processing unit is used to determine whether the input text contains prompt words for transmission line relay protection setting prediction. If so, the transmission line parameters following the prompt word are converted into input value tokens through numerical range deviation encoding; otherwise, the input text is segmented and converted into input text tokens.

[0100] The large language model prediction unit inputs the input numerical tokens or input text tokens into the finely tuned pre-trained large language model for transmission line relay protection setting prediction, and generates output numerical tokens or output text tokens.

[0101] The prediction result generation unit generates a transmission line relay protection setting prediction set based on the output numerical tokens, and generates the transmission line relay protection prediction response dialogue content based on the output text tokens.

[0102] By adopting the above system, the automation and intelligence level of relay protection is improved, and the further development of secondary operation and maintenance of relay protection is promoted. In this embodiment, the prompt term recognition and processing unit includes: a prompt term recognition module, an interval deviation encoding module, and a word segmentation module; the prompt term recognition module is used to determine whether the input text contains a prompt term for transmission line relay protection setting prediction; when the input text contains a prompt term for transmission line relay protection setting prediction, the interval deviation encoding module converts the transmission line parameters following the prompt term into input value Tokens through numerical interval deviation encoding; when the input text does not contain a prompt term for transmission line relay protection setting prediction, the word segmentation module performs word segmentation processing on the input text and converts the input text into input text Tokens.

[0103] When implementing the transmission line relay protection setting prediction method based on a large language model using the system proposed in this embodiment, the input text includes a question dialogue, transmission line parameters, and relay protection control words. The transmission line parameters and relay protection control words are converted into tensor forms, resulting in transmission line parameter tensors and relay protection control word tensors, respectively. The elements in the transmission line parameter tensor correspond to different transmission line parameter types, and the elements in the relay protection control word tensor correspond to different relay protection control word types. The transmission line relay protection setting prediction prompt is followed by the transmission line parameter tensor.

[0104] When implementing the transmission line relay protection setting prediction method based on a large language model using the system proposed in this embodiment, the transmission line parameters following the prompt terms are converted into input values ​​(Tokens) using an interval deviation coding module. The process includes:

[0105] Based on the distribution range of transmission line parameter values, discrete intervals are defined and numbered. Each transmission line parameter value is mapped to a discrete interval to obtain the interval deviation value.

[0106] The transmission line parameter values ​​are converted into tensors composed of Tokens, resulting in the input values ​​Tokens.

[0107] The interval number and interval deviation value are combined to form the ID of the input value Tokens, thereby realizing the encoding of the transmission line parameter values ​​into the input value Tokens.

[0108] When implementing the transmission line relay protection setting prediction method based on a large language model using the system proposed in this embodiment, the Tokennizer word segmenter is used as a word segmentation module. The input text is segmented into words, and the input text is converted into a tensor composed of word tokens to form input text tokens.

[0109] In this embodiment, the pre-trained large language model for transmission line relay protection setting prediction is the LLaMa2-7B large language model, such as... Figure 2 As shown, after initial word segmentation and interval bias encoding, the language enters the LLaMa2-7B large language model. The LLaMa2-7B large language model includes an embedding layer, several Transformer encoder layers, a root mean square normalization layer, a linear layer, and a softmax layer. These layers are sequentially connected. In this embodiment, there are 32 Transformer encoder layers. The output of the linear layer, in addition to being connected to the softmax layer, can also be directly output. Each Transformer encoder layer includes a multi-head attention mechanism unit and a feedforward network unit. The specific structural connections of the multi-head attention mechanism unit and the feedforward network unit can be found in [reference needed]. Figure 2 .

[0110] The fine-tuning process of the pre-trained large language model for transmission line relay protection setting prediction includes:

[0111] The process involves acquiring and preprocessing the transmission line relay protection setting sheet data, transmission line parameters, and relay protection control words to obtain tensors for the transmission line relay protection setting sheet data, transmission line parameters, and relay protection control words. This step transforms the transmission line relay protection setting sheet data, transmission line parameters, and relay protection control words into a format suitable for input to the LLaMa2-7B large language model.

[0112] The transmission line parameter tensor and the relay protection control word tensor are used as input samples, and the transmission line relay protection setting value sheet data tensor is used as the input target sample. One input sample and one target sample constitute a training sample. In this step, each element corresponds to a different setting value type. Several training samples form a training sample library.

[0113] The training samples are divided into several mutually exclusive subsets, which are used as the training set to train a pre-trained large language model for transmission line relay protection setting prediction. This pre-trained model is then validated using cross-validation to obtain a fine-tuned version. In this embodiment, the training samples can be divided into k mutually exclusive subsets. Each time, the union of k-1 subsets is used as the training set, and the remaining subset is used as the validation set. This process is repeated k times, and the mean of the k test results is returned. This method utilizes the training samples more comprehensively. In the process of training the pre-trained large language model for transmission line relay protection setting prediction, the mean squared error function (MSE(X)) of the training set and the mean squared error function (MSE(Y)) of the cross-validation are obtained. The MSE(X) and MSE(Y) are combined to form a new loss function, Loss, expressed as:

[0114] Loss = σMSE(X) + (1-σ)MSE(Y)

[0115] Where σ represents the weight to be optimized, and the value of σ is between 0 and 1;

[0116] A model optimizer is used to train a pre-trained large-scale language model for transmission line relay protection setting prediction. After several training rounds, the pre-trained large-scale language model for transmission line relay protection setting prediction is obtained when the loss function is minimized or the mean square error function (MSE(X)) of the training set is minimized. The model parameters of the pre-trained large-scale language model for transmission line relay protection setting prediction at this point are saved.

[0117] Example 4

[0118] This embodiment proposes a platform that integrates the large language model-based transmission line relay protection setting prediction system proposed in Embodiment 3. The platform is based on the large language model-based transmission line relay protection setting prediction system. The system is built upon this platform, which provides the necessary operating environment and infrastructure, enabling the system to operate normally and perform its functions. The platform includes: a prompt word recognition processing unit that determines whether the input text contains prompt words for transmission line relay protection setting prediction; if so, it converts the transmission line parameters following the prompt word into input value tokens through numerical range deviation encoding; otherwise, it performs word segmentation on the input text, converting it into input text tokens. A large language model prediction unit inputs the input value tokens or input text tokens into a finely tuned pre-trained large language model for transmission line relay protection setting prediction, generating output value tokens or output text tokens. A prediction result generation unit generates a transmission line relay protection setting prediction setting sheet based on the output value tokens and generates the transmission line relay protection prediction response dialogue content based on the output text tokens.

[0119] The platform proposed in this embodiment is equipped with the aforementioned system, which can automatically predict the settings of transmission line relay protection, reduce reliance on manual calculation and verification, thereby reducing labor costs and operational errors; it can process and analyze large amounts of power system data, extract useful information from it, support intelligent setting of settings, significantly improve the automation and intelligence level of relay protection systems, and promote the further development of secondary operation and maintenance of relay protection.

[0120] The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. Clearly, the above embodiments of the present invention are merely examples to clearly illustrate the invention and are not intended to limit the implementation of the invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of this invention.

Claims

1. A method for predicting transmission line relay protection settings based on a large language model, characterized in that, Includes the following steps: Get the input text; The input text includes a question dialogue text, transmission line parameters, and relay protection control words; the transmission line parameters and relay protection control words are converted into tensor forms respectively to obtain transmission line parameter tensors and relay protection control word tensors. The elements in the transmission line parameter tensor correspond to different transmission line parameter types, and the elements in the relay protection control word tensor correspond to different relay protection control word types. The transmission line relay protection setting prediction prompt entry is followed by the transmission line parameter tensor; Determine whether the input text contains a prediction prompt for the setting of the transmission line relay protection. If so, convert the transmission line parameters following the prompt into input value tokens by encoding the numerical range deviation. Otherwise, perform word segmentation on the input text and convert the input text into input text tokens. Input numerical tokens or text tokens into the finely tuned pre-trained large language model for transmission line relay protection setting prediction, and generate output numerical tokens or output text tokens. The transmission line relay protection setting prediction setting sheet is generated based on the output numerical tokens, and the transmission line relay protection prediction response dialogue content is generated based on the output text tokens. The process of generating a transmission line relay protection prediction setting sheet based on the output numerical tokens is as follows: The output numerical tokens are converted into a numerical tensor, and then multiplied element-wise with the relay protection control word tensor. The expression is: in, i Indicates the order of numerical tensors. x i Represents the first in a numerical tensor i A fixed value, y i express x i The corresponding transmission line relay protection prediction settings are collected, and the transmission line relay protection prediction settings are combined to obtain the transmission line relay protection prediction setting list. x max express x i The corresponding protection setting maximum value or recommended maximum value; C i This represents a relay protection control word tensor, which is a binary tensor. The process of generating the relay protection prediction response dialogue content of the transmission line based on the output text tokens is as follows: convert the output text tokens into text format; perform post-processing on the text, including removing redundant information and adjusting the text structure.

2. The method for predicting transmission line relay protection settings based on a large language model according to claim 1, characterized in that, The process of converting the transmission line parameters following the prompt term into input values ​​(Tokens) through numerical range deviation encoding includes: Based on the distribution range of transmission line parameter values, discrete intervals are defined and numbered. Each transmission line parameter value is mapped to a discrete interval to obtain the interval deviation value. The transmission line parameter values ​​are converted into tensors composed of Tokens, resulting in the input values ​​Tokens. The interval number and interval deviation value are combined to form the ID of the input value Tokens, thereby realizing the encoding of the transmission line parameter values ​​into the input value Tokens.

3. The method for predicting transmission line relay protection settings based on a large language model according to claim 1, characterized in that, The input text is segmented into words, and then converted into a tensor composed of word tokens, forming the input text tokens.

4. The method for predicting transmission line relay protection settings based on a large language model according to claim 1, characterized in that, The fine-tuning process of the pre-trained large language model for transmission line relay protection setting prediction includes: The system acquires and preprocesses the transmission line relay protection setting sheet data, transmission line parameters, and relay protection control words to obtain the transmission line relay protection setting sheet data tensor, transmission line parameter tensor, and relay protection control word tensor. The transmission line parameter tensor and the relay protection control word tensor are used as input samples, and the transmission line relay protection setting value sheet data tensor is used as input target samples. One input sample and one target sample constitute a training sample. The training samples are divided into several mutually exclusive subsets, which are used as the training set to train a pre-trained large language model for transmission line relay protection setting prediction. The model is then validated using cross-validation to obtain the trained pre-trained large language model for transmission line relay protection setting prediction, thus enabling fine-tuning of the pre-trained large language model for transmission line relay protection setting prediction.

5. The method for predicting transmission line relay protection settings based on a large language model according to claim 4, characterized in that, During the training of the large language model for predicting transmission line relay protection settings, the mean square error function of the training set is obtained. MSE (X) and the mean squared error function of cross-validation MSE (Y), the mean square error function MSE (X) and mean square error function MSE (Y) combined into a new loss function Loss The expression is: in, σ This represents the weights to be optimized. σ The value is between 0 and 1; The model optimizer is used to train a pre-trained large language model for transmission line relay protection setting prediction. After several training rounds, the model with the minimum loss function or the minimum mean square error function of the training set is selected. MSE When (X) is at its minimum, the pre-trained large language model for predicting the setting value of the transmission line relay protection is obtained, and the model parameters of the pre-trained large language model for predicting the setting value of the transmission line relay protection at this time are saved.

6. A transmission line relay protection setting prediction system based on a large language model, characterized in that, The system is used to implement the transmission line relay protection setting prediction method based on a large language model as described in any one of claims 1 to 5, including: The text acquisition unit is used to acquire the input text; The prompt word recognition and processing unit is used to determine whether the input text contains prompt words for transmission line relay protection setting prediction. If so, the transmission line parameters following the prompt word are converted into input value tokens through numerical range deviation encoding; otherwise, the input text is segmented and converted into input text tokens. The large language model prediction unit inputs the input numerical tokens or input text tokens into the finely tuned pre-trained large language model for transmission line relay protection setting prediction, and generates output numerical tokens or output text tokens. The prediction result generation unit generates a transmission line relay protection setting prediction set based on the output numerical tokens, and generates the transmission line relay protection prediction response dialogue content based on the output text tokens.

7. The transmission line relay protection setting prediction system based on a large language model according to claim 6, characterized in that, The prompt word recognition and processing unit includes: a prompt word recognition module, an interval deviation encoding module, and a word segmentation module; The prompt term recognition module is used to determine whether the input text contains prompt terms for transmission line relay protection setting prediction; when the input text contains prompt terms for transmission line relay protection setting prediction, the interval deviation encoding module converts the transmission line parameters following the prompt term into input value Tokens through numerical interval deviation encoding; when the input text does not contain prompt terms for transmission line relay protection setting prediction, the word segmentation module performs word segmentation on the input text and converts the input text into input text Tokens.

8. A platform, characterized in that, The platform is equipped with the transmission line relay protection setting prediction system based on a large language model as described in claim 6.

Images