Data processing method and device and program product

Abstract

The invention discloses a data processing method and device and a program product. The method comprises the steps that inquiry information associated with the power system is acquired, and the inquiry information is used for inquiring the operation state of the power system; in the electric power system, operation data associated with the inquiry information is determined, and the operation data is used for representing the operation state of the electric power system; a question and answer model is called to analyze the operation data and the inquiry information to obtain a reply result matched with the inquiry information, and the question and answer model is obtained by training historical feedback information obtained in at least one feedback information channel; the historical feedback information is used for representing a feedback result for feeding back the accuracy degree of the historical reply result, and the historical reply result is matched with the historical inquiry information. The technical problem of low answering accuracy of the question and answer model is solved.

Description

Technical Field

[0001] The present invention relates to the field of power systems, and in particular to a data processing method, device and program product. Background Art

[0002] At present, the application of question-answering models in power systems relies on a static knowledge base and lacks the ability of dynamic learning and self-optimization. It is unable to respond and adapt to changes in power system demand in a timely manner, resulting in a technical problem of low accuracy of the question-answering model's responses. Summary of the invention

[0003] The embodiments of the present invention provide a data processing method, device and program product to at least solve the technical problem of low response accuracy of the question-answering model.

[0004] According to one aspect of an embodiment of the present invention, a data processing method is provided, which may include: obtaining inquiry information associated with an electric power system, wherein the inquiry information is used to inquire about the operating status of the electric power system; in the electric power system, determining operating data associated with the inquiry information, wherein the operating data is used to characterize the operating status of the electric power system; calling a question-and-answer model to analyze the operating data and the inquiry information to obtain a reply result that matches the inquiry information, wherein the question-and-answer model is trained using historical feedback information obtained from at least one feedback information channel, the historical feedback information is used to characterize a feedback result that provides feedback on the accuracy of a historical reply result, and the historical reply result matches the historical inquiry information.

[0005] Optionally, the method may also include: obtaining historical inquiry information, historical reply results, and historical feedback information of historical reply results in multiple feedback information channels; using the historical inquiry information, historical reply results, and historical feedback information to train a sub-question and answer model to obtain a question and answer model.

[0006] Optionally, historical inquiry information, historical reply results, and historical feedback information are used to train a sub-question and answer model to obtain a question and answer model, including: preprocessing the multiple historical inquiry information and multiple historical reply results obtained respectively to obtain multiple preprocessed historical inquiry information and multiple preprocessed historical reply results; using the preprocessed multiple historical inquiry information and the preprocessed multiple historical reply results to construct a first training data set and a validation set, and using multiple historical inquiry information, multiple historical reply results and historical feedback information to construct a second training data set, wherein the data in the first training data set is different from the data in the validation set; using the first training data set, the validation set, and the second training data set to train the sub-question and answer model to obtain a question and answer model.

[0007] Optionally, the sub-question and answer model is trained using the first training data set, the validation set, and the second training data set to obtain a question and answer model, including: using the first training data set to train the sub-question and answer model to obtain a first question and answer model; obtaining a third training data set whose degree of correlation with the power system is greater than a correlation degree threshold, wherein the third training data set is different from the first training data set and contains relevant inquiry information and relevant response results of the relevant inquiry information; using the second training data set and the third training data set to train the first question and answer model to obtain a question and answer model.

[0008] Optionally, the first question-answering model is trained using a validation set, a second training data set, and a third training data set to obtain a question-answering model, including: constructing a loss function, wherein the loss function is associated with historical response results in the third training data set; using the loss function and the third training data set to train the first question-answering model to obtain a second question-answering model; using the validation set to adjust model parameters of the trained second question-answering model to obtain an adjusted second question-answering model; and using the second training data set to train the adjusted second question-answering model to obtain a question-answering model.

[0009] Optionally, the adjusted second question-answering model is trained using the second training data set to obtain a question-answering model, including: converting the second training data set to obtain a converted data set; determining an action data set corresponding to the converted data set, wherein the action data set includes multiple action data, and the action data are used to characterize the adjustment actions of the second question-answering model on the historical response results; calling the second question-answering model to convert the second training data set and the action data set to obtain evaluation data corresponding to multiple action data, wherein the evaluation data is used to characterize the impact of the action data on the historical response results; and using the evaluation data to adjust the second question-answering model to obtain a question-answering model.

[0010] Optionally, obtaining historical inquiry information and historical reply results includes: obtaining multiple historical inquiry information and multiple historical reply results from a database corresponding to the power system.

[0011] Optionally, obtaining historical feedback information of historical reply results in multiple feedback information channels includes: in response to a client triggering an evaluation instruction, obtaining historical feedback information in the evaluation instruction.

[0012] According to another aspect of an embodiment of the present invention, a data processing device is also provided, which may include: an acquisition unit, used to acquire query information associated with an electric power system, wherein the query information is used to inquire about the operating status of the electric power system; a determination unit, used to determine operating data associated with the query information in the electric power system, wherein the operating data is used to characterize the operating status of the electric power system; a processing unit, used to call a question-and-answer model to analyze the operating data and the query information to obtain a reply result that matches the query information, wherein the question-and-answer model is trained using historical feedback information obtained from at least one feedback information channel, the historical feedback information is used to characterize a feedback result that provides feedback on the accuracy of a historical reply result, and the historical reply result matches the historical query information.

[0013] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the data processing method of an embodiment of the present invention.

[0014] According to another aspect of an embodiment of the present invention, a processor is further provided. The processor is used to run a program, wherein the program executes the data processing method of an embodiment of the present invention when run by the processor.

[0015] According to another aspect of an embodiment of the present invention, a program product is further provided. The program product includes computer instructions. When the computer instructions are executed by a processor, the data processing method of the embodiment of the present invention is implemented.

[0016] According to another aspect of the embodiments of the present invention, a vehicle is further provided. The vehicle can be used to execute the data processing method of the embodiments of the present invention.

[0017] In an embodiment of the present invention, query information associated with the power system is obtained, wherein the query information is used to inquire about the operating status of the power system; in the power system, operating data associated with the query information is determined, wherein the operating data is used to characterize the operating status of the power system; a question-answering model is called to analyze the operating data and the query information to obtain a reply result that matches the query information, wherein the question-answering model is trained using historical feedback information obtained in at least one feedback information channel, the historical feedback information is used to characterize the feedback result of the accuracy of the historical reply result, and the historical reply result matches the historical query information. That is to say, in an embodiment of the present invention, by using the historical feedback information fed back in the feedback information channel (for example, user feedback information and sentiment scores), the self-learning of the question-answering model is realized without manual intervention, so that the reinforcement learning algorithm is used to continuously adjust and optimize the model strategy, thereby achieving the technical effect of improving the reply accuracy of the question-answering model and solving the technical problem of low reply accuracy of the question-answering model. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 is a flow chart of a data processing method according to an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of reinforcement learning according to an embodiment of the present invention;

[0021] Figure 3 It is a flow chart of a method for optimizing intelligent question-answering results of a large model of a power system based on reinforcement learning according to an embodiment of the present invention;

[0022] Figure 4 is a schematic diagram of a data processing device according to an embodiment of the present invention;

[0023] Figure 5 is a structural block diagram of a computer terminal according to an embodiment of the present invention;

[0024] Figure 6 The invention is a block diagram of an electronic device according to a method for processing data in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] According to an embodiment of the present invention, an embodiment of a method for processing data is provided, and the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] In this embodiment, a data processing method is proposed. This method can achieve self-learning of the question-answering model by utilizing historical feedback information (for example, user feedback information and sentiment scores) fed back in the feedback information channel without human intervention, thereby utilizing the reinforcement learning algorithm to continuously adjust and optimize the model strategy, thereby achieving the technical effect of improving the answer accuracy of the question-answering model and solving the technical problem of low answer accuracy of the question-answering model.

[0029] Figure 1 FIG. 1 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0030] Step S102: acquiring inquiry information associated with the power system, wherein the inquiry information is used to inquire about the operating status of the power system.

[0031] In the technical solution provided in the above step S102 of the present invention, the above power system may include a power grid. The above inquiry information may be used to inquire about the operating status of the power system, may be issued by a user or a maintainer of the power system, and may be information such as text, image, or voice. For example, the inquiry information may be "how to deal with a sudden increase in the load on the power grid", "what is the load of the power system at 3 pm", etc. It should be noted that this is only for illustrative purposes, and no specific restrictions are imposed on the content and form of the inquiry information.

[0032] For example, a user may input inquiry information related to the power system through a mobile terminal, such as a computer, a mobile phone, a tablet, etc. The intelligent question-answering system may obtain the inquiry information input by the user.

[0033] Step S104: in the power system, determining operating data associated with the query information, wherein the operating data is used to characterize the operating state of the power system.

[0034] In the technical solution provided in the above step S104 of the present invention, the operation data associated with the query information can be determined and retrieved. The above operation data can be data generated during the operation of the power system, for example, data generated during the operation of the power grid, which can be used to characterize the operating state of the power system. It should be noted that this is only an example, and there is no specific limitation on the type of operation data.

[0035] For example, real-time data of power equipment (referred to as equipment) in the power system is obtained from multi-source data such as substation equipment, transmission lines, and dispatch center systems in the power system, and operating data associated with the query information is determined from the acquired real-time data. Alternatively, relevant data can be collected from sources such as operation logs, equipment manuals, and technical documents, and operating data associated with the query information can be determined from the acquired relevant data. It should be noted that this is only an example, and there is no specific limitation on the method of determining operating data.

[0036] Step S106, calling the question-answering model to analyze the operating data and the inquiry information to obtain an answer result that matches the inquiry information, wherein the question-answering model is trained using historical feedback information obtained from at least one feedback information channel, and the historical feedback information is used to characterize the feedback result of the accuracy of the historical answer result, and the historical answer result matches the historical inquiry information.

[0037] In the technical solution provided in the above step S106 of the present invention, the operation data and the query information are used as the input of the question-answering model, and the operation data and the query information are processed by the question-answering model to obtain a reply result matching the query information. Among them, the above question-answering model can be a large model deployed in the intelligent question-answering system, a pre-trained model obtained by training, or a large model in the field of natural language processing (Natural Language Processing, referred to as NLP), which can be used to reply to the query information. For example, the question-answering model can be a third-generation generative pre-trained transformer (Generative Pre-trained Transformer 3, referred to as GPT-3), a bidirectional encoder representation based on a transformer (Bidirectional Encoder Representations from Transformers, referred to as BERT), etc. It should be noted that this is only an example, and there is no specific restriction on the type of question-answering model.

[0038] Optionally, the above-mentioned feedback information channels may include multiple channels such as web pages, mobile applications, and on-site terminals. It should be noted that this is only an example, and there is no specific limitation on the type of feedback information channel. Historical feedback information can be used to characterize the feedback results of the accuracy of historical reply results, can be used to characterize the user's satisfaction with the historical reply results, or can be used to characterize the accuracy of historical question and answer results, and the historical reply results match the historical inquiry information.

[0039] In this embodiment, in the process of calling the question-answering model to analyze the operating data and the query information to obtain a response result that matches the query information, reinforcement learning can be used to determine the most suitable response result among multiple response results that match the query information, thereby achieving the technical effect of improving the response accuracy of the question-answering model and solving the technical problem of the low response accuracy of the question-answering model.

[0040] Optionally, a reinforcement learning framework may be constructed in advance, and a question-answering model may be trained using historical query information, historical response results corresponding to the historical query information, historical feedback information corresponding to the historical response results, and the reinforcement learning framework.

[0041] For example, suppose a user inputs a query in the power dispatching system: "How to deal with a sudden increase in power grid load?" The query input by the user is input into the question-answering model. At the same time, real-time data (i.e., operating data) in the power system can be extracted, for example, operating data such as substation load and line current can be extracted, and the operating data associated with the query information in the extracted data is determined, and the operating data is used as part of the input of the question-answering model. The input query information and operating data are preprocessed to remove noise and outliers in the above data, so as to ensure that the question-answering model can process clean data. The preprocessed query information and operating data can be input into the pre-trained BERT model (i.e., the question-answering model) to obtain a preliminary response result generated by the question-answering model, such as: "It is recommended to increase the output of the backup generator set to balance the load", "Check whether there is a backup power supply that can be enabled to ensure load balance". Furthermore, the potential impact and user satisfaction of each answer result can be determined. The Q value in the reinforcement learning framework can be used to evaluate the potential impact and user satisfaction of each answer result. If the question-answering model determines that the Q value of the second answer result is the highest based on previous data and user feedback, it can be determined that the second answer result is more in line with the actual operating habits of the power grid dispatcher. Therefore, this answer result can be selected as the final output.

[0042] Optionally, if the user believes that the second answer solves the problem, they can provide positive feedback on the answer, and the system can update the Q value based on the feedback to strengthen the use of this strategy in similar questions. As more user feedback is accumulated, the question-answering model can be retrained regularly. New training data and updated Q values ​​will be used to further optimize the model to provide more accurate suggestions in future questions and answers.

[0043] Through the above-mentioned steps S102 and S106 of the present invention, query information associated with the power system is obtained, wherein the query information is used to inquire about the operating status of the power system; in the power system, the operating data associated with the query information is determined, wherein the operating data is used to characterize the operating status of the power system; the question-answering model is called to analyze the operating data and the query information to obtain a reply result matching the query information, wherein the question-answering model is trained using historical feedback information obtained in at least one feedback information channel, the historical feedback information is used to characterize the feedback result of the accuracy of the historical reply result, and the historical reply result matches the historical query information. That is to say, in an embodiment of the present invention, by using the historical feedback information fed back in the feedback information channel (for example, user feedback information and emotional scores), the self-learning of the question-answering model is realized without manual intervention, so that the model strategy is continuously adjusted and optimized using the reinforcement learning algorithm, thereby achieving the technical effect of improving the reply accuracy of the question-answering model and solving the technical problem of low reply accuracy of the question-answering model.

[0044] The above method of this embodiment is further introduced below.

[0045] As an optional implementation, step S102 obtains historical inquiry information, historical reply results, and historical feedback information of historical reply results in multiple feedback information channels; uses the historical inquiry information, historical reply results, and historical feedback information to train a sub-question-and-answer model to obtain a question-and-answer model.

[0046] In this embodiment, historical inquiry information, historical reply results, and historical feedback information of historical reply results in multiple feedback information channels are obtained; the sub-question-answering model can be trained using the historical inquiry information, historical reply results, and historical feedback information to obtain a question-answering model. The sub-question-answering model can be a large model deployed in an intelligent question-answering system, an original model or a basic model, or a large model in the field of natural language processing, and can be used to train a question-answering model.

[0047] For example, power data can be collected from multi-source databases such as substation equipment, transmission lines, and dispatch center systems in the power system, or from sources such as operation logs, equipment manuals, and technical documents to obtain multiple historical query information and historical response results, and historical feedback information generated by users for different historical response results can be obtained from feedback information channels. The historical query information, historical response results, and historical feedback information can be used as training data to train a question-answering model.

[0048] Optionally, the sub-question answering model can capture contextual information in text (e.g., historical query information) through a bidirectional transformer architecture. The configuration parameters of the sub-question answering model may include the number of model layers (L), the number of hidden units (H), the number of attention heads (A), etc. It should be noted that this is only an example, and there is no specific restriction on the configuration parameters of the sub-question answering model.

[0049] As an optional implementation, historical inquiry information, historical reply results, and historical feedback information are used to train a sub-question and answer model to obtain a question and answer model, including: preprocessing the multiple historical inquiry information and multiple historical reply results obtained respectively to obtain multiple preprocessed historical inquiry information and multiple preprocessed historical reply results; using the preprocessed multiple historical inquiry information and the preprocessed multiple historical reply results to construct a first training data set and a validation set, and using multiple historical inquiry information, multiple historical reply results and historical feedback information to construct a second training data set, wherein the data in the first training data set is different from the data in the validation set; using the first training data set, the validation set, and the second training data set to train the sub-question and answer model to obtain a question and answer model.

[0050] In this embodiment, the above-mentioned preprocessing can be used to clean and process the collected power data, and can be used to ensure the quality and consistency of the collected data. For example, the preprocessing may include missing value filling, outlier detection and other preprocessing methods. It should be noted that the preprocessing method here is only an example and is not specifically limited.

[0051] In this embodiment, power data can be collected from multiple source databases such as substation equipment, transmission lines, and dispatch center systems in the power system, or from sources such as operation logs, equipment manuals, and technical documents to obtain multiple historical query information and historical response results. The collected power data can be cleaned and preprocessed, such as missing value filling, outlier detection, and other preprocessing to ensure the quality and consistency of the collected data.

[0052] Optionally, the obtained multiple historical query information and multiple historical reply results are preprocessed respectively to obtain multiple preprocessed historical query information and multiple preprocessed historical reply results. Further, the preprocessed multiple historical query information and the preprocessed multiple historical reply results can be used to construct a first training data set and a validation set, and the multiple historical query information, the multiple historical reply results and the historical feedback information can be used to construct a second training data set.

[0053] For example, multiple pre-processed historical query information and multiple pre-processed historical reply results are obtained, and the pre-processed historical query information and the pre-processed historical reply results can be converted into the input format required by BERT. For example, the text data in the pre-processed historical query information and the pre-processed historical reply results can be converted into vocabulary identifiers (Identity, abbreviated as ID), and input embeddings, position embeddings, and segment embeddings are created to ensure the diversity and representativeness of the data set, and the processed data is divided, so as to divide the pre-processed data into a first training data set, a validation set, and a third training data set, thereby completing the preparation process of the training data.

[0054] Optionally, the sub-question-answering model is trained using the processed first training data, validation set, and third training data set to obtain a question-answering model.

[0055] As an optional implementation, a sub-question and answer model is trained using a first training data set, a validation set, and a second training data set to obtain a question and answer model, including: using the first training data set to train the sub-question and answer model to obtain a first question and answer model; obtaining a third training data set having a degree of correlation with the power system greater than a correlation degree threshold, wherein the third training data set is different from the first training data set and contains relevant inquiry information and relevant response results of the relevant inquiry information; using the second training data set and the third training data set to train the first question and answer model to obtain a question and answer model.

[0056] In this embodiment, the sub-question and answer model can be trained using the first training data set to obtain a first question and answer model.

[0057] Optionally, the BERT model (i.e., the sub-question answering model) can be preliminarily trained using the following loss:

[0058] H BERT =H MLM +H NSP

[0059]

[0060] Among them, H MLM Can be used to represent the Masked Language Model loss. NSP It can be used to represent the next sentence prediction loss. BERT Can be used to characterize the loss of the BERT model. i Can be used to represent masked words, x' i Can be used to represent unmasked context words, yi Can be used to represent the label of the next sentence, y' i Can be used to indicate context sentences.

[0061] Optionally, the masked vocabulary can come from the first training data set, which provides basic data for learning the sub-question-answering model. In the initial model training stage, by training the masked language model on the first training data set, the trained question-answering model can effectively learn the contextual features of the language, laying the foundation for subsequent fine-tuning and application.

[0062] Optionally, after training the first question-answering model (i.e., the pre-trained BERT model), a third training data set may be obtained whose correlation with the power system is greater than a correlation threshold, and the third training data set may include relevant inquiry information related to the power system and relevant reply results corresponding to the relevant inquiry information. The first question-answering model may be further fine-tuned using the second training data set and the third training data set to obtain a question-answering model. The third training data set is different from the first training data set.

[0063] The following further illustrates a method of further fine-tuning the first question-answering model using the second training data set and the third training data set to obtain a question-answering model. It should be noted that any method of obtaining a question-answering model by training using historical data of the power system should be within the scope of protection of the present invention and is not specifically limited here.

[0064] As an optional implementation, the first question-answering model is trained using a validation set, a second training data set, and a third training data set to obtain a question-answering model, including: constructing a loss function, wherein the loss function is associated with historical response results in the third training data set; using the loss function and the third training data set to train the first question-answering model to obtain a second question-answering model; using the validation set to adjust model parameters of the trained second question-answering model to obtain an adjusted second question-answering model; and using the second training data set to train the adjusted second question-answering model to obtain a question-answering model.

[0065] In this embodiment, a loss function is constructed, and the loss function can be preset, and the result of the loss function is associated with the historical answer results in the third training data set. The first question-answering model is trained using the loss function and the third training data set, so that the second question-answering model can be obtained. Further, the model parameters of the trained second question-answering model can be adjusted using the validation set to obtain the second question-answering model; the adjusted second question-answering model is trained using the second training data set to obtain the question-answering model.

[0066] Optionally, after the first question-answering model is trained, the pre-trained BERT model can be fine-tuned using a third training data set related to the power system that has been acquired in advance. During the fine-tuning process, a task-specific loss function H can be defined. QA :

[0067]

[0068] Among them, s i and e i They can be used to indicate the start and end positions of historical response results, context i Used to indicate context, question i Used to indicate a problem.

[0069] Furthermore, the training parameters of the first question-answering model during the training process may include the learning rate α, the batch size B, and the optimizer. The learning rate decay β is adjusted. Multiple training parameters jointly determine the training effect and performance of the trained question-answering model. Selecting the appropriate learning rate, batch size, and optimizer can improve the training effect and convergence speed of the model to achieve the purpose of the best training effect of the model:

[0070]

[0071] Among them, θ can be used to represent the model parameters.

[0072] Optionally, during fine-tuning, the loss function H during training can be monitored in real time. QA to improve the accuracy and avoid overfitting or underfitting.

[0073] Furthermore, after the second question-answering model is trained in the above manner, the validation set can be used to adjust the model parameters of the trained second question-answering model to obtain an adjusted second question-answering model.

[0074] In this embodiment, the trained BERT model can be evaluated using a validation set to determine the performance of the trained second question-answering model. The data in the validation set does not participate in the model training and verification process, and it is an independent evaluation criterion for the generalization ability of the model. The trained second question-answering model is evaluated using the validation set to determine the performance of the model on unseen data, to ensure the reliability of the model in practical applications, to determine whether it is necessary to continue to adjust the second question-answering model. If the second question-answering model needs to be adjusted, the test data can be reacquired and the second question-answering model can be adjusted using the test data to obtain the adjusted second question-answering model.

[0075] Optionally, the validation set can be used to determine indicators such as accuracy, precision, recall, and F1 score of the question-answering results output by the second question-answering model:

[0076]

[0077]

[0078] The following further describes a method for training the adjusted second question-answering model using the second training data set to obtain a question-answering model.

[0079] As an optional implementation, the second question-answering model is trained with the second training data set to obtain a question-answering model, including: converting the second training data set to obtain a converted data set; determining an action data set corresponding to the converted data set, wherein the action data set includes a plurality of action data, and the action data are used to characterize the adjustment action of the second question-answering model on the historical response results; calling the second question-answering model to convert the second training data set and the action data set to obtain evaluation data corresponding to the plurality of action data, wherein the evaluation data is used to characterize the influence of the action data on the historical response results; and using the evaluation data to adjust the second question-answering model to obtain a question-answering model.

[0080] In this embodiment, the second training data set can be constructed based on historical inquiry information, historical answer results and historical feedback information. The second data set can be transformed to obtain a transformed data set. The transformed data set can be used to characterize the state of the second question-answering model, which can be a state, which can be expressed as s t express.

[0081] Furthermore, the action data set (A = {a1, a2, ..., a n}), the action data set may include multiple action data, which can be used to characterize the adjustment action of the second question-answering model on the historical answer result, for example, it can be actions such as modifying the answer content, adjusting the answer structure, and regenerating the answer. By selecting different actions, the second question-answering model can continuously adjust and optimize its output to improve the accuracy of the questions and answers of the final generated question-answering model and user satisfaction. It can be determined that different action data correspond to historical feedback information.

[0082] Optionally, the second question-answering model is called to transform the second training data set and the action data set to obtain evaluation data corresponding to the plurality of action data. The evaluation data can be used to characterize the influence of the action data on the historical answer results, and can be an action value function (also called a reward function), which can be expressed by R tIndicates that the second question-answering model can be adjusted using the evaluation data to obtain a question-answering model.

[0083] In this embodiment, by using the reinforcement learning framework, the agent can be trained to select the best action according to the current state to maximize the cumulative reward. By continuously interacting and learning with the environment, the agent can gradually master how to achieve goals in a complex environment, that is, the large model can output a response result that meets the query information according to the current environment (for example, the current operation data of the power system).

[0084] In this embodiment, a reinforcement learning framework can be built, which can include state, action and reward. The above state can be used to characterize the state of the current second question-answering model, which can be a conversion data set, and can include historical query information input by the user and historical answer results generated by the system. The above state can be expressed by the following formula:

[0085] s t ={historical inquiry information, historical reply results}

[0086] The above actions can be used to represent the operations that can be performed by the second question-answering model. For example, it can be used to modify the content of the answer, adjust the structure of the answer, regenerate the answer, etc. A = {a1, a2, ..., a n} means that, where a i It can be used to represent the actions that the second question-answering model can take in a specific state, such as modifying the answer content, adjusting the answer structure, regenerating the answer, etc. By selecting different actions, the second question-answering model can continuously adjust and optimize its own output to improve the accuracy of the questions and answers of the final generated question-answering model and user satisfaction.

[0087] The above rewards can be used to represent user feedback or the evaluation score automatically generated by the system, and can be used to guide model optimization. The reward function R can be expressed by the following formula:

[0088] R t =λ1U s,web +λ2U c,web +λ3E s,web +λ4U s,app +λ5F app +

[0089] λ6I app +λ7E s,app +λ8U s,term +λ9I term +λ 10 E term +λ 11E s,term +λ 12 A text

[0090] Among them, λ i It can be used to characterize the weight coefficients corresponding to different historical feedback information.

[0091] Optionally, obtain a second training data set, and use the second training data set to obtain a transformed data set s t . Determine the action data sets corresponding to different conversion data sets respectively, call the second question and answer model, process the second training data set and the action data set to obtain evaluation data corresponding to multiple action data. For example, assuming that the conversion data set is {historical inquiry information one, historical reply result one, historical reply result two, historical reply result three}, determine that the action data corresponding to different historical reply results in the conversion data set are {delete part of the answer, change part of the answer, merge multiple answers} respectively. Call the second question and answer model to process the historical inquiry information one in the second training data set. Under the merge action, the result output by the second question and answer model is historical reply result three, and the historical feedback information at this time is determined. According to the above method, the historical feedback information corresponding to the second question and answer model under different action data is determined, so that based on multiple historical feedback information, the evaluation data (R t ). Using R t , adjust the second question-answering model to obtain the final question-answering model.

[0092] Optionally, the learning algorithm of reinforcement learning can be a model-free reinforcement learning (Q-Learning) algorithm. Define a Q function, where the Q function can be used to represent the state s t Next, perform action a t Expected reward after:

[0093]

[0094] Among them, Q(s t ,a t ) can be used to characterize the t Execute action a t The current Q value of , i.e., can be used to represent the expected cumulative reward of performing action a in state s. t It can be used to represent the current state and can include the current historical query information and historical response results. t It can be used to characterize the current action, for example, it can include modifying the answer, regenerating the answer, etc. α can be used to characterize the learning rate and can be used to control the step size of the Q value update. t Can be used to represent the current state s t Execute action a tThe immediate reward obtained. γ can be used to represent the discount factor, which can be used to weigh the importance of future rewards. The value range can be [0,1]. a' can be used to represent the next state s t+ 1 All possible actions in . Can be used to represent the next state s t+1 The maximum Q value of all possible actions.

[0095] Optionally, the Q function is initialized by using a small random value or zero value.

[0096]

[0097] Optionally, the strategy of the reinforcement learning architecture can be updated to explore and exploit via an epsilon-greedy strategy, selecting actions:

[0098]

[0099] Among them, ∈ can be used to represent the exploration probability, which can be used to determine the frequency of selecting random actions arg max a Q(s t ,a) can be used to indicate the selection of the action with the largest Q value in the current state.

[0100] In this embodiment, reinforcement learning may be updated.

[0101] Optionally, initialize the second question-answering model and set the initial state s 0 According to the current state s t Select a t , to complete the action selection. Execute the selected action and generate a new answer t+1 , get the new state s t+1 , thus obtaining new question and answer pairs: t+1 ={question t+1 ,answer t+1}. Further, the reward R of the current action can be calculated t , combined with feedback from various channels and system evaluation:

[0102] R t =λ1U s,web +λ2U c,web +λ3E s,web +λ4U s,app +λ5F app +

[0103] λ6I app +λ7E s,app +λ8U s,term +λ9Iterm +λ 10 E term +λ 11 E s,term +λ 12 A text

[0104] Update the Q value using the Q-Learning algorithm:

[0105]

[0106] Based on the updated Q-value, adjust the strategy of the question-answering system to optimize the generated answers.

[0107] In this embodiment, for the optimization and replacement of the model, the collected new data and updated Q values ​​are regularly used to retrain the BERT model (i.e., the second question-answering model) to ensure that the model parameters can adapt to the new optimization strategy:

[0108]

[0109] Among them, H QA It can be used to represent the loss function of the question-answering task, namely the cross entropy loss function.

[0110] Optionally, new data and feedback are combined to continuously perform retraining and strategy updates to ensure continuous optimization and efficient application of the question-answering system.

[0111] Optionally, the Q value affects the value of the loss function by affecting the strategy selection of the second question-answering model, and the gradient descent update rule updates the model parameters of the second question-answering model according to the gradient of this loss function. Ultimately, the change in the Q value indirectly leads to the optimization and improvement of the model parameters.

[0112] As an optional implementation manner, obtaining historical inquiry information and historical response results includes: obtaining multiple historical inquiry information and multiple historical response results from a database corresponding to the power system.

[0113] In this embodiment, multiple historical inquiry information and historical reply results can be obtained from the database corresponding to the power system, wherein the historical inquiry information can be the inquiry information historically input by the user. The historical reply result can be the result obtained by replying to the historical inquiry information. The database of the above-mentioned power system may include databases corresponding to systems such as substation equipment, transmission lines, and dispatching center systems. It should be noted that this is only for example, and there is no specific limitation on the type of database.

[0114] For example, power data can be collected from multiple source databases such as substation equipment, transmission lines, and dispatching center systems in the power system, or from sources such as operation logs, equipment manuals, and technical documents to obtain multiple historical query information and historical response results.

[0115] As an optional implementation, obtaining historical feedback information of historical reply results in multiple feedback information channels includes: in response to a client triggering an evaluation instruction, obtaining historical feedback information in the evaluation instruction.

[0116] In this embodiment, the user can issue historical feedback information on the historical reply results by triggering an evaluation instruction. Based on the feedback information, the user's score on the accuracy of the historical reply results, satisfaction level, and other information can be determined. It should be noted that this is only an example and there is no specific restriction on the content represented by the historical feedback information.

[0117] Optionally, historical feedback information of the main grid in the power system can be collected through multiple channels such as web pages, mobile applications, and on-site terminals. A special feedback mechanism can be designed, such as a pop-up window on the substation operation interface, embedded feedback in the transmission line inspection mobile application, and updated technical documents of the dispatch center system, to obtain user feedback information on the response results and historical feedback information on historical response results. In the process of model training, historical feedback information can be used to reinforce the model. In the process of model use, feedback information can be used to reinforce the model.

[0118] For webpage feedback collection, a feedback collection module can be integrated into the main network dispatch management platform. After the dispatcher performs a dispatch operation or queries the running status, a feedback window can be automatically popped up on the display interface. The dispatcher can be invited to rate the reply result and fill in comments through the feedback window. In response to the evaluation instruction triggered by the client during feedback, the historical feedback information in the evaluation instruction can be obtained. Among them, the historical feedback information can include the score for the reply result (U s,web )、Comment content(U c,web ) and sentiment score (E s,web ) etc. It should be noted that this is only an example and there is no specific restriction on the content contained in the historical feedback information.

[0119] Optionally, a feedback collection function can be integrated into the power transmission line inspection mobile application to collect feedback information for the power transmission line inspection mobile application. When performing inspection tasks, a feedback button can be provided, through which a feedback operation can be triggered to trigger an evaluation instruction, so that historical feedback information can be obtained after the feedback is completed. The historical feedback information obtained by this channel may include a score for the historical response result (U s,app), the frequency of use of the historical response results (F app ), interaction log (I app ), and the sentiment score (E s,app ).

[0120] In order to collect historical feedback information in the field terminal, the feedback collection function can be integrated in the monitoring terminal of the substation and the transmission line. After the operator performs the field operation, he can provide instant feedback through the terminal device to obtain the historical feedback information of the operator. The historical feedback information of the operator can include the operator score (U s,term ), system log (I term ), Error Report (E term ), sentiment score (E s,term ).

[0121] In an embodiment of the present invention, by utilizing historical feedback information fed back in the feedback information channel (for example, user feedback information and sentiment scores), self-learning of the question-answering model is achieved without human intervention, thereby utilizing the reinforcement learning algorithm to continuously adjust and optimize the model strategy, thereby achieving the technical effect of improving the answer accuracy of the question-answering model and solving the technical problem of low answer accuracy of the question-answering model.

[0122] The technical solution of the embodiment of the present invention is illustrated below in conjunction with preferred implementation modes.

[0123] At present, with the continuous expansion of the scale of my country's power system and the increasingly complex network structure, the rapid development of the power Internet of Things has enabled the power system to enter the information age. The massive data generated by the operation of the power grid makes it difficult for users to quickly obtain accurate answers from the massive data through natural language.

[0124] With the widespread application of large models (such as GPT-3, BERT, etc.) in the field of natural language processing, the performance of intelligent question-answering systems has been significantly improved. However, although large models have powerful language understanding and generation capabilities, it is difficult to meet the diverse needs of power systems by relying solely on large models, resulting in a lack of adaptive capabilities in intelligent question-answering systems, poor performance of question-answering systems, and low accuracy of responses.

[0125] In view of the current problem that it is difficult to quickly obtain accurate answers from the massive data generated by the operation of the power grid through natural language, in this embodiment, a method for tuning the intelligent question and answer results of a large model of the power system based on reinforcement learning is proposed. This method uses the advantages of reinforcement learning. The question and answer model can continuously adjust and optimize itself according to the real-time feedback of the user, significantly improving the adaptive ability of the question and answer model to meet the diverse needs of users, so that the question and answer model can quickly obtain accurate answers from the massive data generated by the operation of the power grid and continuously adapt to the ever-changing power grid environment, thereby achieving the technical effect of improving the answer accuracy of the question and answer model and solving the technical problem of low answer accuracy of the question and answer model.

[0126] Optionally, reinforcement learning is also called evaluation learning. Reinforcement learning can enable the large model to continuously learn knowledge and better adapt to the current usage environment through interaction with environmental variables (such as feedback information).

[0127] Figure 2 is a schematic diagram of reinforcement learning according to an embodiment of the present invention, such as Figure 2 As shown, the reinforcement learning-based method can include incentives, perception, performance, external environment and state. Among them, perception can refer to the recognition and judgment results of the current sample after the learning subject perceives the environment world (also known as state). Action can refer to the fine-tuning of model parameters after interacting with the environment. Incentives (also known as benefits) can be used to characterize the final recognition accuracy of the model.

[0128] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments; the size of the numbers in the embodiments is only for illustration and is not specifically limited here. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention and are not specifically limited here.

[0129] In this embodiment, in order to solve the problem that it is currently difficult to quickly obtain accurate answers from the massive data generated by power grid operation through natural language, a method for optimizing intelligent question-answering results of a large power system model based on reinforcement learning is proposed. Figure 3 is a flow chart of a method for optimizing intelligent question-answering results of a large power system model based on reinforcement learning according to an embodiment of the present invention. Figure 3 As shown, the method may include the following steps:

[0130] Step S301: power data collection and preprocessing.

[0131] In this embodiment, power data can be collected from multiple source databases such as substation equipment, transmission lines, and dispatch center systems in the power system, or from sources such as operation logs, equipment manuals, and technical documents to obtain multiple historical query information and historical response results. The collected power data can be cleaned and preprocessed, such as missing value filling, outlier detection, etc., to ensure the quality and consistency of the collected data.

[0132] Step S302, training the original model.

[0133] In this embodiment, the BERT model can be selected as the basic model. The sub-question-answering model captures contextual information in the text (e.g., historical query information) through a bidirectional converter architecture. The configuration parameters of the sub-question-answering model may include the number of model layers (L), the number of hidden units (H), the number of attention heads (A), etc. It should be noted that this is only an example, and there is no specific restriction on the model parameters of the sub-question-answering model.

[0134] In this embodiment, training the original model may include the following steps: selecting a model architecture, data preparation, preliminary training of the model based on the prepared data, fine-tuning the trained model, and evaluating the fine-tuned model to obtain an evaluation result. The evaluation result may be used to determine whether to continue fine-tuning the model.

[0135] In this embodiment, the preprocessed power system data set, such as the preprocessed historical query information and the preprocessed historical reply results, is converted into the input format required by BERT, which may include converting text data into vocabulary tags, creating input embeddings, position embeddings, and segment embeddings. The diversity and representativeness of the data set are ensured to divide the preprocessed data into a first training data set, a validation set, and a third training data set, thereby completing the preparation process of the training data.

[0136] In this embodiment, after obtaining the first training data, the validation set and the third training data set, the sub-question and answer model can be trained to obtain a question and answer model.

[0137] Optionally, the BERT model (i.e., the sub-question answering model) can be preliminarily trained using the following formula:

[0138] H BERT =H MLM +H NSP

[0139]

[0140] Optionally, after the first question-answering model is trained, the pre-trained BERT model can be fine-tuned using a third training data set related to the power system that has been acquired in advance. During the fine-tuning process, a task-specific loss function H can be defined. QA :

[0141]

[0142] Among them, s i and e i They can be used to indicate the start and end positions of historical response results, context i Used to indicate context, question i Used to indicate a problem.

[0143] Furthermore, the training parameters of the first question-answering model during the training process may include the learning rate α, the batch size B, and the optimizer. The learning rate decay β is adjusted. Multiple training parameters jointly determine the training effect and performance of the trained question-answering model. Selecting the appropriate learning rate, batch size, and optimizer can improve the training effect and convergence speed of the model to achieve the purpose of the best training effect of the model:

[0144]

[0145] Among them, θ can be used to represent the model parameters.

[0146] Optionally, during fine-tuning, the loss function H during training can be monitored in real time. QA to improve the accuracy and avoid overfitting or underfitting.

[0147] In this embodiment, the trained BERT model can be evaluated using a validation set to determine the performance of the trained second question-answering model. The data in the validation set does not participate in the model training and verification process, and it is an independent evaluation criterion for the generalization ability of the model. The trained second question-answering model is evaluated using the validation set to determine the performance of the model on unseen data, to ensure the reliability of the model in practical applications, to determine whether it is necessary to continue to adjust the second question-answering model. If the second question-answering model needs to be adjusted, the test data can be reacquired and the second question-answering model can be adjusted using the test data to obtain the adjusted second question-answering model.

[0148] Optionally, the validation set can be used to determine indicators such as accuracy, precision, recall, and F1 score of the question-answering results output by the second question-answering model:

[0149]

[0150]

[0151] Step S303: Collect historical feedback information.

[0152] In this embodiment, feedback information of the main grid of the power system can be collected through multiple channels such as web pages, mobile applications and on-site terminals. A special feedback mechanism can be designed, such as a pop-up window on the substation operation interface, embedded feedback in the transmission line inspection mobile application, and updated technical documents of the dispatching center system, to obtain user feedback information on the response results.

[0153] Optionally, the collection of feedback information may include: collection of feedback information in web pages, collection of feedback information in mobile applications for transmission line inspection, collection of feedback information in field terminals, and collection of feedback information in updated equipment manuals and technical documents.

[0154] Optionally, for webpage feedback collection, a feedback collection module can be integrated into the main network dispatch management platform. After the dispatcher performs a dispatch operation or queries the running status, a feedback window can be automatically popped up on the display interface. The dispatcher can be invited to rate the reply result and fill in comments through the feedback window. In response to the evaluation instruction triggered by the client during feedback, the historical feedback information in the evaluation instruction can be obtained. Among them, the historical feedback information may include the score for the reply result (U s,web )、Comment content(U c,web ) and sentiment score (E s,web ) etc. It should be noted that this is only an example and there is no specific restriction on the content contained in the historical feedback information.

[0155] Optionally, a feedback collection function can be integrated into the power transmission line inspection mobile application to collect feedback on the power transmission line inspection mobile application. When performing inspection tasks, a feedback button can be provided. Through the feedback button, a feedback operation can be triggered to trigger an evaluation instruction, thereby obtaining historical feedback information after the feedback is completed. The historical feedback information obtained by this channel may include a score for the historical response result (U s,app ), the frequency of use of the historical response results (F app ), interaction log (I app ), and the sentiment score (E s,app ).

[0156] Optionally, for the collection of historical feedback information in the field terminal, the feedback collection function is integrated into the monitoring terminal of the substation and the transmission line. After the operator performs the field operation, the operator provides instant feedback through the terminal device to obtain the historical feedback information of the operator. The historical feedback information of the operator can include the operator score (U s,term ), system log (I term ), Error Report (E term ), sentiment score (E s,term ).

[0157] Optionally, obtain updated equipment manuals and technical documents in the power system. The above data can be used with A text express.

[0158] In this embodiment, after the historical feedback information is obtained, step S304 may be implemented.

[0159] Step S304, building a reinforcement learning framework.

[0160] In the neural network model, the reinforcement learning framework is a machine learning method that can train a large model to achieve a specific goal through interaction with the environment. The reinforcement learning framework can contain an agent, an environment, a state, an action, a reward, a policy, and a value function.

[0161] Among them, the agent is the subject in the learning process, which obtains information and makes decisions by interacting with the environment. The agent may correspond to the question-answering model, the first question-answering model or the second question-answering model in this embodiment. The above environment may be the external world in which the agent interacts, and may provide information such as state and reward. The above state is a description at a certain moment, which may be represented by a vector, and the agent may make a decision based on the current state. The above action may refer to the action taken by the agent in a certain state, and the choice of action may affect the state change of the output result of the large model and the reward obtained by the agent. The above reward may refer to the feedback signal obtained by the agent after performing a certain action, which may be used to guide the learning process of the agent, and the reward may be positive (encouraging a certain behavior) or negative (punishing a certain behavior). The above strategy may be a rule for the agent to select an action based on the current state, which may be deterministic (each state corresponds to a fixed action) or random (each state corresponds to a probability distribution of an action). The above value function may be used to evaluate the expected return of a certain state or state-action pair, and may include a state value function and an action value function. The above learning algorithm can be a method for the agent to update parameters such as strategy and value function according to the feedback information, for example, it can be Q-learning, policy gradient, etc. It should be noted that this is only for example, and there is no specific restriction on the type of value function and the type of learning algorithm.

[0162] By utilizing the reinforcement learning framework, the agent can be trained to select the best action based on the current state to maximize the cumulative reward. By continuously interacting and learning with the environment, the agent can gradually master how to achieve goals in a complex environment, that is, the large model can output a response that meets the query information based on the current environment (for example, the current operation data of the power system).

[0163] In this embodiment, a reinforcement learning framework can be built, which can include state, action and reward. The above state can be used to characterize the state of the current second question-answering model, which can include historical query information input by the user and historical answer results generated by the system. The above state can be expressed by the following formula:

[0164] s t ={historical inquiry information; historical reply results}

[0165] The above actions can be used to represent the operations that can be performed by the second question-answering model. For example, it can be used to modify the content of the answer, adjust the structure of the answer, regenerate the answer, etc. A = {a1, a2, ..., a n} means that, where a i It can be used to represent the actions that the second question-answering model can take in a specific state, such as modifying the answer content, adjusting the answer structure, regenerating the answer, etc. By selecting different actions, the second question-answering model can continuously adjust and optimize its own output to improve the accuracy of the questions and answers of the final generated question-answering model and user satisfaction.

[0166] The above rewards can be used to represent user feedback or the evaluation score automatically generated by the system, and can be used to guide model optimization. The reward function R can be expressed by the following formula:

[0167] R t =λ1U s,web +λ2U c,web +λ3E s,web +λ4U s,app +λ5F app +

[0168] λ6I app +λ7E s,app +λ8U s,term +λ9I term +λ 10 E term +λ 11 E s,term +λ 12 A text

[0169] Among them, λ i It can be used to characterize the weight coefficients corresponding to different historical feedback information.

[0170] Optionally, remove invalid data and ensure data format is consistent.

[0171] In this embodiment, the learning algorithm of the reinforcement learning may be a model-free reinforcement learning (Q-Learning) algorithm.

[0172] Optionally, define a Q function, where the Q function can be used to represent the state s t Next, perform action a t Expected reward after:

[0173]

[0174] Among them, Q(s t ,a t ) can be used to characterize the tExecute action a t The current Q value of , i.e., can be used to represent the expected cumulative reward of performing action a in state s. t It can be used to represent the current state and can include the current historical query information and historical response results. t It can be used to characterize the current action, for example, it can include modifying the answer, regenerating the answer, etc. α can be used to characterize the learning rate and can be used to control the step size of the Q value update. t Can be used to represent the current state s t Execute action a t The immediate reward obtained. γ can be used to represent the discount factor, which can be used to weigh the importance of future rewards. The value range can be [0,1]. a' can be used to represent the next state s t+1 All possible actions in . Can be used to represent the next state s t+1 The maximum Q value of all possible actions.

[0175] Optionally, the Q function is initialized by using a small random value or zero value.

[0176]

[0177] Optionally, the strategy of the reinforcement learning architecture can be updated to explore and exploit via an epsilon-greedy strategy, selecting actions:

[0178]

[0179] Among them, ∈ can be used to represent the exploration probability, which can be used to determine the frequency of selecting random actions arg max a Q(s t ,a) can be used to indicate the selection of the action with the largest Q value in the current state.

[0180] In this embodiment, reinforcement learning may be updated.

[0181] Optionally, initialize the second question-answering model and set the initial state s0. t Select a t , to complete the action selection. Execute the selected action and generate a new answer t+1 , get the new state s t+1 , thus obtaining new question and answer pairs: t+1 ={question t+1 ,answer t+1}. Further, the reward R of the current action can be calculated t , combined with feedback from various channels and system evaluation:

[0182] R t =λ1U s,web +λ2U c,web +λ3E s,web +λ4U s,app +λ5F app +

[0183] λ6I app +λ7E s,app +λ8U s,term +λ9I term +λ 10 E term +λ 11 E s,term +λ 12 A text

[0184] Update the Q value using the Q-Learning algorithm:

[0185]

[0186] Based on the updated Q-value, adjust the strategy of the question-answering system to optimize the generated answers.

[0187] In this embodiment, for the optimization and replacement of the model, the collected new data and updated Q values ​​are regularly used to retrain the BERT model (i.e., the second question-answering model) to ensure that the model parameters can adapt to the new optimization strategy:

[0188]

[0189] Among them, H QA It can be used to represent the loss function of the question-answering task, namely the cross entropy loss function.

[0190] Optionally, new data and feedback are combined for continuous retraining and strategy updates to ensure continuous optimization and efficient application of the question-answering system.

[0191] Optionally, the Q value affects the value of the loss function by affecting the strategy selection of the second question-answering model, and the gradient descent update rule updates the model parameters according to the gradient of this loss function. Ultimately, the change in the Q value indirectly leads to the optimization and improvement of the model parameters.

[0192] The application of existing intelligent question-answering systems in power systems faces multiple challenges. First, these systems often rely on static knowledge bases, lack the ability of dynamic learning and self-optimization, and cannot respond to and adapt to changes in grid demand in a timely manner. Second, existing systems usually rely on system feedback from a single channel, ignoring the comprehensive use of multi-channel data, resulting in insufficient feedback data and affecting system optimization effects. In addition, existing systems have poor adaptability in complex application scenarios, making it difficult to meet the diverse actual needs of power systems, limiting their potential for widespread application. These defects seriously restrict the effectiveness and user experience of intelligent question-answering systems in power systems.

[0193] To solve the above problems, in this embodiment, a method for optimizing the intelligent question-answering results of a large model of a power system based on reinforcement learning is adopted. The method significantly improves the accuracy and practicality of the intelligent question-answering system of the power system by introducing reinforcement learning and multi-channel feedback optimization strategies. By collecting staff feedback through web pages, mobile applications and on-site terminals, the system can fully understand the needs of relevant personnel and power grid changes, and dynamically adjust the question-answering strategy in combination with the emotional scoring mechanism to provide more considerate services, thereby further adapting to environmental changes in the power system, continuously learning and improving, and keeping the system in the best state. The optimized intelligent question-answering system can quickly and accurately answer questions from operators and engineers, reduce the time for finding information and solving problems, improve work efficiency, reduce the risk of operational errors, and ensure the safe operation of the power system. The system also has the ability to flexibly adapt to different application scenarios, such as fault diagnosis, operation guidance, equipment maintenance, etc., to ensure the provision of high-quality services in various complex environments. Through periodic retraining and model updates, the system can optimize itself in combination with new data and feedback from relevant personnel to ensure continuous improvement of performance and service quality.

[0194] For example, suppose a user asks a question in the power dispatching system: "How to deal with the sudden increase in power grid load?" The query information entered by the user is first received by the question-answering model. Real-time data (i.e., operating data) in the power system is extracted, such as substation load, line current and other operating data, and used as part of the model input. The input data is preprocessed to remove noise and outliers to ensure that the model can process clean data. The preprocessed data is input into the pre-trained BERT model to obtain preliminary response results generated by the question-answering model, such as: "It is recommended to increase the output of the backup generator set to balance the load" and "Check whether there is a backup power supply that can be enabled to ensure load balance." Using reinforcement learning, the best answer is selected from multiple preliminary response results.

[0195] Optionally, the potential impact and user satisfaction of each answer can be evaluated by the Q value in the reinforcement learning framework. The question-answering model can find that the second answer has the highest Q value based on past data and user feedback, because it is more in line with the actual operating habits of the power grid dispatcher, and therefore, the answer can be selected as the final output.

[0196] If the user thinks that the second answer solves the problem and gives positive feedback, the system can update the Q value based on the feedback to strengthen the use of this strategy in similar questions. As more user feedback information is accumulated, the question-answering model can be retrained regularly. New training data and updated Q values ​​will be used to further optimize the model to provide more accurate suggestions in future questions and answers.

[0197] In an embodiment of the present invention, by utilizing historical feedback information fed back in the feedback information channel (for example, user feedback information and sentiment scores), self-learning of the question-answering model is achieved without human intervention, thereby utilizing the reinforcement learning algorithm to continuously adjust and optimize the model strategy, thereby achieving the technical effect of improving the answer accuracy of the question-answering model and solving the technical problem of low answer accuracy of the question-answering model.

[0198] According to an embodiment of the present invention, a data processing device is also provided. It should be noted that the data processing device of this embodiment can be used to execute the data processing method of embodiment 1 of the present invention.

[0199] Figure 4 Schematic diagram of a data processing device according to an embodiment of the present invention. Figure 4 As shown, the data processing device 40 may include: an acquisition unit 402 , a determination unit 404 and a processing unit 406 .

[0200] The acquisition unit 402 is used to acquire inquiry information associated with the power system, wherein the inquiry information is used to inquire about the operating status of the power system.

[0201] The determination unit 404 is used to determine, in the power system, operating data associated with the query information, wherein the operating data is used to characterize an operating state of the power system.

[0202] Processing unit 406 is used to call the question-answering model to analyze the operating data and the inquiry information to obtain an answer result that matches the inquiry information, wherein the question-answering model is trained using historical feedback information obtained from at least one feedback information channel, and the historical feedback information is used to characterize the feedback result of the accuracy of the historical answer result, and the historical answer result matches the historical inquiry information.

[0203] The data processing device of this embodiment obtains query information associated with the power system through an acquisition unit, wherein the query information is used to inquire about the operating status of the power system; determines operating data associated with the query information in the power system through a determination unit, wherein the operating data is used to characterize the operating status of the power system; and calls a question-and-answer model through a processing unit to analyze the operating data and the query information to obtain a reply result matching the query information, wherein the question-and-answer model is trained using historical feedback information obtained from at least one feedback information channel, the historical feedback information is used to characterize the feedback result of providing feedback on the accuracy of historical reply results, and the historical reply results match the historical query information, thereby achieving the technical effect of improving the reply accuracy of the question-and-answer model and solving the technical problem of low reply accuracy of the question-and-answer model.

[0204] The embodiment of the present invention can provide a computer terminal, which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0205] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0206] In this embodiment, the computer terminal can execute the program codes of the above steps in the data processing method.

[0207] Optionally, Figure 5 is a structural block diagram of a computer terminal according to an embodiment of the present invention. Figure 5 As shown, the computer terminal 508 may include: one or more (only one is shown in the figure) processors 502 , a memory 504 and a transmission device 506 .

[0208] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the data processing method and device in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned data processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 508 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0209] The processor can call the information and application programs stored in the memory through the transmission device to execute the steps of the above-mentioned data processing method.

[0210] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only, and the computer terminal 508 may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID for short), a PAD, or other terminal devices. Figure 5 The structure of the computer terminal 508 is not limited. For example, the computer terminal 508 may also include Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations are shown.

[0211] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0212] According to an embodiment of the present invention, a computer-readable storage medium is further provided. The computer-readable storage medium includes a stored program, wherein the program executes the data processing method in Embodiment 1.

[0213] According to an embodiment of the present invention, a processor is further provided. The processor is used to run a program. When the program is run by the processor, the data processing method in Embodiment 1 is executed.

[0214] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0215] In this embodiment, the computer terminal can execute the program code of the steps of the data processing method.

[0216] Wherein, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the multilingual translation method and device in the embodiment of the present invention, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned multilingual translation method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0217] According to an embodiment of the present invention, a computer program product is further provided. The computer program product includes computer instructions. When the computer instructions are executed by a processor, the data processing method in Embodiment 1 is implemented.

[0218] An embodiment of the present invention may provide an electronic device, which may include a memory and a processor.

[0219] Figure 6 1 is a block diagram of an electronic device for a data processing method according to an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0220] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 to a random access memory (RAM) 603. In RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0221] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 604, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0222] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as a verification method for data. For example, in some embodiments, the verification method for data may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the verification method for data described above may be executed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the data verification method in any other appropriate manner (for example, by means of firmware).

[0223] According to an embodiment of the present invention, a method for processing data is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0224] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0225] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0226] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0227] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or an LCD (liquid crystal display, monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a path ball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0228] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0229] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0230] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0231] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0232] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units can be a logical function division, and there can be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0233] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0234] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0235] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0236] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: include: Acquiring inquiry information associated with the power system, wherein the inquiry information is used to inquire about the operating status of the power system; In the power system, determining operating data associated with the query information, wherein the operating data is used to characterize an operating state of the power system; A question-and-answer model is called to analyze the operating data and the inquiry information to obtain an answer result that matches the inquiry information, wherein the question-and-answer model is trained using historical feedback information obtained from at least one feedback information channel, and the historical feedback information is used to characterize feedback results for the accuracy of historical answer results, and the historical answer results match the historical inquiry information.

2. The method according to claim 1, characterized in that: The method further comprises: Acquire the historical inquiry information, the historical reply results, and the historical feedback information of the historical reply results in the plurality of feedback information channels; The sub-question-answering model is trained using the historical inquiry information, the historical reply results, and the historical feedback information to obtain the question-answering model.

3. The method according to claim 2, characterized in that The using the historical inquiry information, the historical reply results, and the historical feedback information to train the sub-question-answer model to obtain the question-answer model includes: Preprocessing the acquired multiple historical query information and the acquired multiple historical reply results respectively to obtain the preprocessed multiple historical query information and the preprocessed multiple historical reply results; Using the preprocessed multiple historical query information and the preprocessed multiple historical reply results, a first training data set and a validation set are constructed, and using the multiple historical query information, the multiple historical reply results and the historical feedback information, a second training data set is constructed, wherein the data in the first training data set is different from the data in the validation set; The sub-question-answering model is trained using the first training data set, the validation set, and the second training data set to obtain the question-answering model.

4. The method according to claim 3, characterized in that: The step of training the sub-question-answering model by using the first training data set, the validation set, and the second training data set to obtain the question-answering model includes: Using the first training data set, training the sub-question-answering model to obtain a first question-answering model; Acquire a third training data set having a correlation degree with the power system greater than a correlation degree threshold, wherein the third training data set is different from the first training data set and includes relevant query information and relevant response results of the relevant query information; The first question-answering model is trained using the validation set, the second training data set, and the third training data set to obtain the question-answering model.

5. The method according to claim 4, characterized in that The step of training the first question-answering model by using the validation set, the second training data set, and the third training data set to obtain the question-answering model includes: Constructing a loss function, wherein the loss function is associated with historical response results in the third training data set; Using the loss function and the third training data set, training the first question-answering model to obtain a second question-answering model; Using the validation set, adjusting the model parameters of the trained second question-answering model to obtain an adjusted second question-answering model; The adjusted second question-answering model is trained using the second training data set to obtain the question-answering model.

6. The method according to claim 5, characterized in that Using the second training data set, training the adjusted second question-answering model to obtain the question-answering model includes: Converting the second training data set to obtain a converted data set; Determine an action data set corresponding to the conversion data set, wherein the action data set includes a plurality of action data, and the action data is used to represent an adjustment action of the second question-answering model on the historical answer result; Calling the second question-answering model to transform the second training data set and the action data set to obtain evaluation data corresponding to the plurality of action data, wherein the evaluation data is used to characterize the influence of the action data on the historical answer result; The second question-answering model is adjusted using the evaluation data to obtain the question-answering model.

7. The method according to claim 2, characterized in that The obtaining of the historical inquiry information and the historical reply results includes: A plurality of the historical inquiry information and a plurality of the historical reply results are obtained from a database corresponding to the power system.

8. The method according to claim 2, characterized in that: The obtaining of the historical feedback information of the historical reply results in the plurality of feedback information channels includes: In response to the client triggering the evaluation instruction, historical feedback information in the evaluation instruction is obtained.

9. A data processing device, characterized in that: include: an acquisition unit, configured to acquire inquiry information associated with the power system, wherein the inquiry information is used to inquire about the operating status of the power system; a determining unit, configured to determine, in the power system, operating data associated with the query information, wherein the operating data is used to characterize an operating state of the power system; A processing unit is used to call a question-answering model to analyze the operating data and the inquiry information to obtain an answer result that matches the inquiry information, wherein the question-answering model is trained using historical feedback information obtained from at least one feedback information channel, and the historical feedback information is used to characterize feedback results for the accuracy of historical answer results, and the historical answer results match the historical inquiry information.

10. A computer program product, characterized in that The method comprises computer instructions, which implement the method according to any one of claims 1 to 8 when executed by a processor.

Images