Method and device for performing question and answer processing by applying electric power data model and storage medium

Abstract

The invention provides a method and device for performing question and answer processing by applying a power data model and a storage medium. The method comprises the following steps: acquiring a power industry data set, wherein the power industry data set comprises power industry public data and power enterprise internal private data; according to the data set of the power industry, constructing an instruction data set for a preset task of the power industry, obtaining a pre-training model of a natural language processing task, training the pre-training model by adopting the instruction data set so as to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-training model, obtaining a power data model; and deploying the power data model to a question-answering system of the practical application power industry for answering related questions of the power industry. The problems that in the prior art, efficiency is low in the aspect of utilizing data in the specific field, a model cannot fully capture and utilize professional knowledge of the power industry, and therefore the performance and accuracy of the model actually applied to the power industry system question-answering system are affected are solved.

Description

Technical Field

[0001] The present application relates to the technical field of applying an electric power data model to perform question-answering processing, and in particular, to a method, device, storage medium, and electronic device for applying an electric power data model to perform question-answering processing. Background Art

[0002] As the digitalization of enterprises accelerates, the requirements for management sophistication are increasing, and the complexity of the rules and regulations in the power industry is also growing. The rules and regulations of the power industry constitute a comprehensive system of norms and guidelines, which aims to ensure the safety, reliability, efficiency and sustainability of power supply, while promoting fair competition and healthy development of the power market. The existing large number of rules and regulations covers multiple aspects such as laws, industry standards, management regulations, operating procedures and technical specifications, and plays a vital role in the smooth operation of power companies.

[0003] In order to make the learning and query of the power industry system more convenient and to quickly locate the required specific specifications from numerous rules and regulations, it is particularly necessary to develop an efficient and intelligent question-answering assistant system. With the advancement of artificial intelligence technology, especially the significant development in the field of natural language processing, large pre-trained models have shown their excellent performance in multiple application scenarios. Although a large number of existing models have acquired extensive language understanding capabilities through pre-training on massive text data, a large number of existing general models still have deficiencies in understanding professional field knowledge and systems such as the power industry. In order to improve the performance of the model in a specific professional field, researchers usually need to fine-tune the model, that is, to further train the model using data from a specific field.

[0004] With the increasing sophistication of the power industry's institutional management needs and the increasing complexity of the system, it is essential to quickly and accurately retrieve and interpret relevant systems and documents. However, existing technologies have shortcomings in meeting this challenge, including the lack of technical solutions specifically tailored to the needs of power industry institutional management, the limitations of traditional retrieval methods, the lack of semantic understanding of deep learning methods in specific fields, the poor interpretability of large models in specific fields, and the low efficiency of data utilization. There is currently a lack of technical solutions specifically tailored to the needs of power industry institutional management. Traditional retrieval and keyword matching methods are difficult to provide satisfactory retrieval results when dealing with power industry systems due to the similarity of professional vocabulary and the repetitiveness of content. Deep learning methods such as pre-trained models can be used for semantic search, but these methods have insufficient semantic understanding of specific fields and lack the ability to deeply interpret content. Existing large models usually have complex structures containing hundreds of millions to billions of parameters. In specific fields such as the power industry, this complexity may make it difficult to optimize the model for specific tasks. Existing technologies are inefficient in utilizing data in specific fields, and the model may not be able to fully capture and utilize the expertise of the power industry, thus affecting the performance and accuracy of the model.

[0005] In the process of digital transformation of the power industry system, managers have begun to use computer systems and various software tools to retrieve and manage relevant information. Database management systems are used to store and manage a large amount of power system data, such as electricity price mechanisms, operating rules, personnel management rules, etc. However, although database retrieval can locate relevant rules, it cannot provide explanations and further understanding of these rules. Summary of the invention

[0006] The main purpose of this application is to provide a method, device, storage medium and electronic device for question and answer processing using an electric power data model, so as to at least solve the problem that the existing technology is inefficient in utilizing data in specific fields, and the model cannot fully capture and utilize the professional knowledge of the electric power industry, thereby affecting the performance and accuracy of the model in the actual application of the electric power industry system question and answer system.

[0007] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for applying an electric power data model for question and answer processing is provided, comprising: an acquisition step, acquiring an electric power industry data set, wherein the electric power industry data set includes public data of the electric power industry and private data within the electric power enterprise; a construction step: constructing instructions for preset tasks of the electric power industry according to the electric power industry data set to obtain an instruction data set, wherein the instructions for the preset tasks of the electric power industry include question and answer instructions, text classification instructions and summary generation instructions; an optimization processing step: obtaining a pre-trained model for a natural language processing task, using the instruction data set to train the pre-trained model to optimize at least one of the cross entropy loss parameter and the mean square error parameter of the pre-trained model to obtain an electric power data model; a deployment step: deploying the electric power data model to a question and answer system that is actually applied to the electric power industry, so as to answer questions related to the electric power industry.

[0008] Optionally, before using the instruction data set to train the pre-trained model to optimize at least one of the cross entropy loss parameters and mean square error parameters of the pre-trained model, the method also includes: obtaining parameters of a weight matrix in a pre-trained attention mechanism of the pre-trained model, and decomposing the parameters of the weight matrix into a first low-rank matrix and a second low-rank matrix, wherein the product of the first low-rank matrix and the second low-rank matrix is ​​equal to the parameters of the weight matrix.

[0009] Optionally, decomposing the parameters of the weight matrix into a first low-rank matrix and a second low-rank matrix, including: using the formula W0+ΔW=W0+BA to decompose the parameters of the weight matrix into the first low-rank matrix and the second low-rank matrix, wherein W0 is the weight matrix, ΔW is the parameters of the weight matrix, A is the first low-rank matrix, and B is the second low-rank matrix.

[0010] Optionally, the pre-trained model is trained using the instruction data set to optimize at least one of the cross entropy loss parameter and the mean square error parameter of the pre-trained model to obtain the power data model, including: training the pre-trained model using the instruction data set; during the training process, adjusting the first low-rank matrix and the second low-rank matrix using a gradient descent optimization algorithm to optimize the mean square error parameter of the pre-trained model to obtain the power data model.

[0011] Optionally, using a gradient descent optimization algorithm to adjust the first low-rank matrix and the second low-rank matrix to optimize the mean square error parameter of the pre-trained model includes:

[0012] According to the formula Determine the mean square error parameter of the pre-trained model, where MSE is the mean square error parameter, Y i is the true value, W0 is the weight matrix, ΔW is the parameter of the weight matrix, A is the first low-rank matrix, B is the second low-rank matrix, x is the data of the instruction data set, and n is the number of data in the instruction data set.

[0013] Optionally, after using the instruction data set to train the pre-trained model to optimize at least one of the cross entropy loss parameter and the mean square error parameter of the pre-trained model to obtain the power data model, the method also includes: using the instruction data set to perform a verification operation on the power data model to verify whether the accuracy of the power data model is greater than the accuracy threshold, whether the recall rate is greater than the recall rate threshold, and whether the F1 score is greater than the score threshold, wherein the F1 score is the harmonic mean of the accuracy and the recall rate; when it is verified that the power data model satisfies the conditions that the accuracy is greater than the accuracy threshold, the recall rate is greater than the recall rate threshold, and the F1 score is greater than the score threshold, it is determined that the power data model has passed the verification.

[0014] Optionally, after verifying whether the accuracy of the power data model is greater than the accuracy threshold, whether the recall rate is greater than the recall rate threshold, and whether the F1 score is greater than the score threshold, the method further includes: when verifying that the power data model satisfies at least one of the accuracy rate being less than or equal to the accuracy threshold, the recall rate being less than or equal to the recall rate threshold, and the F1 score being less than or equal to the score threshold, determining that the power data model verification has failed, and continuing to train the power data model.

[0015] According to another aspect of the present application, a device for generating a system model for an electric power industry is provided, comprising: an acquisition unit for executing an acquisition step to acquire an electric power industry data set, wherein the electric power industry data set includes public data of the electric power industry and private data within electric power enterprises; a construction unit for executing a construction step: constructing instructions for preset tasks for the electric power industry according to the electric power industry data set to obtain an instruction data set, wherein the instructions for the preset tasks for the electric power industry include question-and-answer instructions, text classification instructions, and summary generation instructions; a fine-tuning processing unit for executing an optimization processing step: acquiring a pre-trained model for a natural language processing task, training the pre-trained model using the instruction data set to optimize at least one of the cross entropy loss parameter and the mean square error parameter of the pre-trained model to obtain an electric power data model; a deployment unit for executing a deployment step: deploying the electric power data model to a question-and-answer system for actual application in the electric power industry to answer questions related to the electric power industry.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute any one of the methods for applying the power data model to perform question and answer processing.

[0017] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the methods for question and answer processing using an application power data model.

[0018] Applying the technical solution of the present application, through the acquisition step, the power industry data set is obtained, wherein the power industry data set includes public data of the power industry and private data within the power enterprise; the construction step: according to the power industry data set, construct instructions for preset tasks of the power industry to obtain an instruction data set, wherein the instructions for preset tasks of the power industry include question-answering instructions, text classification instructions and summary generation instructions; the optimization step: obtain a pre-trained model for natural language processing tasks, use the instruction data set to train the pre-trained model to optimize at least one of the cross entropy loss parameter and mean square error parameter of the pre-trained model, and obtain a power data model; the deployment step: deploy the power data model to the question-answering system of the actual application power industry, which is used to answer questions related to the power industry. By training the model with public data of the power industry and private data within the power enterprise, the model parameters are adjusted in a targeted manner, the model's understanding and adaptability to the power industry system is improved, and the time and computing resources required for model training are greatly reduced, thereby improving the training efficiency. It solves the problem that the existing technology is inefficient in utilizing data in specific fields, and the model cannot fully capture and utilize the professional knowledge of the power industry, thereby affecting the performance and accuracy of the model's actual application of the power industry system question-answering system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for question-answering processing using a power data model provided in an embodiment of the present application is shown;

[0021] Figure 2 A flow chart of a method for question-answering processing using a power data model provided in accordance with an embodiment of the present application is shown;

[0022] Figure 3 A flowchart of a method for question-answering processing using a specific power data model provided in an embodiment of the present application is shown;

[0023] Figure 4 A structural block diagram of a device for performing question-and-answer processing using a power data model provided according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] As introduced in the background technology, the existing technology is inefficient in utilizing data in specific fields, and the model is unable to fully capture and utilize the professional knowledge of the electric power industry, thereby affecting the performance and accuracy of the model in actually applying the electric power industry system question and answer system. In order to solve the problem that the existing technology is inefficient in utilizing data in specific fields, and the model is unable to fully capture and utilize the professional knowledge of the electric power industry, thereby affecting the performance and accuracy of the model in actually applying the electric power industry system question and answer system, the embodiments of the present application provide a method, device, storage medium and electronic device for question and answer processing using a power data model.

[0028] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal for a method of performing question-answering processing using a power data model according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for question-answering processing using the power data model in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The transmission device 106 is used to receive or send data via a network. The above network specific examples may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a method for question and answer processing using an application power data model running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0032] Figure 2 1 is a flow chart of a method for performing question-answering processing using a power data model according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0033] Step S201, acquisition step: acquiring a power industry data set, wherein the power industry data set includes public data of the power industry and private data within power enterprises;

[0034] Among them, the power industry data set covers data on laws, industry standards, management regulations, operating procedures, technical specifications and other aspects.

[0035] Specifically, after obtaining the electric power industry data set, preprocessing operations are performed on the data of the electric power industry data set, including data cleaning, data deduplication, data segmentation and other preprocessing operations.

[0036] Step S202, construction step: constructing instructions for preset tasks in the power industry according to the power industry data set to obtain an instruction data set, wherein the instructions for preset tasks in the power industry include question-answering instructions, text classification instructions, and summary generation instructions;

[0037] For example: Question: What is the electricity price mechanism in the power industry? Answer: The power system adopts a dual-track electricity price mechanism of planning and market. Planned electricity is mainly based on historical electricity demand, future development plans, supply environment and policy environment to formulate power balance plans and power generation plans. Market electricity is in the power market, where the supply and demand sides directly negotiate electricity prices, and the power grid companies only play the role of transmitting electricity. This dual-track electricity price mechanism aims to promote the market-oriented reform of electricity and achieve flexibility and fairness in electricity prices.

[0038] The method can divide the instruction data set into a training set for model training and a validation set for model validation according to a ratio, wherein the ratio of the training set to the validation set can be 8:2, 7:3, etc.

[0039] Step S203, optimization processing step: obtaining a pre-trained model of a natural language processing task, training the pre-trained model using an instruction data set to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-trained model, and obtaining a power data model;

[0040] Among them, the pre-trained model can be a ChatGLM-10B model, which has been pre-trained on a large-scale Chinese corpus and can understand the characteristics of the Chinese language.

[0041] Among them, the cross entropy loss parameter is a commonly used loss function in machine learning, which is used to measure the difference between the model prediction result and the true label. The cross entropy loss is usually used in classification problems, especially multi-classification problems. It is calculated by comparing the probability distribution of the true label with the probability distribution of the model prediction, and calculating the loss value by comparing the difference between the two. The smaller the value of the cross entropy loss, the closer the model's prediction result is to the true label, and the better the model performance. The mean square error parameter is a statistic used to measure the degree of difference between the predicted value and the true value, and is usually used to measure the prediction accuracy of the model. It is calculated by averaging the square of the difference between each predicted value and the true value. The smaller the mean square error parameter, the higher the prediction accuracy of the model.

[0042] Step S204, deployment step: deploying the power data model to a question-and-answer system that is actually used in the power industry to answer questions related to the power industry.

[0043] Specifically, the fine-tuned power data model is deployed to the actual application of the power industry system question-answering system. The model deployment can be run on a server or integrated into a mobile application or web service.

[0044] Through this embodiment, through the acquisition step, the power industry data set is obtained, wherein the power industry data set includes public data of the power industry and private data within the power enterprise; the construction step: according to the power industry data set, the instructions for the preset tasks of the power industry are constructed to obtain the instruction data set, wherein the instructions for the preset tasks of the power industry include question-answering instructions, text classification instructions and summary generation instructions; the optimization processing step: obtaining the pre-trained model of the natural language processing task, using the instruction data set to train the pre-trained model to optimize at least one of the cross entropy loss parameter and the mean square error parameter of the pre-trained model, and obtaining the power data model; the deployment step: deploying the power data model to the question-answering system of the actual application of the power industry, for answering questions related to the power industry. By training the model with public data of the power industry and private data within the power enterprise, the model parameters are adjusted in a targeted manner, the model's understanding and adaptability to the power industry system is improved, and the time and computing resources required for model training are greatly reduced, thereby improving the training efficiency. It solves the problem that the existing technology is inefficient in utilizing data in specific fields, and the model cannot fully capture and utilize the professional knowledge of the power industry, thereby affecting the performance and accuracy of the model's actual application of the power industry system question-answering system.

[0045] In the specific implementation process, before using the instruction data set to train the pre-trained model to optimize at least one of the cross entropy loss parameters and mean square error parameters of the pre-trained model, the method also includes: obtaining the parameters of the weight matrix in the pre-trained attention mechanism of the pre-trained model, and decomposing the parameters of the weight matrix into a first low-rank matrix and a second low-rank matrix, wherein the product of the first low-rank matrix and the second low-rank matrix is ​​equal to the parameters of the weight matrix.

[0046] This method uses Lora (Low-Rank Adaptation) training to adjust fewer parameters. The core idea of ​​Lora is to introduce a low-rank structure into the weight matrix of the model, which is usually achieved by adding a learnable low-rank matrix. This low-rank matrix can be regarded as an adaptive transformation, which is multiplied by the original weights during the forward propagation of the model to adjust the effect of the weights. During the fine-tuning process, the original weights are kept unchanged, and only the first low-rank matrix A and the second low-rank matrix B are optimized. This allows the behavior of the model to be adjusted with fewer parameters and computational costs.

[0047] Specifically, the parameters of the weight matrix are decomposed into a first low-rank matrix and a second low-rank matrix, including: using the formula W0+ΔW=W0+BA to decompose the parameters of the weight matrix into a first low-rank matrix and a second low-rank matrix, wherein W0 is the weight matrix, ΔW is the parameter of the weight matrix, A is the first low-rank matrix, and B is the second low-rank matrix.

[0048] This method is used for the weight matrix in the pre-trained attention mechanism in the pre-trained model ChatGLM-10B. Fine-tune and decompose the weight matrix parameter ΔW through low-rank matrix decomposition. ΔW is decomposed into the product of two low-rank matrices, namely: W0+ΔW=W0+BA; where, and is a low-rank matrix, r is the rank number of the low rank, and r is usually much smaller than d and k.

[0049] More specifically, the pre-trained model is trained using an instruction data set to optimize at least one of the cross entropy loss parameter and the mean square error parameter of the pre-trained model to obtain a power data model, including: training the pre-trained model using an instruction data set; during the training process, adjusting the first low-rank matrix and the second low-rank matrix using a gradient descent optimization algorithm to optimize the mean square error parameter of the pre-trained model to obtain the power data model.

[0050] Among them, the gradient descent optimization algorithm is a commonly used mathematical optimization algorithm used to minimize the value of a function. The basic idea is to gradually adjust the parameters along the gradient direction of the function until the minimum point of the function is found. There are many variants of the gradient descent algorithm, including batch gradient descent, stochastic gradient descent, and mini-batch gradient descent. In each iteration, the algorithm calculates the gradient of the loss function with respect to the parameters and updates the parameters in the direction of the negative gradient. This iteration continues step by step until the stopping condition is met.

[0051] Furthermore, the first low-rank matrix and the second low-rank matrix are adjusted by using a gradient descent optimization algorithm to optimize the mean square error parameter of the pre-trained model, including: according to the formula Determine the mean square error parameter of the pre-trained model, where MSE is the mean square error parameter, Y i is the true value, W0 is the weight matrix, ΔW is the parameter of the weight matrix, A is the first low-rank matrix, B is the second low-rank matrix, x is the data of the instruction data set, and n is the number of data in the instruction data set.

[0052] During the fine-tuning process, this method only updates the decomposed low-rank matrix B and low-rank matrix A, while the original large parameter matrix remains unchanged. The forward propagation formula changes to: h = W0x + ΔWx = W0x + BAx; where x represents the training set data and h is the calculated value. The model is trained using the instruction data set of the power industry system, and the parameters of the low-rank matrix are updated through optimization algorithms such as gradient descent. The optimization target is cross entropy loss or mean square error, etc.

[0053] The formula for calculating the mean square error is: Among them, Y i represents the true value, MSE is the mean square error; the forward propagation formula is introduced into the error calculation formula, and the low-rank matrix B and the low-rank matrix A are adjusted to optimize the mean square error parameters of the model.

[0054] Furthermore, after the pre-trained model is trained using the instruction data set to optimize at least one of the cross entropy loss parameter and the mean square error parameter of the pre-trained model to obtain the power data model, the method also includes: using the instruction data set to perform a verification operation on the power data model to verify whether the accuracy of the power data model is greater than the accuracy threshold, whether the recall rate is greater than the recall rate threshold, and whether the F1 score is greater than the score threshold, wherein the F1 score is the harmonic mean of the accuracy and the recall rate; when it is verified that the power data model satisfies the accuracy greater than the accuracy threshold, the recall rate greater than the recall rate threshold, and the F1 score greater than the score threshold, it is determined that the power data model verification has passed.

[0055] Among them, the F1 score refers to the harmonic mean of the model's precision and recall. Precision is the proportion of samples predicted by the model to be positive that are actually positive, and recall is the proportion of all samples predicted to be positive by the model. The F1 score takes into account both precision and recall, and is usually used to balance the accuracy and completeness of the model. The F1 score ranges from 0 to 1. The closer it is to 1, the better the performance of the model. The score threshold can be set to 0.8 or 0.9. Among them, the accuracy threshold can be set to 90% or 95%, and the recall threshold can be set to 90% or 95%.

[0056] Specifically, after verifying whether the accuracy of the power data model is greater than the accuracy threshold, whether the recall rate is greater than the recall rate threshold, and whether the F1 score is greater than the score threshold, the method also includes: when verifying that the power data model satisfies at least one of the accuracy less than or equal to the accuracy threshold, the recall rate less than or equal to the recall rate threshold, and the F1 score less than or equal to the score threshold, determining that the power data model verification has failed, and continuing to train the power data model.

[0057] After the fine-tuning training phase, this method uses a validation set to comprehensively evaluate the performance of the model. During the evaluation process, a series of key indicators are set, including accuracy, recall, F1 score, etc. These indicators together provide a comprehensive perspective on the performance of the model. By monitoring these evaluation indicators, possible problems with the model can be discovered in a timely manner and adjusted accordingly. In addition, the model hyperparameters are tuned to optimize the model configuration. Hyperparameter tuning includes but is not limited to the adjustment of parameters such as learning rate, batch size, and number of layers to ensure that the model can achieve optimal performance on the institutional question-answering task in the power industry.

[0058] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the method for performing question and answer processing using an electric power data model of the present application will be described in detail below in combination with specific embodiments.

[0059] This embodiment relates to a specific method for question-answering processing using a power data model, such as Figure 3 As shown, including the following:

[0060] Collect public and private data sets related to the power industry. Preprocess the data, including cleaning, deduplication, and word segmentation. Build instruction data sets: Design instructions for specific tasks in the power industry, such as question answering, text classification, and summary generation.

[0061] For example: Question: What is the electricity price mechanism in the power industry? Answer: The power system adopts a dual-track electricity price mechanism of planning and market. Planned electricity is mainly based on historical electricity demand, future development plans, supply environment and policy environment to formulate power balance plans and power generation plans. Market electricity is in the power market, where the supply and demand sides directly negotiate electricity prices, and the power grid companies only play the role of transmitting electricity. This dual-track electricity price mechanism aims to promote the market-oriented reform of electricity and achieve flexibility and fairness in electricity prices.

[0062] Construct a dataset containing these instructions for model training and evaluation. Divide the preprocessed dataset into a training set and a validation set in a ratio (such as 8:2 or 7:3).

[0063] Then we use ChatGLM-10B, a pre-trained model suitable for Chinese natural language processing tasks. This model has been pre-trained on a large-scale Chinese corpus and can understand the characteristics of the Chinese language.

[0064] For the weight matrix in the pre-trained attention mechanism in the pre-trained model ChatGLM-10B Fine-tune and decompose the weight matrix parameter ΔW through low-rank matrix decomposition. ΔW is decomposed into the product of two low-rank matrices, namely: W0+ΔW=W0+BA;

[0065] in, and is a low-rank matrix, r is the rank number of the low rank, and r is usually much smaller than d and k.

[0066] During fine-tuning, only the decomposed low-rank matrices B and A are updated, while the original large parameter matrix remains unchanged. The forward propagation formula changes to: h = W0x + ΔWx = W0x + BAx;

[0067] Where x represents the training set data and h is the calculated value. The model is trained using the power industry system instruction data set, and the parameters of the low-rank matrix are updated through optimization algorithms such as gradient descent. The optimization target is cross entropy loss or mean square error, etc.

[0068] The formula for calculating the mean square error is:

[0069] Among them, Y i represents the true value, and MSE is the mean square error.

[0070] After the fine-tuning training phase is completed, the validation set is used to comprehensively evaluate the performance of the model. During the evaluation process, a series of key indicators are set, including accuracy, recall, F1 score, etc., where the F1 score refers to the harmonic mean of the model's precision and recall. Together, these indicators provide us with a comprehensive perspective on the performance of the model. By monitoring these evaluation indicators, possible problems with the model can be discovered in a timely manner and adjusted accordingly. In addition, the model hyperparameters are tuned to optimize the model configuration. Hyperparameter tuning includes but is not limited to the adjustment of parameters such as learning rate, batch size, and number of layers to ensure that the model can achieve optimal performance on the institutional question-answering task in the power industry.

[0071] The fine-tuned model is deployed to the actual application of the power industry system question answering system. The model deployment can be run on a server or integrated into a mobile application or web service.

[0072] This embodiment adopts the Lora (low-rank adaptive) parameter adjustment method, with the goal of fine-tuning quality and efficiency, and defines a Lora power industry system large model training method that integrates domain and private data. It solves technical problems such as data fusion, model customization, and training efficiency improvement. By integrating the power industry system and the internal system data of the enterprise, data silos are broken and the synergy between data is fully utilized; the Lora (low-rank adaptive) training method is used on the basis of the open source large model to maintain the versatility of the model while adjusting key parameters in a targeted manner to improve the model's understanding and adaptability to the power industry system; by adjusting fewer parameters, the time and computing resources required for training are greatly reduced, and the training efficiency is improved.

[0073] The embodiment of the present application also provides a device for question-and-answer processing using an electric power data model. It should be noted that the device for question-and-answer processing using an electric power data model in the embodiment of the present application can be used to execute the method for question-and-answer processing using an electric power data model provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been described will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0074] The following introduces a device for performing question and answer processing using a power data model provided in an embodiment of the present application.

[0075] Figure 4 Schematic diagram of a device for question-answering processing using a power data model according to an embodiment of the present application. Figure 4As shown, the device includes: an acquisition unit 41, a construction unit 42, a fine-tuning processing unit 43, and a deployment unit 44.

[0076] The acquisition unit 41 is used to perform an acquisition step: acquiring a power industry data set, wherein the power industry data set includes public data of the power industry and private data within the power enterprise;

[0077] Specifically, after obtaining the electric power industry data set, preprocessing operations are performed on the data of the electric power industry data set, including data cleaning, data deduplication, data segmentation and other preprocessing operations.

[0078] The construction unit 42 is used to perform the construction step: construct instructions for preset tasks in the power industry according to the power industry data set to obtain an instruction data set, wherein the instructions for preset tasks in the power industry include question-answering instructions, text classification instructions, and summary generation instructions;

[0079] The instruction data set may be divided into a training set for model training and a validation set for model validation according to a ratio, wherein the ratio of the training set to the validation set may be 8:2, 7:3, etc.

[0080] The fine-tuning processing unit 43 is used to perform an optimization processing step: obtaining a pre-trained model of a natural language processing task, training the pre-trained model using an instruction data set to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-trained model, and obtaining a power data model;

[0081] Among them, the cross entropy loss parameter is a commonly used loss function in machine learning, which is used to measure the difference between the model prediction result and the true label. The cross entropy loss is usually used in classification problems, especially multi-classification problems. It is calculated by comparing the probability distribution of the true label with the probability distribution of the model prediction, and calculating the loss value by comparing the difference between the two. The smaller the value of the cross entropy loss, the closer the model's prediction result is to the true label, and the better the model performance. The mean square error parameter is a statistic used to measure the degree of difference between the predicted value and the true value, and is usually used to measure the prediction accuracy of the model. It is calculated by averaging the square of the difference between each predicted value and the true value. The smaller the mean square error parameter, the higher the prediction accuracy of the model.

[0082] The deployment unit 44 is used to perform the deployment step: deploying the power data model to a question-answering system that is actually applied in the power industry to answer questions related to the power industry.

[0083] Specifically, the fine-tuned power data model is deployed to the actual application of the power industry system question-answering system. The model deployment can be run on a server or integrated into a mobile application or web service.

[0084] In this embodiment, the acquisition unit is used to perform the acquisition step: acquiring the power industry data set, wherein the power industry data set includes public data of the power industry and private data within the power enterprise; the construction unit is used to perform the construction step: constructing instructions for preset tasks of the power industry according to the power industry data set, and obtaining an instruction data set, wherein the instructions for preset tasks of the power industry include question-answering instructions, text classification instructions, and summary generation instructions; the fine-tuning processing unit is used to perform the optimization processing step: obtaining a pre-trained model for a natural language processing task, using the instruction data set to train the pre-trained model to optimize at least one of the cross entropy loss parameter and mean square error parameter of the pre-trained model, and obtaining a power data model; the deployment unit is used to perform the deployment step: deploying the power data model to the question-answering system of the actual power industry for answering questions related to the power industry. By using the public data of the power industry and the private data within the power enterprise to train the model, the model parameters are adjusted in a targeted manner, the model's understanding and adaptability to the power industry system is improved, and the time and computing resources required for model training are greatly reduced, thereby improving the training efficiency. It solves the problem that the existing technology is inefficient in utilizing data in specific fields, and the model is unable to fully capture and utilize the expertise of the power industry, thus affecting the performance and accuracy of the model's actual application of the power industry institutional question-and-answer system.

[0085] As an optional scheme, the device also includes a decomposition unit, which is used to obtain the parameters of the weight matrix in the pre-trained attention mechanism of the pre-trained model before using the instruction data set to train the pre-trained model to optimize at least one of the cross entropy loss parameters and the mean square error parameters of the pre-trained model, and decompose the parameters of the weight matrix into a first low-rank matrix and a second low-rank matrix, wherein the product of the first low-rank matrix and the second low-rank matrix is ​​equal to the parameters of the weight matrix.

[0086] Specifically, the Lora (Low-Rank Adaptation) training method is used to adjust fewer parameters. The core idea of ​​Lora is to introduce a low-rank structure into the weight matrix of the model, which is usually achieved by adding a learnable low-rank matrix. This low-rank matrix can be regarded as an adaptive transformation, which is multiplied by the original weights during the forward propagation of the model to adjust the effect of the weights. In the process of fine-tuning, the original weights are kept unchanged, and only the first low-rank matrix A and the second low-rank matrix B are optimized, so that the behavior of the model can be adjusted with fewer parameters and computational costs.

[0087] An optional solution, the decomposition unit includes a decomposition module, which is used to decompose the parameters of the weight matrix into a first low-rank matrix and a second low-rank matrix using the formula W0+ΔW=W0+BA, wherein W0 is the weight matrix, ΔW is the parameters of the weight matrix, A is the first low-rank matrix, and B is the second low-rank matrix.

[0088] Specifically, for the weight matrix in the pre-trained attention mechanism in the pre-trained model ChatGLM-10B Fine-tune and decompose the weight matrix parameter ΔW through low-rank matrix decomposition. ΔW is decomposed into the product of two low-rank matrices, namely: W0+ΔW=W0+BA; where, and is a low-rank matrix, r is the rank number of the low rank, and r is usually much smaller than d and k.

[0089] An optional solution, the optimization processing unit includes a training module and an adjustment module; the training module is used to train the pre-trained model using an instruction data set; the adjustment module is used to adjust the first low-rank matrix and the second low-rank matrix using a gradient descent optimization algorithm during the training process to optimize the mean square error parameters of the pre-trained model to obtain a power data model.

[0090] Among them, the gradient descent optimization algorithm is a commonly used mathematical optimization algorithm used to minimize the value of a function. The basic idea is to gradually adjust the parameters along the gradient direction of the function until the minimum point of the function is found. There are many variants of the gradient descent algorithm, including batch gradient descent, stochastic gradient descent, and mini-batch gradient descent. In each iteration, the algorithm calculates the gradient of the loss function with respect to the parameters and updates the parameters in the direction of the negative gradient. This iteration continues step by step until the stopping condition is met.

[0091] In an optional solution, the adjustment module includes a determination submodule for determining the Determine the mean square error parameter of the pre-trained model, where MSE is the mean square error parameter, Y i is the true value, W0 is the weight matrix, ΔW is the parameter of the weight matrix, A is the first low-rank matrix, B is the second low-rank matrix, x is the data of the instruction data set, and n is the number of data in the instruction data set.

[0092] Specifically, during the fine-tuning process, only the decomposed low-rank matrix B and low-rank matrix A are updated, while the original large parameter matrix remains unchanged. The forward propagation formula changes to: h = W0x + ΔWx = W0x + BAx; where x represents the training set data and h is the calculated value. The model is trained using the instruction data set of the power industry system, and the parameters of the low-rank matrix are updated through optimization algorithms such as gradient descent. The optimization target is cross entropy loss or mean square error, etc.

[0093] The formula for calculating the mean square error is: Among them, Y i represents the true value, and MSE is the mean square error.

[0094] In an optional scheme, the device also includes a verification unit and a first determination unit; the verification unit is used to train the pre-trained model using the instruction data set to optimize at least one of the cross entropy loss parameter and the mean square error parameter of the pre-trained model to obtain the power data model, and then use the instruction data set to perform a verification operation on the power data model to verify whether the accuracy of the power data model is greater than the accuracy threshold, whether the recall rate is greater than the recall rate threshold, and whether the F1 score is greater than the score threshold, wherein the F1 score is the harmonic mean of the accuracy and the recall rate; the first determination unit is used to determine that the power data model verification has passed when verifying that the power data model satisfies the accuracy greater than the accuracy threshold, the recall rate greater than the recall rate threshold, and the F1 score greater than the score threshold.

[0095] Specifically, the F1 score refers to the harmonic mean of the model's precision and recall. Precision is the proportion of samples predicted by the model to be positive that are actually positive, and recall is the proportion of all samples predicted to be positive by the model. The F1 score takes into account both precision and recall, and is usually used to balance the accuracy and completeness of the model. The F1 score ranges from 0 to 1. The closer it is to 1, the better the performance of the model. The score threshold can be set to 0.8 or 0.9. Among them, the accuracy threshold can be set to 90% or 95%, and the recall threshold can be set to 90% or 95%.

[0096] In an optional scheme, the device also includes a second determination unit, which is used to determine whether the power data model verification has failed and continue to train the power data model after verifying whether the accuracy of the power data model is greater than the accuracy threshold, whether the recall rate is greater than the recall rate threshold, and whether the F1 score is greater than the score threshold. If the power data model satisfies at least one of the accuracy less than or equal to the accuracy threshold, the recall rate is less than or equal to the recall rate threshold, and the F1 score is less than or equal to the score threshold.

[0097] Specifically, after the fine-tuning training phase was completed, the validation set was used to comprehensively evaluate the performance of the model. During the evaluation process, a series of key indicators were set, including accuracy, recall, F1 score, etc. These indicators together provide a comprehensive perspective on the performance of the model. By monitoring these evaluation indicators, possible problems with the model can be discovered in a timely manner and adjusted accordingly. In addition, the model hyperparameters are tuned to optimize the model configuration. Hyperparameter tuning includes but is not limited to the adjustment of parameters such as learning rate, batch size, and number of layers to ensure that the model can achieve optimal performance on the institutional question-answering task in the power industry.

[0098] The apparatus for question-answering processing using the power data model includes a processor and a memory, wherein the acquisition unit, construction unit, fine-tuning processing unit, deployment unit, etc. are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0099] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to solve the problem that the existing technology is inefficient in utilizing data in specific fields, and the model cannot fully capture and utilize the professional knowledge of the power industry, thus affecting the performance and accuracy of the model in the actual application of the power industry system question-answering system.

[0100] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0101] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute the method of applying the power data model to perform question and answer processing.

[0102] Specifically, the method of applying the power data model to perform question-answering processing includes:

[0103] Step S201, acquisition step: acquiring a power industry data set, wherein the power industry data set includes public data of the power industry and private data within power enterprises;

[0104] Step S202, construction step: constructing instructions for preset tasks in the power industry according to the power industry data set to obtain an instruction data set, wherein the instructions for preset tasks in the power industry include question-answering instructions, text classification instructions, and summary generation instructions;

[0105] Step S203, optimization processing step: obtaining a pre-trained model of a natural language processing task, training the pre-trained model using an instruction data set to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-trained model, and obtaining a power data model;

[0106] Step S204, deployment step: deploying the power data model to a question-and-answer system that is actually used in the power industry to answer questions related to the power industry.

[0107] An embodiment of the present invention provides a processor, which is used to run a program, wherein the method of applying a power data model to perform question and answer processing is executed when the program is running.

[0108] Specifically, the method of applying the power data model to perform question-answering processing includes:

[0109] Step S201, acquisition step: acquiring a power industry data set, wherein the power industry data set includes public data of the power industry and private data within power enterprises;

[0110] Step S202, construction step: constructing instructions for preset tasks in the power industry according to the power industry data set to obtain an instruction data set, wherein the instructions for preset tasks in the power industry include question-answering instructions, text classification instructions, and summary generation instructions;

[0111] Step S203, optimization processing step: obtaining a pre-trained model of a natural language processing task, training the pre-trained model using an instruction data set to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-trained model, and obtaining a power data model;

[0112] Step S204, deployment step: deploying the power data model to a question-and-answer system that is actually used in the power industry to answer questions related to the power industry.

[0113] An embodiment of the present invention provides a device, the device including a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0114] Step S201, acquisition step: acquiring a power industry data set, wherein the power industry data set includes public data of the power industry and private data within power enterprises;

[0115] Step S202, construction step: constructing instructions for preset tasks in the power industry according to the power industry data set to obtain an instruction data set, wherein the instructions for preset tasks in the power industry include question-answering instructions, text classification instructions, and summary generation instructions;

[0116] Step S203, optimization processing step: obtaining a pre-trained model of a natural language processing task, training the pre-trained model using an instruction data set to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-trained model, and obtaining a power data model;

[0117] Step S204, deployment step: deploying the power data model to a question-and-answer system that is actually used in the power industry to answer questions related to the power industry.

[0118] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0119] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0120] Step S201, acquisition step: acquiring a power industry data set, wherein the power industry data set includes public data of the power industry and private data within power enterprises;

[0121] Step S202, construction step: constructing instructions for preset tasks in the power industry according to the power industry data set to obtain an instruction data set, wherein the instructions for preset tasks in the power industry include question-answering instructions, text classification instructions, and summary generation instructions;

[0122] Step S203, optimization processing step: obtaining a pre-trained model of a natural language processing task, training the pre-trained model using an instruction data set to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-trained model, and obtaining a power data model;

[0123] Step S204, deployment step: deploying the power data model to a question-and-answer system that is actually used in the power industry to answer questions related to the power industry.

[0124] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0125] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0126] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0130] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0131] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0132] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0133] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0134] 1) A method for question-answering processing using a power data model of the present application includes: an acquisition step, obtaining a power industry data set, wherein the power industry data set includes public data of the power industry and private data within power enterprises; a construction step: constructing instructions for preset tasks of the power industry according to the power industry data set to obtain an instruction data set, wherein the instructions for preset tasks of the power industry include question-answering instructions, text classification instructions, and summary generation instructions; an optimization processing step: obtaining a pre-trained model for a natural language processing task, using the instruction data set to train the pre-trained model to optimize at least one of the cross entropy loss parameter and mean square error parameter of the pre-trained model, and obtaining a power data model; a deployment step: deploying the power data model to a question-answering system for the actual application of the power industry, for answering questions related to the power industry. By using the public data of the power industry and the private data within the power enterprise to train the model, the model parameters are adjusted in a targeted manner, the model's understanding and adaptability to the power industry system is improved, and the time and computing resources required for model training are greatly reduced, thereby improving the training efficiency. It solves the problem that the existing technology is inefficient in utilizing data in specific fields, and the model is unable to fully capture and utilize the expertise of the power industry, thus affecting the performance and accuracy of the model's actual application of the power industry institutional question-and-answer system.

[0135] 2) A device for question-answering processing using a power data model of the present application includes: an acquisition unit for executing an acquisition step: acquiring a power industry data set, wherein the power industry data set includes public data of the power industry and private data within the power enterprise; a construction unit for executing a construction step: constructing instructions for preset tasks of the power industry according to the power industry data set to obtain an instruction data set, wherein the instructions for preset tasks of the power industry include question-answering instructions, text classification instructions, and summary generation instructions; a fine-tuning processing unit for executing an optimization processing step: acquiring a pre-trained model for a natural language processing task, training the pre-trained model using the instruction data set to optimize at least one of the cross entropy loss parameter and mean square error parameter of the pre-trained model, and obtaining a power data model; a deployment unit for executing a deployment step: deploying the power data model to a question-answering system for the actual application power industry, for answering questions related to the power industry. By using the public data of the power industry and the private data within the power enterprise to train the model, the model parameters are adjusted in a targeted manner, the model's understanding and adaptability to the power industry system is improved, and the time and computing resources required for model training are greatly reduced, thereby improving training efficiency. It solves the problem that the existing technology is inefficient in utilizing data in specific fields, and the model is unable to fully capture and utilize the expertise of the power industry, thus affecting the performance and accuracy of the model's actual application of the power industry institutional question-and-answer system.

[0136] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for question-answering processing using a power data model, characterized in that: include: Acquisition step: acquiring a power industry data set, wherein the power industry data set includes public data of the power industry and private data within power enterprises; Construction step: constructing instructions for preset tasks in the power industry according to the power industry data set to obtain an instruction data set, wherein the instructions for preset tasks in the power industry include question-answering instructions, text classification instructions, and summary generation instructions; Optimization processing step: obtaining a pre-trained model of a natural language processing task, using the instruction data set to train the pre-trained model to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-trained model, and obtaining a power data model; Deployment step: deploy the power data model to a question-answering system that is actually used in the power industry to answer questions related to the power industry.

2. The method according to claim 1, characterized in that Before using the instruction data set to train the pre-trained model to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-trained model, the method further includes: Obtain parameters of a weight matrix in a pre-trained attention mechanism of the pre-trained model, and decompose the parameters of the weight matrix into a first low-rank matrix and a second low-rank matrix, wherein the product of the first low-rank matrix and the second low-rank matrix is ​​equal to the parameters of the weight matrix.

3. The method according to claim 2, characterized in that Decomposing the parameters of the weight matrix into a first low-rank matrix and a second low-rank matrix includes: The formula W0+ΔW=W0+BA is used to decompose the parameters of the weight matrix into the first low-rank matrix and the second low-rank matrix, wherein W0 is the weight matrix, ΔW is the parameter of the weight matrix, A is the first low-rank matrix, and B is the second low-rank matrix.

4. The method according to claim 2, characterized in that: The pre-trained model is trained using the instruction data set to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-trained model to obtain a power data model, including: Using the instruction data set to train the pre-trained model; During the training process, a gradient descent optimization algorithm is used to adjust the first low-rank matrix and the second low-rank matrix to optimize the mean square error parameter of the pre-trained model to obtain the power data model.

5. The method according to claim 4, characterized in that The first low-rank matrix and the second low-rank matrix are adjusted using a gradient descent optimization algorithm to optimize the mean square error parameter of the pre-trained model, including: According to the formula Determine the mean square error parameter of the pre-trained model, where MSE is the mean square error parameter, Y i is the true value, W0 is the weight matrix, ΔW is the parameter of the weight matrix, A is the first low-rank matrix, B is the second low-rank matrix, x is the data of the instruction data set, and n is the number of data in the instruction data set.

6. The method according to claim 1, characterized in that After the pre-trained model is trained using the instruction data set to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-trained model to obtain a power data model, the method further includes: Using the instruction data set to perform a verification operation on the power data model, verifying whether the accuracy of the power data model is greater than an accuracy threshold, whether the recall rate is greater than a recall threshold, and whether the F1 score is greater than a score threshold, wherein the F1 score is a harmonic mean of the accuracy and the recall rate; When it is verified that the power data model satisfies the accuracy greater than the accuracy threshold, the recall greater than the recall threshold, and the F1 score greater than the score threshold, it is determined that the power data model verification has passed.

7. The method according to claim 6, characterized in that After verifying whether the accuracy of the power data model is greater than the accuracy threshold, whether the recall rate is greater than the recall rate threshold, and whether the F1 score is greater than the score threshold, the method further includes: When it is verified that the power data model satisfies at least one of the accuracy being less than or equal to the accuracy threshold, the recall being less than or equal to the recall threshold, and the F1 score being less than or equal to the score threshold, it is determined that the power data model verification has failed, and the power data model continues to be trained.

8. A device for generating a system model for the electric power industry, characterized in that: include: The acquisition unit is used to perform the acquisition step: acquiring a power industry data set, wherein the power industry data set includes public data of the power industry and private data within the power enterprise; The construction unit is used to perform the construction step: construct instructions for preset tasks in the power industry according to the power industry data set to obtain an instruction data set, wherein the instructions for preset tasks in the power industry include question-answering instructions, text classification instructions, and summary generation instructions; The fine-tuning processing unit is used to perform an optimization processing step: obtaining a pre-trained model for a natural language processing task, training the pre-trained model using the instruction data set to optimize at least one of a cross entropy loss parameter and a mean square error parameter of the pre-trained model, and obtaining a power data model; The deployment unit is used to execute the deployment step: deploying the power data model to a question-answering system that is actually applied in the power industry to answer questions related to the power industry.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for question-and-answer processing using an electric power data model as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for performing question and answer processing using an electric power data model as described in any one of claims 1 to 7.

Images