Electric power knowledge extraction method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an electric power knowledge extraction method and device, electronic equipment and a storage medium. Constructing a power knowledge data set; based on the electric power knowledge data set, performing parameter fine tuning on the basic language large model through a low-rank adaptive method to obtain an electric power knowledge extraction model; and performing knowledge extraction on the electric power knowledge through the electric power knowledge extraction model. According to the embodiment of the invention, the knowledge extraction accuracy and efficiency of the electric power knowledge are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to information processing technology, and in particular to a method, device, electronic device and storage medium for extracting power knowledge. Background Art

[0002] Electricity knowledge, especially electricity safety knowledge, is a basic skill that power industry workers must master. It is not only related to personal safety, but also to the safe and stable operation of the entire power system. Therefore, effective methods are needed to manage and utilize the large amount of accumulated electricity safety data, that is, electricity safety knowledge, to improve work efficiency, reduce accident risks and enhance overall safety.

[0003] Knowledge extraction technology can help automate the processing of large amounts of document data. Existing technologies usually perform knowledge extraction through deep learning algorithms.

[0004] However, traditional deep learning algorithms require a large amount of high-quality sample data for training. Otherwise, it is difficult for the model to fully learn the specific knowledge in the professional field, and there is a certain risk of overfitting, making it difficult to ensure the accuracy and efficiency of knowledge extraction. Summary of the invention

[0005] The present application provides a method, device, electronic device and storage medium for extracting electric power knowledge to improve the accuracy and efficiency of extracting electric power safety knowledge.

[0006] In a first aspect, an embodiment of the present application provides a method for extracting electric power knowledge, the method comprising:

[0007] Constructing power knowledge dataset;

[0008] Based on the power knowledge dataset, the parameters of the basic language model are fine-tuned through a low-rank adaptive method to obtain a power knowledge extraction model.

[0009] The power knowledge is extracted through the power knowledge extraction model.

[0010] In a second aspect, an embodiment of the present application further provides a power knowledge extraction device, the power knowledge extraction device comprising:

[0011] An electric power knowledge data set construction module is used to construct an electric power knowledge data set;

[0012] The parameter fine-tuning module is used to fine-tune the parameters of the basic language large model based on the power knowledge dataset through a low-rank adaptive method to obtain a power knowledge extraction model;

[0013] The power knowledge extraction module is used to extract power knowledge through the power knowledge extraction model.

[0014] In a third aspect, an embodiment of the present application further provides an electronic device, the electronic device comprising:

[0015] one or more processors;

[0016] A storage device for storing one or more programs;

[0017] When one or more programs are executed by one or more processors, the one or more processors implement any one of the power knowledge extraction methods provided in the embodiments of the present application.

[0018] In a fourth aspect, an embodiment of the present application further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute any one of the power knowledge extraction methods provided in the embodiments of the present application.

[0019] This application constructs an electric power knowledge data set; provides professional sample data for fine-tuning parameters of the basic language large model; based on the electric power knowledge data set, fine-tunes the parameters of the basic language large model through a low-rank adaptive method to obtain an electric power knowledge extraction model; based on the powerful general language understanding ability and stronger generalization of the basic language large model, fine-tunes the parameters through a low-rank adaptive method, so that the obtained electric power knowledge extraction model has professional knowledge in the electric power field, and reduces the number of parameters that need to be updated during the fine-tuning process, thereby improving the fine-tuning efficiency; extracts electric power knowledge through the electric power knowledge extraction model, which can improve the accuracy and efficiency of knowledge extraction of electric power knowledge. Therefore, through the technical solution of this application, the problem that it is difficult for the model to fully learn the specific knowledge in the professional field and it is difficult to ensure the accuracy and efficiency of knowledge extraction is solved, and the accuracy and efficiency of knowledge extraction of electric power knowledge are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a method for extracting power knowledge in Embodiment 1 of the present application;

[0021] Figure 2 is a flow chart of a method for extracting power knowledge in Embodiment 2 of the present application;

[0022] Figure 3 It is a structural schematic diagram of a power knowledge extraction device in Embodiment 3 of the present application;

[0023] Figure 4 It is a structural diagram of an electronic device in Embodiment 4 of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first" and "second" etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Embodiment 1

[0027] Figure 1 A flow chart of a method for extracting electric power knowledge is provided for the first embodiment of the present application. This embodiment is applicable to the situation where safety knowledge of the electric power system is extracted. The method can be executed by an electric power knowledge extraction device, which can be implemented in software and / or hardware and specifically configured in an electronic device.

[0028] See also Figure 1 The power knowledge extraction method shown in the figure specifically includes the following steps:

[0029] S110. Construct an electric power knowledge dataset.

[0030] The power knowledge dataset is used to train the basic language model so that the basic language model can learn power knowledge, such as power safety knowledge, and improve the accuracy of knowledge extraction of power knowledge. Knowledge extraction technology mainly includes entity extraction, relationship extraction and event extraction. Therefore, the power knowledge dataset includes original files, as well as entity labels, relationship labels and event labels. For example, historical cases, regulations and operating specifications related to power safety can be collected, and relevant entities, relationships and events can be extracted manually or through certain algorithms for annotation.

[0031] In an optional embodiment, constructing an electric power knowledge data set includes: obtaining original electric power data from multiple sources; the source of the original electric power data includes at least one of laws and regulations, operating manuals, safety specifications and safety cases; annotating entities, relationships and events in the original electric power data to obtain the electric power knowledge data set.

[0032] The original power data may be relevant power data obtained through existing documents from various sources, and the source of the original power data may include at least one of laws and regulations, operation manuals, safety specifications, and safety cases.

[0033] The power industry is a basic industry. The safety of power production is not only related to the benefits and development of the enterprise itself, but also affects the stability of the entire society, economic development and the normal life of the people. Therefore, it is the top priority of power companies to do a good job in safety knowledge education for production personnel and ensure safe production. It is also an important measure to ensure the safety and health of operators during the production process.

[0034] The country has formulated a series of legal and regulatory documents in response to the safety characteristics of the power industry: the National Energy Administration revised the "Regulations on Supervision and Management of Power Safety Hazards" in 2022, aiming to strengthen the investigation, management and supervision and management of power safety hazards; "Power Safety Knowledge and Cases" published by China Electric Power Press describes power safety knowledge in detail, including electrical safety equipment, electrical operation safety technology, electrical maintenance safety technology, electrical testing and measurement work safety technology, power line operation and maintenance safety technology, live working safety technology and electrical fire and explosion prevention, etc.; there are also some related safety specification documents, such as the "Safety Specifications for Power Engineering Construction" and "Power System Operation Safety Regulations". These legal and regulatory documents and safety specification documents together constitute the power industry safety knowledge system, providing legal protection and guidance for the standardized operation and safe operation of the power industry.

[0035] In addition, you can also collect power safety knowledge through some operation manuals and safety case books to obtain original power data. According to the "Power Safety Supervision Work Manual" compiled by Zhang Ruibing et al., "Power Safety Knowledge and Cases" compiled by Gu Biao, and "New Employee Safety Education" compiled by Shanxi Electric Power Company and other safety knowledge books, all the power safety knowledge involved is sorted out, including the Work Safety Law, the Electricity Law, the State Grid Corporation's Work Safety Work Regulations, daily safety work, key safety work, special work, major safety risk prevention and control, and various electrical equipment knowledge and safety knowledge cases, etc., to obtain original power data. You can also obtain original power data through some historical cases.

[0036] The knowledge extraction task mainly includes three subtasks: entity extraction, relationship extraction, and event extraction. By labeling the entities, relationships, and events in the original power data, a power knowledge data set can be obtained. The labeling of entities, relationships, and events in the original power data can be done manually or by algorithm, and this application does not make specific restrictions on this. For example, taking the substation equipment relationship in the power grid equipment as an example, the entity types include substation, voltage, capacity, and switch, etc., the event types include overvoltage and normal operation, etc., and the relationships between entities include affiliation, non-inclusion, and topological relationships.

[0037] Entity extraction, also known as named entity recognition (NER), is the process of identifying specific types of entities from unstructured text, such as place names, organization names, time and values. In the power grid industry, there are many types of entities, such as equipment entities (transformers and lines, etc.), parameter entities (voltage and current frequency, etc.), geographical locations (such as cities, regions and substations, etc.), organizational entities (grid companies and dispatching centers, etc.), time entities (date and time, etc.), and event entities (faults and power outages, etc.). Entity extraction technology continues to develop, from traditional rule matching to unsupervised learning, and then from unsupervised learning to supervised learning based on feature engineering, and finally to research methods based on deep learning. For example, rule-based methods can extract relations through predefined grammar and vocabulary rules; supervised learning methods can use labeled data sets to train models for relation extraction; and semi-supervised and unsupervised learning can extract relations with a small amount of labeled data or no labeled data. For example, entity extraction can be performed based on semantic similarity and mutual information maximization, and the automatic extraction of unstructured power grid regulation text knowledge can be achieved based on the deep learning model of attention-based bidirectional long short-term memory network and conditional random field.

[0038] Entity extraction technology plays an important role in the power grid industry. It helps to extract valuable information from a large amount of complex power grid data and supports the intelligent management and operation of the power grid. Therefore, it is necessary to annotate the entities in the original power data so that the power knowledge extraction model has entity extraction capabilities. Entity annotation can be achieved through manual annotation or through the above-mentioned entity extraction algorithm.

[0039] Relation extraction is a basic task in knowledge extraction, which mainly extracts the semantic relationship between entities in the text. Relation extraction in the power grid industry refers to the process of identifying and extracting the relationship between entities from power grid-related data sources. These relationships can be connection relationships between equipment (such as lines connecting transformers), relationships between operations and results (such as maintenance operations causing equipment downtime), event causal relationships (such as lightning strikes causing line failures), equipment parameter relationships (such as excessive current affecting transformer operating efficiency) and organizational hierarchical relationships (such as the dispatch center managing multiple substations), or the relationship between power grid events and other entities.

[0040] Relation extraction technology mainly includes three aspects: rule-based methods, machine learning-based methods, and deep learning-based methods. Rule-based methods can use predefined rules and patterns to match entities and relationships in text. Machine learning-based methods can include traditional machine learning methods, such as support vector machines and decision trees. Deep learning-based methods can include convolutional neural networks, recursive neural networks, and attention mechanisms.

[0041] The application of relationship extraction technology in the power grid industry not only helps to improve the level of intelligent power grid management, but also provides strong support for the safe and stable operation of the power grid. By automatically identifying and understanding the relationship between power grid entities, power grid monitoring, fault analysis, and decision support can be more effective. Therefore, it is necessary to annotate the relationships in the original power data so that the power knowledge extraction model has the ability to extract relationships. Relationship annotation can be achieved through manual annotation or through the above-mentioned relationship extraction algorithm.

[0042] Event extraction is a core component of knowledge extraction, which aims to identify event information in text and structure the event information. In the power grid industry, event extraction can be used to automatically extract key information from various documents for subsequent analysis and utilization. The basic steps of event extraction are as follows: entity recognition, such as equipment name, fault type and location; event trigger word recognition, determining which words in the text can trigger the occurrence of events, such as "fault" and "replacement"; event element extraction, identifying elements related to the event, such as time, location and participants; event classification, classifying events into predefined event types, such as "equipment failure" or "grid outage"; relationship extraction, determining the relationship between event elements and constructing a complete event description. Therefore, it is necessary to annotate the events in the original power data so that the power knowledge extraction model has event extraction capabilities. Event annotation can be achieved through manual annotation or through the above-mentioned event extraction algorithm.

[0043] By acquiring original power data from multiple sources; the source of original power data includes at least one of laws and regulations, operating manuals, safety specifications and safety cases, original power data can be acquired from multiple aspects to improve the richness of original power data; the entities, relationships and events in the original power data are labeled to obtain a power knowledge data set, so that the power knowledge extraction model trained by the power knowledge data set has the ability to extract entities, relationships and events.

[0044] S120. Based on the electric power knowledge dataset, the parameters of the basic language large model are fine-tuned through a low-rank adaptive method to obtain an electric power knowledge extraction model.

[0045] Parameter fine-tuning can be fine-tuning the parameters of the basic language large model through the power knowledge data set, so that the basic language large model can learn professional knowledge related to power knowledge. The basic language large model has achieved good results in general fields and has learned a wide range of language knowledge, but for specific fields and downstream tasks, the performance and accuracy of the basic language large model are still insufficient. Exemplarily, the basic language large model can be the Chinese Llama2-7B model (professional term, a large model) optimized and improved for the Chinese context. In order to achieve better results, parameter fine-tuning can be used to adjust the parameters of the basic language large model so that the basic language large model can adapt to the data related to the power knowledge extraction task. Fine-tuning can not only enable the basic language large model to understand specific fields, but also save resource efficiency and more effectively meet the needs of use.

[0046] The low-rank adaptive method can be a parameter fine-tuning technology for a basic language large model, which is used to adapt the basic language large model to the power knowledge extraction task. Since the basic language large model has a large number of parameters, if the full parameter fine-tuning technology is used, it will consume a lot of resources. Therefore, the use of efficient parameter fine-tuning methods can better adapt to downstream tasks under existing resources. The low-rank adaptive method simulates small changes in parameters by introducing low-rank matrices. These low-rank matrices can capture the main changes in the parameters of the basic language large model during the fine-tuning process. Since the number of parameters of the low-rank matrix is ​​relatively small, the low-rank adaptive technology only needs to add a small number of parameters to the fine-tuning task, which can greatly reduce the number of parameters that need to be updated during the fine-tuning process.

[0047] The power knowledge extraction model can be a large model with power knowledge extraction capability, which is used for power safety knowledge extraction. The power knowledge dataset is used as sample data, and the parameters of the basic language large model are fine-tuned through a low-rank adaptive method to obtain the power knowledge extraction model.

[0048] S130. Extracting power knowledge through a power knowledge extraction model.

[0049] Knowledge extraction can refer to the process of extracting knowledge from unstructured text into a structured form by identifying and understanding the text in the power knowledge data. For example, the structured form can be a label or attribute with richer semantic information, and the scenarios widely used in the industry can include content understanding and clause understanding. Knowledge extraction is a basic core task of natural language processing. The power knowledge extraction model obtains the user's questions through the question dialog box on the page, obtains the user's questions, and performs knowledge extraction of power knowledge.

[0050] Electricity knowledge, especially electricity safety knowledge, is a basic skill that must be mastered by power industry workers. It is not only related to personal safety, but also to the safe and stable operation of the entire power system. However, at the same time, electricity safety knowledge has multiple characteristics such as professionalism, complexity, standardization, dynamism, practicality and systematicity, which makes it not so easy for production personnel to master. Professionalism can mean that electricity safety knowledge involves professional knowledge of the power system, including equipment operation, maintenance and troubleshooting, and covers professional knowledge of multiple disciplines such as electrical engineering, physics and materials science. Therefore, its content is highly specialized and technical. Complexity can mean that the power system is a complex system, and its safety knowledge covers multiple levels, such as technology, management and environment. Specific safety knowledge includes detailed procedures, operating guidelines and emergency handling steps, etc. These details are crucial to the safety of production personnel and equipment and have strong complexity. Standardization can mean that electricity safety knowledge must comply with relevant laws and regulations, and strictly follow industry standards and specifications to ensure legal and compliant operations. Dynamicity can mean that with the advancement of technology and the introduction of new standards, electricity safety knowledge needs to be continuously updated and improved to adapt to new safety needs and technical requirements. Practicality means that power safety knowledge requires not only theoretical understanding, but also emphasizes application and practice in actual operations. Many safety knowledge and operating procedures need to be flexibly applied according to specific situations, such as different weather conditions and equipment types. Systematicity means that power safety knowledge is not isolated, and it needs to consider the interaction and impact between various components from a system perspective. Because of so many characteristics, the use of knowledge extraction technology is very necessary for power safety knowledge, which is of great help in training, employment, and decision-making assistance.

[0051] Based on the above characteristics of power safety knowledge, it is very necessary to use knowledge extraction technology. The use of knowledge extraction technology can not only improve the efficiency and quality of power safety management, but also promote the intelligent development of the power industry and ensure the safe and stable operation of the power system. By extracting knowledge from power knowledge, the following effects can be achieved: high efficiency, high accuracy, auxiliary decision-making, promotion of sharing, strengthening emergency response and promotion of intelligence. Improving efficiency can mean: the amount of power safety knowledge is large and scattered. Through the power knowledge extraction model, key information can be automatically extracted from a large amount of text, improving the efficiency of knowledge organization and application. Improving accuracy can mean: manual organization of power safety knowledge is prone to errors, while knowledge extraction through the power knowledge extraction model can reduce human errors and ensure the accuracy and reliability of knowledge. Assisting decision-making can mean: extracting key safety knowledge through the power knowledge extraction model can provide data support for the operation and maintenance, fault diagnosis and risk management of the power system, and help make more informed decisions. Promoting sharing can mean: unstructured text data can be converted into structured data through knowledge extraction of the power knowledge extraction model, which helps to convert implicit knowledge into explicit knowledge, facilitates sharing and dissemination within the organization, and improves the overall safety level. Strengthening emergency response can mean: in an emergency, quickly extracting relevant safety knowledge through the power knowledge extraction model can guide emergency response and reduce accident losses. Promoting intelligence can mean: building the foundation of the intelligent power system through the power knowledge extraction model, which provides the necessary data and knowledge support for the intelligent monitoring, diagnosis and maintenance of the power system.

[0052] Electricity safety knowledge is highly professional, complex, and updated quickly. Using the electricity knowledge extraction model to extract knowledge can not only improve the speed and accuracy of information processing, but also improve efficiency, assist decision-making, and promote intelligence, thereby effectively improving the safety and operational efficiency of the power system.

[0053] With the increasing complexity of power systems and the development of technology, it is particularly important to ensure the safe operation of power systems. Power safety refers to ensuring the safety of personnel, equipment and the environment during the production, transmission, distribution and use of power systems. Specifically, it can include preventing accidents such as electrical fires and electric shock injuries, and ensuring the stable operation of power systems. Therefore, there is power safety knowledge in various aspects, including equipment operation, environmental operation, operating specifications and emergency response. Therefore, power companies and research institutions need effective methods to manage and utilize the large amount of accumulated power safety data, that is, power safety knowledge, to improve work efficiency, reduce accident risks and improve overall safety.

[0054] In the field of power safety, knowledge extraction through power knowledge extraction models can automatically process a large amount of document data and extract key information from them, such as cause analysis, preventive measures and safe operating procedures in accident reports. However, power safety knowledge extraction also faces many challenges. On the one hand, there is document diversity and complexity. Power safety documents are of various types and contain a large number of professional terms and technical details. On the other hand, the accuracy of information extraction, accurately identifying entities, events or relationships from texts is the core of knowledge extraction.

[0055] Traditional artificial intelligence neural network methods can achieve good knowledge extraction results under the drive of large-scale and high-quality data, but the extraction effect is not satisfactory in the face of professional fields and low resources. In the field of power safety knowledge, due to the large number of power equipment, complex safety knowledge and diverse scenarios, insufficient data and data preprocessing and labeling operations, it is very time-consuming and labor-intensive to complete knowledge extraction. In addition, using traditional neural networks to realize knowledge extraction of power safety knowledge text data makes it difficult for the model to fully learn the specific knowledge in the professional field, and there is a certain risk of overfitting.

[0056] Therefore, the present invention proposes to complete the knowledge extraction work based on the basic language big model for the power safety knowledge field scenario, combined with fine-tuning technology. The basic language big model has a strong general language understanding ability, which makes the big model more generalizable, and at the same time has a wider applicability and scalability, suitable for a variety of power application scenarios. Extracting power safety knowledge through the power knowledge extraction model can not only improve the efficiency and quality of power safety management, but also promote the intelligent development of the power industry and ensure the safe and stable operation of the power system.

[0057] The technical solution of this embodiment is to construct an electric power knowledge data set; provide professional sample data for fine-tuning parameters of the basic language large model; based on the electric power knowledge data set, fine-tune the parameters of the basic language large model through a low-rank adaptive method to obtain an electric power knowledge extraction model; based on the powerful general language understanding ability and stronger generalization of the basic language large model, fine-tune the parameters through a low-rank adaptive method, so that the obtained electric power knowledge extraction model has professional knowledge in the electric power field, and reduces the number of parameters that need to be updated during the fine-tuning process, thereby improving the fine-tuning efficiency; extracting electric power knowledge through the electric power knowledge extraction model can improve the accuracy and efficiency of knowledge extraction of electric power knowledge. Therefore, through the technical solution of this application, the problem that it is difficult for the model to fully learn the specific knowledge in the professional field and it is difficult to ensure the accuracy and efficiency of knowledge extraction is solved, and the accuracy and efficiency of knowledge extraction of electric power knowledge are improved.

[0058] Embodiment 2

[0059] Figure 2 This is a flow chart of a method for extracting power knowledge provided in Example 2 of the present application. The technical solution of this embodiment is further refined on the basis of the above technical solution.

[0060] Furthermore, the step “based on the electric power knowledge dataset, the parameters of the basic language big model are fine-tuned by a low-rank adaptive method to obtain the electric power knowledge extraction model” is refined as: “based on the electric power knowledge dataset, the parameters of the basic language big model are frozen and low-rank matrix pairs are added to the basic language big model to fine-tune the parameters of the basic language big model to obtain the electric power knowledge extraction model” to obtain the electric power knowledge extraction model.

[0061] See also Figure 2 A method for extracting electric power knowledge is shown, comprising:

[0062] S210. Construct an electric power knowledge dataset.

[0063] S220. Based on the electric power knowledge data set, the parameters of the basic language big model are frozen and low-rank matrix pairs are added to the basic language big model to fine-tune the parameters of the basic language big model to obtain the electric power knowledge extraction model.

[0064] In an optional embodiment, the basic language large model is the Chinese Llama2-7B model.

[0065] At present, the research on various large language models is flourishing, and many general language understanding models have emerged, and have achieved good results in various tasks. Among them, the LLaMa (professional term, a large model) model designed and developed by Meta (professional term, name) has been widely used in various fields due to its good model effect and comprehensive training data set. However, the model has poor understanding and adaptation to Chinese. In order to make up for the problem of insufficient Chinese understanding ability, relevant researchers later added a large amount of Chinese data for training and optimization, thus obtaining the Chinese Llama2-7B model optimized for Chinese. The model currently has a stronger understanding ability of the Chinese language, and is compatible with the relevant optimizations of the original LLaMa, thus providing the possibility of fine-tuning the model.

[0066] The Chinese Llama2-7B model has been specifically optimized and improved for the Chinese context, featuring rich representation capabilities, powerful Chinese optimization, and strong context expansion capabilities. Among them, rich representation capabilities may refer to the fact that the Chinese Llama2-7B model has 7 billion parameters, enabling it to learn more complex and refined language patterns, especially performing well in understanding complex texts and generating high-quality natural language. Chinese optimization may refer to the in-depth optimization of the vocabulary of the Chinese processing model, expanding the vocabulary and improving the encoding / decoding speed, while also expanding the coverage of the Chinese character set. Strong context expansion capabilities may refer to the fact that the model natively supports 4K context, but through fine-tuning, the context length can be increased to 32K, which enables the model to process longer text information.

[0067] By selecting the Chinese Llama2-7B model as the basic language large model, one can leverage its characteristics of rich representation capabilities, powerful Chinese optimization, and strong context expansion capabilities to reduce the number of parameter fine-tuning, improve the efficiency of parameter fine-tuning, and at the same time ensure the accuracy of the power knowledge extraction model.

[0068] Freezing the parameters of the basic language large model may refer to the fact that during the fine-tuning process, the parameters of the basic language large model are frozen and do not participate in gradient updates. Adding low-rank matrix pairs to the basic language large model for parameter fine-tuning may refer to adding a pair of low-rank matrices (A and B), and the product of these two matrices is used to simulate the changes in the original parameters. Exemplarily, a new path can be added beside the original pre-trained model. By multiplying the front and back matrices A and B, the first matrix A is responsible for dimensionality reduction, the second matrix B is responsible for dimensionality increase, and the middle layer dimension is r, so as to simulate the intrinsic rank.

[0069] In an optional embodiment, based on the power knowledge dataset, by freezing the parameters of the basic language large model and adding low-rank matrix pairs to the basic language large model for parameter fine-tuning, it includes: based on the power knowledge dataset, in the basic language large model, performing forward propagation through the added low-rank matrix pairs to obtain the forward propagation result; based on the forward propagation result, freezing the parameters in the basic language large model and adjusting the parameters of the added low-rank matrices to achieve parameter fine-tuning of the basic language large model.

[0070] During the forward propagation process of the basic language large model, the input passes through the original pre-trained layer and then is transformed by the added low-rank matrix. Since the dimension of the trainable layer is the same as that of the pre-trained model layer, both being d, first reduce the dimension d to r through a fully connected layer, and then map it back to the d dimension through a fully connected layer, where r << d and r is the rank of the matrix. In this way, the matrix calculation changes from d×d to d×r + r×d, reducing the number of parameters.

[0071] During the back propagation process of the basic language large model, only the gradients of the low-rank matrices A and B are updated and calculated, and the parameters of these two matrices are updated. Specifically, when training the downstream task of power knowledge extraction, the other parameters of the model are fixed, and only the weight parameters of the two newly added matrices are optimized. The results of the pre-trained model and the newly added pathway are added as the final result (the input and output dimensions of the pathways on both sides are consistent). The weight parameters of the first matrix A will be initialized by a Gaussian function, while the weight parameters of the second matrix B will be initialized to a zero matrix. This ensures that the newly added pathway BA=0 at the beginning of training and has no effect on the model results. The parameter update formula is as follows:

[0072] h=W0x+ΔWx=W0x+BAx;

[0073] Among them, W0 is the parameter for initializing the basic language model, ΔW is the parameter that needs to be updated, x is the input, and h is the output.

[0074] If all parameters are fine-tuned, the parameter amount is W0. However, if the low-rank adaptive method is used, only ΔW needs to be fine-tuned. The process of low-rank adaptive parameter update can use low-rank decomposition. The formula is as follows:

[0075] W0+ΔW=W0+BA.

[0076] During low-rank adaptive training, W0 is frozen and does not accept gradient updates, while A and B contain trainable parameters. In the forward process, W0 and ΔW are multiplied by the same x and added together. During initialization, A is initialized with a random Gaussian and B is initialized with zero. The training process can be represented by ΔW = BA. The core idea of ​​the low-rank adaptive method is to simulate the change in parameters through low-rank decomposition, thereby achieving indirect training of large models with a very small number of parameters.

[0077] Based on the electric power knowledge dataset, in the basic language large model, forward propagation is performed through the added low-rank matrix to obtain the forward propagation result; based on the forward propagation result, the parameters in the basic language large model are frozen, and the parameters of the added low-rank matrix are adjusted to achieve parameter fine-tuning of the basic language large model, reduce the number of adjusted parameters, and improve the efficiency of parameter fine-tuning.

[0078] In an optional embodiment, a low-rank matrix pair is added to each layer in the base language model or a layer where matrix multiplication occurs.

[0079] The low-rank matrix is ​​added to each layer of the basic language model or the layer where matrix multiplication occurs. The product of two matrices is used to simulate the change of the original parameters, reduce the amount of data in the parameter calculation process, and reduce the occupation of computing resources.

[0080] In various knowledge extraction tasks, accuracy, recall and harmonic mean F1 are usually used as evaluation indicators to objectively describe the extraction effect. Such indicators divide the knowledge extraction results into four categories according to the correctness of the classification, namely: positive samples TP predicted as positive examples, negative samples FP predicted as positive examples, positive samples FN predicted as negative examples, and negative samples TN predicted as negative examples. These four results are represented by confusion matrix as shown in Table 1.

[0081] Table 1 Confusion matrix diagram

[0082]

[0083] The accuracy of knowledge extraction is used to measure the proportion of correct knowledge in the acquired knowledge and is calculated according to the following formula:

[0084]

[0085] TP is the number of true positives, i.e., the number of entities, relationships between entities, or entity attributes that are extracted and consistent with the truth; FP is the number of false positives, i.e., the number of entities, relationships between entities, or entity attributes that are extracted but do not correspond to the truth.

[0086] The recall rate Recall in the knowledge extraction stage is used to measure the extent to which the acquired knowledge covers the correct knowledge and is calculated according to the following formula:

[0087]

[0088] Where: FN is the number of false negatives, the number of real but unrecognized entities, inter-entity relations or entity attributes.

[0089] The value of F1 is calculated as follows:

[0090]

[0091] Among them, the precision measures the accuracy of the model in predicting positive samples. The higher the precision, the greater the probability that the true label is also positive in the samples predicted to be positive; the recall measures the ability of the model to detect positive samples. The higher the recall, the greater the probability that the samples with the true label are predicted to be positive; the F1 value is an evaluation indicator that combines the precision and recall rates, and is used to reflect the overall effect of the model.

[0092] In an optional embodiment, the model prompt words of the power knowledge extraction model include: at least one of: information type, indicator word and example.

[0093] In order to better utilize the knowledge extraction performance of the power knowledge extraction model, it is necessary to design prompt words. For example, prompt words can be designed with the help of Propmt (professional term, a prompt template).

[0094] When designing a prompt, you should pay attention to the following points:

[0095] Specificity: The prompt should be as specific as possible, indicating the type of information that the model needs to extract. For example, when extracting equipment types, the prompt word is "List all types of power equipment mentioned in the following paragraph: [paragraph text]."; when extracting safety regulations, the prompt word is "Based on the following text, summarize the safe operation steps of power operations: [paragraph text]."; when extracting the cause of an accident, the prompt word is "Analyze the following accident report and point out the possible causes of the accident: [paragraph text].";

[0096] Use indicator words: Use indicator words to guide the model, such as "extract", "list", "summarize", "analyze", etc., which is also to make the specific extraction task clearer;

[0097] Consider the context: provide enough context information to help the model better understand the extraction task;

[0098] Provide examples and formats: Provide one or more examples to show the expected output format, as appropriate. "For example, in the text '... Transformer failed...', the equipment type is 'Transformer'.", to help the model better identify entities and relationships, etc.

[0099] Therefore, the model prompt words of the power knowledge extraction model include: at least one of information type, indicator words and examples, so that the power knowledge extraction model can better display the knowledge extraction results to the user.

[0100] The power knowledge extraction model has a strong ability to understand general language and has stronger generalization. By establishing a text data set, the power knowledge extraction model can learn infrequently occurring safety terms or attributes, and has better knowledge extraction capabilities for power knowledge. Secondly, the power knowledge extraction model has wide applicability and scalability, and is suitable for a variety of power application scenarios, including but not limited to fault diagnosis, equipment maintenance, and safety audits. The power knowledge extraction model can be fine-tuned according to new data sets, and can be updated by continuously learning new knowledge to adapt to the latest developments and changes in the field of power safety. Thirdly, the power knowledge extraction model can more conveniently promote knowledge sharing. By extracting and organizing safety knowledge, an easily accessible knowledge base can be created to promote knowledge sharing and learning within the organization, while also helping to train new employees and improve the overall safety skills of the team. Finally, for such complex extraction tasks in the field of power safety, the power knowledge extraction model provides a more flexible and varied extraction method, and with the help of multiple rounds of question-answering, it can effectively enhance the interpretability of knowledge extraction.

[0101] S230. Extracting power knowledge through a power knowledge extraction model.

[0102] The technical solution of this embodiment is to fine-tune the parameters of the basic language big model based on the electric power knowledge data set by freezing the parameters of the basic language big model and adding low-rank matrix pairs to the basic language big model to obtain an electric power knowledge extraction model, thereby reducing the amount of parameter adjustment of the basic language big model and improving the efficiency of parameter adjustment.

[0103] Embodiment 3

[0104] Figure 3 The figure shows a schematic diagram of the structure of a power knowledge extraction device provided in the third embodiment of the present application. This embodiment is applicable to the case of extracting safety knowledge of the power system. The specific structure of the power knowledge extraction device is as follows:

[0105] An electric power knowledge data set construction module 310 is used to construct an electric power knowledge data set;

[0106] A parameter fine-tuning module 320 is used to fine-tune the parameters of the basic language large model based on the power knowledge data set by a low-rank adaptive method to obtain a power knowledge extraction model;

[0107] The power knowledge extraction module 330 is used to extract power knowledge through a power knowledge extraction model.

[0108] The technical solution of this embodiment is to construct an electric power knowledge data set; provide professional sample data for fine-tuning parameters of the basic language large model; based on the electric power knowledge data set, fine-tune the parameters of the basic language large model through a low-rank adaptive method to obtain an electric power knowledge extraction model; based on the powerful general language understanding ability and stronger generalization of the basic language large model, fine-tune the parameters through a low-rank adaptive method, so that the obtained electric power knowledge extraction model has professional knowledge in the electric power field, and reduces the number of parameters that need to be updated during the fine-tuning process, thereby improving the fine-tuning efficiency; extracting electric power knowledge through the electric power knowledge extraction model can improve the accuracy and efficiency of knowledge extraction of electric power knowledge. Therefore, through the technical solution of this application, the problem that it is difficult for the model to fully learn the specific knowledge in the professional field and it is difficult to ensure the accuracy and efficiency of knowledge extraction is solved, and the accuracy and efficiency of knowledge extraction of electric power knowledge are improved.

[0109] Optionally, the parameter fine-tuning module 320 includes:

[0110] The parameter fine-tuning unit is used to fine-tune the parameters of the basic language big model based on the power knowledge data set by freezing the parameters of the basic language big model and adding low-rank matrix pairs to the basic language big model to obtain the power knowledge extraction model.

[0111] Optional, parameter fine-tuning unit, including:

[0112] The low-rank matrix pair adding sub-unit is used to perform forward propagation through the added low-rank matrix pair in the basic language large model based on the power knowledge data set to obtain the forward propagation result;

[0113] The parameter freezing subunit is used to freeze the parameters in the basic language model based on the forward propagation results, adjust the parameters of the added low-rank matrix, and achieve parameter fine-tuning of the basic language model.

[0114] Optionally, low-rank matrix pairs are added to each layer in the base language model or layers where matrix multiplication occurs.

[0115] Optionally, the power knowledge data set construction module 310 includes:

[0116] The original power data mobile phone unit is used to obtain original power data from multiple sources; the original power data source includes at least one of laws and regulations, operation manuals, safety specifications and safety cases;

[0117] The label annotation unit is used to annotate the entities, relations and events in the original power data to obtain the power knowledge dataset.

[0118] Optionally, the model prompt words of the power knowledge extraction model include: at least one of: information type, indicator word and example.

[0119] The power knowledge extraction device provided in the embodiment of the present application can execute the power knowledge extraction method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects for executing the power knowledge extraction method.

[0120] Embodiment 4

[0121] Figure 4 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present application is shown in FIG. Figure 4 As shown, the electronic device includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of the processor 410 in the electronic device can be one or more. Figure 4 A processor 410 is taken as an example; the processor 410, the memory 420, the input device 430 and the output device 440 in the electronic device can be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0122] The memory 420, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the power knowledge extraction method in the embodiment of the present application (for example, the power knowledge data set construction module 310, the parameter fine-tuning module 320 and the power knowledge extraction module 330). The processor 410 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 420, that is, implements the above-mentioned power knowledge extraction method.

[0123] The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 420 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 420 may further include a memory remotely arranged relative to the processor 410, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0124] The input device 430 may be used to receive input character information and generate key signal input related to user settings and function control of the electronic device. The output device 440 may include a display device such as a display screen.

[0125] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0126] Embodiment 5

[0127] Embodiment 5 of the present application also provides a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to execute a method for extracting electric power knowledge, the method comprising: constructing an electric power knowledge data set; based on the electric power knowledge data set, fine-tuning parameters of a basic language large model through a low-rank adaptive method to obtain an electric power knowledge extraction model; and extracting electric power knowledge through the electric power knowledge extraction model.

[0128] Of course, the storage medium containing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the method operations described above, can also execute related operations in the power knowledge extraction method provided in any embodiment of the present application.

[0129] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present application can be implemented with the help of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for an electronic device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0130] It is worth noting that in the embodiment of the above-mentioned power knowledge extraction device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0131] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A method for extracting power knowledge, It is characterized in that include: Constructing power knowledge dataset; Based on the electric power knowledge dataset, the parameters of the basic language large model are fine-tuned by a low-rank adaptive method to obtain an electric power knowledge extraction model; The power knowledge extraction model is used to extract power knowledge.

2. The method according to claim 1, It is characterized in that Based on the electric power knowledge data set, the parameters of the basic language large model are fine-tuned by a low-rank adaptive method to obtain an electric power knowledge extraction model, including: Based on the electric power knowledge data set, the parameters of the basic language big model are frozen and low-rank matrix pairs are added to the basic language big model to fine-tune the parameters of the basic language big model, so as to obtain the electric power knowledge extraction model.

3. The method according to claim 2, It is characterized in that The method of fine-tuning the parameters of the basic language model based on the electric power knowledge data set by freezing the parameters of the basic language model and adding a low-rank matrix pair to the basic language model comprises: Based on the electric power knowledge data set, in the basic language large model, forward propagation is performed through the added low-rank matrix to obtain a forward propagation result; Based on the forward propagation result, the parameters in the basic language large model are frozen, and the parameters of the added low-rank matrix are adjusted to achieve parameter fine-tuning of the basic language large model.

4. The method according to claim 3, It is characterized in that The low-rank matrix is ​​added to each layer in the base language model or a layer where matrix multiplication occurs.

5. The method according to claim 1, It is characterized in that The constructing of the electric power knowledge data set includes: Acquire original power data from multiple sources; the original power data source includes at least one of laws and regulations, operation manuals, safety specifications and safety cases; The entities, relations and events in the original power data are labeled to obtain a power knowledge data set.

6. The method according to any one of claims 1 to 5, It is characterized in that The model prompt words of the power knowledge extraction model include: at least one of information type, indicator word and example.

7. A device for extracting electric power knowledge, It is characterized in that include: An electric power knowledge data set construction module, used to construct an electric power knowledge data set; A parameter fine-tuning module, used to fine-tune the parameters of the basic language large model based on the power knowledge data set by a low-rank adaptive method to obtain a power knowledge extraction model; The power knowledge extraction module is used to extract power knowledge through the power knowledge extraction model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the power knowledge extraction method as described in any one of claims 1-6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the power knowledge extraction method as described in any one of claims 1 to 6 is implemented.

10. A computer program product, It is characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the power knowledge extraction method according to any one of claims 1 to 6.

Images