Electric power knowledge graph construction method and system based on active learning

Abstract

The invention discloses an active learning-based electric power knowledge graph construction method and system, and relates to the technical field of electric power market transaction data analysis, and the method comprises the steps: collecting data, cleaning the collected data, generating word prediction task data of an electric power transaction scene, and training an electric power field word embedding model; generating an entity relationship training sample, inputting a word embedding model, performing entity relationship label prediction through an entity relationship joint extraction model, extracting a high-quality sample based on active learning to perform part-of-speech tagging, and training the model based on a tagged sample; and applying the trained entity relationship joint extraction model to power transaction data to construct a knowledge graph, performing manual correction and entity unification, and storing the knowledge graph into a Neo4j database. According to the method, active learning and deep learning are combined, so that the sample labeling cost is effectively reduced, and meanwhile, the model performance is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of power market transaction data analysis, and in particular to a method and system for constructing a power knowledge graph based on active learning. Background Art

[0002] With the advent of the big data era, the volume of data is expanding rapidly, and the data form has also extended from traditional numerical structured data to semi-structured and unstructured text data. As a graph-based data structure, knowledge graphs have powerful knowledge learning and relationship storage capabilities, providing people with a more effective way to store and manage data. At the same time, knowledge graphs can also effectively assist new scenarios such as intelligent decision-making and intelligent search, and have broad application prospects.

[0003] The knowledge graph in the field of power trading refers to a knowledge graph built based on data in the power field. It can effectively collect information such as entities and relationships in the data in the field of power trading, thereby effectively assisting the trading decisions of power supply and power purchasing companies. Since the data in the field of power trading is highly knowledge-based, data annotation needs to be performed by professionals. Therefore, there are problems such as long time consumption and high cost in annotating massive amounts of data, and too little annotated data will greatly constrain the performance of deep learning in constructing the power knowledge graph. The existing patents for constructing knowledge graphs all assume that annotated data already exists, and rarely discuss the difficulty and high cost of obtaining annotated data, making it difficult to meet existing needs. Summary of the invention

[0004] In view of the above-mentioned existing problems, the present invention provides a method and system for constructing an electric power knowledge graph based on active learning. Through data cleaning, preprocessing and text segmentation technology, the problem that unstructured text data is difficult to be directly used for knowledge graph construction is solved; through active learning and fine-tuning of the general language model, the adaptability problem of the domain-specific language model is solved; by adopting an entity-relationship joint extraction model combined with an active learning strategy, the accuracy problem of entity relationship extraction is solved; through entity vectorization representation and semantic similarity calculation, the problem of entity unification and consistency maintenance is solved; through a visualization display module, the problem of intuitive display of the knowledge graph is solved; by building an integrated system implemented by computer equipment, the automation and efficiency problems of knowledge graph construction are solved.

[0005] In order to solve the above technical problems, a method for constructing a power knowledge graph based on active learning is proposed, including:

[0006] Unstructured text data is obtained from power trading documents, and the obtained data is cleaned, preprocessed and text-blocked; the preprocessed text data is used to fine-tune the model to obtain a word embedding language model in the power field, and the model is used to jointly extract entity relationships to obtain entity relationships; the extracted entity relationships are processed in a unified manner, the unified entity relationship data is imported into the database, and a power trading knowledge graph is constructed for visual display.

[0007] As a preferred solution of the method for constructing an electric power knowledge graph based on active learning described in the present invention, the unstructured text data obtained includes electric power transaction reports, contracts and academic literature collected from government websites, company disclosure reports and academic platforms, and experts determine and preliminarily screen the text types, delete redundant data not related to the business, and obtain text data that can construct a knowledge graph.

[0008] As a preferred solution of the method for constructing an electric power knowledge graph based on active learning described in the present invention, the preprocessing includes using regular expressions to identify and delete irrelevant noise information in the text, unifying the date and time formats, correcting spelling errors, standardizing currency units and numerical expressions, eliminating duplicate records, and combining the Chinese stop word list to traverse the stop word list according to samples of the text data and delete samples of stop words;

[0009] The text segmentation includes segmenting text data into blocks using a text segmentation tool to form samples; the noise information includes special characters, spaces and HTML tags.

[0010] As a preferred solution of the method for constructing an electric power knowledge graph based on active learning described in the present invention, wherein: the fine-tuning of the model includes selecting electric power-related literature from the cleaned data, the content of which covers electric power text data, and masking the text to obtain an input format adapted to the requirements of the RoBERTa model training task, selecting the middle layer of the RoBERTa model for fine-tuning, training the language model based on the word prediction task, iteratively updating the low-rank matrix weights, and using the output of the trained model before softmax as the final output to obtain a word embedding language model with knowledge related to the electric power field, and obtaining a word vector representation;

[0011] The fine-tuning consists of freezing the weight matrix W∈R in the original RoBERTa d×d , and insert a trainable low-rank matrix A, B for each weight matrix: A∈R d×r ,B∈R r×d And initialize it.

[0012] As a preferred solution of the method for constructing an electric power knowledge graph based on active learning described in the present invention, wherein: the entity relationship joint extraction includes inputting the word vector into the entity relationship extraction model for label prediction, and combining the active learning strategy to generate samples to be labeled;

[0013] The constructing of the entity relationship extraction model includes defining an entity label set and a relationship label set, inputting the electric power field literature for entity relationship extraction into the pre-trained model according to sentence samples, obtaining hidden layer representation, initializing the entity relationship extraction model, inputting the obtained head representation and tail representation into the double affine attention layer to capture the interaction information between word pairs, and obtaining the probability distribution of the model prediction label through the Softmax layer, and calculating the information entropy of the sentence as the uncertainty score of the sentence, annotating the input sample and calculating the loss function, specifying a time stop mechanism, and repeating the entity relationship joint extraction until the specified time stop mechanism is triggered;

[0014] The hidden layer is represented as:

[0015] {h1,h2,…,h |s|}=RoBERTa({x1,x2,…,x |s|})

[0016] Among them, x i is the phrase in the original input sequence, h i is the word embedding vector, i=1, 2, …, s, |s| is the input text length, RoBERTa is the fine-tuned word embedding model, and i is the variable index;

[0017] The initialization entity relationship extraction model includes inputting the word embedding vector into the head entity multi-layer perceptron and the tail entity multi-layer perceptron respectively, and the formula is expressed as:

[0018]

[0019] in, and are the i-th and j-th hidden layer vectors after the multi-layer perceptron, i and j are variable indices, MLP head and MLP tail is the head entity multilayer perceptron and the tail entity multilayer perceptron, |s| is the length of the input text;

[0020] The probability distribution formula of the model prediction label is:

[0021] foreachi,j∈(0,|s|)

[0022]

[0023] P i,j=softmax(g i,j ),P∈R s|×|s|×|y |

[0024] Among them, h h and h t is the hidden layer vector after the head entity and the tail entity multilayer perceptron, U1 and U2 are trainable parameter matrices, P is the label prediction matrix, P i,j is the probability distribution of the i-th and j-th positions in the sentence matrix of |s|×|s| in the label space, i and j are variable indices, |y| is the length of the label space, and b is an empty label;

[0025] The formula for calculating information entropy is:

[0026]

[0027] Among them, |s| is the length of the input text, t is the identifier of the label space, P i,j,t is the probability value of the label in the label matrix at each position in the matrix, i and j are variable indexes, Entropy_score is the information entropy, and Y is the label space of the model;

[0028] The label space expression of the model is:

[0029] Y=Y r ∪Y e ∪{⊥}

[0030] Among them, Y is the label space of the model, Y e is the entity label set, Y r is a set of relation labels, ⊥ is an empty label;

[0031] The loss function formula is expressed as:

[0032]

[0033]

[0034] in, is the cross entropy loss between the predicted label and the true label, is the constraint loss in the symmetric region, The implicit loss of mutual constraints between relations and entities, is the total loss, |s| is the input text length, Y e is the entity label set, Y r is the set of relation labels, P i,:,t is the probability distribution of any label space in the i-th row, P :,i,t is the probability distribution of any label space in the jth column, i and j are variable indices, Y is the label space of the model, and t is the identifier of the label space.

[0035] As a preferred solution of the method for constructing an electric power knowledge graph based on active learning described in the present invention, wherein: the entity unified processing of the extracted entity relationship includes, based on knowledge judgment, removing the wrong entity relationship, performing entity-level entity vectorization representation on the extracted named entities, and calculating the semantic similarity respectively according to the cosine formula, after calculating the similarity, setting the similarity threshold to 0.95, taking entities above the threshold as fusion objects, extracting the main keywords from them and delivering the entity list to be merged to experts for evaluation, judging whether the merged entities meet the professional requirements and business logic, and selecting a unified entity list;

[0036] The formula for calculating semantic similarity is:

[0037]

[0038] Among them, A is the semantic vector of entity 1, B is the semantic vector of entity 2, ∥A∥ is the modulus of the semantic vector of entity 1, and ∥B∥ is the modulus of the semantic vector of entity 2.

[0039] As a preferred solution of the method for constructing an electric power knowledge graph based on active learning described in the present invention, the visual display includes obtaining triples based on knowledge graph entity relationship extraction technology, using a python development kit import tool and importing entity relationship data, and establishing and storing an electric power transaction knowledge graph;

[0040] A knowledge graph query is performed on the transaction subject based on the Cypher language. The Cypher is a declarative query language exclusive to Neo4j. The data of power transaction knowledge is updated and queried based on the Cypher language.

[0041] Another object of the present invention is to provide a system for constructing an electric power knowledge graph based on active learning. The present invention combines active learning with deep learning, effectively reducing the sample annotation cost while ensuring the model performance. The system of the present invention constructs an electric power trading knowledge graph through efficient data cleaning, model fine-tuning and entity relationship joint extraction technology, realizes the intelligent frequency and peak control of thermal power units, improves the operating efficiency of the power system, reduces operating costs, and ensures the transparency and accuracy of electric power transactions, providing strong data support and decision-making assistance for the power market.

[0042] As a preferred solution of the power knowledge graph construction system based on active learning described in the present invention, it is characterized by comprising a text data acquisition and processing module, a model fine-tuning module, an entity relationship joint extraction module, an entity unified processing module and a visualization display module;

[0043] The text data acquisition and processing module is used to collect reports, contracts and academic documents related to power transactions to provide a data source for building a knowledge graph, clean text data, and use the Huggingface text segmentation tool to segment the cleaned text into blocks to form samples suitable for model training;

[0044] The model fine-tuning module is used to fine-tune the pre-trained RoBERTa model to better adapt to the language characteristics of the power field and output a word embedding language model in the power field;

[0045] The entity relationship joint extraction module is used to extract entity relationships from the labeled samples, define entity and relationship label sets, predict the probability distribution of labels through the model, and realize the extraction of entity relationships;

[0046] The entity unified processing module is used to perform entity-level vector representation of the extracted entity relationships, calculate semantic similarity, merge entities, and ensure entity consistency in the knowledge graph;

[0047] The visualization display module is used to import the unified entity relationship data into the database, use Neo4j and py2neo tools to build and store the power transaction knowledge graph, and use Neo4j's Cypher query language to perform visualization and data query of the knowledge graph based on the triple data extracted from the knowledge graph.

[0048] A computer device includes a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of a method described in the method for constructing an electric power knowledge graph based on active learning are implemented.

[0049] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of a method described in the method for constructing an electric power knowledge graph based on active learning are implemented.

[0050] Beneficial effects of the present invention: The present invention realizes the standardization of text data through preprocessing steps, such as using regular expressions to delete noise information, unify formats, and eliminate duplicate records. The beneficial effect of this step is to improve the quality of text data, provide a pure data environment for subsequent model training and entity relationship extraction, thereby improving the accuracy of the knowledge graph; by fine-tuning the model, the model is better adapted to the power field, and the construction of a word embedding language model in the power field is realized. The beneficial effect of this step is to improve the performance of the model in the power field, make the entity relationship extraction more accurate, and lay the foundation for building a high-quality knowledge graph; through the joint extraction of entity relationships and combined with active learning strategies, the effective identification and extraction of entity relationships are realized. The beneficial effect of this step is to improve the efficiency of entity relationship extraction, reduce the workload of manual annotation, and ensure the integrity of the knowledge graph; by performing entity unified processing on the extracted entity relationships, the consistency of entities is achieved. The beneficial effect of this step is to reduce the redundant information in the knowledge graph and improve the accuracy and availability of the knowledge graph.

[0051] Through visual display, the visualization and query of the power trading knowledge graph are realized. The beneficial effect of this step is to make the knowledge graph more intuitive, which is convenient for users to understand and analyze the knowledge structure in the field of power trading and provide support for decision-making; by constructing a system including text data acquisition and processing module, model fine-tuning module, entity relationship joint extraction module, entity unified processing module and visual display module, the automation and integration of power knowledge graph construction are realized. The beneficial effect of this step is to improve the efficiency of knowledge graph construction, reduce the cost of manual intervention, and make the knowledge graph construction process more convenient; the above method is implemented by computer equipment, making the power knowledge graph construction process more efficient and convenient. The beneficial effect of this step is to promote the application of power knowledge graph in the industry and provide strong technical support for the power trading field; by storing computer programs in computer-readable storage media, the convenient dissemination and wide application of power knowledge graph construction methods are realized. The beneficial effect of this step is to promote the popularization of power knowledge graph technology and provide convenience for research and practice in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0053] Figure 1An overall flow chart of a method for constructing an electric power knowledge graph based on active learning provided for an embodiment of the present invention.

[0054] Figure 2 A schematic diagram of the structure of an entity relationship extraction model for a method for constructing an electric power knowledge graph based on active learning provided by an embodiment of the present invention.

[0055] Figure 3 A system solution flow chart of a power knowledge graph construction system based on active learning provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0058] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it an embodiment that is mutually exclusive with other embodiments, either individually or selectively.

[0059] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0060] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0061] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0062] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a method for constructing an electric power knowledge graph based on active learning, comprising:

[0063] S1: Obtain unstructured text data from power trading literature, and clean, preprocess and segment the acquired data.

[0064] Furthermore, the acquisition of unstructured text data includes electricity trading reports, contracts and academic literature collected from government websites, company disclosure reports and academic platforms, and experts determine and preliminarily screen the text types, delete redundant data not related to the business, and obtain text data that can construct a knowledge graph.

[0065] It should be noted that the preprocessing includes using regular expressions to identify and delete irrelevant noise information in the text, unifying the date and time formats, correcting spelling errors, standardizing monetary units and numerical expressions, eliminating duplicate records, and combining the Chinese stop word list to traverse the stop word list according to samples of the text data and delete samples of stop words.

[0066] The text segmentation includes segmenting text data into blocks using the Huggingface text segmentation tool to form samples; the noise information includes special characters, spaces and HTML tags.

[0067] S2: Use the preprocessed text data to fine-tune the model to obtain the word embedding language model in the power field, and use the model to jointly extract entity relationships to obtain entity relationships.

[0068] The fine-tuning of the model includes selecting literature in the power-related field from the cleaned data, covering power text data, and masking the text to obtain an input format that meets the requirements of the RoBERTa model training task, selecting the middle layer of the RoBERTa model for fine-tuning, training the language model based on the word prediction task, iteratively updating the low-rank matrix weights, and using the output of the trained model before softmax as the final output to obtain a word embedding language model with knowledge related to the power field, and obtain a word vector representation.

[0069] It should be noted that the masking process includes segmenting the text data and adding special marks, masking 15% of the professional terms in the input text, masking 80% of the words with special marks [mask], randomly replacing 10% of the words with non-original words, and leaving the remaining 10% of the words unchanged.

[0070] The fine-tuning consists of freezing the weight matrix W∈R in the original RoBERTa d×d , and insert a trainable low-rank matrix A, B for each weight matrix: A∈R d×r ,B∈R r×d And initialize it.

[0071] like Figure 2 The entity relationship joint extraction includes inputting word vectors into the entity relationship extraction model for label prediction, and combining active learning strategies to generate samples to be labeled.

[0072] The construction of the entity relationship extraction model includes defining an entity label set and a relationship label set, inputting the power field literature for entity relationship extraction into the pre-trained model according to sentence samples, obtaining hidden layer representation, initializing the entity relationship extraction model, inputting the obtained head representation and tail representation into the double affine attention layer to capture the interaction information between word pairs, and obtaining the probability distribution of the model prediction label through the Softmax layer, and calculating the information entropy of the sentence as the uncertainty score of the sentence, annotating the input samples and calculating the loss function, specifying a time-stop mechanism, and repeating the entity relationship joint extraction until the specified time-stop mechanism is triggered.

[0073] The hidden layer is represented as:

[0074] {h1,h2,…,h |s|}=RoBERTa({x1,x2,…,x |s|})

[0075] Among them, x i is the phrase in the original input sequence, h i is the word embedding vector, i=1, 2,…, s, |s| is the input text length, RoBERTa is the fine-tuned word embedding model, and i is the variable index.

[0076] The initialization entity relationship extraction model includes inputting the word embedding vector into the head entity multi-layer perceptron and the tail entity multi-layer perceptron respectively, and the formula is expressed as:

[0077]

[0078] in, and are the i-th and j-th hidden layer vectors after the multi-layer perceptron, i and j are variable indices, MLP head and MLP tail are the head entity multilayer perceptron and the tail entity multilayer perceptron, |s| is the length of the input text.

[0079] Furthermore, the probability distribution formula of the model prediction label is:

[0080] foreachi,j∈(0,|s|)

[0081]

[0082] P i,j =softmax(g i,j ),P∈R |s|×|s|×|y|

[0083] Among them, h h and h t is the hidden layer vector after the head entity and the tail entity multilayer perceptron, U1 and U2 are trainable parameter matrices, P is the label prediction matrix, P i,j is the probability distribution of the i-th and j-th positions in the sentence matrix of |s|×|s| in the label space, i and j are variable indices, |y| is the length of the label space, and b is the empty label.

[0084] The formula for calculating information entropy is:

[0085]

[0086] Among them, |s| is the length of the input text, t is the label space, P i,j,t is the probability value of the label in the label matrix for each position in the matrix, i and j are variable indexes, Entropy_score is the information entropy, and Y is the label space of the model.

[0087] The label space expression of the model is:

[0088] Y=Y r ∪Y e ∪{⊥}

[0089] Among them, Y is the label space of the model, Y e is the entity label set, Y r is a set of relation labels, ⊥ is an empty label.

[0090] The loss function formula is expressed as:

[0091]

[0092]

[0093] in, is the cross entropy loss between the predicted label and the true label, is the constraint loss in the symmetric region, The implicit loss of mutual constraints between relations and entities, is the total loss, |s| is the input text length, Y e is the entity label set, Y r is the set of relation labels, P i,:,t is the probability distribution of any label space in the i-th row, P :,i,t is the probability distribution of any label space in the jth column, i and j are variable indices, Y is the label space of the model, and t is the identifier of the label space.

[0094] The time-stop mechanism includes designing measurement indicators to confirm whether the model performance has reached expectations and whether training can be stopped from the perspective of model prediction stability. Based on the loss results of model training, the rate of decrease of the model loss value is calculated by differential approximation. The current loss value is subtracted from the loss value of the previous iteration to approximate the rate of decrease of the loss value. Since the loss value of the model is related to the specific prediction task and data set, a relative threshold is set here for time-stopping. When the difference in the decrease of the model loss value between two iterations is less than 1% of the initial loss value of the model, it is considered that the prediction result of the model has stabilized and further training and sampling can be stopped.

[0095] S3: Perform entity unification processing on the extracted entity relationships, import the unified entity relationship data into the database, and construct a power transaction knowledge graph for visual display.

[0096] Furthermore, the entity unification processing of the extracted entity relationships includes, based on knowledge judgment, removing erroneous entity relationships, entity-level entity vectorization representation of the extracted named entities, and calculating the semantic similarity according to the cosine formula. After calculating the similarity, the similarity threshold is set to 0.95, and entities above the threshold are taken as fusion objects, and the main keywords are extracted from them and the list of entities to be merged is delivered to experts for evaluation to determine whether the merged entities meet professional requirements and business logic, and select a unified entity list.

[0097] The formula for calculating semantic similarity is:

[0098]

[0099] Among them, A is the semantic vector of entity 1, B is the semantic vector of entity 2, ∥A∥ is the modulus of the semantic vector of entity 1, and ∥B∥ is the modulus of the semantic vector of entity 2.

[0100] It should be noted that the visual display includes obtaining triples based on knowledge graph entity relationship extraction technology, importing entity relationship data using the Neo4j-import batch import tool of the Neo4j-based python development package py2neo, and establishing and storing the power trading knowledge graph.

[0101] A knowledge graph query is performed on the transaction subject based on the Cypher language, wherein Cypher is a declarative query language exclusive to Neo4j, and data update and query operations on the power transaction knowledge are performed based on the Cypher language.

[0102] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0103] Example 2, reference Figure 3 , which is the second embodiment of the present invention, and provides an active learning-based power knowledge graph construction system, including a text data acquisition and processing module 100, a model fine-tuning module 200, an entity relationship joint extraction module 300, an entity unified processing module 400 and a visualization display module 500;

[0104] The text data acquisition and processing module 100 is used to collect reports, contracts and academic documents related to power transactions to provide a data source for building a knowledge graph, clean the text data, and use the Huggingface text segmentation tool to segment the cleaned text into blocks to form samples suitable for model training;

[0105] The model fine-tuning module 200 is used to fine-tune the pre-trained RoBERTa model to better adapt to the language characteristics of the power field, and output a word embedding language model in the power field;

[0106] The entity relationship joint extraction module 300 is used to extract entity relationships from the labeled samples, define entity and relationship label sets, predict the probability distribution of labels through the model, and realize the extraction of entity relationships;

[0107] The entity unified processing module 400 is used to perform entity-level vector representation of the extracted entity relationship, calculate semantic similarity, merge entities, and ensure entity consistency in the knowledge graph;

[0108] The visualization display module 500 is used to import the unified entity relationship data into the database, use Neo4j and py2neo tools to build and store the power trading knowledge graph, and use Neo4j's Cypher query language to perform visualization and data query on the knowledge graph based on the triple data extracted from the knowledge graph.

[0109] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0110] Embodiment 3, the third embodiment of the present invention, is different from the first two embodiments in that:

[0111] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0112] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0113] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0114] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. A method for constructing an electric power knowledge graph based on active learning, characterized in that: include, Obtain unstructured text data from power trading documents, and clean, preprocess and segment the acquired data; The preprocessed text data is used to fine-tune the model to obtain a word embedding language model in the power field, and the model is used to perform entity relationship joint extraction to obtain entity relationships; The extracted entity relationships are processed in a unified manner, the unified entity relationship data is imported into the database, and a knowledge graph of power trading is constructed for visual display.

2. The method for constructing an electric power knowledge graph based on active learning according to claim 1, characterized in that: The unstructured text data obtained includes power transaction reports, contracts and academic literature collected from government websites, company disclosure reports and academic platforms, and experts determine and preliminarily screen the text types, delete redundant data not related to the business, and obtain text data that can be used to construct a knowledge graph.

3. The method for constructing an electric power knowledge graph based on active learning according to claim 2, characterized in that: The preprocessing includes using regular expressions to identify and delete irrelevant noise information in the text, unifying the date and time formats, correcting spelling errors, standardizing currency units and numerical expressions, eliminating duplicate records, and combining the Chinese stop word list to traverse the stop word list according to samples of the text data and delete samples of the stop words; The text segmentation includes segmenting text data into blocks using a text segmentation tool to form samples; the noise information includes special characters, spaces and HTML tags.

4. The method for constructing an electric power knowledge graph based on active learning according to claim 3, characterized in that: The fine-tuning of the model includes selecting literature in the field of electricity-related fields from the cleaned data, covering electricity text data, and masking the text to obtain an input format adapted to the requirements of the RoBERTa model training task, selecting the middle layer of the RoBERTa model for fine-tuning, training the language model based on the word prediction task, iteratively updating the low-rank matrix weights, and using the output of the trained model before softmax as the final output to obtain a word embedding language model with knowledge related to the electricity field, and obtain a word vector representation; The fine-tuning consists of freezing the weight matrix W∈R in the original RoBERTa d×d , and insert a trainable low-rank matrix A, B for each weight matrix: A∈R d×r ,B∈R r×d And initialize it.

5. The method for constructing an electric power knowledge graph based on active learning according to claim 4, characterized in that: The entity relationship joint extraction includes inputting the word vector into the entity relationship extraction model for label prediction, and combining the active learning strategy to generate samples to be labeled; The constructing of the entity relationship extraction model includes defining an entity label set and a relationship label set, inputting the electric power field literature for entity relationship extraction into the pre-trained model according to sentence samples, obtaining hidden layer representation, initializing the entity relationship extraction model, inputting the obtained head representation and tail representation into the double affine attention layer to capture the interaction information between word pairs, and obtaining the probability distribution of the model prediction label through the Softmax layer, and calculating the information entropy of the sentence as the uncertainty score of the sentence, annotating the input sample and calculating the loss function, specifying a time stop mechanism, and repeating the entity relationship joint extraction until the specified time stop mechanism is triggered; The hidden layer is represented as: {h1,h2,…,h |s| }=RoBERTa({x1,x2,…,x |s| }) Among them, x i is the phrase in the original input sequence, h i is the word embedding vector, i=1, 2, …, s, |s| is the input text length, RoBERTa is the fine-tuned word embedding model, and i is the variable index; The initialization entity relationship extraction model includes inputting the word embedding vector into the head entity multi-layer perceptron and the tail entity multi-layer perceptron respectively, and the formula is expressed as: in, and are the i-th and j-th hidden layer vectors after the multi-layer perceptron, i and j are variable indices, MLP head and MLP tail is the head entity multilayer perceptron and the tail entity multilayer perceptron, |s| is the length of the input text; The probability distribution formula of the model prediction label is: foreachi,j∈(0,|s|) P i,j =softmax(g i,j ),P∈R |s|×|s|×|y| Among them, h h and h t is the hidden layer vector after the head entity and the tail entity multilayer perceptron, U1 and U2 are trainable parameter matrices, P is the label prediction matrix, P i,j is the probability distribution of the i-th and j-th positions in the sentence matrix of |s|×|s| in the label space, i and j are variable indices, |y| is the length of the label space, and b is an empty label; The formula for calculating information entropy is: Among them, |s| is the length of the input text, t is the label space, P i,j,t is the probability value of the label in the label matrix at each position in the matrix, i and j are variable indexes, Entropy_score is the information entropy, and Y is the label space of the model; The label space expression of the model is: Y=Y r ∪And e ∪{⊥} Among them, Y is the label space of the model, Y e is the entity label set, Y r is a set of relation labels, ⊥ is an empty label; The loss function formula is expressed as: in, is the cross entropy loss between the predicted label and the true label, is the constraint loss in the symmetric region, is the implicit loss of mutual constraints between relations and entities, is the total loss, |s| is the input text length, Y e is the entity label set, Y r is the set of relation labels, P i,:,t is the probability distribution of any label space in the i-th row, P :,i,t is the probability distribution of any label space in the jth column, i and j are variable indices, Y is the label space of the model, and t is the identifier of the label space.

6. The method for constructing an electric power knowledge graph based on active learning according to claim 5, characterized in that: The entity unified processing of the extracted entity relationships includes: based on knowledge judgment, removing erroneous entity relationships, performing entity-level entity vectorization representation on the extracted named entities, and calculating semantic similarity respectively according to the cosine formula. After calculating the similarity, setting the similarity threshold to 0.95, taking entities above the threshold as fusion objects, extracting main keywords from them, and delivering the entity list to be merged to experts for evaluation, judging whether the merged entities meet professional requirements and business logic, and selecting a unified entity list; The formula for calculating semantic similarity is: Among them, A is the semantic vector of entity 1, B is the semantic vector of entity 2, ∥A∥ is the modulus of the semantic vector of entity 1, and ∥B∥ is the modulus of the semantic vector of entity 2.

7. The method for constructing an electric power knowledge graph based on active learning according to claim 6, characterized in that: The visual display includes obtaining triples based on knowledge graph entity relationship extraction technology, using a python development kit import tool to import entity relationship data, and establishing and storing a power transaction knowledge graph; A knowledge graph query is performed on the transaction subject based on the Cypher language. The Cypher is a declarative query language exclusive to Neo4j. The data of power transaction knowledge is updated and queried based on the Cypher language.

8. A system using a method for constructing an electric power knowledge graph based on active learning as claimed in any one of claims 1 to 7, characterized in that: It includes text data acquisition and processing module, model fine-tuning module, entity relationship joint extraction module, entity unified processing module and visualization display module; The text data acquisition and processing module is used to collect reports, contracts and academic documents related to power transactions to provide a data source for building a knowledge graph, clean text data, and use the Huggingface text segmentation tool to segment the cleaned text into blocks to form samples suitable for model training; The model fine-tuning module is used to fine-tune the pre-trained RoBERTa model to better adapt to the language characteristics of the power field and output a word embedding language model in the power field; The entity relationship joint extraction module is used to extract entity relationships from the labeled samples, define entity and relationship label sets, predict the probability distribution of labels through the model, and realize the extraction of entity relationships; The entity unified processing module is used to perform entity-level vector representation of the extracted entity relationships, calculate semantic similarity, merge entities, and ensure entity consistency in the knowledge graph; The visualization display module is used to import the unified entity relationship data into the database, use Neo4j and py2neo tools to build and store the power transaction knowledge graph, and use Neo4j's Cypher query language to perform visualization and data query of the knowledge graph based on the triple data extracted from the knowledge graph.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the processor implements the steps of a method for constructing an electric power knowledge graph based on active learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for constructing an electric power knowledge graph based on active learning as described in any one of claims 1 to 7 are implemented.

Images