Hydropower centralized control service knowledge graph construction method and device based on mixed large model

Abstract

The invention discloses a hydroelectric centralized control service knowledge graph construction method and device based on a mixed large model. The method comprises the following steps: constructing a knowledge graph mode layer; establishing a mapping relationship between the unstructured data and the triple data; constructing a mixed large model; and outputting the water and electricity centralized control service knowledge graph triple through the mixed large model, and further obtaining a water and electricity centralized control service knowledge graph data layer. According to the method, the efficiency and precision of knowledge graph construction can be effectively improved, the method has important significance on intelligent operation, equipment management and the like of a centralized control center, and a new solution is provided for business decision making and fault diagnosis.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graph construction for hydropower centralized control business, and specifically relates to a method and device for constructing a knowledge graph for hydropower centralized control business based on a hybrid large model. Background Art

[0002] With the continuous development of the hydropower industry, the hydropower centralized control center has gradually become the core of power system dispatching and operation. However, the challenges faced by the centralized control center mainly come from the form of data it processes - a large amount of unstructured data. These data come from a variety of information sources such as equipment operation logs, fault records, dispatching procedures, maintenance manuals, etc., and the forms are complex and diverse, making it difficult to effectively integrate and utilize them through traditional linear analysis models. Existing data processing methods often lack sufficient flexibility and accuracy when dealing with the diversity and uncertainty of such data, resulting in slow information analysis and delayed decision support, thus affecting the overall operating efficiency and reliability of the centralized control center.

[0003] At present, the hydropower centralized control center relies on traditional rule-based models and manual analysis methods in the fields of equipment status monitoring, fault diagnosis, emergency response, etc. These methods often require a lot of manual intervention and the participation of domain experts, making it difficult to achieve rapid integration and real-time analysis of multi-source data. Moreover, due to the lack of automated information extraction methods, traditional methods are difficult to provide fast and accurate decision support in the face of emergency situations such as equipment failures and scheduling anomalies.

[0004] Traditional knowledge graph construction models (such as rule bases, bag-of-words models, etc.) still have shortcomings. When processing unstructured data, traditional models have difficulty accurately parsing the semantic information and complex associations in the data, resulting in low accuracy in information extraction and relationship modeling. At the same time, these models usually rely on rule-driven or shallow machine learning, making it difficult to mine deep semantic associations in large-scale, multi-source heterogeneous data, and have poor scalability and adaptability. Therefore, traditional models lack the ability to model global dependencies between data, limiting the ability of logical reasoning and inference.

[0005] The introduction of hybrid large models has become a promising technological innovation. By combining specialized modules for identifying unstructured data, hybrid large models can not only effectively reduce computational complexity, but also enhance the ability to capture global information of long sequence data. This model can extract refined features from multi-source heterogeneous data through multi-level and multi-modular design, and reveal the complex correlation structure between data to the greatest extent. For example, Chinese patent document CN117035076A discloses a method for automatically constructing domain knowledge graphs based on large language models and prompt engineering. The knowledge graph is constructed through a large language model. However, the current large model construction technology for knowledge graphs still has the following shortcomings: (1) It is general but not precise, and performs poorly in dealing with tasks in specific vertical fields, especially in the field of hydropower centralized control business; (2) The model is proficient in processing text data, but ignores the large amount of sequence data that will appear in the field of hydropower centralized control business, such as flow process, rainfall process, etc. Summary of the invention

[0006] In order to solve at least one of the problems existing in the prior art, the present invention proposes a method and device for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model, and uses a hybrid large model to analyze the massive heterogeneous data generated in the hydropower centralized control business, and constructs a business logic graph, equipment entity graph, concept graph, and case graph of the centralized control center. Compared with traditional rule-based models and manual analysis methods, it has higher model efficiency, can effectively improve the comprehensive utilization rate of data in the hydropower centralized control center, and can provide technical references for the daily operation of the hydropower centralized control center and emergency response to emergencies. It is of great significance to the development of the dispatching work of the hydropower centralized control center and the reduction of human and property losses caused by abandoned water and sudden accidents.

[0007] In order to achieve the purpose of the present invention, the present invention provides a method for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model, comprising the following steps: Relying on the data of the research area, the model layer of the knowledge graph of the centralized control center is constructed using the network ontology language OWL. The model layer includes the model layers of the business logic graph, equipment entity graph, concept graph and case graph of the centralized control center; Through open source data and annotated hydropower centralized control center heterogeneous data, a mapping relationship between unstructured data and triple data is established. The training set and test set of unstructured text mapping of sequence data, the training set and test set of triple mapping of text data, and the training set and test set of triple data mapping from sequence data and triple splicing data to information fusion are established respectively. The triple represents the relationship between one entity and another entity. Construct a hybrid large model, which includes an embedding layer, BERT1 layer, BERT2 layer, and Transformer fusion layer for capturing semantic relationships between data. The BERT1 layer uses sequence data as an independent variable and the water situation form description as a dependent variable; the BERT2 layer uses text data as an independent variable and the knowledge graph triple as a dependent variable; the Transformer fusion layer uses the TOKEN concatenation value of the expert analysis text and the knowledge graph triple after the sequence data is converted as an independent variable, and the output is a hydropower centralized control graph triple; adjust and optimize the model hyperparameters; Based on the pattern layer, the hybrid big model is used to extract triple data from the centralized control unstructured data, instantiate the business logic graph, equipment entity graph, concept graph and case graph of the centralized control center, and form the knowledge graph data layer of the hydropower centralized control business.

[0008] Furthermore, by collecting the business processes, operating specifications, historical water conditions data, work tickets and operation ticket data in the study area, a knowledge graph model layer of the control center based on the network ontology language OWL was constructed.

[0009] Furthermore, in the constructed model layer, the ontological relationship between salesmen, equipment, operations, concepts, and cases is constructed using scheduling procedures, business processes, operating specifications, historical water conditions data, work tickets, and operation ticket data; OWL is used to define the hierarchical relationship, attributes, and constraints between salesmen, equipment, operations, concepts, and cases, and to define the business logic graph model layer, the equipment entity graph model layer, the concept graph model layer, and the case graph model layer.

[0010] The built-in reasoning mechanism of the web ontology language OWL can be used to perform logical consistency checking and reasoning analysis on the defined pattern layer to automatically identify and correct potential inconsistencies.

[0011] Furthermore, the open source data may be data from open source databases such as DBpedia, Wikidata, and Freebase.

[0012] The open source database contains a large amount of unstructured data, entities, attributes and relationship information. The mapping relationship from unstructured data to structured triples is established by using the unstructured data and the corresponding entities, attributes and relationship information. According to the type of unstructured data, the unstructured data is classified into sequence data and text data, and the training set and test set of sequence data unstructured text mapping and text data triple group mapping are established respectively.

[0013] Furthermore, by converting the sequence data into expert analysis text, the expert analysis text and the triples extracted from the plain text data are used for knowledge fusion, and new triples that combine the information of the sequence data are extracted; then the triple annotations that fuse the sequence description information and the text description information are analyzed and annotated, and a training set and a test set are established to map the triple splicing data to the triple data after information fusion.

[0014] Furthermore, the hybrid model is a model that trains the BERT1 and BERT2 models separately in the sequence data recognition task and the text data recognition task, and semantically fuses the extracted features in the Transformer fusion layer with an attention mechanism.

[0015] Furthermore, in the hybrid large model, the main task of the embedding layer is to convert discrete input data into dense low-dimensional vector representations, and to capture the semantic relationship between data through word embedding methods. Its role is to reduce feature dimensions, optimize the computational performance of the model, reduce the consumption of memory and computing resources during the operation, and improve the expressiveness of data. During the mapping process, the embedding layer can better capture the semantic information of the input data, enhance the model's ability to understand complex features, and effectively reduce the impact of noise and irrelevant information; The Transformer fusion layer includes a multi-head attention layer and a feedforward neural network to capture the contextual relationship between words. Its multi-head attention mechanism is as follows: The input of the Transformer fusion layer is a one-dimensional vector , ,..., , a one-dimensional vector is by taking the output of the BERT1 layer , ,..., and the output of the BERT2 layer , ,..., spliced ​​together, where Represents the first The feature vector at each position, ∈ , Represents the length of the entire one-dimensional vector of the input, for each position , whose corresponding query vector , key vector Sum value vector It is obtained by the following linear transformation: ; ; ; in, , , are the linear transformation matrices for query, key, and value, respectively; By calculating the query vector Key vectors with other positions The similarity score between them is used to obtain the attention weight ; ; in, Represents the dimensions of the query vector and key vector.

[0016] Using attention weights to map the value vector Perform weighted summation to obtain the output attention vector of the corresponding position

[0017] ; BERT1 layer and BERT2 layer: Both BERT1 layer and BERT2 are BERT large models, which use the bidirectional encoder architecture in the Transformer model and multi-layer stacked Transformers to achieve deep semantic understanding of unstructured data. The BERT large model captures contextual information between sentences and words by introducing the masked language model MLM and the next sentence prediction NSP task during the training process.

[0018] The principle of the masked language model MLM is as follows: Given an input vector , ,..., , Indicates One-dimensional vectors, the masked language model MLM randomly masks some words in the input vector, recorded as masked sequence , ,..., , For the masked words; for each masked word , the masked language model MLM will generate a probability distribution , which represents the masked word In the given context The distribution of The principle of NSP for post-sentence prediction is as follows: Given an input sentence and sentences , Represents a TOKEN converted from one-dimensional vectors of different types, which is a sentence , Add the [CLS] tag as the start of the sentence pair, add the [SEP] tag to separate the two sentences, and merge the input sentences into an embedding vector ; Use a linear classifier to embed the vector Processing to get the sentence coherence score .

[0019] ; in represents the weight matrix of the linear classifier, represents the embedding vector, Represents the bias value of the linear classifier.

[0020] Furthermore, the cross entropy loss function is used to optimize the weight matrix of the linear classifier With offset value , the formula of the cross entropy loss function is as follows: ; in, represents the cross entropy loss of the linear classifier; Represents the total number of samples, that is, the number of sentence pairs in the training set; Indicates The true labels of samples; Represents the masked language model MLM or the next sentence prediction NSP for the The predicted probability of a sample.

[0021] The BERT1 layer is trained using the sequence data unstructured text mapping training set and test set, and the BERT2 layer is trained using the text data triple group mapping training set and test set. The collected vector features are fused in the Transformer fusion layer.

[0022] Furthermore, the dispatching procedures, business processes, operating specifications, historical water conditions data, work tickets and operation ticket data are input into the constructed hybrid large model, and the business logic graph triples, equipment entity graph triples, concept graph triples, and case graph triples are output to instantiate the business logic graph, equipment entity graph, concept graph, and case graph of the control center to form a knowledge graph data layer.

[0023] Furthermore, the steps of forming the hydropower centralized control business knowledge graph data layer include: Extract triples based on the hydropower centralized control knowledge graph triples in the format of "subject-predicate-object". The subject and object are defined as nodes in the knowledge graph, while the predicate represents the edge connecting the nodes; The graph database is used to import nodes and relationships to form a graph structure, create nodes and edges, and form a knowledge graph data layer for hydropower centralized control business.

[0024] The device for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model provided by the present invention includes the following modules: The model layer construction module is used to construct the model layer of the knowledge graph of the centralized control center using the network ontology language OWL. The model layer includes the business logic graph of the centralized control center, the equipment entity graph, the concept graph and the case graph. The pattern layer of the example map; The data set construction module is used to establish the mapping relationship between unstructured data and triple data through open source data and annotated hydropower centralized control center heterogeneous data, and respectively establish training sets and test sets for mapping sequence data unstructured text, training sets and test sets for mapping text data triples, and training sets and test sets for mapping sequence data and triple splicing data to triple data after information fusion, where a triple represents the relationship between one entity and another entity; The model building module is used to build a hybrid large model. The hybrid large model includes an embedding layer, a BERT1 layer, a BERT2 layer, and a Transformer fusion layer for capturing the semantic relationship between data. The BERT1 layer uses sequence data as an independent variable and the water situation form description as a dependent variable; the BERT2 layer uses text data as an independent variable and the knowledge graph triple as a dependent variable; the Transformer fusion layer uses the expert analysis text converted from sequence data and the TOKEN concatenation value of the knowledge graph triple as an independent variable, and the output is a hydropower centralized control graph triple; The knowledge graph construction module is used to extract triple data from the centralized control unstructured data based on the pattern layer and utilize the hybrid large model to instantiate the centralized control center business logic graph, equipment entity graph, concept graph and case graph to form the hydropower centralized control business knowledge graph data layer.

[0025] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) The present invention applies the hybrid large model to the construction of the knowledge graph of the centralized control center, which can realize the analysis and processing of heterogeneous data driven by actual semantics. Compared with the shortcomings of traditional knowledge graph construction models (such as rule bases, bag-of-words models, etc.), such as heavy workload for experts and inability to understand context, the hybrid large model only needs to build one model to complete the simultaneous analysis of a large amount of heterogeneous data. It also uses the BERT model's ability to understand semantics and can consider the semantic spatial correlation of all data instances after training. The accuracy and construction efficiency of the knowledge graph can be greatly improved.

[0026] (2) The present invention separates the sequence data task from the text data task, and uses BERT1 layer and BERT2 layer for training respectively, which reduces the mutual interference between semantic information and improves the model training efficiency and reasoning efficiency. At the same time, it conducts special optimization training for the large amount of sequence data (such as flow time series, etc.) in the hydropower centralized control and dispatching business, thereby realizing the specialization of the hydropower centralized control business model and being more suitable for the working scenarios of the hydropower centralized control business.

[0027] (3) The present invention constructs a knowledge graph in the vertical field of hydropower centralized control by integrating text data and sequence data through a hybrid large model, so that the constructed knowledge graph is more in line with actual business needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flow chart of a method for constructing a semantically driven hydropower centralized control business knowledge graph based on a hybrid large model according to an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of the business logic knowledge graph model layer of an embodiment of the present invention.

[0030] Figure 3 Schematic diagram of the hybrid large model structure used in the implementation of the present invention.

[0031] Figure 4 This is a schematic diagram of the business logic knowledge graph after the unstructured data is instantiated in the implementation of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not limited to the present invention.

[0033] The present invention provides a method for constructing a semantically driven hydropower centralized control business knowledge graph of a hybrid large model, such as Figure 1 As shown, the following steps are included: S1. Relying on the basic data of the study area, the network ontology language OWL is used to construct the model layer of the knowledge graph of the centralized control center. The knowledge graph model layer of the centralized control center includes the business logic graph model layer, the equipment entity graph model layer, the concept graph model layer and the case graph model layer.

[0034] In each constructed model layer, the ontological relationship between salespersons, equipment, operations, concepts (basic concepts of water conservancy), and cases (historical disposal cases) is constructed using dispatching procedures, business processes, operating specifications, historical water conditions data, work tickets, and operation ticket data; the network ontology language OWL is used to define the hierarchical relationship, attributes, and constraints between salespersons, equipment, operations, concepts, and cases, and to define the business logic graph model layer, equipment entity graph model layer, concept graph model layer, and case graph model layer.

[0035] This step collects the business processes, operating specifications, historical water conditions data, work tickets and operation ticket data of the study area, and constructs a knowledge graph model layer of the centralized control center based on the network ontology language OWL. In some embodiments of the present invention, a small-scale business logic graph is constructed by taking the local business logic graph as an example. In the constructed small-scale business logic graph, a small-scale business logic graph can be generated in a hybrid large model by using the remote centralized control city power outage scheduling procedure data and model layer. The remote centralized control city power outage logic graph model layer is one of the contents of the centralized control center business logic graph model layer. This part of the content data is now taken as an example, such as Figure 2 As shown in the figure, the business logic graph, equipment entity graph, concept graph and case graph of the centralized control center are all constructed using the same method under different data, that is, firstly building a model layer based on the network ontology language OWL, and then using a hybrid large model to construct an instance graph under the constraints of the model layer. Figure 2 As shown in the figure, a business logic graph pattern layer of remote centralized control of mains power outage is constructed based on the network ontology language OWL. Starting from the node "remote centralized control of mains power outage", the business logic graph pattern layer uses the edge "phenomenon" to connect the fault phenomenon nodes corresponding to the fault when it occurs. The edge "next step" indicates that when the execution condition is met, enter the next node. The "operation start point" node is a flag node, which indicates the operation to be taken next. The "operation start point" node corresponds to the measures to be taken by three different perspectives: "dispatcher", "dispatcher", and "power station side". In the perspective of the "dispatcher", there is an "edge" composed of conditional branches, namely "line forced transmission success" and "hydropower station bus differential protection action or line forced transmission failure", and different branch conditions correspond to different measures. The equipment entity graph pattern layer, the concept graph pattern layer, and the case graph pattern layer are all constructed based on the network ontology language OWL. Figure 2 The specific logic of the mode layer graph in is as follows: The business logic knowledge graph model layer creates a total of 15 nodes and 20 relationships, among which the 15 nodes include 1 plan node (remote emergency response plan for depressurization of the entire hydropower and new energy plant in the centralized control center), 1 situation node (remote centralized control, closed-door operation, depressurization of the entire plant), 4 phenomenon nodes and 8 operation nodes.

[0036] The 4 phenomenon nodes include: Phenomenon node 1: Phenomenon (non-monitoring - manual confirmation of protection action, accident tripping); Phenomenon node 2: Phenomenon (equipment monitoring - the two outgoing lines 201 and 202 of the first and second circuits tripped at the same time, and the 220kV line and bus lost voltage at the same time); Phenomenon node 3: Phenomenon (equipment monitoring - unit outlet switches 021, 022, 023 tripped); Phenomenon node 4: Phenomenon (equipment monitoring - 10kV and 400V factory power supplies all disappeared); The 8 operation nodes include: Operation node 1: Operation (dispatcher - report to the superior dispatching agency, disconnect all switches on the bus without waiting for orders, and notify the power station to try to restore power supply); Operation Node 2: Operation (dispatcher - report the situation to the superior organization and notify the power station to check the primary and secondary equipment); Operation Node 3: Operation (dispatcher - report the factors affecting the line force transmission to the superior dispatching agency, and report the details of fault recording and traveling wave ranging to the superior dispatching agency); Operation Node 4: Operation (Chief Dispatcher - Chief Dispatcher reports the situation to the person in charge of the dispatching operation department); Operation node 5: Operation (power station side - controlled power station reports equipment inspection status); Operation Node 6: Operation (Dispatcher - Power supply to the affected plant should be restored as soon as possible); Operation Node 7: Operation (dispatcher - the superior agency applies for remote startup or on-site disposal, performs remote startup or orders on-site personnel to handle the situation); Operation Node 8: Operation (Chief Dispatcher - Chief Dispatcher reports the handling situation to the department head and makes a record of the situation); The relationship between the nodes is: Plan node → Situation node: The relationship is "situation", which indicates the specific situation that the plan targets; Situation node → Phenomenon node 1: The relationship is "phenomenon", indicating a phenomenon in the situation; Situation node → Phenomenon node 2: The relationship is "phenomenon", indicating a phenomenon in this situation; Situation node → Phenomenon node 3: The relationship is "phenomenon", indicating a phenomenon in this situation; Situation node → Phenomenon node 4: The relationship is "phenomenon", indicating a phenomenon in this situation; Situation node → Operation starting point: The relationship is "Next", indicating the starting point of the operation process from the situation; Operation start point → Operation node 1: The relationship is "dispatcher", indicating the first operation of the dispatcher; Operation start point → Operation node 4: The relationship is "Scheduler", indicating the first operation of the scheduler; Operation start point → Operation node 5: The relationship is "power station side", indicating the first operation on the power station side; Operation Node 1 → Operation Node 2: The relationship is "Next", indicating the order of operations; Operation Node 2 → Operation Node 3: The relationship is "Next", indicating the order of operations; Operation Node 3 → Operation Node 6: The relationship is "Line Forced Delivery Successful", indicating the next operation in the case of success; Operation node 3 → Operation node 7: The relationship is "the busbar differential protection of the hydropower station is actuated or the line forced transmission is unsuccessful", indicating the next operation in the case of failure; Operation Node 4 → Operation Node 8: The relationship is "Next", indicating a long operation process.

[0037] By using the built-in reasoning mechanism of the web ontology language OWL, logical consistency checking and reasoning analysis are performed on each defined model layer to automatically identify and correct potential inconsistencies.

[0038] S2. Through the open source database and the annotated heterogeneous data of the hydropower centralized control center, the mapping relationship between unstructured data and triple data is established, and the training set and test set of the mapping of sequence data unstructured text, the training set and test set of the mapping of text data triples, and the training set and test set of the mapping of sequence data and triple splicing data to triple data after information fusion are established. The triple represents the relationship between one entity and another entity, and the format is (head entity, relationship, tail entity).

[0039] In some embodiments of the present invention, the required open source data can be extracted from open source databases such as DBpedia, Wikidata, Freebase, etc.

[0040] The open source database contains a large amount of unstructured data, entities, attributes and relationship information. The mapping relationship from unstructured data to structured triples is established by using the unstructured data and the corresponding entities, attributes and relationship information. According to the type of unstructured data, the unstructured data is classified into sequence data and text data, and the training set and test set of sequence data unstructured text mapping and the training set and test set of text data triple mapping are established respectively.

[0041] In some embodiments of the present invention, the steps of constructing the annotated heterogeneous data of the hydropower centralized control center include: Experts analyze the logical structure of daily operation documents such as procedures and work tickets, select key entities in the documents based on their operation experience, and establish relationships between key entities to form (head entity, relationship, tail entity) triple annotations; For sequence data, such as rainfall and flow, experts analyze the key concepts in the sequence data and provide relevant description standards; The annotations of the text description data (entity triples extracted from the text description data) and the annotations of the sequence data (expert analysis text based on the sequence data) are concatenated as the input of the Transformer fusion layer. The output of the Transformer fusion layer is the hydropower centralized control map triple.

[0042] By converting sequence data into expert analysis text, the knowledge fusion is performed using the expert analysis text and the triplets extracted from plain text data such as scheduling procedures and work tickets, and new triplets that combine the information of sequence data are extracted. Then the expert analysis annotation fuses the triple annotations of sequence description information and text description information, and establishes the training set and test set of triple splicing data to the triple data after information fusion.

[0043] Labeled heterogeneous data of hydropower centralized control center is used to optimize the performance of hybrid large models.

[0044] S3. Build a hybrid large model. The hybrid large model includes an embedding layer, a BERT1 layer, a BERT2 layer, and a Transformer fusion layer. For the BERT1 layer, the sequence data is used as the independent variable, and the water situation form is described as the dependent variable; for the BERT2 layer, the text data is used as the independent variable, and the knowledge graph triple is used as the dependent variable; for the Transformer fusion layer, the TOKEN concatenation value of the expert analysis text converted from the sequence data and the knowledge graph triple extracted by BERT2 is used as the independent variable, and the knowledge graph triple is used as the dependent variable, and the model hyperparameters are adjusted and optimized. The input and output data structures used in the hybrid large model are as follows: Figure 3 .

[0045] In some embodiments of the present invention, the independent variables of the BERT1 layer include historical rainfall data and flow data. The rainfall lasts for 180 minutes, and the rainfall every 10 minutes is used as a feature, with a total of 18 features. The flow data includes the flow changes in the past 30 minutes, with a total of 18 flow features. The dependent variable is a formal description text of the flood control and power generation tasks under the current rainfall and flow conditions. For the sequence data generated under 60 simulated rainstorm waterlogging scenarios, they are randomly divided into training sets and test sets in a ratio of 70:30. The training set contains 42 samples and the test set contains 18 samples, which are used to evaluate the performance of the BERT1 model.

[0046] The main contents of the triples of the text data of the BERT2 layer include topics, related terms and background information. The structure of the triple is (head entity, relationship, tail entity), with the topic as the input feature and the related terms and background information as the output features. The topic refers to the name, approximation, colloquialism, etc. of the current main water conservancy task; the related terms refer to the relative pronouns that appear in a series of water conservancy tasks generated around the topic; the background information is various concepts and important knowledge points related to the topic. In some embodiments of the present invention, a total of 100 topics are extracted, and 300 related terms are associated. The extracted topics include "flood risk", "drainage system", "reservoir management", etc. The related terms include "inclusive expansion", "drainage", "flood storage", etc., and the background information includes "basic geographic information", "topography", etc. In the text data processing, the text is divided according to the completeness and information richness, and a 60:40 ratio is used for random allocation to ensure the consistency of the text data triple mapping training set and the test set in the topic distribution. The training set contains 60 topics and the test set contains 40 topics.

[0047] In some embodiments of the present invention, the Transformer fusion layer is trained with a concatenated TOKEN mixed data set with a ratio of 1:1 between knowledge graph triples and sequence data. The BERT1 layer is trained with a sequence data unstructured text mapping training set, and the BERT2 layer is trained with a text data triple mapping training set. The input and output data structures used in the hybrid large model are specifically shown in Table 1.

[0048] Table 1 Input and output examples of hybrid large models

[0049] The hybrid model is a model that trains the BERT1 and BERT2 models separately in the sequence data recognition task and the text data recognition task, and uses the attention mechanism to semantically fuse the extracted features in the Transformer fusion layer. The main task of the embedding layer is to convert discrete input data, i.e. sequence data and text data, into dense low-dimensional vector representations, and to capture the semantic relationship between data through word embedding methods. Its role is to reduce feature dimensions, optimize the computing performance of the hybrid large model, reduce the consumption of memory and computing resources during the operation, and improve the expressiveness of data. In the mapping process, the embedding layer can better capture the semantic information of the input data, enhance the hybrid large model's ability to understand complex features, and effectively reduce the impact of noise and irrelevant information, such as Figure 3 As shown, , ,..., A single TOKEN representing sequence data, , ,..., A single TOKEN representing text data. The Transformer fusion layer includes a multi-head attention layer and a feedforward neural network, which are used to capture the contextual relationship between words. The feedforward neural network includes a feedforward layer and an embedding layer. The input of the Transformer fusion layer is a one-dimensional sequence, and the working mechanism of its multi-head attention layer is as follows: Input one-dimensional vector , ,..., , a one-dimensional vector is by taking the output of the BERT1 layer , ,..., and the output of BERT2 layer , ,..., spliced ​​together, where Represents the first The feature vector at each position, ∈ , Represents the length of the entire one-dimensional vector of the input, for each position , whose corresponding query vector , key vector Sum value vector It can be obtained by the following linear transformation: ; ; ; in, , , are the linear transformation matrices for query, key, and value, respectively. By calculating the query vector Key vectors with other positions The similarity score between them can get the attention weight : ; in, Represents the dimensions of the query vector and key vector.

[0050] After obtaining the attention weight, use the attention weight to pair the value vector Perform weighted summation to obtain the output attention vector of the corresponding position : ; in, Indicates Data for each location; The feedforward layer is a simple linear connection layer, and the calculation formula is as follows: ; in Indicates The output of the position component, Represents the feed-forward layer transformation matrix.

[0051] Both the BERT1 layer and the BERT2 layer are large BERT models. Through the bidirectional encoder architecture in the Transformer model, they use multi-layer stacked Transformer encoders to achieve deep semantic understanding of unstructured data.

[0052] The BERT model captures contextual information between sentences and words by introducing the masked language model MLM and the next sentence prediction NSP task during the training process. The principles of the masked language model MLM and the next sentence prediction NSP are as follows: Masked Language Model MLM: Given an input vector , ,..., , Indicates A one-dimensional vector with a certain number of dimensions. TOKEN or the intermediate variables in the model operation are all in the form of one-dimensional vectors; the masked language model MLM randomly masks some words in the input vector, recorded as a masked sequence , ,..., , For the A masked one-dimensional vector, which can be regarded as a word here; for each masked word , the masked language model MLM will generate a probability distribution , which represents the masked word In the given context The distribution in .

[0053] Next sentence prediction NSP: Given an input sentence and sentences , Represents a TOKEN converted from one-dimensional vectors of different types, which is a sentence , Add the [CLS] tag as the start of the sentence pair, add the [SEP] tag to separate the two sentences, and merge the input sentences into an embedding vector ; Use a linear classifier to embed the vector Processing to get the sentence coherence score : ; in, represents the weight matrix of the linear classifier, represents the embedding vector, Represents the bias value of the linear classifier.

[0054] Using the cross entropy loss function to optimize the weight matrix of the linear classifier With offset value , the formula of the cross entropy loss function is as follows: ; in, represents the cross entropy loss of the linear classifier; Represents the total number of samples, that is, the number of sentence pairs in the text data triple mapping training set; Indicates The true labels of samples; Indicates the masked language model MLM or the next sentence prediction NSP for the The predicted probability of samples. Position and There is a one-to-one correspondence between samples, which essentially masks the first position, Each position is considered as a sample.

[0055] The BERT1 layer is trained using the sequence data unstructured text mapping training set and test set, and the BERT2 layer is trained using the text data triplet mapping training set and test set. The collected vector features are fused in the Transformer fusion layer to output the hydropower centralized control knowledge graph triples. Subsequently, based on the hydropower centralized control knowledge graph triples, the hydropower centralized control knowledge graph can be created using the graph database.

[0056] When constructing a large hybrid model, parameters must be set and optimized. In some embodiments of the present invention, the parameters of the large hybrid model are optimized by a grid search method and a 5-fold cross validation method. The optimized parameter settings are shown in Table 2.

[0057] Table 2 Parameters and values ​​of the BERT1 layer, BERT2 layer, and Transformer fusion layer used

[0058] S4. Input the unstructured data into the hybrid big model, output the business logic graph triples, equipment entity graph triples, concept graph triples, and case graph triples, instantiate the business logic graph, equipment entity graph, concept graph, and case graph of the centralized control center, and form a knowledge graph data layer for the hydropower centralized control business.

[0059] Based on the pattern layer, a hybrid big model is used to convert unstructured data (centralized control center dispatching procedures, business processes, operating specifications, historical water conditions data, work tickets and operation ticket data) into triple data. The triple data extracts the key information in the above unstructured data, and establishes a directed graph relationship between the key information, instantiates the central control center business logic graph, equipment entity graph, concept graph, and case graph, and forms a knowledge graph data layer for hydropower centralized control business.

[0060] The process of constructing the business logic graph of the centralized control center based on the hydropower centralized control knowledge graph triples produced by the hybrid large model (the process of equipment entity graph, concept graph, and case graph is consistent with the process of building business logic, but the data and model layers are different) includes extracting triples in the format of "subject-predicate-object", covering the equipment, processes, and responsible persons of the centralized control center. Subjects and objects are defined as nodes in the knowledge graph, while predicates represent the edges connecting these nodes. For example, in the triple (emergency work group, responsible, equipment maintenance), "emergency work group" and "equipment maintenance" are nodes, and "responsible" is the connecting edge.

[0061] Nodes and relationships are imported into a graph database to form a graph structure, and nodes and edges are created through the query language of the graph database using the corresponding syntax (in some embodiments of the present invention, the graph database may use Neo4j, and the syntax may use Cypher. It is understandable that other graph databases and syntax may also be used in other embodiments). At the same time, additional node attributes (such as device status, responsible person, etc.) and complex relationships (such as the sequence of events) are added. Finally, the graph is checked for rationality using an inference engine to ensure logical consistency. In some embodiments of the present invention, the example results are as follows: Figure 4 , Figure 4 It includes key objects such as "batteries", "USP power supplies", and "environmental monitoring systems", business personnel such as "centralized control dispatchers" and "emergency work groups", and measures that need to be taken, such as "prepare tools and materials", "go to the scene to deal with it", "turn off unnecessary equipment", "notify the emergency work group", and "check the operation of equipment". These node projects are connected by the relationships of "trigger", "facility", "execution", "leadership", "dissatisfaction", and "monitoring". This knowledge graph can meet basic business logic relationships and can be applied to logical reasoning and judgment.

[0062] In some embodiments of the present invention, a device for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model is provided, and the device includes the following modules: A model layer construction module is used to construct a model layer of the knowledge graph of the centralized control center using the network ontology language OWL, wherein the model layer includes the model layers of the business logic graph, the equipment entity graph, the concept graph and the case graph of the centralized control center; The data set construction module is used to establish the mapping relationship between unstructured data and triple data through open source data and annotated hydropower centralized control center heterogeneous data, and respectively establish training sets and test sets for mapping sequence data unstructured text, training sets and test sets for mapping text data triples, and training sets and test sets for mapping sequence data and triple splicing data to triple data after information fusion, where a triple represents the relationship between one entity and another entity; The model building module is used to build a hybrid large model. The hybrid large model includes an embedding layer, a BERT1 layer, a BERT2 layer, and a Transformer fusion layer for capturing the semantic relationship between data. The BERT1 layer uses sequence data as an independent variable and the water situation form description as a dependent variable; the BERT2 layer uses text data as an independent variable and the knowledge graph triple as a dependent variable; the Transformer fusion layer uses the expert analysis text converted from sequence data and the TOKEN concatenation value of the knowledge graph triple as an independent variable, and the output is a hydropower centralized control graph triple; The knowledge graph construction module is used to extract triple data from the centralized control unstructured data based on the pattern layer and utilize the hybrid large model to instantiate the centralized control center business logic graph, equipment entity graph, concept graph and case graph to form the hydropower centralized control business knowledge graph data layer.

[0063] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined in the present invention may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in the present invention, but will conform to the widest scope consistent with the principles and novel features disclosed in the present invention.

Claims

1. A method for constructing a knowledge graph for hydropower centralized control business based on a hybrid large model, characterized in that: The steps include: The model layer of the knowledge graph of the centralized control center is constructed by using the network ontology language OWL; the model layer of the knowledge graph of the centralized control center includes the model layers of the business logic graph of the centralized control center, the equipment entity graph, the concept graph and the case graph; Through open source data and labeled heterogeneous data of hydropower centralized control center, a mapping relationship between unstructured data and triple data is established; Based on the model layer of the centralized control center knowledge graph, a pre-built hybrid large model is used to extract triple data from the centralized control unstructured data, instantiate the centralized control center knowledge graph, and form a hydropower centralized control business knowledge graph data layer; The hybrid large model includes an embedding layer for capturing semantic relationships between data, a BERT1 layer, a BERT2 layer, and a Transformer fusion layer. The BERT1 layer takes sequence data as an independent variable and the water situation form description as a dependent variable; the BERT2 layer takes text data as an independent variable and the knowledge graph triple as a dependent variable; the Transformer fusion layer takes the expert analysis text converted from sequence data and the TOKEN concatenation value of the knowledge graph triple as an independent variable, and the output is a hydropower centralized control graph triple.

2. The method for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model according to claim 1 is characterized in that: The method of establishing a mapping relationship between unstructured data and triple data includes: establishing a training set and a test set for mapping sequence data to unstructured text, a training set and a test set for mapping text data to triples, and a training set and a test set for mapping sequence data and triple concatenated data to triple data after information fusion, wherein a triple represents a relationship from one entity to another.

3. The method for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model according to claim 1 is characterized in that: In the constructed model layer, the ontological relationship between salesmen, equipment, operations, concepts, and cases is constructed using scheduling procedures, business processes, operating specifications, historical water conditions data, work tickets, and operation ticket data. The network ontology language OWL is used to define the hierarchical relationship, attributes, and constraints between salesmen, equipment, operations, concepts, and cases, and to define the business logic graph model layer, equipment entity graph model layer, concept graph model layer, and case graph model layer.

4. The method for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model according to claim 1 is characterized in that: The open source data contains a large amount of unstructured data, entities, attributes and relationship information. The unstructured data and the corresponding entities, attributes and relationship information are used to establish a mapping relationship from unstructured data to structured triples. According to the type of unstructured data, a training set and a test set for unstructured text mapping of sequence data, and a training set and a test set for text data triple group mapping are established respectively.

5. The method for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model according to claim 1 is characterized in that: By converting sequence data into expert analysis text, using the expert analysis text and triples extracted from plain text data for knowledge fusion, new triples combining information from sequence data are extracted; Then, the triple annotations that fuse the sequence description information and the text description information are analyzed and annotated, and a training set and a test set are established to map the triple splicing data to the triple data after information fusion.

6. The method for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model according to claim 1 is characterized in that: The Transformer fusion layer includes a multi-head attention layer and a feed-forward neural network, which is used to capture the contextual relationship between words; The BERT1 layer and BERT2 are both BERT large models, which use the bidirectional encoder architecture in the Transformer model and multi-layer stacked Transformers to achieve deep semantic understanding of unstructured data. The BERT large model captures contextual information between sentences and words by introducing the masked language model MLM and next sentence prediction NSP during the training process.

7. The method for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model according to claim 6 is characterized in that: The following sentence prediction NSP includes: given an input sentence and sentences , Represents a TOKEN converted from one-dimensional vectors of different types, which is a sentence , Add the [CLS] tag as the start of the sentence pair, add the [SEP] tag to separate the two sentences, and merge the input sentences into an embedding vector ; Use a linear classifier to embed the vector Processing to get the sentence coherence score .

8. The method for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model according to claim 7 is characterized in that: The weight matrix and bias value of the linear classifier are optimized using the cross entropy loss function. The formula of the cross entropy loss function is as follows: in, represents the cross entropy loss of the linear classifier; represents the total number of samples; Indicates The true labels of samples; Indicates the masked language model MLM or the next sentence prediction NSP for the The predicted probability of a sample.

9. The method for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model according to any one of claims 1 to 8, characterized in that: The steps to form the hydropower centralized control business knowledge graph data layer include: Extract triples based on the hydropower centralized control knowledge graph triples in the format of "subject-predicate-object". The subject and object are defined as nodes in the knowledge graph, while the predicate represents the edge connecting the nodes; The graph database is used to import nodes and relationships to form a graph structure, create nodes and edges, and form a knowledge graph data layer for hydropower centralized control business.

10. A device for implementing the method for constructing a knowledge graph of hydropower centralized control business based on a hybrid large model as described in claim 1, characterized in that: The device comprises the following modules: The model layer construction module is used to construct the model layer of the knowledge graph of the centralized control center using the network ontology language OWL; The data set construction module is used to establish the mapping relationship between unstructured data and triple data through open source data and labeled heterogeneous data of the hydropower centralized control center; The model building module is used to build a hybrid large model. The hybrid large model includes an embedding layer, a BERT1 layer, a BERT2 layer, and a Transformer fusion layer for capturing the semantic relationship between data. The BERT1 layer uses sequence data as an independent variable and the water situation form description as a dependent variable; the BERT2 layer uses text data as an independent variable and the knowledge graph triple as a dependent variable; the Transformer fusion layer uses the expert analysis text converted from sequence data and the TOKEN concatenation value of the knowledge graph triple as an independent variable, and the output is a hydropower centralized control graph triple; The knowledge graph construction module is used to extract triple data from the centralized control unstructured data based on the pattern layer and utilize the hybrid large model to instantiate the centralized control center business logic graph, equipment entity graph, concept graph and case graph to form the hydropower centralized control business knowledge graph data layer.

Images