Power plant metal supervision entity relationship extraction method based on dual coding

Abstract

The invention belongs to the technical field of new-generation information, and particularly relates to a power plant metal supervision entity relationship extraction method based on dual coding, which comprises the following steps: multi-modal data preprocessing: collecting and cleaning data, and carrying out data labeling; constructing a dual coding joint learning model: designing a network layer architecture, and training the dual coding joint learning model; constructing and querying a dynamic knowledge graph: extracting a model to generate a triple, and performing time sequence evolution analysis, causal reasoning interface and dynamic updating; and incremental knowledge updating and dynamic model optimization: an incremental learning and feedback module forms a bidirectional closed loop between the knowledge graph and the relationship extraction model, and continuous evolution of the system is ensured through dynamic knowledge updating and adaptive model optimization. According to the method, through bidirectional feature modeling of dual paths, collaborative representation optimization between entities and relationships is realized while context information is captured, so that the requirements of complicated data types and diversified semantic associations in an engineering scene are met.

Description

Technical Field

[0001] The invention belongs to the field of new generation information technology, and in particular relates to a method for extracting metal supervision entity relationships in a power plant based on dual coding. Background Art

[0002] As a crucial component in ensuring the safe operation of power industry equipment, power plant metal inspection primarily covers the inspection, evaluation, and maintenance of key components, including metal materials, welding quality, main steam piping, reheat steam piping, heating surface pipes, boiler drums, headers and feedwater piping, turbine generator rotors, high-temperature bolts, and large castings. With the widespread use of flexible operating units and frequent deep peak-shaving operations, the metal components of these critical equipment are exposed to increased fatigue stress, temperature fluctuations, and complex operating environments, exacerbating the generation and development of defects. This significantly increases the workload of metal inspection, and traditional manual analysis methods, unable to process large amounts of paper reports and unstructured data, are no longer able to meet the efficient operation requirements of modern power plants.

[0003] Currently, power plant technical archives include multi-dimensional information, including historical operating data, structural property records, metal wear and explosion inspections, annual pressure vessel inspection records, equipment accident and abnormality analysis records, chemical monitoring and pickling records, and more. These archives are massive and complex, making management difficult and data utilization inefficient. This makes it difficult for operators and maintenance personnel to quickly and accurately identify defects requiring attention, thus impacting equipment safety and reliability. Furthermore, with the rapid development of new energy and smart power plant technologies, the need for dynamic fault diagnosis and knowledge sharing is becoming increasingly urgent. Traditional single-task data analysis methods are no longer sufficient to meet the current complex metal monitoring needs. Summary of the Invention

[0004] In response to the above-mentioned problems existing in the prior art, the present invention takes the power plant metal supervision scenario as the core and proposes a power plant metal supervision entity relationship extraction method based on dual coding.

[0005] In order to achieve the above effects, the technical solutions adopted by the present invention are as follows:

[0006] The method for extracting entity relations of metal supervision in power plants based on dual coding includes the following steps:

[0007] Step S1. Multimodal data preprocessing: collecting and cleaning the data, and labeling the data;

[0008] Furthermore, the step S1 includes the following steps:

[0009] Step S1.1. Data collection and cleaning

[0010] Multimodal data is collected from the power plant's technical archives and divided into structured data and unstructured data. Structured data includes historical operating data, structural property ledgers, pressure vessel annual inspection records, chemical supervision and pickling records. Unstructured data includes metal anti-wear and explosion-proof inspection report texts, metal inspection report texts, equipment accident and abnormality analysis records, and metal X-ray inspection images. Cleaning operations are performed on the above collected data: missing values in structured data are filled by reverse inference through associating historical maintenance records; for unstructured image data, the resolution pixels are uniformly adjusted and grayscale processing is performed.

[0011] Step S1.2. Data annotation

[0012] After completing data cleaning, we began to annotate the multimodal data. For text data, we used the BIO format to annotate the equipment, defects, and parameters. For X-ray inspection images, we used the image labeling tool to annotate the bounding box coordinates of the defect area. The annotation results must ensure standardized alignment between the image and text annotations.

[0013] Step S2. Dual coding joint learning model construction: Design the network layer architecture and perform dual coding joint learning model training;

[0014] Step S2 specifically includes the following steps:

[0015] Step S2.1. Design the network layer architecture

[0016] The dual encoding joint learning model consists of the following four layers of networks, which achieve joint extraction of entities and relations through layered collaboration:

[0017] The first layer is the dynamic word embedding layer, which is responsible for converting the input text sequence into a context-aware dynamic word vector.

[0018] Specifically, the pre-trained BERT model is used to generate the embedding vector of each word The dimension of the embedding vector is set to 768. During the training process, the word embedding parameters are shared between entity recognition and relation classification tasks to enhance the word vector's ability to capture semantic associations.

[0019] The second layer is a bidirectional long short-term memory network layer. The bidirectional long short-term memory network layer receives the output of the dynamic word embedding layer and simultaneously introduces a forward LSTM network and a backward LSTM network to process the input sequence from both sides to capture more comprehensive semantic information.

[0020] The processing process is that the forward LSTM processes the input sequence from left to right and generates a forward hidden state sequence {h f1 ,h f2 ,…,h fn}; The reverse LSTM processes the same input sequence from right to left and generates a backward hidden state sequence {h b1 ,h b2 ,…,h bn Through semantic fusion, the hidden state sequence of the bidirectional LSTM with the forward and reverse hidden states is represented as h t =[h ft ;h bt ], h t It also contains the semantic information before and after the current moment.

[0021] The third layer is the multi-head attention mechanism layer, which introduces a parallel attention module to focus on key entities and relationship indicators in the text.

[0022] Specifically, we set up independent attention heads, each of which calculates the query matrix Q, key matrix K, and value matrix V. We dynamically assign weights by scaling dot product attention to improve the model's ability to capture key features. By computing multiple attention matrices in parallel, we can enhance our focus on important tokens in the sentence.

[0023] Among them, the attention mechanism calculates the attention weight by mapping the query matrix Q, the key matrix K and the value matrix V:

[0024]

[0025] where d k is the scaling factor for the matrix dimensions.

[0026] The fourth layer is the global normalization and joint optimization layer, which solves the problem of inter-task goal conflict by jointly optimizing entity recognition and relationship classification tasks.

[0027] Specifically, the input of the global normalization layer is the hidden state sequence of the bidirectional LSTM obtained from the previous layers, and the score vector v of the entity and relationship categories is generated through the fully connected layer. z :

[0028]

[0029] Where W z is the parameter matrix, h z is the hidden layer feature vector, v z is a category score vector, which represents the category possibility of entities and relations; in the joint optimization process, the objective function is expressed as:

[0030]

[0031] in are entity types, is the relationship type;

[0032] In the decoding stage of the model, the conditional random field method is used to find the optimal sequence through the Viterbi algorithm. Given the label sequence y={y1,y2,…,y n}, the model needs to calculate the sequence score:

[0033]

[0034] Where Q(y i ,y i+1 ) represents the transfer score between labels, d(y i ) represents the label score of position i. Finally, the model outputs the optimal label sequence:

[0035]

[0036] Through the collaborative optimization of the global normalization layer, the model can improve the complementarity and robustness between tasks while ensuring the performance of a single task.

[0037] Step S2.2. Model training

[0038] Step S2.2.1. Parameter initialization

[0039] A hierarchical strategy is adopted in the parameter initialization stage of the model: the dynamic word embedding layer loads the weights of the BERT-base pre-trained model and freezes some underlying parameters to accelerate convergence, only adjusting the upper-layer parameters to adapt to the semantic characteristics of the power plant metal supervision field; the bidirectional long short-term memory network layer sets the number of hidden units and the dropout rate, and prevents overfitting by randomly dropping some neurons; each attention head in the multi-head attention layer independently initializes the query, key, and value matrices, and the outputs are concatenated and reduced in dimensionality through linear transformation to ensure the stability of feature fusion.

[0040] Step S2.2.2. Training strategy

[0041] Multiple optimization strategies are adopted during the training process: insufficient text paragraphs are padded, and the influence of invalid positions is ignored through a masking mechanism; the optimizer uses an adaptive learning rate algorithm, sets the initial learning rate, and adds weight decay to control the parameter amplitude; the dynamic adjustment mechanism automatically adjusts the learning rate and task loss weight according to the performance of the validation set, gradually enhancing the model's ability to capture complex relationships; an early stopping mechanism is set to terminate training when the validation loss has not decreased for multiple consecutive rounds to avoid invalid calculations.

[0042] Step S2.2.3. Model validation

[0043] To verify the performance of the model, typical test text was input, and the model output was consistent with the expected entity recognition and relationship classification results.

[0044] Step S3. Dynamic knowledge graph construction and query: Extract the model to generate triples, perform temporal evolution analysis, and then perform causal reasoning interface and dynamic update;

[0045] Furthermore, the specific steps are as follows:

[0046] Step S3.1. Triple extraction and storage

[0047] This system uses a relation extraction model based on joint decoding to generate triples, including a parameter-sharing joint model and a joint decoding model, and realizes the synchronous extraction of entities and relations through end-to-end joint decoding.

[0048] Specifically, the model design transforms entity recognition and relation classification into a sequence labeling problem, and defines special label types to simultaneously represent entity boundaries and relation semantics.

[0049] The model architecture includes the following core mechanisms:

[0050] Step S3.1.1. Establishing a parameter-sharing joint model

[0051] The process of extracting triples using a parameter-sharing joint model is divided into multiple steps and adopts a dual network architecture, an entity detection network and a relation extraction network.

[0052] Among them, the entity detection network encodes the text sequence based on a bidirectional recurrent neural network to generate fused entity features:

[0053]

[0054] Where E(X) is the word embedding matrix, middle represents a set of real numbers, n represents the length of the input sequence, and d represents the hidden layer dimension.

[0055] Based on the bidirectional recurrent neural network with dependency tree structure, the input fusion entity feature H ent And the dependency syntax tree node information, output entity pairs (e i ,e j )’s relationship probability:

[0056] P(r|e i ,e j )=Softmax(W r [h i ;h j ;h dep ]+b r )

[0057] where e i ,e j Represents the head entity and tail entity in an entity pair; h i,h j Represents the head entity e i and tail entity e j The hidden state vectors are generated by the entity detection network and encode the semantic information of the entity in the context; dep Represents the dependency tree path feature, encoding the grammatical relationship between entity pairs through the dependency syntactic analysis tree; W r ,b r are learnable parameters.

[0058] The entity and relation networks share some hidden layer parameters of the word embedding layer, and the loss function is the weighted subtask loss:

[0059] L joint = entity +βL relation

[0060] Among them L entity Cross entropy loss for entity recognition, used to optimize entity boundaries and type classification. rElAtion is the cross entropy loss for relation classification, which is used to optimize the entity-to-relation type. α is the entity recognition task (L entity ) is the weight coefficient that controls the contribution ratio of the task to the total loss. β is the relationship classification task (L relation ) to control the contribution ratio of the task to the total loss.

[0061] Step S3.1.2. Establish a joint decoding model

[0062] A label type is designed for the relation extraction model based on joint decoding. The extended label set L = {h, r, t} is defined, where h represents the head entity, r represents the relation, and t represents the tail entity. Entity recognition and relation classification are transformed into sequence labeling problems. An end-to-end model is then adopted. This model uses a bidirectional long short-term memory network to generate a contextual hidden vector and encodes the sentence through the encoding layer:

[0063]

[0064] in, represents a real matrix, Represents a set of real numbers, n represents the length of the input sequence, and 2d represents the hidden state dimension of the bidirectional long short-term memory network. The decoding layer decodes the hidden layer vector input and uses the improved single long short-term memory network combined with the label transfer matrix to gradually predict the label sequence:

[0065] P(y t |y t-1 ,H)=Softmax(W d h t +b d +Tyt-1,yt )

[0066] where y t represents the predicted label of the current time step t; y t-1 represents the historical label of the previous time step t-1; W d is the weight matrix of the linear transformation, transforming the hidden state h t Mapping to label space; h t represents the hidden state vector of the encoder at time step t; b d The bias vector representing the linear transformation; Indicates the label transfer matrix from label y t-1 to y t The transfer score of ,establishes the dependency relationship between labels.

[0067] Then the triples are directly output. The system stores the entity-relationship triples {h, r, t} extracted by the model, where h, r, and t represent the head entity, relationship, and tail entity, respectively, into a structured knowledge graph, and defines the core node types (equipment, defect, event, parameter) and their attributes: the equipment node contains ID, model, and material fields; the defect node records type, location, and size attributes; the relationship types include "cause", "associated parameters", and "repair plan", and a timestamp field is attached to each relationship to form a structured knowledge network for the entire life cycle of the equipment.

[0068] Step S3.2. Perform temporal evolution analysis

[0069] Based on the timestamp field, knowledge graph snapshots are generated according to preset periods, a multi-dimensional time-series evolution model of the device status is constructed, and the dynamic changes of the device status are recorded. At the same time, knowledge graph snapshots are generated according to the time dimension, which supports the visual display of the device status evolution path and provides historical version comparison function to facilitate tracing the root cause of degradation.

[0070] Step S3.3. Provide causal reasoning interface and dynamic update

[0071] It provides a query interface and supports complex causal chain tracing. When the model outputs new triples through incremental learning, the system automatically triggers the knowledge graph update process: the new nodes and relationships are stored in real time after permission verification, the graph version is updated synchronously, and the operation log is recorded to ensure the timeliness and security of the knowledge base.

[0072] Step S4. Incremental knowledge update and dynamic model optimization: The incremental learning and feedback module forms a bidirectional closed loop between the knowledge graph and the relationship extraction model, ensuring the continuous evolution of the system through dynamic knowledge update and adaptive model optimization;

[0073] Step S4.1. Incremental knowledge update

[0074] First, the incremental learning part compares the newly extracted triples (h, r, t) with the existing knowledge in the knowledge base, and determines whether to add them to the knowledge base through confidence threshold filtering. The confidence of the new knowledge is defined as

[0075] P new =max(P(h,r,t))

[0076] Among them, P(h,r,t) is the model's predicted probability of the triple. When P new When the value is greater than θ, the system stores it in the knowledge base as incremental data. Otherwise, it enters the active learning queue and waits for manual review or further model learning. θ is the set credibility threshold.

[0077] The system uses entity alignment strategy to detect redundant information and ensure data consistency. If the similarity between the knowledge (h′, r′, t′) in the existing knowledge base and (h, r, t) is

[0078] S=cos(v (h,r,t) ,v (h′,r′,t′) )

[0079] If S>δ is satisfied, it is judged as redundant and not stored in the knowledge base. Otherwise, it is stored as new knowledge. δ is the set similarity threshold.

[0080] Step S4.2. Dynamic model optimization

[0081] The feedback mechanism maintains the adaptability of the relation extraction model through dynamic model optimization, and adopts the knowledge distillation method to optimize the original model P T (y|x) guides the new model P S (y|x), and optimize its learning process with distillation loss:

[0082]

[0083] Where x is the input text, y i is a possible tag, P T (y i |x) is the predicted probability of the original model, P S (y i |x) is the predicted probability of the new model. A user rating mechanism is designed to allow experts or end users to score the relationships extracted by the model R∈[0,1]. If the score is below the set threshold, the relationship is re-labeled and added to the model training set to ensure that the knowledge base and model maintain high quality over the long term.

[0084] Ultimately, the incremental learning and feedback module ensures that the knowledge base and the relationship extraction model are updated synchronously, forming a knowledge-driven self-optimization closed loop, continuously improving the accuracy of entity relationship extraction, and enabling the system to have long-term evolution capabilities.

[0085] The technical effects achieved by the present invention are:

[0086] 1. This paper proposes a joint entity-relationship extraction method based on a multi-task dual encoding framework. Through bidirectional feature modeling of dual paths, it achieves collaborative representation optimization between entities and relationships while capturing contextual information. A multi-head attention mechanism layer assigns higher weights to keywords through parallelized attention modules, while dynamically adjusting the focus of entities and relationships to address the complex data types and diverse semantic associations required in engineering scenarios.

[0087] 2. The present invention coordinates the objective functions of entity recognition and relationship classification through task joint optimization via a global normalization layer to maximize the capture of the dependencies of entity relationships in the text, dynamically construct a knowledge graph, and provide technical support for operation status monitoring, equipment defect analysis, and fault diagnosis.

[0088] 3. By integrating the core concepts of multi-task learning and dual coding, the model of this invention realizes the deep integration of entity extraction and relationship extraction tasks, overcoming the problems of information isolation and insufficient collaboration in traditional task separation methods, while fully meeting the requirements of engineering applications in metal supervision scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 This is a schematic diagram of this application. DETAILED DESCRIPTION

[0090] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0091] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0092] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0093] In the description of this application, it should be noted that the terms "upper," "vertical," "inner," "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the product of the application is typically placed when in use, or are the orientations or positional relationships commonly understood by those skilled in the art. These terms are intended only to facilitate the description of this application and simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting this application. Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0094] It should also be noted that, in the description of this application, unless otherwise expressly specified or limited, the terms "disposed," "installed," and "connected" should be understood broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0095] Example 1

[0096] The method for extracting entity relations of metal supervision in power plants based on dual coding includes the following steps:

[0097] Step S1. Multimodal data preprocessing: collecting and cleaning the data, and labeling the data;

[0098] Step S2. Dual coding joint learning model construction: Design the network layer architecture and perform dual coding joint learning model training;

[0099] Step S3. Dynamic knowledge graph construction and query: Extract the model to generate triples, perform temporal evolution analysis, and then perform causal reasoning interface and dynamic update;

[0100] Step S4. Incremental knowledge update and dynamic model optimization: The incremental learning and feedback module forms a bidirectional closed loop between the knowledge graph and the relationship extraction model, ensuring the continuous evolution of the system through dynamic knowledge update and adaptive model optimization.

[0101] This paper constructs a joint entity-relationship extraction model based on a multi-task dual encoding framework. Through bidirectional feature modeling of dual paths, it achieves collaborative representation optimization between entities and relationships while capturing contextual information. A multi-head attention mechanism layer assigns higher weights to keywords through parallelized attention modules, while dynamically adjusting the focus of entities and relationships to address the complex data types and diverse semantic associations required in engineering scenarios.

[0102] Example 2

[0103] The method for extracting entity relations of metal supervision in power plants based on dual coding includes the following steps:

[0104] Step S1. Multimodal data preprocessing: collecting and cleaning the data, and labeling the data;

[0105] Furthermore, the step S1 includes the following steps:

[0106] Step S1.1. Data collection and cleaning

[0107] The system collects multimodal data from the power plant's technical archives, which is divided into structured data and unstructured data. Structured data mainly includes historical operating data, structural property ledgers, pressure vessel annual inspection records, chemical supervision and pickling records, etc. Unstructured data includes metal anti-wear and explosion-proof inspection report texts, metal inspection report texts, equipment accident abnormality analysis records, etc. (such as defect descriptions in PDF or Word format), and metal X-ray flaw detection images, etc. (JPEG / PNG format); for the above-mentioned collected data, the system performs cleaning operations: for missing values in structured data (such as missing equipment material fields), reverse inference and filling are performed by associating historical maintenance records; for unstructured image data, the resolution pixels are uniformly adjusted and grayscale processing is performed to eliminate noise and improve the compatibility of subsequent analysis.

[0108] Step S1.2. Data annotation

[0109] After data cleaning, we begin annotating multimodal data. For textual data (such as inspection reports), we use the BIO (Begin-Inside-Outside) format to annotate entities such as equipment, defects, and parameters. For example, in the sentence "Boiler pipe A has a creep crack," we annotate "Boiler pipe A" as equipment entity B and "Creep crack" as defect entity I. For X-ray inspection images, we use image labeling tools to annotate the bounding box coordinates of the defect area (such as the crack location and size). The annotation results must ensure standardized alignment between the image and text annotations to provide high-quality annotated data for subsequent model training.

[0110] Step S2. Dual coding joint learning model construction: Design the network layer architecture and perform dual coding joint learning model training;

[0111] Step S2 specifically includes the following steps:

[0112] Step S2.1. Design the network layer architecture

[0113] The dual encoding joint learning model consists of the following four layers of networks, which achieve joint extraction of entities and relations through layered collaboration:

[0114] The first layer is the dynamic word embedding layer, which is responsible for converting the input text sequence (such as the descriptive sentences in the inspection report) into a context-aware dynamic word vector.

[0115] Specifically, the pre-trained BERT model is used to generate the embedding vector of each word The dimension of the representation embedding vector is set to 768. The larger the dimension, the richer the semantic information can be captured. Its characteristic is that it can dynamically adjust the semantic representation according to the context (for example, the vector of "crack" in "creep crack" and "hydrogen-induced crack" is different). During the training process, through the multi-task learning mechanism (entity recognition and relationship classification tasks share word embedding parameters), the word vector's ability to capture semantic associations is enhanced, laying the foundation for subsequent tasks.

[0116] The second layer is the Bidirectional Long Short-Term Memory (BiLSTM) layer. This layer receives the output of the dynamic word embedding layer and simultaneously introduces a forward LSTM network and a backward LSTM network to process the input sequence from both sides, capturing more comprehensive semantic information.

[0117] The processing process is that the forward LSTM processes the input sequence from left to right and generates a forward hidden state sequence {h f1 ,h f2 ,…,h fn}; The reverse LSTM processes the same input sequence from right to left and generates a backward hidden state sequence {h b1 ,h b2 ,…,h bn Through semantic fusion, the hidden state sequence of the bidirectional LSTM with the forward and reverse hidden states is represented as h t =[h ft ;h bt ], h t It also contains the semantic information before and after the current moment.

[0118] The bidirectional LSTM layer adapts to data characteristics, using the time series information of the equipment description in the inspection report as input to the bidirectional LSTM. For example, "Repair of the boiler pipe crack was completed in 2023." The model extracts contextual associations through dual encoding. Dynamic initialization is also used to generate dynamic word embeddings using a pre-trained model, providing high-quality input representations for the bidirectional LSTM. The core of the bidirectional LSTM is to model long-range dependencies, addressing the lack of contextual understanding of traditional unidirectional models, optimizing entity boundary recognition (for example, accurately distinguishing the start and end positions of "boiler pipe A"), and alleviating entity nesting issues (for example, the semantic association between "weld" and "crack" in "cracks in the pipeline weld"), ensuring comprehensive modeling of complex semantics in text sequences.

[0119] The third layer is the multi-head attention mechanism layer, which introduces a parallel attention module to focus on key entities and relationship indicators in the text.

[0120] Specifically, we set up independent attention heads, each of which calculates the query matrix Q, key matrix K, and value matrix V. We dynamically assign weights by scaling dot product attention to improve the model's ability to capture key features. By computing multiple attention matrices in parallel, we can enhance our focus on important tokens in the sentence.

[0121] Among them, the attention mechanism calculates the attention weight by mapping the query matrix Q, the key matrix K and the value matrix V:

[0122]

[0123] where d k is the scaling factor for the matrix dimensions. Through multi-head computation, the attention mechanism can focus on different levels of features in a sentence, such as entities, attributes, and actions in a device description. Ultimately, the outputs of multiple attention heads are concatenated to form a unified feature representation, providing more refined contextual dependency information for entity recognition and relation extraction.

[0124] The fourth layer is the global normalization and joint optimization layer, which solves the problem of inter-task goal conflict by jointly optimizing entity recognition and relationship classification tasks.

[0125] Specifically, the input of the global normalization layer is the hidden state sequence of the bidirectional LSTM obtained from the previous layers, and the score vector v of the entity and relationship categories is generated through the fully connected layer. z :

[0126]

[0127] Where W z is the parameter matrix, h z is the hidden layer feature vector, v z is a category score vector, which represents the category possibility of entities and relations. In the joint optimization process, the model needs to consider the dependencies between entities and relations at the same time. The objective function is expressed as:

[0128]

[0129] in are entity types, For relation types, by maximizing the above joint probability, the model can ensure that the prediction results of entities and relations are semantically consistent.

[0130] In the decoding stage of the model, the conditional random field (CRF) method is used to find the optimal sequence through the Viterbi algorithm. Assuming that the given label sequence y={y1,y2,…,y n}, the model needs to calculate the sequence score:

[0131]

[0132] Where Q(y i ,y i+1 ) represents the transfer score between labels, d(y i ) represents the label score of position i. Finally, the model outputs the optimal label sequence:

[0133]

[0134] Through the collaborative optimization of the global normalization layer, the model can improve the complementarity and robustness between tasks while ensuring the performance of a single task.

[0135] Step S2.2. Model training

[0136] Step S2.2.1. Parameter initialization

[0137] A hierarchical strategy is adopted in the parameter initialization stage of the model: the dynamic word embedding layer loads the weights of the BERT-base pre-trained model and freezes some underlying parameters to accelerate convergence, adjusting only the upper-layer parameters to adapt to the semantic characteristics of the power plant metal supervision domain; the bidirectional long short-term memory network (BiLSTM) layer sets the number of hidden units and the dropout rate, and prevents overfitting by randomly dropping some neurons; each attention head in the multi-head attention layer independently initializes the query, key, and value matrices, and the outputs are concatenated and then reduced in dimensionality through linear transformation to ensure the stability of feature fusion.

[0138] Step S2.2.2. Training strategy

[0139] Multiple optimization strategies are adopted during the training process: a reasonable batch size is set to fill text paragraphs of insufficient length, and the influence of invalid positions is ignored through a masking mechanism; the optimizer uses an adaptive learning rate algorithm, sets the initial learning rate and adds weight decay to control the parameter amplitude; a dynamic adjustment mechanism automatically adjusts the learning rate and task loss weight according to the performance of the validation set, gradually enhancing the model's ability to capture complex relationships; an early stopping mechanism is set to terminate training when the validation loss has not decreased for multiple consecutive rounds to avoid invalid calculations.

[0140] Step S2.2.3. Model validation

[0141] To verify the performance of the model, typical test text (including descriptions of equipment, defects, and causal relationships) was input, and the model output met the expected entity recognition and relationship classification results.

[0142] Step S3. Dynamic knowledge graph construction and query: Extract the model to generate triples, perform temporal evolution analysis, and then perform causal reasoning interface and dynamic update;

[0143] Furthermore, the specific steps are as follows:

[0144] Step S3.1. Triple extraction and storage

[0145] This system uses a joint decoding-based relation extraction model to generate triples. The system consists of two models: a parameter-sharing joint model and a joint decoding model. Unlike traditional vertical models (where entity recognition and relation classification are performed independently), this model achieves simultaneous extraction of entities and relations through end-to-end joint decoding.

[0146] Specifically, the model design transforms entity recognition and relation classification into a sequence labeling problem, and defines special label types to simultaneously represent entity boundaries and relation semantics.

[0147] The model architecture includes the following core mechanisms:

[0148] Step S3.1.1. Establishing a parameter-sharing joint model

[0149] The process of extracting triples using a joint model with parameter sharing is a multi-step process, employing a dual-network architecture: an entity detection network and a relation extraction network. The key difference is that the loss of the joint model based on parameter sharing is the sum of the losses of each sub-process. Gradient calculations and backward parameter updates update the parameters of all sub-processes in the entire model, while the sub-processes in the pipeline model are independent of each other. The joint model using joint decoding is more in line with the "joint" concept, as the subject, object, and relation type are identified and derived in the same step.

[0150] Among them, the entity detection network encodes the text sequence based on the bidirectional recurrent neural network (BiRNN) to generate fused entity features:

[0151]

[0152] Where E(X) is the word embedding matrix, middle represents a set of real numbers, n represents the length of the input sequence (i.e., the number of time steps), and d represents the hidden layer dimension.

[0153] Based on the bidirectional recurrent neural network (Tree-BiRNN) of the dependency tree structure, the input fusion entity feature H ent And the dependency syntax tree node information, output entity pairs (e i ,e j )’s relationship probability:

[0154] P(r|e i ,e j )=Softmax(W r [h i ;h j ;h dep ]+b r )

[0155] where e i ,e j Represents the head entity and tail entity in an entity pair; h i ,h j Represents the head entity e i and tail entity e j The hidden state vectors are generated by the entity detection network and encode the semantic information of the entity in the context; dep Represents the dependency tree path feature, encoding the grammatical relationship between entity pairs through the dependency syntactic analysis tree; W r ,b r are learnable parameters.

[0156] The entity and relation networks share some hidden layer parameters of the word embedding layer, and the loss function is the weighted subtask loss:

[0157] L joint =αL eniity +βL relation

[0158] Among them L entity Cross entropy loss for entity recognition, used to optimize entity boundaries and type classification. relation is the cross entropy loss for relation classification, which is used to optimize the entity-to-relation type. α is the entity recognition task (L entity ) is the weight coefficient that controls the contribution ratio of the task to the total loss. β is the relationship classification task (L relation ) to control the contribution ratio of the task to the total loss.

[0159] Step S3.1.2. Establish a joint decoding model

[0160] The joint decoding-based relation extraction model designs a special label type and defines an extended label set L = {h, r, t}, where h represents the head entity, r represents the relation, and t represents the tail entity. This transforms entity recognition and relation classification into a sequence labeling problem. An end-to-end model is then employed that uses a bidirectional long short-term memory network (BiLSTM) to generate contextual hidden vectors and encode sentences through the encoding layer:

[0161]

[0162] in, represents a real matrix, Represents a set of real numbers, n represents the length of the input sequence (i.e., the number of time steps), and 2d represents the hidden state dimension of the bidirectional long short-term memory network (the concatenation of the forward and backward hidden states). The decoding layer decodes the hidden layer vector input and uses an improved single-item long short-term memory network (LSTM) combined with a label transfer matrix to gradually predict the label sequence:

[0163]

[0164] where y t represents the predicted label of the current time step t; y t-1 represents the historical label of the previous time step t-1; W d is the weight matrix of the linear transformation, transforming the hidden state h t Mapping to label space; h t represents the hidden state vector of the encoder at time step t; b d The bias vector representing the linear transformation; Indicates the label transfer matrix from label y t-1 to y t The transfer score of ,establishes the dependency relationship between labels.

[0165] Then the triples are directly output. The system stores the entity-relationship triples {h, r, t} extracted by the model, where h, r, and t represent the head entity, relationship, and tail entity, respectively (for example, <equipment A, causes, defect B>) into a structured knowledge graph, and defines the core node types (equipment, defect, event, parameter) and their attributes: the equipment node contains fields such as ID, model, and material; the defect node records attributes such as type, location, and size; the relationship types include "cause", "associated parameters", and "repair plan", etc., and a timestamp field is attached to each relationship to form a structured knowledge network for the entire life cycle of the equipment.

[0166] Step S3.2. Perform temporal evolution analysis

[0167] Based on the timestamp field, the system generates knowledge graph snapshots at preset intervals, constructs a multi-dimensional time-series evolution model of the device state, and records dynamic changes in the device state. For example, it can count the frequency changes of a certain type of defect in multiple time slices, calculate degradation trend indicators, analyze the correlation coefficient between parameters and defects, and explain the impact of parameter fluctuations on defects. At the same time, the system generates knowledge graph snapshots by time dimension, supports visual display of the device state evolution path (for example, the causal chain of "material aging → creep cracks → leakage accidents"), and provides a historical version comparison function to facilitate tracing the root cause of degradation.

[0168] Step S3.3. Provide causal reasoning interface and dynamic update

[0169] The system provides a query interface that supports complex causal chain tracing. For example, a query statement can be entered to retrieve all potential factors leading to a defect, and combined with parameter correlation analysis, a root cause hypothesis can be generated. Furthermore, when the model outputs new triples through incremental learning, the system automatically triggers a knowledge graph update process: new nodes and relationships are stored in real time after permission verification, the graph version is updated simultaneously, and operation logs are recorded to ensure the timeliness and security of the knowledge base.

[0170] Step S4. Incremental knowledge update and dynamic model optimization: The incremental learning and feedback module forms a bidirectional closed loop between the knowledge graph and the relationship extraction model, ensuring the continuous evolution of the system through dynamic knowledge update and adaptive model optimization;

[0171] Step S4.1. Incremental knowledge update

[0172] First, the incremental learning part compares the newly extracted triples (h, r, t) (head entity, relationship, tail entity) with the existing knowledge in the knowledge base, and determines whether to add them to the knowledge base through confidence threshold filtering. The confidence of the new knowledge is defined as

[0173] P new =max(P(h,r,t))

[0174] Among them, P(h,r,t) is the model's predicted probability of the triple. When P new When the value is greater than θ, the system stores it in the knowledge base as incremental data. Otherwise, it enters the active learning queue and waits for manual review or further model learning. θ is the set credibility threshold.

[0175] In addition, the system uses entity alignment strategies (such as embedding vector similarity calculation) to detect redundant information and ensure data consistency. If the similarity between the knowledge (h′, r′, t′) in the existing knowledge base and (h, r, t) is

[0176] S=cos(v (h,r,t) ,v (h′,r′,t′) )

[0177] If S>δ is satisfied, it is judged as redundant and not stored in the knowledge base. Otherwise, it is stored as new knowledge. δ is the set similarity threshold.

[0178] Step S4.2. Dynamic model optimization

[0179] On the other hand, the feedback mechanism maintains the adaptability of the relation extraction model through dynamic model optimization. The system regularly evaluates the performance of the model on new data and adopts the knowledge distillation method to optimize the original model P. T (y|x) guides the new model P S (y|x), and optimize its learning process with distillation loss:

[0180]

[0181] Where x is the input text, y i are possible labels (relationship types), P T (y i |x) is the predicted probability of the original model, P S (y i |x) is the predicted probability of the new model. This method ensures that the model does not forget old knowledge when introducing new data. In addition, the system has a user scoring mechanism that allows experts or end users to score the relationships extracted by the model R∈[0,1]. If the score is below the set threshold, it is re-labeled and added to the model training set, ensuring that the knowledge base and model maintain high quality over the long term.

[0182] Ultimately, the incremental learning and feedback module ensures that the knowledge base and the relationship extraction model are updated synchronously, forming a knowledge-driven self-optimization closed loop, continuously improving the accuracy of entity relationship extraction, and enabling the system to have long-term evolution capabilities.

Claims

1. A method for extracting entity relations of metal supervision in power plants based on dual coding, characterized by: The steps include: Step S1. Multimodal data preprocessing: collecting and cleaning the data, and labeling the data; Step S2. Dual coding joint learning model construction: Design the network layer architecture and perform dual coding joint learning model training; Step S3. Dynamic knowledge graph construction and query: Extract the model to generate triples, perform temporal evolution analysis, and then perform causal reasoning interface and dynamic update; Step S4. Incremental knowledge update and dynamic model optimization: The incremental learning and feedback module forms a bidirectional closed loop between the knowledge graph and the relationship extraction model, ensuring the continuous evolution of the system through dynamic knowledge update and adaptive model optimization.

2. The method for extracting metal supervision entity relationships in power plants based on dual coding according to claim 1 is characterized by: The step S1 comprises the following steps: Step S1.

1. Data collection and cleaning; Multimodal data is collected from power plant technical archives and divided into structured and unstructured data. Structured data includes historical operating data, structural characteristics records, pressure vessel annual inspection records, chemical supervision and pickling records. Unstructured data includes metal anti-wear and explosion-proof inspection report texts, metal inspection report texts, equipment accident and abnormality analysis records, and metal X-ray inspection images. Cleaning operations are performed on the collected data: missing values in the structured data are filled by reverse inference through association with historical maintenance records. For unstructured image data, the resolution pixels are uniformly adjusted and grayscale processing is performed. Step S1.

2. Data annotation; After completing data cleaning, we began to annotate the multimodal data. For text data, we used the BIO format to annotate the equipment, defects, and parameters. For X-ray inspection images, we used the image labeling tool to annotate the bounding box coordinates of the defect area. The annotation results must ensure standardized alignment between the image and text annotations.

3. The method for extracting metal supervision entity relationships in power plants based on dual coding according to claim 1 is characterized in that: The step S2 specifically includes the following steps: Step S2.

1. Design the network layer architecture; The dual encoding joint learning model consists of the following four layers of networks, which achieve joint extraction of entities and relations through layered collaboration: The first layer is the dynamic word embedding layer, which is responsible for converting the input text sequence into context-aware dynamic word vectors; The second layer is a bidirectional long short-term memory network layer. The bidirectional long short-term memory network layer receives the output of the dynamic word embedding layer and simultaneously introduces a forward LSTM network and a backward LSTM network to process the input sequence from both sides to capture more comprehensive semantic information. The third layer is the multi-head attention mechanism layer, which introduces a parallel attention module to focus on key entities and relationship indicators in the text; The fourth layer is the global normalization and joint optimization layer, which solves the problem of inter-task conflict by jointly optimizing entity recognition and relationship classification tasks; Step S2.

2. Model training.

4. The method for extracting metal supervision entity relationships in power plants based on dual coding according to claim 3 is characterized in that: Its characteristics are: The first layer uses the pre-trained BERT model to generate the embedding vector for each word The dimension of the embedding vector is set to 768. During the training process, the word embedding parameters are shared between entity recognition and relation classification tasks to enhance the word vector's ability to capture semantic associations.

5. The method for extracting metal supervision entity relationships in power plants based on dual coding according to claim 3 is characterized in that: Its characteristics are: The second layer processing process is that the forward LSTM processes the input sequence from left to right and generates a forward hidden state sequence {h f1 ,h f2 ,…,h fn }; The reverse LSTM processes the same input sequence from right to left and generates a backward hidden state sequence {h b1 ,h b2 ,…,h bn Through semantic fusion, the hidden state sequence of the bidirectional LSTM with the forward and reverse hidden states is represented as h t =[h ft ;h bt ], h t It also contains the semantic information before and after the current moment, h fn is the forward hidden state, h bn is the backward hidden state, h ft Refers to the forward hidden state of any time step t, h bt Generally refers to the backward hidden state at any time step t.

6. The method for extracting entity relationships of metal supervision in power plants based on dual coding according to claim 3 is characterized in that: Its characteristics are: The third layer is implemented as follows: independent attention heads are set up, each of which calculates the query matrix Q, key matrix K, and value matrix V. The weights are dynamically assigned by scaling the dot product attention to improve the model's ability to capture key features. By computing multiple attention matrices in parallel, the focus on important tokens in the sentence is enhanced. Among them, the attention mechanism calculates the attention weight by mapping the query matrix Q, the key matrix K and the value matrix V: where d k is the scaling factor for the matrix dimensions.

7. The method for extracting entity relationships of metal supervision in power plants based on dual coding according to claim 3 is characterized in that: Its characteristics are: The fourth layer generates the score vector v of entity and relationship categories through the fully connected layer z : Where W z is the parameter matrix, h z is the hidden layer feature vector, v z is a category score vector, which represents the category possibility of entities and relations; in the joint optimization process, the objective function is expressed as: in are entity types, is the relationship type; In the decoding stage of the model, the conditional random field method is used to find the optimal sequence through the Viterbi algorithm. Given the label sequence y={y1,y2,…,y n }, the model needs to calculate the sequence score: Where Q(y i ,y i+1 ) represents the transfer score between labels, d(y i ) represents the label score of position i. Finally, the model outputs the optimal label sequence:

8. The method for extracting entity relationships of metal supervision in power plants based on dual coding according to claim 3 is characterized in that: Its characteristics are: Step S2.2 specifically includes: Step S2.2.

1. Parameter initialization; The model's parameter initialization phase uses a hierarchical strategy: the dynamic word embedding layer loads the weights of the BERT-base pre-trained model and freezes some underlying parameters to accelerate convergence, adjusting only the upper-layer parameters to adapt to the semantic characteristics of the power plant metal monitoring domain. The bidirectional long short-term memory network layer sets the number of hidden units and the dropout rate, randomly dropping some neurons to prevent overfitting. Each attention head in the multi-head attention layer independently initializes the query, key, and value matrices, and the concatenated outputs are then linearly transformed for dimensionality reduction to ensure the stability of feature fusion. Step S2.2.

2. Training strategy; Multiple optimization strategies are employed during training: insufficient text paragraphs are padded, and the impact of invalid positions is ignored through a masking mechanism. The optimizer uses an adaptive learning rate algorithm, setting an initial learning rate and adding weight decay to control parameter amplitude. A dynamic adjustment mechanism automatically adjusts the learning rate and task loss weight based on validation set performance, gradually enhancing the model's ability to capture complex relationships. An early stopping mechanism is implemented to terminate training when the validation loss fails to decrease for multiple consecutive rounds, avoiding inefficient computations. Step S2.2.

3. Model validation; To verify the performance of the model, typical test text was input, and the model output met the expected entity recognition and relationship classification results.

9. The method for extracting entity relationships of metal supervision in power plants based on dual coding according to claim 1 is characterized in that: Its characteristics are: Step S3. The specific steps are as follows: Step S3.

1. Triple extraction and storage; A relation extraction model based on joint decoding is used to generate triples, including a parameter-sharing joint model and a joint decoding model, to achieve synchronous extraction of entities and relations through end-to-end joint decoding. In the model design, entity recognition and relationship classification are unified into a sequence labeling problem, and special label types are defined to simultaneously represent entity boundaries and relationship semantics; In step S3.1., the model architecture includes the following core mechanisms: Step S3.1.

1. Establish a parameter sharing joint model; The process of extracting triples using a parameter-sharing joint model is divided into multiple steps and adopts a dual network architecture: an entity detection network and a relation extraction network. Among them, the entity detection network encodes the text sequence based on a bidirectional recurrent neural network to generate fused entity features: Where E(X) is the word embedding matrix, middle represents a set of real numbers, n represents the length of the input sequence, and d represents the hidden layer dimension; Based on the bidirectional recurrent neural network with dependency tree structure, the input fusion entity feature H ent And the dependency syntax tree node information, output entity pairs (e i ,e j )’s relationship probability: P(r|e i ,e j )=Softmax(W r [h i ;h j ;h dep ]+b r ) where e i ,e j Represents the head entity and tail entity in an entity pair; h i ,h j Represents the head entity e i and tail entity e j The hidden state vectors are generated by the entity detection network and encode the semantic information of the entity in the context; dep Represents the dependency tree path feature, encoding the grammatical relationship between entity pairs through the dependency syntactic analysis tree; W r ,b r is a learnable parameter; The entity and relation networks share some hidden layer parameters of the word embedding layer, and the loss function is the weighted subtask loss: L joint =αL entity +βL relation Among them L entity Cross entropy loss for entity recognition, used to optimize entity boundaries and type classification; L relation is the cross entropy loss for relation classification, which is used to optimize the entity-to-relation type; α is the entity recognition task L entity The weight coefficient controls the contribution ratio of the task to the total loss; β is the relationship classification task L relation The weight coefficient controls the contribution ratio of the task to the total loss; Step S3.1.

2. Establish a joint decoding model; A label type is designed for the relation extraction model based on joint decoding. The extended label set L = {h, r, t} is defined, where h represents the head entity, r represents the relation, and t represents the tail entity. Entity recognition and relation classification are transformed into sequence labeling problems. An end-to-end model is then adopted. This model uses a bidirectional long short-term memory network to generate a contextual hidden vector and encodes the sentence through the encoding layer: in, represents a real matrix, Represents a set of real numbers, n represents the length of the input sequence, and 2d represents the hidden state dimension of the bidirectional long short-term memory network. The decoding layer decodes the hidden layer vector input and uses the improved single long short-term memory network combined with the label transfer matrix to gradually predict the label sequence: where y t represents the predicted label of the current time step t; y t-1 represents the historical label of the previous time step t-1; W d is the weight matrix of the linear transformation, transforming the hidden state h t Mapping to label space; h t represents the hidden state vector of the encoder at time step t; b d The bias vector representing the linear transformation; Indicates the label transfer matrix from label y t-1 to y t Transfer score, establish the dependency relationship between labels; The system then directly outputs triples. The entity-relationship triples {h, r, t} extracted by the model, where h, r, and t represent the head entity, relationship, and tail entity, are stored in a structured knowledge graph. Core node types (equipment, defect, event, and parameter) and their attributes are defined: equipment nodes contain ID, model, and material fields; defect nodes record type, location, and size attributes; relationship types include "cause," "associated parameters," and "repair solution." A timestamp field is attached to each relationship, forming a structured knowledge network covering the entire life cycle of the equipment. Step S3.

2. Perform temporal evolution analysis; Based on the timestamp field, knowledge graph snapshots are generated at preset intervals to build a multi-dimensional time-series evolution model of device status and record the dynamic changes of device status. At the same time, knowledge graph snapshots are generated by time dimension, supporting the visualization of the device status evolution path and providing historical version comparison function to facilitate tracing the root cause of degradation. Step S3.

3. Provide causal reasoning interface and dynamic update; It provides a query interface and supports complex causal chain tracing. When the model outputs new triples through incremental learning, the system automatically triggers the knowledge graph update process: the new nodes and relationships are stored in real time after permission verification, the graph version is updated synchronously, and the operation log is recorded to ensure the timeliness and security of the knowledge base.

10. The method for extracting metal supervision entity relationships in power plants based on dual coding according to claim 1 is characterized in that: Its characteristics are: Step S4 specifically includes: Step S4.

1. Incremental knowledge update; First, the incremental learning part compares the newly extracted triples (h, r, t) with the existing knowledge in the knowledge base, and determines whether to add them to the knowledge base through confidence threshold filtering. The confidence of the new knowledge is defined as P new =max(P(h,r,t)) Among them, P(h,r,t) is the model's predicted probability of the triple. When P new When the value is greater than θ, the system stores it in the knowledge base as incremental data. Otherwise, it enters the active learning queue and waits for manual review or further model learning. θ is the set credibility threshold. The system uses entity alignment strategy to detect redundant information and ensure data consistency. If the similarity between the knowledge (h′, r′, t′) in the existing knowledge base and (h, r, t) is S=cos(v (h,r,t) ,in (h′,r′,t′) ) If S>δ, it is considered redundant and not stored in the knowledge base. Otherwise, it is stored as new knowledge. δ is the set similarity threshold. Step S4.

2. Dynamic model optimization; The feedback mechanism maintains the adaptability of the relation extraction model through dynamic model optimization, and adopts the knowledge distillation method to optimize the original model P T (y|x) guides the new model P S (y|x), and optimize its learning process with distillation loss: Where x is the input text, y i is a possible tag, P T (y i |x) is the predicted probability of the original model, P S (y i |x) is the predicted probability of the new model; a user scoring mechanism is designed to allow experts or end users to score the relationships extracted by the model R∈[0,1]. If the score is lower than the set threshold, it is re-labeled and added to the model training set to ensure that the knowledge base and model maintain high quality in the long term; Ultimately, the incremental learning and feedback module ensures that the knowledge base and the relationship extraction model are updated synchronously, forming a knowledge-driven self-optimization closed loop, continuously improving the accuracy of entity relationship extraction, and enabling the system to have long-term evolution capabilities.

Images