Electric carbon market transaction participant relationship extraction method and system based on cascading pointer network

Abstract

The application relates to a kind of electric carbon market transaction participant relationship extraction method and system based on cascade pointer network, comprising the following steps: constructing electric carbon market transaction field segmentation dictionary;Build PowerSpanBERT model;Improved gradient accumulation-projection gradient descent counter-training strategy is introduced to train PowerSpanBERT model;Identify and extract candidate head entity set;Introduce multi-head attention mechanism to calculate the word embedding vector output by PowerSpanBERT model, use CPN to decode the feature vector and word embedding vector of candidate head entity set to obtain candidate tail entity set, predict the relationship type between the head-tail entity pair in candidate head entity set and candidate tail entity set;Output head-tail entity pair and its corresponding relationship type, generate entity relationship triple according to the irrigation result.The application provides a more accurate and efficient entity relationship extraction solution for electric carbon market transaction.

Description

Technical Field

[0001] This invention belongs to the field of relation extraction technology, and relates to a method and system for extracting relations of entities participating in the carbon market, particularly a method and system for extracting relations of entities participating in the carbon market based on a cascaded pointer network. Background Technology

[0002] Against the backdrop of the growing carbon market, it is crucial to study the impact of carbon market expansion on the low-carbon operation of new power systems. This study investigates the identification and relationship extraction techniques of multiple stakeholders and participants in the carbon market trading of new power systems, analyzes the trend of carbon market expansion, and examines the mechanism, key influencing factors, and transmission mechanisms of the comprehensive impact of typical carbon emission reduction policies, such as carbon market expansion, on green electricity, green certificates, and nationally certified voluntary emission reductions. Furthermore, based on the transmission mechanism of key factors, it studies the relationship map of the integration and evolution of the electricity-carbon market across all stages of the new power system. This research not only promotes the low-carbon and stable operation of the new power system but also contributes to the achievement of "dual carbon" goals.

[0003] Knowledge extraction primarily targets open, linked data, using automated techniques to extract usable knowledge units. These units mainly consist of three elements: entities (the extension of a concept), relationships, and attributes. Based on these, a series of high-quality factual representations are formed, laying the foundation for the construction of higher-level schema layers. Relation extraction aims to solve the problem of semantic links between entities. Early relation extraction mainly identified entity relationships by manually constructing semantic rules and templates. Subsequently, relation models between entities gradually replaced manually predefined syntax and rules.

[0004] Currently, the interactions among market participants in the electricity carbon trading market are complex and intertwined. This study investigates techniques for identifying and extracting relationships among multiple stakeholders and stages in the electricity carbon market of new power systems. It analyzes the expansion trend of the carbon market and examines the mechanisms, key influencing factors, and transmission mechanisms of the comprehensive impact of typical carbon emission reduction policies, such as carbon market expansion, on green electricity, green certificates, and nationally certified voluntary emission reductions. However, traditional relationship extraction methods are ill-suited to the rapid changes in the market.

[0005] Traditional pipeline extraction methods separate named entity recognition and relation extraction into two independent tasks, offering advantages in flexibility and independence. However, the lack of effective interaction between these two tasks makes it difficult to fully uncover the implicit relationships between entities, resulting in unsatisfactory relation extraction performance. Joint extraction models based on sequence-to-sequence joint decoding handle entity recognition and relation extraction tasks simultaneously within a unified framework, thereby improving the accuracy of relation extraction. However, they face challenges in extracting from long sequence data and suffer from data redundancy issues. They cannot effectively address the problems of data redundancy and relation overlap. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and propose a method for extracting entity relationships of participants in the electric carbon market based on cascaded pointer networks. By using the PowerSpanBERT-GA-PGD-CPN combined model to jointly decode the entity relationships of participants in the electric carbon market, entity relationship triples are extracted, providing a more accurate and efficient entity relationship extraction solution for electric carbon market transactions.

[0007] The present invention solves its practical problem by adopting the following technical solution:

[0008] A method for extracting the relationships between participants in the electric carbon market based on cascaded pointer networks includes the following steps:

[0009] Construct a word segmentation dictionary for the field of electricity carbon market trading;

[0010] Based on the constructed word segmentation dictionary for the electric carbon market trading domain, a PowerSpanBERT model is built;

[0011] Based on the obtained PowerSpanBERT model, an improved GA-PGD adversarial training strategy is introduced to conduct adversarial training of the PowerSpanBERT model.

[0012] Using a trained PowerSpanBERT model, the input text is converted into word embedding vectors that include contextual information. A cascaded pointer network is then used to decode the word embedding vectors to identify and extract a set of candidate head entities.

[0013] The feature vectors of the candidate head entity set are extracted, and a multi-head attention mechanism is introduced to calculate the word embedding vectors output by the PowerSpanBERT model. The CPN is used to decode the feature vectors of the candidate head entity set and the word embedding vectors to obtain the candidate tail entity set. The relationship type between the head-tail entity pairs in the candidate head entity set and the candidate tail entity set is predicted by a linear layer and a sigmoid activation function.

[0014] The extraction module constructs the PowerSpanBERT-GA-PGD-CPN model, outputs head-to-tail entity pairs and their corresponding relation types, generates entity relation triples based on irrigation results, and completes the joint extraction of entity relations for participants in the electricity carbon market.

[0015] Furthermore, the specific steps for constructing a word segmentation dictionary for the field of electric carbon market trading and determining the word segmentation vocabulary for the field of electric carbon market trading include:

[0016] First, calculate the average information content of a word. The formula for this calculation is as follows:

[0017] Character nThe average information content is estimated using the following formula on a large-scale corpus:

[0018] (1)

[0019] In the formula, p ν For the character n The probability of occurrence, -log p ν It is a word n The amount of information contained therein.

[0020] Based on the average information content of a character, calculate the point mutual information between two adjacent characters;

[0021] (2)

[0022] In the formula, n 1 and n 2 represents two adjacent words, PMI ( n 1, n 2) Represented as a character n 1. n 2 points of mutual information, words n 1. n 2. The probability of appearing alone in a text dataset is used P ( n 1) P ( n 2) indicates that P ( n 1, n 2) for n 1 and n 2. The probability of being adjacent in this corpus.

[0023] According to the words n The formula for calculating the average information content (1) is to calculate the word information content separately. x The left and right entropies are calculated, and the formula for their product is derived as shown in equation (3).

[0024] (3)

[0025] The minimum threshold for point mutual information in word segmentation, PMImin, is set to 0.5, and the minimum threshold for the left-right entropy product, H(ξ), is set to 2, thereby generating a word segmentation dictionary for the field of carbon market trading.

[0026] Furthermore, the specific steps for constructing the PowerSpanBERT model based on the constructed word segmentation dictionary for the electric carbon market trading domain include:

[0027] The SpanBERT module was pre-trained using text data issued in existing electric carbon market transactions.

[0028] Based on the pre-trained SpanBERT module, a domain-specific PowerSpanBERT model is trained using the word segmentation dictionary for the electric carbon market trading domain constructed in step 1.

[0029] Furthermore, based on the obtained PowerSpanBERT model, an improved GA-PGD adversarial training strategy is introduced to perform adversarial training of the PowerSpanBERT model:

[0030] Based on the obtained PowerSpanBERT model, a gradient accumulation strategy is proposed on the basis of PGD adversarial training. In the process of multiple forward and backward propagation, the model parameters are not updated immediately, but these gradients are accumulated, as shown in Equation (6):

[0031] (6)

[0032] exist K After each iteration, the average of the cumulative gradients is calculated, and the model parameters are updated using the average gradient, as shown in formula (7):

[0033] (7)

[0034] The improved GA-PGD adversarial training strategy is shown in Equation (8):

[0035] (8)

[0036] Furthermore, using the trained PowerSpanBERT model, the input text is converted into word embedding vectors that include contextual information. A cascaded pointer network is then used to decode these word embedding vectors, and the specific method for identifying and extracting the candidate head entity set is as follows:

[0037] The PowerSpanBERT model is trained using the GA-PGD adversarial training strategy, which converts the input text data into word embedding vector representations that the model can process.

[0038] The generated word embedding vectors are input into a cascaded pointer network, which scores the position of each word through an attention mechanism and assigns a vector label of 1 or 0 using binary classification labels start and end positions to identify and extract the possible head entity set in the text.

[0039] Furthermore, the feature vectors of the candidate head entity set are extracted, and a multi-head attention mechanism is introduced to calculate the word embedding vectors output by the PowerSpanBERT model. The candidate tail entity set is obtained by decoding the candidate head entity set feature vectors and word embedding vectors using CPN. The specific method for predicting the relationship type between head-tail entity pairs in the candidate head entity set and the candidate tail entity set using a linear layer and a sigmoid activation function is as follows:

[0040] The features of the candidate head entities are extracted, and the vectors that fuse the multi-head attention mechanism output and the candidate head entity features are decoded using CPN to obtain the candidate tail entity set. The word embedding vectors output by the PowerSpanBERT model are passed to the joint recognition layer of relation and tail entities, and the multi-head attention mechanism outputs a vector that integrates the information of each attention head.

[0041] Based on the obtained candidate head entity set, the possible relationship types are determined by using a linear layer and a sigmoid activation function. The relationship type with the highest probability between the head entity and the tail entity is then identified as the relationship type between the head and tail entity pairs.

[0042] Furthermore, following the above steps are the following steps: constructing the PowerSpanBERT-GA-PGD-CPN model, outputting head-tail entity pairs and their corresponding relation types, generating entity relation triples based on irrigation results, and evaluating the model's performance using accuracy, recall, and F1 score metrics.

[0043] Furthermore, the specific method for constructing the PowerSpanBERT-GA-PGD-CPN model, outputting head-to-tail entity pairs and their corresponding relation types, generating entity relation triples based on irrigation results, and evaluating the model's performance using accuracy, recall, and F1 score metrics is as follows:

[0044] The output of the PowerSpanBERT-GA-PGD-CPN model is represented as a triple. The generated triples are manually screened and optimized to remove incorrect matches and redundant information, ensuring the accuracy and reliability of the triples.

[0045] The model's performance is evaluated using metrics such as accuracy, recall, and F1 score. Based on the evaluation results, the model is adjusted and optimized, including adjusting the network structure and hyperparameters.

[0046] A system for extracting the relationships between participants in the electric carbon market based on a cascaded pointer network includes:

[0047] The word segmentation dictionary construction module builds a word segmentation dictionary for the field of electric carbon market trading and determines the word segmentation vocabulary for the field of electric carbon market trading.

[0048] The PowerSpanBERT model building module constructs the PowerSpanBERT model based on the constructed word segmentation dictionary for the electric carbon market trading domain.

[0049] An improved GA-PGD adversarial training strategy module is introduced. Based on the obtained PowerSpanBERT model, an improved GA-PGD adversarial training strategy is introduced to conduct adversarial training of the PowerSpanBERT model.

[0050] The candidate head entity set extraction module uses a trained PowerSpanBERT model to convert the input text into word embedding vectors that include contextual information, and uses a cascaded pointer network to decode the word embedding vectors to identify and extract the candidate head entity set.

[0051] The candidate tail entity set and relationship extraction module extracts the feature vector of the candidate head entity set. At the same time, it introduces a multi-head attention mechanism to calculate the word embedding vector output by the PowerSpanBERT model. The CPN is used to decode the feature vector of the candidate head entity set and the word embedding vector to obtain the candidate tail entity set. The relationship type between the head-tail entity pairs in the candidate head entity set and the candidate tail entity set is predicted by a linear layer and a sigmoid activation function.

[0052] Furthermore, the word segmentation dictionary construction module also includes:

[0053] The module for calculating the average information content of a character first calculates the average information content of the character, using the following formula:

[0054] Character n The average information content is estimated using the following formula on a large-scale corpus:

[0055] (1)

[0056] In the formula, p ν For the character n The probability of occurrence, -log p ν It is a word n The amount of information contained therein.

[0057] The point mutual information calculation module between two adjacent characters calculates the point mutual information between two adjacent characters based on the average information content of the characters.

[0058] (2)

[0059] In the formula, n 1 and n 2 represents two adjacent words, PMI ( n 1, n 2) Represented as a character n 1. n 2 points of mutual information, words n 1. n 2. The probability of appearing alone in a text dataset is used P ( n 1) P ( n 2) indicates that P ( n 1, n 2) For n 1 and n 2. The probability of being adjacent in this corpus.

[0060] The left and right entropy calculation modules for words are based on the word... n The formula for calculating the average information content (1) is to calculate the word information content separately. x The left and right entropies are calculated, and the formula for their product is derived as shown in equation (3).

[0061] (3)

[0062] The word segmentation dictionary generation module for the field of electric carbon market trading sets the minimum threshold of point mutual information PMImin for word formation to 0.5 and the minimum threshold of left and right entropy product H(ξ)min for word formation to 2, thereby generating a word segmentation dictionary for the field of electric carbon market trading.

[0063] Advantages and beneficial effects of the present invention:

[0064] 1. This invention proposes a method for extracting entity relations in the electricity carbon market based on cascaded pointer networks. First, a professional dictionary for the electricity carbon market trading domain is constructed to improve the accuracy of text segmentation and facilitate more accurate extraction of information units during knowledge extraction. Second, a domain-specific PowerSpanBERT model is trained by combining the electricity carbon market trading domain text and the professional dictionary. An adversarial training method with gradient accumulation is introduced during pre-training to improve the accuracy of the model in extracting electricity knowledge and its ability to predict unfamiliar information. Then, a multi-head attention mechanism is introduced in the joint recognition layer of relations and tail entities to enhance the feature representation of the output vector of the encoding layer, realize the joint extraction of entities and relations, and extract multiple relation triples at the same time, effectively addressing the relation overlap problem in the extraction process and significantly improving the accuracy of knowledge extraction.

[0065] 2. This invention proposes a method for extracting entity relationships in the electricity carbon market based on cascaded pointer networks. Addressing the challenge of densely distributed entities and overlapping relationships in Chinese texts used in the electricity carbon trading field, the method transforms the knowledge extraction task into a pointer network annotation problem. It annotates the input electricity text sequence with entities and relationships at the same location, directly modeling triples to jointly extract entities and relationships. This solves the problems of entity redundancy and overlapping relationship extraction, avoids error propagation, and significantly improves the accuracy and efficiency of entity relationship extraction and annotation in the electricity carbon trading field. Attached Figure Description

[0066] Figure 1 This is a structural diagram of the PowerSpanBERT-GA-PGD-CPN model of the present invention;

[0067] Figure 2 This is a schematic diagram of the SpanBERT model masking strategy of the present invention;

[0068] Figure 3 This is a schematic diagram of the PowerSpanBERT layer architecture and input representation of the present invention. Detailed Implementation

[0069] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings:

[0070] A method for extracting the relationships between participants in the electric carbon market based on cascaded pointer networks includes the following steps:

[0071] Step 1: First, construct a word segmentation dictionary for the field of electricity carbon market trading;

[0072] The specific steps of step 1 include:

[0073] (1) First, calculate the average information content of the character. The calculation formula is as follows:

[0074] Character n The average information content is estimated using the following formula on a large-scale corpus:

[0075] (1)

[0076] In the formula, p ν For the character n The probability of occurrence, -log p ν It is a word n The amount of information contained therein.

[0077] (2) Calculate the point mutual information between two adjacent characters based on the average information content of the characters;

[0078] (2)

[0079] In the formula, n 1 and n 2 represents two adjacent words, PMI ( n 1, n 2) Represented as a character n 1. n 2 points of mutual information, words n 1. n 2. The probability of appearing alone in a text dataset is used P ( n 1) P ( n 2) indicates that P ( n 1, n 2) For n 1 and n 2. The probability of being adjacent in this corpus.

[0080] (3) According to the word n The formula for calculating the average information content (1) is to calculate the word information content separately. x The left and right entropies are calculated, and the formula for their product is derived as shown in equation (3).

[0081] (3)

[0082] (3) Based on step (2), the minimum threshold PMImin for word segmentation is set to 0.5, and the minimum threshold H(ξ)min for left and right entropy product is set to 2, thereby generating a word segmentation dictionary for the field of carbon market trading.

[0083] In this embodiment, the working principle of step 1 is as follows:

[0084] Step 1: The field of electric carbon market trading involves many proper nouns and industry terms. In order to help the model accurately identify and understand these proper nouns, avoid misjudging them as other irrelevant words, and improve the accuracy of entity recognition and relation extraction, this invention first constructs a professional dictionary for the field of electric carbon market trading to determine the professional vocabulary in the field of electric carbon market trading.

[0085] To address the specialized terminology in the electricity and carbon market trading field, this paper proposes a minimum entropy word segmentation method based on left and right entropy to construct a specialized dictionary for the electricity and carbon market. The method calculates the pointwise mutual information (PMI) value between adjacent characters to assess their co-occurrence probability. Simultaneously, the product of left and right entropy is introduced as an auxiliary basis for word formation. Fine-grained vocabulary is aggregated, and the contextual adaptability of words is evaluated to achieve more accurate word segmentation, thereby optimizing the word segmentation algorithm for electricity and carbon market trading texts.

[0086] Character n The average information content is estimated using the following formula on a large-scale corpus:

[0087] (1)

[0088] In the formula, p ν For the character n The probability of occurrence, -log p ν It is a word n The amount of information contained therein.

[0089] The point mutual information (PMI) between two adjacent words is a metric based on mutual information (PMI) and is mainly used to assess the strength of the correlation between two things.

[0090] The formula for calculating PMI is as follows:

[0091] (2)

[0092] In the formula, n 1 and n 2 represents two adjacent words, PMI ( n 1, n 2) Represented as a character n 1. n 2 points of mutual information, words n 1. n 2. The probability of appearing alone in a text dataset is used P ( n 1) P ( n 2) indicates that P ( n 1, n 2) For n 1 and n 2. The probability of being adjacent in this corpus.

[0093] Calculate the words according to formula (1) x The left and right entropies are calculated, and the formula for their product is derived as shown in equation (3). The minimum threshold H for the product of left and right entropies when forming a word is set. x ) min When the value is 2, a better vocabulary aggregation effect can be achieved.

[0094] (3)

[0095] Based on the understanding of the above-mentioned characteristics of Chinese word segmentation and the principles of minimum entropy, left and right entropy and mutual information, and considering the current lack of a ready-made word segmentation dictionary for the field of carbon market trading, this invention takes the minimum entropy Chinese word segmentation algorithm based on left and right entropy as the research basis, aiming to build a professional dictionary to improve the accuracy of extraction.

[0096] Step 2: Based on the word segmentation dictionary for the electric carbon market trading domain constructed in Step 1, construct the PowerSpanBERT model;

[0097] The specific steps of step 2 include:

[0098] (1) The SpanBERT module is pre-trained using text data issued in the existing electric carbon market;

[0099] (2) Based on the pre-trained SpanBERT module in step (1), the domain-specific PowerSpanBERT model is trained in combination with the word segmentation dictionary for the electric carbon market trading domain constructed in step 1.

[0100] The working principle of step 2 is as follows:

[0101] Step 2: The professional dictionary constructed above provides specific terms and concepts in the field of electric carbon market trading. When the PowerSpanBERT encoding layer converts the text into word embedding vectors, it can more accurately embed proper nouns in the field, avoiding mis-embedding or incorrect parsing, so as to better capture and understand the complex semantic relationships in electric carbon market trading.

[0102] For terms such as "carbon emission trading" and "electricity market access standards," specialized dictionaries can help PowerSpanBERT better identify the boundaries of these complex entities, ensuring that the complete entity is accurately captured during the encoding process, without information loss or errors due to word segmentation or unclear boundaries.

[0103] SpanBERT is an improved model based on BERT. It introduces a new training objective—SpanBoundary Objective (SBO)—to better capture the internal structure and boundary information of spans (such as phrases and entities). In addition to using the cross-entropy of the predicted mask words as the training objective, it introduces continuous mask words as mask boundary words as auxiliary training objectives. During training, it selects two words that are outside the mask boundary and extracts feature vectors from these two words. These feature vectors are then combined with the positional encoding vectors of the mask words to predict the final word. This allows the model to better learn span-level dependencies and contextual information.

[0104] like Figure 2 As shown, x 5,x 6, x 7, x 8, x 9 is the feature vector corresponding to the mask word, used to predict the vector. x Taking the character corresponding to 5 as an example, the probability that the character is correctly predicted as "carbon" is:

[0105] (4)

[0106] In the formula, L MLM The calculation method is consistent with BERT's MLM calculation method. L SBO This is calculated using a two-layer fully connected network. p 3 indicates the position encoding of the word to be predicted in the mask word.

[0107] The input to the SpanBERT model consists of three parts: word embeddings, paragraph embeddings, and positional embeddings. Given a length of... n -2 tag sequence S = [ s 1, s 2, s 3,…, s n-3 , s n-2 Special markers [CLS] and [SEP] are added to the beginning and end of the sequence, respectively. Each marker in the sequence is then converted into its corresponding word vector. e i The word embedding matrix is ​​obtained. E Assign a position vector to each position in the sequence. p i This yields a position embedding matrix containing all positions. P Paragraph embeddings are used to distinguish between two sentences and are typically used only once in a sequence, denoted as [ SEP Paragraph embedding vector s i [SEG] This indicates the beginning of a paragraph. For each marker in the sequence... s i Its input i i It is composed of word embedding, position embedding and paragraph embedding, as shown in formula (5):

[0108] (5)

[0109] Represent the obtained input I = [ i 1, i 2,…,i n-1 , i n Each label is fed into the Transformer layer of the BERT model. s i To obtain a context-aware vector representation x i : X = [ x 1, x 2,…, x n ].

[0110] The SpanBERT module was pre-trained using regulations, management methods, and policy documents issued in the electricity and carbon market trading, such as the "Interim Regulations on the Administration of Carbon Emission Trading," the "Administrative Measures for Carbon Emission Trading (Trial)," the "Guidelines for Enterprise Greenhouse Gas Emission Accounting and Reporting (Trial)," the "Reference Guidelines for Emission Monitoring Plan Review and Emission Report Verification," and the "Guidelines for the Design and Implementation of Voluntary Greenhouse Gas Emission Reduction Projects." Furthermore, a domain-specific PowerSpanBERT model was trained using the professional dictionary for the electricity and carbon market trading field constructed in step one. The model architecture and input description are as follows: Figure 3 As shown.

[0111] Step 3: Based on the PowerSpanBERT model obtained in Step 2, introduce an improved GA-PGD adversarial training strategy to perform adversarial training on the PowerSpanBERT model.

[0112] The specific method for step 3 is as follows:

[0113] Based on the PowerSpanBERT model obtained in step 2, a gradient accumulation strategy is proposed on the basis of PGD adversarial training. In the process of multiple forward and backward propagation, the model parameters are not updated immediately, but these gradients are accumulated, as shown in formula (6):

[0114] (6)

[0115] exist K After each iteration, the average of the cumulative gradients is calculated, and the model parameters are updated using the average gradient, as shown in formula (7):

[0116] (7)

[0117] The improved GA-PGD adversarial training strategy is shown in Equation (8):

[0118] (8)

[0119] The working principle of step 3 is as follows:

[0120] Step 3: During PowerSpanBERT training, to improve the robustness and generalization ability of the model, an improved adversarial training method is introduced. Small perturbations (adversarial examples) are introduced into the input data to enhance the model's resistance to noise and attacks.

[0121] PGD ​​(Projected Gradient Descent) is a gradient-based adversarial sample generation method that, in each iteration, moves along the loss function L( x,θ Gradient direction perturbation of input samples x Perturbation is generated based on the gradient direction. r Since forward and backward propagation are required for each iteration, the computational cost increases significantly. Therefore, based on PGD adversarial training, a gradient accumulation strategy is proposed. During multiple forward and backward propagations, the model parameters are not updated immediately, but rather these gradients are accumulated, as shown in equation (6):

[0122] (6)

[0123] exist K After each iteration, the average of the cumulative gradients is calculated, and the model parameters are updated using the average gradient, as shown in formula (7):

[0124] (7)

[0125] The improved GA-PGD optimization objective is shown in Equation (8):

[0126] (8)

[0127] The improved GA-PGD adversarial training described above virtually expands the original data by a factor of K without increasing additional computational overhead, thereby improving the utilization of gradients during training. At the same time, after multiple iterations and gradient accumulation, the model can learn from multiple different norm constraints, resulting in smaller generalization error compared to the PGD algorithm with a single norm constraint, thus enhancing the model's generalization ability.

[0128] Step 4: Using the trained PowerSpanBERT model, the input text is converted into word embedding vectors that include contextual information. A cascaded pointer network is used to decode the word embedding vectors and identify and extract the candidate head entity set.

[0129] The specific method for step 4 is as follows:

[0130] (1) Use the GA-PGD adversarial training strategy in step 3 to train the PowerSpanBERT model and convert the input text data into word embedding vector representations that the model can process;

[0131] (2) Input the generated word embedding vector into the cascade pointer network. The cascade pointer network scores the position of each word through an attention mechanism and assigns vector label 1 or label 0 to the start and end positions using binary classification to identify and extract the possible head entity set in the text.

[0132] The working principle of step 4 is as follows:

[0133] Step 4: The PowerSpanBERT model encodes the input raw text, that is, it uses the PowerSpanBERT model to convert the text into word embedding vectors. First, it uses the BertTokenizer component to segment the words and convert the words into IDs. Then, it uses the BertModel component to encode the words, encoding each word into a 768-dimensional word vector representation.

[0134] like Figure 1 As shown, the word embedding vector obtained in the previous step is... H n The input is fed into a Cascade Pointer Network (CPN), which decodes the word vectors to identify all possible head entities and uses binary classification to label the start and end positions.

[0135] First, the first linear layer is used to compute the... i Each word is a score representing the start and end positions of the head entity:

[0136] (9)

[0137] In the formula, For weight parameters, This is the bias parameter.

[0138] The output of the linear layer is then passed through the sigmoid activation function to obtain the probability that each position is the start or end position of the head entity, as shown in formula (10):

[0139] (10)

[0140] In the formula, and These represent the first and second parts of the input sequence, respectively. i The probability that a marker is identified as the start and end position of a head entity.

[0141] The probability is converted into a binary label according to the threshold (set to 0.5), as shown in formula (11). If the probability exceeds the threshold, the corresponding label will be assigned as label 1, otherwise it will be label 0.

[0142] (11)

[0143] Then, using the nearest matching principle, the identified start and end are paired to obtain a set of candidate head entities, each candidate head entity is denoted as ( i,j ).

[0144] Step 5: Extract the feature vector of the candidate head entity set, and at the same time, introduce a multi-head attention mechanism to calculate the word embedding vector output by the PowerSpanBERT model. Use CPN to decode the feature vector of the candidate head entity set and the word embedding vector to obtain the candidate tail entity set. Predict the relationship type between head-tail entity pairs in the candidate head entity set and the candidate tail entity set through a linear layer and a sigmoid activation function.

[0145] The specific method for step 5 is as follows:

[0146] (1) Extract the feature vector of the candidate head entity set obtained in step 4;

[0147] (2) Based on (1), a multi-head attention mechanism is introduced to calculate the word embedding vector output by the PowerSpanBERT model in step 4. The decoding method is the same as in step 4. CPN is used to decode the feature vector of the candidate head entity set and the word embedding vector to obtain the candidate tail entity set.

[0148] (3) Based on the candidate head entity set obtained in step 4, the possible relationship types are determined by using a linear layer and a sigmoid activation function. The relationship type with the highest probability between the head entity and the tail entity in (2) can be identified as the relationship type between the head-tail entity pair.

[0149] The working principle of step 5 is as follows:

[0150] The head entity identified by the head entity decoding layer is passed to the joint relation and tail entity recognition layer, along with the relevant text embedding vector and positional information. The input to the joint relation and tail entity recognition layer includes the output of the PowerSpanBERT encoding layer. H n In addition, the characteristics of the head entities identified in the head entity decoding layer were also considered.

[0151] A multi-headed attention mechanism is introduced in the joint recognition layer of relation and tail entity. This mechanism is used to capture complex dependencies in the text, enhances the feature representation of the output vector of the encoding layer, and improves the accuracy of the semantic vector describing the meaning of the entity.

[0152] For the input vector H n Perform a linear transformation to obtain its query vector. Q n Key vector K n Sum value vector V n For each head, calculate the dot product of the query vector and the key vector. To prevent the dot product from becoming too large and causing gradient vanishing, use the dimension of the key vector. d k The attention required to obtain the scaled dot product after scaling is as follows:

[0153] (12)

[0154] Each head independently computes attention, then the outputs of all heads are concatenated together, and the final output is generated through a linear transformation:

[0155]

[0156] In the formula, W sub Let be the weight matrix for the linear transformation.

[0157] Calculate the correlation score and scale it:

[0158] (13)

[0159] Next, the scores are normalized so that the association score of each entity pair is converted into a probability distribution, and the result is obtained by using the softmax function. i Attention weights of individual entities α i Then, the attention weights obtained through normalization are used. α i With the i The value vectors of each entity are summed using weighted methods to obtain the weighted global vector. Sum i As shown in formula (14).

[0160] (14)

[0161] In the formula, V iFor the first i A vector of values ​​for each entity.

[0162] The input to the joint relationship and tail entity recognition layer, in addition to the output of the encoding layer, is... H n In addition, the characteristics of the head entity were also considered. vsub j (referring to the first) j (The average of the vectors of all tags contained in each candidate head entity)

[0163] (15)

[0164] In the formula, and These represent the first and second parts of the input sequence, respectively. i The probability that a marker is identified as the start and end position of a tail entity.

[0165] Similarly, the probabilities of the start and end positions of the tail entities are converted into binary labels based on a threshold. The start and end of the identified tail entities are then paired using the nearest matching principle to obtain a set of candidate tail entities. Each candidate tail entity is denoted as ( m,n ).

[0166] For each candidate head entity ( i , j The relationship between the head entity and each possible tail entity is determined using a linear layer and a sigmoid activation function. The probability of the relationship between each candidate head entity and tail entity is shown in formula (16).

[0167] (16)

[0168] In the formula, [ Sum i ; Sum j ] represents the sequence representation of the head entity. W rel For weight parameters, b rel This is the bias parameter.

[0169] For each candidate head entity ( i , j ) and candidate tail entities ( m , n ), the probability of using relational classification p rel k To determine the type of relationship between them:

[0170] (17)

[0171] Step 6: Construct the PowerSpanBERT-GA-PGD-CPN model, output the identification results of the head entity in Step 4 and the tail entity and relationship in Step 5, generate entity relationship triples based on the irrigation results, and evaluate the performance of the model through accuracy, recall and F1 score.

[0172] The specific method for step 6 is as follows:

[0173] (1) The output of the PowerSpanBERT-GA-PGD-CPN model is represented as a triple. The generated triples are manually screened and optimized to remove incorrect matches and redundant information, so as to ensure the accuracy and reliability of the triples.

[0174] (2) The performance of the model is evaluated by the accuracy, recall and F1 score indicators. The model is adjusted and optimized according to the evaluation results, including adjusting the network structure and hyperparameters, in order to improve the overall performance of the model.

[0175] The working principle of step 6 is as follows:

[0176] Precision, recall, and F1 score are used as evaluation criteria for entity relation identification results. Precision (Prec) represents the proportion of correctly identified entity relations to the total number of correctly identified entity relations, and its calculation formula is as follows:

[0177] (18)

[0178] Recall (Rec) represents the proportion of correctly identified entity relationships out of the total number that should theoretically have been identified.

[0179] (19)

[0180] Furthermore, the F1 score is used to comprehensively evaluate precision and recall, and the F1 score can be used to evaluate the model's recognition performance. The formula for calculating the F1 score is:

[0181] (20)

[0182] N A This represents the total number of entities in the labeled dataset, i.e., the actual standard answer given by the labeled dataset; N B This represents the total number of entities identified by the model. N C This represents the total number of entities correctly identified by the model, that is, the number of entities identified by the model that match the entities in the actual labeled dataset.

[0183] A system for extracting the relationships between participants in the electric carbon market based on a cascaded pointer network includes:

[0184] A word segmentation dictionary construction module is used to build a word segmentation dictionary for the field of electricity carbon market trading.

[0185] The PowerSpanBERT model building module constructs the PowerSpanBERT model based on the word segmentation dictionary built for the electric carbon market trading domain.

[0186] An improved GA-PGD adversarial training strategy is introduced into the module. Based on the obtained PowerSpanBERT model, an improved GA-PGD adversarial training strategy is introduced to perform PowerSpanBERT model pre-training.

[0187] The candidate head entity set extraction module uses a trained PowerSpanBERT model to convert the input text into word embedding vectors that include contextual information. Then, a cascaded pointer network is used to decode the word embedding vectors to identify and extract the candidate head entity set.

[0188] The candidate tail entity set and relationship extraction module extracts the feature vector of the candidate head entity set. At the same time, it introduces a multi-head attention mechanism to calculate the word embedding vector output by the PowerSpanBERT model. The CPN is used to decode the feature vector of the candidate head entity set and the word embedding vector to obtain the candidate tail entity set. The linear layer and sigmoid activation function are used to predict the relationship type between the head-tail entity pairs in the candidate head entity set and the candidate tail entity set.

[0189] The extraction module constructs the PowerSpanBERT-GA-PGD-CPN model, outputs head-to-tail entity pairs and their corresponding relation types, generates entity relation triples based on irrigation results, and completes the joint extraction of entity relations for participants in the electricity carbon market.

[0190] The word segmentation dictionary construction module also includes:

[0191] The module for calculating the average information content of a character first calculates the average information content of the character, using the following formula:

[0192] Character n The average information content is estimated using the following formula on a large-scale corpus:

[0193] (1)

[0194] In the formula, p ν For the character n The probability of occurrence, -log p νIt is a word n The amount of information contained therein.

[0195] The point mutual information calculation module between two adjacent characters calculates the point mutual information between two adjacent characters based on the average information content of the characters.

[0196] (2)

[0197] In the formula, n 1 and n 2 represents two adjacent words, PMI ( n 1, n 2) Represented as a character n 1. n 2 points of mutual information, words n 1. n 2. The probability of appearing alone in a text dataset is used P ( n 1) P ( n 2) indicates that P ( n 1, n 2) For n 1 and n 2. The probability of being adjacent in this corpus.

[0198] The left and right entropy calculation modules for words are based on the word... n The formula for calculating the average information content (1) is to calculate the word information content separately. x The left and right entropies are calculated, and the formula for their product is derived as shown in equation (3).

[0199] (3)

[0200] The word segmentation dictionary generation module for the field of electric carbon market trading sets the minimum threshold of point mutual information PMImin for word formation to 0.5 and the minimum threshold of left and right entropy product H(ξ)min for word formation to 2, thereby generating a word segmentation dictionary for the field of electric carbon market trading.

[0201] It should be emphasized that the embodiments described in this invention are illustrative rather than limiting. Therefore, this invention includes, but is not limited to, the embodiments described in the specific implementation. Any other implementations derived by those skilled in the art based on the technical solutions of this invention are also within the scope of protection of this invention.

Claims

1. A method for extracting the entity relationships of participants in the electric carbon market based on cascaded pointer networks, characterized in that: Includes the following steps: Construct a word segmentation dictionary for the field of electricity carbon market trading; Based on the constructed word segmentation dictionary for the electric carbon market trading domain, a PowerSpanBERT model is built; The SpanBERT module was pre-trained using text data issued in existing electric carbon market transactions. Based on the pre-trained SpanBERT module, a domain-specific PowerSpanBERT model is trained using a constructed word segmentation dictionary for the electric carbon market trading domain. Based on the obtained PowerSpanBERT model, an improved gradient accumulation-projection gradient descent adversarial training strategy, GA-PGD, is introduced to conduct adversarial training of the PowerSpanBERT model. Using a trained PowerSpanBERT model, the input text is converted into word embedding vectors that include contextual information. A cascaded pointer network is then used to decode the word embedding vectors to identify and extract a set of candidate head entities. Next, extract the feature vector of the candidate head entity set, and introduce a multi-head attention mechanism to calculate the word embedding vector output by the PowerSpanBERT model. Use a cascaded pointer network (CPN) to decode the feature vector and word embedding vector of the candidate head entity set to obtain the candidate tail entity set. Predict the relationship type between head-tail entity pairs in the candidate head entity set and the candidate tail entity set through a linear layer and a sigmoid activation function. Construct the PowerSpanBERT-GA-PGD-CPN model, output the head-tail entity pairs and their corresponding relation types, generate entity relation triples based on irrigation results, and complete the joint extraction of entity relations for participants in the electricity carbon market. The specific method for adversarial training of the PowerSpanBERT model based on the obtained PowerSpanBERT model, by introducing an improved GA-PGD adversarial training strategy, is as follows: Based on the obtained PowerSpanBERT model, a gradient accumulation strategy is proposed on the basis of PGD adversarial training. In the process of multiple forward and backward propagation, the model parameters are not updated immediately, but these gradients are accumulated, as shown in Equation (6): （6） exist K After each iteration, the average of the cumulative gradients is calculated, and the model parameters are updated using the average gradient, as shown in formula (7): （7） The improved GA-PGD adversarial training strategy is shown in Equation (8): （8）； An improved GA-PGD adversarial training strategy is adopted to train the PowerSpanBERT model. Using the trained PowerSpanBERT model, the input text is converted into word embedding vectors that include contextual information. A cascaded pointer network is then used to decode these word embedding vectors to identify and extract the candidate head entity set. The specific method is as follows: The PowerSpanBERT model is trained using the GA-PGD adversarial training strategy, which converts the input text data into word embedding vector representations that the model can process. The generated word embedding vectors are input into a cascaded pointer network, which scores the position of each word through an attention mechanism and assigns the vector label 1 or label 0 to the start and end positions using binary classification to identify and extract the possible head entity set in the text. The feature vectors of the candidate head entity set are extracted, and a multi-head attention mechanism is introduced to calculate the word embedding vectors output by the PowerSpanBERT model. CPN is used to decode the candidate head entity set feature vectors and word embedding vectors to obtain the candidate tail entity set. The specific method for predicting the relationship type between head-tail entity pairs in the candidate head entity set and the candidate tail entity set using a linear layer and a sigmoid activation function is as follows: The features of the candidate head entities are extracted, and the vectors that fuse the multi-head attention mechanism output and the candidate head entity features are decoded using CPN to obtain the candidate tail entity set. The word embedding vectors output by the PowerSpanBERT model are passed to the joint recognition layer of relation and tail entities, and the multi-head attention mechanism outputs a vector that integrates the information of each attention head. Based on the obtained candidate head entity set, the possible relationship types are determined by using a linear layer and a sigmoid activation function. The relationship type with the highest probability between the head entity and the tail entity can be identified as the relationship type between the head-tail entity pair. Following the above steps, the following steps are also included: constructing the PowerSpanBERT-GA-PGD-CPN model, outputting the identification results of head entities, tail entities, and relationships, generating entity relationship triples based on irrigation results, and evaluating the model's performance using accuracy, recall, and F1 score metrics. The specific method for constructing the PowerSpanBERT-GA-PGD-CPN model, outputting head-tail entity pairs and their corresponding relation types, generating entity relation triples based on irrigation results, and evaluating the model's performance using accuracy, recall, and F1 score metrics is as follows: (1) The output of the PowerSpanBERT-GA-PGD-CPN model is represented as a triple. The generated triples are manually screened and optimized to remove incorrect matches and redundant information, so as to ensure the accuracy and reliability of the triples. (2) The performance of the model is evaluated by the accuracy, recall and F1 score indicators. The model is adjusted and optimized according to the evaluation results, including adjusting the network structure and hyperparameters.

2. The method for extracting entity relationships in the electric carbon market based on cascaded pointer networks according to claim 1, characterized in that: The specific steps for constructing a word segmentation dictionary for the field of electricity carbon market trading include: First, calculate the average information content of a word. The formula for this calculation is as follows: Character ν The average information content is estimated using the following formula on a large-scale corpus: （1） In the formula, p ν For the character ν The probability of occurrence, -log p ν It is a word ν The amount of information contained; Based on the average information content of a character, calculate the point mutual information between two adjacent characters; （2） In the formula, ν 1 and ν 2 represents two adjacent words, PMI ( ν 1, ν 2) Represented as a character ν 1. ν 2 points of mutual information, words ν 1. ν 2. The probability of appearing alone in a text dataset is used P ( ν 1) P ( ν 2) indicates that P ( ν 1, ν 2) for ν 1 and ν 2. The probability of being adjacent in this corpus; According to the words ν The formula for calculating the average information content (1) is to calculate the word information content separately. ξ The left and right entropies are calculated, and the formula for their product is derived as shown in equation (3). （3） The minimum threshold for point mutual information in word segmentation, PMImin, is set to 0.5, and the minimum threshold for the left-right entropy product, H(ξ), is set to 2, thereby generating a word segmentation dictionary for the field of carbon market trading.

3. The extraction system of the method for extracting the entity relationship of participants in the electric carbon market based on a cascaded pointer network as described in claim 1 or 2, characterized in that: include: A word segmentation dictionary construction module is used to build a word segmentation dictionary for the field of electricity carbon market trading. The PowerSpanBERT model building module constructs the PowerSpanBERT model based on the constructed word segmentation dictionary for the electric carbon market trading domain. An improved GA-PGD adversarial training strategy is introduced into the module. Based on the obtained PowerSpanBERT model, an improved GA-PGD adversarial training strategy is introduced to perform PowerSpanBERT model pre-training. The candidate head entity set extraction module uses a trained PowerSpanBERT model to convert the input text into word embedding vectors that include contextual information. Then, a cascaded pointer network is used to decode the word embedding vectors to identify and extract the candidate head entity set. The candidate tail entity set and relationship extraction module extracts the feature vector of the candidate head entity set. At the same time, it introduces a multi-head attention mechanism to calculate the word embedding vector output by the PowerSpanBERT model. The CPN is used to decode the feature vector of the candidate head entity set and the word embedding vector to obtain the candidate tail entity set. The linear layer and sigmoid activation function are used to predict the relationship type between the head-tail entity pairs in the candidate head entity set and the candidate tail entity set. The extraction module constructs the PowerSpanBERT-GA-PGD-CPN model, outputs head-to-tail entity pairs and their corresponding relation types, generates entity relation triples based on irrigation results, and completes the joint extraction of entity relations for participants in the electricity carbon market.

4. The system for extracting the entity relationship of participants in the electric carbon market based on a cascaded pointer network according to claim 3, characterized in that: The word segmentation dictionary construction module also includes: The module for calculating the average information content of a character first calculates the average information content of the character, using the following formula: Character ν The average information content is estimated using the following formula on a large-scale corpus: （1） In the formula, p ν For the character ν The probability of occurrence, -log p ν It is a word ν The amount of information contained; The point mutual information calculation module between two adjacent characters calculates the point mutual information between two adjacent characters based on the average information content of the characters. （2） In the formula, ν 1 and ν 2 represents two adjacent words, PMI ( ν 1, ν 2) Represented as a character ν 1. ν 2 points of mutual information, words ν 1. ν 2. The probability of appearing alone in a text dataset is used P ( ν 1) P ( ν 2) indicates that P ( ν 1, ν 2) for ν 1 and ν 2. The probability of being adjacent in this corpus; The left and right entropy calculation modules for words are based on the word... ν The formula for calculating the average information content (1) is to calculate the word information content separately. ξ The left and right entropies are calculated, and the formula for their product is derived as shown in equation (3). （3） The word segmentation dictionary generation module for the field of electric carbon market trading sets the minimum threshold of point mutual information PMImin for word formation to 0.5 and the minimum threshold of left and right entropy product H(ξ)min for word formation to 2, thereby generating a word segmentation dictionary for the field of electric carbon market trading.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-2.

Images