A relation extraction method for knowledge graph construction of low-label industrial corpora

Abstract

A relation extraction method for knowledge graph construction using low-labeled industrial corpora is proposed. First, dynamic prompt templates are constructed by combining contextual information from the industrial text data, and a semantic collaboration mechanism is introduced to enhance the semantic association between template tags. Second, a feature alignment mechanism and phased learning are combined to enable the model to gradually adapt to multi-task settings with increasing difficulty and to enhance the model's ability to discriminate semantic differences. Finally, the matching probability between relation representations and label representations is calculated, and the model is optimized using cross-entropy loss to predict the most likely relation type. This invention fully utilizes the semantics of the industrial text data context, combined with a phased learning strategy, enabling the model to gradually adapt to the complexity of relation extraction tasks, requiring low-scale labeled data, and achieving high relation extraction accuracy.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, and in particular relates to a relation extraction method for constructing knowledge graphs from low-labeled industrial corpora. Background Technology

[0002] With the accelerating digital transformation in industrial manufacturing, equipment operation and maintenance, and production management, industrial knowledge graphs, as a crucial technology for achieving semantic management and intelligent decision-making of industrial data, are gradually being applied to key scenarios such as equipment fault diagnosis, process optimization, and knowledge reasoning. Relation extraction, as a core component of knowledge graph construction, is responsible for identifying semantic relationships between entities from unstructured industrial text, and is of great significance for improving the completeness and usability of knowledge graphs.

[0003] However, text data in the industrial domain often features highly specialized terminology, complex language styles, and strong contextual dependencies, posing significant challenges to relation extraction tasks. Especially in practical applications, high-quality manually labeled data is extremely limited. The high cost of labeling and the high level of professional knowledge required result in the widespread existence of "low-labeled corpora." This data scarcity severely restricts the performance and generalization ability of traditional supervised learning methods in industrial scenarios.

[0004] Existing relation extraction methods mostly rely on large-scale, high-quality labeled data, making them difficult to directly transfer to low-resource industrial corpora. Furthermore, industrial texts contain numerous long-tail, complex, and implicit relations, further increasing the learning difficulty of the models. Rule-based methods exhibit poor generalization ability when faced with complex sentence structures, while deep learning-based methods are prone to overfitting and lack robustness in small-sample scenarios. Therefore, there is an urgent need for a relation extraction method that can still possess good generalization and semantic understanding capabilities even with low-labeled corpora, in order to support the construction and practical application of high-quality industrial knowledge graphs. Summary of the Invention

[0005] To overcome the challenges of scarce labeled data and complex relational semantics in industrial knowledge graph construction, this invention proposes a relation extraction method for low-label industrial corpora. This method obtains industrial entity representations through dynamically optimizable prompt templates. In course learning tasks, a training strategy progressing from easy to difficult is adopted to gradually improve the model's ability to discriminate complex relations. This achieves efficient and robust relation extraction in low-resource scenarios, supporting the construction of industrial knowledge graphs.

[0006] The technical solution adopted by this invention to solve the technical problem is:

[0007] A relation extraction method for knowledge graph construction of low-labeled industrial corpora is proposed. First, dynamic prompt templates are constructed by combining contextual information from industrial text data, and a semantic collaboration mechanism is introduced to enhance the semantic association between template tags. Second, by combining feature alignment mechanism and phased course learning, the model can gradually adapt to multi-task settings with increasing difficulty and enhance the model's ability to discriminate semantic differences. Finally, the matching probability between relation representation and label representation is calculated, and the model is optimized through cross-entropy loss to predict the most likely relation type.

[0008] Furthermore, the method includes the following steps:

[0009] Step 1: Construct an entity annotation set E by combining industry-specific terminology. Utilize existing industry terminology dictionaries to build entity recognition templates. Extract the corresponding head and tail entities from the text through precise matching. Use trainable continuous identifiers to replace fixed words in traditional prompts to construct dynamic prompts T1 = [t1][t2]...[t] m [sub][MASK][obj], where [sub] and [obj] represent identified industrial entities, [t1][t2]...[t m [MASK] ∈ V, where V represents the pre-trained model vocabulary, m is the number of trainable tags, and [MASK] represents the complex semantic relationships that industrial entities may have in the context of industrial text data;

[0010] Step 2: Use the Word2Vec model to train word vectors for industrial entities, obtain words that are semantically similar to the target entity, and embed the first k related words into their embedding vectors e. h and head entity embedding e sub Tail entity embedded e obj The trainable labels t in the mean initialization template T1 j Embedded representation

[0011]

[0012] Step 3: Convert the original relation label set Y = {y1, y2, ..., y...} λ The relations in} are removed and interpreted into natural language form, resulting in a relation description set Y' = {y1', y2', ..., y λ '}, together with the manually constructed hint T2 = means[MASK], form a new relation label sequence [CLS]y w '[SEP] means [MASK], w∈λ, embedding the mask positions as answer words to construct a continuous answer space Q={q1,q2,...,q λ},

[0013]

[0014] Step 4: Combine the dynamic prompt T1 with the industrial domain text data X i =(x1,x2,x3,...,x n ), where i∈N are merged, and N is the number of samples in each batch. The merged sequence is then encoded to obtain the corresponding word embedding representation. When input into a pre-trained model, the output embedding F(χ) of the masked word can be obtained at the mask position. i ,y) and the output vector G(χ) before classification i ,y),

[0015] F(χ i ,y)=PLM(χ i );

[0016] G(χ i ,y)=W(F(χ) i ,y))+b;

[0017] Where W∈R |V|×d It is the output projection matrix of the model, b∈R |V| |v| represents the size of the vocabulary;

[0018] Step 5: Input sequence X i After randomly covering a marker, the masked sequence X is obtained. i ', Replace the answer word corresponding to the real label at the mask position of the prompt template T1, and construct a new input sequence χ. i ', using a pre-trained language model to predict masked token words, and employing cross-entropy loss L f Maximize the conditional probability p(word|χ′) i ,y),

[0019]

[0020] Where M is the set of all occluded locations, and BCE() represents the binary cross-entropy loss function.

[0021] The above steps 1 to 5 are a specific implementation process for constructing dynamic prompt templates by combining the contextual information of industrial text data and introducing a semantic collaboration mechanism to enhance the semantic association between template tags.

[0022] Furthermore, the relation extraction method also includes the following steps:

[0023] Step 6: Divide the training samples into two formats: masked format and unmasked format. In the masked format, use the entity pairs [sub] and [obj] as input to the model, and combine them with the cue T1 to form a new input sequence χ.entity =[CLS][sub][obj][SEP].T1[SEP], which uses the pre-classification output vector of the mask position as the entity bias G(χ). entity The input sequence χ consists of blank input and prompt T1. prompt =[CLS][SEP].T1[SEP], obtain the pre-classification output vector at the mask position as the cue bias G(χ). prompt The bias is subtracted from the original input pre-classification output vector (y), .

[0024] x out =G(χ) i ,y)-G(χ entity ,y)-G(χ prompt ,y);

[0025] The fine-tuning process uses cross-entropy loss L mlm optimization,

[0026]

[0027] p(y|x)=w*ReLU(x out )+c;

[0028] Where, w∈R |V|×d It is the output projection matrix of the model, c∈R |V| For bias variables;

[0029] Step 7: Inject relational knowledge into the answer space Q = {r1, r2, ..., r...} n Select the current input X in} i Real Relationship Tag y i The corresponding answer word is r. i As a positive sample, randomly select except r i Any answer word outside of r is used as a negative sample. i By calculating the cosine similarity, the model prediction result F(χ) is... i (y) is closer to the true answer words and further away from irrelevant answer words.

[0030] L rel =max{cos(F(χ i ,y),r i )-cos(F(χ i ,y),r i ')+γ,0};

[0031] Step 8: In the unmasked format, replace the [MASK] markers in the original prompt template with relational answer words in the continuous space. Concatenate the output vectors corresponding to the positions of the two starting markers to construct the relational representation e.rel The data is then fed into a classifier to output a probability distribution over the label set Y. The fine-tuning process utilizes cross-entropy loss L... cls Optimize,

[0032] e rel =[e [CLS] ;e [SEP] ];

[0033] p i =softmax(W*e rel +b);

[0034] L cls =-logp i .

[0035] The processing steps 6 to 8 above are a specific implementation process that combines feature alignment and phased course learning strategies to enable the model to gradually adapt to the difficulty of the task and improve its ability to distinguish semantic differences.

[0036] Furthermore, the relation extraction method also includes the following step: Step 9, Calculate the overall loss L,

[0037] L=α*(L mlm +L rel )+(1-α)*L cls +β*L f ;

[0038] When the input is in mask format, α is 1, otherwise it is 0, and β is a hyperparameter;

[0039] Step 10: In the early stage of training, a lower proportion of masked format samples are used and a higher proportion of unmasked format data is retained. As the training process progresses, the proportion of masked samples is gradually increased, and finally transitions to the full masked format.

[0040] Step 11: Repeat steps 4 to 10. When L is less than the specified minimum loss value, end the calculation and take the relationship corresponding to the position index with the highest probability in the prediction result as the final result of the industrial entity relationship in the industrial domain text.

[0041] The processing steps 9 to 11 above are a specific implementation process of dynamically adjusting model parameters through joint loss and using the converged model to infer the relationships between entities. The technical concept of this invention is as follows: It constructs a dynamic cue learning set of knowledge features from industrial domain annotations such as equipment, production processes, and quality control; it enhances the semantic association between templates using semantic coherence constraints; it enhances the model's adaptability to multi-task settings through phased course learning; and it enhances the model's perception of relational semantics through a feature alignment mechanism.

[0042] The beneficial effects of this invention are as follows: it can integrate multiple types of labeled knowledge in the industrial field and construct dynamic prompt templates with enhanced semantic association, effectively adapting to multi-task relationship extraction scenarios; it overcomes the problems of insufficient understanding of domain semantics and insensitivity to changes in task difficulty in traditional methods, improves the stability and generalization ability of relationship recognition in low-labeling environments, and thus significantly enhances the quality and efficiency of industrial knowledge graph construction. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating a relation extraction method for constructing knowledge graphs from low-labeled industrial corpora. Detailed Implementation

[0044] The invention will be further described below with reference to the accompanying drawings.

[0045] Reference Figure 1 A relation extraction method for knowledge graph construction using low-labeled industrial corpora includes the following steps:

[0046] Step 1: Construct an entity annotation set E by combining industry-specific terminology. Utilize existing industry terminology dictionaries to build entity recognition templates. Extract the corresponding head and tail entities from the text through precise matching. Use trainable continuous identifiers to replace fixed words in traditional prompts to construct dynamic prompts T1 = [t1][t2]...[t] m [sub][MASK][obj], where [sub] and [obj] represent identified industrial entities, [t1][t2]...[t m [MASK] ∈ V, where V represents the pre-trained model vocabulary, m is the number of trainable tags, and [MASK] represents the complex semantic relationships that industrial entities may have in the context of industrial text data;

[0047] In this embodiment, the text data "Car Fault Report No. 363: Fault Phenomenon: Steering gear makes abnormal noise when the vehicle is driving on a bumpy road" in the automotive repair field is used to construct an entity annotation set E containing automotive-related terms such as "engine" and "accelerator" using GPT or the car manual. The entities "steering gear" and "abnormal noise" in the text data are identified through rule matching, and dynamic prompts [t1][t2]...[t] are constructed. m Steering gear [MASK] unusual noise;

[0048] Step 2: Use the Word2Vec model to train word vectors for industrial entities, obtain words that are semantically similar to the target entity, and embed the first k related words into their embedding vectors e. h and head entity embedding e sub Tail entity embedded e obj The trainable labels t in the mean initialization template T1j Embedded representation

[0049]

[0050] In this embodiment, a large number of documents related to industrial equipment, such as maintenance manuals, operating procedures, and technical reports, are collected and trained to obtain a Word2Vec model capable of learning word vector representations for each industrial term. Word2Vec is used to obtain k words similar to "steering gear" and "abnormal noise," and the average embedding of these similar words is used to initialize the tags in the differentiable cue template [t]. i ];

[0051] Step 3: Convert the original relation label set Y = {y1, y2, ..., y...} λ The relations in} are removed and interpreted into natural language form, resulting in a relation description set Y' = {y1', y2', ..., y λ '}, together with the manually constructed hint T2 = means[MASK], form a new relation label sequence [CLS]y w '[SEP] means [MASK], w∈λ, embedding the mask positions as answer words to construct a continuous answer space Q={q1,q2,...,q λ},

[0052]

[0053] In this embodiment, the original relation label "component failure" can be briefly expanded into a natural language expression of "the component that has failed". Then, it is combined with the manually constructed prompt template T2 and input into the pre-trained model. The embedded mask position is used to initialize the answer word corresponding to the relation label.

[0054] Step 4: Combine the dynamic prompt T1 with the industrial domain text data X i =(x1,x2,x3,...,x n ), where i∈N are merged, and N is the number of samples in each batch. The merged sequence is then encoded to obtain the corresponding word embedding representation. When input into a pre-trained model, the output embedding F(χ) of the masked word can be obtained at the mask position. i ,y) and the output vector G(χ) before classification i ,y),

[0055] F(χ i ,y)=PLM(χ i );

[0056] G(χ i ,y)=W*F(χi ,y)+b;

[0057] Where W∈R |V|×d It is the output projection matrix of the model, b∈R |V| Here, |v| is the size of the vocabulary, and |v| is the bias variable.

[0058] In this embodiment, the prompt T1 is concatenated with the original industrial text data "363 Car Fault Report: Fault Phenomenon: Steering gear makes abnormal noise when the vehicle is driving on a bumpy road" to form a complete input. During model training, such inputs are packaged into a batch, with each batch containing several such enhanced prompt texts. These merged text sequences are then fed into an encoder for processing, which converts each word in the entire sentence into a corresponding embedding vector. These embeddings are then fed into a pre-trained language model (such as BERT or RoBERTa), where the model outputs a vector representation at the [MASK] position and calculates the probability of each word being the answer.

[0059] Step 5: Input sequence X i After randomly covering a marker, the masked sequence X is obtained. i ', Replace the answer word corresponding to the real label at the mask position of the prompt template T1, and construct a new input sequence χ. i ', using a pre-trained language model to predict masked token words, and employing cross-entropy loss L f Maximize the conditional probability p(word|χ′) i ,y),

[0060]

[0061] Where M is the set of all occluded locations, and BCE() represents the binary cross-entropy loss function;

[0062] In this embodiment, a word is randomly selected from the original industrial text data "363 Car Fault Report Fault Phenomenon: Steering gear abnormal noise when the vehicle is driving on a bumpy road" to mask it. For example, "car" can be masked to obtain the new text input "363 [MASK] Fault Report Fault Phenomenon Fault Phenomenon: Steering gear abnormal noise when the vehicle is driving on a bumpy road". At the [MASK] position in the prompt template T1, the answer word r corresponding to the relationship label between "steering gear" and "abnormal noise" is used as the entity pair. i Replace to get a new prompt template [t1][t2]...[t n Steering gear iAn unusual noise was detected. These inputs were then fed into a pre-trained language model, which predicted the randomly masked word "car." The entire process involved training by maximizing the probability of the correct word appearing, thereby continuously optimizing the model's ability to understand context.

[0063] Step 6: Divide the training samples into two formats: masked format and unmasked format. In the masked format, use the entity pairs [sub] and [obj] as input to the model, and combine them with the cue T1 to form a new input sequence χ. entity =[CLS][sub][obj][SEP].T1[SEP], which uses the pre-classification output vector of the mask position as the entity bias G(χ). entity The input sequence χ consists of blank input and prompt T1. prompt =[CLS][SEP].T1[SEP], obtain the pre-classification output vector at the mask position as the cue bias G(χ). prompt The bias is subtracted from the original input pre-classification output vector (y), .

[0064] x out =G(χ) i ,y)-G(χ entity ,y)-G(χ prompt ,y);

[0065] The fine-tuning process uses cross-entropy loss L mlm optimization,

[0066]

[0067] p(y|x)=w*ReLU(x out )+c;

[0068] Where, w∈R |V|×d It is the output projection matrix of the model, c∈R |V| For bias variables;

[0069] In this embodiment, the original text input and template T1 are directly merged as the input to the model under the mask format, and the predicted score logits of the label can be obtained at the [MASK] position. In addition, another blank input format containing only prompts is constructed, only including "[t1][t2]...[t... n When the entity pair "steering gear [MASK] abnormal noise" is input into the model, the model's prediction result for "[MASK]" mainly reflects the template's own preference or the guiding effect of the prompt, which is called prompt bias. The entity pairs "steering gear" and "abnormal noise" are used as input, forming a new input "[CLS]steering gear, abnormal noise [SEP][t1][t2]...[t... nWhen the steering gear [MASK] abnormal noise [SEP] is input into the model, the model's prediction of [MASK] mainly reflects the guiding effect of the entity, which is called entity bias. In order to eliminate the interference of bias on the results, the bias score is subtracted from the original prediction score, retaining the true semantic differences related to the entity. Finally, during training, cross-entropy loss is used to optimize this adjusted output, so that the model focuses more on the semantic relationship between entities rather than the influence of the template itself.

[0070] Step 7: Inject relational knowledge into the answer space Q = {r1, r2, ..., r...} n Select the current input X in} i Real Relationship Tag y i The corresponding answer word is r. i As a positive sample, randomly select except r i Any answer word outside of r is used as a negative sample. i By calculating the cosine similarity, the model prediction result F(χ) is... i (y) is closer to the true answer words and further away from irrelevant answer words.

[0071] L rel =max{cos(F(χ i ,y),r i )-cos(F(χ i ,y),r i ')+γ,0};

[0072] In this embodiment, the relationship between the entity pairs "steering gear" and "abnormal noise" in the original industrial text data "363 car fault report: When the vehicle is driving on a bumpy road, there is abnormal noise from the steering gear" is "component fault". During training, the word corresponding to the relation label "component" in the answer space is used as a positive sample, and another relation word, such as "fault occurred" or "dependence", is randomly selected from the answer space as a negative sample. After the model processes the prompt template and performs context encoding, it outputs a vector in the answer space. The cosine similarity between this vector and the word vector of the positive sample ("component fault") is calculated, and the similarity between this vector and the negative sample is also calculated. The training objective is to make the vector output by the model more closely resemble "component fault" and less closely resemble irrelevant relations such as "fault occurred".

[0073] Step 8: In the unmasked format, replace the [MASK] markers in the original prompt template with relational answer words in the continuous space. Concatenate the output vectors corresponding to the positions of the two starting markers to construct the relational representation e. rel The data is then fed into a classifier to output a probability distribution over the label set Y. The fine-tuning process utilizes cross-entropy loss L... cls Optimize,

[0074] e rel =[e [CLS] ;e [SEP] ];

[0075] p i =soft max(W*e rel +b);

[0076] L cls =-log p i ;

[0077] Where W∈R |V|×d It is the output projection matrix of the model, b∈R |V| For bias variables;

[0078] In this embodiment, the relationship between the entity pairs "steering gear" and "abnormal noise" in the original industrial text data "363 car fault report: Fault phenomenon: When the vehicle is driving on a bumpy road, the steering gear makes abnormal noise" is "component fault". In the unmasked format, [MASK] is no longer used as a placeholder in template T1; instead, the relation answer word "component fault" is directly filled into the [MASK] position in the prompt template. Next, the output vectors from the two starting positions of the input are extracted and concatenated to form the semantic representation of the entire relation. This relation representation is then fed into a classifier, which outputs the probability distribution on all possible relation labels (such as "component fault", "control", "dependency", etc.). Finally, cross-entropy loss is used to optimize the model.

[0079] Step 9: Calculate the overall loss L.

[0080] L=α*(L mlm +L rel )+(1-α)*L cls +β*L f ;

[0081] When the input is in mask format, α is 1, otherwise it is 0, and β is a hyperparameter;

[0082] Step 10: In the early stages of training, use a lower proportion of masked samples and retain a higher proportion of unmasked data. As training progresses, gradually increase the proportion of masked samples, eventually transitioning to a full-masked format.

[0083] In this embodiment, for the task of extracting relationships from industrial text, the model initially uses unmasked samples, with a ratio of approximately 80% unmasked and 20% masked. During this stage, explicit relational clues help the model quickly establish basic semantic connections between entities and relationships. As training progresses and the model's understanding of the task improves, the number of masked samples is gradually increased by approximately 20% per stage, reducing reliance on explicit relational clues. Finally, in the later stages of training, the training data is completely transformed into masked format, enabling the model to determine relationships between entities based solely on contextual information, even without relational clues, thus better adapting to the relationship prediction needs of real-world applications.

[0084] Step 11: Repeat steps 4 to 10. When L is less than the specified minimum loss value, end the calculation and take the relationship corresponding to the position index with the highest probability in the prediction result as the final result of the industrial entity relationship in the industrial domain text.

[0085] In this embodiment, the model iteratively performs the relationship prediction step multiple times, updating parameters, calculating the loss, and evaluating the gap between the current output and the true relationship each time. The training process ends when the loss value gradually decreases until it falls below a set minimum threshold. At this point, the relationship label corresponding to the position with the highest probability among all possible relationship labels is selected as the final output, which is used for subsequent applications such as knowledge graph construction or fault analysis.

[0086] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A relation extraction method for knowledge graph construction using low-annotation industrial corpora, characterized in that, First, a semantic collaboration mechanism is introduced to optimize the dynamic prompt template constructed by combining contextual information. Second, by combining feature alignment and phased course learning strategies, the model gradually adapts to the difficulty of the task and improves its ability to distinguish semantic differences. Finally, the model parameters are dynamically adjusted through joint loss, and the converged model is used to infer the relationships between entities. The relation extraction method includes the following steps: Step 1: Construct an entity annotation set by combining industry-specific terminology. This method utilizes existing terminology dictionaries in the industrial field to construct entity recognition templates. Through precise matching, it extracts corresponding head and tail entities from the text. It also uses trainable continuous identifiers to replace fixed words in traditional prompts to construct dynamic prompts. ,in , This represents identified industrial entities. , This represents the vocabulary of the pre-trained model. The number of trainable labels. This represents the complex semantic relationships that may exist between industrial entities within the context of industrial text data; Step 2: Use the Word2Vec model to train word vectors for industrial entities, obtain words that are semantically similar to the target entities, and then... k Embedding vectors of relevant words and head entity embedding Tail entity embedding Mean initialization template Trainable tags in Embedded representation , ； Step 3: Set the original relation labels By removing symbols and interpreting the relations in the table into natural language representations, we obtain a relation description set. Tips for hand-building Construct a new relation label sequence , The mask positions are embedded as answer words to construct a continuous answer space. , ； Step 4: Dynamic prompts Text data in the industrial sector merge, The number of samples in each batch is determined, and the merged sequence is encoded to obtain the corresponding word embedding representation. When input into a pre-trained model, the output embedding of the masked word can be obtained at the mask position. and the output vector before classification , ； ； in, It is the output projection matrix of the model. For bias variables, For vocabulary size; Step 5: Input sequence The sequence after randomly covering a marker in the middle is obtained by masking. In the prompt template The mask positions are replaced with the answer words corresponding to the actual labels to construct a new input sequence. Using pre-trained language models to predict masked tokens Through cross-entropy loss Maximize conditional probability , ； ； in, It is the set of all obscured locations. This represents the binary cross-entropy loss function.

2. The relation extraction method for knowledge graph construction of low-labeled industrial corpora as described in claim 1, characterized in that, The relation extraction method further includes the following steps: Step 6: Divide the training samples into two formats: masked format and unmasked format. In the masked format, entity pairs are... , As input to the model, and with prompts Construct a new input sequence The pre-classification output vector obtained from the mask position is used as the entity bias. Blank input and prompts Construct the input sequence The pre-classification output vector at the mask position is used as the cue bias. Subtract the bias from the original input pre-classification output vector. ； The fine-tuning process uses cross-entropy loss optimization, ； ； in, It is the output projection matrix of the model. For bias variables; Step 7: Injecting relational knowledge into the answer space Select current input Real Relationship Tags Corresponding answer words As a positive sample, randomly select except Any answer word outside of the sample is used as a negative sample. By calculating cosine similarity, the model's prediction results are... It gets closer to the actual answer words and further away from irrelevant answer words. ； Step 8: In the unmasked format, replace the [MASK] markers in the original prompt template with relational answer words in the continuous space. Concatenate the output vectors corresponding to the positions of the two starting markers to construct the relational representation. The data is then fed into a classifier to output a probability distribution over the label set Y. The fine-tuning process uses cross-entropy loss. Optimize, ； ； ； in, It is the output projection matrix of the model. This is a bias variable.

3. The relation extraction method for knowledge graph construction of low-labeled industrial corpora as described in claim 2, characterized in that, The relation extraction method further includes the following steps: Step 9: Calculate the overall loss , ； When the input is in mask format, It is 1 if it is true, otherwise it is 0; For hyperparameters; Step 10: In the early stage of training, a lower proportion of masked format samples are used and a higher proportion of unmasked format data is retained. As the training process progresses, the proportion of masked samples is gradually increased, and finally transitions to the full masked format. Step 11: Repeat steps 4 to 10, when Once the value is less than the specified minimum loss, the calculation ends, and the relationship corresponding to the position index with the highest probability in the prediction results is taken as the final result of the industrial entity relationship in the industrial domain text.

Images