Method for constructing relationship triple, knowledge enhancement method and related device thereof

Abstract

The application provides a relationship triple construction method, a knowledge enhancement method and related devices, which can fine-tune a pre-training model using knowledge corpus of an application scenario of a knowledge graph to obtain a language model. The language model learns the knowledge of the application scenario and can distinguish whether a target sentence meets the part-of-speech requirement of a relationship triple, whether it is a knowledge element meeting the knowledge of the application scenario of the knowledge graph, and then construct a relationship triple according to the target sentence meeting the above requirements. Manual annotation is not required, which avoids the problem that manual annotation may not cover all possible entities and relationships, resulting in low quality of the knowledge graph.

Description

Technical Field

[0001] This application relates to the field of knowledge graph technology, and in particular to a method for constructing relation triples, a knowledge enhancement method, and related apparatus. Background Technology

[0002] A relation triple is a form of representation for the relationships between entities in a knowledge graph. It typically consists of three elements: a head entity, a relation, and a tail entity, and is one of the most basic representations in a knowledge graph. Currently, extracting relation triples requires manual annotation of entities, relations, and related information to facilitate supervised training based on the annotations. However, manually annotated content may not cover all possible entities and relations, thus reducing the quality of the knowledge graph. Summary of the Invention

[0003] In view of this, this application proposes a method for constructing relation triples, a knowledge enhancement method, and related devices to solve the problem that manually annotated content in the prior art may not be able to cover all possible entities and relations, thereby reducing the quality of the knowledge graph.

[0004] The technical solution proposed in this application is as follows:

[0005] In a first aspect, embodiments of this application provide a method for constructing relational triples, including:

[0006] Target sentences are extracted from the raw data; the target sentences include sentences whose parts of speech match the part-of-speech requirements of relation triples, and the relation triples are used to construct a knowledge graph;

[0007] The target statement is input into the language model to obtain the element type prediction result output by the language model; the language model is obtained by retraining the pre-trained model using knowledge corpus belonging to the application scenario of the knowledge graph; the element type prediction result includes a first prediction result that the target statement is a knowledge element that conforms to the application scenario of the knowledge graph.

[0008] The target statement whose element type prediction result is the first prediction result is input into the language model to obtain the relation prediction result output by the language model; the relation prediction result includes relation triples constructed based on the target statement.

[0009] Secondly, embodiments of this application provide an apparatus for constructing relational triples, comprising:

[0010] The extraction module is used to extract target sentences from the raw data; the target sentences include sentences whose parts of speech match the part-of-speech requirements of relation triples, and the relation triples are used to construct a knowledge graph;

[0011] An input module is used to input the target statement into a language model to obtain the element type prediction result output by the language model; the language model is obtained by retraining a pre-trained model using knowledge corpus belonging to the application scenario of the knowledge graph; the element type prediction result includes a first prediction result that the target statement is a knowledge element that conforms to the application scenario of the knowledge graph; the target statement with the first prediction result is input into the language model to obtain the relation prediction result output by the language model; the relation prediction result includes relation triples constructed based on the target statement.

[0012] Thirdly, embodiments of this application provide a knowledge enhancement method, including:

[0013] The first word embedding of the same sentence in the language model is replaced by graph embedding of the knowledge graph; the language model is obtained by retraining the pre-trained model using knowledge corpus of the application scenario belonging to the knowledge graph.

[0014] By fixing the parameters of the language model and the first word embedding, and updating the second word embedding in the language model, a knowledge-enhanced language model is obtained; the second word embedding includes the word embeddings in the language model that have not been replaced.

[0015] The method for extracting relation triples from the knowledge graph includes: extracting target sentences from raw data; the target sentences include sentences whose parts of speech match the part-of-speech requirements of relation triples, and the relation triples are used to construct the knowledge graph; inputting the target sentences into a language model to obtain the element type prediction results output by the language model; the language model is obtained by retraining a pre-trained model using knowledge corpus belonging to the application scenario of the knowledge graph, and the element type prediction results include a first prediction result that the target sentences are knowledge elements that conform to the application scenario of the knowledge graph; inputting the target sentences with the first prediction result into the language model to obtain the relation prediction results output by the language model; the relation prediction results include relation triples constructed based on the target sentences.

[0016] Fourthly, embodiments of this application provide a knowledge enhancement device, including:

[0017] The replacement module is used to replace the first word embedding of the same sentence in the language model using graph embedding of the knowledge graph; the language model is obtained by retraining the pre-trained model using knowledge corpus belonging to the application scenario of the knowledge graph.

[0018] An update module is used to fix the parameters and first word embedding of the language model, and update the second word embedding in the language model to obtain a knowledge-enhanced language model; the second word embedding includes the word embeddings in the language model that have not been replaced.

[0019] The method for extracting relation triples from the knowledge graph includes: extracting target sentences from raw data; the target sentences include sentences whose parts of speech match the part-of-speech requirements of relation triples, and the relation triples are used to construct the knowledge graph; inputting the target sentences into a language model to obtain the element type prediction results output by the language model; the language model is obtained by retraining a pre-trained model using knowledge corpus belonging to the application scenario of the knowledge graph, and the element type prediction results include a first prediction result that the target sentences are knowledge elements that conform to the application scenario of the knowledge graph; inputting the target sentences with the first prediction result into the language model to obtain the relation prediction results output by the language model; the relation prediction results include relation triples constructed based on the target sentences.

[0020] Fifthly, embodiments of this application provide an electronic device, including:

[0021] Memory and processor;

[0022] The memory is used to store programs;

[0023] The processor is configured to implement the method for constructing relation triples as described above, and / or the method for knowledge enhancement as described above, by running a program in the memory.

[0024] Sixthly, embodiments of this application provide a storage medium storing a computer program, which, when executed by a processor, implements the method for constructing relation triples as described above, and / or implements the knowledge enhancement method as described above.

[0025] The proposed method for constructing relation triples extracts target sentences from raw data. These target sentences include those whose parts of speech match the part-of-speech requirements of relation triples. The target sentences are then input into a pre-trained language model to obtain element type predictions. These predictions are used to determine relation predictions. The relation triples are used to construct a knowledge graph. The language model is obtained by retraining the pre-trained model using knowledge corpus from application scenarios within the knowledge graph. In other words, this application fine-tunes the pre-trained model using knowledge corpus from application scenarios within the knowledge graph, resulting in the aforementioned language model. This allows the language model to learn the knowledge of the application scenario, enabling it to distinguish whether target sentences whose parts of speech match the part-of-speech requirements of relation triples are knowledge elements relevant to the application scenario of the knowledge graph. Relation triples are then constructed based on these target sentences, eliminating the need for manual annotation. This avoids the problem that manually annotated content may not cover all possible entities and relations, leading to low-quality knowledge graphs. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating a method for constructing relation triples provided in an embodiment of this application;

[0028] Figure 2 This is a flowchart illustrating a knowledge enhancement method provided in an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of the structure of a relation triplet construction device provided in an embodiment of this application;

[0030] Figure 4 This is a schematic diagram of the structure of a knowledge enhancement device provided in an embodiment of this application;

[0031] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0032] The technical solution of this application embodiment is applicable to the application scenario of constructing knowledge graphs. By adopting the technical solution of this application embodiment, the problem that the content of manual annotation cannot cover all possible entities and relationships, resulting in low quality of knowledge graphs can be avoided.

[0033] A relation triple is a form of representation of relationships between entities in a knowledge graph. It typically consists of three elements: a head entity, a relation, and a tail entity, and is one of the most basic representations in a knowledge graph. Extracting entities, relations, and events from text through three key technical tasks—entity extraction, relation extraction, and event extraction—is a commonly used method for relation triple extraction. Specifically, entity extraction identifies entity names in text, relation extraction identifies semantic relationships between entities, and event extraction identifies events in text and the relationships between events and participants.

[0034] This approach requires significant manpower and time to annotate entities, relations, and events, along with their related information, and then supervise the training of the entity, relation, and event extraction model based on the annotated content. This allows the trained model to extract entities, relations, and events from natural language text. However, manually annotated content may not cover all possible entities, relations, and events, leading to incomplete extraction of relation triples and thus reducing the quality of the knowledge graph.

[0035] For example, when constructing a knowledge graph for medical scenarios, the sheer variety of text types in medical contexts—such as medical records, examination reports, medical textbooks, and medical guidelines—and the highly specialized nature of these texts, containing numerous medical terms and common knowledge, necessitates significant manpower and time for annotating entities, relationships, events, and related information within these texts when automatically extracting relation triples based on entity, relation, and event extraction. Furthermore, the data annotated by medical experts may not cover all possible entity, relation, and event types, thus reducing the quality of the constructed knowledge graph.

[0036] Based on this, embodiments of this application provide a method for constructing relation triples, a knowledge enhancement method, and related apparatus to address the problem that manually annotated content in the prior art may not be able to cover all possible entities and relations, thereby reducing the quality of the knowledge graph.

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] This application proposes a method for constructing relational triples. This method can be executed by an electronic device, which can be any device with data and instruction processing capabilities, such as a computer, smart terminal, or server. See also Figure 1 As shown, the method includes:

[0039] S101. Extract the target statement from the original data.

[0040] The aforementioned raw data refers to a corpus set that matches the application scenario of the knowledge graph to be constructed. For example, if a knowledge graph for a medical scenario needs to be constructed, a corpus set for the medical scenario needs to be collected as the raw data; if a knowledge graph for an enterprise knowledge management scenario needs to be constructed, a corpus set for the enterprise knowledge management scenario needs to be collected as the raw data. The corpus can be text data from various sources such as books, news, social media, and academic papers, or it can be audio data from recordings or speech transcriptions; this implementation does not impose any limitations.

[0041] The target statements mentioned above are used to construct relation triples. These target statements include sentences whose parts of speech match the part-of-speech requirements of the relation triples. The part-of-speech requirements of the relation triples are related to the application scenario of the knowledge graph constructed from them. In other words, the part-of-speech requirements of the relation triples can be determined based on the application scenario of the knowledge graph, and then sentences whose parts of speech match the part-of-speech requirements can be extracted from the original data.

[0042] In most knowledge graph applications, relation triples are used to represent specific objects, concepts, or relationships, such as names of people, places, companies, products, brands, events, attributes, and so on. In these applications, the part-of-speech requirement for relation triples is nouns, and nouns need to be extracted from the raw data as the target sentence. However, in some specific applications, relation triples can also be adjectives, verbs, adverbs, or even entire sentences. For example, in the construction of sentiment knowledge graphs, sentiment and attitude can be considered entities, and these entities usually appear in the form of adjectives. In such applications, the part-of-speech requirement for relation triples is both nouns and adjectives, and nouns and adjectives need to be extracted from the raw data as the target sentence.

[0043] In some embodiments, relevant personnel can set part-of-speech requirements for relation triples based on the application scenario of the knowledge graph, so as to extract sentences whose parts of speech meet the part-of-speech requirements of relation triples from the original data.

[0044] After determining the part-of-speech (POS) requirements for relation triples, Chinese POS tagging tools can be used to process the raw data and extract the target sentences. These tools include THULAC and Jieba. Chinese POS tagging tools typically employ machine learning and natural language processing techniques to tag each word in the Chinese text with its corresponding part of speech, such as noun, verb, or adjective.

[0045] For example, if the application scenario of the knowledge graph is a medical scenario, and the part-of-speech requirement for the relation triple is noun, the original data in the medical scenario can be obtained, and the sentences belonging to nouns can be extracted from the original data using Chinese part-of-speech tagging tools as target sentences.

[0046] S102. Input the target statement into the language model to obtain the element type prediction result output by the language model.

[0047] The aforementioned relation triples are used to construct a knowledge graph. The pre-trained language model is obtained by retraining the pre-trained model using knowledge corpus belonging to the application scenario of the knowledge graph. In this embodiment, the language model can generate complete sentences by supplementing the provided corpus. Retraining refers to using knowledge corpus belonging to the application scenario of the knowledge graph to perform unsupervised or weakly supervised training on the pre-trained model again.

[0048] The aforementioned pre-trained model refers to a general language model obtained through unsupervised or weakly supervised training using large-scale unlabeled data. In the embodiments of this application, the pre-trained model is retrained using knowledge corpus belonging to the application scenarios of the knowledge graph, so that the resulting language model can learn the knowledge of the application scenarios of the knowledge graph. In some embodiments, the basic model that the pre-trained model can adopt includes: Bidirectional Encoder Representations from Transformers (BERT) model, Generative Adversarial Network (GAN) model, etc., which are not limited in this embodiment.

[0049] In some embodiments, the pre-trained model is retrained using knowledge corpus belonging to the application scenarios of the knowledge graph through a masking strategy. There are three masking strategies: random replacement with a mask, random word replacement, or retention of original words. 15% of the aforementioned knowledge corpus can be randomly selected for masking. Of this masked portion, 80% can be randomly replaced with a mask, 10% can be randomly replaced, and 10% can be retained. After retraining the pre-trained model, the resulting language model can learn comprehensive knowledge of the application scenarios of the knowledge graph.

[0050] The aforementioned knowledge elements include entity elements or relation elements. Entity elements refer to the entities within a relation triple, while relation elements refer to the relations within a relation triple. Furthermore, entity elements include head entity elements and tail entity elements.

[0051] The element type prediction results include a first prediction result if the target statement is a knowledge element that conforms to the application scenario of the knowledge graph, or a second prediction result if the target statement is not a knowledge element that conforms to the application scenario of the knowledge graph.

[0052] Specifically, in this embodiment, the target statement is input into a language model, which then predicts whether the target statement is a knowledge element that conforms to the application scenario of the knowledge graph. In other words, the language model predicts whether the target statement is an entity element or relation element that conforms to the application scenario of the knowledge graph. If the predicted target statement is an entity element or relation element that conforms to the application scenario of the knowledge graph, the prediction result is determined as the first prediction result; if the predicted target statement is not an entity element or relation element that conforms to the application scenario of the knowledge graph, the prediction result is determined as the second prediction result.

[0053] S103. Input the target statement whose element type prediction result is the first prediction result into the language model to obtain the relation prediction result output by the language model.

[0054] Based on the above process, target statements with the second prediction result can be deleted, and only target statements with the first prediction result can be used to determine the relation prediction result. That is, the target statements with the first prediction result of element type are input into the language model to obtain the relation prediction result output by the language model. This reduces the noise brought by target statements of knowledge elements that are not applicable to the application scenario of knowledge graph, thereby improving the processing speed and quality of the language model.

[0055] The aforementioned relation prediction results include relation triples constructed based on the target statement. In other words, in the embodiments of this application, the target statement with the first prediction result is input into a pre-trained language model to obtain relation triples constructed based on the target statement output by the language model.

[0056] As described in the above embodiments, the language model can supplement and generate complete sentences based on the provided corpus. Therefore, the relational text constructed based on the target sentence with the first prediction result can be input into the pre-trained language model to obtain the text prediction result of the language model for the relational text, and the above text prediction result is determined as the relational prediction result.

[0057] The relational text is used to request the output of relational triples constructed based on the target statement whose prediction result is the first prediction result. In some embodiments, the relational text is a text containing a mask constructed based on the target statement whose prediction result is the first prediction result. The language model predicts the content of the mask portion to achieve the purpose of predicting relational triples based on the target statement whose prediction result is the first prediction result. Specifically, any three target statements can be extracted from the target statement whose prediction result is the first prediction result and arranged and combined to automatically construct the relational text according to the following template:

[0058] "Is A the C of B? Yes or No? MASK", where A, B, and C are the target statements whose prediction result is the first prediction result.

[0059] The above relational text is input into the language model so that the language model can predict the "MASK" part. If the language model predicts "yes" for the "MASK" part, it means that A, B, and C are relation triples, where A and B are entities in the relation triple and C is the relation in the relation triple. If the language model predicts "no" for the "MASK" part, it means that A, B, and C are not relation triples.

[0060] In the embodiments of this application, all target statements whose prediction result is the first prediction result can be traversed, and multiple relational texts can be constructed in the manner described above to automatically predict all possible relational triples without human intervention.

[0061] In the above embodiments, the pre-trained model can be fine-tuned using the knowledge corpus of the application scenario of the knowledge graph to obtain a language model. The language model learns the knowledge of the application scenario and can distinguish whether the target sentence whose part of speech meets the part-of-speech requirements of the relation triple is a knowledge element that conforms to the application scenario of the knowledge graph. Then, relation triples are constructed based on the target sentences that meet the above requirements. No manual annotation is required, which avoids the problem that the content of manual annotation may not be able to cover all possible entities and relations, resulting in low quality of the knowledge graph.

[0062] As an optional implementation, another embodiment of this application discloses that the steps of the above embodiments, which input the target statement into the language model and obtain the element type prediction result output by the language model, may specifically include the following steps:

[0063] The first text is input into the language model, and the text prediction result of the language model for the first text is obtained; the text prediction result for the first text is determined to be the element type prediction result.

[0064] The first text mentioned above is used to request confirmation as to whether the target statement is a knowledge element that conforms to the application scenario of the knowledge graph. The first text is a text containing a mask constructed based on the target statement. The language model predicts whether the target statement is a knowledge element that conforms to the application scenario of the knowledge graph by predicting the content of the mask part.

[0065] In the embodiments of this application, the first text is input into the language model to obtain the text prediction result of the MASK part in the first text output by the language model, and the text prediction result of the MASK part in the first text is determined as the element type prediction result.

[0066] Specifically, the first text can be automatically constructed using the following template:

[0067] "Is D a knowledge element in the application scenario of the knowledge graph? Yes or no? MASK", where D is the target statement.

[0068] The first text is constructed using each target statement. Each first text is then input into a language model so that the language model can predict the "MASK" part. If the language model predicts "yes" for the "MASK" part, it means that the target statement in the first text is a knowledge element of the application scenario of the knowledge graph. If the language model predicts "no" for the "MASK" part, it means that the target statement in the first text is not a knowledge element of the application scenario of the knowledge graph.

[0069] Furthermore, since knowledge elements include entity elements and relation elements, the first text constructed for the target statement can include entity element prediction text and relation element prediction text. Entity element prediction text can be used to predict whether the target statement is an entity element that conforms to the application scenario of the knowledge graph, and relation element prediction text can be used to predict whether the target statement is a relation element that conforms to the application scenario of the knowledge graph.

[0070] Specifically, the predicted text for entity elements can be constructed as: "Is D an entity element in the application scenario of the knowledge graph? Yes or no? MASK"; the predicted text for relation elements can be constructed as: "Is D a relation element in the application scenario of the knowledge graph? Yes or no? MASK".

[0071] Entity element prediction text is constructed using each target statement. Then, each entity element prediction text is input into a language model so that the language model can predict the "MASK" part. If the language model predicts "yes" for the "MASK" part, it means that the target statement in the first text is an entity element of the knowledge graph application scenario. If the language model predicts "no" for the "MASK" part, it means that the target statement in the first text is not an entity element of the knowledge graph application scenario.

[0072] The system constructs relational element prediction text using each target statement, and then inputs each relational element prediction text into a language model so that the language model can predict the "MASK" part. If the language model predicts "yes" for the "MASK" part, it means that the target statement in the first text is a relational element of the knowledge graph application scenario. If the language model predicts "no" for the "MASK" part, it means that the target statement in the first text is not a relational element of the knowledge graph application scenario.

[0073] The target statements of entity elements in the application scenario predicted by the knowledge graph can be grouped into an entity element set, and the target statements of relation elements in the application scenario predicted by the knowledge graph can be grouped into a relation element set.

[0074] For example, if the application scenario of the knowledge graph is a medical scenario, then the entity element prediction text can be constructed as: "Is D an entity element related to the medical scenario? Yes or no? MASK"; the relation element prediction text can be constructed as: "Is D a relation element related to the medical scenario? Yes or no? MASK". Then, the above entity element prediction text and relation element prediction text are respectively input into the language model so that the language model can predict whether the target statement belongs to the relation element or entity element of the medical scenario.

[0075] In the above embodiments, the target sentences that are predicted as the first prediction result can be selected by using the language model. This not only reduces noise and improves the processing speed and quality of the language model, but also allows the language model to learn about relevant application scenarios without manual annotation, thus avoiding the problem that manually annotated content cannot cover all possible entities and relationships.

[0076] As an optional implementation, another embodiment of this application discloses that the steps of the above embodiments, which input the target statement whose element type prediction result is the first prediction result into the language model to obtain the relation prediction result output by the language model, may specifically include the following steps:

[0077] The second text is input into the language model to obtain the text prediction result of the language model for the second text; based on the text prediction result for the second text, the relationship prediction result is determined.

[0078] The second text mentioned above is used to request the output of statements that belong to the same relation triplet as the target statement whose element type prediction result is the first prediction result. The second text is a text containing a mask constructed based on the target statement whose element type prediction result is the first prediction result. The language model predicts the content of the mask part to achieve the purpose of predicting statements that belong to the same relation triplet as the target statement whose element type prediction result is the first prediction result.

[0079] In the embodiments of this application, the second text is input into a language model to obtain the text prediction result of the language model for the MASK part of the second text. Based on the text prediction result for the MASK part of the second text, the relation prediction result is determined. That is, for any second text, the target sentence that constructs the second text and the prediction result of the language model for the second text can be combined into a relation triple.

[0080] Specifically, the following template can be used to automatically construct the second text:

[0081] The statement "A's B is C" is defined where any one of A, B, or C can be masked. The unmasked portions of A, B, and C are the target statements whose element type prediction is the first prediction result. Furthermore, if A is not masked, A is the statement belonging to an entity element in the target statements whose element type prediction is the first prediction result; if B is not masked, B is the statement belonging to a relation element in the target statements whose element type prediction is the first prediction result; and if C is not masked, C is the statement belonging to an entity element in the target statements whose element type prediction is the first prediction result. The language model's prediction results for the masked portions of the second text, along with the unmasked target statements in the second text, are combined to form relation triples.

[0082] For example, if A is the MASK part, then the second text is "MASK's B is C". The entity element set and relation element set in the above embodiments can be traversed to extract target statements belonging to entity elements and relation elements, constructing the second text. Each second text is then input into the language model to predict the "MASK" part. The language model's prediction of the "MASK" part, B, and C form a relation triple. In this relation triple, the prediction of the "MASK" part is the head entity element, B is the relation element, and C is the tail entity element.

[0083] For example, if B is the MASK part, then the second text is "A's MASK is C". The entity element set in the above embodiments can be traversed, and any two target statements belonging to entity elements can be extracted from the entity element set to construct the second text. Each second text is then input into the language model so that the language model predicts the "MASK" part. A, the language model's prediction result for the "MASK" part, and C are combined into a relation triple. In this relation triple, A is the head entity element, the prediction result for the "MASK" part is the relation element, and C is the tail entity element.

[0084] The language models used in the above embodiments can master the knowledge of relevant application scenarios without manual annotation training. They can achieve the goal of predicting relation triples while avoiding the problem that manually annotated content cannot cover all possible entities and relations.

[0085] As an optional implementation, another embodiment of this application discloses that if the target statement whose element type prediction result is the first prediction result includes a first target statement belonging to an entity element and a second target statement belonging to a relation element, then the steps of the above embodiment input the second text into the language model to obtain the text prediction result for the second text output by the language model, which may specifically include the following steps:

[0086] Construct a second text for requesting the output of a third target statement that belongs to the same relation triplet as the first target statement and the second target statement; input the second text into the language model to obtain the third target statement output by the language model.

[0087] If the target statement whose element type prediction result is the first prediction result includes a first target statement belonging to an entity element and a second target statement belonging to a relation element, then a second text can be constructed to request the output of a third target statement that belongs to the same relation triplet as both the first and second target statements.

[0088] As described in the above embodiments, the second text is a text containing a mask constructed from the target statement whose element type prediction result is the first prediction result. In the embodiments of this application, when constructing the second text, the first target statement can be placed at the position of the head entity element, and the position of the tail entity element can be masked to facilitate the prediction of the third target statement belonging to the tail entity element based on the first target statement and the second target statement. Alternatively, the first target statement can be placed at the position of the tail entity element, and the position of the head entity element can be masked to facilitate the prediction of the third target statement belonging to the head entity element based on the first target statement and the second target statement.

[0089] If the first target statement is placed in the header entity element, the second text can be constructed as follows:

[0090] "A's B is MASK"; where A is the first target statement, B is the second target statement, the second text is input into the language model, and the language model's prediction result for the MASK part is the third target statement.

[0091] The entity element set and relation element set in the above embodiments can be traversed to extract target statements belonging to entity elements and target statements belonging to relation elements, thus constructing second text. Each second text is then input into a language model to predict the "MASK" part. A, B, and the language model's prediction of the "MASK" part are combined to form a relation triple. In this relation triple, A is the first target statement and belongs to the head entity element, B is the second target statement and belongs to the relation element, and the language model's prediction of the "MASK" part is the third target statement and belongs to the tail entity element.

[0092] If the first target statement is placed at the position of the last entity element, the second text can be constructed as follows:

[0093] “The B of MASK is C”; where C is the first target statement and B is the second target statement. The second text is input into the language model, and the language model predicts the third target statement for the MASK part.

[0094] The entity element set and relation element set in the above embodiments can be traversed to extract target statements belonging to entity elements and relation elements, constructing second text. Each second text is then input into a language model to predict the "MASK" part. The language model's prediction of the "MASK" part, along with B and C, forms a relation triple. In this relation triple, the language model's prediction of the "MASK" part is the third target statement and belongs to the head entity element; B is the second target statement and belongs to the relation element; and C is the first target statement and belongs to the tail entity element.

[0095] The language models used in the above embodiments can master the knowledge of relevant application scenarios without manual annotation training. They can achieve the goal of predicting relation triples while avoiding the problem that manually annotated content cannot cover all possible entities and relations.

[0096] As an optional implementation, another embodiment of this application discloses the method of the above embodiments, which may specifically include the following steps:

[0097] The third text is input into the language model, and the text prediction result of the language model is obtained. If the text prediction result of the third text includes relation triples that do not have triple relations, the relation triples are deleted.

[0098] To improve the accuracy of relation triples and further enhance the quality of knowledge graphs, speech models can be used to further validate relation triples. Specifically, a third text can be constructed based on the relation triples. This third text is used to request confirmation of whether the relation triples possess a triple relationship. The third text is a text containing a mask constructed based on the relation triples. The language model can predict whether the relation triples possess a triple relationship by predicting the content of the mask portion.

[0099] In some embodiments, the third text may include a binary classification problem. Specifically, the third text can be automatically constructed using the following template:

[0100] "Is B of A C? Yes or No? MASK"; where A is the head entity element in the relation triple, B is the relation element in the relation triple, and C is the tail entity element in the relation triple.

[0101] Each set of relation triples predicted by the language model can be used to construct a corresponding third text. Each third text is then input into the language model so that the language model can predict the "MASK" part. If the language model predicts "yes" for the "MASK" part, it means that the relation triple has a triple relation. If the language model predicts "no" for the "MASK" part, it means that the relation triple does not have a triple relation.

[0102] If prediction determines that a certain relation triple does not possess a triple relation, then such relation triples need to be deleted to avoid using them to construct a knowledge graph, thereby improving the accuracy of the knowledge graph.

[0103] In the above embodiments, relation triples can be verified to further improve the accuracy of the knowledge graph.

[0104] As an optional implementation, another embodiment of this application discloses the method of the above embodiments, which may specifically include the following steps:

[0105] When retraining a pre-trained model using knowledge corpus belonging to the application scenario of knowledge graph, a set number of model parameters of the pre-trained model are saved, and the model parameters are distributed and stored on each graphics card participating in the retraining of the pre-trained model.

[0106] When retraining a pre-trained model, in order to improve training speed and effectiveness, it is necessary to reduce the computational resources required during training. The traditional multi-machine, multi-GPU training method (Distributed Data Parallel, DDP) requires storing model parameters on each GPU involved in training, resulting in parameter redundancy in the pre-trained model and consuming a large amount of memory.

[0107] To reduce redundant parameters, this embodiment only saves a predetermined number of model parameters for the pre-trained model and distributes these parameters across the various GPUs involved in retraining the pre-trained model. The specific number of predetermined numbers can be set according to actual needs. In a particular embodiment, only one set of model parameters needs to be saved and distributed across the various GPUs involved in retraining the pre-trained model.

[0108] The model parameters of the pre-trained model mentioned above include the parameters of the pre-trained model itself and / or the optimizer parameters, which are not limited in this embodiment.

[0109] In some embodiments, BERT-large can be used as a pre-trained model. Compared to BERT-base, BERT-large has stronger expressive power, more knowledge, and performs better in downstream tasks. However, BERT-large requires more computation and places higher demands on GPUs. In the embodiments of this application, during the retraining process of BERT-large, only one set of model parameters needs to be saved, and the model parameters are evenly distributed across the GPUs, thereby significantly reducing the GPU memory required during training.

[0110] Furthermore, in the embodiments of this application, the Lion (EvoLved Sign Momentum) optimizer is used to retrain the pre-trained model. Compared to the commonly used optimizer AdamW, Lion has fewer parameters, caches fewer parameters, and eliminates the most computationally intensive division and square root operations during AdamW's update process, making it 2-15% faster in practice. Therefore, the Lion optimizer is chosen in this embodiment to reduce GPU memory usage and improve convergence speed.

[0111] In the above embodiments, by saving a set number of pre-trained model parameters and distributing the model parameters among the graphics cards participating in the retraining of the pre-trained model, the video memory occupied by the model parameters is reduced, thereby improving the training speed when retraining the pre-trained model.

[0112] As an optional implementation, another embodiment of this application discloses the method of the above embodiments, which may specifically include the following steps:

[0113] Extract word embeddings of relation triples from the language model; determine the graph embeddings of the knowledge graph constructed from relation triples based on the word embeddings of relation triples.

[0114] Specifically, commonly used knowledge graph embedding methods currently rely on a predefined scoring function to learn vector representations of entities and relations: embedding is learned by maximizing the score of true triples and minimizing the score of false triples. For example, the scoring function proposed by TransE (Translating Embedding) is f. r (h,t)=-||h+rt|| 1 / 2 (where h is the head entity element, r is the relation element, and t is the tail entity element), requiring that the embeddings in the embedding space satisfy translation principles as much as possible; while similarity-based embedding models utilize the multiplication formula h T M r ≈t is used to transform the head entity in the embedding space to make it more similar to the tail entity. For example, the Latent Factor Model (LFM) proposes an evaluation function h. T M r t.

[0115] However, current methods define triplet scoring functions based on relatively simple distance or similarity functions, which cannot well represent complex relationships. Moreover, knowledge graph embedding methods only utilize the structural information in the knowledge graph during the embedding learning process, ignoring the semantic information of entities and relationships in the real text context. This results in embeddings that cannot well reflect the semantics of entities and relationships, ultimately limiting the application of embeddings in downstream tasks.

[0116] Based on this, in the embodiments of this application, a language model is used to extract the graph embedding of the knowledge graph.

[0117] The word embeddings of relation triples are extracted from the language model. Specifically, the word embeddings of relation triples are vector representations of the relation triples. Specifically, relation triples can be input into the language model, and the language model can encode the relation triples to obtain vector representations of the relation triples, which serve as the word embeddings of the relation triples.

[0118] Based on the word embeddings of relation triples, the graph embeddings of the knowledge graph constructed from the relation triples are determined. In some embodiments, the word embeddings of relation triples can be determined as the graph embeddings of the knowledge graph constructed from those relation triples.

[0119] Since the language model is trained on the knowledge corpus of the application domain of the knowledge graph, the language model learns the knowledge of the application scenario, including the dependencies and associations between contexts. Based on the word embedding of relation triples, the language model determines the graph embedding of the knowledge graph constructed by relation triples, which can better reflect the semantics of entities and relations.

[0120] As an optional implementation, another embodiment of this application discloses that the steps of the above embodiments, based on the word embeddings of relation triples, determine the graph embedding of the knowledge graph constructed from relation triples, which may specifically include the following steps:

[0121] Using the features of the knowledge graph as samples and classifying the nodes in the knowledge graph as the goal, a classification model is trained. The classification model includes a graph neural network, and the features of the knowledge graph are word embeddings of relation triples extracted from the language model. The node features of the knowledge graph are extracted from the trained classification model as the graph embeddings of the knowledge graph.

[0122] Graph neural networks can directly learn entity and relation embeddings using the graph structure of knowledge graphs without pre-defining other features. They use graph convolution operations to learn the graph structure, obtaining overall contextual embeddings to represent entity and relation semantics. This allows for the organic learning of dependencies and associations within the knowledge graph, resulting in stronger expressive power. Based on this, in the embodiments of this application, a graph neural network is trained to classify nodes in the knowledge graph, achieving the goal of extracting graph embeddings from the knowledge graph.

[0123] Specifically, word embeddings of relation triples extracted from the language model can be used as features of the knowledge graph. Specifically, word embeddings of entity elements in the relation triples serve as node features of the knowledge graph, and word embeddings of relation elements in the relation triples serve as edge features. Using word embeddings from relation triples as node and edge features of the knowledge graph can provide contextual information for knowledge graph learning, helping GINE learn better node embedding features.

[0124] Using the features of a knowledge graph as samples, and aiming to classify the nodes in the knowledge graph, a classification model is trained. This classification model includes a Graph Isomorphism Network (GIN). It should be noted that a knowledge graph is a directed graph with edge features; therefore, this embodiment selects GIN, which can accept edge features. Assume the feature of a node is x. i Let N(i) be the set of neighboring nodes of a node, and let e be the feature of the edge from node i to node i. j,i The formula for GINE is:

[0125] x' i =h θ((1+∈)x i +∑ j∈N(i) ReLU(x j +e j,i ))

[0126] Among them, h θ It is a learnable multilayer perceptron (MLP).

[0127] In the embodiments of this application, GINE is trained in a semi-supervised manner. During the specific training process, 15% of the node features are randomly masked. It is assumed that the embedding of node i after passing through multiple GINE layers is... Using GINE's MLP to perform node classification tasks:

[0128]

[0129] Using real labels y i and predicted value The cross-entropy function is used as the loss function to optimize the parameters of the entire graph neural network model until the graph neural network is trained and a classification model is obtained.

[0130] Furthermore, node features of the knowledge graph are extracted from the trained classification model as graph embeddings of the knowledge graph. Specifically, the knowledge graph can be input into the trained classification model so that the classification model can perform a classification task for the knowledge graph. During the classification task, the classification model generates embeddings of each node of the knowledge graph. In the embodiments of this application, the embeddings of each node of the knowledge graph generated by the classification model are extracted as the embeddings of the knowledge graph.

[0131] In the above embodiments, using word embeddings in relation triples as features of nodes and edges in the knowledge graph can bring contextual information to the learning of the knowledge graph, so that the resulting embeddings can well reflect the semantics of entities and relations.

[0132] This application proposes a knowledge enhancement method, which can be executed by an electronic device. The electronic device can be any device with data and instruction processing capabilities, such as a computer, smart terminal, or server. See also... Figure 2 As shown, the method includes:

[0133] S201. Use graph embedding from a knowledge graph to replace the first word embedding of the same sentence in the language model.

[0134] The language model described above was obtained by retraining the pre-trained model using knowledge corpus belonging to application scenarios of knowledge graphs.

[0135] Specifically, in the embodiments of this application, graph embeddings of knowledge graphs can be used to replace the first word embeddings of the same statement in the language model. For example, if the statement corresponding to a certain node in the knowledge graph is "A", then the word embeddings of the same statement "A" in the pre-trained model can be replaced with the graph embeddings of the statement "A" in the knowledge graph.

[0136] In some embodiments, the methods for extracting relation triples for constructing knowledge graphs and the methods for extracting graph embeddings for knowledge graphs can be referred to the descriptions in the above embodiments, and will not be repeated here.

[0137] S202. Fix the parameters of the language model and the first word embedding, update the second word embedding in the language model, and obtain the knowledge-enhanced language model.

[0138] The second word embedding includes the word embeddings in the language model that have not been replaced.

[0139] Since the language model has already been retrained, there is no need to spend computational resources to fine-tune the parameters of the language model again. Instead, the parameters of the language model and the first word embedding are fixed, so that the language model learns embedding knowledge based on the first word embedding and the sentence corresponding to the first word embedding, and updates the second word embedding based on the learned embedding knowledge. The knowledge enhancement brought by the knowledge graph is implicitly fed back to the remaining word embeddings, so that the knowledge-enhanced word embeddings can bring greater help to downstream tasks.

[0140] Corresponding to the above-described method for constructing relation triples, this application also discloses a device for constructing relation triples, see [link to relevant documentation]. Figure 3 As shown, the device includes:

[0141] The extraction module 100 is used to extract target sentences from the raw data; the target sentences include sentences whose parts of speech match the part-of-speech requirements of relation triples, and the relation triples are used to construct a knowledge graph;

[0142] The input module 110 is used to input the target statement into the language model and obtain the element type prediction result output by the language model. The language model is obtained by retraining the pre-trained model using knowledge corpus belonging to the application scenario of the knowledge graph. The element type prediction result includes the first prediction result that the target statement is a knowledge element that conforms to the application scenario of the knowledge graph.

[0143] The target statement whose element type prediction result is the first prediction result is input into the language model to obtain the relation prediction result output by the language model; the relation prediction result includes relation triples constructed based on the target statement.

[0144] As an optional implementation, another embodiment of this application discloses that the input module 110 of the above embodiments, when inputting the target statement into the language model and obtaining the element type prediction result output by the language model, is specifically used for:

[0145] The first text is input into the language model, and the text prediction result of the language model is obtained. The first text is used to request confirmation whether the target statement is a knowledge element that conforms to the application scenario of the knowledge graph.

[0146] The text prediction result for the first text is determined to be the element type prediction result.

[0147] As an optional implementation, another embodiment of this application discloses that the input module 110 of the above embodiments, when inputting a target statement whose element type prediction result is the first prediction result into the language model to obtain the relation prediction result output by the language model, is specifically used for:

[0148] The second text is input into the language model to obtain the text prediction result of the language model for the second text; the second text is used to request the output of the statement that belongs to the same relation triple as the target statement whose element type prediction result is the first prediction result.

[0149] Based on the text prediction results for the second text, determine the relationship prediction results.

[0150] As an optional implementation, another embodiment of this application discloses that the knowledge elements include entity elements and relation elements. If the target statement whose element type prediction result is the first prediction result includes a first target statement belonging to entity elements and a second target statement belonging to relation elements, then the input module 110 of the above embodiment, when inputting the second text into the language model and obtaining the text prediction result for the second text output by the language model, is specifically used for:

[0151] Construct a second text for requesting the output of a third target statement that belongs to the same relation triplet as the first target statement and the second target statement;

[0152] The second text is input into the language model, and the third target sentence is output by the language model.

[0153] As an optional implementation, another embodiment of this application discloses that the apparatus for constructing relation triples in the above embodiments further includes:

[0154] The storage module is used to save a set number of model parameters of the pre-trained model when retraining the pre-trained model using knowledge corpus belonging to the application scenario of knowledge graph, and to distribute and store the model parameters in the various graphics cards participating in the retraining of the pre-trained model.

[0155] As an optional implementation, another embodiment of this application discloses that the apparatus for constructing relation triples in the above embodiments further includes:

[0156] The module is used to extract word embeddings of relation triples from the language model; and to determine the graph embeddings of the knowledge graph constructed from the relation triples based on the word embeddings of the relation triples.

[0157] As an optional implementation, another embodiment of this application discloses that when the determining module of the above embodiments determines the graph embedding of the knowledge graph constructed from relation triples based on the word embedding of relation triples, it is specifically used for:

[0158] Using the features of the knowledge graph as samples and classifying the nodes in the knowledge graph as the goal, a classification model is trained. The classification model includes a graph neural network, and the features of the knowledge graph are word embeddings of relation triples extracted from the language model. The node features of the knowledge graph are extracted from the trained classification model as the graph embeddings of the knowledge graph.

[0159] Specifically, for the specific working content of each unit of the above-mentioned relation triplet construction device, please refer to the content of the above-mentioned relation triplet construction method embodiment, which will not be repeated here.

[0160] Corresponding to the above-described knowledge enhancement method, this application also discloses a knowledge enhancement device, see [link to relevant documentation]. Figure 4 As shown, the device includes:

[0161] Replacement module 200 is used to replace the first word embedding of the same sentence in the language model using graph embedding of knowledge graph; the language model is obtained by retraining the pre-trained model using knowledge corpus of application scenarios belonging to knowledge graph;

[0162] The update module 210 is used to fix the parameters and first word embedding of the language model, update the second word embedding in the language model, and obtain a knowledge-enhanced language model; the second word embedding includes the word embeddings in the language model that have not been replaced.

[0163] The method for extracting relation triples in a knowledge graph includes: extracting target sentences from raw data; target sentences include sentences whose parts of speech match the part-of-speech requirements of relation triples, which are used to construct the knowledge graph; inputting the target sentences into a language model to obtain the element type prediction results output by the language model; the language model is obtained by retraining a pre-trained model using knowledge corpus belonging to the application scenario of the knowledge graph, and the element type prediction results include the first prediction result that the target sentences are knowledge elements that conform to the application scenario of the knowledge graph; inputting the target sentences with the first prediction result into the language model to obtain the relation prediction results output by the language model; the relation prediction results include relation triples constructed based on the target sentences.

[0164] This application also proposes a control device, which includes a processor and an interface circuit. The processor in the control device is connected to an input / output component through the interface circuit of the control device.

[0165] The input / output component specifically refers to the hardware component that enables the user to input information and output information to the user, such as a microphone, keyboard, handwriting tablet, touch screen, monitor, speaker, printer, etc.

[0166] The aforementioned interface circuit can be any interface circuit capable of implementing data communication functions, such as a USB interface circuit, a Type-C interface circuit, a serial port circuit, a PCIe circuit, etc.

[0167] The processor in the control device is a circuit with signal processing capabilities, which can execute any of the relation triple construction methods described in the above embodiments, and / or, knowledge enhancement methods.

[0168] When the control device is applied to a device with human-computer interaction function, the input and output components of the control device can be input components and output components on the device, such as microphone, keyboard, handwriting tablet, touch screen, display, audio player, etc. Meanwhile, the processor of the control device can be the device's built-in CPU or GPU, etc., and the interface circuit of the control device can be the interface circuit between the device's information input components and the CPU or GPU processor.

[0169] This application also discloses an electronic device, see [link to relevant documentation] Figure 5 As shown, the electronic device includes:

[0170] Memory 300 and processor 310;

[0171] The memory 300 is connected to the processor 310 and is used to store programs;

[0172] Processor 310 is configured to implement the relation triple construction method and / or knowledge enhancement method disclosed in any of the above embodiments by running a program stored in memory 300.

[0173] Specifically, the aforementioned electronic device may further include: a bus, a communication interface 320, an input device 330, and an output device 340.

[0174] The processor 310, memory 300, communication interface 320, input device 330, and output device 340 are interconnected via a bus. Among them:

[0175] A bus can include a pathway for transmitting information between various components of a computer system.

[0176] The processor 310 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0177] Processor 310 may include a main processor, as well as a baseband chip, modem, etc.

[0178] The memory 300 stores a program for executing the technical solution of this application, and may also store an operating system and other critical business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 300 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0179] Input device 330 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0180] Output device 340 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0181] The communication interface 320 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0182] The processor 310 executes the program stored in the memory 300 and calls other devices, which can be used to implement the relationship triple construction method and / or the knowledge enhancement method provided in the above embodiments of this application.

[0183] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the methods for constructing relation triples and / or the various steps of the knowledge enhancement methods provided in the above embodiments.

[0184] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0185] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the relation triple construction method and / or knowledge enhancement method provided in the above embodiments.

[0186] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0187] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0188] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0189] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0190] The modules and sub-modules in the apparatus and terminal in the various embodiments of this application can be merged, divided, and deleted according to actual needs.

[0191] It should be understood, in the several embodiments provided in this application, that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are merely illustrative; for instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0192] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0193] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0194] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0195] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0196] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0197] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing relational triples, characterized in that, include: Target sentences are extracted from the raw data; the target sentences include sentences whose parts of speech match the part-of-speech requirements of relation triples, and the relation triples are used to construct a knowledge graph; The target statement is input into the language model to obtain the element type prediction result output by the language model; The language model is obtained by retraining the pre-trained model using knowledge corpus belonging to the application scenario of the knowledge graph. The element type prediction result includes the first prediction result that the target sentence is a knowledge element that conforms to the application scenario of the knowledge graph. The second text containing a mask, constructed from the target statement whose element type prediction result is the first prediction result, is input into the language model to obtain the relation prediction result output by the language model; the relation prediction result includes relation triples constructed based on the target statement.

2. The method according to claim 1, characterized in that, The step of inputting the target statement into the language model and obtaining the element type prediction result output by the language model includes: The first text is input into the language model to obtain the text prediction result of the language model for the first text; the first text is a text containing a mask constructed based on the target statement, used to request confirmation whether the target statement is a knowledge element that conforms to the application scenario of the knowledge graph. The text prediction result for the first text is determined to be the element type prediction result.

3. The method according to claim 1, characterized in that, The step of inputting a second text containing a mask, constructed from the target statement whose element type prediction result is the first prediction result, into the language model to obtain the relation prediction result output by the language model includes: The second text is input into the language model to obtain the text prediction result of the language model for the second text; the second text is used to request the output of a statement that belongs to the same relation triplet as the target statement whose element type prediction result is the first prediction result; The relationship prediction result is determined based on the text prediction result for the second text.

4. The method according to claim 3, characterized in that, The knowledge elements include entity elements and relation elements. If the target statement with the element type prediction result being the first prediction result includes a first target statement belonging to entity elements and a second target statement belonging to relation elements, then inputting the second text into the language model to obtain the text prediction result for the second text output by the language model includes: Construct a second text for requesting the output of a third target statement that belongs to the same relation triplet as the first target statement and the second target statement; The second text is input into the language model to obtain the third target sentence output by the language model.

5. The method according to claim 1, characterized in that, Also includes: The third text is input into the language model to obtain the text prediction result of the language model for the third text. The third text is a text containing a mask constructed based on the relation triple, used to request confirmation as to whether the relation triple has a triple relationship; If the text prediction result of the third text includes a relation triplet that does not have a triplet relation, then the relation triplet is deleted.

6. The method according to claim 1, characterized in that, include: Extract word embeddings of the relation triples from the language model; Based on the word embeddings of the relation triples, the graph embeddings of the knowledge graph constructed from the relation triples are determined.

7. The method according to claim 6, characterized in that, The step of determining the graph embedding of the knowledge graph constructed from the relation triples based on the word embeddings of the relation triples includes: Using the features of the knowledge graph as samples, and aiming to classify the nodes in the knowledge graph, a classification model is trained; wherein, the classification model includes a graph neural network, and the features of the knowledge graph are word embeddings of the relation triples extracted from the language model; The node features of the knowledge graph are extracted from the trained classification model and used as the graph embedding of the knowledge graph.

8. A knowledge enhancement method, characterized in that, include: We use graph embedding from knowledge graphs to replace the first word embedding of the same sentence in the language model; The language model is obtained by retraining the pre-trained model using knowledge corpus belonging to the application scenarios of the knowledge graph. By fixing the parameters of the language model and the first word embedding, and updating the second word embedding in the language model, a knowledge-enhanced language model is obtained. The second word embedding includes the unreplaced word embeddings in the language model; The method for extracting relation triples from the knowledge graph includes: extracting target sentences from raw data; the target sentences include sentences whose parts of speech match the part-of-speech requirements of relation triples, and the relation triples are used to construct the knowledge graph; inputting the target sentences into a language model to obtain the element type prediction results output by the language model; the language model is obtained by retraining a pre-trained model using knowledge corpus belonging to the application scenario of the knowledge graph, and the element type prediction results include a first prediction result that the target sentences are knowledge elements that conform to the application scenario of the knowledge graph; inputting a second text containing a mask constructed based on the target sentences whose element type prediction results are the first prediction results into the language model to obtain the relation prediction results output by the language model; the relation prediction results include relation triples constructed based on the target sentences.

9. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is configured to implement the method for constructing relation triples as described in any one of claims 1 to 7, and / or the knowledge enhancement method as described in claim 8, by running a program in the memory.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method for constructing relation triples as described in any one of claims 1 to 7, and / or the knowledge enhancement method as described in claim 8.

Images