Construction method and device for domain-oriented knowledge graph

Abstract

The invention relates to a domain-oriented knowledge graph construction method and device, and the method comprises the steps: obtaining the related text data of a bid invitation file, carrying out the filtering of stop words, and constructing a text data sample set for training; constructing an entity type and a type guide vector by utilizing priori knowledge and a text data sample set, and generating a prompt engineering structure; performing domain self-learning training on the first preset language model to grasp bid invitation file auditing domain knowledge; performing comparative learning training on the second preset language model to identify a related entity; through entity identification and a thinking chain strategy, entity relationship prediction training is performed on the first language model to identify the relationship between entities, through the above mode, the accuracy of entity identification and information extraction is improved, the compliance and the high efficiency of bid invitation auditing work are ensured, and thus a more perfect knowledge graph is constructed.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and in particular to a method and device for constructing a domain-oriented knowledge graph. Background Art

[0002] In today's era of rapid development of informatization, various organizations are facing increasing challenges in information processing and management, especially in the fields of bidding and auditing. With the continuous expansion of business, the number of audit documents has increased dramatically, resulting in a significant reduction in the efficiency of information retrieval and management. This not only makes it difficult for relevant staff to handle documents, but also seriously affects the progress and quality of bidding work. Therefore, an efficient document management tool can not only improve the efficiency of bidding and auditing work, but also reduce the necessary waste of resources.

[0003] In this context, knowledge graphs, as a method of organizing and constructing knowledge based on artificial intelligence technology, express information in a form that is closer to the human cognitive world. They can represent complex associations at a semantic level and effectively display the relationships between documents and the flow of information, providing a way to better manage and understand massive amounts of information.

[0004] The existing knowledge graph construction methods mainly include the following methods:

[0005] Rule-based methods rely on manually defined rules and patterns to extract and construct knowledge graphs. The rules are usually formed through the knowledge of domain experts and can effectively capture the relationships and entities in a specific field. For example, natural language processing technology is used, combined with regular expressions and template matching, to identify entities (such as names, places, organizations, etc.) and their relationships from text.

[0006] Statistical learning methods use machine learning algorithms to automatically extract knowledge from large amounts of data. This method usually involves training with labeled data or identifying potential entities and relationships through techniques such as clustering.

[0007] For example, models such as Support Vector Machine (SVM) and Conditional Random Field (CRF) are used to identify entities in text.

[0008] Deep learning methods extract knowledge from complex structures and sequence data by constructing neural networks, especially graph neural networks (GNNs) and recurrent neural networks (RNNs). This method performs well in processing natural language and graph data and can automatically learn feature representations, thereby improving the effect of knowledge graph construction.

[0009] To sum up, the traditional knowledge graph construction method relies on a large amount of labeled data, which is time-consuming and costly for institutions with relatively limited resources. How to build an efficient knowledge graph based on a small amount of labeled data has become a technical problem that needs to be solved urgently. Summary of the invention

[0010] The purpose of the present invention is to propose a method and device for constructing a domain-oriented knowledge graph, so as to realize the construction of an efficient knowledge graph based on a small amount of annotated data.

[0011] To achieve the above object, according to a first aspect of the present invention, a method for constructing a domain-oriented knowledge graph is provided, the method comprising:

[0012] Acquire text data related to the bidding document, and perform stop word filtering on the text data related to the bidding document to obtain a text data sample set for training;

[0013] Constructing a plurality of entity types according to the text data sample set and prior knowledge, generating type guidance vectors corresponding to the plurality of entity types, and generating a prompt engineering structure according to the type guidance vectors corresponding to the plurality of entity types;

[0014] Performing domain self-learning training on the first preset language model according to the text data sample set, so that the first preset language model can master knowledge related to the bidding document review field;

[0015] Performing comparative learning training on a second preset language model according to the text data sample set and the prompt engineering structure, so that the second preset language model can identify entities related to the bidding document;

[0016] Based on the trained second preset language model, entity recognition is performed on each text data sample in the text data sample set to obtain the head entity and the tail entity of each text data sample, and the thought chain between the head entity and the tail entity is obtained. The head entity, the tail entity and the thought chain are used as model inputs, and entity relationship prediction training is performed on the first preset language model that has undergone domain self-learning training, so that the first preset language model can recognize the relationship between entities.

[0017] Preferably, the plurality of entity types include a plurality of known entity types and one unknown entity type;

[0018] The generating type guidance vectors corresponding to the multiple entity types includes:

[0019] Obtain description information of the plurality of known entity types, and encode the description information of the plurality of known entity types to obtain type guidance vectors of the plurality of known entity types And obtain the type guidance vector w of the unknown entity type according to the type guidance vectors of the multiple known entity types other ; The type indicator vector w of the unknown entity type other A type reference vector with the known entity types Orthogonal;

[0020] The generating a prompt engineering structure according to the type guidance vectors corresponding to the multiple entity types includes:

[0021] The type reference vector w of the unknown entity type other A type reference vector with the known entity types By separating the mark h sep And splice in a specific order to get the prompt engineering structure [w other ,h sep ,w1,h sep ,...,w m ].

[0022] Preferably, the performing domain self-learning training on the first preset language model according to the text data sample set includes:

[0023] Performing multiple rounds of domain self-learning training on the first preset language model based on a random masking strategy and a sequence masking strategy;

[0024] The random masking strategy includes:

[0025] The text data sample is a sequence including a plurality of text units. For each text data sample, a certain number of text units are randomly selected from the text data sample according to a preset masking probability, and the selected text units are masked according to a preset masking rule to obtain a new text data sample, and the new text data sample is input into the first preset language model for domain self-learning, so that the first preset language model can predict the masked text unit;

[0026] The sequence masking strategy includes:

[0027] The text data sample is a sequence containing multiple text units. For each text data sample, the text data sample is divided into multiple continuous subsequences according to a preset mask length, a certain number of subsequences are randomly selected according to a preset masking probability, and the selected subsequences are masked according to a preset masking rule to obtain a new text data sample, and the new text data sample is input into the first preset language model for domain self-learning, so that the first preset language model can predict the masked subsequence.

[0028] Preferably, performing comparative learning training on the second preset language model according to the text data sample set and the prompt engineering structure includes:

[0029] The text data sample is a sequence of multiple text units. An indicator symbol [CLS] is added to the beginning of each text data sample and an indicator symbol [CLS] is added to the end of each text data sample. <s>Used to indicate its boundaries;

[0030] Inputting the text data sample with the indicator symbol added into a pre-trained encoder including an embedding layer to perform feature extraction to obtain a first feature vector;

[0031] The first feature vector is concatenated with the prompt engineering structure and input into a pre-trained encoder that does not include an embedding layer to obtain a second feature vector;

[0032] The entity type of each text unit in the text data sample is predicted according to the second feature vector.

[0033] Preferably, predicting the entity type of each text unit in the text data sample according to the second feature vector comprises:

[0034] Let the second eigenvector be expressed as [h′1,h′2,...,h′ t ,h′ t+1 ,...,h′ t+m ],in,

[0035] [h′1,h′2,...,h′ t ] is the feature vector corresponding to the text unit of the text data sample, [h′ t+1 ,...,h′ t+m ] are feature vectors corresponding to the multiple entity types;

[0036] According to [h′1,h′2,...,h′ t ] and [h′ t+1 ,...,h′ t+m ] Calculate the degree of association between each text unit and each entity type in the text data sample, and determine the prediction probability of each text unit in the text data sample on different entity types based on the degree of association, and select the entity type corresponding to the highest prediction probability as the prediction label of the text unit; through comparative learning training, the second preset language model can recognize the entity type of each text unit in the text data sample.

[0037] Preferably, predicting the entity type of each text unit in the text data sample according to the second feature vector comprises:

[0038] In contrastive learning training, the type guidance vector corresponding to the entity type of each text unit is regarded as a positive example, and the type guidance vectors corresponding to other entity types are regarded as negative examples. The degree of association is determined by calculating the similarity between the feature vector unit corresponding to each text unit and the feature vector unit corresponding to each type guidance vector.

[0039] Preferably, performing comparative learning training on the second preset language model according to the text data sample set and the prompt engineering structure includes:

[0040] A large amount of comparative learning training is performed on the second preset language model using publicly annotated datasets, so that the second preset language model gradually masters the comparative strategy. Then, a small amount of annotated datasets in the field of bidding document review are used to fine-tune the model parameters of the second preset language model to adapt to the new label space after the training stage.

[0041] Preferably, obtaining the thought chain between the head entity and the tail entity includes:

[0042] Mapping the head entity and the tail entity to a type representation space according to the entity types of the head entity and the tail entity to obtain a head entity feature vector and a tail entity feature vector;

[0043] Extract evidence from text data samples according to a preset evidence extraction function to obtain evidence;

[0044] A thought chain between the head entity and the tail entity is generated according to the head entity, the tail entity, the head entity feature vector, the tail entity feature vector and the evidence.

[0045] According to a second aspect of the present invention, there is provided a device for constructing a domain knowledge graph, the device comprising a module for executing the method described in the first aspect of the present invention.

[0046] The present invention proposes a method and device for constructing a domain knowledge graph, which has the following beneficial effects:

[0047] The present invention uses a domain self-learning strategy to enable the pre-trained first language model to understand and master the knowledge in the bidding review field, which is convenient for processing a large amount of unlabeled text. At the same time, a prompt project is constructed with the help of a small amount of labeled data, and the second preset language model is trained by comparative learning using a text data sample set and a prompt project structure to improve entity recognition capabilities. Finally, a specific thinking chain strategy is combined to further improve the accuracy of information extraction, build a more complete knowledge map, and further ensure the compliance and efficiency of bidding review work. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 This is a flowchart of a method for constructing a knowledge graph in an embodiment of the present invention.

[0050] Figure 2 This is a schematic diagram of a method for constructing a knowledge graph in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The detailed description of the accompanying drawings is intended as an illustration of the current embodiments of the present invention and is not intended to represent the only form in which the present invention can be implemented. It should be understood that the same or equivalent functions can be accomplished by different embodiments that are intended to be included in the spirit and scope of the present invention.

[0052] See also Figures 1-2 An embodiment of the present invention provides a method for constructing a domain knowledge graph, comprising the following steps:

[0053] Step S10, obtaining text data related to the bidding document, and performing stop word filtering on the text data related to the bidding document to obtain a text data sample set for training.

[0054] Specifically, the text data related to the bidding documents include but are not limited to bidding documents, bidding documents, bid evaluation reports, etc. There may be useless noise words in the text data related to the bidding documents that affect the named entity recognition work, such as the English abbreviations of some special nouns, various special symbols and other irrelevant words. By analyzing and constructing a stop word list, invalid information in the text data related to the bidding documents is eliminated according to the stop word list during the training and application of the knowledge graph.

[0055] Step S20, constructing multiple entity types according to the text data sample set and prior knowledge, generating type guidance vectors corresponding to the multiple entity types, and generating a prompt engineering structure according to the type guidance vectors corresponding to the multiple entity types.

[0056] Specifically, we can build an entity category set Y = {y1, y2, ..., y n }, providing support for subsequent named entity recognition.

[0057] Step S30, performing domain self-learning training on the first preset language model according to the text data sample set, so that the first preset language model can grasp the knowledge related to the bidding document review field.

[0058] Specifically, the first preset language model can be a pre-trained language model, which may have been trained on a large general corpus, but has not yet been optimized for the specific field of tender document review. The purpose of step S30 is to further train the pre-trained language model by using a text data sample set in the field of tender document review, so that the model can better understand and process text information related to tender document review, thereby providing support for subsequent tasks such as knowledge graph construction.

[0059] Step S40, performing comparative learning training on a second preset language model according to the text data sample set and the prompt engineering structure, so that the second preset language model can identify entities related to the bidding document.

[0060] Specifically, the second preset language model can be a pre-trained language model, which may have been trained on a large general corpus, but has not yet been optimized for the specific field of tender document review. The purpose of step S40 is to further train the pre-trained language model by using a sample set of text data in the field of tender document review, so that the second preset language model learns how to identify relevant entities from the text of the tender document. Contrastive learning training is a machine learning technique that trains a model by comparing the differences between different samples. In contrastive learning, the model is trained to distinguish between positive samples (similar samples) and negative samples (dissimilar samples). In the context of entity recognition, contrastive learning can be used to distinguish between samples with the same entity type and samples that do not have the same entity type.

[0061] Step S50, based on the trained second preset language model, entity recognition is performed on each text data sample in the text data sample set to obtain the head entity and the tail entity of each text data sample, and the thought chain between the head entity and the tail entity is obtained, and the head entity, the tail entity and the thought chain are used as model inputs, and entity relationship prediction training is performed on the first preset language model after domain self-learning training, so that the first preset language model can recognize the relationship between entities.

[0062] Specifically, each text data sample in the text data sample set is processed using the second preset language model trained in step S40. During this process, the model scans the text and identifies the head entity and the tail entity in the text. The head entity and the tail entity usually refer to the key entities mentioned in the text, which may be individuals, organizations, places, events, etc.

[0063] A thought chain refers to the semantic connection or logical path between the head entity and the tail entity. It may contain multiple words or phrases that connect the head entity and the tail entity in the text and describe the relationship between them. After entity recognition, the text is analyzed to determine the thought chain between the head entity and the tail entity.

[0064] After identifying the head entity, the tail entity, and the chain of thoughts between them, this information will be combined as model input data for training the model to identify the relationship between entities. Using the above model input data, the first preset language model that has been trained by domain self-learning in step S30 is further trained. The purpose of this training is entity relationship prediction. In this training stage, the first preset language model will learn how to predict the relationship between entities based on the head entity, the tail entity, and the chain of thoughts between them. Through step S50, the first preset language model is trained to understand and predict the relationship between entities, such as "supplier A is associated with project B" or "contract C stipulates clause D". The ultimate goal is to enable the first preset language model to accurately identify and infer the relationship between entities in the field of bidding document review, which is crucial to building a comprehensive and accurate knowledge graph.

[0065] After training, the first preset language model and the second preset language model in the method of this embodiment constitute a domain knowledge graph model.

[0066] In some embodiments, the plurality of entity types includes a plurality of known entity types and an unknown entity type;

[0067] The generating type guidance vectors corresponding to the multiple entity types includes:

[0068] Obtain description information of the plurality of known entity types, and encode the description information of the plurality of known entity types to obtain type guidance vectors of the plurality of known entity types And obtain the type guidance vector w of the unknown entity type according to the type guidance vectors of the multiple known entity types other ; The type indicator vector w of the unknown entity type other A type reference vector with the known entity types Orthogonal;

[0069] The generating a prompt engineering structure according to the type guidance vectors corresponding to the multiple entity types includes:

[0070] The type reference vector w of the unknown entity type other A type reference vector with the known entity types By separating the mark h sep And splice in a specific order to get the prompt project structure

[0071] [w other ,h sep ,w1,h sep ,...,w m ].

[0072] Specifically, the method of this embodiment obtains the description information of each entity type by constructing a type-description mapping relationship F(x), as shown below:

[0073] desc i =F(x i );

[0074] Through a specific function F(x i ), for each entity type x, from an external knowledge base (such as the first sentence of a Wikipedia page, expert experience) i Get the description sentence desc containing its key semantic features i . Through the language model learned in the field, the description sentence desc i Encode to get the type indicator vector w i ,in LLM stands for Large Language Model.

[0075] The unknown entity type (OTHER) does not belong to the defined entity category range. For example, the defined known entity types include {organization, location}, which become {organization, location, OTHER} after processing. If the person "C" is recognized, it will be classified as OTHER; if the organization "A" is recognized, it will be classified as organization. The type guidance vector w for the special unknown entity type (OTHER) is other The method of this embodiment randomly initializes a type guidance vector matrix T with other types and uses an orthogonal triangular decomposition operation to make the type guidance vector w of the unknown entity type other The vectors of all known types are kept at a distinct distance in vector space, using the following formula:

[0076]

[0077] Different from other methods that rely on predefined templates for prompting engineering effects, the method of this embodiment generates a type guide vector w of the OTHER type. other , special separators (such as h sep ) and the type indicator vector w for each entity type i They are spliced ​​together in a specific order to form a template-independent prompt engineering structure. This process integrates the entity category and its description information to prepare for the subsequent connection and processing with the input sentence. The formula is as follows:

[0078] prompt=[w other ,h sep ,w1,h sep ,...,w m ].

[0079] In some embodiments, the performing domain self-learning training on the first preset language model according to the text data sample set includes:

[0080] Performing multiple rounds of domain self-learning training on the first preset language model based on a random masking strategy and a sequence masking strategy;

[0081] The random masking strategy includes:

[0082] The text data sample is a sequence including a plurality of text units. For each text data sample, a certain number of text units are randomly selected from the text data sample according to a preset masking probability, and the selected text units are masked according to a preset masking rule to obtain a new text data sample, and the new text data sample is input into the first preset language model for domain self-learning, so that the first preset language model can predict the masked text unit;

[0083] Specifically, when processing text data related to tender document review, the core of the random masking strategy adopted by the method of this embodiment is to randomly mask some words in the input text with a certain probability, forcing the first preset language model to predict these masked parts based on contextual information, so as to better learn the semantic and grammatical features of the text, and then improve the understanding and mastery of the knowledge in the field of tender document review.

[0084] The specific operations are as follows:

[0085] Assume that the input text is a sequence of words X = (x1, x2, ..., x n ), x i represents the i-th word, to mask the probability p(x i ) is masked in the text X in the sequence to obtain X'=(x'1,x'2,...,x' n );

[0086] in:

[0087]

[0088] It has been verified that p(x i ) is 15%, the effect is best, that is, 15% of the words are randomly selected from the input text for masking. Among the selected 15% of the text to be masked, there is an 80% probability of replacing it with [MASK], a 10% probability of replacing it with a random word, and the remaining 10% probability of keeping the original word unchanged.

[0089] The sequence masking strategy includes:

[0090] The text data sample is a sequence containing multiple text units. For each text data sample, the text data sample is divided into multiple continuous subsequences according to a preset mask length, a certain number of subsequences are randomly selected according to a preset masking probability, and the selected subsequences are masked according to a preset masking rule to obtain a new text data sample, and the new text data sample is input into the first preset language model for domain self-learning, so that the first preset language model can predict the masked subsequence.

[0091] Specifically, the sequence masking strategy aims to enhance the model's ability to learn contextual information by breaking down the sequence of the input text, and while improving computational efficiency, enable the first preset language model to better understand the domain-specific knowledge structure. In the sequence masking strategy, the method of this embodiment masks a continuous subsequence in the input text, while ensuring that the first preset language model can perform effective contextual reasoning through the broken information, thereby improving the adaptability to the field of tender document review.

[0092] The specific operations are as follows:

[0093] Assume that the input text is a sequence of words X = (x1, x2, ..., x n ), x i represents the i-th word (text unit), dividing the text into multiple continuous subsequences, each of which contains k consecutive words, such as X j =(x j ,x j+1 ,...,x j+k-1 ). Randomly select several subsequences and mask them with special markers. The masked text can be represented as X′=(x1,x2,...,x j-1 , <x>,x j+k ,...,x n ),in, <x>is the masked continuous subsequence, and the number of masked regions is Where p(x i ) is the masking probability, n is the length of the input text, and k is the mask length.

[0094] It has been verified that p(x i ) is 15%, the effect is best, that is, 15% of subsequences are randomly selected from the input text for masking. By comparing the performance under different mask ranges, the best effect is achieved when the mask length is 3.

[0095] Specifically, when the first preset language model can accurately infer the content of the masked part according to the context, the model training ends. Based on the above two masking strategies, the method of this embodiment formulates a multi-round domain learning strategy for the language model, so that the first preset language model can better adapt to and master the knowledge of this specific field by inferring the masked part according to the context.

[0096] In some embodiments, performing comparative learning training on the second preset language model according to the text data sample set and the prompt engineering structure includes:

[0097] The text data sample is a sequence of multiple text units. An indicator symbol [CLS] is added to the beginning of each text data sample and an indicator symbol [CLS] is added to the end of each text data sample. <s>Used to indicate its boundaries;

[0098] Inputting the text data sample with the indicator symbol added into a pre-trained encoder including an embedding layer to perform feature extraction to obtain a first feature vector;

[0099] The first feature vector is concatenated with the prompt engineering structure and input into a pre-trained encoder that does not include an embedding layer to obtain a second feature vector;

[0100] The entity type of each text unit in the text data sample is predicted according to the second feature vector.

[0101] Specifically, the indicator [CLS] indicates the start of a sequence, and the indicator <s>Indicates the end of the sequence. The text data sample with the indicator symbol added is input into a pre-trained encoder. The encoder contains an embedding layer. The embedding layer converts the text unit (such as a word) into a vector representation of a fixed length. The encoder performs feature extraction to obtain a first feature vector. The formula is as follows:

[0102] [h CLS ,h1,...,h n ,h s ]=LLM([CLS],word1,...,word n , <s>);

[0103] is the feature representation of the input sentence, h s For special marks <s>The characteristic representation of h CLS Represents the feature representation that integrates the context information of the input sentence;

[0104] The first eigenvector [h CLS ,h1,...,h n ,h s ] and prompt engineering structure [w other ,h sep ,w1,h sep ,...,w m ] to splice, the spliced ​​vector [h1|h2|...|h t |w1|w2|...|w m ] is input into the encoder of a pre-trained large language model. The encoder does not contain an embedding layer. The role of this encoder is to further process the concatenated feature vector. The formula is as follows:

[0105] [h′1,h′2,...,h′ t ,h′ t+1 ,...,h′ t+m ]=LLM enc ([h1|h2|...|h t |w1|w2|...|w m ])

[0106] LLM enc represents the second feature vector [h′1,h′2,...,h′ obtained by the above steps using the LLM encoder without the embedding layer t ,h′ t+1 ,...,h′ t+m ] is used to predict the entity type of each text unit in the text data sample. Entity type prediction refers to identifying which entity category each unit in the text belongs to, such as a person's name, an organization name, a place name, etc., t=n+2.

[0107] In some embodiments, predicting the entity type of each text unit in the text data sample according to the second feature vector includes:

[0108] The second eigenvector is denoted as [h′1,h′2,...,h′ t ,h′ t+1 ,...,h′ t+m ], [h′1,h′2,...,h′ t ] is the feature vector corresponding to the text unit of the text data sample, [h′ t+1 ,...,h′ t+m ] are feature vectors corresponding to the multiple entity types;

[0109] According to [h′1,h′2,...,h′ t ] and [h′ t+1 ,...,h′ t+m ] Calculate the degree of association between each text unit and each entity type in the text data sample, and determine the prediction probability of each text unit in the text data sample on different entity types based on the degree of association, and select the entity type corresponding to the highest prediction probability as the prediction label of the text unit, so that the second preset language model can recognize the entity type of each text unit in the text data sample through comparative learning training.

[0110] Specifically, for each text unit in the text data sample, the feature vector h′ i (i=1,…,t), calculate its feature vector h′ with each entity type t+j (j=1,…,m), and based on the calculated correlation degree, determine the predicted probability of each text unit in the text data sample on different entity types. For each text unit, select the entity type with the highest prediction probability as the predicted label of the text unit. The goal of contrastive learning is to optimize the model by comparing positive samples (correct entity types) and negative samples (wrong entity types). During the training process, the model will try to increase the similarity between positive samples and reduce the similarity between negative samples. In this way, the model learns how to better identify the entity types of each text unit in the text data sample.

[0111] In some embodiments, predicting the entity type of each text unit in the text data sample according to the second feature vector includes:

[0112] In contrastive learning training, the type guidance vector corresponding to the entity type of each text unit is regarded as a positive example, and the type guidance vectors corresponding to other entity types are regarded as negative examples. The degree of association is determined by calculating the similarity between the feature vector unit corresponding to each text unit and the feature vector unit corresponding to each type guidance vector.

[0113] Specifically, the feature vector h′ of each text unit i (i=1,…,t), calculate its feature vector h′ with each entity type t+j The degree of association between (j=1,…,m) can be determined by calculating their similarity. In this embodiment, the similarity is calculated using the normalized cosine similarity function d(.) as a criterion. Based on this similarity, a specific probability prediction function is used to calculate the predicted probability of each text unit belonging to each entity type. The formula is as follows:

[0114]

[0115] Finally, by comparing the predicted probabilities of each tag on different entity categories, the predicted probability of each text unit x is determined. i The predicted label The formula is as follows:

[0116]

[0117] Specifically, when the second preset model can accurately identify the type of entity, the model training ends.

[0118] In some embodiments, performing comparative learning training on the second preset language model according to the text data sample set and the prompt engineering structure includes:

[0119] A large amount of comparative learning training is performed on the second preset language model using publicly annotated datasets, so that the second preset language model gradually masters the comparative strategy. Then, a small amount of annotated datasets in the field of bidding document review are used to fine-tune the model parameters of the second preset language model to adapt to the new label space after the training stage.

[0120] Specifically, the second preset language model of the method of this embodiment first performs a large amount of such comparative training on a public annotated dataset, so that the second preset language model gradually masters the comparison strategy and can keenly capture the similarities and differences between the text units and different types of guidance vectors in the input text sequence. After the model is trained on the public annotated dataset and masters the comparison strategy, it is applied to the target dataset and fine-tuned with a small amount of annotated data in the target field to adapt to the new label space after the training stage. The second preset language model generates a text unit encoding representation for the target text, compares it with the type guidance vector, evaluates the correlation between word embedding and entity type according to the learned strategy, and calculates the probability of the tag type with a probability prediction function. In the process, it gradually adapts to the characteristics of the target dataset, thereby accurately identifying named entities, providing strong support for the construction of the bidding review knowledge graph and overcoming the adaptability defects of traditional methods.

[0121] When applied to the actual task of named entity recognition in bidding review texts, the second preset language model efficiently calculates the similarity between the input sentence text unit and the type guidance vector based on the learned comparison strategy to evaluate the degree of association between word embeddings and each entity type, and then uses a specific probability prediction function to calculate the predicted probability that each text unit belongs to each entity type, thereby accurately completing the named entity recognition task.

[0122] In some embodiments, obtaining the thought chain between the head entity and the tail entity includes:

[0123] Mapping the head entity and the tail entity to a type representation space according to the entity types of the head entity and the tail entity to obtain a head entity feature vector and a tail entity feature vector;

[0124] Extract evidence from text data samples according to a preset evidence extraction function to obtain evidence;

[0125] A thought chain between the head entity and the tail entity is generated according to the head entity, the tail entity, the head entity feature vector, the tail entity feature vector and the evidence.

[0126] Specifically, the method of this embodiment is based on a small amount of annotated data, and establishes a thinking chain for the head entity and the tail entity in the input text, assuming that the named entity recognition task has been completed. head , the tail entity is e tail , through domain knowledge and semantic understanding ability of pre-trained models, combined with entity categories, e head Mapped to the category representation space, we get the head entity feature vector C(e head ), and similarly, we get the tail entity feature vector C(e tail ), this process is done through the semantic mapping function f in the pre-trained model concept To achieve, that is:

[0127] C(e head )=f concept (e head ), C(e tail )=f concept (e tail );

[0128] Next, we accurately locate the key evidence fragments in the text that can effectively support the relationship between the head entity and the tail entity. Let the text be T and use the evidence extraction function f evidence To obtain evidence, namely:

[0129] E=f evidence (T,e head ,e tail ,);

[0130] Where E represents the extracted evidence set. Finally, the head entity, tail entity, concept-level representation and evidence are integrated to construct a logically coherent and semantically clear thinking chain structure. Let the constructed thinking chain be COT, then:

[0131] COT = g(C(e head ),C(e tail ),e head ,e tail ,E);

[0132] Among them, g is the thinking chain construction function, which is responsible for combining various elements according to a specific logical order and grammatical rules to form a thinking chain expression such as "[head entity (concept level)] has a [relationship] with [tail entity (concept level)] through [evidence]".

[0133] It should be noted that the evidence extraction function f evidence The input of the function is the text sentence and the obtained head and tail entities, and the output is the content in the text that is related to these two entities. For example, if the text mentions "Company A launched mobile phone B", the input of the function is the sentence and "Company A" and "Mobile phone B", and the output is the key verb "launch" as evidence. The thinking chain constructed based on this evidence is "'Company A (enterprise)' proves that it has an 'affiliation' with 'Mobile phone B' through the evidence of 'launching'."

[0134] The method of this embodiment combines the constructed thought chain COT with the text pair (e head ,e tail ) are inputted together into the first preset language model after domain self-learning and training, and the first preset language model is assumed to be M language , the model processes the input and obtains the relationship expressed as follows:

[0135] R=M language (COT,e head ,e tail );

[0136] Then, the relationship representation R is compared with the relationship set in the knowledge graph for similarity. The method in this embodiment selects the cosine similarity algorithm based on the vector space model as the similarity measurement function, calculates the similarity score between R and each relationship, and finally, selects the relationship with the highest similarity score as the prediction result. The model is trained based on the above principles. When the first preset language model can accurately identify the relationship between entities, the model training is completed.

[0137] In summary, the method of this embodiment has the following advantages:

[0138] (1) The domain self-learning strategy designed by the method of this embodiment combines the random masking strategy and the sequence masking strategy. The random masking strategy can fully mine the contextual information of the text, so that the model can deeply understand the semantics and grammatical structure of the text in the process of predicting the masked words, thereby quickly adapting to the language expression habits and knowledge characteristics in the field of bidding document review. The sequence masking strategy improves the model's association and reasoning ability for long-distance contextual information by destroying and reconstructing the text sequence, and effectively improves the accuracy and efficiency of the model when processing complex text relationships. Compared with traditional single learning strategies or general learning methods that lack specificity, this multi-strategy collaborative domain learning strategy can enable the large language model to master the bidding review domain knowledge more accurately and efficiently with a small amount of annotated data, reduce dependence on a large amount of annotated data, reduce resource consumption and time costs, and provide a high-quality information extraction foundation for building a knowledge graph.

[0139] (2) The type guidance vector in the method of this embodiment provides rich semantic information basis for entity recognition, so that the model can more accurately understand the connotation and difference of different entity types. For the special OTHER type, the orthogonal triangular decomposition operation is used to keep it at a clear distance from other type guidance vectors in the vector space, which effectively avoids recognition errors caused by type ambiguity in the process of multi-type entity recognition and improves the accuracy and specificity of entity classification.

[0140] (3) The method of this embodiment adopts a template-independent prompt construction method, which splices the OTHER type representation, special separation mark and type guidance vector of each entity type in a specific order, and integrates the entity category and its description information. Compared with the traditional prompt engineering that relies on pre-defined templates, this method is more flexible and adaptable, and can better cope with the situation of diverse entity types and complex expressions in the tender document review text, significantly improves the stability and accuracy of named entity recognition in different text contexts, and provides more accurate and complete entity information for knowledge graph construction.

[0141] (4) The method of this embodiment introduces a thinking chain strategy in the relationship extraction link. Based on a small amount of annotated data, with the help of domain knowledge and the semantic understanding ability of the pre-trained model, a logically coherent and semantically clear thinking chain structure is established for the head entity and the tail entity. By integrating the head entity, the tail entity, the concept-level representation and the key evidence fragments, the model is no longer limited to the surface text information when processing entity relationships, but deeply explores the logical relationships and semantic associations behind the text. This method can effectively improve the accuracy and reliability of the model's relationship judgment when processing complex entity relationships in the bidding review text, and avoid relationship misjudgment caused by information fragmentation or semantic understanding deviation. Compared with traditional relationship extraction methods, the thinking chain strategy can better utilize the implicit knowledge and long-distance semantic dependencies in the text, so that the extracted relationships are more in line with the actual business logic, thereby constructing a more accurate and practical bidding review knowledge graph, further ensuring the compliance and efficiency of the bidding review work, and improving the overall work quality and scientific nature of decision-making.

[0142] Another embodiment of the present invention also provides a device for constructing a domain-oriented knowledge graph, including a module for executing the knowledge graph construction method described in the above embodiment. This device can be a hardware device, a software device, or a combination of hardware and software.

[0143] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.< / s> < / s> < / s> < / s> < / x> < / x> < / s>

Claims

1. A method for constructing a domain knowledge graph, characterized in that: include: Acquire text data related to the bidding document, and perform stop word filtering on the text data related to the bidding document to obtain a text data sample set for training; Constructing a plurality of entity types according to the text data sample set and prior knowledge, generating type guidance vectors corresponding to the plurality of entity types, and generating a prompt engineering structure according to the type guidance vectors corresponding to the plurality of entity types; Performing domain self-learning training on the first preset language model according to the text data sample set, so that the first preset language model can master knowledge related to the bidding document review field; Performing comparative learning training on a second preset language model according to the text data sample set and the prompt engineering structure, so that the second preset language model can identify entities related to the bidding document; Based on the trained second preset language model, entity recognition is performed on each text data sample in the text data sample set to obtain the head entity and the tail entity of each text data sample, and the thought chain between the head entity and the tail entity is obtained. The head entity, the tail entity and the thought chain are used as model inputs, and entity relationship prediction training is performed on the first preset language model that has undergone domain self-learning training, so that the first preset language model can recognize the relationship between entities.

2. The method according to claim 1, characterized in that The multiple entity types include multiple known entity types and one unknown entity type; The generating type guidance vectors corresponding to the multiple entity types includes: Obtain description information of the plurality of known entity types, and encode the description information of the plurality of known entity types to obtain type guidance vectors of the plurality of known entity types And obtain the type guidance vector w of the unknown entity type according to the type guidance vectors of the multiple known entity types other ; The type indicator vector w of the unknown entity type other A type reference vector with the known entity types Orthogonal; The generating a prompt engineering structure according to the type guidance vectors corresponding to the multiple entity types includes: The type reference vector w of the unknown entity type other A type reference vector with the known entity types By separating the mark h sep And splice in a specific order to get the prompt project structure [w other ,h sep ,w1,h sep ,...,w m ]。 3. The method according to claim 2, characterized in that The performing domain self-learning training on the first preset language model according to the text data sample set includes: Performing multiple rounds of domain self-learning training on the first preset language model based on a random masking strategy and a sequence masking strategy; The random masking strategy includes: The text data sample is a sequence including a plurality of text units. For each text data sample, a certain number of text units are randomly selected from the text data sample according to a preset masking probability, and the selected text units are masked according to a preset masking rule to obtain a new text data sample, and the new text data sample is input into the first preset language model for domain self-learning, so that the first preset language model can predict the masked text unit; The sequence masking strategy includes: The text data sample is a sequence containing multiple text units. For each text data sample, the text data sample is divided into multiple continuous subsequences according to a preset mask length, a certain number of subsequences are randomly selected according to a preset masking probability, and the selected subsequences are masked according to a preset masking rule to obtain a new text data sample, and the new text data sample is input into the first preset language model for domain self-learning, so that the first preset language model can predict the masked subsequence.

4. The method according to claim 3, characterized in that The performing comparative learning training on the second preset language model according to the text data sample set and the prompt engineering structure includes: The text data sample is a sequence of multiple text units. An indicator symbol [CLS] is added to the beginning of each text data sample and an indicator symbol [CLS] is added to the end of each text data sample. <s> Used to indicate its boundaries;< / s> <s> Inputting the text data sample with the indicator symbol added into a pre-trained encoder including an embedding layer to perform feature extraction to obtain a first feature vector; The first feature vector is concatenated with the prompt engineering structure and input into a pre-trained encoder that does not include an embedding layer to obtain a second feature vector; The entity type of each text unit in the text data sample is predicted according to the second feature vector.

5. The method according to claim 4, characterized in that The step of predicting the entity type of each text unit in the text data sample according to the second feature vector comprises: Let the second eigenvector be expressed as [h ′ 1,h ′ 2,...,h ′ t ,h ′ t+1 ,...,h ′ t+m ],in, [h ′ 1,h ′ 2,...,h ′ t ] is the feature vector corresponding to the text unit of the text data sample, [h ′ t+1 ,...,h ′ t+m ] are feature vectors corresponding to the multiple entity types; According to [h ′ 1,h ′ 2,...,h ′ t ] and [h ′ t+1 ,...,h ′ t+m ] Calculate the degree of association between each text unit and each entity type in the text data sample, and determine the prediction probability of each text unit in the text data sample on different entity types based on the degree of association, and select the entity type corresponding to the highest prediction probability as the prediction label of the text unit; through comparative learning training, the second preset language model can recognize the entity type of each text unit in the text data sample.

6. The method according to claim 5, characterized in that The step of predicting the entity type of each text unit in the text data sample according to the second feature vector comprises: In contrastive learning training, the type guidance vector corresponding to the entity type of each text unit is regarded as a positive example, and the type guidance vectors corresponding to other entity types are regarded as negative examples. The degree of association is determined by calculating the similarity between the feature vector unit corresponding to each text unit and the feature vector unit corresponding to each type guidance vector.

7. The method according to any one of claims 3 to 6, characterized in that: The performing comparative learning training on the second preset language model according to the text data sample set and the prompt engineering structure includes: A large amount of comparative learning training is performed on the second preset language model using publicly annotated datasets, so that the second preset language model gradually masters the comparative strategy. Then, a small amount of annotated datasets in the field of bidding document review are used to fine-tune the model parameters of the second preset language model to adapt to the new label space after the training stage.

8. The method according to claim 1, characterized in that The step of obtaining the thought chain between the head entity and the tail entity includes: Mapping the head entity and the tail entity to a type representation space according to the entity types of the head entity and the tail entity to obtain a head entity feature vector and a tail entity feature vector; Extract evidence from text data samples according to a preset evidence extraction function to obtain evidence; A thought chain between the head entity and the tail entity is generated according to the head entity, the tail entity, the head entity feature vector, the tail entity feature vector and the evidence.

9. A device for constructing a domain knowledge graph, characterized in that: The method comprises a module for executing the method according to any one of claims 1 to 8. < / s>

Images