Knowledge base construction method and device and computer equipment

Abstract

The invention relates to a knowledge base construction method and device and computer equipment. The method comprises the following steps: in response to a construction request for a domain knowledge base of a target domain, screening a new corpus set of the target domain through a text classification model; the text classification model is obtained by training in advance based on a positive sample formed by a historical corpus set of a target field and a negative sample formed by corpus sets of other fields matched with the category of the target field; performing relation extraction on the basis of the screened new corpus set to obtain a relation between a plurality of entities, and performing error information correction on the relation between the plurality of entities; and constructing a domain knowledge base of the target domain according to the corrected multiple entities and the relationship among the entities. By adopting the method, the quality of the domain knowledge base can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a knowledge base construction method, apparatus and computer equipment. Background Art

[0002] A knowledge base, also known as an intelligent database or an artificial intelligence database, is a structured, easy-to-operate, easy-to-use and fully organized knowledge cluster built in knowledge engineering to solve problems in a certain field.

[0003] The difference between a domain knowledge base and a general knowledge base is that a domain knowledge base is targeted at a specific vertical field and is highly professional and accurate. In related technologies, when constructing a domain knowledge base, the corpus of a specific field is usually collected and cleaned first, and then the corpus is manually annotated according to the domain ontology to obtain the entities in the corpus and the relationship between entities. Finally, the annotation results are incorporated into the knowledge base to obtain a domain knowledge base for a specific field.

[0004] However, the method of constructing a domain knowledge base in related technologies has a technical problem of low quality of the domain knowledge base. Summary of the invention

[0005] Based on this, it is necessary to provide a knowledge base construction method, device and computer equipment to improve the quality of the domain knowledge base in response to the above technical problems.

[0006] In a first aspect, an embodiment of the present application provides a knowledge base construction method, comprising:

[0007] In response to a request to build a domain knowledge base of a target domain, a new corpus of the target domain is screened by a text classification model; the text classification model is pre-trained based on positive samples consisting of a historical corpus of the target domain and negative samples consisting of corpuses of other domains matching the categories of the target domain;

[0008] Relationship extraction is performed based on the screened new corpus to obtain multiple entities and the relationships between them, and the wrong information of the relationships between them is corrected;

[0009] According to the revised multiple entities and the relationships between the entities, a domain knowledge base of the target domain is constructed.

[0010] In one embodiment, the method further comprises:

[0011] Determine, according to the target domain data of the target domain and the candidate domain data of the plurality of candidate domains, the inter-class distance between the target domain data and the candidate domain data;

[0012] According to the distance between categories, other fields matching the target field category are determined from multiple candidate fields, and the corpus of other fields is determined as negative samples.

[0013] In one embodiment, the method further comprises:

[0014] When there is a history domain knowledge base in the target domain, the historical corpus in the history domain knowledge base is determined as a positive sample;

[0015] When there is no historical domain knowledge base in the target domain, multiple historical corpus texts of the target domain are obtained from multiple third-party systems, and each historical corpus text is determined as a positive sample.

[0016] In one embodiment, relationship extraction is performed based on the screened new corpus to obtain multiple entities and the relationships between the entities, including:

[0017] The screened new corpus is input into a pre-trained relation extraction model to obtain multiple entities and the relations between entities output by the relation extraction model; the relation extraction model is obtained by training an initial relation extraction model based on multiple text data and annotation information of each text data in the historical corpus.

[0018] In one embodiment, the training process of the relation extraction model includes:

[0019] Input each text data into the initial relation extraction model to obtain multiple predicted entities output by the initial relation extraction model and predicted relations between the predicted entities;

[0020] Determine the relationship extraction loss value of the initial relationship extraction model according to each predicted entity, each predicted relationship, each annotation information and a preset loss function;

[0021] Based on the relationship extraction loss value, the model parameters of the initial relationship extraction model are adjusted until the training is completed to obtain the relationship extraction model.

[0022] In one embodiment, correcting incorrect information of multiple entities and relationships between the entities includes:

[0023] Obtain multiple candidate entities pre-annotated by multiple users on the filtered new corpus and the relationship between the candidate entities;

[0024] Determining a labeling consistency coefficient between users based on multiple candidate entities and the relationship between the candidate entities;

[0025] Based on the labeling consistency coefficient, each user is instructed to correct the erroneous information in the multiple entities and the relationships between the entities, so as to obtain the corrected multiple entities and the relationships between the entities.

[0026] In one embodiment, determining the tagging consistency coefficient of each user according to multiple candidate entities and the relationship between the candidate entities includes:

[0027] According to each candidate entity, the relationship between each candidate entity, the number of users of each user, and the amount of data in the screened new corpus, the actual consistency ratio and the random consistency ratio are determined;

[0028] The annotation consistency coefficient is determined based on the actual consistency ratio and the random consistency ratio.

[0029] In one embodiment, a domain knowledge base of a target domain is constructed based on the modified multiple entities and the relationships between the entities, including:

[0030] According to the corrected multiple entities and the relationships between the entities, multiple entity relationship triples are constructed;

[0031] Each entity relationship triple is stored in the database to obtain the domain knowledge base of the target domain.

[0032] In a second aspect, the present application also provides a knowledge base construction device, including:

[0033] A corpus screening module is used to respond to a request for building a domain knowledge base of a target domain and screen a new corpus of the target domain through a text classification model; the text classification model is pre-trained based on positive samples consisting of a historical corpus of the target domain and negative samples consisting of corpuses of other domains matching the target domain category;

[0034] The relation extraction module is used to extract relations based on the screened new corpus, obtain multiple entities and the relations between them, and correct the wrong information between them;

[0035] The knowledge base construction module is used to construct a domain knowledge base of the target domain based on the corrected multiple entities and the relationships between the entities.

[0036] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method in any one of the embodiments of the first aspect are implemented.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in any one of the embodiments of the first aspect above.

[0038] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method in any one of the embodiments of the first aspect above.

[0039] The knowledge base construction method, apparatus and computer device provided in the embodiments of the present application respond to a request for constructing a domain knowledge base of a target domain, screen a new corpus of the target domain through a text classification model, and then extract relationships based on the screened new corpus to obtain multiple entities and relationships between each entity, and correct erroneous information between the multiple entities and the relationships between each entity, and then construct a domain knowledge base of the target domain based on the corrected multiple entities and the relationships between each entity, wherein the text classification model is pre-trained based on positive samples consisting of a historical corpus of the target domain and negative samples consisting of corpora of other domains matching the target domain category. In this method, when constructing a domain knowledge base of a target domain, after obtaining a new corpus of the target domain required for constructing the domain knowledge base, a text classification model is first used to screen the new corpus of the target domain. The positive samples during training of the text classification model are the historical corpus of the target domain, while the negative samples are the corpus of other domains that match the target domain category. In this way, the trained text classification model can more accurately identify corpus texts that do not belong to the target domain from the new corpus of the target domain, thereby improving the data quality of the data required for constructing the domain knowledge base, thereby improving the quality of the constructed domain knowledge base; moreover, after performing relationship extraction on the screened new corpus, the erroneous information of the extracted multiple entities and the relationships between the entities is corrected, thereby further improving the data quality of constructing the domain knowledge base, thereby further improving the quality of the domain knowledge base, which is equivalent to making double quality assurance in terms of the data required for construction when constructing the domain knowledge base, thereby making the quality of the constructed domain knowledge base higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 An application environment diagram of a knowledge base construction method in one embodiment;

[0042] Figure 2 A schematic diagram of a process of building a knowledge base in one embodiment;

[0043] Figure 3A schematic diagram of a process for obtaining negative samples in one embodiment;

[0044] Figure 4 A schematic diagram of a process for obtaining positive samples in one embodiment;

[0045] Figure 5 A schematic diagram of a process for training a relationship extraction model in one embodiment;

[0046] Figure 6 A schematic diagram of a flow chart of correcting error information of multiple entities and relationships between the entities in one embodiment;

[0047] Figure 7 A schematic diagram of a process for determining a label consistency coefficient in one embodiment;

[0048] Figure 8 A schematic diagram of a process for building a domain knowledge base in one embodiment;

[0049] Fig. 9 A schematic diagram of a process of building a knowledge base in another embodiment;

[0050] Fig.10 A schematic diagram of a framework for constructing an iterative domain knowledge base in one embodiment;

[0051] Fig.11 A schematic diagram of the structure of a knowledge base construction device in one embodiment;

[0052] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] The technical background of the embodiments of the present application is first described below.

[0055] The difference between a domain knowledge base and a general knowledge base is that a domain knowledge base is targeted at a specific vertical field and is highly professional and accurate. Related technologies include domain ontology design, domain corpus collection and cleaning, data set annotation, training entity and relationship extraction algorithms, and knowledge fusion. In the domain ontology design phase, the definitions of entity concepts in the domain and their relationships are sorted out based on the knowledge of domain experts. After collecting domain corpus, the corpus is manually annotated according to the domain ontology. The constructed data set is used to train the relationship extraction algorithm, and then the trained algorithm is used to automatically annotate the new corpus, and the annotation results are incorporated into the knowledge base using knowledge fusion technology. There are several problems with the technical solutions in related technologies: (1) They are not iterative: In the process of dataset construction, model training, and information extraction, each link is pipelined and does not constitute an iterative loop; (2) The labor cost of dataset annotation is high: the training of relationship extraction algorithms relies on large-scale supervised datasets in the field. Annotating datasets in specific fields requires annotators to have a certain level of expertise in the field. The annotation process is difficult and time-consuming and labor-intensive; (3) Corpus cleaning is difficult: Common data cleaning operations such as illegal character removal, format alignment, and deduplication can only solve the problem of data format specification, but cannot screen out corpus in a specific vertical field in a targeted manner.

[0056] Based on this, the present application provides a knowledge base construction method. When constructing a domain knowledge base for a target domain, after obtaining a new corpus of the target domain required for constructing the domain knowledge base, a text classification model is first used to screen the new corpus of the target domain. The positive sample during training of the text classification model is a historical corpus of the target domain, while the negative sample is a corpus of other fields that matches the target domain category. In this way, the trained text classification model can more accurately identify corpus texts that do not belong to the target domain from the new corpus of the target domain, thereby improving the data quality of the data required for constructing the domain knowledge base, thereby improving the quality of the constructed domain knowledge base; moreover, after performing relationship extraction on the screened new corpus, the erroneous information of the extracted multiple entities and the relationships between the entities is corrected, thereby further improving the data quality of constructing the domain knowledge base, thereby further improving the quality of the domain knowledge base, which is equivalent to making double quality assurance in terms of the data required for construction when constructing the domain knowledge base, thereby making the quality of the constructed domain knowledge base higher.

[0057] It should be noted that the beneficial effects or technical problems solved by the embodiments of the present application are not limited to this one, but may also include other implicit or related problems. For details, please refer to the description of the following embodiments.

[0058] The application environment of the knowledge base construction method provided in the embodiment of the present application is described below. The knowledge base construction method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Optionally, the terminal 102 is used by the user to trigger a request to build a domain knowledge base for the target field. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0059] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0060] In an exemplary embodiment, Figure 2 As shown, a knowledge base construction method is provided, which is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including the following steps 201 to 203. Among them:

[0061] S201, in response to a request to build a domain knowledge base of a target domain, a new corpus of the target domain is screened through a text classification model; the text classification model is pre-trained based on positive samples consisting of a historical corpus of the target domain and negative samples consisting of corpora of other domains matching the target domain category.

[0062] The target domain can be any specific vertical field, including but not limited to finance, medicine, history, agriculture, energy, space science, electronic information, education, law, etc. The domain knowledge base of the target domain refers to a special database composed of the corpus of the target domain, which is usually used to store, manage and query knowledge resources.

[0063] The construction request of the domain knowledge base of the target domain refers to the request that is triggered when the domain knowledge base of the target domain needs to be constructed in the knowledge base construction application of the server. Naturally, after the server receives the construction request, it needs to construct the domain knowledge base of the target domain. Exemplarily, the request is triggered by the user through the interface in the knowledge base construction application; wherein, the triggering method includes but is not limited to external input device clicks, voice control, touch screen triggering, etc., and the embodiment of the present application does not limit the triggering method.

[0064] The text classification model refers to a neural network model for text classification of a new corpus in the target field, and is mainly used to filter out corpus texts that do not belong to the target field. Among them, the new corpus in the target field can be obtained by calling the search engine interface based on the keyword set of the target field. The text classification model is pre-trained based on positive samples consisting of a historical corpus in the target field and negative samples consisting of a corpus in other fields that matches the category of the target field. Among them, the positive sample is a historical corpus in the target field. In an implementable manner, the historical corpus is obtained from the historical corpus of the target field. The negative sample is a corpus in other fields that matches the category of the target field, wherein the other fields that match the category of the target field refer to other fields that are similar to the field type of the target field, that is, other fields that have a certain association with the target field and the application scenario. In an implementable manner, one (or more) other fields that are similar to the field type of the target field are determined from multiple specific vertical fields, and the historical corpus is obtained from the historical corpus of the one (or more) other fields as negative samples for training the text classification model.

[0065] In one embodiment, the positive samples and negative samples obtained above are input into a preset initial text classification model to obtain positive sample prediction results and negative sample prediction results, and then the classification loss value of the initial text classification model is determined based on the positive sample prediction results and the negative sample prediction results in combination with a preset loss function, and the model parameters of the initial text classification model are adjusted based on the classification loss value until the training is completed to obtain a text classification model. Among them, the initial text classification model can use BERT (Bidirectional Encoder Representations from Transformers, bidirectional transformer model).

[0066] After the text classification model is trained based on the above method, the new corpus in the target field is input into the text classification model, and the text classification model screens the new corpus in the target field to obtain a screened new corpus.

[0067] S202, performing relationship extraction based on the screened new corpus to obtain multiple entities and relationships between the entities, and correcting erroneous information between the multiple entities and the relationships between the entities.

[0068] Among them, relationship extraction means that given a piece of unstructured text, it is necessary to identify the named entities in the text and extract the relationships between the named entities.

[0069] In one embodiment, the method for extracting relationships from the filtered new corpus may be to display the filtered new corpus to the user through an interface, and the user may extract relationships from the filtered new corpus. The relationship extraction here may be that the user directly performs entity labeling and relationship labeling on the filtered new corpus, and then the server extracts multiple entities and the relationships between the entities from the user-labeled information.

[0070] In another embodiment, the method of performing relation extraction on the filtered new corpus may be to perform relation extraction on the filtered new corpus through a relation extraction algorithm. The relation extraction algorithm may be a rule-based method, which completes relation extraction by manually constructing grammar and semantic rules and matching sentence fragments with patterns. It may also be a dictionary-driven method, which expands the dictionary, uses a string matching algorithm to identify entities, and uses verbs in the domain dictionary and their relation structures to identify relation types.

[0071] After performing relationship extraction on the screened new corpus, in order to improve the accuracy of the relationships between the multiple entities and each entity, the multiple entities and the relationships between each entity are further corrected for erroneous information. In one embodiment, a labeler with professional capabilities in the target field can correct the erroneous information in the relationships between the multiple entities and each entity to obtain corrected relationships between the multiple entities and each entity.

[0072] S203: construct a domain knowledge base of the target domain according to the corrected multiple entities and the relationships between the entities.

[0073] After correcting errors in the multiple entities and the relationships between the entities, the new corpus and the modified multiple entities and the relationships between the entities can be stored in a database to obtain a domain knowledge base of the target domain.

[0074] In the knowledge base construction method provided in the embodiment of the present application, in response to a request for constructing a domain knowledge base of a target domain, a new corpus of the target domain is screened through a text classification model, and then relationship extraction is performed based on the screened new corpus to obtain multiple entities and relationships between each entity, and erroneous information between the multiple entities and the relationships between each entity is corrected, and then the domain knowledge base of the target domain is constructed based on the corrected multiple entities and the relationships between each entity, wherein the text classification model is pre-trained based on positive samples consisting of a historical corpus of the target domain and negative samples consisting of corpora of other domains matching the target domain category. In this method, when constructing a domain knowledge base of a target domain, after obtaining a new corpus of the target domain required for constructing the domain knowledge base, a text classification model is first used to screen the new corpus of the target domain. The positive samples during training of the text classification model are the historical corpus of the target domain, while the negative samples are the corpus of other domains that match the target domain category. In this way, the trained text classification model can more accurately identify corpus texts that do not belong to the target domain from the new corpus of the target domain, thereby improving the data quality of the data required for constructing the domain knowledge base, thereby improving the quality of the constructed domain knowledge base; moreover, after performing relationship extraction on the screened new corpus, the erroneous information of the extracted multiple entities and the relationships between the entities is corrected, thereby further improving the data quality of constructing the domain knowledge base, thereby further improving the quality of the domain knowledge base, which is equivalent to making double quality assurance in terms of the data required for construction when constructing the domain knowledge base, thereby making the quality of the constructed domain knowledge base higher.

[0075] Based on the above embodiment, an embodiment is provided to illustrate the process of obtaining the above negative samples.

[0076] In an exemplary embodiment, Figure 3 As shown, the method also includes:

[0077] S301 , determining the inter-class distance between the target domain data and each candidate domain data according to the target domain data of the target domain and the candidate domain data of multiple candidate domains.

[0078] In the embodiment of the present application, the target domain data of the target domain and the candidate domain data of the multiple candidate domains can be any data of the corresponding domains. In order to ensure the accuracy of the selection of other domains that match the target category, thereby ensuring the quality and accuracy of negative samples, the domain data in each domain should be sufficient in quantity, so that the accurate inter-class distance can be determined based on the data characteristics of different domains, and at the same time, the data in each domain should not include error information. Among them, the calculation of the inter-class distance can use Euclidean distance, Manhattan distance, Mahalanobis distance, etc.

[0079] Taking Euclidean distance as an example, for any candidate field, the mean of the multiple candidate field data is calculated based on the multiple candidate field data of the candidate field, and the mean of the multiple target field data is calculated based on the multiple target field data of the target field. Then, based on the mean of the multiple candidate field data and the mean of the multiple target field data, combined with the Euclidean distance calculation formula, the inter-class distance between the target field and the candidate field is obtained.

[0080] S302, determining other fields that match the target field category from multiple candidate fields according to the distances between the categories, and determining the corpus of the other fields as negative samples.

[0081] Based on the multiple inter-class distances obtained above, the inter-class distances can be compared with the preset distance threshold, and the candidate fields corresponding to the inter-class distances less than the distance threshold are determined as other fields that match the target field category, and the corpus of the candidate field is obtained as a negative sample for training the text classification model. In addition, in order to improve the classification accuracy of the text classification model, an appropriate proportion of samples with large inter-class distances from the positive samples can also be selected.

[0082] In the knowledge base construction method provided in the embodiment of the present application, the inter-class distance between the target domain data and the candidate domain data of each candidate domain is determined based on the target domain data of the target domain and the candidate domain data of multiple candidate domains, and then other domains matching the target domain category are determined from multiple candidate domains based on the inter-class distance, and the corpus of other domains is determined as a negative sample. In this method, the inter-class distance between the target domain and each candidate domain is calculated based on the characteristics of the data in the target domain and the characteristics of the data in multiple candidate domains, so that one or more candidate domains are selected from multiple candidate domains as other domains matching the target domain based on the inter-class distance, and the corpus in the selected candidate domain can be a negative sample for training a text classification model to improve the classification accuracy of the text classification model.

[0083] Based on the above embodiment, an embodiment is provided to illustrate the process of obtaining the above positive samples.

[0084] In an exemplary embodiment, Figure 4 As shown, the method also includes:

[0085] S401, when there is a history domain knowledge base in the target domain, determine the history corpus in the history domain knowledge base as a positive sample.

[0086] Among them, the historical domain knowledge base is a domain knowledge base constructed based on the corpus of the target domain in historical time.

[0087] In the embodiment of the present application, if there is a history domain knowledge base in the target domain, the corpus in the history domain knowledge base is the history corpus set, and the history corpus set in the history domain knowledge base is directly determined as a positive sample.

[0088] S402, when there is no history domain knowledge base in the target domain, multiple historical corpus texts in the target domain are obtained from multiple third-party systems, and each historical corpus text is determined as a positive sample.

[0089] If there is no historical domain knowledge base in the target domain, that is, the domain knowledge base of the target domain has not been constructed in historical time, it is necessary to obtain positive samples through other means.

[0090] In an embodiment of the present application, multiple historical corpus texts of the target field can be obtained from multiple third-party systems, and the obtained multiple historical corpus texts can be used as positive samples. Among them, the third-party system can include vertical websites, professional field databases, natural corpus processing toolkits, etc. It is understandable that in order to improve the data quality of positive samples, after obtaining multiple historical corpus texts, the historical corpus texts are subjected to a data cleaning process. Among them, data cleaning includes illegal character removal, format alignment, deduplication, error information processing, etc.

[0091] In the knowledge base construction method provided in the embodiment of the present application, when there is a history domain knowledge base in the target domain, the historical corpus set in the history domain knowledge base is determined as a positive sample; when there is no history domain knowledge base in the target domain, multiple history corpus texts in the target domain are obtained from multiple third-party systems, and each history corpus text is determined as a positive sample. In this method, based on the two different situations of the existence of the history domain knowledge base in the target domain and the absence of the history domain knowledge base, different methods are used to obtain positive samples for training the text classification model, which improves the flexibility and diversity of positive sample acquisition and lays a foundation for reliable training of the text classification model.

[0092] Based on the above embodiment, an embodiment is provided to illustrate the process of obtaining multiple entities and the relationships between the entities.

[0093] In an exemplary embodiment, relationship extraction is performed based on the screened new corpus to obtain multiple entities and the relationships between the entities, including:

[0094] The screened new corpus is input into a pre-trained relation extraction model to obtain multiple entities and the relations between entities output by the relation extraction model; the relation extraction model is obtained by training the initial relation extraction model based on multiple text data and annotation information of each text data in the historical corpus.

[0095] In the embodiment of the present application, a relation extraction model that can achieve the relation extraction effect is pre-trained, and the new corpus after screening is processed based on the trained relation extraction model to obtain the relationship between multiple entities and each entity. Among them, the relation extraction model is based on the historical corpus of the target field and the annotation information of each text data, and the initial relation extraction model is trained. The historical corpus of the target field is the same as the above-mentioned way of obtaining positive samples, which will not be repeated here. There are multiple text data in the historical corpus, and the annotation information in each text data, wherein the annotation information refers to the relationship between the entities annotated by multiple text data and each entity. If the historical corpus is obtained from the historical domain knowledge base, the annotation information directly exists in the historical corpus. If the historical corpus is obtained from a third-party system, it is necessary to annotate multiple text data. For example, in order to provide annotation quality, multiple text data can be annotated by professional annotation personnel in the target field to obtain the annotation information of each text data. The initial relation extraction model can select the commonly used extractive model and generative model that can realize relation extraction.

[0096] In one embodiment, Figure 5 As shown in Figure 1, the training process of the relation extraction model includes:

[0097] S501, input each text data into the initial relationship extraction model to obtain multiple predicted entities output by the initial relationship extraction model and the predicted relations between the predicted entities.

[0098] In an embodiment of the present application, after obtaining multiple text data and annotation information of each text data for training the relationship extraction model, the multiple text data are input into the selected initial relationship extraction model. The initial relationship extraction model analyzes and processes the multiple text data and outputs its own predicted entities and the relationships between each entity, that is, multiple predicted entities and the predicted relationships between each predicted entity.

[0099] S502, determining the relationship extraction loss value of the initial relationship extraction model according to each predicted entity, each predicted relationship, each annotation information and a preset loss function.

[0100] After the initial relationship extraction model outputs multiple predicted entities and the predicted relationships between each predicted entity, the multiple predicted entities, the predicted relationships between each predicted entity, and the annotation information of multiple text data are substituted into the preset loss function to calculate the relationship extraction loss value of the initial relationship extraction model. Among them, the loss function can adopt cross entropy loss, mean square error loss, mean absolute error loss, etc.

[0101] S503: Based on the relationship extraction loss value, adjust the model parameters of the initial relationship extraction model until the training is completed to obtain the relationship extraction model.

[0102] Based on the relationship extraction loss value calculated above, the model parameters of the initial relationship extraction model are adjusted, and training is continued after the adjustment until the calculated relationship extraction loss value is less than the preset loss threshold, thereby obtaining a relationship extraction model.

[0103] In this embodiment, multiple text data are input into the initial relationship extraction model, and the initial relationship extraction model processes the multiple text data to output predicted entities and the relationships between entities. Based on the prediction results and the annotation information of the multiple text data, the relationship extraction loss value is calculated in combination with the loss function, so that the model parameters of the initial relationship extraction model are continuously adjusted based on the relationship extraction loss value, so that the prediction results of the initial relationship extraction model are continuously close to the true value, thereby improving the prediction ability of the model, and thus training a higher performance relationship extraction model.

[0104] In the knowledge base construction method provided in the embodiment of the present application, the screened new corpus is input into a pre-trained relation extraction model to obtain multiple entities and the relations between the entities output by the relation extraction model; the relation extraction model is obtained by training the initial relation extraction model based on multiple text data and the annotation information of each text data in the historical corpus. In this method, the initial relation extraction model is trained with the historical corpus of the target field to obtain a relation extraction model that can automatically annotate the new corpus, which reduces the efficiency of manual annotation and improves the annotation quality, thereby improving the accuracy and reliability of relation extraction for the new corpus.

[0105] Based on the above embodiment, an embodiment is provided to illustrate the process of correcting the erroneous information of multiple entities and the relationships between the entities.

[0106] In an exemplary embodiment, Figure 6 As shown, the incorrect information of multiple entities and the relationships between them is corrected, including:

[0107] S601, obtaining multiple candidate entities pre-annotated by multiple users on the filtered new corpus and the relationship between the candidate entities.

[0108] In an embodiment of the present application, before formally correcting erroneous information on multiple entities and the relationships between entities, multiple annotators first pre-annotate the screened new corpus, so as to balance the annotation consistency of multiple annotators based on the pre-annotation of multiple annotators, thereby improving the annotation quality when formally correcting.

[0109] In practical applications, each user annotates each data in the filtered new corpus to obtain multiple candidate entities pre-annotated by multiple users on the filtered new corpus and the relationship between the candidate entities.

[0110] S602: Determine the labeling consistency coefficient between users according to the multiple candidate entities and the relationship between the candidate entities.

[0111] Among them, the labeling consistency system is an indicator to measure the consistency when multiple labelers label the same data.

[0112] In one embodiment, if Figure 7 As shown, determining the labeling consistency coefficient of each user according to the multiple candidate entities and the relationship between the candidate entities may include:

[0113] S701, determining an actual consistency ratio and a random consistency ratio according to each candidate entity, the relationship between each candidate entity, the number of users of each user, and the amount of data in the screened new corpus.

[0114] The actual consistency ratio refers to the percentage of samples with consistent annotations to all annotated samples, while the random consistency ratio refers to the percentage of samples with consistent annotations due to random factors to all annotated samples.

[0115] For example, the actual consistency ratio and the random consistency ratio may be determined using the following formula (1) to formula (3).

[0116] (1)

[0117] (2)

[0118] (3)

[0119] in, is the actual consistency ratio; is the random consistency ratio; m is the number of users; n is the number of data; For the The data are labeled as categories ( ), which is determined based on each candidate entity and the relationship between each candidate entity (for example, for any data, the number of entities labeled as the same entity or the same relationship is determined from multiple candidate entities or the relationship between each candidate entity); is the number of triple categories (two entities and the relationship between the two entities constitute a triple).

[0120] S702: Determine a labeling consistency coefficient according to the actual consistency ratio and the random consistency ratio.

[0121] In one implementation, the labeling consistency coefficient can be determined based on the following formula (4) in combination with the actual consistency ratio and the random consistency ratio.

[0122] (4)

[0123] in, is the annotation consistency coefficient.

[0124] In this embodiment, the actual annotation consistency between the labelers and the annotation consistency caused by random factors are determined through the multiple candidate entities labeled by each user and the relationship between the candidate entities, combined with the number of users and the amount of data that needs to be pre-labeled, so as to quickly and accurately determine the annotation consistency coefficient based on the actual consistency ratio and the random consistency ratio, thereby providing a basis for balancing the annotation consistency between the labelers.

[0125] S603: Based on the annotation consistency coefficient, instruct each user to correct the erroneous information in the multiple entities and the relationships between the entities, so as to obtain the corrected multiple entities and the relationships between the entities.

[0126] After obtaining the annotation consistency coefficient, the annotation consistency coefficient is compared with the preset consistency threshold. If the annotation consistency coefficient does not meet the consistency threshold, it means that the annotation personnel have not yet reached the annotation consistency. Each annotation personnel continues to pre-annotate the screened new corpus and recalculate the annotation consistency coefficient between the annotation personnel until the annotation consistency coefficient meets the consistency threshold. Then, the pre-annotation is stopped and each annotation personnel is instructed to correct the erroneous information of the multiple entities obtained by relationship extraction and the relationships between the entities. At this time, the data can be divided into batches and given to multiple annotation personnel for correction, because the pre-annotation has made each annotation personnel have a consistent understanding of the knowledge concepts in the field.

[0127] In the knowledge base construction method provided in the embodiment of the present application, multiple candidate entities and the relationships between the candidate entities that are pre-annotated by multiple users on the screened new corpus are obtained, and then the annotation consistency coefficient between the users is determined based on the relationships between the multiple candidate entities and the candidate entities. Then, based on the annotation consistency coefficient, each user is instructed to correct the erroneous information in the relationships between the multiple entities and the entities, and the corrected relationships between the multiple entities and the entities are obtained. In this method, before correcting the erroneous information between the multiple entities and the relationships between the entities, the annotation consistency of the multiple annotation personnel is first balanced based on the pre-annotation results of the screened new corpus by multiple annotation personnel, that is, to ensure that each annotation personnel has a consistent understanding of the knowledge concepts in the target field. On this basis, multiple annotation personnel correct the erroneous information between the multiple entities and the relationships between the entities to improve the correction accuracy, and then improve the accuracy of the relationships between the entities and the entities, thereby improving the quality of the domain knowledge base.

[0128] Based on the above embodiment, an embodiment is provided to illustrate the process of constructing the domain knowledge base of the target domain.

[0129] In an exemplary embodiment, Figure 8 As shown, based on the modified multiple entities and the relationships between the entities, a domain knowledge base of the target domain is constructed, including:

[0130] S801, constructing a plurality of entity relationship triples according to the corrected plurality of entities and the relationships between the entities.

[0131] A triple is a data structure consisting of three elements, usually represented as (a, b, c), where a, b, and c are data elements of any type. An entity-relationship triple is a triple consisting of two entities and the relationship between the two entities.

[0132] In the embodiment of the present application, for any two entities, an entity relationship triple corresponding to the two entities is constructed according to the relationship between the two entities.

[0133] S802, storing each entity relationship triplet in a database to obtain a domain knowledge base of the target domain.

[0134] Based on the multiple entity relationship triples obtained above, each entity relationship triple is stored in the database, which constitutes the domain knowledge base of the target domain. It should be noted that if there is a historical domain knowledge base in the target domain, each entity relationship triple is filled into the historical domain knowledge base of the target domain to obtain an expanded domain knowledge base.

[0135] In the knowledge base construction method provided in the embodiment of the present application, a plurality of entity relationship triples are constructed according to the revised plurality of entities and the relationships between the entities, and each entity relationship triple is stored in a database to obtain a domain knowledge base of the target domain. In this method, a plurality of entity relationship triples are formed by combining the revised plurality of entities and the relationships between the entities, and then the plurality of entity relationship triples are stored in a database to form a domain knowledge base of the target domain, which is used for computer processing and application, thereby improving the construction efficiency and construction accuracy of the domain knowledge base.

[0136] In addition, in an exemplary embodiment, the present application also provides a knowledge base construction method, such as Fig. 9 As shown, the following steps may be included:

[0137] S901, obtaining a historical corpus of a target field; the historical corpus includes a plurality of text data and annotation information of each of the text data.

[0138] S902: Input each text data into the initial relationship extraction model to obtain a plurality of predicted entities output by the initial relationship extraction model and the predicted relationships between the predicted entities.

[0139] S903, determining the relationship extraction loss value of the initial relationship extraction model according to each predicted entity, each predicted relationship, each annotation information and a preset loss function.

[0140] S904: Based on the relationship extraction loss value, adjust the model parameters of the initial relationship extraction model until the training is completed to obtain the relationship extraction model.

[0141] S905 , in response to a request to construct a domain knowledge base of the target domain, screening a new corpus of the target domain by using a text classification model.

[0142] Among them, the text classification model is pre-trained based on positive samples consisting of a historical corpus of the target field and negative samples consisting of a corpus of other fields that matches the target field category.

[0143] S906, inputting the screened new corpus into a pre-trained relation extraction model to obtain multiple entities and the relations between the entities output by the relation extraction model.

[0144] S907, obtaining multiple candidate entities pre-annotated by multiple users on the filtered new corpus and the relationship between the candidate entities.

[0145] S908: Determine the labeling consistency coefficient between users according to the multiple candidate entities and the relationship between the candidate entities.

[0146] Optionally, the candidate entities, the relationships between the candidate entities, and the number of users of each user are screened, and the actual consistency ratio and the random consistency ratio are determined; and the annotation consistency coefficient is determined according to the actual consistency ratio and the random consistency ratio.

[0147] S909: Based on the annotation consistency coefficient, instruct each user to correct the erroneous information in the multiple entities and the relationships between the entities, so as to obtain the corrected multiple entities and the relationships between the entities.

[0148] S910: construct a plurality of entity relationship triples according to the corrected plurality of entities and the relationships between the entities.

[0149] S911, storing each entity relationship triplet in a database to obtain a domain knowledge base of the target domain.

[0150] The above-mentioned process of S901-S911 can refer to the description of the above-mentioned method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0151] In addition, in an exemplary embodiment, the construction framework of the iterative domain knowledge base in the embodiment of the present application is described.

[0152] like Fig.10 As shown in the figure, in order to improve the efficiency of domain knowledge base construction, the manual workload is reduced from the two links of corpus collection and manual annotation. In the corpus collection link, the efficiency of corpus collection in the field is improved with the help of search engine retrieval capabilities and text classification algorithms, while also ensuring the quality of the corpus. In the manual annotation link, the existing annotation data is used to train the relation extraction model to automatically annotate the new corpus, and only a small amount of manual correction is required, reducing the work of manual annotation. In order to ensure the quality of data annotation, the model annotation results are manually corrected, and the annotation errors of the model are continuously reduced through multiple iterations. The above scheme is used to dynamically expand the data set, the scale of the data set is continuously expanded, the effect of the trained relation extraction model is continuously improved, the quality of the model automatic annotation is continuously improved, and the workload of manual correction is continuously reduced, forming an iterative enhancement loop.

[0153] This embodiment achieves simultaneous iterative enhancement of data set scale, quality and model training effect, reduces expert labor costs in the process of building a domain knowledge base, and further screens the corpus through a text classification algorithm, thereby reducing the difficulty of collecting domain corpora and improving the quality of domain corpora. At the same time, the quality of data set annotation is controlled based on pre-annotation and annotation consistency coefficients, which reduces data annotation errors caused by human understanding bias and ensures data annotation quality.

[0154] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0155] Based on the same inventive concept, the embodiment of the present application also provides a knowledge base construction device for implementing the knowledge base construction method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more knowledge base construction device embodiments provided below can refer to the limitations on the knowledge base construction method above, and will not be repeated here.

[0156] In an exemplary embodiment, Fig.11 As shown, a knowledge base construction device 1 is provided, comprising: a corpus screening module 10, a relationship extraction module 20 and a knowledge base construction module 30, wherein:

[0157] A corpus screening module 10 is used to screen a new corpus set of the target domain through a text classification model in response to a request for building a domain knowledge base of the target domain; the text classification model is pre-trained based on positive samples consisting of a historical corpus set of the target domain and negative samples consisting of corpus sets of other domains matching the target domain category;

[0158] A relation extraction module 20 is used to perform relation extraction based on the screened new corpus, obtain multiple entities and the relations between the entities, and correct the wrong information between the multiple entities and the relations between the entities;

[0159] The knowledge base construction module 30 is used to construct a domain knowledge base of the target domain according to the modified multiple entities and the relationships between the entities.

[0160] In one embodiment, the knowledge base construction device 1 further includes:

[0161] A distance determination module, for determining an inter-class distance between the target domain data and each candidate domain data based on the target domain data of the target domain and the candidate domain data of the plurality of candidate domains;

[0162] The negative sample determination module is used to determine other fields that match the target field category from multiple candidate fields based on the distance between the categories, and determine the corpus of other fields as negative samples.

[0163] In one embodiment, the knowledge base construction device 1 further includes:

[0164] The positive sample determination module is used to determine the historical corpus set in the historical domain knowledge base as a positive sample when there is a historical domain knowledge base in the target domain; when there is no historical domain knowledge base in the target domain, obtain multiple historical corpus texts in the target domain from multiple third-party systems and determine each historical corpus text as a positive sample.

[0165] In one embodiment, the relationship extraction module 20 is further used for:

[0166] The screened new corpus is input into a pre-trained relation extraction model to obtain multiple entities and the relations between entities output by the relation extraction model; the relation extraction model is obtained by training the initial relation extraction model based on multiple text data and annotation information of each text data in the historical corpus.

[0167] In one embodiment, the knowledge base construction device 1 further includes:

[0168] An information prediction module, used to input each text data into the initial relation extraction model to obtain a plurality of predicted entities output by the initial relation extraction model and predicted relations between each predicted entity;

[0169] A loss determination module, used to determine the relationship extraction loss value of the initial relationship extraction model according to each predicted entity, each predicted relationship, each annotation information and a preset loss function;

[0170] The model training module is used to adjust the model parameters of the initial relationship extraction model based on the relationship extraction loss value until the training is completed to obtain the relationship extraction model.

[0171] In one embodiment, the relationship extraction module 20 is further used for:

[0172] A plurality of candidate entities and the relationships between the candidate entities pre-annotated by multiple users on the screened new corpus are obtained; based on the plurality of candidate entities and the relationships between the candidate entities, the annotation consistency coefficient between the users is determined; based on the annotation consistency coefficient, each user is instructed to correct the erroneous information in the plurality of entities and the relationships between the entities, so as to obtain the corrected plurality of entities and the relationships between the entities.

[0173] In one embodiment, the relationship extraction module 20 is further used for:

[0174] According to each candidate entity, the relationship between each candidate entity, the number of users of each user and the amount of data in the screened new corpus, the actual consistency ratio and the random consistency ratio are determined; according to the actual consistency ratio and the random consistency ratio, the annotation consistency coefficient is determined.

[0175] In one embodiment, the knowledge base construction module 30 is further used for:

[0176] According to the corrected multiple entities and the relationships between the entities, multiple entity relationship triples are constructed; each entity relationship triple is stored in a database to obtain a domain knowledge base of the target domain.

[0177] Each module in the above-mentioned knowledge base construction device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0178] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.12 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store knowledge base construction data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a knowledge base construction method is implemented.

[0179] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0180] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0181] In response to a request to build a domain knowledge base of a target domain, a new corpus of the target domain is screened by a text classification model; the text classification model is pre-trained based on positive samples consisting of a historical corpus of the target domain and negative samples consisting of corpuses of other domains matching the categories of the target domain;

[0182] Relationship extraction is performed based on the screened new corpus to obtain multiple entities and the relationships between them, and the wrong information of the relationships between them is corrected;

[0183] According to the revised multiple entities and the relationships between the entities, a domain knowledge base of the target domain is constructed.

[0184] The implementation principles and technical effects of each step implemented by the processor in the embodiment of the present application are similar to the principles of the above-mentioned knowledge base construction method and will not be repeated here.

[0185] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0186] In response to a request to build a domain knowledge base of a target domain, a new corpus of the target domain is screened by a text classification model; the text classification model is pre-trained based on positive samples consisting of a historical corpus of the target domain and negative samples consisting of corpuses of other domains matching the categories of the target domain;

[0187] Relationship extraction is performed based on the screened new corpus to obtain multiple entities and the relationships between them, and the wrong information of the relationships between them is corrected;

[0188] According to the revised multiple entities and the relationships between the entities, a domain knowledge base of the target domain is constructed.

[0189] The implementation principles and technical effects of the various steps implemented when the computer program in the embodiment of the present application is executed by the processor are similar to the principles of the above-mentioned knowledge base construction method and will not be repeated here.

[0190] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0191] In response to a request to build a domain knowledge base of a target domain, a new corpus of the target domain is screened by a text classification model; the text classification model is pre-trained based on positive samples consisting of a historical corpus of the target domain and negative samples consisting of corpuses of other domains matching the categories of the target domain;

[0192] Relationship extraction is performed based on the screened new corpus to obtain multiple entities and the relationships between them, and the wrong information of the relationships between them is corrected;

[0193] According to the revised multiple entities and the relationships between the entities, a domain knowledge base of the target domain is constructed.

[0194] The implementation principles and technical effects of the various steps implemented when the computer program in the embodiment of the present application is executed by the processor are similar to the principles of the above-mentioned knowledge base construction method, and will not be repeated here.

[0195] It should be noted that the personnel information (including but not limited to personnel equipment information, personnel personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the personnel or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0196] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0197] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0198] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for constructing a knowledge base, characterized in that: The method comprises: In response to a request for building a domain knowledge base of a target domain, a new corpus of the target domain is screened by a text classification model; the text classification model is pre-trained based on positive samples consisting of a historical corpus of the target domain and negative samples consisting of corpuses of other domains matching the category of the target domain; Performing relationship extraction based on the screened new corpus to obtain multiple entities and relationships between the entities, and correcting erroneous information between the multiple entities and the relationships between the entities; A domain knowledge base of the target domain is constructed according to the modified multiple entities and the relationships between the entities.

2. The method according to claim 1, characterized in that The method further comprises: Determining, according to the target domain data of the target domain and the candidate domain data of a plurality of candidate domains, an inter-class distance between the target domain data and each of the candidate domain data; According to each of the inter-class distances, other fields matching the target field category are determined from the multiple candidate fields, and corpora of the other fields are determined as the negative samples.

3. The method according to claim 1, characterized in that: The method further comprises: In the case where there is a history domain knowledge base in the target domain, determining the history corpus in the history domain knowledge base as the positive sample; In the case that the target domain does not have the historical domain knowledge base, a plurality of historical corpus texts of the target domain are acquired from a plurality of third-party systems, and each of the historical corpus texts is determined as the positive sample.

4. The method according to any one of claims 1 to 3, characterized in that: The extracting of relations based on the screened new corpus to obtain the relations between the multiple entities and the entities includes: The screened new corpus is input into a pre-trained relation extraction model to obtain the multiple entities and the relations between the entities output by the relation extraction model; the relation extraction model is obtained by training an initial relation extraction model based on multiple text data in the historical corpus and the annotation information of each text data.

5. The method according to claim 4, characterized in that The training process of the relation extraction model includes: Inputting each of the text data into the initial relation extraction model to obtain a plurality of predicted entities output by the initial relation extraction model and predicted relations between each of the predicted entities; Determining a relationship extraction loss value of the initial relationship extraction model according to each of the predicted entities, each of the predicted relationships, each of the annotation information and a preset loss function; Based on the relationship extraction loss value, the model parameters of the initial relationship extraction model are adjusted until the training is completed to obtain the relationship extraction model.

6. The method according to any one of claims 1 to 3, characterized in that: The correcting of error information between the plurality of entities and the relationships between the entities includes: Acquire multiple candidate entities pre-annotated by multiple users on the filtered new corpus and the relationship between the candidate entities; Determining a labeling consistency coefficient between the users according to the plurality of candidate entities and the relationship between the candidate entities; Based on the annotation consistency coefficient, each of the users is instructed to correct erroneous information in the plurality of entities and the relationships between the entities, so as to obtain corrected relationships between the plurality of entities and the entities.

7. The method according to claim 6, characterized in that The determining, according to the plurality of candidate entities and the relationship between the candidate entities, a labeling consistency coefficient of each user includes: Determining an actual consistency ratio and a random consistency ratio according to each of the candidate entities, the relationship between each of the candidate entities, the number of users of each of the users, and the amount of data in the screened new corpus; The annotation consistency coefficient is determined according to the actual consistency ratio and the random consistency ratio.

8. The method according to any one of claims 1 to 3, characterized in that: The constructing a domain knowledge base of the target domain according to the modified multiple entities and the relationships between the entities includes: Constructing a plurality of entity relationship triples according to the modified plurality of entities and the relationships between the entities; Each of the entity relationship triples is stored in a database to obtain a domain knowledge base of the target domain.

9. A knowledge base construction device, characterized in that: The device comprises: A corpus screening module, for responding to a request for building a domain knowledge base of a target domain, screening a new corpus of the target domain through a text classification model; the text classification model is pre-trained based on positive samples consisting of a historical corpus of the target domain and negative samples consisting of corpuses of other domains matching the category of the target domain; A relationship extraction module, used to perform relationship extraction based on the screened new corpus, obtain multiple entities and the relationships between the entities, and correct the erroneous information of the relationships between the multiple entities and the entities; The knowledge base construction module is used to construct a domain knowledge base of the target domain according to the modified multiple entities and the relationships between the entities.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images