Method and apparatus for constructing vertical domain knowledge graph and storage medium

Abstract

The present application discloses a method and apparatus for constructing a vertical domain knowledge graph and a storage medium. The method comprises: acquiring initial annotated text data and auxiliary training data, wherein the initial annotated text data is text data, currently annotated with an entity and an entity relationship, in a preset vertical domain, and the auxiliary training data is text data annotated with an entity and an entity relationship and constructed on the basis of data related to domain nouns in the preset vertical domain (S101); adjusting a preset pre-training model on the basis of the initial annotated text data and the auxiliary training data to obtain a target model (S102); obtaining newly added annotated text data on the basis of the target model and data to be annotated (S103); and on the basis of the initial annotated text data and the newly added annotated text data, obtaining a current knowledge graph in the preset vertical domain (S104).

Description

Method, device and storage medium for constructing vertical domain knowledge graph

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese patent application with application number 202310755894.4 and application date of June 26, 2023, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0003] The present application relates to the technical field of knowledge graph construction, and in particular to a method for constructing a vertical field knowledge graph, a device for constructing a vertical field knowledge graph, and a storage medium. Background Art

[0004] Currently, in practical applications, the technology for extracting information from unstructured data and constructing domain knowledge graphs is still under development. Generally speaking, there are roughly three methods: 1) manual extraction; 2) building templates for extraction; 3) using natural language processing technology (NLP) for extraction. With the development of NLP technology, the method of using NLP technology to extract knowledge from text has achieved good results in general fields. However, in vertical fields, such as the field of communications operation and maintenance, because entities and entity relationships are relatively professional, and entities and entity relationships are constantly changing, when using NLP technology to extract entities and entity relationships, the annotation threshold and cost are high, which leads to low efficiency in constructing vertical domain knowledge graphs.

[0005] Summary of the Invention

[0006] Based on this, the embodiments of the present application provide a method, device and existence medium for constructing a vertical field knowledge graph.

[0007] In a first aspect, the present application provides a method for constructing a vertical field knowledge graph, the method comprising: obtaining initial annotated text data and auxiliary training data, the initial annotated text data being text data currently annotated with entities and entity relationships in a preset vertical field, and the auxiliary training data being text data annotated with entities and entity relationships constructed based on data related to domain nouns in the preset vertical field; adjusting a preset pre-trained model based on the initial annotated text data and the auxiliary training data to obtain a target model; obtaining newly annotated text data based on the target model and the data to be annotated; and obtaining the current knowledge graph of the preset vertical field based on the initial annotated text data and the newly annotated text data.

[0008] In a second aspect, the present application provides a device for constructing a vertical field knowledge graph, the device comprising: a memory and a processor, the memory being used to store a computer program; the processor being used to execute the computer program and implement the method for constructing a vertical field knowledge graph as described above when executing the computer program.

[0009] In a third aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the method of constructing a vertical field knowledge graph as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG1 is a flow chart of an embodiment of a method for constructing a vertical domain knowledge graph according to the present application;

[0011] FIG2 is a flow chart of another embodiment of the method for constructing a vertical field knowledge graph according to the present application;

[0012] FIG3 is a flowchart of another embodiment of the method for constructing a vertical field knowledge graph in the present application;

[0013] FIG4 is a flowchart of another embodiment of the method for constructing a vertical field knowledge graph in the present application;

[0014] FIG5 is a flowchart of another embodiment of the method for constructing a vertical field knowledge graph in the present application;

[0015] FIG6 is a flowchart of another embodiment of the method for constructing a vertical field knowledge graph in the present application;

[0016] FIG7 is a flowchart of another embodiment of the method for constructing a vertical field knowledge graph according to the present application;

[0017] FIG8 is a flowchart of another embodiment of the method for constructing a vertical field knowledge graph according to the present application;

[0018] FIG9 is a flowchart of another embodiment of the method for constructing a vertical field knowledge graph in the present application;

[0019] FIG10 is a schematic diagram of the structure of an embodiment of an apparatus for constructing a vertical field knowledge graph according to the present application;

[0020] FIG11 is a schematic diagram of the structure of another embodiment of the apparatus for constructing a vertical field knowledge graph according to the present application;

[0021] FIG12 is a flow chart of an embodiment of a method for constructing a vertical domain knowledge graph executed by the apparatus of FIG11 ;

[0022] FIG13 is a schematic diagram of the preset pre-trained question-answering model used in FIG12 ;

[0023] FIG14 is a flow chart of an embodiment of constructing training data in the communication operation and maintenance field in FIG12;

[0024] FIG15 is a schematic diagram of an embodiment of a knowledge graph obtained by taking the communication operation and maintenance field as an example. DETAILED DESCRIPTION

[0025] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] The flowcharts shown in the accompanying drawings are for illustration only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0027] In the subsequent description, suffixes such as "module," "component," or "unit" used to represent elements are used only to facilitate the description of the present application and have no specific meaning. Therefore, "module," "component," or "unit" may be used interchangeably.

[0028] Currently, in practical applications, the technology for extracting information from unstructured data and constructing domain knowledge graphs is still under development. Generally speaking, there are roughly three methods: 1) manual extraction; 2) building templates for extraction; 3) using natural language processing technology (NLP) for extraction. With the development of NLP technology, the method of using NLP technology to extract knowledge from text has achieved good results in general fields. However, in vertical fields, such as the field of communications operation and maintenance, because entities and entity relationships are relatively professional, and entities and entity relationships are constantly changing, when using NLP technology to extract entities and entity relationships, the annotation threshold and cost are high, which leads to low efficiency in constructing vertical domain knowledge graphs.

[0029] The embodiment of the present application, when training the preset pre-training model, not only uses the initial annotated text data of the preset vertical field, but also specifically uses annotated auxiliary training data related to the domain nouns of the preset vertical field for training. The auxiliary training data is related to the domain nouns of the preset vertical field, and can expand the corresponding training data volume of the preset vertical field-related entities, which helps to improve the model's ability to extract related entities, improve the extraction effect of related entities and entity relationships, and thus improve the efficiency of constructing the vertical field knowledge graph.

[0030] The embodiments of the present application are described in detail below.

[0031] See Figure 1, which is a flow chart of an embodiment of the method for constructing a vertical field knowledge graph in the present application.

[0032] A vertical field can be a small field that is vertically subdivided under a larger field. It is a vertical extension of the field, not a horizontal extension of the field. It is a field that provides specific services to a limited group of people, such as communications, education, sports, medical care, environmental protection, entertainment, etc. Among them, communications, education, sports, medical care, environmental protection, entertainment and other fields can be further subdivided in the vertical direction. For example, the communications field can be further subdivided into the communications operation and maintenance field. The method of the embodiment of the present application is a method for constructing a knowledge graph for a certain vertical field.

[0033] A knowledge graph is a graph-based data structure composed of nodes and edges. Each node represents an entity, and each edge represents a relationship between entities, or entity relationships. A knowledge graph is essentially a semantic network. Entities can be real-world objects, such as people, places, companies, phone numbers, and animals; relationships can be used to express connections between different entities. In simple terms, a knowledge graph is a relational network formed by connecting all different types of information. Therefore, a knowledge graph provides the ability to analyze problems from a "relational" perspective. A key step in building a knowledge graph is extracting data from various data sources and then adding it to the knowledge graph according to specific rules. This process is called knowledge extraction. Data sources can be divided into two types: structured and unstructured. Structured data is relatively easy to process, while processing unstructured data typically requires natural language processing techniques, such as entity name recognition, relationship extraction, and entity unification.

[0034] The method includes: step S101, step S102, step S103 and step S104.

[0035] Step S101: Obtain initial annotated text data and auxiliary training data, wherein the initial annotated text data is text data currently annotated with entities and entity relationships in a preset vertical field, and the auxiliary training data is text data annotated with entities and entity relationships constructed based on data related to domain nouns in the preset vertical field.

[0036] In the embodiment of the present application, the initially annotated text data is text data currently annotated with entities and entity relationships in a preset vertical field. The initially annotated text data can be annotated manually.

[0037] The data related to the domain nouns of the preset vertical field includes the domain nouns of the preset vertical field (including proper nouns, common nouns, etc.), attribute data of the domain nouns of the preset vertical field, etc. The attribute data of the domain nouns of the preset vertical field can be data describing or illustrating the domain nouns. The attribute data of the domain nouns includes but is not limited to: data explaining and illustrating the domain nouns, data defining the domain nouns, feature data of the domain nouns, etc. In some embodiments, the data related to the domain nouns of the preset vertical field includes at least one of the domain data of the preset vertical field and the attribute data of the domain nouns.

[0038] Auxiliary training data is text data annotated with entities and entity relationships, which is constructed based on data related to the domain nouns of the preset vertical field. Therefore, auxiliary training data is usually text data of entities and entity relationships related to proper nouns, common nouns, etc. related to the preset vertical field. Auxiliary training data can be obtained manually, for example, by having professionals in the preset vertical field annotate the domain nouns and the relationships between domain nouns in the documents of the preset vertical field; auxiliary training data can also be obtained automatically, for example, by first finding the pattern of the appearance of domain nouns and the relationships between domain nouns in the documents of the preset vertical field, and automatically annotating the domain nouns and the relationships between domain nouns in the documents of the preset vertical field based on the pattern.

[0039] Step S102: adjusting the preset pre-training model based on the initial annotated text data and the auxiliary training data to obtain a target model.

[0040] A pre-trained model can be a model pre-trained on the original task, which can be used on the target task later. The model can be adjusted according to the characteristics of the target task to achieve the purpose of improving the target task. Therefore, in essence, the pre-trained model is a method of transfer learning, using a model trained by others on one's own target task. It can also be understood that a pre-trained model means first learning human knowledge to obtain a model, and then substituting it into a certain task to adjust the model, which is relatively easier and simpler. Pre-trained models include but are not limited to BERT (Bidirectional Encoder Representation from Transformers, a deep bidirectional encoder based on Transformer), GPT (Generative Pre-Trained Transformer, a generative pre-trained Transformer model), T5 (Text-to-Text Transfer Transformer) and other pre-trained models. A preset pre-trained model can refer to a pre-set pre-trained model.

[0041] Pre-trained models are usually obtained by training on large-scale data sets, so they have better generalization performance. Adjustment, which can also be called fine-tuning, refers to using the parameters of the pre-trained model as the initial parameters, and then training on a new data set so that the adjusted pre-trained model can obtain better performance. The preset pre-trained model is adjusted based on the initial annotated text data and the auxiliary training data, that is, the preset pre-trained model is adjusted using the initial annotated text data and the auxiliary training data, and the adjusted model of the preset pre-trained model is the target model. The main advantage of pre-training is that it can reduce the demand for labeled data, improve the generalization ability of the pre-trained model, and at the same time speed up the training speed of the pre-trained model and reduce the risk of overfitting of the pre-trained model.

[0042] Compared with the related art in which only the initial annotated text data is used when adjusting a pre-trained model, the embodiment of the present application uses not only the initial annotated text data but also the auxiliary training data as the annotated text data. In this way, on the one hand, the amount of initial annotated text data can be relatively reduced. If the initial annotated text data is obtained manually, the amount of manual annotation can be reduced, and the labor cost can be reduced; on the other hand, the adjusted target model can be made more targeted and objective, so it is easier to extract the entities and entity relationships of the data to be annotated in the preset vertical field.

[0043] Step S103: obtaining newly added annotated text data based on the target model and the data to be annotated.

[0044] Because the target model is a model that is obtained by adjusting a preset pre-trained model using the initial annotated text data and the auxiliary training data, the target model is capable of extracting entities and entity relationships from the data to be annotated in the preset vertical field. The target model can be used to automatically annotate the data to be annotated in a targeted and objective manner, thereby obtaining newly annotated text data annotated with entities and entity relationships. Automatically annotating the data to be annotated in a targeted and objective manner using the target model can improve annotation efficiency and effectiveness while also avoiding extensive manual annotation, significantly reducing labor costs.

[0045] Step S104: Based on the initial annotated text data and the newly added annotated text data, obtain the current knowledge graph of the preset vertical field.

[0046] Since the efficiency and effectiveness of obtaining the newly annotated text data by using the target model are greatly improved, the efficiency of constructing vertical field knowledge graphs will also be improved.

[0047] The embodiment of the present application, when training the preset pre-training model, not only uses the initial annotated text data of the preset vertical field, but also specifically uses annotated auxiliary training data related to the domain nouns of the preset vertical field for training. The auxiliary training data is related to the domain nouns of the preset vertical field, and can expand the corresponding training data volume of the preset vertical field-related entities, which helps to improve the model's ability to extract related entities, improve the extraction effect of related entities and entity relationships, and thus improve the efficiency of constructing the vertical field knowledge graph.

[0048] In some embodiments, in step S101 , obtaining auxiliary training data may include: sub-step S101A1 , sub-step S101A2 , sub-step S101A3 , and sub-step S101A4 , as shown in FIG. 2 .

[0049] Sub-step S101A1: determining a template set of context templates according to the domain nouns and the frequencies of occurrence of the attribute data of the domain nouns in the documents of the preset vertical domain.

[0050] The positions where domain nouns and the attribute data of the domain nouns appear in documents of the preset vertical field are usually relatively fixed, and the positions where domain nouns and the attribute data of the domain nouns appear in documents of the preset vertical field usually have a certain regularity. Therefore, the template set of the context template can be determined based on the frequency of appearance of the domain nouns and the attribute data of the domain nouns in documents of the preset vertical field.

[0051] Sub-step S101A2: filling the common nouns in the preset vertical field, the field nouns, and the attribute data of the field nouns into corresponding positions according to any template in the template set to obtain constructed text data.

[0052] By selecting any template from the template set, and filling the domain nouns (such as common nouns, professional nouns, etc.) in the preset vertical field and the attribute data of the domain nouns into the corresponding positions, the constructed text data can be constructed.

[0053] Sub-step S101A3: Determine the position of the entity in the constructed text data according to the filled template and the filled data.

[0054] Since the constructed text data is obtained by the filled template and the filled data, the template and the filled data are known, and therefore the position of the entity in the constructed text data can be calculated based on the filled template and the filled data.

[0055] Sub-step S101A4: annotating the constructed text data with entities and entity relationships according to the positions of the entities in the constructed text data to obtain the auxiliary training data.

[0056] When the locations of entities in constructed text data are known, the entities and entity relationships can be automatically annotated in the constructed text data to obtain auxiliary training data.

[0057] Since the template set is determined according to the frequency of occurrence of domain nouns and their attribute data in the document, the data related to the domain nouns are filled into the corresponding positions according to any template in the template set to obtain constructed text data. According to the filled template and the filled data, the position of the entity in the constructed text data is determined, and then the entities and entity relationships of the constructed text data are annotated to obtain auxiliary training data. In this way, auxiliary training data can be obtained simply, quickly and automatically.

[0058] In some embodiments, sub-step S101A1, determining the template set of the context template based on the frequency of occurrence of the domain noun and the attribute data of the domain noun in the documents of the preset vertical domain, may also include: sub-step S101A11 and sub-step S101A12, as shown in Figure 3.

[0059] Sub-step S101A11: mining the initial annotated text data, and determining the contexts of multiple patterns and / or multiple sentence patterns whose occurrence frequency is greater than or equal to a preset frequency based on the occurrence frequency of the domain nouns and the attribute data of the domain nouns in the initial annotated text data.

[0060] In an embodiment of the present application, the initial annotated text data is text data currently annotated with entities and entity relationships, and the initial annotated text data is mined to further mine the domain nouns and attribute data of the domain nouns in the initial annotated text data. The occurrence frequency of the domain nouns and the attribute data of the domain nouns in the initial annotated text data is determined, and based on whether the occurrence frequency is greater than or equal to a preset frequency, it is determined whether the domain nouns and the attribute data of the domain nouns appear occasionally or appear regularly in the initial annotated text data. If the occurrence frequency is greater than or equal to the preset frequency, it can be determined that the domain nouns and the attribute data of the domain nouns appear regularly in the initial annotated text data, and thus the context of multiple patterns and / or multiple sentence patterns with an occurrence frequency greater than or equal to the preset frequency is determined.

[0061] Sub-step S101A12: abstracting the contexts of the multiple patterns and / or multiple sentence patterns whose occurrence frequencies are greater than or equal to the preset frequencies to obtain a template set consisting of context templates of the multiple patterns and / or multiple sentence patterns.

[0062] Then, the contexts of the multiple patterns and / or multiple sentence patterns whose occurrence frequencies are greater than or equal to the preset frequency are abstracted to obtain a template set consisting of context templates of the multiple patterns and / or multiple sentence patterns.

[0063] Abstract the context of the pattern and / or sentence structure, that is, replace the domain nouns and other information in the context related to the mined domain nouns with the type of the corresponding entity (i.e., the specific entity). For example, "By checking the background data, it was found that Base Station No. 1 on Taoyuan Road had a small coverage area starting from June 5th." "Base Station No. 1 on Taoyuan Road" is abstracted as "location," "June 5th" is abstracted as "time," "coverage area" is an instance of the [performance indicator] type, and "small coverage area" is an instance of [fault phenomenon], which is composed of the noun of the performance indicator plus a "negative adjective." Therefore, after abstracting the sentence structure in the example sentence, the following template is obtained: "By checking the background data, it was found that {performance indicator name} {negative adjective} appeared at {location} starting from {time}."

[0064] In some embodiments, the template set may be obtained in the following manner:

[0065] The first method can be: after abstract processing to obtain multiple templates, the abstracted templates are sorted from high to low according to the proportion of the templates in the initial annotated text data, and added to the candidate template set in sequence. When the sum of the proportions of the templates in the candidate template set in the initial annotated text data reaches a set threshold (for example, 80%), stop adding new templates. At this time, the candidate template set is the final template set composed of context templates of multiple patterns and / or multiple sentence patterns.

[0066] The second method can be: after abstract processing to obtain multiple templates, sort the abstracted templates from high to low according to the proportion of the initial annotated text data, and take the top k templates. The template set composed of the top k templates is a template set composed of context templates of multiple patterns and / or multiple sentence patterns; for example, k = 1, the most common context template of the noun in this field is taken.

[0067] In some embodiments, step S102, adjusting the preset pre-trained model based on the initial annotated text data and the auxiliary training data to obtain a target model, may include: adjusting the parameters of the preset pre-trained question-answering model based on the initial annotated text data and the auxiliary training data to obtain a target model, wherein the parameters of the target model are the adjusted parameters of the preset pre-trained question-answering model.

[0068] In an embodiment of the present application, a pre-trained question-answering model is used as a pre-training model; the pre-trained question-answering model is to input a question and a description into the model at the same time based on the pre-training model, and finally the model is required to predict the location of the answer from the given description, that is, to essentially predict the starting position (start position) of the answer in the description and its end position (end position). Therefore, the pre-trained question-answering model can be to construct a classifier on the basis of the pre-training model to classify each token (Token) output of the last layer of the pre-training model to determine whether they belong to the start position or the end position. For example, the way to construct the pre-trained question-answering model can be: splice the original question and the context description into a sequence, separate them with the [SEP] symbol in the middle, and then input them into the pre-training model for feature extraction. After the pre-training model encoding is completed, the output of the last layer is taken to classify each Token to obtain the predicted output of the start position and the end position.

[0069] In some embodiments, in step S101 , before obtaining the initial annotated text data, the process may further include: step S105 , step S106 , and step S107 , as shown in FIG. 4 .

[0070] Step S105: Obtain unstructured raw data.

[0071] Step S106: pre-processing the unstructured original data to obtain data to be labeled.

[0072] Step S107: Divide the data to be annotated into multiple groups, so as to obtain the initial annotated text data from the multiple groups of data to be annotated.

[0073] At this time, in step S101, the obtaining of the initial annotated text data may also include: selecting a group of data to be annotated from the data to be annotated divided into multiple groups according to preset marking requirements, annotating the entities and entity relationships of the selected group of data to be annotated to obtain the initial annotated text data, the preset marking requirements are used to indicate the entity types and entity relationship types that need to be marked, and the number of the selected group of data to be annotated is less than or equal to the preset number.

[0074] In an embodiment of the present application, after preprocessing the unstructured raw data to obtain data to be labeled, the data to be labeled is divided into multiple groups, and the data to be labeled in one group is selected for entity and entity relationship labeling. The labeling is performed according to preset labeling requirements indicating the entity types and entity relationship types to be labeled, and the number of data to be labeled in the selected group is less than or equal to the preset number. In this way, on the one hand, labeling can be performed according to user needs based on the preset labeling requirements, and on the other hand, the number of data to be labeled is less than or equal to the preset number, which allows for small sample labeling, thereby reducing the amount of manual labeling and labor costs.

[0075] In some embodiments, the preset number may be less than or equal to 100, for example, the preset number may be 30-50.

[0076] In some embodiments, step S102, adjusting the preset pre-training model based on the initial annotated text data and the auxiliary training data to obtain a target model, may include: adjusting the parameters of the preset pre-training model based on the initial annotated text data and the auxiliary training data to obtain a first target model, wherein the parameters of the first target model are the adjusted parameters of the preset pre-training model.

[0077] The parameters of the pre-trained model include, but are not limited to, learning rate, objective function adjustment, coefficients of each part, number of training rounds, batch size, etc. In the embodiment of the present application, the parameters of the preset pre-trained model are adjusted using the initial annotated text data and the auxiliary training data to obtain a first target model, so that the first target model can be further adjusted subsequently to obtain a target model with better extraction effect and more optimization.

[0078] In some embodiments, step S103, obtaining newly annotated text data based on the target model and the data to be annotated, may include: sub-step S1031 and sub-step S1032, as shown in FIG5 .

[0079] Sub-step S1031: Based on the first target model and two or more groups of data to be labeled, an iterative processing method is used to obtain a target model that meets preset iteration termination requirements and a portion of newly labeled text data, wherein the data to be labeled is divided into two or more groups of data to be labeled.

[0080] Sub-step S1032: annotating the remaining group of to-be-annotated data using the target model and revising the annotation results to obtain the remaining newly annotated text data.

[0081] In the embodiment of the present application, the data to be annotated is divided into two or more groups of data to be annotated. The first target model is used to annotate the data to be annotated in one group to generate a group of newly annotated text data. The newly annotated text data is then used to further adjust the first target model. In this way, a target model that meets the preset iteration termination requirements and a portion of the newly annotated text data are obtained through iteration. Finally, the target model is used to annotate the remaining groups of data to be annotated and the annotation results are corrected to obtain the remaining newly annotated text data. In this way, a target model with better extraction effect and more optimization can be obtained, thereby improving the target model's subsequent annotation capabilities for the remaining groups of data to be annotated.

[0082] It should be noted that the number of newly added annotated text data generated after annotating a certain group of data to be annotated can be less than or equal to the number of data to be annotated in the certain group. For example, when the accuracy of annotation cannot be completely correct, the annotated text data generated after annotating a certain group of data to be annotated includes correctly annotated text data and incorrectly annotated text data. The correctly annotated text data can be used as a group of newly added annotated text data, and for incorrectly annotated text data, the data to be annotated corresponding to the incorrectly annotated text data can be re-annotated after the first target model is readjusted; or all the incorrectly annotated text data can be manually corrected to correct text data, and the correctly annotated text data and the corrected text data can be used as a group of newly added text data.

[0083] In some embodiments, sub-step S1031, based on the first target model and more than two groups of data to be annotated, using an iterative processing method to obtain a target model that meets the preset iteration termination requirements and a portion of the newly annotated text data, may also include: sub-step S10311, sub-step S10312 and sub-step S10313, as shown in Figure 6.

[0084] Sub-step S10311: annotate a group of to-be-annotated data using the first target model and modify the annotation results to obtain a group of newly annotated text data.

[0085] Sub-step S10312: adjusting the parameters of the first target model based on the current newly added annotated text data to obtain a second target model, where the parameters of the second target model are the adjusted parameters of the first target model.

[0086] Sub-step S10313: Use the second target model as the new first target model. When it is determined that the new first target model meets the preset iteration termination requirements, use the new first target model as the target model. When it is determined that the new first target model does not meet the preset iteration termination requirements, return to the step of annotating a group of data to be annotated using the first target model and correcting the annotation results to obtain a group of newly annotated text data, until the new first target model meets the preset iteration termination requirements, and use the new first target model as the target model.

[0087] In an embodiment of the present application, the first target model annotates a group of data to be annotated and then corrects the annotation results to obtain a group of newly annotated text data. Since the annotation results are corrected, the incorrectly annotated results are not returned for re-annotation. Therefore, the number of newly annotated text data in this group is the same as the number of a group of data to be annotated, avoiding returning for re-annotation and improving the annotation efficiency. At the same time, the number of incorrectly annotated results is limited. Based on this annotation result, the correction method is adopted to effectively reduce the workload of manual annotation. As the model extraction capability becomes stronger, the workload of correction continues to decrease, and the cost of correction is within an acceptable range.

[0088] The current newly added annotated text data is the newly added annotated text data that is annotated and corrected by the first target model and accumulated. For example, there are five groups of data to be labeled. The first target model labels the data to be labeled in the first group and modifies the labeling results to obtain the newly labeled text data of the first group. At this time, the current newly labeled text data is the newly labeled text data of the first group. The newly labeled text data of the first group is then used to adjust the parameters of the first target model to obtain a new first target model. If the preset iteration termination requirement is not met, the new first target model labels the data to be labeled in the second group and modifies the labeling results to obtain the newly labeled text data of the second group. At this time, the current newly labeled text data is the newly labeled text data of the first group and the newly labeled text data of the second group. The newly labeled text data of the first and second groups are then used to adjust the parameters of the new first target model to obtain another new first target model. If the preset iteration termination requirement is not met, another new first target model labels the data to be labeled in the third group and modifies the labeling results to obtain the newly labeled text data of the third group. At this time, the current newly labeled text data is the newly labeled text data of the first, second, and third groups. This cycle is repeated until the new first target model meets the preset iteration termination requirement, and the new first target model is used as the target model.

[0089] In some embodiments, the preset iteration termination requirement includes the evaluation indicators of the most recent preset number of new first target models being substantially the same, where the evaluation indicators are used to indicate the labeling effect of the new first target model. The evaluation indicators of the first target model include but are not limited to: micro F1, precision, recall, etc.

[0090] In some embodiments, sub-step S10312, adjusting the parameters of the first target model based on the current newly added annotated text data to obtain the second target model, may also include: adjusting the parameters of the first target model based on the initial annotated text data, the current newly added annotated text data and the auxiliary training data to obtain the second target model.

[0091] In an embodiment of the present application, considering that overfitting may cause the model to forget previously learned information, when adjusting the first target model, the newly added annotated text data is not directly used to continue adjusting the first target model. Instead, the initial annotated text data and all newly added annotated text data and auxiliary training data are used to adjust the first target model from scratch to obtain a second target model, that is, a new first target model.

[0092] In some embodiments, step S104, obtaining the current knowledge graph of the preset vertical field based on the initial annotated text data and the newly annotated text data, may also include: sub-step S1041 and sub-step S1042, as shown in Figure 7.

[0093] Sub-step S1041: performing entity alignment on the initial annotated text data and the newly added annotated text data to identify and aggregate entities belonging to synonyms.

[0094] Sub-step S1042: Obtain the current knowledge graph of the preset vertical field based on the initial annotated text data and the newly added annotated text data after entity alignment.

[0095] In an embodiment of the present application, since there may be many entities in the original data of the preset vertical field that are not expressed in a standardized manner and have serious problems of colloquialism, that is, multiple entities may express the same meaning, it is necessary to perform entity alignment on the initial annotated text data and the newly added annotated text data to identify and aggregate entities that are synonyms.

[0096] In some embodiments, the method may further include: step S108 , step S109 , and step S110 , as shown in FIG8 .

[0097] Step S108: Acquire new data to be labeled.

[0098] Step S109: obtaining new annotated text data based on the target model and the new data to be annotated.

[0099] Step S110: Based on the previously annotated text data and the newly added annotated text data, a new knowledge graph of the preset vertical field is obtained.

[0100] In the embodiment of the present application, as time accumulates, the target model can be used to annotate the newly added data to be annotated to obtain new annotated text data. Then, the previous annotated text data and the newly annotated text data can be used to update the knowledge graph, that is, to obtain a new knowledge graph for the preset vertical field. In other words, the knowledge graph of the embodiment of the present application can be continuously updated as the data to be annotated continues to accumulate.

[0101] In some embodiments, the method may further include: step S111 and step S112, as shown in FIG9 .

[0102] Step S111: Acquire new auxiliary training data.

[0103] Step S112: adjusting the target model based on the previous annotated text data, the previous auxiliary training data and the new auxiliary training data to obtain a new target model.

[0104] In the embodiments of the present application, as technology develops, business changes, etc., the types of entities and entity relationships that need to be extracted may change, and the previously adjusted target model may not be fully applicable to new developments, new situations, new businesses, etc. Therefore, new auxiliary training data can be obtained, and then the target model is adjusted using the previous annotated text data, the previous auxiliary training data, and the new auxiliary training data to obtain a new target model. In actual applications, the sampling weights of the previous annotated text data, the previous auxiliary training data, and the new auxiliary training data can be adjusted to enable the new target model to quickly adapt to the needs of new developments, new situations, new businesses, etc.

[0105] See Figure 10, which is a structural diagram of an embodiment of the device for constructing a vertical field knowledge graph in the present application. It should be noted that the device of the embodiment of the present application can implement the above-mentioned method of constructing a vertical field knowledge graph. For detailed description of the relevant content, please refer to the above-mentioned method part, which will not be repeated here.

[0106] The apparatus 100 includes a memory 1 and a processor 2. The memory 1 is configured to store a computer program. The processor 2 is configured to execute the computer program and implement any of the above-described methods for constructing a vertical domain knowledge graph when executing the computer program. The memory 1 is connected to the processor 2 via a bus.

[0107] The processor 2 may be a microcontroller unit, a central processing unit or a digital signal processor, etc. The memory 1 may be a Flash chip, a read-only memory, a magnetic disk, an optical disk, a USB flash drive or a mobile hard disk, etc.

[0108] It should be noted that the embodiments of the present application do not limit the specific implementation method of the device.

[0109] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements any of the methods for constructing a vertical field knowledge graph as described above.

[0110] The computer-readable storage medium may be an internal storage unit of the apparatus, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of the apparatus, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc.

[0111] The following describes the embodiments of the present application in detail by taking the preset vertical field of communication operation and maintenance as an example.

[0112] In the field of communications operation and maintenance, a large number of operation and maintenance-related case documents have been accumulated. In order to further promote the intelligent development of the communications operation and maintenance field, it is urgently necessary to manage and share the information of the accumulated document materials, extract the operation and maintenance knowledge contained in the documents, and build a knowledge graph in the field of communications operation and maintenance to provide it as basic data for upper-level intelligent operation and maintenance products.

[0113] With the development of NLP technology, the method of using NLP technology to extract knowledge from text has achieved good results in general fields. However, in the field of communication operation and maintenance, the amount of annotated corpus data is extremely scarce. At the same time, only professional engineers have the ability to annotate, that is, the annotation threshold and cost are very high. In addition, with the development of communication technology and changes in business, the types of entities and entity relationships will continue to change. The existing annotated corpus cannot be applied to the new types of entities and relationships. These problems make it very difficult to build a knowledge graph in the field of communication operation and maintenance.

[0114] In the process of constructing the knowledge graph in the field of communication operation and maintenance, there are problems such as non-standard expressions, serious colloquialism, and serious lack of annotated corpus. The threshold and cost of corpus annotation are high. With the continuous development of communication technology and business changes, the types of entities and relationships will continue to change. The embodiment of the present application provides a method and device for constructing a knowledge graph in the field of communication operation and maintenance based on small sample learning. It uses large-scale pre-trained models in general fields and the proper noun information accumulated in the field of communication operation and maintenance to conduct small sample learning, and preliminarily extract text data in the field of communication operation and maintenance, thereby improving the efficiency of constructing the knowledge graph in the field of communication operation and maintenance.

[0115] The embodiment of the present application uses the unstructured original data of communication operation and maintenance documents as the data source and is applied to the scenario of quickly building a knowledge graph in the field of communication operation and maintenance; it can also be applied to the knowledge graph construction service in the field of communication operation and maintenance, or in the online knowledge graph construction system, especially in the scenario where the types of entities and entity relationships that need to be extracted will change with technological development and business changes.

[0116] The structure of the device is shown in Figure 11, and may include the following modules: a data management module 101 to be processed, a labeling module 102, an auxiliary training data construction module 103, an information extraction module 104, a training data management module 105, a model training module 106, a model management module 107, a schema management module 108, an entity alignment module 109, and a knowledge storage module 110.

[0117] To-be-processed data management module 101: manages the communication operation and maintenance texts to be processed;

[0118] Annotation module 102: The annotation module 102 supports users to annotate the text data to be processed from scratch (initial annotated text data can be obtained). The types of entities and entity relationships (i.e., the required schema) that the annotation module 102 needs to annotate are defined in the schema management module 108. When annotating, the required schema can be selected. In addition, the annotation module supports users to review and modify the model extraction results, that is, it supports displaying the data processed by the information extraction module 104 to the user, and the user can modify or confirm the model extraction results.

[0119] Auxiliary training data construction module 103: constructs auxiliary training data using domain terms and attribute data accumulated in the field of communication operation and maintenance. The characteristics of this auxiliary training data construction module are:

[0120] 1) Domain nouns, attribute data of domain nouns, or domain nouns and attribute data of domain nouns are entities that need to be extracted to build a knowledge graph.

[0121] 2) By mining and summarizing (or summarizing expert experience) the annotated documents (which can be initial annotated text data), the contexts of various patterns and / or various sentence structures in which domain nouns and their attribute data frequently appear (i.e., the frequency of occurrence is greater than or equal to the preset frequency) in the annotated documents are obtained.

[0122] 3) Abstract the contexts of the multiple patterns and / or multiple sentence patterns that often appear in the mined domain nouns and their attribute data, and obtain multiple sets of templates of fixed patterns and / or fixed sentence patterns to form a template set.

[0123] The context of the pattern and / or sentence is abstracted, i.e., the information related to the common nouns, domain nouns, and communication operation and maintenance domain nouns in the context of the mined domain nouns is replaced with the type of the corresponding entity. For example, "By checking the background data, it was found that the wireless call drop rate at LFZ0704842H1 Guandi Temple had been poor since July 2nd." In the sentence, "LFZ0704842H1 Guandi Temple" is abstracted to "location" and "July 2nd" is abstracted to "time." The wireless call drop rate belongs to the "performance indicator" type of the communication operation and maintenance domain, and "poor wireless call drop rate" is an instance of the "fault phenomenon" in the communication operation and maintenance domain, consisting of the name of the performance indicator plus a "negative adjective." Therefore, after abstracting the context of the sentence in the example, the following template is obtained: "By checking the background data, it was found that {performance indicator name} {negative adjective} appeared at {location} starting from {time}."

[0124] There are two ways to select templates in the template set:

[0125] Method 1: Sort the abstracted templates by their proportion in the initial annotated text data from high to low, and add them to the candidate template set in sequence. When the sum of the proportions of the templates in the candidate template set in the initial annotated text data reaches a set threshold, for example, 80%, stop adding new templates.

[0126] Method 2: Sort the abstracted templates by their proportion in the initial annotated text data from high to low, and take the top k templates to form a template set. For example, k = 1, and take the most common context pattern of the noun in this field.

[0127] 4) Based on the template set of fixed patterns and / or sentences obtained above, a template (i.e., a filled template) is selected according to a certain sampling strategy, and the domain nouns, the attribute data of the domain nouns, or the domain nouns and the attribute data of the domain nouns are filled into the positions specified by the selected template for filling, so as to obtain the constructed text data of the entity instance (i.e., a specific entity) composed of the domain nouns and their attribute data; based on the filled template used and the filled data, the position of the constructed entity instance in the constructed text data can be calculated, and then the constructed text data can be automatically labeled.

[0128] The selection of a template from the template set according to a certain sampling strategy may be uniform extraction, or sampling according to the proportion of the templates appearing in the initial annotated text data, that is, templates that appear in a large proportion in the initial annotated text data have a high probability of being extracted.

[0129] For example, in the template in 3), for common nouns such as time and place, specific words can be randomly extracted from the prepared time noun table and place noun table, such as "Yesterday at 3 pm" and "Nanjing West Road Subway Station", to replace "{time}" and "{place}" in the template; fill in the common noun information in the corresponding position, and in the template in 3), replace "{performance indicator name}" with the domain noun information of the [performance indicator] class to be constructed, for example, "wireless connection rate", to obtain the following context text about "wireless connection rate": By checking the background data, it is found that the wireless connection rate of Nanjing West Road Subway Station has been low since 3 pm yesterday.

[0130] For the filled template and filled data, the constructed text data is annotated according to the rules. Taking the context text about "wireless connection rate" obtained by the above construction as an example, it is necessary to annotate the positions of the entities corresponding to [performance indicators] and [fault manifestations]. It is easy to know that the starting position of the performance indicator start_1 = 10 + len ("Nanjing West Road Subway Station") + 1 + len ("Yesterday at 3 pm") + 4, and the ending position end_1 = start_1 + len ("performance indicator name"); similarly, the starting position corresponding to [fault manifestation] start_2 = start_1, and the ending position end_2 = end_1 + len ("low"), where len (x) represents the length of the string x.

[0131] The following example illustrates the construction of a knowledge graph in the communications operations domain. The schema used in this example is ["fault case," "fault manifestation," "troubleshooting method," "fault root cause," "solution," "performance indicator," "alarm"]. Using the domain nouns for alarms accumulated in this domain as an example, the following illustrates the content of the auxiliary training data constructed above:

[0132] Example: Alarms are used to notify and alert users of abnormalities in communication system equipment or software, helping them improve troubleshooting. Accumulated alarm data includes [Alarm Name], [Alarm Cause], and [Handling Suggestions]. [Alarm Name] is typically an instance of "Alarm" in the aforementioned schema, [Alarm Cause] is typically an instance of "Fault Root Cause," and [Handling Suggestions] is an instance of "Solution." Mining sentences containing [Alarm] information in the initial annotated text data revealed the following patterns, which account for approximately 90% of the total: "Checking current / historical alarms at this site revealed the presence of {Alarm Name}. {Handling Suggestions}, and {Alarm Name} has disappeared. Therefore, {Alarm Name} at this site was caused / caused by {Alarm Cause}," and "{Alarm Name} exists at the base station at {Location}, suspected to be {Alarm Cause}. Following {Handling Suggestions}, the alarm resolved at {Time}." Select a template and fill the alarm attribute data into the corresponding template to obtain constructed text data. Then, based on the selected template and the filled-in information, automatically mark the alarm name, root cause of the fault, and solution in the constructed text data, thus obtaining an auxiliary training data constructed with alarm information, thereby expanding the number of samples of alarm name, root cause of the fault, and solution in the training data.

[0133] The auxiliary training data construction module expands the amount of corresponding training data for related entities by utilizing accumulated domain nouns and their attribute data, thereby improving the model's ability to extract such entities. The auxiliary training data construction module supports the creation, editing, importing, and management of templates used for data construction.

[0134] The information extraction module 104 selects a suitable extraction model from the model management module 107 and a required schema from the schema management module 108 to extract information from the communication operation and maintenance text data to be processed (ie, the data to be processed).

[0135] Training data management module 105: manages different batches of training data (which may be data to be processed divided into different groups) for use in model adjustment (fine-tuning).

[0136] Model training module 106: The user can select a basic pre-training model from the models provided by the model management module 107 as the preset pre-training model, select the training data in the training data management module 105 and the auxiliary training data in the auxiliary training data construction module 103, and set different sampling weights for different batches of training data (for example, a relatively high sampling weight is set for manually reviewed annotated text data). In addition, the preset pre-training model can also set some hyperparameters, such as learning rate, objective function adjustment, coefficients of each part, training rounds, batch size, etc.; the objective function adjustment includes a combination of objective functions (including: cross entropy, KL divergence between the current model and the reference model, etc.).

[0137] Model management module 107: This module manages the basic pre-trained model and the adjusted model from the model training module 106, and provides them to the information extraction module 104 for use.

[0138] Schema management module 108: provides functions for defining, editing, modifying, and managing schemas.

[0139] Entity Alignment Module 109: This module identifies synonyms within the extracted entities and clusters different names representing the same entity. This module uses the Levenshtein distance to initially identify candidate synonym pairs. It then further filters these candidate synonym pairs based on entity pair semantic similarity. This ensures that the candidate synonyms obtained by Entity Alignment Module 109 have a high accuracy rate, reducing the workload of subsequent manual verification.

[0140] Knowledge storage module 110: converts the extracted information confirmed by the user or automatically reviewed, as well as the information annotated by the user, into triple information, and stores it in a database that meets business needs, as the basis for upper-level applications of the knowledge graph in the communication operation and maintenance field; if multiple users use the device and their preferences are different, then using the same communication operation and maintenance data source may eventually result in multiple different communication operation and maintenance knowledge graphs to meet the usage needs of different users.

[0141] The prerequisites for using the method flow of the embodiment of the present application are: 1) the schema required for the knowledge graph needs to be designed in advance, that is, the types of entities and entity relationships that need to be extracted; 2) a certain amount of unstructured text related to communication operation and maintenance has been accumulated.

[0142] As shown in FIG12 , the method includes the following steps:

[0143] S1. Obtain unstructured communication operation and maintenance data (i.e., unstructured raw data), which usually includes communication operation and maintenance case documents and other data. Generally, it can be directly obtained from the company's internal document database.

[0144] S2. Preprocess the unstructured communication operation and maintenance data to obtain data to be labeled. The preprocessing mainly involves obtaining the text portion of the data by writing rule codes, deleting redundant spaces and meaningless character strings, and converting English letters to lowercase.

[0145] S3. Divide the data to be labeled into multiple groups according to N items, randomly select a group of data to be labeled, and obtain initial labeled text data; label-studio or doccano platform can be used to manually label the entities and entity relationships of the selected group of data to be labeled, where N can usually be 30-50.

[0146] S4. If the entities to be extracted contain domain nouns (such as professional nouns), then the existing domain nouns related to the communication operation and maintenance field (such as professional nouns) and the attribute data of the domain nouns (such as data described by professional nouns) can be used to construct auxiliary training data; therefore, optionally, the domain nouns and the attribute data of the domain nouns accumulated in the communication operation and maintenance field are obtained.

[0147] S5. Optionally, for the domain nouns and attribute data of the domain nouns accumulated in the communication operation and maintenance field, auxiliary training data is automatically constructed by the following method:

[0148] A) discovering (or summarizing through expert experience) through mining the initial annotated text data that domain nouns and attribute data of domain nouns frequently appear (i.e., the frequency of occurrence is greater than or equal to a preset frequency) in the context of several patterns and / or sentence patterns in the initial annotated text data;

[0149] B) abstracting the context of the frequently appearing patterns and / or sentence patterns of the mined domain nouns and their attribute data to obtain several sets of templates of fixed patterns and / or sentence patterns, forming a template set;

[0150] C) Based on the template set obtained above, a template is selected according to a certain sampling strategy, and the domain noun, the attribute data of the domain noun, or the domain noun and the attribute data of the domain noun are filled into the position specified by the template to obtain the constructed text data of the entity instance (i.e., the specific entity) composed of the domain noun and the attribute data of the domain noun; based on the filled template used and the filled data, the position of the entity in the constructed text data is calculated, and the constructed text data is automatically annotated according to the position of the entity in the constructed text data.

[0151] S6. Based on the initial annotated text data and the auxiliary training data, the preset pre-trained question-answering model is adjusted (i.e., fine-tuned) to obtain a first target model; wherein, the preset pre-trained question-answering model can be selected based on large-scale pre-trained language models such as BERT and ERNIE, and the adjusted question-answering model is used on a general extractive question-answering dataset. The model structure is shown in FIG13 , and the model includes an encoding layer and a classification layer, wherein e0, e1, ..., e18, ... represent position numbers, Ysi represents the probability that position i is the starting position of the entity, and Yei represents the probability that position i is the ending position of the entity; the embodiment of the present application utilizes a small sample learning of a large-scale pre-trained question-answering model In order to improve learning ability, a mechanism based on multi-round dialogue is adopted to extract entities and entity relationships from the labeled data in turn; therefore, the initial labeled text data and auxiliary training data need to be converted into question-answering training data, as shown in Figure 14. The type of entity to be extracted and the type of entity relationship are converted into questions. The text in the initial labeled text data identifies whether each position in the sequence is the start position and end position of the entity to be extracted; the schema designed in the first step of Figure 14 is: [{“fault case”: [“phenomenon”, “cause”, “solution”]}], where “fault case” is the type of entity to be extracted, and “phenomenon”, “cause”, and “solution” are The type of entity relationship to be extracted; the initial annotated text data in the second step is "Coverage problem analysis and processing case: poor video calls, poor Tel speed test experience. Based on KPI correlation analysis, the cell rate is low, mainly because the road loss ratio is too high, about 50%-60%; optimization method: downtilt angle is reduced by 6 degrees (3 degrees adjusted to 9 degrees). Optimization effect: After the downtilt angle is adjusted, the road loss distribution of uplink business users (>135) is improved from 50% to 8%, and the uplink rate is improved from 0.4Mbps to 2.20Mbps." Among them, the "fault case" is the "coverage problem analysis and processing case", its "phenomenon" is "poor video calls, poor Tel speed test experience", and its "cause" is " Based on KPI correlation analysis, the low cell rate is primarily due to an excessively high path loss ratio of approximately 50%-60%. The solution is to reduce the downtilt angle by 6 degrees (adjusting from 3 degrees to 9 degrees). The third step identifies whether each position in the sequence represents the start and end positions of the entity to be extracted, thereby obtaining training samples. For example, [CLS] Fault Case [SEP] text [SEP], labels: [{start:0,end:10}], [CLS] Coverage Problem Analysis and Treatment Case Phenomenon [SEP] text [SEP], labels: [{start:11,end:16},{start:17,end:25}], etc. As shown in Figure 13, the training samples obtained in the third step of Figure 14 are input into the encoding and classification layers of a pre-trained language model such as BERT or ERNIE to adjust the pre-trained language model.Based on the converted question-answering training data, the preset pre-training model is fine-tuned. The objective function defaults to the cross-entropy loss function. Settings such as the learning rate, training rounds, and the sampling weights of each training sample are set according to specific circumstances.

[0152] S7: Determine whether the iteration termination condition is met. If not, proceed to S8; if yes, proceed to S11.

[0153] S8. Use the first target model to extract entities and entity relationships from a randomly selected set of data to be labeled to obtain prediction results.

[0154] S9. Correct the prediction results to obtain newly added annotated text data: Since the target model fine-tuned using the communication operation and maintenance training data (i.e., the initial annotated text data and auxiliary training data) has a certain information extraction capability for the communication operation and maintenance data to be annotated, the initial target model is used to perform preliminary entity and entity relationship extraction on a set of data to be annotated. Based on the model prediction results, the erroneous parts in the prediction results are manually corrected, and some entities and entity relationships that the model failed to extract are supplemented with annotations to obtain newly added annotated text data. At the same time, the corrected data is used to calculate the evaluation index of the initial target model on the set of data to be annotated, that is, the micro F1 or precision or recall rate is calculated as the evaluation index of the extraction effect of this model.

[0155] The method of using the model to preliminarily label the data to be labeled and then manually making modifications based on this preliminary labeling can effectively reduce the workload of manual labeling. As the model extraction capability becomes stronger, the workload of manual correction continues to decrease. In addition, since the communications field has relatively high requirements for the quality of extracted information, the extraction accuracy and recall rate of model extraction are usually difficult to reach 95% or above. Therefore, the results of model extraction usually need to be manually corrected and confirmed.

[0156] S10. Based on the initial annotated data and all the newly annotated text data, as well as the auxiliary training data, fine-tune the first target model to obtain a new first target model: It should be noted that, considering that fine-tuning overfitting may cause the model to forget previously learned information, this step does not directly use the newly annotated text data to continue fine-tuning the above-mentioned first target model. Instead, the initial annotated text data and all the newly annotated text data, as well as the auxiliary training data, are used to fine-tune the first target model from scratch to obtain a new first target model.

[0157] Repeat steps S7 to S10 until the iteration terminates: that is, while using the new first target model to label a set of data to be labeled, continue to adjust the steps of the new first target model. The iteration termination condition is: the evaluation indicators of the three latest new first target models are basically equal, indicating that the model extraction effect does not become stronger with the increase of training data. The iterative process can be stopped and the new first target model can be used as the target model.

[0158] S11. Use the target model to predict the remaining groups of unlabeled data.

[0159] S12. Based on the model prediction results, manually correct the erroneous parts of the prediction results, and perform additional annotations on the parts where the model fails to extract entities and entity relationships, to obtain newly annotated text data for the remaining parts.

[0160] S13. Perform entity alignment on all annotated text data.

[0161] Since many entities in communication operation and maintenance documents have non-standard expressions and serious colloquialisms, that is, multiple entities may express the same meaning, it is necessary to perform synonym entity recognition and clustering on the acquired entities. For each entity, perform the following operations:

[0162] A) Use jieba, FastHan, or a word segmenter corresponding to the target model to segment the entity. For example, use jieba to segment the cell uplink effective throughput rate to obtain [cell / uplink / effective / throughput / rate].

[0163] B) Optionally, further obtain the semantic representation of entities and all entity relationships: obtain all triple data containing entity i, directly splice them to obtain all relationship description texts of the target entity, use the encoding layer of the target model, or a selected encoding model to encode entity i and the entity i relationship description text respectively, take the corresponding [cls] vector, and represent the entity and entity relationship data, recorded as E_i, E_ri.

[0164] C) Synonym identification: For any entity pair (i, j), calculate the ratio a of the Levenshtein distance dij to (entity i length + entity j length - dij), as well as the Jaccard similarity b between the phrases after the entity pair (i, j) is segmented. If a is less than the set threshold T1, or b is greater than the set threshold T2, the entity pair (i, j) is considered to be similar in form. Otherwise, (i, j) is considered not to be a synonym, and the next entity pair is judged.

[0165] D) Optionally, semantic similarity is used to further determine whether the entity pair (i, j) is a synonym, that is, by calculating cosine(E_i, E_j), the semantic similarity s1 of the entity pair is obtained; optionally, by calculating cosine(E_ri, E_rj), the semantic similarity s2 between the corresponding relationship description texts of the entity pair is obtained; if s1 is greater than the set threshold T3, or s2 is greater than the set threshold T4, then (i, j) is considered to be a candidate synonym pair, otherwise, (i, j) is considered not to be a synonym pair.

[0166] E) Finally, the candidate synonym pairs are manually confirmed to obtain a synonym set.

[0167] S14. Obtain a knowledge graph based on all the annotated text data after entity alignment.

[0168] The obtained knowledge graph is stored in a suitable database for use by other intelligent systems. As shown in Figure 15, taking the field of communications operations and maintenance as an example, a schematic diagram of the knowledge graph subgraph is extracted. Each ellipse represents an entity, and the ellipses with different dashed and solid lines indicate different entity types. The lines between entities indicate the relationships between entities, i.e., entity relationships.

[0169] In summary, the method and device of the embodiments of the present application use a large-scale pre-trained question-answering model to convert the extraction task into a multi-round question-answering task, effectively utilizing the general semantic understanding and prompt learning technology of the pre-trained question-answering model. Through small sample learning, a model with good extraction capabilities in the field of communication operation and maintenance can be obtained, thereby improving the efficiency of extracting entities and entity relationships from text data.

[0170] For scenarios where it is necessary to quickly build a knowledge graph in the field of communication operation and maintenance, especially scenarios where unstructured data is used as the original data for building the knowledge graph and data annotation requires professional communication operation and maintenance engineers, the method of the embodiment of the present application can be used to quickly build a knowledge graph in the field of communication operation and maintenance through small sample learning and iterative annotation adjustment.

[0171] In the scenario of building a communication operation and maintenance knowledge graph online, as the business changes, the types of entities and entity relationships that need to be extracted change, and the previously adjusted model is no longer fully applicable to the new business. By using the device of the embodiment of the present application, by modifying the schema, the data to be extracted and labeled are divided into multiple groups, the data are iteratively labeled, the sampling weights of various training data are adjusted, and the model is adjusted, it can quickly adapt to new business needs.

[0172] Documents in the field of communication operation and maintenance have non-standard expressions and are more colloquial, and the same entity has multiple expressions. Using the method and device of the embodiments of the present application, synonyms in the entity can be identified, and different expressions of the same entity can be effectively obtained.

[0173] The embodiments of the present application provide a method, device and medium for constructing a vertical domain knowledge graph. When training a preset pre-trained model, not only the initial annotated text data of the preset vertical domain is used, but also the annotated auxiliary training data related to the domain nouns of the preset vertical domain is specially used for training. The auxiliary training data is related to the domain nouns of the preset vertical domain, and can expand the corresponding training data volume of the preset vertical domain-related entities, which helps to improve the model's ability to extract related entities, improve the extraction effect of related entities and entity relationships, and thus improve the efficiency of constructing the vertical domain knowledge graph.

[0174] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0175] In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0176] The above description of some embodiments of the present application with reference to the accompanying drawings does not limit the scope of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present application shall be within the scope of the present application.

Claims

1. A method for constructing a vertical domain knowledge graph, the method comprising: Acquire initial annotated text data and auxiliary training data, wherein the initial annotated text data is text data currently annotated with entities and entity relationships in a preset vertical field, and the auxiliary training data is text data annotated with entities and entity relationships constructed based on data related to domain nouns in the preset vertical field; Adjusting a preset pre-training model based on the initial annotated text data and the auxiliary training data to obtain a target model; Based on the target model and the data to be annotated, newly added annotated text data is obtained; Based on the initial annotated text data and the newly added annotated text data, the current knowledge graph of the preset vertical field is obtained.

2. The method according to claim 1, wherein: Obtain auxiliary training data, including: Determine a template set of context templates according to the frequency of occurrence of the domain nouns and the attribute data of the domain nouns in the documents of the preset vertical domain; Fill the common nouns in the preset vertical field, the field nouns, and the attribute data of the field nouns into corresponding positions according to any template in the template set to obtain constructed text data; Determining the position of the entity in the constructed text data according to the filled template and the filled data; According to the positions of the entities in the constructed text data, the constructed text data is annotated with entities and entity relationships to obtain the auxiliary training data.

3. The method according to claim 2, wherein: The template set of determining the context template according to the frequency of occurrence of the domain noun and the attribute data of the domain noun in the documents of the preset vertical domain includes: Mining the initial annotated text data, and determining the contexts of multiple modes and / or multiple sentence patterns whose occurrence frequencies are greater than or equal to a preset frequency according to the occurrence frequencies of the domain nouns and the attribute data of the domain nouns in the initial annotated text data; Abstract processing is performed on the contexts of the multiple modes and / or multiple sentence patterns whose occurrence frequencies are greater than or equal to the preset frequencies to obtain a template set consisting of context templates of the multiple modes and / or multiple sentence patterns.

4. The method according to claim 1, wherein: The step of adjusting the preset pre-training model based on the initial annotated text data and the auxiliary training data to obtain a target model includes: Based on the initial annotated text data and the auxiliary training data, the parameters of the preset pre-trained question-answering model are adjusted to obtain a target model, wherein the parameters of the target model are the adjusted parameters of the preset pre-trained question-answering model.

5. The method according to claim 4, wherein: Before obtaining the initial annotation text data, it also includes: Get unstructured raw data; Preprocessing the unstructured raw data to obtain data to be labeled; The data to be annotated are divided into multiple groups, so that the initial annotated text data is obtained from the multiple groups of data to be annotated.

6. The method according to claim 5, wherein: The obtaining of initial annotated text data includes: According to preset marking requirements, one group of data to be labeled is selected from the data to be labeled that is divided into multiple groups, and entities and entity relationships are labeled for the selected group of data to be labeled to obtain the initial labeled text data. The preset marking requirements are used to indicate the entity types and entity relationship types that need to be labeled, and the number of the selected group of data to be labeled is less than or equal to the preset number.

7. The method according to claim 1, wherein: The step of adjusting the preset pre-training model based on the initial annotated text data and the auxiliary training data to obtain a target model includes: The parameters of the preset pre-training model are adjusted based on the initial annotated text data and the auxiliary training data to obtain a first target model, wherein the parameters of the first target model are the adjusted parameters of the preset pre-training model.

8. The method according to claim 7, wherein: The step of obtaining newly added annotated text data based on the target model and the data to be annotated includes: Based on the first target model and more than two groups of data to be labeled, an iterative processing method is used to obtain a target model that meets a preset iteration termination requirement and a portion of newly added labeled text data, wherein the data to be labeled is divided into more than two groups of data to be labeled; The target model is used to annotate the remaining group of data to be annotated and the annotated results are corrected to obtain the remaining newly annotated text data.

9. The method according to claim 8, wherein: The method of obtaining a target model and a portion of newly annotated text data that meet preset iteration termination requirements by using an iterative processing method based on the first target model and more than two groups of data to be annotated includes: Annotating a group of to-be-annotated data by using the first target model and correcting the annotation results to obtain a group of newly annotated text data; Adjusting the parameters of the first target model based on the current newly added annotated text data to obtain a second target model, wherein the parameters of the second target model are the adjusted parameters of the first target model; The second target model is used as a new first target model. When it is determined that the new first target model meets the preset iteration termination requirement, the new first target model is used as the target model. When it is determined that the new first target model does not meet the preset iteration termination requirement, Return to the step of annotating a group of data to be annotated by using the first target model and correcting the annotation results to obtain a group of newly annotated text data, until the new first target model meets the preset iteration termination requirement, and the new first target model is used as the target model.

10. The method according to claim 9, wherein: The adjusting the parameters of the first target model based on the current newly added annotated text data to obtain the second target model includes: The parameters of the first target model are adjusted based on the initial annotated text data, the current newly annotated text data and the auxiliary training data to obtain a second target model.

11. The method according to claim 9, wherein: The preset iteration termination requirement includes that evaluation indicators of the most recent preset number of new first target models are substantially the same, and the evaluation indicators are used to indicate the labeling effect of the new first target model.

12. The method according to claim 1, wherein: The obtaining of the current knowledge graph of the preset vertical field based on the initial annotated text data and the newly added annotated text data includes: Performing entity alignment on the initial annotated text data and the newly added annotated text data to identify and aggregate entities belonging to synonyms; According to the initial annotated text data and the newly added annotated text data after entity alignment, the current knowledge graph of the preset vertical field is obtained.

13. The method according to claim 1, wherein: The method further comprises: Get new data to be labeled; Based on the target model and the new data to be annotated, new newly added annotated text data is obtained; Based on the previously annotated text data and the newly added annotated text data, a new knowledge graph for the preset vertical field is obtained.

14. The method according to claim 1, wherein: The method further comprises: Obtain new auxiliary training data; The target model is adjusted based on the previous annotated text data, the previous auxiliary training data and the new auxiliary training data to obtain a new target model.

15. A device for constructing a vertical domain knowledge graph, the device comprising: A memory and a processor, wherein the memory is used to store a computer program; The processor is used to execute the computer program and implement the method for constructing a vertical field knowledge graph as described in any one of claims 1-14 when executing the computer program.

16. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor implements the method for constructing a vertical field knowledge graph as described in any one of claims 1 to 14.

Images