Knowledge graph labeling model generation method and system

Abstract

The embodiment of the invention provides a knowledge graph labeling model generation method and system. The method comprises the steps of obtaining first text data; generating a first regular expression based on the first text data; and generating a knowledge graph labeling model based on the first regular expression and the first text data.

Description

Technical Field

[0001] This specification relates to the field of machine learning, and in particular to a method and system for generating a knowledge graph annotation model. Background Art

[0002] Knowledge graph is a major visual data display method in the field of machine learning. It can intuitively display the relationship between multiple entities in a graphical way. In the process of knowledge graph construction, it is necessary to label data such as sample text, and the labeling accuracy and coverage will directly affect the effect of downstream tasks in the construction process, and then affect the accuracy and comprehensiveness of the knowledge graph portrait. At present, there are two main methods for text labeling in the process of knowledge graph construction, namely rule-based labeling and machine learning-based labeling. Rule-based labeling is labeling that uses symbolic rules to match text patterns (for example, regular expression matching method). Its rule formulation requires expert knowledge in related fields, and the coverage is low, and it cannot use labeled data for learning. Machine learning-based labeling is to use labeled data to learn a usable machine learning model, and use the machine learning model for labeling. In the case of less labeled data, the labeling effect is even worse than rule-based labeling, and its labeling results are less interpretable. Especially in the process of building knowledge graphs in multilingual scenarios, formulating rules or annotating corpora not only requires staff to have multilingual capabilities, but also the rule-making or corpus annotation process is time-consuming and labor-intensive; moreover, the knowledge graph labeling needs to have a certain degree of interpretability in order to explain and iterate the knowledge graph labeling results.

[0003] Therefore, it is hoped to provide a knowledge graph annotation model generation method to improve the data labeling quality and efficiency in the knowledge graph construction process. Summary of the invention

[0004] One of the embodiments of this specification provides a method for generating a knowledge graph annotation model. The method includes: obtaining first text data; generating a first regular expression based on the first text data; and generating a knowledge graph annotation model based on the first regular expression and the first text data.

[0005] In some embodiments, the first text data may be preprocessed, and the preprocessing may include at least one of normalization processing and word segmentation processing.

[0006] In some embodiments, keywords may be extracted from the first text data; the first text data may be annotated based on preset rules and the keywords to obtain first annotated data; and the keywords may be converted into the first regular expression based on the preset rules.

[0007] In some embodiments, the first annotated data can be used as a first training sample to pre-train the knowledge graph annotation model; the second annotated data can be used as a second training sample to optimize the training of the knowledge graph annotation model to obtain a trained knowledge graph annotation model.

[0008] In some embodiments, the second text data can be input into the trained knowledge graph annotation model to obtain the annotation results of the second text data.

[0009] In some embodiments, the first text data may include multiple levels of labels, and the method may further include: training labels at different levels in the multiple levels of labels through different training tasks.

[0010] In some embodiments, the method may further include: converting the trained knowledge graph annotation model into a second regular expression; and optimizing the trained knowledge graph annotation model based on the second regular expression.

[0011] In some embodiments, the knowledge graph annotation model may include a word vector component, a finite automaton recurrent neural network model component and a classifier component. The word vector component can be used to generate a first word vector based on the first text data, the finite automaton recurrent neural network model component can be used to generate a text feature vector, and the classifier component can be used to classify the text data.

[0012] In some embodiments, a first word vector can be generated through the word vector component based on the first text data; the finite automaton recurrent neural network model component can be generated based on the first regular expression and the first word vector; and the knowledge graph annotation model can be constructed based on the finite automaton recurrent neural network model component and the classifier component.

[0013] One of the embodiments of the present specification provides a knowledge graph annotation model generation system, including a data acquisition module, a regular expression generation module and a model generation module; the data acquisition module is used to acquire first text data; the regular expression generation module is used to generate a first regular expression based on the first text data; the model generation module is used to generate a knowledge graph annotation model based on the first regular expression and the first text data.

[0014] One of the embodiments of the present specification provides a knowledge graph annotation model generation device, including a processor, wherein the processor is used to execute the knowledge graph annotation model generation method.

[0015] One of the embodiments of the present specification provides a computer-readable storage medium, which stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the knowledge graph annotation model generation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0017] Figure 1 It is a schematic diagram of an application scenario of a knowledge graph annotation model generation system according to some embodiments of this specification;

[0018] Figure 2 is a schematic diagram of a knowledge graph annotation model generation system according to some embodiments of this specification;

[0019] Figure 3 is an exemplary flow chart of a method for generating a knowledge graph annotation model according to some embodiments of this specification;

[0020] Figure 4 is an exemplary flow chart of a method for generating a knowledge graph annotation model according to some embodiments of this specification;

[0021] Figure 5 It is a schematic diagram of a method for generating a knowledge graph annotation model according to some embodiments of this specification;

[0022] Figure 6 It is a schematic diagram of a method for generating a knowledge graph annotation model according to some embodiments of this specification;

[0023] Figure 7 is an exemplary flow chart of a method for generating a knowledge graph annotation model according to some embodiments of this specification;

[0024] Figure 8 is a schematic diagram of a knowledge graph annotation model according to some embodiments of this specification;

[0025] Fig. 9 It is a schematic diagram of a method for generating a knowledge graph annotation model according to some embodiments of this specification;

[0026] Fig. 10A , 10B is a schematic diagram of a multi-level dish category labeling system according to some embodiments of this specification;

[0027] Fig.11It is an exemplary flowchart of the knowledge graph annotation model generation method shown in some embodiments of this specification. DETAILED DESCRIPTION

[0028] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0029] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0030] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0031] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0032] Figure 1 1 is a schematic diagram of an application scenario of a knowledge graph annotation model generation system according to some embodiments of this specification. In this specification, the knowledge graph annotation model generation system 100 is referred to as system 100 hereinafter.

[0033] like Figure 1 As shown, in some embodiments, the system 100 may include a server 110 , a network 120 , a terminal 130 , and a storage device 140 .

[0034] The server 110 can process various information and data. In some embodiments, the server 110 can execute the knowledge graph annotation model generation method as shown in some embodiments of this specification to generate a knowledge graph annotation model based on sample text data, etc. In some embodiments, the server 110 can train the knowledge graph annotation model to obtain a trained knowledge graph annotation model. In some embodiments, the server 110 can mark the text data obtained from the terminal 130, etc. through the trained knowledge graph annotation model to obtain the annotation result. In some embodiments, the server 110 can exchange information and data with other components in the system 100 (e.g., terminal 130, storage device 140) through the network 120. For example, text data for model generation, model training, model marking, etc. are obtained from the storage device 140. In some embodiments, the server 110 may include multiple devices, wherein the same device or different devices may be used for model generation, model training, and model marking.

[0035] The server 110 is a system with computing and processing capabilities, and may include various computers, such as servers, personal computers, or a computing platform composed of multiple computers connected in various structures. The server 110 may include one or more sub-processing devices (e.g., a single-core processing device or a multi-core multi-core processing device), and the processing device may execute program instructions. As an example only, the processing device may include various common general-purpose central processing units (CPUs), graphics processing units (GPUs), microprocessors, application-specific integrated circuits (ASICs), or other types of integrated circuits.

[0036] The network 120 can connect the components of the system and / or connect the system with external resources. The network 120 enables communication between the components and with other parts outside the system to facilitate the exchange of data and / or information. In some embodiments, one or more components in the system 100 (e.g., server 110, terminal 130, storage device 140) can send data and / or information to other components via the network 120. In some embodiments, the network 120 can be any one or more of a wired network or a wireless network.

[0037] The terminal 130 can be used to interact with the user. In some embodiments, the terminal 130 can obtain text data input by the user, and these text data can be used as text data used by the server 110 to generate a model and / or train a model. In some embodiments, the terminal 130 can obtain the annotation results of the text data input by the user through the server 110. In some embodiments, the terminal 130 can be one of the mobile devices 130-1, tablet computers 130-2, laptop computers 130-3, desktop computers 130-4, and other devices with input and / or output functions, or any combination thereof.

[0038] The storage device 140 can store data or information generated by other devices. In some embodiments, the storage device 140 can store text data used as model training samples and / or model input (for example, text data input by a user obtained by the terminal 130, etc.). In some embodiments, the storage device 140 can store data and / or information processed by the server 110, for example, a generated knowledge graph annotation model, a trained knowledge graph annotation model, etc. The storage device 140 may include one or more storage components, each of which may be an independent device or part of other components in the system 100 (for example, the terminal 130, the storage device 140). The storage device may be local or implemented through the cloud.

[0039] It should be noted that the above description is provided for illustrative purposes only and is not intended to limit the scope of this specification. For those of ordinary skill in the art, under the guidance of the contents of this specification, various changes and modifications can be made. The features, structures, methods and other features of the exemplary embodiments described in this specification can be combined in various ways to obtain additional and / or alternative exemplary embodiments. For example, server 110 can be based on a cloud computing platform, such as a public cloud, a private cloud, a community and a hybrid cloud. However, these changes and modifications will not deviate from the scope of this specification.

[0040] Figure 2 It is a schematic diagram of a knowledge graph annotation model generation system according to some embodiments of this specification.

[0041] like Figure 2 As shown, in some embodiments, the knowledge graph annotation model generation system 200 may include a data acquisition module 210, a regular expression generation module 220, and a model generation module 230. In some embodiments, each module in the knowledge graph annotation model generation system 200 may be implemented by the server 110.

[0042] In some embodiments, the data acquisition module 210 may be used to acquire first text data.

[0043] In some embodiments, the data acquisition module 210 may pre-process the first text data, wherein the pre-processing may include at least one of normalization processing and word segmentation processing.

[0044] In some embodiments, the regular expression generation module 220 may be configured to generate a first regular expression based on the first text data.

[0045] In some embodiments, the regular expression generation module 220 may extract keywords from first text data; annotate the first text data based on preset rules and keywords to obtain first annotated data; and convert the keywords into a first regular expression based on preset rules.

[0046] In some embodiments, the model generation module 230 can be used to generate a knowledge graph annotation model based on the first regular expression and the first text data.

[0047] In some embodiments, the knowledge graph annotation model may include a word vector component, a finite automata-recurrent neural network (FA-RNN) model component, and a classifier component. Among them, the word vector component can be used to generate a first word vector based on the first text data, the finite automata-recurrent neural network model component can be used to generate a text feature vector, and the classifier component can be used to classify the text data.

[0048] In some embodiments, the model generation module 230 can generate a first word vector based on the first text data through a word vector component; generate a FA-RNN model component based on the first regular expression and the first word vector; and construct a knowledge graph annotation model based on the FA-RNN model component and the classifier component.

[0049] In some embodiments, the knowledge graph annotation model generation system 200 may further include a model training module 240. The model training module 240 may be used to train and / or optimize the generated knowledge graph annotation model to obtain a trained knowledge graph annotation model.

[0050] In some embodiments, the model training module 240 can train the knowledge graph annotation model in the following manner: use the first annotation data as the first training sample to pre-train the knowledge graph annotation model; use the second annotation data as the second training sample to optimize the training of the knowledge graph annotation model to obtain a trained knowledge graph annotation model.

[0051] In some embodiments, the first text data may include multiple levels of labels. For labels at different levels in the multiple levels of labels, the model training module 240 may perform training through different training tasks.

[0052] In some embodiments, the model training module 240 may convert the trained knowledge graph annotation model into a second regular expression; and optimize the trained knowledge graph annotation model based on the second regular expression.

[0053] In some embodiments, the knowledge graph annotation model generation system 200 may further include a data annotation module 250. The data annotation module 250 may be used to input the second text data into the trained knowledge graph annotation model to obtain the annotation result of the second text data.

[0054] Figure 3 It is an exemplary flowchart of the knowledge graph annotation model generation method shown in some embodiments of this specification.

[0055] like Figure 2 As shown, the process 200 includes the following steps: In some embodiments, the process 200 may be executed by the server 110 .

[0056] Step 310 , obtaining first text data. In some embodiments, step 310 may be performed by the data obtaining module 210 .

[0057] The first text data is sample data that is not labeled (labeling means annotating, referred to as labeling), which is used to generate and train the knowledge graph annotation model, and may include various types of text information and / or data, such as text files, tables, etc. In some embodiments, the server 110 may obtain text information and / or data from other files such as images, audio and video files as the first text data. For example, the text content in an image and / or video is obtained through image recognition. For another example, a voice file is converted into text through voice recognition. In some embodiments, the server 110 may obtain the first text data from various types of data warehouses and / or databases, such as hive tables, MySQL, Sybase, MongoDB, Redis, etc.

[0058] The second text data is data to be annotated, and may include various data containing text information, such as text files, tables, etc. In some embodiments, the second text data may be acquired in the same or similar manner as the first text data.

[0059] In some embodiments, the server 110 may preprocess the first text data, and use the preprocessed first text data for processing in subsequent steps. The preprocessing of the first text data may include at least one of normalization processing, word segmentation processing, etc. In some embodiments, the normalization processing may include one or more processing of removing accented characters, normalizing upper and lower cases, etc. In some embodiments, the word segmentation processing may be performed by various word segmentation algorithms, models, and tools (e.g., Natural Language Toolkit (NLTK), Jieba Chinese word segmentation, etc.).

[0060] In some embodiments, server 110 may use different preprocessing methods for different types of text. For example, for texts in Chinese or Japanese, word segmentation can be performed directly. For another example, for texts in alphabetic languages ​​such as English, Spanish, and Portuguese, normalization can be performed first, that is, accented characters are removed and case normalized, and then the normalized text is segmented using word segmentation tools such as NLTK. Just as an example, taking the Brazilian dish category labeling in a takeaway scenario as an example, the text content before and after the normalization of the name, menu, and description text information of a dish can be shown in Table 1: Table 1 Brazilian cuisine information and normalization results

[0061] Step 320 , generating a first regular expression based on the first text data. In some embodiments, step 320 may be performed by the regular expression generation module 220 .

[0062] In some embodiments, the server 110 may extract keywords from the first text data. The keywords may include multiple types of keywords, such as black and white list keywords, etc. In some embodiments, the keywords may be represented in various forms, such as a list of multiple keywords, etc. In some embodiments, the keywords may be matched with tags, wherein the tags may represent a certain type of thing, and the keywords matched with the tags may be used to represent or characterize whether it is a specific thing in the class of things. For example, for the category tag "food", the matching keywords may be "rice", "noodles", etc.

[0063] In some embodiments, the server 110 can formulate preset rules based on keywords, and the preset rules are used to determine which labels should be added to the text. For blacklist and whitelist keywords, the preset rules may include: for a certain category label, (1) if the text information of a text sample contains keywords that match the whitelist keywords of the label, the text sample can be marked with the label; (2) if the text information of a text sample contains keywords that match the blacklist keywords of the label, the text sample cannot be marked with the label; (3) if the text information of a text sample contains keywords that match both the whitelist keywords and the blacklist keywords of a category label, the sample cannot be marked with the label. Taking the labeling of dish categories in a takeaway scenario as an example, for dish categories label, assuming that its whitelist keywords are: sake, vinhojapones, vinho japones, saque; its blacklist keywords are: sushi, then if the text information of text sample 1 contains sake but does not contain sushi, then text sample 1 is labeled If the text sample 1 contains sushi but does not contain sake, vinhojapones, vinhojapones, or saque, the text sample will not be labeled If the text sample 1 contains both sake and sushi, the text sample will not be labeled Label.

[0064] In some embodiments, matching a keyword in the black and white list keywords may include being identical to the keyword in the black and white list keywords, or matching a specific pattern of the black and white list keywords, for example, any one or more of first letter matching, first and last matching, first word matching, containing a specified string, etc., as long as it can be converted into a regular expression by a tool or manually.

[0065] In some embodiments, the server 110 may extract keywords from the first text data by manually generating, generating based on an algorithm, generating based on a model, etc., or any combination thereof. Hereinafter, the keyword is described as a blacklist or whitelist keyword.

[0066] In some embodiments, the server 110 may observe and analyze text features of labeled samples and extract a number of keywords based on manual experience to generate blacklist and whitelist keywords.

[0067] In some embodiments, the server 110 can automatically generate keywords through algorithms and / or models based on the correlation between words and tags, for example, through algorithms such as term frequency-inverse document frequency (TF-IDF), through models such as bidirectional encoder representation technology (BERT) based on transformers. Among them, the keywords generated in this way may include whitelist keywords. Experiments have shown that in the process of knowledge graph labeling, whitelist keywords play a major role. Therefore, only obtaining whitelist keywords will not have a significant impact on the labeling effect of the knowledge graph.

[0068] TF-IDF can be used to evaluate the importance of a word to a document set or a document in a corpus. The importance of a word can be represented by its TF-IDF score, which increases in proportion to the number of times it appears in a document, but decreases inversely with the frequency of its appearance in the corpus, that is, the TF-IDF score of a word = word frequency * word inverse document frequency. For example, the total number of words in a text file in a text set is 100, and the word "cow" appears 3 times, then the word frequency of the word "cow" in the file is 3 / 100 = 0.03. The method of calculating the document frequency is to divide the total number of documents in the document set by the number of documents in which the word "cow" appears. So, assuming that the word "cow" appears in 1,000 documents and the total number of documents is 10,000,000, its inverse document frequency is lg(10,000,000 / 1,000) = 4, and the final TF-IDF score is 0.03*4 = 0.12.

[0069] In some embodiments, the server 110 can generate whitelist keywords based on TF-IDF. Specifically, for each preset tag, the server 110 can calculate the TF-IDF score corresponding to each word in each text sample corresponding to the tag in the first text data, wherein the higher the TF-IDF score, the more important the word is in the sample, and the more it can represent the characteristics of the sample. After obtaining the TF-IDF score corresponding to each word, the server 110 can calculate the mean of the TF-IDF scores of each word under each tag, and extract several words with larger TF-IDF scores as whitelist keywords for the sample. In some embodiments, when there are duplicate keywords under any two tags, the server 110 can retain the keyword under the tag with the larger score and remove the keyword under the tag with the smaller score according to the respective TF-IDF scores of the duplicate keyword under the two tags, so that there are no duplicate keywords in any two tags.

[0070] For each whitelist keyword under a label category, it should represent a word that is closely related to the label word. If a word can be used to replace the label word, it means that the word is closely related to the label word and can be used as a whitelist keyword for the label category. The Masked Language Modeling (MLM) training method in the pre-trained model BERT masks a certain word to predict which word the masked word may be, or which word the masked word can be replaced by.

[0071] In some embodiments, the server 110 can generate whitelist keywords based on BERT in a variety of ways. For example, using at least one of a tag word and a non-tag word. A tag word refers to a word used as a tag, and a non-tag word refers to a word not used as a tag. For example, pizza can be used as a tag word, and pizzag can be used as a non-tag word.

[0072] In some embodiments, the server 110 can use the label words to generate corresponding keywords through BERT. Specifically, the server 110 can mask the position containing the label words in the corpus (for example, the first text data), use the encoder in BERT and its classifier head during MLM pre-training to predict the masked position, and obtain the most likely multiple (for example, 50, 60, etc.) replacement words for the position, where the number of replacement words can be recorded as N; then all possible replacement words for all label words in the entire corpus are counted, and the most likely multiple (for example, 50, 60, etc.) replacement words for each label word are selected in descending order according to the frequency of the replacement words as the whitelist keywords of the label, where the number of replacement words can be recorded as M, where N≥M. For example, if all the corpora contain 50 sentences in total, then for each tag, each sentence is used as a corpus, and 50 replacement keywords are obtained. For all the corpora, a total of 50*50=2500 replacement keywords are obtained. The frequency of these 2500 words is counted, and the top 50 replacement keywords are taken as the whitelist keywords of the tag.

[0073] Figure 5 This is a schematic diagram of a method for obtaining whitelist keywords using tag words. Figure 5 As shown, it is assumed that the label words in the first text data include pastel, de, pizza, etc., where label 2 can correspond to pizza, and label 1 and label 3 correspond to other label words one by one. The server 110 can input the first text data and these label words into the BERT Encoder, and obtain 50 keywords for each label through the MLM training method, and these keywords are whitelist keywords. Taking the label word pizza as an example, the server 110 can mask the position where the label word pizza exists in each text in the first text data, and then use the encoder in BERT (i.e., BERT Encoder) and MLM to predict the masked position to obtain the 50 most likely replacement words for the position; then all possible replacement words for the label word pizza of all texts in the first text data are counted, and the frequency of the replacement words is counted, and the top 50 replacement words are taken in descending order as the keywords of label 2.

[0074] In some embodiments, the server 110 can use non-label words to generate corresponding keywords through BERT. Specifically, based on the above-mentioned use of label words to generate corresponding keywords through BERT, the server 110 can further mask and predict all words in the corpus. For each masked word, among the N replacement words obtained, if P words appear in the N keywords of a certain label screened in the previous step, then the masked word is considered to be similar to the label and can be used as a keyword for the label, where P≤N, for example, if N is 50, then P can be any one of 20, 30, 40, etc.

[0075] Figure 6 A schematic diagram of a method for obtaining whitelist keywords using non-label words. Figure 6 The method shown in the figure uses non-label words to obtain whitelist keywords. Figure 5 The above is performed based on the results of obtaining whitelist keywords using tag words. Figure 5 Similarly, assume that the label words in the first text data include pastel, de, pizza, etc., where label 2 can correspond to pizza, label 1 and label 3 correspond to other label words one by one, and pizzag is a word that is not marked as a label word in the first text data. Taking the word pizzag as an example, the server 110 can mask the position where the word pizzag exists in each text in the first text data, and then use BERT Encoder and MLM to predict the masked position to obtain the 50 most likely replacement words for the position. Since 20 of these replacement words are consistent with the keywords obtained by label 2 ( Figure 5 If “pizzag” is repeated in the 50 keywords of label 2 obtained in , it is considered that “pizzag” can be used as the keyword of label 2 (corresponding to the label word “pizza”).

[0076] In some embodiments of the present specification, keywords are automatically generated by using algorithms and / or models (eg, TF-IDF, Bert, etc.) based on the correlation between words and tags, making the generation of keywords more automated and convenient.

[0077] In some embodiments, the server 110 may annotate the first text data based on preset rules and keywords extracted from the first text data to obtain first annotated data. Taking the case where the keywords are black and white list keywords and the preset rules are rules that match the black and white list keywords as an example, the server 110 may mark (i.e., annotate) the text in the first text data based on the black and white list keywords and the rules that match the black and white list keywords, and use the marked text as the first annotated data. In some embodiments, the method of marking text data may include: one of the following, manually, automatically through an automated tool, or through model processing, or any combination thereof. In some embodiments, the black and white list keywords of each tag may include a list of words, and the server 110 may write the black and white list keywords in the form of a configuration file. When subsequently marking based on the black and white list keywords, the server 110 may replace the configuration file.

[0078] In some embodiments, the server 110 may also obtain second annotation data, wherein the second annotation data may include annotated text, etc., and its annotation quality is better than the first annotation data. In some embodiments, the server 110 may obtain the second annotation data in various ways, such as manual annotation, etc. In some embodiments, when the second annotation data is manually generated, the amount of the first annotation data may be more than the second annotation data. In some embodiments, the amount of the first annotation data may also be less than or equal to the second annotation data, and this specification does not limit this.

[0079] In some embodiments, the server 110 can convert the keywords extracted from the first text data into a first regular expression based on preset rules. A regular expression describes a string matching pattern, which can be used to check whether a string contains a certain substring, replace a matching substring, extract a substring that meets a certain condition from a string, etc. Since the logic of the black and white list keywords and the preset rules that match them are relatively simple, they can be easily converted into regular expressions. Taking the example of the keywords being the black and white list keywords and the preset rules being the rules that match the black and white list keywords, the server 110 can convert the extracted black and white list keywords into regular expressions based on the rules that match the black and white list keywords. Just as an example, for the aforementioned takeaway scenario, when labeling the dish categories The whitelist keywords of the tag can be converted into the following regular expression: $*(sake|vinhojapones|vinho japones|saque)$*, where "$" represents any character, "*" represents matching the preceding sub-expression zero or more times, "()" represents the start and end positions of the sub-expression, and "|" represents a choice between two adjacent items. In some embodiments, the method of converting keywords into regular expressions may include: manually, automatically, through an automated tool, through model processing, etc., or any combination thereof.

[0080] In some embodiments of the present specification, by labeling based on black and white list keywords, a large number of preliminary labeling results can be quickly obtained for pre-training of subsequent models, thereby improving the final labeling accuracy of the subsequent models.

[0081] Step 330 , generating a knowledge graph annotation model based on the first regular expression and the first text data. In some embodiments, step 330 may be performed by the model generation module 230 .

[0082] The knowledge graph annotation model is a model for annotating text data to generate a knowledge graph based on the annotated text data, and may include various machine learning models, for example, a Recurrent Neural Networks (RNN) model, a Convolutional Neural Network (CNN) model such as TextCNN, a BERT model, etc. In some embodiments, the knowledge graph model may include a FA-RNN model, and the server 110 may generate the knowledge graph annotation model based on the first regular expression and the first text data.

[0083] In some embodiments, the input of the knowledge graph annotation model may include various text information (e.g., first text data, etc.) and preset label information, and the output may include the labeled input text.

[0084] In some embodiments, the knowledge graph annotation model may include a word vector component, a finite automaton recurrent neural network (FA-RNN) model component, and a classifier component. The word vector component may be used to generate a first word vector based on the first text data, the finite automaton recurrent neural network model component may be used to generate a text feature vector, and the classifier component may be used to classify the text data.

[0085] In some embodiments, the server 110 can generate a first word vector through a word vector component according to the first text data. In some embodiments, the word vector component can obtain word vector information from the first text data through a word vector model (for example, a GloVe (Global Vectors for Word Representation) model, a BERT model, etc.) to generate a first word vector. For example, in the Brazilian dish category labeling scenario in the aforementioned takeaway scenario, assuming that the first text data is the text content in Table 1, the processed text content in Table 1 can be segmented, and the three types of information after segmentation, namely, the dish name, dish menu, and dish description, can be input into the word vector model to obtain the output three word vectors, and the three word vectors are weighted and summed to obtain the word vector information of the first text data, that is, the first word vector; or directly splice these three types of information and input them into the word vector model to obtain the first word vector.

[0086] In some embodiments, the server 110 can generate a FA-RNN model component based on the first regular expression and the first word vector. Specifically, the server 110 can generate a FA-RNN model based on the first regular expression, and add the first vector information when generating the model. Since the FA-RNN model is converted from the regular expression of keywords (for example, black and white list keywords), the FA-RNN model has already incorporated preset rules for matching keywords during the initialization phase, so that the FA-RNN can be trained using the annotated data later, thereby making full use of the rule knowledge and annotated data for training and obtaining a knowledge graph annotation model with better results. By adding word vector information to the FA-RNN model, the quality of the output results of the FA-RNN model can be improved, thereby improving the labeling effect. For more information on how to generate a FA-RNN model based on regular expressions, please refer to Fig. 9 The relevant description will not be repeated here.

[0087] In some embodiments, the server 110 may construct a knowledge graph annotation model based on the FA-RNN model component and the classifier component. In some embodiments, the input of the FA-RNN model component may include various text information and preset tag information, and the output may include a vector representing the input text features.

[0088] In some embodiments, the FA-RNN model component may be composed of a FA-RNN model corresponding to a whitelist and a FA-RNN model corresponding to a blacklist. The server 110 may extract the corresponding black and white list keywords for each tag category, and write out the first regular expression and convert it to obtain the FA-RNN model according to the whitelist and blacklist keyword lists, that is, obtain the FA-RNN model corresponding to the whitelist and the FA-RNN model corresponding to the blacklist. In some embodiments, the server 110 may input the text information and the preset tag information into the FA-RNN model corresponding to the whitelist and the FA-RNN model corresponding to the blacklist, respectively, and obtain the output vectors under the FA-RNN model corresponding to the whitelist and the FA-RNN model corresponding to the blacklist, which may be recorded as r_white and r_black, respectively.

[0089] In some embodiments, the server 110 may fuse the vectors r_white and r_black respectively output by the FA-RNN model corresponding to the blacklist and whitelist to obtain a final vector r that can characterize the input text features, as shown in the following formula: Among them, p and q represent the elements in the vector.

[0090] In some embodiments, the classifier component can be used as a labeling classifier to output the scores of the text information on each label, and sort them according to the scores, and label the text information with the label that is ranked first, so as to obtain the labeling result, wherein the labeling result includes the labeled text. The classifier component may include various models or tools that can be used for information classification, for example, artificial neural networks (ANN) such as multilayer perceptrons (Multilayer Perceptron, MLP). In some embodiments, the server 110 can be connected to the classifier component (for example, at least one MLP layer) after the FA-RNN model to obtain a knowledge graph labeling model. In some embodiments, when used to label text data, that is, using a knowledge graph labeling model, the server 110 can be connected to an MLP layer after the FA-RNN model. In some embodiments, when used for training a knowledge graph labeling model, especially for training a multi-level labeling system, the server 110 can be connected to at least two MLP layers after the FA-RNN model to train different levels of labels for the text information respectively.

[0091] Figure 8 Schematic diagram of a knowledge graph annotation model according to some embodiments of this specification. Figure 8As shown, the knowledge graph annotation model 820 may include a FA-RNN model 821 and an MLP layer 822, wherein the FA-RNN model 821 corresponds to the FA-RNN model component, and the MLP layer 822 corresponds to the classifier component. The FA-RNN model 821 is generated by converting the regular expression 830, and word vector information 840 is added during the generation process, wherein the word vector information 840 can be generated by the first text data through the word vector component. The workflow of the knowledge graph annotation model 820 may include: the server 110 inputs the preset tag information and the text information 810 into the FA-RNN model component 821, and then obtains the output vector representing the features of the text information 810, and after the vector is processed by the MLP layer 822, the label score 850 of the text information 810 on each preset label is obtained, and according to the label score 850, the corresponding label with the highest score is taken to label the text information 810, thereby obtaining the annotated text of the text information 810, that is, the annotation result.

[0092] In some embodiments, the step of annotating the first text data based on preset rules and keywords to obtain the first annotated data, and the step of converting the keywords into the first regular expression based on the preset rules, can be performed in any order, that is, they can be performed simultaneously, or any one of the two steps can be performed first and the other one can be performed later.

[0093] In some embodiments, some or all of the steps in the aforementioned steps 310-330 until the knowledge graph annotation model is generated can be performed by a generation model (e.g., a machine learning model, etc.) or an automatic generation tool (e.g., a program script, etc.). For example, the server 110 can input the first text data and the preset tag information into the generation model, and the generation model automatically completes a series of operations such as normalizing the first text data, generating a first regular expression, and generating a knowledge graph annotation model, and then obtains the output knowledge graph annotation model. For another example, the server 110 can input the normalized first text data and the preset tag information into the generation model, and the generation model automatically completes a series of subsequent operations of generating a knowledge graph annotation model.

[0094] In some embodiments, the training samples of the knowledge graph annotation model may include various annotated data (for example, first annotated data, second annotated data, etc.), the training labels are the annotations in the training data, and the output is the annotation results of the input training samples.

[0095] In some embodiments, the server 110 can use the first labeled data as the first training sample to pre-train the knowledge graph annotation model to obtain a pre-trained knowledge graph annotation model; and then use the second labeled data as the second training sample to optimize the pre-trained knowledge graph annotation model to obtain a trained knowledge graph annotation model.

[0096] In some embodiments of the present specification, a large amount of labeled data is first obtained by labeling using black and white list keywords and preset rules, and the generated knowledge graph labeling model is pre-trained. Then, the optimized training is performed using manually labeled data with better quality. Since the quality of the labeled data used for optimized training is better than that of the labeled data used for initial training, the knowledge graph model obtained after two stages of training has better adaptability to various data and higher labeling accuracy.

[0097] Since the FA-RNN model in the trained knowledge graph annotation model is converted from regular expressions and trained, the trained FA-RNN model can also be converted back to regular expressions. Since the trained FA-RNN model has better data annotation effect, and the regular expression is more interpretable and understandable than the FA-RNN model itself, after converting the trained FA-RNN back to regular expressions, the calculation logic of the current FA-RNN can be analyzed based on the regular expression, and the regular expression can be added or deleted accordingly to optimize the knowledge graph annotation model.

[0098] In some embodiments, the server 110 may convert the trained knowledge graph annotation model into a second regular expression, and optimize the trained knowledge graph annotation model according to the second regular expression. Specifically, the server 110 may obtain the calculation logic of the current FA-RNN (i.e., the FA-RNN model in the trained knowledge graph annotation model) according to the second regular expression (for example, what parameters are added or reduced in the expression, what parameters are modified, how to modify, etc.); adjust the first regular expression or the second regular expression according to the obtained calculation logic (for example, add or delete regular expressions, modify parameters in regular expressions, etc.), and then regenerate the knowledge graph annotation model according to the adjusted first regular expression or second regular expression. The above optimization process can be performed iteratively. For information on how to convert the knowledge graph annotation model into the second regular expression, please refer to Fig. 9 The relevant description will not be repeated here.

[0099] In some embodiments of the present specification, the conversion between regular expressions and FA-RNN models facilitates understanding of the computational logic of the current model and enables more efficient optimization of the knowledge graph annotation model.

[0100] When the knowledge graph labeling system is structured (for example, a tree structure), for the data that needs to be labeled, due to the existence of multi-level labels, the generated knowledge graph annotation model is prone to overfitting, that is, the labeling result cannot distinguish between multi-level labels. Fig. 10A ,10B is a schematic diagram of a multi-level dish category label system according to some embodiments of this specification, Fig. 10A , 10B The following is a three-level label system for food categories in a takeaway scenario, which is a hierarchical system in a tree diagram. Fig. 10A As shown in the figure, the dish category label system includes: 4 L1-level labels, 14 L2-level labels, and 98 L3-level labels. The labels at different levels in the dish category label system have a parent-child relationship. Fig. 10B Shown Fig. 10A In the L1-level label "beverage" and its L2 and L3-level labels, "beverage" (L1 label) is the parent label of "alcoholic beverage" (L2-level label), "alcoholic beverage" (L2-level label) is the parent label of "beer" (L3-level label), and "beer" is a leaf label. In the takeaway scenario, the category label of the dish is generally directly marked on the leaf label (for example, Fig. 10B Then, through the mapping relationship between the leaf label and the parent label, it can be traced back to the corresponding parent label.

[0101] In some embodiments, when the first text data contains multiple levels of labels, the server 110 can train labels of different levels in the multiple levels of labels through different training tasks. Specifically, when performing model training, the server 110 can connect at least two MLP layers after the FA-RNN model of the knowledge graph annotation model, and perform parallel training on different levels of labels for the text data (for example, Fig. 10A , 10B The L2 and L3 level labels in the training are trained separately), where one MLP layer is used to train one level of labels. This multi-task training method can avoid overfitting of the knowledge graph annotation model and has better adaptability to the structured knowledge graph labeling system, thereby improving the labeling accuracy of the model.

[0102] In some embodiments, after the training is completed, the server 110 can save the trained knowledge graph annotation model to a storage device, for example, to the storage device 140 .

[0103] In some embodiments, the server 110 may input the second text data into a trained knowledge graph annotation model to obtain an annotation result of the second text data, wherein the second text data is data to be annotated.

[0104] In some embodiments, before inputting the second text data into the trained knowledge graph annotation model, the server 110 may preprocess the second text data, wherein the preprocessing method may be the same as or similar to the preprocessing method for the first text data.

[0105] Figure 4 It is an exemplary flowchart of the knowledge graph annotation model generation method shown in some embodiments of this specification.

[0106] like Figure 4 As shown, the process 400 may include the following steps. In some embodiments, the process 400 may be executed by the server 110 .

[0107] Step 410 , obtaining marking input information. In some embodiments, step 410 may be performed by the data acquisition module 210 .

[0108] The labeling input information is data used to generate the knowledge graph annotation model, which can be the first text data. The specific operation of step 410 is similar to that of step 310, and the description of step 310 can be referred to, which will not be repeated here.

[0109] In some embodiments, after step 410, steps 420-440 may be performed by the regular expression generation module 220. Steps 430 and 440 may be performed in any order, i.e., simultaneously; or step 430 may be performed first and then step 440; or step 430 may be performed first and then step 430.

[0110] Step 420, generating blacklist and whitelist keywords.

[0111] In some embodiments, the server 110 may generate blacklist and whitelist keywords according to the marking input information, wherein the blacklist and whitelist keywords may be keywords extracted from the first text data. For more information on how to generate blacklist and whitelist keywords, please refer to the relevant content of step 320, which will not be repeated here.

[0112] Step 430, generate a regular expression.

[0113] In some embodiments, the server 110 may convert the blacklist and whitelist keywords generated in step 420 into a regular expression according to a preset rule, wherein the preset rule matches the blacklist and whitelist keywords, and the regular expression may be a first regular expression. For more information on how to generate a regular expression, see step 320 and Fig. 9 The relevant content will not be repeated here.

[0114] Step 440: tagging based on the blacklist and whitelist keywords.

[0115] In some embodiments, the server 110 may annotate the input information for marking obtained in step 410 based on preset rules and blacklist and whitelist keywords to obtain annotated data for the input information for marking, wherein the annotated data may be first annotated data. For more information on how to annotate the input information for marking, please refer to the relevant description of step 320, which will not be repeated here.

[0116] Step 450 , constructing a model. In some embodiments, step 450 may be performed by the model generation module 230 .

[0117] In some embodiments, the server 110 can generate a knowledge graph annotation model based on the regular expression generated in step 430 and the labeling input information obtained in step 410, wherein the knowledge graph annotation model can be constructed by a FA-RNN model (i.e., a FA-RNN model component) and an MLP layer (i.e., a classifier component). Specifically, the server 110 can obtain word vector information (i.e., a first word vector) from the labeling input information; convert the regular expression to generate a FA-RNN model, and add the word vector information during the FA-RNN model generation process; and connect the MLP layer to the FA-RNN model to obtain a knowledge graph annotation model. For more information on how to build the model, please refer to the relevant description of step 330, which will not be repeated here.

[0118] Step 460 , training the model. In some embodiments, step 460 may be performed by the model training module 240 .

[0119] In some embodiments, the server 110 can use the labeling data of the labeling input information generated in step 440 as a training sample (i.e., the first training sample) to pre-train the knowledge graph labeling model generated in step 450, and then use the labeling data obtained by manual training (i.e., the second labeling data) as a training sample (i.e., the second training sample) to optimize the pre-trained knowledge graph labeling model, thereby obtaining a trained knowledge graph labeling model. For more information on how to train the model, please refer to the relevant description of step 330, which will not be repeated here.

[0120] Step 470 , model reasoning. In some embodiments, step 470 may be performed by the data annotation module 250 .

[0121] Model reasoning is to use the knowledge graph annotation model to obtain the annotation results of the data to be annotated, that is, to label the data to be annotated. In some embodiments, the server 110 can input the data to be annotated into the trained knowledge graph annotation model obtained in step 460 to obtain the annotation results corresponding to the data to be annotated, that is, the data annotated with the label of the data to be annotated. Among them, the data to be annotated can be the second text data. For more information on how to use the knowledge graph annotation model to label the data to be annotated, please refer to the relevant description of step 330, which will not be repeated here.

[0122] Step 480, model interpretation. In some embodiments, step 480 may be performed by the model training module 240. In some embodiments, step 480 may be performed at any time after step 460, for example, before step 470.

[0123] Model interpretation is to understand the parameters of the knowledge graph annotation model based on the regular expression converted from the knowledge graph annotation model, so as to optimize and adjust the knowledge graph annotation model. In some embodiments, the server 110 can convert the trained knowledge graph annotation model obtained in step 460 into a regular expression (i.e., a second regular expression), specifically, the FA-RNN model in the knowledge graph annotation model is converted into a regular expression; the server 110 can optimize the trained knowledge graph annotation model according to the obtained regular expression. Among them, the above process can be performed iteratively. For more information on how to optimize the knowledge graph annotation model based on the regular expression converted from the trained knowledge graph annotation model, please refer to the relevant description of step 330, which will not be repeated here.

[0124] Figure 7 It is an exemplary flowchart of the knowledge graph annotation model generation method shown in some embodiments of this specification.

[0125] like Figure 7 As shown, process 700 may include the following steps. In some embodiments, process 700 may be executed by server 110. In some embodiments, by executing the steps shown in process 700, the operation of annotating the first text data based on the preset rules and keywords to obtain the first annotated data described in step 320 may be implemented.

[0126] Step 710 , obtaining marking input information. In some embodiments, steps 710 and 720 may be performed by the data acquisition module 210 .

[0127] In some embodiments, the server 110 may obtain the marking input information, wherein the marking input information is sample data without a label, which may be the first text data. For more information on how to obtain the marking input information, please refer to the relevant description of step 310, which will not be repeated here.

[0128] Step 720: text preprocessing.

[0129] In some embodiments, the server 110 may preprocess the marking input information obtained in step 710, wherein the preprocessing may include at least one of normalization processing, word segmentation processing, etc. For more information on how to preprocess the data, please refer to the relevant description of step 310, which will not be repeated here.

[0130] In some embodiments, after step 720 , steps 730 - 750 may be performed by the regular expression generation module 220 to annotate the labeling input data.

[0131] Step 730, read the blacklist and whitelist keywords.

[0132] In some embodiments, the server 110 can read the blacklist and whitelist keywords, ie, the keywords corresponding to the first annotation data, from the preprocessed marking input information. For how to obtain the content of the blacklist and whitelist keywords, please refer to the relevant description of step 320, which will not be repeated here.

[0133] Step 740: Text matching and tagging.

[0134] In some embodiments, the server 110 may annotate the label input information based on the blacklist and whitelist keywords obtained in step 730 and the preset rules matching the blacklist and whitelist keywords, that is, label the label input information with a label matching the blacklist and whitelist keywords. Thus, the data after the label input information is annotated is obtained, that is, the first annotated data. For more information on how to annotate the label input information, please refer to the relevant description of step 320, which will not be repeated here.

[0135] Step 750: output the marking result.

[0136] In some embodiments, the server 110 may output the data obtained in step 740 after annotating the labeled input data to a storage device, or directly output it to a word vector model (for example, a GloVe model, a BERT model, etc.) to obtain word vector information for generating a knowledge graph annotation model, i.e., a first word vector.

[0137] Fig. 9 is a schematic diagram of a method for generating a knowledge graph annotation model according to some embodiments of this specification. In some embodiments, Fig. 9The method shown can be performed by the server 110 or the model generation module 230. In some embodiments, by executing Fig. 9 The method shown in FIG. 1 can implement the operation of generating a FA-RNN model based on the first regular expression and the first word vector in step 330 and step 450, wherein the first regular expression can include Fig. 9 In the regular expression 910, the FA-RNN model may include Fig. 9 The RNN930 in .

[0138] like Fig. 9 As shown, in some embodiments, the processor 110 can convert the regular expression 910 into a weighted finite state automaton (WFSA or WFA) with equivalent semantics through an algorithm (e.g., Thompson's Construction Algorithm, etc.), that is, WFA 920. WFA is a finite state automaton (FSA or FA) with only 0 and 1 weight values. In some embodiments, WFA 920 can be expressed by the following formula: A=(Σ,S,T,α0,α ∞ ) (2) Among them, Σ represents the input vocabulary set, with a total of V words; S represents the state set, with a total of K states; is the state transfer weight tensor, T[σ,i,j] means when the input is σ, from state s i Transfer to state s j The weight of , that is, T[σ] represents the transfer matrix of word σ; is the initial state weight, that is, α0[i] is when t=0, s i The weight value at , the set of initial states is recorded as S0; is the terminal state weight, i.e., α ∞ [i] means after reading all the input, s i The weight value at the end state is recorded as S ∞ .

[0139] In some embodiments, for an input sequence x, each word in the sequence x is sequentially input into path p, where p = (u1, u2, ..., u N+1 ), then the score B(A,p) corresponding to the input sequence x can be expressed as follows: Among them, u i Represents the point on path p as an index, i is the subscript, 1≤i≤N+1.

[0140] In some embodiments, based on formula (3), let π(x) be the state from the initial state s0, after all words in the x sequence are input into the path p to reach the terminal state s i The set of all paths where s0∈S0, s i ∈S ∞ , then the total path score B can be obtained by forward algorithm calculation forward (A,x), as shown in the following formula: Wherein, T has the same meaning as in formula (2); T[x i ] is similar to T[σ] in formula (2), where x i This is equivalent to σ in formula (2).

[0141] In some embodiments, formula (4) can be converted into a cyclic form, that is, an RNN form, and the RNN obtained after the conversion can be recorded as O = (α0, T, α ∞ ), which can be expressed by the following formula: The meaning of each parameter is the same as the corresponding parameter in formula (2) to formula (4).

[0142] like Fig. 9 As shown, in some embodiments, in order to reduce parameters, the server 110 can obtain three matrices E from the WFA 920. R Specifically, the server 110 can convert the formula (2) into Decomposed into three matrices E R , D1, D2, where Then formula (4) can be transformed into the following formula: in, Represents bitwise multiplication of elements.

[0143] In some embodiments, a word vector (i.e., the first word vector) can be added to the RNN shown in formula (6). Specifically, let E w is the external word vector matrix, u t is word x t In E w The word embedding vector in , where We can introduce a matrix G to embed a D dimension into the vector u t Convert to r dimension to replace v t ,in, When introducing the matrix G, we can introduce a hyperparameter β to control v tand u t The weight of , then formula (6) can be transformed into the following formula:

[0144] like Fig. 9 As shown, in some embodiments, by adding word vectors, the server 110 can R , D1, and D2, we get the RNN shown in formula (6), and then by adding the word vector to the RNN, we get RNN930, which is the RNN shown in formula (7), where RNN930 is converted from FA and is therefore FA-RNN.

[0145] In some embodiments, the server 110 may reconvert the FA-RNN into a regular expression, for example, by converting the FA-RNN model component in the trained knowledge graph annotation model described in process 300 into a second regular expression.

[0146] In some embodiments, the server 110 may use the parameters O in the trained FA-RNN. RE and the external word vector matrix E introduced in the above formula (7) w , reconstructing the tensor T in the WFA (e.g., WFA920), where WFA can be shown as formula (2), where T has the same meaning as T in formula (2). The process of reconstructing the tensor T can be shown as follows:

[0147] in, is the reconstructed vector T; ⊙ represents the Khatri-Rao product; β is the weight, which can be obtained by using a threshold function to convert the weights to 0-1 values ​​(i.e., the weight value is 0 or 1), where γ is a fixed scalar.

[0148] In some embodiments, the server 110 may convert the FA-RNN into a WFA; convert the WFA into a FA by using the above formulas (8) and (9); and perform a similar process on the weights in the soft aggregation layer as in formula (8) (i.e., convert the weights into 0-1 values ​​by using a threshold function) to reconstruct the logical aggregation layer. The above process may be used to implement the operations of converting the trained FA-RNN back into the second regular expression in steps 330 and 480. In some embodiments, the server 110 may use any existing method to implement any step in the process of converting the FA-RNN into a regular expression, for example, the step of converting the FA-RNN into a WFA, and this specification does not limit this.

[0149] Fig.11 It is an exemplary flowchart of the knowledge graph annotation model generation method shown in some embodiments of this specification.

[0150] like Fig.11 As shown, process 1100 may include the following steps. In some embodiments, process 1100 may be executed by server 110. In some embodiments, by executing the steps shown in process 1100, the model reasoning operation shown in process 300 and step 470 may be implemented, that is, the operation of obtaining the annotation result of the second text data by inputting the trained knowledge graph annotation model through the second text data.

[0151] Step 1110 , obtaining data to be marked. In some embodiments, steps 1110 and 1120 may be performed by the data acquisition module 210 .

[0152] In some embodiments, the server 110 may obtain marking input data, wherein the marking input data is data to be marked, which may be second text data. For more information on how to obtain the data to be marked, please refer to the relevant description of step 310, which will not be repeated here.

[0153] Step 1120, data preprocessing.

[0154] In some embodiments, the server 110 may preprocess the data to be labeled obtained in step 1110, wherein the preprocessing may include at least one of normalization processing, word segmentation processing, etc. For more information on how to preprocess the data, please refer to the relevant description of step 310, which will not be repeated here.

[0155] In some embodiments, after step 720 , the data labeling module 250 may perform steps 1130 - 1150 to label the data to be labeled.

[0156] Step 1130, load the saved model.

[0157] In some embodiments, the server 110 may obtain a saved knowledge graph annotation model, that is, the trained knowledge graph annotation model in process 300 , from a storage device (e.g., storage device 140 ), and load the model.

[0158] Step 1140, model calculation.

[0159] In some embodiments, the server 110 can input the preprocessed data to be labeled into the knowledge graph annotation model loaded in step 1130 to obtain the annotation results of the data to be labeled, that is, the data after the data to be labeled is annotated.

[0160] Step 1150, output the result.

[0161] In some embodiments, the server 110 may output the labeling results of the data to be labeled obtained in step 1140 to a storage device to save the labeling results, or output them to an output device such as a display device for user browsing.

[0162] It should be noted that the above descriptions of processes 300, 400, 700, and 1100 are only for illustration and explanation, and do not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to processes 300, 400, 700, and 1100 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification. For example, steps 720 and 1120 can be cancelled, where the preprocessing operation of the data can be performed through the knowledge graph annotation model.

[0163] The beneficial effects that may be brought about by the embodiments of this specification include but are not limited to: (1) by obtaining a regular expression from sample data to be annotated, generating a knowledge graph annotation model based on the regular expression, and using sample annotation data and manually annotated data to train the model, it is possible to combine simple rules (black and white lists) with machine learning models, and in the process of knowledge graph annotation, both rule knowledge is integrated and annotation data can be used for training, thereby improving the labeling effect of the knowledge graph annotation model; (2) by using convenient and easy-to-understand black and white list keywords, the difficulty of generating the knowledge graph annotation model is reduced, and the generation efficiency of the knowledge graph annotation model is improved; (3) by automatically generating black and white list keywords through automatic generation methods (for example, TF-IDF generation, BERT generation, etc.), the model generation efficiency is improved; (4) by converting regular expressions and FA-RNN models into each other, the knowledge graph annotation model can be continuously optimized, thereby improving the accuracy of the knowledge graph annotation model labeling; (5) by constructing multi-task training according to a multi-level label architecture, the overfitting of the knowledge graph annotation model is alleviated, and the accuracy of the knowledge graph annotation model labeling is improved. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other beneficial effects that may be obtained.

[0164] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0165] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0166] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0167] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0168] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0169] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0170] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A method for generating a knowledge graph annotation model, comprising: Acquire first text data; generating a first regular expression based on the first text data; Generate a knowledge graph annotation model based on the first regular expression and the first text data.

2. The method according to claim 1, wherein obtaining the first text data comprises: The first text data is preprocessed, where the preprocessing includes at least one of normalization processing and word segmentation processing.

3. The method of claim 1, wherein generating a first regular expression based on the first text data comprises: extracting keywords from the first text data; Annotating the first text data based on a preset rule and the keyword to obtain first annotated data; The keyword is converted into the first regular expression based on the preset rule.

4. The method of claim 3, further comprising: Using the first labeled data as a first training sample to pre-train the knowledge graph labeling model; The second labeled data is used as a second training sample to optimize the training of the knowledge graph annotation model to obtain a trained knowledge graph annotation model.

5. The method of claim 4, further comprising: The second text data is input into the trained knowledge graph annotation model to obtain the annotation result of the second text data.

6. The method according to claim 4, wherein the first text data comprises multi-level tags, and the method further comprises: For labels at different levels in the multi-level labels, training is performed through different training tasks.

7. The method of claim 4, further comprising: Convert the trained knowledge graph annotation model into a second regular expression; The trained knowledge graph annotation model is optimized based on the second regular expression.

8. According to the method as claimed in claim 1, the knowledge graph annotation model includes a word vector component, a finite automaton recurrent neural network model component and a classifier component, the word vector component is used to generate a first word vector based on the first text data, the finite automaton recurrent neural network model component is used to generate a text feature vector, and the classifier component is used to classify the text data.

9. The method according to claim 8, wherein generating a knowledge graph annotation model based on the first regular expression and the first text data comprises: Generate a first word vector based on the first text data through the word vector component; Generate the finite automaton recurrent neural network model component based on the first regular expression and the first word vector; The knowledge graph annotation model is constructed based on the finite automaton recurrent neural network model component and the classifier component.

10. A knowledge graph annotation model generation system, comprising a data acquisition module, a regular expression generation module and a model generation module; The data acquisition module is used to acquire first text data; The regular expression generation module is used to generate a first regular expression based on the first text data; The model generation module is used to generate a knowledge graph annotation model based on the first regular expression and the first text data.

Images