Data processing method and apparatus

Abstract

Embodiments of the present specification provide a data processing method and device, wherein the data processing method comprises: obtaining a to-be-processed keyword with a hierarchical relationship in an initial knowledge graph and an initial hierarchical relationship of the to-be-processed keyword, predicting a hierarchical relationship between the to-be-processed keywords, obtaining a predicted hierarchical relationship, and performing accuracy evaluation on the predicted hierarchical relationship to generate a corresponding evaluation result, obtaining a data graph constructed based on a target data source, and determining an index value of an evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the data graph and the to-be-processed keyword, determining a target hierarchical relationship between the to-be-processed keywords according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result and the index value, and updating the initial knowledge graph based on the target hierarchical relationship.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer, in particular to a data processing method. BACKGROUND

[0002] With the development of computers and the Internet, the amount of data that can be used for research and analysis is growing exponentially, and the era of big data has also arrived. Big data is a collection of data structures with huge amounts of data, complex structures, and numerous types. The data in it reflects the information of the objective world, so through the analysis of big data, a large amount of useful information can be obtained. In order to extract useful information from massive data and make full use of big data, advanced data analysis technologies such as artificial intelligence are also becoming increasingly active, and the foundation of artificial intelligence is the construction of knowledge graph.

[0003] Knowledge graph is a natural language processing technology that uses structured visual graph to display entity information in natural language. It is often used to process NLP (Natural Language Processing) tasks that contain complex relationships and require logical reasoning, and form an image of knowledge macro connection.

[0004] However, the current knowledge graph is constructed by manual method. With the acceleration of data update speed, manual data processing not only needs to consume a lot of time and energy, but also brings information update delay. Therefore, how to realize data extraction automation is very important for constructing knowledge graph, that is, an effective method is urgently needed to solve such problems. SUMMARY

[0005] Therefore, the embodiments of the present specification provide a data processing method. One or more embodiments of the present specification also relate to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects in the prior art.

[0006] According to a first aspect of the embodiments of the present specification, a data processing method is provided, comprising:

[0007] Obtaining a to-be-processed keyword with a hierarchical relationship in an initial knowledge graph and an initial hierarchical relationship of the to-be-processed keyword;

[0008] Predicting the hierarchical relationship between the to-be-processed keywords, obtaining a predicted hierarchical relationship, and evaluating the accuracy of the predicted hierarchical relationship to generate a corresponding evaluation result;

[0009] acquire a data graph constructed based on a target data source, and determine an index value of an evaluation index for evaluating the initial hierarchical relationship accuracy according to the data graph and the to-be-processed keyword;

[0010] According to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result and the index value, a target hierarchical relationship between the to-be-processed keywords is determined, and the initial knowledge graph is updated based on the target hierarchical relationship.

[0011] Optionally, the predicting the hierarchical relationship between the to-be-processed keywords to obtain a predicted hierarchical relationship comprises:

[0012] A to-be-processed sentence is constructed based on the to-be-processed keywords, and the to-be-processed sentence is input into a prediction model for prediction to obtain the predicted hierarchical relationship between the to-be-processed keywords.

[0013] Optionally, the constructing a to-be-processed sentence based on the to-be-processed keywords and inputting the to-be-processed sentence into a prediction model for prediction to obtain the predicted hierarchical relationship between the to-be-processed keywords comprises:

[0014] A first to-be-processed sentence corresponding to the to-be-processed keywords is constructed in combination with a first ordering relationship of the to-be-processed keywords and a masking identifier, and the first to-be-processed sentence is input into a prediction model for prediction to obtain a first predicted hierarchical relationship between the to-be-processed keywords;

[0015] A second to-be-processed sentence corresponding to the to-be-processed keywords is constructed in combination with a second ordering relationship of the to-be-processed keywords and a masking identifier, and the second to-be-processed sentence is input into a prediction model for prediction to obtain a second predicted hierarchical relationship between the to-be-processed keywords.

[0016] Optionally, the evaluating the accuracy of the predicted hierarchical relationship to generate a corresponding evaluation result comprises:

[0017] A to-be-evaluated sentence is constructed based on the to-be-processed keywords and the predicted hierarchical relationship, and the to-be-evaluated sentence is input into an evaluation model for accuracy evaluation to generate an evaluation result corresponding to the predicted hierarchical relationship.

[0018] Optionally, the evaluating the accuracy of the predicted hierarchical relationship to generate a corresponding evaluation result comprises:

[0019] A to-be-evaluated sentence is constructed based on the to-be-processed keywords and the predicted hierarchical relationship, and the to-be-evaluated sentence is input into an evaluation model for accuracy evaluation to generate an evaluation result corresponding to the predicted hierarchical relationship.

[0020] Correspondingly, after the corresponding evaluation result is generated, the method further comprises:

[0021] According to a preset evaluation rule, a feature value corresponding to the first predicted hierarchical relationship and / or the second predicted hierarchical relationship is determined, and the feature value is updated according to the evaluation result.

[0022] Optionally, the determining, according to the data graph and the to-be-processed keyword, of an index value of an evaluation index used for evaluating accuracy of the initial hierarchical relationship comprises:

[0023] determining a concept word in the data graph that has a semantic similarity greater than a preset similarity threshold with the to-be-processed keyword;

[0024] obtaining a parent node and a child node in the data graph that have a connection relationship with a node in which the concept word is located;

[0025] determining a first coincidence ratio of the parent node based on a node quantity and node information of the parent node, and determining a second coincidence ratio of the child node based on a node quantity and node information of the child node;

[0026] determining an index value of a first evaluation index used for evaluating accuracy of the initial hierarchical relationship according to the first coincidence ratio and the second coincidence ratio.

[0027] Optionally, the to-be-processed keyword comprises a first to-be-processed keyword and a second to-be-processed keyword.

[0028] Correspondingly, the determining, according to the data graph and the to-be-processed keyword, of an index value of an evaluation index used for evaluating accuracy of the initial hierarchical relationship comprises:

[0029] determining a first concept word in the data graph that has a semantic similarity greater than a preset similarity threshold with the first to-be-processed keyword, and determining a second concept word in the data graph that has a semantic similarity greater than a preset similarity threshold with the second to-be-processed keyword;

[0030] obtaining a first parent node and a first child node in the data graph that have a connection relationship with a node in which the first concept word is located, and obtaining a second parent node and a second child node in the data graph that have a connection relationship with a node in which the second concept word is located;

[0031] determining a first coincidence ratio based on a node quantity and node information of the first parent node and the second parent node, and determining a second coincidence ratio based on a node quantity and node information of the first child node and the second child node;

[0032] determining an index value of a first evaluation index used for evaluating accuracy of the initial hierarchical relationship according to the first coincidence ratio and the second coincidence ratio.

[0033] Optionally, the determining the first coincidence ratio based on the first parent node and the second parent node includes:

[0034] determining a first number of nodes of the first parent node, a second number of nodes of the second parent node, and a third number of nodes of the first parent node and the second parent node that exist in overlap;

[0035] in a case where a first ratio between the third number of nodes and the first number of nodes is less than a preset threshold value, and the first number of nodes is greater than or equal to the second number of nodes, determining a product of the first ratio and the first number of nodes, and determining whether a second ratio of the product and the second number of nodes is greater than or equal to the preset threshold value;

[0036] if yes, determining a third ratio between the second ratio and the preset threshold value, and taking a product of the third ratio and a preset ratio coefficient as the first coincidence ratio.

[0037] Optionally, the to-be-processed keyword includes a first to-be-processed keyword and a second to-be-processed keyword.

[0038] Correspondingly, the determining an index value of an evaluation index for evaluating the accuracy of the initial hierarchical relationship based on the data graph and the to-be-processed keyword includes:

[0039] determining a plurality of first concept words in the data graph that have a semantic similarity greater than a preset similarity threshold value with the first to-be-processed keyword, and determining a plurality of second concept words in the data graph that have a semantic similarity greater than a preset similarity threshold value with the second to-be-processed keyword;

[0040] performing encoding processing on the plurality of first concept words to generate a first encoding vector set, and performing encoding processing on the plurality of second concept words to generate a second encoding vector set;

[0041] constructing a first node cluster based on a plurality of encoding vectors in the first encoding vector set and determining a first center point in the first node cluster, and constructing a second node cluster based on a plurality of encoding vectors in the second encoding vector set and determining a second center point in the second node cluster;

[0042] calculating a distance between the first center point and the second center point, and taking the distance as an index value of a second evaluation index for evaluating the accuracy of the initial hierarchical relationship.

[0043] Optionally, after the generating the second encoding vector set, the method further includes:

[0044] respectively, and determining target principal axes corresponding to the first and second sets of encoded vectors according to the processing results;

[0045] According to the projection coordinates of the plurality of encoded vectors in the first and second sets of encoded vectors on the target principal axes, an overlap ratio between the first and second encoded vectors is calculated, and the overlap ratio is taken as an index value of a third evaluation index for evaluating the accuracy of the initial hierarchical relationship.

[0046] Optionally, the determining of the target principal axes corresponding to the first and second sets of encoded vectors according to the processing results comprises:

[0047] determining a first principal axis vector corresponding to the first set of encoded vectors and a second principal axis vector corresponding to the second set of encoded vectors according to the processing results;

[0048] taking a direction of a sum vector of the first and second principal axis vectors as the target principal axes corresponding to the first and second sets of encoded vectors.

[0049] Optionally, the determining of the target hierarchical relationship between the to-be-processed keywords according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result and the index value comprises:

[0050] determining an accuracy rate of the initial hierarchical relationship according to the predicted hierarchical relationship, the evaluation result and the index value;

[0051] in a case where the accuracy rate is greater than a preset accuracy threshold, taking the initial hierarchical relationship as the target hierarchical relationship between the to-be-processed keywords.

[0052] According to a second aspect of an embodiment of the present specification, a data processing apparatus is provided, comprising:

[0053] an acquisition module configured to acquire to-be-processed keywords having a hierarchical relationship in an initial knowledge graph and an initial hierarchical relationship of the to-be-processed keywords;

[0054] an evaluation module configured to predict a hierarchical relationship between the to-be-processed keywords, obtain a predicted hierarchical relationship, and evaluate the accuracy of the predicted hierarchical relationship to generate a corresponding evaluation result;

[0055] a determination module configured to acquire a data graph constructed based on a target data source, and determine an index value of an evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the data graph and the to-be-processed keywords;

[0056] An updating module configured to determine a target hierarchical relationship between the to-be-processed keywords according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result and the index value, and update the initial knowledge graph based on the target hierarchical relationship.

[0057] According to a third aspect of the embodiments of the present specification, a computing device is provided, comprising:

[0058] a memory and a processor;

[0059] The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions.

[0060] obtain to-be-processed keywords with hierarchical relationships in an initial knowledge graph and initial hierarchical relationships of the to-be-processed keywords;

[0061] predict hierarchical relationships between the to-be-processed keywords, obtain predicted hierarchical relationships, and evaluate the predicted hierarchical relationships to generate corresponding evaluation results;

[0062] obtain a data graph constructed based on a target data source, and determine an index value of an evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the data graph and the to-be-processed keywords;

[0063] determine a target hierarchical relationship between the to-be-processed keywords according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result and the index value, and update the initial knowledge graph based on the target hierarchical relationship.

[0064] According to a fourth aspect of the embodiments of the present specification, a computer readable storage medium is provided, which stores computer executable instructions, and the instructions are executed by a processor to implement the steps of any one of the data processing methods.

[0065] According to a fifth aspect of the embodiments of the present specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data processing method.

[0066] One embodiment of the present specification obtains a to-be-processed keyword and an initial hierarchical relationship of the to-be-processed keyword in an initial knowledge graph that exists a hierarchical relationship, predicts a hierarchical relationship between the to-be-processed keywords, obtains a predicted hierarchical relationship, and performs accuracy evaluation on the predicted hierarchical relationship to generate a corresponding evaluation result, obtains a data graph constructed based on a target data source, and determines an index value of an evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the data graph and the to-be-processed keyword, determines a target hierarchical relationship between the to-be-processed keywords according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result and the index value, and updates the initial knowledge graph based on the target hierarchical relationship.

[0067] The embodiment of the present specification predicts the hierarchical relationship between the to-be-processed keywords, and performs accuracy evaluation on the initial hierarchical relationship between the to-be-processed keywords in the initial knowledge graph based on the data graph of the target data source, to determine the target hierarchical relationship between the to-be-processed keywords according to the prediction result and the accuracy evaluation result, and update the initial knowledge graph based on the target hierarchical relationship, which is beneficial to guarantee the accuracy of the knowledge graph construction result. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 FIG. 1 is a flowchart of a data processing method according to one embodiment of the present specification;

[0069] Figure 2 FIG. 2 is a schematic diagram of a projection result according to one embodiment of the present specification;

[0070] Figure 3 FIG. 3 is a flowchart of a processing process of a data processing method according to one embodiment of the present specification;

[0071] Figure 4 FIG. 4 is a structural schematic diagram of a data processing device according to one embodiment of the present specification;

[0072] Figure 5 FIG. 5 is a structural block diagram of a computing device according to one embodiment of the present specification. DETAILED DESCRIPTION

[0073] In the following description, a lot of specific details are set forth in order to facilitate a thorough understanding of the present specification. However, the present specification can be implemented in many different ways than those described herein, and those skilled in the art can make similar extensions without departing from the connotation of the present specification, so the present specification is not limited to the specific implementation disclosed below.

[0074] The terminology used in this disclosure, in one or more embodiments, is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0075] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal sequence, but to distinguish between different types of information. For example, without departing from the scope of one or more embodiments, first can be termed second, and similarly, second can be termed first. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "in response to determining."

[0076] First, the noun terms related to one or more embodiments of the present disclosure are explained.

[0077] Knowledge graph: From an academic point of view, a knowledge graph is essentially a semantic network knowledge base. From the perspective of practical application, a knowledge graph can be understood as a multi-relational graph.

[0078] Domain knowledge graph (DKG): refers to a knowledge graph limited to a specific field, which requires higher accuracy of concepts and stronger professionalism.

[0079] Domain knowledge graph with professional concept hierarchical relationship: special attention is paid to the construction of hierarchical relationships of professional concepts in the field, such as computer science-artificial intelligence, metal-iron, fruit-apple, etc., and the hierarchical relationship is required to be accurate and consistent with professional knowledge.

[0080] Bert pre-training model: Bert model is a natural language pre-training model.

[0081] In the present specification, a data processing method is provided, and the present specification also relates to a data processing device, a computing device, a computer readable storage medium, and a computer program, which are described in detail one by one in the following embodiments.

[0082] At present, in the construction of the academic field knowledge graph (industry field knowledge graph), the method commonly used is: analyzing and constructing a hierarchical relationship tree for the keywords extracted from different scientific and technological literatures, for example, "computer science", which has a subclass "computer application", and then constructing a hierarchical relationship tree based on the relationship between "computer science" and "computer application". The implementation of this process is an abstraction and induction of knowledge. Similarly, such an abstraction and induction process is also a key problem for the construction of an industry field knowledge graph. The current solution to this problem is mainly to use a concept layering algorithm; however, this algorithm requires a very comprehensive and detailed professional data related text, and needs manual data labeling to train the type recognition model. Such an algorithm is not conducive to the rapid, low-cost and accurate construction of an actual industry field knowledge graph.

[0083] Based on the foregoing concept layering algorithm, the embodiment of the present specification proposes a keyword hierarchical relationship determination method based on a pre-trained Bert model and a Chinese wiki data set that can be obtained at low cost, and constructs four effective evaluation indexes based on the Bert MLM model, the wiki subgraph mapping distance, etc., which can realize the rapid elimination of the noise of the keyword hierarchical relationship, thereby ensuring the accuracy of the determined hierarchical relationship between keywords. Moreover, the test results obtained by testing using a public test data set show that the hierarchical relationship between keywords determined by the present scheme can achieve an accuracy of 80% and a recall rate of about 80%.

[0084] Figure 1 A flowchart of a data processing method according to one embodiment of the present specification is shown, which specifically includes the following steps.

[0085] Step 102, obtaining a to-be-processed keyword with a hierarchical relationship in an initial knowledge graph and an initial hierarchical relationship of the to-be-processed keyword.

[0086] Specifically, the initial knowledge graph, i.e., the existing knowledge graph, contains multiple nodes, and the node name of each node can be used as the to-be-processed keyword described in the embodiment of the present specification. In the initial knowledge graph, if there is a connection relationship between two nodes, the connection relationship can be used as the initial hierarchical relationship between the two to-be-processed keywords corresponding to the two nodes with the connection relationship, and the initial hierarchical relationship is used to represent that there is a hierarchical relationship of superior and subordinate between the two to-be-processed keywords in the initial knowledge graph.

[0087] The relationship type corresponding to the relationship between the different to-be-processed keywords corresponding to different nodes in the initial knowledge graph can be different, and the hierarchical relationship is one of the multiple relationship types. The hierarchical relationship between the to-be-processed keywords can represent the inclusion relationship between the two, for example, if the A concept is a subclass of the B concept (the A concept includes the B concept), there is a connection relationship between the A concept and the B concept in the knowledge graph constructed based on the A concept and the B concept, and the A concept is located at the upper level of the B concept.

[0088] Therefore, the extraction of the accurate hierarchical relationship between different to-be-processed keywords can realize the logical structure of the knowledge in different fields. After determining the initial hierarchical relationship between two to-be-processed keywords, the initial hierarchical relationship can be evaluated to determine the accuracy of the initial hierarchical relationship, so that the initial knowledge graph is optimized according to the evaluation result.

[0089] Step 104, predicting the hierarchical relationship between the to-be-processed keywords, obtaining a predicted hierarchical relationship, and evaluating the accuracy of the predicted hierarchical relationship to generate a corresponding evaluation result.

[0090] Specifically, after determining the initial hierarchical relationship between two to-be-processed keywords, the first way to evaluate the accuracy of the initial hierarchical relationship is to predict the hierarchical relationship between the two to-be-processed keywords again, and then evaluate the accuracy of the predicted hierarchical relationship obtained by prediction to obtain a corresponding evaluation result.

[0091] In specific implementation, the hierarchical relationship between the to-be-processed keywords is predicted to obtain a predicted hierarchical relationship, that is, a to-be-processed sentence is constructed based on the to-be-processed keywords, and the to-be-processed sentence is input into a prediction model for prediction to obtain the predicted hierarchical relationship between the to-be-processed keywords.

[0092] Further, a to-be-processed sentence is constructed based on the to-be-processed keywords, and the to-be-processed sentence is input into a prediction model for prediction to obtain the predicted hierarchical relationship between the to-be-processed keywords, including:

[0093] The first to-be-processed sentence corresponding to the to-be-processed keywords is constructed in combination with the first ordering relationship and the masking identifier of the to-be-processed keywords, and the first to-be-processed sentence is input into a prediction model for prediction to obtain the first predicted hierarchical relationship between the to-be-processed keywords;

[0094] The second to-be-processed sentence corresponding to the to-be-processed keywords is constructed in combination with the second ordering relationship and the masking identifier of the to-be-processed keywords, and the second to-be-processed sentence is input into a prediction model for prediction to obtain the second predicted hierarchical relationship between the to-be-processed keywords.

[0095] Specifically, the prediction model can be a Bert MLM model; and the mask identifier can be [mask].

[0096] During pre-training of the Bert model, a commonly used training mode is to remove some words in a complete sentence (and replace them with a specific mask symbol), and a sentence generated by replacement is used for model training (the training target is the accuracy of predicting the missing words), to obtain a Bert MLM model (Masked Language Model).

[0097] Therefore, after obtaining the to-be-processed keywords, the to-be-processed sentence can be constructed based on the to-be-processed keywords, and the to-be-processed sentence is input into the Bert MLM model for prediction, to obtain the predicted hierarchical relationship between the to-be-processed keywords.

[0098] After obtaining two to-be-processed keywords having a connection relationship (i.e., a hierarchical relationship) in the initial knowledge graph, the to-be-processed sentence is constructed based on the to-be-processed keywords, i.e., the to-be-processed sentence is constructed by using the to-be-processed sentence and the mask identifier. During the construction of the to-be-processed sentence, it is considered that the ordering relationship between the two to-be-processed sentences is not unique, and the predicted hierarchical relationship obtained based on different ordering relationships may also have differences. Therefore, the two to-be-processed keywords are used to construct two to-be-processed sentences by using two ordering relationships and the mask identifier, and the two to-be-processed sentences are input into the Bert MLM model, to predict the words masked by the mask identifier by the Bert MLM model, and to obtain two groups of prediction results (predicted hierarchical relationships).

[0099] In actual application, if the two to-be-processed keywords are key_child and key_parent, a first to-be-processed sentence constructed by using the first ordering relationship of the to-be-processed keywords and the mask identifier can be key_child[mask]key_parent, which is input into the Bert MLM model for prediction, to obtain a candidate set m1 of the words masked by the model; and a second to-be-processed sentence constructed by using the second ordering relationship of the to-be-processed keywords and the mask identifier can be key_parent[mask]key_child, which is input into the Bert MLM model for prediction, to obtain a candidate set m2 of the words masked by the model.

[0100] The m1 and m2 respectively include one or at least two predicted hierarchical relationships, and the predicted hierarchical relationship includes but is not limited to "belongs to", "contains", "and", "or", "vs", " / ", and ":".

[0101] Alternatively, after the Bert MLM model processes the first to-be-processed sentence and the second to-be-processed sentence to generate the candidate set m1 and the candidate set m2, the feature value corresponding to the predicted hierarchical relationship of the two to-be-processed keywords can be evaluated according to a pre-established feature value evaluation rule, in combination with the candidate sets m1 and m2, to obtain a corresponding evaluation result.

[0102] In actual applications, the feature value evaluation rule can be: if “belongs to” is contained in m1, the feature value is 1; if “contains” is contained in m2, the feature value is 1; if “vs” or “+” is contained in m1 or m2, the feature value is -1; if “,” is contained in m1, the feature value is 0.1; and in other cases, the feature value is 0.5. According to the feature value evaluation rule, the feature value corresponding to the predicted hierarchical relationship of the two to-be-processed keywords can be evaluated, so that the feature value of one dimension that is consistent with the predicted hierarchical relationship in m1 and m2 can be selected from the above four dimensions as the evaluation result.

[0103] The embodiments of the present specification are only described by taking the above feature value evaluation rule as an example. In actual applications, the feature value evaluation rule can be determined according to actual needs, which is not limited herein.

[0104] In specific implementation, after the hierarchical relationship between the to-be-processed keywords is predicted by using the Bert MLM model to obtain a predicted hierarchical relationship, the accuracy of the predicted hierarchical relationship is evaluated to generate a corresponding evaluation result. Specifically, a to-be-evaluated sentence can be constructed based on the to-be-processed keywords and the predicted hierarchical relationship, and the to-be-evaluated sentence is input into an evaluation model for accuracy evaluation to generate an evaluation result corresponding to the predicted hierarchical relationship.

[0105] Specifically, the evaluation model can be an electra pre-training model.

[0106] After the predicted hierarchical relationship between the to-be-processed keywords is obtained, a to-be-evaluated sentence can be constructed based on the to-be-processed keywords and the predicted hierarchical relationship, and the to-be-evaluated sentence is input into the electra pre-training model to evaluate the accuracy of the predicted hierarchical relationship in the to-be-processed sentence by the electra pre-training model to generate a corresponding evaluation result.

[0107] With the above example, if the two keywords to be processed are key_child and key_parent, the predicted hierarchical relationship contained in the alternative set m1 is "belongs to", and the predicted hierarchical relationship contained in the alternative set m2 is "contains", then the evaluation statements constructed based on the keywords to be processed and the predicted hierarchical relationship can be "key_child belongs to key_parent" and "key_parent contains key_child", and then the evaluation statements are input into the electra pre-training model, so as to obtain the accuracy evaluation result of the predicted hierarchical relationship in the statement to be processed.

[0108] Based on this, after generating the corresponding evaluation result, it further includes:

[0109] According to the preset evaluation rule, the feature value corresponding to the first predicted hierarchical relationship and / or the second predicted hierarchical relationship is determined, and the feature value is updated according to the evaluation result.

[0110] Specifically, in addition to the above-mentioned Bert MLM model, the electra pre-training model is introduced in the embodiments of the present specification. The electra pre-training model is similar to the Bert MLM model, but instead of directly outputting the mask alternative word, it outputs the accuracy or rationality evaluation value of the predicted hierarchical relationship contained in a sentence. For example, it can output the evaluation value of "belongs to" in "key_child belongs to key_parent", or output the average value of the evaluation values of "is" and "one kind" in "key_child is one kind of key_parent". When the evaluation value of the predicted hierarchical relationship output by the electra pre-training model is less than or equal to the feature value corresponding to the predicted hierarchical relationship of the two keywords to be processed output by the above-mentioned Bert MLM model, the feature value is not processed. When the evaluation value of the predicted hierarchical relationship output by the electra pre-training model is greater than the feature value corresponding to the predicted hierarchical relationship output by the above-mentioned Bert MLM model, the feature value needs to be updated, and specifically, the feature value can be updated to 1.

[0111] Step 106, obtaining a data graph constructed based on a target data source, and determining an index value of an evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the data graph and the keywords to be processed.

[0112] Specifically, the target data source can be a Chinese encyclopedia, or it can also be other devices or original media that can provide certain required data.

[0113] Since in the process of determining the superior-inferior hierarchical relationship between the keywords to be processed, if the keywords to be processed are specific to a certain field, the corpus in this field is usually very limited, and in order to ensure the accuracy of the determination result, some public data sources, or public texts, can be used as supplements to determine the target hierarchical relationship between the keywords to be processed in the embodiments of the present specification.

[0114] In practical applications, in the process of determining the target hierarchical relationship between the keywords to be processed using a public data source, i.e., a target data source, a data graph can be constructed based on the target data source. Since the entire open source target data source can include multiple different concept words (such as computer science, metal, and fruit) and description texts corresponding to each concept word, in constructing the data graph, the concept word and the description text corresponding to the concept word can be obtained, and then other multiple concept words contained in the description text can be obtained, and the connection relationship between the concept word and the other multiple concept words can be established, and then the concept word is taken as a node, and the connection relationship between the concept words is taken as an edge, to generate the data graph.

[0115] In specific implementation, after obtaining the data graph, the index value of the evaluation index for evaluating the accuracy of the initial hierarchical relationship is determined according to the data graph and the keywords to be processed, including:

[0116] determining a concept word in the data graph that has a semantic similarity greater than a preset similarity threshold with the keywords to be processed;

[0117] obtaining parent nodes and child nodes in the data graph that have a connection relationship with the node in which the concept word is located;

[0118] determining a first coincidence rate of the parent nodes based on the node quantity and node information of the parent nodes, and determining a second coincidence rate of the child nodes based on the node quantity and node information of the child nodes;

[0119] determining the index value of the first evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the first coincidence rate and the second coincidence rate.

[0120] Specifically, after determining the initial hierarchical relationship between the to-be-processed keywords, a second way of accuracy evaluation of the initial hierarchical relationship is performed, i.e., a concept word corresponding to the to-be-processed keyword is searched in the data graph. Specifically, a concept word in the data graph having a semantic similarity greater than a preset similarity threshold with the to-be-processed keyword is searched, then parent nodes and child nodes of upper and lower levels having a connection relationship with the node where the concept word is located are searched in the data graph, then a first coincidence ratio of the parent nodes is determined based on the node quantity and node information of the parent nodes, and a second coincidence ratio of the child nodes is determined based on the node quantity and node information of the child nodes; and then, an index value of a first evaluation index for evaluating the accuracy of the initial hierarchical relationship is determined according to the first coincidence ratio and the second coincidence ratio.

[0121] Alternatively, in a case where the concept word corresponding to the to-be-processed keyword cannot be directly obtained from the data graph, the to-be-processed keyword can be disassembled, and a concept word corresponding to the disassembled keyword is obtained from the data graph, then parent nodes and child nodes of upper and lower levels having a connection relationship with the node where the concept word is located are searched in the data graph, and an index value of a first evaluation index for evaluating the accuracy of the initial hierarchical relationship is further determined.

[0122] Further, the to-be-processed keyword includes a first to-be-processed keyword and a second to-be-processed keyword.

[0123] Correspondingly, determining an index value of an evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the data graph and the to-be-processed keyword includes:

[0124] determining a first concept word in the data graph having a semantic similarity greater than a preset similarity threshold with the first to-be-processed keyword, and determining a second concept word in the data graph having a semantic similarity greater than a preset similarity threshold with the second to-be-processed keyword;

[0125] obtaining first parent nodes and first child nodes in the data graph having a connection relationship with the node where the first concept word is located, and obtaining second parent nodes and second child nodes in the data graph having a connection relationship with the node where the second concept word is located;

[0126] determining a first coincidence ratio based on the node quantity and node information of the first parent nodes and the second parent nodes, and determining a second coincidence ratio based on the node quantity and node information of the first child nodes and the second child nodes;

[0127] determining an index value of a first evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the first coincidence ratio and the second coincidence ratio.

[0128] Specifically, if the to-be-processed keywords include a first to-be-processed keyword and a second to-be-processed keyword, after determining the initial hierarchical relationship between the two to-be-processed keywords, the initial hierarchical relationship is evaluated for accuracy, that is, the concept words corresponding to the two to-be-processed keywords are searched in the data graph, and specifically, a first concept word with a semantic similarity greater than a preset similarity threshold with the first to-be-processed keyword in the data graph is searched, and then a second concept word with a semantic similarity greater than the preset similarity threshold with the second to-be-processed keyword in the data graph is searched; then the first parent node and the first child node of the upper and lower levels which have a connection relationship with the node where the first concept word is located, and the second parent node and the second child node of the upper and lower levels which have a connection relationship with the node where the second concept word is located are searched in the data graph; then the first coincidence rate of the parent node can be determined based on the node quantity and node information of the first parent node and the second parent node, and the second coincidence rate of the child node is determined based on the node quantity and node information of the first child node and the second child node; and then the index value of the first evaluation index for evaluating the accuracy of the initial hierarchical relationship is determined according to the first coincidence rate and the second coincidence rate. The node information of the first parent node can be used to represent the connection relationship or hierarchical relationship between the first parent node and the node where the first concept word is located.

[0129] Alternatively, in the case where the concept words corresponding to the first to-be-processed keyword and the second to-be-processed keyword cannot be directly obtained from the data graph, the first to-be-processed keyword and the second to-be-processed keyword can be disassembled, and the concept words corresponding to the disassembled keywords are obtained from the data graph, and then the parent node and the child node of the upper and lower levels which have a connection relationship with the node where the concept word is located are searched in the data graph, and further the index value of the first evaluation index for evaluating the accuracy of the initial hierarchical relationship is determined.

[0130] Further, determining the first coincidence rate based on the node quantity and node information of the first parent node and the second parent node comprises:

[0131] determining the first node quantity of the first parent node, the second node quantity of the second parent node, and the third node quantity of the nodes that exist in the first parent node and the second parent node and overlap;

[0132] in the case where the first ratio between the third node quantity and the first node quantity is less than a preset threshold, and the first node quantity is greater than or equal to the second node quantity, the product of the first ratio and the first node quantity is determined, and it is judged whether the second ratio of the product to the second node quantity is greater than or equal to the preset threshold;

[0133] if yes, the third ratio between the second ratio and the preset threshold is determined, and the product of the third ratio and a preset ratio coefficient is taken as the first coincidence rate.

[0134] Specifically, if two keywords to be processed are key_child and key_parent, a concept word w1 corresponding to key_child in the data graph is located in node A1, the first parent node of the upper level connected with node A1 in the data graph is node A2, node A3, node A4, and node A5, and the first child node of the lower level connected with node A1 in the data graph is node A6 and node A7; a concept word w2 corresponding to key_parent in the data graph is located in node B1, the second parent node of the upper level connected with node B1 in the data graph is node B2 and node B3, and the second child node of the lower level connected with node B1 in the data graph is node B4, node B5, and node B6.

[0135] Therefore, the number of nodes N1 of the first parent node of the upper level connected with node A1 is equal to 4, and the number of nodes N2 of the second parent node of the upper level connected with node B1 is equal to 2.

[0136] In addition, since node A1 and node B1 can be one or at least two, and node A1 and node B1 can be the same or different, the intersection between node A1 and node B1 can be determined first, and the number of nodes M (the third number of nodes that overlap in the first parent node and the second parent node) in the intersection is determined. In the case that the first ratio r between the number of nodes M and the number of nodes N1 of the first parent node is less than the preset threshold value, the size between the number of nodes N1 of the first parent node and the number of nodes N2 of the second parent node, and the size of the second ratio of the product of the first ratio r and the number of nodes N1 to the number of nodes N2 to the preset threshold value can be further determined. If the number of nodes N1 of the first parent node is greater than or equal to the number of nodes N2 of the second parent node, and the second ratio of the product of the first ratio r and the number of nodes N1 to the number of nodes N2 is greater than or equal to the preset threshold value, i.e., N1≥N2, and r*N1 / N2≥T, the third ratio between the second ratio and the preset threshold value can be determined, and the product of the third ratio and the preset ratio coefficient is used as the first coincidence ratio.

[0137] In practical applications, the preset ratio coefficient can be 0.5, and therefore the first coincidence ratio is r*N1 / N2 / T*0.5.

[0138] Based on the number of nodes and node information of the first parent node and the second parent node, the first coincidence ratio is determined, and based on the number of nodes and node information of the first child node and the second child node, the second coincidence ratio can be determined.

[0139] The calculation process of the first coincidence ratio is described in detail in the specification, and the calculation process of the second coincidence ratio is similar to that of the first coincidence ratio. For details, refer to the calculation process of the first coincidence ratio, which will not be described here.

[0140] In addition, in the embodiments of the specification, after determining the first concept word corresponding to the first to-be-processed keyword and the second concept word corresponding to the second to-be-processed keyword, and finding the parent node or child node of the node where the first concept word and the second concept word are located in the data graph, if the more the third node number of the overlapping nodes in the parent nodes of the nodes where the two to-be-processed keywords are located, the higher the probability that the two to-be-processed keywords are of the same type. In the case where it is determined that the two to-be-processed keywords have an initial hierarchical relationship, that is, the two to-be-processed keywords have an existing connection relationship in the initial knowledge graph, if it is determined that they belong to the same type, the accuracy of the connection relationship can be further determined.

[0141] Therefore, after the first coincidence ratio and the second coincidence ratio are calculated, they can be compared, and the larger value in the comparison result is taken as the index value of the first evaluation index for evaluating the accuracy of the initial hierarchical relationship. The first evaluation index is the coincidence ratio, and the index value is the specific numerical value of the coincidence ratio.

[0142] Alternatively, in the case where the concept word corresponding to the to-be-processed keyword cannot be directly obtained from the data graph, the to-be-processed keywords key_child and key_parent can be disassembled, and the concept words existing in the data graph in the disassembled keywords are taken to obtain the set key_set_child and key_set_parent. Then, the parent nodes and child nodes of the upper and lower levels in the data graph that have a connection relationship with the nodes where the concept words in the set are located are found, that is, the child node set key_succ_child and key_succ_parent of key_set_child and key_set_parent in the data graph, and the parent node set key_pred_child and key_pred_parent are obtained. Then, the coincidence ratios of key_succ_child and key_succ_parent and the coincidence ratios of key_pred_child and key_pred_parent are calculated by the above calculation method, and the larger value is taken as the index value of the first evaluation index for evaluating the accuracy of the initial hierarchical relationship.

[0143] In addition, the to-be-processed keywords include a first to-be-processed keyword and a second to-be-processed keyword;

[0144] Correspondingly, determining the index value of the evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the data graph and the to-be-processed keyword comprises:

[0145] determining a plurality of first concept words in the data graph having a semantic similarity greater than a preset similarity threshold with the first to-be-processed keyword, and determining a plurality of second concept words in the data graph having a semantic similarity greater than a preset similarity threshold with the second to-be-processed keyword;

[0146] encoding the plurality of first concept words to generate a first set of encoded vectors, and encoding the plurality of second concept words to generate a second set of encoded vectors;

[0147] constructing a first node cluster based on a plurality of encoded vectors in the first set of encoded vectors, and determining a first center point in the first node cluster, constructing a second node cluster based on a plurality of encoded vectors in the second set of encoded vectors, and determining a second center point in the second node cluster;

[0148] calculating a distance between the first center point and the second center point, and taking the distance as an index value of a second evaluation index for evaluating the accuracy of the initial hierarchical relationship.

[0149] Specifically, after determining the initial hierarchical relationship between the two to-be-processed keywords, a third way of evaluating the accuracy of the initial hierarchical relationship is to determine concept words corresponding to the to-be-processed keywords in the data graph, and calculate the distance between the concept words, so as to take the distance as an index value of a second evaluation index for evaluating the accuracy of the initial hierarchical relationship.

[0150] If the to-be-processed keywords include a first to-be-processed keyword and a second to-be-processed keyword, after determining the initial hierarchical relationship between the two to-be-processed keywords, the accuracy of the initial hierarchical relationship can be evaluated, that is, the concept words corresponding to the two to-be-processed keywords in the data graph can be found, specifically, a plurality of first concept words in the data graph having a semantic similarity greater than a preset similarity threshold with the first to-be-processed keyword can be found, and then a plurality of second concept words in the data graph having a semantic similarity greater than a preset similarity threshold with the second to-be-processed keyword can be found; then the plurality of first concept words and the second concept words are encoded to calculate the distance between the concept words according to the encoding result.

[0151] Among them, the plurality of first concept words having a semantic similarity greater than a preset similarity threshold with the first to-be-processed keyword are more similar in type to the type of the first to-be-processed keyword, and the types of the first concept words are also more similar to each other; and the plurality of second concept words having a semantic similarity greater than a preset similarity threshold with the second to-be-processed keyword are more similar in type to the type of the second to-be-processed keyword, and the types of the second concept words are also more similar to each other.

[0152] Therefore, after the first concept words and the second concept words are encoded to generate corresponding encoding vectors, and the plurality of first concept words and the plurality of second concept words are mapped to the coordinate system based on the encoding vectors, in the mapping result, the distances between the encoding vectors corresponding to the plurality of first concept words are short, the mapping nodes are dense, and are aggregated into a first node cluster, while the distances between the encoding vectors corresponding to the plurality of second concept words are short, and are aggregated into another second node cluster. In this case, the type between the first to-be-processed keyword and the second to-be-processed keyword can be evaluated by determining the center points of the two node clusters and calculating the distance between the two center points, and the distance can be used as an index value of a second evaluation index for evaluating the accuracy of the initial hierarchical relationship. The second evaluation index is the distance, and the index value is the specific value of the distance.

[0153] In addition, after the second set of encoding vectors is generated, the following steps are further included:

[0154] The plurality of encoding vectors in the first set of encoding vectors and the second set of encoding vectors are respectively processed in a dimension reduction manner, and a target principal axis corresponding to the first set of encoding vectors and the second set of encoding vectors is determined according to a processing result;

[0155] According to the projection coordinates of the plurality of encoding vectors in the first set of encoding vectors and the second set of encoding vectors on the target principal axis, an overlap ratio between the first encoding vector and the second encoding vector is calculated, and the overlap ratio is used as an index value of a third evaluation index for evaluating the accuracy of the initial hierarchical relationship.

[0156] The target principal axis corresponding to the first set of encoding vectors and the second set of encoding vectors is determined according to the processing result, including:

[0157] According to the processing result, a first principal axis vector corresponding to the first set of encoding vectors and a second principal axis vector corresponding to the second set of encoding vectors are determined;

[0158] A direction of a sum vector of the first principal axis vector and the second principal axis vector is used as the target principal axis corresponding to the first set of encoding vectors and the second set of encoding vectors.

[0159] Specifically, after the initial hierarchical relationship between the two to-be-processed keywords is determined, a fourth way of evaluating the accuracy of the initial hierarchical relationship is to determine the concept words corresponding to the to-be-processed keywords in the data graph, encode the concept words, and then reduce the dimension of the encoding vectors to determine the overlap ratio of the encoding vectors according to the dimension reduction processing result.

[0160] The principal component analysis method PCA (Principal Component Analysis) can be used to reduce the dimension of the plurality of encoding vectors in the first encoding vector set and the second encoding vector set, and based on the dimension reduction result, a first principal axis vector corresponding to the first encoding vector set and a second principal axis vector corresponding to the second encoding vector set are determined; then the direction of the sum vector of the first principal axis vector and the second principal axis vector is taken as the target principal axis corresponding to the first encoding vector set and the second encoding vector.

[0161] After the target principal axis is determined, the plurality of encoding vectors in the first encoding vector set and the second encoding vector set can be projected onto the target principal axis, respectively, to determine the overlap ratio between the projection distance of the plurality of encoding vectors in the first encoding vector set on the target principal axis and the projection distance of the plurality of encoding vectors in the second encoding vector set on the target principal axis according to the projection result, and the overlap ratio is taken as the index value of the third evaluation index for evaluating the accuracy of the initial hierarchical relationship. The third evaluation index is the overlap ratio, and the index value is the specific numerical value of the overlap ratio.

[0162] The schematic diagram of the projection result provided by the embodiments of the present specification is shown in Figure 2 The distance L1 between the coordinate point a and the coordinate point b represents the projection distance of the plurality of encoding vectors in the first encoding vector set on the target principal axis, and the coordinate point a and the coordinate point b are the projection coordinates of two encoding vectors in the first encoding vector set on the target principal axis; the distance L2 between the coordinate point c and the coordinate point d represents the projection distance of the plurality of encoding vectors in the second encoding vector set on the target principal axis, and the coordinate point c and the coordinate point d are the projection coordinates of two encoding vectors in the second encoding vector set on the target principal axis.

[0163] Therefore, the overlap ratio between the first encoding vector and the second encoding vector can be obtained by formula 1.

[0164] R=L3 / L1-L3 / L2 Formula 1

[0165] Wherein, L1 represents the distance between the coordinate point a and the coordinate point b, L2 represents the distance between the coordinate point c and the coordinate point d, and L3 represents the distance between the coordinate point b and the coordinate point c.

[0166] After obtaining the accuracy evaluation result of the predicted hierarchical relationship, the index value of the first evaluation index, the second evaluation index and the third evaluation index by the foregoing manner, the target hierarchical relationship between the keywords to be processed can be determined based on the evaluation results of the four dimensions, so as to optimize the initial knowledge graph.

[0167] In step 108, a target hierarchical relationship between the to-be-processed keywords is determined according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result and the index value, and the initial knowledge graph is updated based on the target hierarchical relationship.

[0168] In a specific implementation, determining the target hierarchical relationship between the to-be-processed keywords according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result and the index value includes:

[0169] The accuracy of the initial hierarchical relationship is determined according to the predicted hierarchical relationship, the evaluation result and the index value.

[0170] In a case where the accuracy is greater than a preset accuracy threshold, the initial hierarchical relationship is taken as the target hierarchical relationship between the to-be-processed keywords.

[0171] Specifically, after obtaining the accuracy evaluation result F1, the first evaluation index F2, the second evaluation index F3 and the index value F4 of the third evaluation index, the accuracy of the initial hierarchical relationship between the to-be-processed keywords can be determined based on the four evaluation results.

[0172] In actual application, if the four evaluation results satisfy ((F1>0.5&F2>0.2)or(F2>T1&F1>-1))&(F3<T2or(F4>T3&F3<0.04)), that is, the accuracy of the initial hierarchical relationship between the to-be-processed keywords is greater than the preset accuracy threshold, in this case, the initial hierarchical relationship can be taken as the target hierarchical relationship between the to-be-processed keywords. In a case where T1 is 0.6, T2 is 0.0035 and T3 is 0.3, a better balance between accuracy and recall can be achieved, but the values of T1, T2 and T3 can be determined according to actual requirements, and are not limited herein.

[0173] In a case where the four evaluation results do not satisfy the condition, it is determined that the initial hierarchical relationship between the to-be-processed keywords is not accurate, the target hierarchical relationship between the to-be-processed keywords is determined again, and the initial knowledge graph is updated based on the target hierarchical relationship.

[0174] One embodiment of the present specification obtains the to-be-processed keywords with hierarchical relationships in the initial knowledge graph and the initial hierarchical relationships of the to-be-processed keywords, predicts the hierarchical relationships between the to-be-processed keywords, obtains the predicted hierarchical relationships, and performs accuracy evaluation on the predicted hierarchical relationships to generate corresponding evaluation results, obtains a data graph constructed based on a target data source, and determines the index value of the evaluation index for evaluating the accuracy of the initial hierarchical relationships according to the data graph and the to-be-processed keywords, determines the target hierarchical relationships between the to-be-processed keywords according to the initial hierarchical relationships, the predicted hierarchical relationships, the evaluation results and the index value, and updates the initial knowledge graph based on the target hierarchical relationships.

[0175] The embodiment of the present specification predicts the hierarchical relationships between the to-be-processed keywords, and performs accuracy evaluation on the initial hierarchical relationships between the to-be-processed keywords in the initial knowledge graph based on the data graph of the target data source, so as to determine the target hierarchical relationships between the to-be-processed keywords according to the prediction results and the accuracy evaluation results, and update the initial knowledge graph based on the target hierarchical relationships, which is beneficial to guarantee the accuracy of the knowledge graph construction results.

[0176] The following describes the data processing method provided by the present specification with reference to the accompanying drawings. Figure 3 The data processing method provided by the present specification is further described below with reference to the accompanying drawings. Figure 3 FIG. 1 shows a process flow diagram of a data processing method provided by one embodiment of the present specification, which specifically includes the following steps.

[0177] In step 302, the to-be-processed keywords with hierarchical relationships in the initial knowledge graph and the initial hierarchical relationships of the to-be-processed keywords are obtained.

[0178] In step 304, a to-be-processed sentence is constructed based on the to-be-processed keywords, and the to-be-processed sentence is input into a Bert pre-training model for prediction to obtain the predicted hierarchical relationships between the to-be-processed keywords.

[0179] In step 306, a to-be-evaluated sentence is constructed based on the to-be-processed keywords and the predicted hierarchical relationships, and the to-be-evaluated sentence is input into an electra pre-training model for accuracy evaluation to generate the evaluation results corresponding to the predicted hierarchical relationships respectively.

[0180] In step 308, the feature value corresponding to the predicted hierarchical relationships is determined according to a preset evaluation rule, and the feature value is updated according to the evaluation results.

[0181] In step 310, a data graph constructed based on a target data source is obtained.

[0182] Step 312, determining a concept word in the data graph having a semantic similarity greater than a preset similarity threshold with the to-be-processed keyword.

[0183] Step 314, acquiring parent nodes and child nodes of the data graph having a connection relationship with a node where the concept word is located.

[0184] Step 316, determining a first coincidence ratio of the parent nodes based on the number and information of the nodes of the parent nodes, and determining a second coincidence ratio of the child nodes based on the number and information of the nodes of the child nodes.

[0185] Step 318, determining an index value of a first evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the first coincidence ratio and the second coincidence ratio.

[0186] Step 320, determining a plurality of first concept words in the data graph having a semantic similarity greater than a preset similarity threshold with the first to-be-processed keyword, and determining a plurality of second concept words in the data graph having a semantic similarity greater than a preset similarity threshold with the second to-be-processed keyword.

[0187] Step 322, performing encoding processing on the plurality of first concept words to generate a first encoding vector set, and performing encoding processing on the plurality of second concept words to generate a second encoding vector set.

[0188] Step 324, constructing a first node cluster based on a plurality of encoding vectors in the first encoding vector set and determining a first center point in the first node cluster, and constructing a second node cluster based on a plurality of encoding vectors in the second encoding vector set and determining a second center point in the second node cluster.

[0189] Step 326, calculating a distance between the first center point and the second center point, and taking the distance as an index value of a second evaluation index for evaluating the accuracy of the initial hierarchical relationship.

[0190] Step 328, performing dimensionality reduction processing on a plurality of encoding vectors in the first encoding vector set and the second encoding vector set respectively, and determining target principal axes corresponding to the first encoding vector set and the second encoding vector set according to the processing results.

[0191] Step 330, calculating an overlap ratio between the first encoding vector and the second encoding vector according to projection coordinates of a plurality of encoding vectors in the first encoding vector set and the second encoding vector set on the target principal axes, and taking the overlap ratio as an index value of a third evaluation index for evaluating the accuracy of the initial hierarchical relationship.

[0192] At step 332, a target hierarchical relationship between the to-be-processed keywords is determined according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result and the index value, and the initial knowledge graph is updated based on the target hierarchical relationship.

[0193] The embodiment of the present specification proposes a keyword hierarchical relationship determination method based on a pre-trained Bert model and a Chinese wiki data set that can be obtained at low cost on the basis of the current concept hierarchical algorithm, constructs four effective evaluation indexes based on a Bert MLM model, a wiki subgraph mapping distance and the like, and can realize rapid elimination of noise of the keyword hierarchical relationship, thereby ensuring the accuracy of the determined keyword hierarchical relationship. Moreover, the test results obtained by testing using a public test data set show that the hierarchical relationship between keywords determined by the present solution can achieve an accuracy of 80% and a recall rate of about 80%.

[0194] Corresponding to the method embodiments described above, the present specification also provides data processing device embodiments, Figure 4 A structural schematic diagram of a data processing device provided by one embodiment of the present specification is shown. As shown in the figure, Figure 4 The device comprises:

[0195] The acquisition module 402 is configured to acquire to-be-processed keywords with hierarchical relationships in an initial knowledge graph and initial hierarchical relationships of the to-be-processed keywords.

[0196] The evaluation module 404 is configured to predict the hierarchical relationships between the to-be-processed keywords, obtain predicted hierarchical relationships, and evaluate the accuracy of the predicted hierarchical relationships to generate corresponding evaluation results.

[0197] The determination module 406 is configured to acquire a data graph constructed based on a target data source, and determine index values of evaluation indexes for evaluating the accuracy of the initial hierarchical relationships according to the data graph and the to-be-processed keywords.

[0198] The update module 408 is configured to determine a target hierarchical relationship between the to-be-processed keywords according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result and the index value, and update the initial knowledge graph based on the target hierarchical relationship.

[0199] Optionally, the evaluation module 404 is further configured to:

[0200] construct a to-be-processed sentence based on the to-be-processed keywords, and input the to-be-processed sentence into a prediction model for prediction to obtain the predicted hierarchical relationships between the to-be-processed keywords.

[0201] Optionally, the evaluation module 404 is further configured to:

[0202] construct a first to-be-processed sentence corresponding to the to-be-processed keywords in combination with the first ordering relationship of the to-be-processed keywords and the masking identifier, input the first to-be-processed sentence into a prediction model for prediction, and obtain a first predicted hierarchical relationship between the to-be-processed keywords;

[0203] construct a second to-be-processed sentence corresponding to the to-be-processed keywords in combination with the second ordering relationship of the to-be-processed keywords and the masking identifier, input the second to-be-processed sentence into a prediction model for prediction, and obtain a second predicted hierarchical relationship between the to-be-processed keywords.

[0204] Optionally, the evaluation module 404 is further configured to:

[0205] construct a to-be-evaluated sentence based on the to-be-processed keywords and the predicted hierarchical relationship, input the to-be-evaluated sentence into an evaluation model for accuracy evaluation, and generate an evaluation result corresponding to the predicted hierarchical relationship respectively.

[0206] Optionally, the evaluation module 404 is further configured to:

[0207] construct a to-be-evaluated sentence based on the to-be-processed keywords and the predicted hierarchical relationship, input the to-be-evaluated sentence into an evaluation model for accuracy evaluation, and generate an evaluation result corresponding to the predicted hierarchical relationship respectively;

[0208] Correspondingly, the data processing apparatus further comprises:

[0209] a processing module configured to determine a feature value corresponding to the first predicted hierarchical relationship and / or the second predicted hierarchical relationship according to a preset evaluation rule, and update the feature value according to the evaluation result.

[0210] Optionally, the determination module 406 is further configured to:

[0211] determine a concept word in the data graph that has a semantic similarity greater than a preset similarity threshold with the to-be-processed keywords;

[0212] obtain a parent node and a child node in the data graph that have a connection relationship with the node where the concept word is located;

[0213] determine a first coincidence ratio of the parent node based on the number and node information of the nodes of the parent node, and determine a second coincidence ratio of the child node based on the number and node information of the nodes of the child node;

[0214] According to the first coincidence ratio and the second coincidence ratio, a value of a first evaluation index for evaluating the accuracy of the initial hierarchical relationship is determined.

[0215] Optionally, the to-be-processed keyword includes a first to-be-processed keyword and a second to-be-processed keyword.

[0216] Correspondingly, the determining module 406 is further configured to:

[0217] determine a first concept word in the data graph that has a semantic similarity greater than a preset similarity threshold with the first to-be-processed keyword, and determine a second concept word in the data graph that has a semantic similarity greater than the preset similarity threshold with the second to-be-processed keyword;

[0218] obtain a first parent node and a first child node in the data graph that have a connection relationship with the node where the first concept word is located, and obtain a second parent node and a second child node in the data graph that have a connection relationship with the node where the second concept word is located;

[0219] determine a first coincidence ratio based on the number and node information of the first parent node and the second parent node, and determine a second coincidence ratio based on the number and node information of the first child node and the second child node;

[0220] According to the first coincidence ratio and the second coincidence ratio, a value of a first evaluation index for evaluating the accuracy of the initial hierarchical relationship is determined.

[0221] Optionally, the determining module 406 is further configured to:

[0222] determine a first node number of the first parent node, a second node number of the second parent node, and a third node number of the nodes that exist in the first parent node and the second parent node;

[0223] In a case where a first ratio between the third node number and the first node number is less than a preset threshold, and the first node number is greater than or equal to the second node number, a product of the first ratio and the first node number is determined, and it is judged whether a second ratio of the product to the second node number is greater than or equal to the preset threshold;

[0224] If yes, a third ratio between the second ratio and the preset threshold is determined, and a product of the third ratio and a preset ratio coefficient is taken as a first coincidence ratio.

[0225] Optionally, the determining module 406 is further configured to:

[0226] determine a plurality of first concept words in the data graph having a semantic similarity greater than a preset similarity threshold with the first to-be-processed keyword, and determine a plurality of second concept words in the data graph having a semantic similarity greater than a preset similarity threshold with the second to-be-processed keyword;

[0227] perform encoding processing on the plurality of first concept words to generate a first set of encoding vectors, and perform encoding processing on the plurality of second concept words to generate a second set of encoding vectors;

[0228] construct a first node cluster based on a plurality of encoding vectors in the first set of encoding vectors, and determine a first center point in the first node cluster, and construct a second node cluster based on a plurality of encoding vectors in the second set of encoding vectors, and determine a second center point in the second node cluster;

[0229] calculate a distance between the first center point and the second center point, and take the distance as an index value of a second evaluation index for evaluating accuracy of the initial hierarchical relationship.

[0230] Optionally, the data processing apparatus further comprises a dimension reduction processing module configured to:

[0231] perform dimension reduction processing on a plurality of encoding vectors in the first set of encoding vectors and the second set of encoding vectors respectively, and determine target principal axes corresponding to the first set of encoding vectors and the second set of encoding vectors according to processing results;

[0232] calculate an overlap ratio between the first set of encoding vectors and the second set of encoding vectors according to projection coordinates of a plurality of encoding vectors in the first set of encoding vectors and the second set of encoding vectors on the target principal axes, and take the overlap ratio as an index value of a third evaluation index for evaluating accuracy of the initial hierarchical relationship.

[0233] Optionally, the dimension reduction processing module is further configured to:

[0234] determine a first principal axis vector corresponding to the first set of encoding vectors and a second principal axis vector corresponding to the second set of encoding vectors according to processing results;

[0235] take a direction of a sum vector of the first principal axis vector and the second principal axis vector as the target principal axes corresponding to the first set of encoding vectors and the second set of encoding vectors.

[0236] Optionally, the updating module 408 is further configured to:

[0237] determine an accuracy rate of the initial hierarchical relationship according to the predicted hierarchical relationship, the evaluation result, and the index value;

[0238] In a case where the accuracy is greater than a preset accuracy threshold, the initial hierarchical relationship is taken as a target hierarchical relationship between the keywords to be processed.

[0239] The above is a schematic scheme of the data processing apparatus of the embodiment. It should be noted that the technical scheme of the data processing apparatus and the technical scheme of the data processing method described above belong to the same concept, and the details of the technical scheme of the data processing apparatus which are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0240] Figure 5 A structural block diagram of a computing device 500 according to one embodiment of the present specification is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 through a bus 530, and a database 550 is used to save data.

[0241] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 can include one or more of any type of network interface (e.g., network interface card (NIC)) such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like, wired or wireless.

[0242] In one embodiment of the present specification, the above-mentioned components of the computing device 500 and other components not shown in the Figure 5 may be connected to each other, for example, through a bus. It should be understood that Figure 5 The structural block diagram of the computing device shown is only for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or replaced as needed by those skilled in the art.

[0243] The computing device 500 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a PC. The computing device 500 can also be a mobile or stationary server.

[0244] The processor 520 is configured to execute the following computer-executable instructions, which implement the steps of the data processing method described above when executed by the processor.

[0245] The above is a schematic scheme of the computing device of the embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the data processing method described above belong to the same concept, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0246] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which implement the steps of the data processing method described above when executed by the processor.

[0247] The above is a schematic scheme of the computer-readable storage medium of the embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the data processing method described above belong to the same concept, and the details of the technical scheme of the storage medium that are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0248] An embodiment of the present specification further provides a computer program, which causes a computer to execute the steps of the data processing method described above when the computer program is executed in the computer.

[0249] The above is a schematic scheme of the computer program of the embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the data processing method described above belong to the same concept, and the details of the technical scheme of the computer program that are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0250] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than the order described in the embodiments and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous.

[0251] The computer readable medium can include any entity or apparatus capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium can include appropriate additions or subtractions according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0252] It should be noted that for the foregoing method embodiments, the descriptions are each simply a combination of a series of acts for the sake of brevity, but those skilled in the art should know that the present application is not limited by the order of the acts described, because some steps can be performed in other orders or at the same time in accordance with the present application. In addition, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the acts and modules involved are not necessarily essential to the present application.

[0253] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0254] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not describe all the details and limit the present application to the specific embodiments described. Obviously, according to the content of the present application, many modifications and changes can be made. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and use the present application. The present application is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: obtaining a to-be-processed keyword with a hierarchical relationship in an initial knowledge graph and an initial hierarchical relationship of the to-be-processed keyword; predicting a hierarchical relationship between the to-be-processed keywords, obtaining a predicted hierarchical relationship, and performing accuracy evaluation on the predicted hierarchical relationship to generate a corresponding evaluation result; obtaining a data graph constructed based on a target data source, and determining an index value of an evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the data graph and the to-be-processed keyword, wherein the target data source is a public data source; determining a target hierarchical relationship between the to-be-processed keywords according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result and the index value, and updating the initial knowledge graph based on the target hierarchical relationship.

2. The data processing method of claim 1, wherein the predicting a hierarchical relationship between the to-be-processed keywords, obtaining a predicted hierarchical relationship comprises: constructing a to-be-processed sentence based on the to-be-processed keywords, and inputting the to-be-processed sentence into a prediction model for prediction to obtain a predicted hierarchical relationship between the to-be-processed keywords.

3. The data processing method of claim 2, wherein the constructing a to-be-processed sentence based on the to-be-processed keywords, and inputting the to-be-processed sentence into a prediction model for prediction to obtain a predicted hierarchical relationship between the to-be-processed keywords comprises: constructing a first to-be-processed sentence corresponding to the to-be-processed keywords in combination with a first ordering relationship and a masking identifier of the to-be-processed keywords, and inputting the first to-be-processed sentence into a prediction model for prediction to obtain a first predicted hierarchical relationship between the to-be-processed keywords; constructing a second to-be-processed sentence corresponding to the to-be-processed keywords in combination with a second ordering relationship and a masking identifier of the to-be-processed keywords, and inputting the second to-be-processed sentence into a prediction model for prediction to obtain a second predicted hierarchical relationship between the to-be-processed keywords.

4. The data processing method of any one of claims 1 to 2, wherein the performing accuracy evaluation on the predicted hierarchical relationship to generate a corresponding evaluation result comprises: constructing an evaluation sentence based on the to-be-processed keywords and the predicted hierarchical relationship, and inputting the evaluation sentence into an evaluation model for accuracy evaluation to generate an evaluation result corresponding to the predicted hierarchical relationship respectively.

5. The data processing method of claim 3, wherein the performing accuracy evaluation on the predicted hierarchical relationship to generate a corresponding evaluation result comprises: constructing an evaluation sentence based on the to-be-processed keywords and the predicted hierarchical relationship, and inputting the evaluation sentence into an evaluation model for accuracy evaluation to generate an evaluation result corresponding to the predicted hierarchical relationship respectively; correspondingly, after the corresponding evaluation result is generated, the method further comprises: determining a feature value corresponding to the first predicted hierarchical relationship and / or the second predicted hierarchical relationship according to a preset evaluation rule, and updating the feature value according to the evaluation result.

6. The data processing method of claim 1, wherein the determining of the index value of the evaluation index for evaluating the accuracy of the initial hierarchical relationship based on the data graph and the keyword to be processed comprises: determining a concept word in the data graph having a semantic similarity greater than a preset similarity threshold with the keyword to be processed; obtaining a parent node and a child node of the data graph having a connection relationship with a node in which the concept word is located; determining a first coincidence ratio of the parent node based on a node quantity and node information of the parent node, and determining a second coincidence ratio of the child node based on a node quantity and node information of the child node; and determining an index value of a first evaluation index for evaluating the accuracy of the initial hierarchical relationship based on the first coincidence ratio and the second coincidence ratio.

7. The data processing method of claim 1, wherein the keyword to be processed comprises a first keyword to be processed and a second keyword to be processed; and the determining of the index value of the evaluation index for evaluating the accuracy of the initial hierarchical relationship based on the data graph and the keyword to be processed comprises: determining a first concept word in the data graph having a semantic similarity greater than a preset similarity threshold with the first keyword to be processed, and determining a second concept word in the data graph having a semantic similarity greater than a preset similarity threshold with the second keyword to be processed; obtaining a first parent node and a first child node of the data graph having a connection relationship with a node in which the first concept word is located, and obtaining a second parent node and a second child node of the data graph having a connection relationship with a node in which the second concept word is located; determining a first coincidence ratio based on a node quantity and node information of the first parent node and the second parent node, and determining a second coincidence ratio based on a node quantity and node information of the first child node and the second child node; and determining an index value of a first evaluation index for evaluating the accuracy of the initial hierarchical relationship based on the first coincidence ratio and the second coincidence ratio.

8. The data processing method of claim 7, wherein the determining of the first coincidence ratio based on the node quantity and the node information of the first parent node and the second parent node comprises: determining a first node quantity of the first parent node, a second node quantity of the second parent node, and a third node quantity of nodes that exist in the first parent node and the second parent node and that overlap; determining a product of a first ratio between the third node quantity and the first node quantity and the first node quantity in a case where the first ratio is less than a preset threshold and the first node quantity is greater than or equal to the second node quantity, and determining whether a second ratio of the second ratio and the preset threshold is greater than or equal to the preset threshold; and if so, determining a third ratio between the second ratio and the preset threshold, and taking a product of the third ratio and a preset ratio coefficient as the first coincidence ratio.

9. The data processing method of claim 1, wherein the keyword to be processed comprises a first keyword to be processed and a second keyword to be processed. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Correspondingly, the determining, according to the data graph and the to-be-processed keyword, of an index value of an evaluation index for evaluating accuracy of the initial hierarchical relationship includes: determining a plurality of first concept words in the data graph that have a semantic similarity greater than a preset similarity threshold with the first to-be-processed keyword, and determining a plurality of second concept words in the data graph that have a semantic similarity greater than the preset similarity threshold with the second to-be-processed keyword; performing encoding processing on the plurality of first concept words to generate a first set of encoding vectors, and performing encoding processing on the plurality of second concept words to generate a second set of encoding vectors; constructing a first node cluster based on a plurality of encoding vectors in the first set of encoding vectors, and determining a first center point in the first node cluster, constructing a second node cluster based on a plurality of encoding vectors in the second set of encoding vectors, and determining a second center point in the second node cluster; calculating a distance between the first center point and the second center point, and taking the distance as an index value of a second evaluation index for evaluating accuracy of the initial hierarchical relationship.

10. The data processing method of claim 9, after the generating a second set of encoding vectors, further comprising: performing dimension reduction processing on a plurality of encoding vectors in the first set of encoding vectors and the second set of encoding vectors respectively, and determining target principal axes corresponding to the first set of encoding vectors and the second set of encoding vectors according to a processing result; calculating an overlap ratio between the first set of encoding vectors and the second set of encoding vectors according to projection coordinates of a plurality of encoding vectors in the first set of encoding vectors and the second set of encoding vectors on the target principal axes, and taking the overlap ratio as an index value of a third evaluation index for evaluating accuracy of the initial hierarchical relationship.

11. The data processing method of claim 10, the determining target principal axes corresponding to the first set of encoding vectors and the second set of encoding vectors according to a processing result includes: determining a first principal axis vector corresponding to the first set of encoding vectors and a second principal axis vector corresponding to the second set of encoding vectors according to a processing result; taking a direction of a sum vector of the first principal axis vector and the second principal axis vector as the target principal axes corresponding to the first set of encoding vectors and the second set of encoding vectors.

12. The data processing method of claim 10, the determining, according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result, and the index value, of a target hierarchical relationship between the to-be-processed keywords includes: determining an accuracy rate of the initial hierarchical relationship according to the predicted hierarchical relationship, the evaluation result, and the index value; in a case where the accuracy rate is greater than a preset accuracy threshold, taking the initial hierarchical relationship as the target hierarchical relationship between the to-be-processed keywords.

13. A data processing apparatus, comprising: an acquisition module configured to acquire to-be-processed keywords having a hierarchical relationship in an initial knowledge graph and an initial hierarchical relationship of the to-be-processed keywords; An evaluation module configured to predict a hierarchical relationship between the keywords to be processed, obtain a predicted hierarchical relationship, and evaluate the predicted hierarchical relationship to generate a corresponding evaluation result; A determination module configured to obtain a data graph constructed based on a target data source, and determine an index value of an evaluation index for evaluating the accuracy of the initial hierarchical relationship according to the data graph and the keywords to be processed, wherein the target data source is a public data source; An updating module configured to determine a target hierarchical relationship between the keywords to be processed according to the initial hierarchical relationship, the predicted hierarchical relationship, the evaluation result, and the index value, and update the initial knowledge graph based on the target hierarchical relationship.

14. A computing device comprising: a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, and the computer executable instructions, when executed by the processor, implement the steps of the data processing method in any one of claims 1 to 12.

15. A computer readable storage medium storing computer executable instructions, and the computer executable instructions, when executed by a processor, implement the steps of the data processing method in any one of claims 1 to 12.

16. A computer program product, characterised in that, computer instructions, and the computer instructions, when executed by a processor, implement the steps of the data processing method in any one of claims 1 to 12.

Images