Data processing method, device, computer equipment and readable storage medium

Abstract

The embodiment of the present application discloses a data processing method, apparatus, computer equipment and readable storage medium, the data processing method comprising: obtaining a knowledge seed set, the knowledge seed set comprising knowledge description information of a text; matching the knowledge description information with text data in a corpus set to obtain a knowledge extraction template; extracting candidate knowledge description information from the corpus set according to the knowledge extraction template; determining the template confidence of the knowledge extraction template, and determining the knowledge confidence of the candidate knowledge description information according to the template confidence; updating the knowledge seed set according to the knowledge confidence and the candidate knowledge description information; and updating the knowledge graph according to the updated knowledge seed set when the updated knowledge seed set meets the convergence condition. By adopting the present application, more knowledge description information can be efficiently extracted from the massive text data on the Internet, continuously providing source data with wide coverage and high accuracy for the knowledge graph.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus, computer device, and readable storage medium. Background Art

[0002] A knowledge graph is a large-scale semantic network represented by a graph data structure, where nodes and edges together describe objects in the objective world and their relationships. Currently, with the continuous development of intelligent information service applications, knowledge graphs have been widely used in intelligent search, intelligent question-answering, personalized recommendations, intelligence analysis, anti-fraud and other fields.

[0003] Currently, the data for mainstream knowledge graphs in the industry all originates from the internet, and rule-based extraction is the primary method for obtaining knowledge description information from internet data. For example, SPO extraction is performed by matching text in a specific format, while unstructured data sources such as web pages can be extracted by formulating parsing rules corresponding to the front-end DOM tree. However, internet text data is heterogeneous, diverse, and of varying quality. There are too many text representations that can contain knowledge description information, and manually configured rules are difficult to cover all text representations in the corpus, resulting in incomplete extracted knowledge description information. Summary of the Invention

[0004] The embodiments of the present application provide a data processing method, apparatus, computer device, and readable storage medium, which can efficiently extract more comprehensive knowledge description information.

[0005] On the one hand, an embodiment of the present application provides a data processing method based on a knowledge graph, including:

[0006] Acquire a knowledge seed set, where the knowledge seed set includes knowledge description information of the text;

[0007] Match the knowledge description information with the text data in the corpus collection to obtain a knowledge extraction template;

[0008] Extract candidate knowledge description information from the corpus collection according to the knowledge extraction template;

[0009] Determine the template confidence of the knowledge extraction template, and determine the knowledge confidence of the candidate knowledge description information according to the template confidence;

[0010] Update the knowledge seed set according to the knowledge confidence and candidate knowledge description information;

[0011] When the updated knowledge seed set meets the convergence condition, the knowledge graph is updated according to the updated knowledge seed set.

[0012] On the one hand, an embodiment of the present application provides a data processing device based on a knowledge graph, including:

[0013] An acquisition module is used to acquire a knowledge seed set, where the knowledge seed set includes textual knowledge description information;

[0014] The template generation module is used to match the knowledge description information with the text data in the corpus to obtain the knowledge extraction template;

[0015] The knowledge extraction module is used to extract candidate knowledge description information from the corpus collection according to the knowledge extraction template;

[0016] A first determination module is used to determine the template confidence of the knowledge extraction template;

[0017] A second determination module is used to determine the knowledge confidence of the candidate knowledge description information according to the template confidence;

[0018] A first updating module is used to update the knowledge seed set according to the knowledge confidence and candidate knowledge description information;

[0019] The second updating module is used to update the knowledge graph according to the updated knowledge seed set.

[0020] Among them, the template generation module includes:

[0021] The query unit is used to traverse the text data in the corpus set and take the paragraphs in the corpus set containing knowledge description information as the target paragraph corpus;

[0022] Basic generation unit, used to generate basic extraction templates based on target paragraph corpus and knowledge description information;

[0023] A generalization generation unit, configured to generate S generalization extraction templates based on the basic extraction template; S is a positive integer;

[0024] The slot filling unit is used to fill the slots of the S generalized extraction templates respectively to obtain the knowledge extraction template.

[0025] Among them, the basic generation unit includes:

[0026] A character acquisition subunit is used to obtain textual relationship characters corresponding to knowledge description information;

[0027] The character replacement subunit is used to replace the knowledge description information with text relationship characters in the target paragraph corpus to obtain the basic extraction template.

[0028] Among them, the generalization generation unit includes:

[0029] The window acquisition subunit is used to acquire text-related characters and characters adjacent to the text-related characters in the basic extraction template by sliding the character window; the adjacent characters are the characters covered by the character window;

[0030] The generalized template generating subunit is used to generate S generalized extraction templates according to the textual relation characters and the adjacent characters; each generalized extraction template includes at least one character among the textual relation characters and the adjacent characters.

[0031] Among them, S generalized extraction templates include generalized extraction template L i , i is a positive integer less than or equal to S;

[0032] Slot filler unit, including:

[0033] Template segmentation subunit, used to extract the generalized template L i Perform word segmentation to obtain text relationship characters and associated word segmentations, and determine the knowledge description information and associated word segmentations associated with the text relationship characters as the word segmentations to be replaced;

[0034] The position determination subunit is based on the text relationship characters of the word to be replaced in the generalized extraction template L i The sorting position in is used as the extraction position index;

[0035] The vocabulary acquisition subunit is used to acquire a slot vocabulary, which includes slot segmentation words, text segmentation words, and a mapping relationship between slot segmentation words and text segmentation words;

[0036] The word search subunit is configured to, if a text word identical to the word to be replaced is found in the slot word table, use the slot word having a mapping relationship with the text word identical to the word to be replaced as the target slot word;

[0037] The word replacement subunit is used to replace the generalized extraction template with the word segmentation according to the target slot segmentation to obtain the slot replacement template;

[0038] The extraction template generation subunit is used to generate a generalized extraction template L according to the slot replacement template and extraction position index i Corresponding knowledge extraction template.

[0039] Among them, the knowledge extraction module includes:

[0040] The text filling unit is used to fill the slots of the text data in the corpus collection to obtain replacement text data;

[0041] A text matching unit, used to match the knowledge extraction template with the replacement text data;

[0042] The knowledge acquisition unit is used to obtain an extraction position index if the knowledge extraction template successfully matches the replacement text data, and obtain candidate knowledge description information in the replacement text data according to the extraction position index.

[0043] The first determination module includes:

[0044] a legitimacy determination unit, configured to verify the knowledge legitimacy of the candidate knowledge description information based on the candidate knowledge description information and the knowledge seed set;

[0045] A current confidence calculation unit, used to determine the current confidence of the knowledge extraction template according to the legitimacy of the knowledge;

[0046] A confidence acquisition unit is used to obtain the generation timestamp of each historical confidence in the historical confidence, and use the historical confidence corresponding to the largest generation timestamp as the relevant historical confidence;

[0047] The template confidence calculation unit is used to determine the template confidence of the knowledge extraction template according to the relevant historical confidence and the current confidence.

[0048] Among them, the knowledge extraction template includes the knowledge extraction template H g , g is a positive integer less than or equal to the total number of knowledge extraction templates;

[0049] Legality determination unit, including:

[0050] The to-be-matched information search subunit is used to search the knowledge description information that describes the same entity as the candidate knowledge description information in the knowledge seed set as the to-be-matched knowledge description information;

[0051] The legitimacy judgment subunit is used to determine the knowledge legitimacy of the candidate knowledge description information based on the candidate knowledge description information and the knowledge description information to be matched;

[0052] The current confidence calculation unit includes:

[0053] Statistics subunit, used to extract template H through knowledge g Among the extracted candidate knowledge description information, the number of candidate knowledge description information whose knowledge legitimacy is legal is counted as a legal parameter;

[0054] The statistical subunit is also used to extract the template H through knowledge g Among the extracted candidate knowledge description information, the number of knowledge description information whose knowledge legitimacy is illegal is counted as the illegal parameter;

[0055] The current confidence determination subunit is used to determine the knowledge extraction template H based on legal parameters and illegal parameters g The current confidence level.

[0056] The candidate knowledge description information includes the candidate knowledge description information K j , j is a positive integer;

[0057] The second template includes:

[0058] The target template determination unit is used to extract the knowledge description information K j The knowledge extraction template of is used as the target knowledge extraction template;

[0059] a target confidence determination unit, configured to use the template confidence of the target knowledge extraction template as the target template confidence;

[0060] Knowledge confidence calculation unit, used to determine the target knowledge description information K according to the target confidence j knowledge confidence.

[0061] The first update module includes:

[0062] a trusted knowledge determination unit, configured to use the target knowledge description information as trusted knowledge description information when the knowledge confidence of the target knowledge description information is greater than a trusted confidence threshold;

[0063] The seed set updating unit is used to add the trusted knowledge description information to the knowledge seed set.

[0064] The second update module includes:

[0065] The knowledge sorting unit is used to sort the candidate knowledge description information in the updated knowledge seed set from high to low according to the knowledge confidence, and use the first L knowledge description information as the knowledge description information to be stored; L is a positive integer;

[0066] The knowledge adding unit is used to add the knowledge description information to be stored into the knowledge graph.

[0067] The data processing device further includes:

[0068] The judgment module is used to determine whether the updated knowledge seed set meets the convergence condition.

[0069] The judgment module includes:

[0070] a first judgment unit, configured to determine that the updated knowledge seed set satisfies a convergence condition if the knowledge seed set and the updated knowledge seed set are the same;

[0071] The second judgment unit is used to determine whether the updated knowledge seed set meets the convergence condition based on the historical confidence of the knowledge extraction template and the template confidence.

[0072] The second judgment unit includes:

[0073] The historical confidence acquisition subunit is used to obtain M historical confidences corresponding to the knowledge extraction template, where M is a positive integer;

[0074] The calculation judgment subunit is used to determine that the updated knowledge seed set meets the convergence condition if the absolute value of the difference between the template confidence and each of the M historical confidences is less than the convergence threshold.

[0075] The acquisition module is specifically used to obtain the knowledge seed set in the xth iteration;

[0076] The above-mentioned data processing device further includes:

[0077] An iteration module is used to use the updated knowledge seed set as the knowledge seed set in the x+1th iteration when the updated knowledge seed set does not meet the convergence condition, continue to obtain the knowledge extraction template and candidate knowledge description information in the x+1th iteration based on the knowledge seed set in the x+1th iteration, and iteratively update the knowledge seed set in the x+1th iteration based on the knowledge extraction template and candidate knowledge description information in the x+1th iteration.

[0078] On one hand, an embodiment of the present application provides a computer device, including: a processor, a memory, and a network interface;

[0079] The above-mentioned processor is connected to the above-mentioned memory and the above-mentioned network interface, wherein the above-mentioned network interface is used to provide data communication function, the above-mentioned memory is used to store computer programs, and the above-mentioned processor is used to call the above-mentioned computer program to execute the method in the embodiment of the present application.

[0080] On one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the method in the embodiment of the present application is executed.

[0081] On the one hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method in the embodiment of the present application.

[0082] The embodiment of the present application proposes to match the knowledge description information in the knowledge seed set with the text data in the corpus set to obtain a knowledge extraction template, and then use the knowledge extraction template to extract the knowledge description information in the corpus set, calculate the template confidence of the knowledge extraction template according to the knowledge legitimacy of the extracted knowledge description information, and then calculate the confidence of the knowledge description information according to the template confidence, and add the knowledge description information with high confidence to the knowledge seed set, so as to obtain a new seed set to extract the knowledge description information of the corpus. By continuously and automatically iterating the above process, the knowledge seed set and the knowledge extraction template are finally continuously expanded to achieve the result of mining the knowledge description information and extraction template of all relevant intentions contained in the corpus. The embodiment of the present application is initially based on credible, small amounts of knowledge description information and automatically mined knowledge extraction templates, and can iteratively mine more new knowledge description information from the corpus set, that is, it can efficiently extract more and more comprehensive knowledge description information from the massive text data on the Internet, and continuously provide source data with wide coverage and high accuracy for the knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0084] Figure 1 This is a network architecture diagram provided by an embodiment of the present application;

[0085] Figure 2a This is a schematic diagram of a data interaction scenario provided by an embodiment of the present application;

[0086] Figure 2b This is a schematic diagram of an iterative extraction scenario provided in an embodiment of the present application;

[0087] Figure 3a This is a flow chart of a data processing method provided by an embodiment of the present application;

[0088] Figure 3b This is a flow chart of a knowledge extraction method provided in an embodiment of the present application;

[0089] Figure 4 This is a schematic diagram of a process for generating a knowledge extraction template provided in an embodiment of the present application;

[0090] Figure 5 Schematic diagram of a character window covering basic extraction template provided in an embodiment of the present application;

[0091] Figure 6 This is a schematic diagram of a process for extracting knowledge description information provided by an embodiment of the present application;

[0092] Figure 7 This is a schematic diagram of a process for calculating template confidence provided in an embodiment of the present application;

[0093] Figure 8 This is a schematic diagram of a knowledge confidence calculation process provided in an embodiment of the present application;

[0094] Figure 9 is a structural diagram of a data processing device provided in an embodiment of the present application;

[0095] Figure 10 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0096] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this invention.

[0097] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive field within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0098] Natural language processing (NLP) is a key area of research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to the study of linguistics. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0099] The solutions provided in the embodiments of this application involve artificial intelligence natural language processing technology and deep learning technology, which are specifically described through the following embodiments. Figure 1 , Figure 1 This is a network architecture diagram provided by the embodiment of this application. Figure 1 As shown, the network architecture may include a business server 1000 and a background server cluster, wherein the background server cluster may include multiple background servers, such as Figure 1 As shown, it may specifically include a backend server 100a, a backend server 100b, a backend server 100c, ..., a backend server 100n. Figure 1 As shown, the backend server 100a, the backend server 100b, the backend server 100c, ..., the backend server 100n can respectively establish a network connection with the above-mentioned business server 1000, so that each backend server can exchange data with the business server 1000 through the network connection.

[0100] like Figure 1 As shown, a knowledge graph is stored in the business server 1000. The business server 1000 can receive business data from each background business server, perform knowledge extraction based on the business data to obtain new credible knowledge description information, and then update the knowledge graph based on the new credible knowledge description information. The business data may include a corpus set for knowledge extraction and a knowledge seed set for knowledge extraction. The corpus set may contain text data that may have knowledge description information; the knowledge seed set may contain a small amount of credible knowledge description information. Knowledge description information may be a description of knowledge, or a set of conventions for knowledge, a data structure that can be accepted by a computer to describe knowledge, such as a Subject-Predication-Object (SPO) triple.

[0101] Each backend server in this application corresponds to a user terminal, and can respond to the user terminal's knowledge extraction instructions for the knowledge graph in the business server 1000, and initiate a business request for knowledge extraction to the business server 1000. When the business server 1000 receives a business request from a certain backend server, it will obtain the business data from the backend server to perform knowledge extraction and update the knowledge graph. During the process of knowledge extraction by the business server 1000, the extraction results can be sent to the backend server that initiated the business request to it. Each backend server in this application can access the knowledge graph in the business server 1000 at any time, and can also obtain the knowledge description information in the knowledge graph.

[0102] It is understandable that the method provided in the embodiments of the present application can be executed by a computer device, and the computer device can be the business server 1000 described above. The business server 1000 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0103] Among them, the background server and the business server can be directly or indirectly connected through wired or wireless communication, and this application does not impose any restrictions on this.

[0104] For easier understanding, see Figure 2a , Figure 2a This is a schematic diagram of a data interaction scenario provided by an embodiment of the present application. Figure 2a The background server shown can be the above Figure 1 The backend server 100a in the backend server cluster shown in the figure can be the above-mentioned Figure 1 The business server 1000 shown in FIG. Figure 2a The user terminal shown can be understood as the user terminal corresponding to the background server 100a. Figure 2aAs shown, the background server can respond to the knowledge extraction instruction from the corresponding user terminal, initiate a business request for knowledge extraction to the business server, and send the business data used for this knowledge extraction to the business server. After receiving the business data, the business server will obtain the corpus set and the knowledge seed set from the business data, and then perform knowledge extraction on the corpus set based on the knowledge seed set to obtain candidate knowledge description information, and then update the knowledge seed set based on the candidate knowledge description information. The business server will determine the convergence of the updated knowledge seed set. If it does not converge, the updated knowledge seed set will be used as the knowledge seed set in the new round of knowledge extraction, and knowledge extraction will continue with the corpus set; if the updated knowledge seed set converges, the knowledge graph will be updated based on the updated knowledge seed set, that is, new knowledge description information will no longer be mined.

[0105] Among them, such as Figure 2a As shown, the knowledge extraction instruction sent by the user terminal may include a knowledge seed set generation instruction, which determines the way the business server obtains the knowledge seed set. The knowledge seed set obtained by the business server can be generated based on the knowledge description information stored in the knowledge graph or manually created knowledge description information. Figure 2a As shown, the background server will transmit business data according to the generation instruction. If the generation instruction is the first generation instruction, the background server will transmit the corpus set used for this knowledge extraction to the business server according to the knowledge extraction instruction; if the generation instruction is the second generation instruction, the background server will obtain the knowledge seed set from the user terminal and transmit it together with the corpus set to the business server.

[0106] Among them, the knowledge description information in the embodiment of the present application can be an SPO triple, where S (Subject) represents the subject of knowledge, P (Predicate) represents the predicate of knowledge, which is a term used to describe entity characteristics or the relationship between entities, and O (Object) represents the object of knowledge. "Xiao Zhang-wife-Xiao Zhu" is a set of SPO triples, Xiao Zhang is the subject, wife is the predicate, and object is Xiao Zhu. The knowledge graph is composed of some interconnected entities and their attributes or relationships, that is, it contains multiple sets of knowledge description information. The purpose of knowledge extraction is to extract new knowledge description information from the corpus collection, add more entities and their attributes or relationships to it, and make the knowledge graph more and more perfect.

[0107] Among them, the knowledge seed set in the embodiment of the present application may contain multiple groups of knowledge description information that describe different entities but have the same specified intention. At this time, the candidate knowledge description information obtained through multiple rounds of knowledge extraction is also the knowledge description information that describes the specified intention. For ease of understanding, the embodiment of the present application takes the SPO triple used to describe the relationship between entities as an example, such as Figure 2a As shown, before knowledge extraction, the business server might obtain the knowledge seed set {Xiaoming - Wife - Xiaoyue, Xiaoqiang - Wife - Xiaolan}. In this knowledge seed set, both sets of SPO triples describe the "wife" relationship. It's understandable that when this knowledge seed set is matched with the corpus, the resulting knowledge description information also contains SPO triples describing the "wife" relationship.

[0108] Knowledge extraction can be achieved by matching the knowledge description information in the knowledge seed set with the corpus set to obtain a knowledge extraction template. The knowledge extraction template is then used to extract the text data in the sentence set to obtain candidate knowledge description information. Credible candidate knowledge description information is then selected and added to the knowledge seed set to obtain an updated knowledge seed set. If the updated knowledge seed set does not converge, the business server will continue to extract knowledge using the updated knowledge seed set. The business server will continuously iterate the above process until the iteration stabilizes, selecting new credible knowledge description information and updating the knowledge graph.

[0109] For further understanding, please refer to Figure 2b , Figure 2b This is a schematic diagram of an iterative extraction scenario provided in an embodiment of the present application. Figure 2b This shows the update changes of the knowledge seed set of the business server from the xth iteration to the x+1th iteration. Figure 2bAs shown, in the xth iteration, the knowledge seed set includes knowledge description information a. This is combined with the corpus set S to perform the aforementioned knowledge extraction process, resulting in the knowledge extraction template set {A1, A2} and the candidate knowledge set {a, b, c, d, e}. The knowledge extraction template set contains all knowledge extraction templates obtained by matching the knowledge seed set with the corpus set. A knowledge extraction template is a paradigm of textual language expression. For example, knowledge extraction template A1 could be "[D:person]'s wife is [D:person]_1,2." [D:person] represents a character slot and can match text corpora representing characters. The numbers 1 and 2 in knowledge extraction template A1 are extraction position indices. Multiple slots can exist in the same template, and the extraction position index describes the location of the text corpus to be extracted. When the knowledge extraction template matches a paragraph corpus with the same language paradigm, the text corpus corresponding to the slot is extracted based on the extraction position index to generate the knowledge description information. For example, when extraction template A1 matches "Xiao Ming's wife is Xiao Yue," the business server extracts Xiao Ming from the first slot and Xiao Yue from the second slot based on the extraction position index, obtaining the candidate knowledge description information "Xiao Ming - Wife - Xiao Yue." The candidate knowledge set contains the candidate knowledge description information extracted from all templates in the knowledge extraction template set.

[0110] like Figure 2b As shown, the business server calculates the confidence of the knowledge extraction template to obtain the template confidence. Template confidence refers to the trustworthiness of the knowledge extraction template. Then, based on the template confidence, the knowledge confidence of the extracted candidate knowledge description information can be calculated. With the knowledge confidence of the candidate knowledge description information, candidates with high knowledge confidence can be selected and added to the original knowledge seed set, resulting in the updated knowledge seed set {a, b, c}.

[0111] like Figure 2bAs shown in the figure, in the xth iteration, the knowledge seed set used before knowledge extraction is {a}, and the updated knowledge seed set obtained after knowledge extraction is {a, b, c}. There is a difference between the two, indicating that the knowledge seed set does not meet the convergence condition and needs to be iterated further. In the x+1th iteration, the knowledge seed set used is {a, b, c} (i.e., the updated knowledge seed set generated in the xth iteration). The above knowledge extraction process is continued with this knowledge seed set {a, b, c} together with the corpus set S, and the knowledge extraction template set {A1, A2, B1, C1, C2, C3} and the candidate knowledge set {a, b, c, d, e, f, g, h, i, j} can be obtained. Then, the template confidence of the knowledge extraction template in the knowledge extraction template set is calculated, and then the confidence of the candidate knowledge description information in the candidate knowledge set is calculated based on the template confidence. The candidate knowledge description information with high knowledge confidence is selected from the candidate knowledge set and added to the original knowledge seed set {a, b, c} to obtain the updated knowledge seed set {a, b, c, i, g}.

[0112] like Figure 2b As shown, the updated knowledge seed set {a, b, c} in the xth iteration is used as the knowledge seed set used in the x+1th iteration. The business server can use the knowledge description information in the x+1th iteration. Compared with the xth iteration, the candidate knowledge description information b and the candidate knowledge description information c are added. Therefore, the obtained knowledge extraction templates are more than those in the xth iteration. The newly obtained knowledge extraction templates can extract new candidate knowledge description information. The candidate knowledge description information obtained in the x+1th iteration is naturally more than the candidate knowledge description information obtained in the xth iteration. Finally, the updated knowledge seed set {a, b, c, i, g} is added with high-confidence candidate knowledge description information i and candidate knowledge description information g. Comparing the updated knowledge seed set {a, b, c, i, g} with the knowledge seed set {a, b, c} used in the x+1 iteration, we find that the convergence condition is still not met. In this case, the updated knowledge seed set from the x+1 iteration is used as the knowledge seed set for the x+2 iteration, and new candidate knowledge description information is continuously extracted from the corpus set S. If the convergence condition is not met, the business server will continue to iterate, continuously expanding the knowledge extraction template set, candidate knowledge set, and knowledge seed set.

[0113] In the kth iteration, if the knowledge seed set used is the same as the updated knowledge seed set, the convergence condition is considered to be met and the iteration is stopped. It is understood that in order to more likely ensure that all knowledge description information in the corpus is mined, it can be set that the knowledge seed set remains unchanged during m consecutive iterations, in which case the convergence condition is considered to be met and the iteration is stopped.

[0114] For further understanding, please see Figure 3a , Figure 3a This is a flow chart of a data processing method provided by an embodiment of the present application. The method can be performed by a business server (such as the above Figure 1 The business server 1000 in the corresponding embodiment is executed, and the embodiment of the present application takes the method executed by the above-mentioned business server as an example for explanation. Figure 3a As shown, the process may include:

[0115] S101: Acquire a knowledge seed set, where the knowledge seed set includes textual knowledge description information.

[0116] Specifically, the knowledge seed set refers to a set of existing credible knowledge description information that is relied upon in the knowledge extraction process. In the embodiment of the present application, knowledge extraction is a continuously iterative process. In the first round of iteration, the knowledge seed set can be constructed and obtained from multiple sources: for example, the head, accurate knowledge description information that already exists in the knowledge graph, or a set of reliable knowledge description information that is manually constructed and maintained. After the first round of knowledge extraction, the knowledge seed set is updated, and part of the extracted knowledge description information is added to the existing knowledge seed set to obtain an updated knowledge seed set, which is used as the knowledge seed set for the second round of knowledge extraction. In subsequent iterations, the knowledge seed set obtained in each round is the updated knowledge seed set in the previous round of knowledge extraction. By iterating in this way, the existing knowledge seed set can be continuously expanded, and all hidden knowledge of the specified intent in the corpus can be eventually excavated.

[0117] Specifically, the specified intent refers to a specific relationship or a specific attribute. That is to say, the knowledge description information in the knowledge seed set can be knowledge description information containing the same specified intent. For ease of understanding, the embodiment of the present application takes the SPO triples used to describe the relationship between entities as an example for explanation. For example, if the knowledge of all the characters' birth dates contained in the corpus is extracted through the embodiment of the present application, the knowledge description information in the knowledge seed set should also be the birth date of the description. To create a knowledge seed set for the date of birth, the user can manually add multiple groups of SPO triples whose predicates are the date of birth; or the SPO set about the date of birth in the knowledge graph can be exported as a knowledge seed set. The knowledge seed set can be {Xiao Zhang - Date of Birth - September 27, 1961, Xiao Cheng - Date of Birth - April 7, 1954, ...}.

[0118] S102: Match the knowledge description information with the text data in the corpus collection to obtain a knowledge extraction template.

[0119] Specifically, the corpus mentioned in the embodiments of the present application is general text data from the Internet. Among them, text data may include structured data, semi-structured data and unstructured data. There is a huge amount of unstructured data on the Internet, which is the subject text data that constitutes the corpus collection. Among them, the unstructured data can use the introduction content of encyclopedia websites or vertical URLs, or directly use general information flow articles, online query logs and even other descriptive text data. The corpus collection can be generated by the background server, and then the business server can obtain it directly from the background server.

[0120] Specifically, if reliable knowledge description information can be mined from a certain corpus, then it can be further assumed that any text that conforms to the linguistic paradigm of that corpus can find corresponding knowledge description information. The linguistic paradigm contained in this text is called a template. Taking the knowledge description information as SPO triples as an example, when a certain corpus contains a pair of Subject and Object, it is believed that the corpus may contain a relationship between the Subject and Object with respect to the Predicate. Furthermore, it can be further assumed that any text that conforms to the linguistic paradigm of that corpus can find a corresponding SPO triple.

[0121] Specifically, the embodiment of the present application matches a knowledge seed set with a corpus set to obtain a template capable of extracting knowledge description information. The knowledge seed set includes multiple pieces of knowledge description information. Taking one piece of knowledge description information as an example, the service server will use the paragraph containing the entity data in the knowledge description information as the target paragraph corpus, then obtain the textual relationship characters corresponding to the knowledge description information. In the target paragraph corpus, the entity text data in the knowledge description information will be replaced with the textual relationship characters to obtain the basic extraction template. For example, the knowledge description information is the SPO triple: Xiao Zhang - Date of Birth - September 27, 1961. The corpus set is searched for a paragraph containing the entity data in the knowledge description information as the target paragraph. For the SPO triple, the paragraph containing the Subject and Object is obtained. The Subject and Object are the textual relationship characters corresponding to the knowledge description information. The Subject of the SPO triple is Xiao Zhang, and the Object is September 27, 1961. If the text data in the corpus set contains a paragraph of corpus such as "Xiao Zhang was born on September 27, 1961 in XX, and his native place is Xinhui, Guangdong," Then we take this paragraph of corpus as the target paragraph corpus, and use text relationship characters to replace the entity text data. The basic extraction template obtained can be "Subject-Object was born in XX and his hometown is Xinhui, Guangdong".

[0122] In a corpus, multiple paragraphs of text data may contain the knowledge description information, so a single knowledge description information can generate multiple basic extraction templates. Alternatively, the corpus may not contain text data containing the knowledge description information, so there will be no corresponding basic extraction template for the knowledge description information. It is understood that each knowledge description information in the knowledge seed set will be matched with text data in the corpus.

[0123] Optionally, if only the direct matching result between the corpus and the knowledge description information is used as a template, it is inevitable that the specificity is more serious, and the matching results with the corpus are less, which will lead to poor subsequent extraction effects. Therefore, after obtaining the basic extraction template, each basic extraction template can be generalized by window sliding, slot replacement and other methods to further expand the number and versatility of extraction templates. Specifically, taking a basic extraction template as an example, window sliding can be to create a character window, slide it in the target paragraph corpus until it covers the text relationship character and the n characters adjacent to the text relationship character. Among them, n is a positive integer, and the specific value can be determined according to different knowledge extraction situations. It can be understood that adjacent can refer to the character in front of the text relationship character, or the character behind the text relationship character, or it can include the characters in front and behind, which is not limited here. The business server obtains the text relationship character and the characters adjacent to the text relationship character in the basic extraction template by sliding the character window. Among them, the adjacent characters are the characters covered by the character window.

[0124] The business server then generates S generalized extraction templates based on the text-relationship character and the characters adjacent to the text-relationship character. Each generalized extraction template includes the text-relationship character and at least one of the adjacent characters. Specifically, the business server obtains i characters adjacent to the text-relationship character from the characters covered by the character window, and generates S generalized extraction templates together with the text-relationship character. Wherein, i is the total number n of characters that are less than the non-text-relationship characters covered by the character window. Specifically, the business server starts from i=1, obtains i characters adjacent to the text-relationship character from the characters covered by the character window, generates a generalized extraction template together with the text-relationship character, then adds 1 to the value of i, and repeats the aforementioned steps of generating a generalized extraction template until the value of i is greater than n, at which point the repetitive process stops. For example, the business server obtains a character adjacent to the text-related character from the characters covered by the character window to generate a generalized extraction template, then obtains two corpus characters adjacent to the text-related character from the characters covered by the character window to generate a generalized extraction template, and so on. Finally, it obtains characters adjacent to the text-related character from the characters covered by the character window to generate a generalized extraction template, and a total of S generalized extraction templates are obtained, and the generation of generalized extraction templates is stopped.

[0125] Next, slot filling is required for each generalized extraction template. This involves performing word segmentation and part-of-speech analysis on the generalized extraction template, replacing some of the characteristic words in the matching results with corresponding concept words. For example, 'XX famous actor' can be replaced with '[D:location]_famous_[D:gender]_actor'. [D:location] represents the region slot, and [D:gender] represents the gender slot. This slot-filling template can match text corpora from other regions or genders, such as 'American famous actress'.

[0126] It can be understood that after matching the corpus set with the knowledge seed set, a basic extraction template with strong specificity can be obtained. After sliding the window of each basic extraction template, S generalized extraction templates can be obtained. After filling the slots of each generalized extraction template, the knowledge extraction template obtained can match a larger range of texts. Therefore, by adopting the method provided in the embodiment of the present application, multiple knowledge extraction templates with a wider matching range can be obtained in the end, and the coverage of knowledge extraction is increased.

[0127] S103: Extract candidate knowledge description information from the corpus set according to the knowledge extraction template.

[0128] Specifically, the corpus set can be the corpus set used to construct the knowledge extraction template mentioned above. After the knowledge extraction template is constructed, new knowledge description information can be extracted from the corpus set through the knowledge extraction template. Among them, the business server will directly perform string matching on the knowledge extraction template and the text data in the corpus set. If the match is successful, the data at the corresponding text relationship character position will be used as the entity of the extracted knowledge description information, and a new set of knowledge description information can be obtained according to the specified intention contained in the knowledge description information in the knowledge seed set. For example, the knowledge description information in the knowledge seed set is "Xiao Ming-Date of Birth-August 19, 1998", then the specified intention is to extract the date of birth of the character. A knowledge extraction template may extract multiple sets of knowledge description information from the corpus set. After matching each knowledge extraction template with the corpus set, the knowledge extraction of this round (as mentioned above) can be obtained. Figure 2bThe candidate knowledge description information extracted by the knowledge extraction of the xth iteration is shown in the figure. For example, the corpus is {Xiao Ming's wife is Xiao Lan, Xiao Yue's husband is Xiao Hua, and coincidentally, Xiao Lan's sister is Xiao Yue}. The knowledge extraction templates include "[D: person]'s wife is [D: person]_1,2" and "[D: person]'s husband is [D: person]_2,1". When both knowledge extraction templates are matched with the corpus, the candidate knowledge description information obtained by "[D: person]'s wife is [D: person]_1,2" is "Xiao Ming - wife - Xiao Lan", and the candidate knowledge description information obtained by "[D: person]'s husband is [D: person]_2,1" is "Xiao Hua - wife - Xiao Yue". Therefore, the candidate knowledge description information set obtained in this round of extraction is {Xiao Ming - wife - Xiao Lan, Xiao Hua - wife - Xiao Yue}. If different knowledge extraction templates extract the same candidate knowledge description information, the business server will only store the candidate knowledge description information once, but will record the knowledge extraction template that can extract the candidate knowledge description information.

[0129] It is understandable that due to the diversity of language and literature, the knowledge description information extracted based on the template is not necessarily correct. Therefore, the extracted knowledge description information is used as candidate knowledge description information and needs to be further processed to determine its reliability.

[0130] S104: Determine the template confidence of the knowledge extraction template, and determine the knowledge confidence of the candidate knowledge description information according to the template confidence.

[0131] Specifically, confidence refers to the degree of credibility, and the higher the confidence, the more credible it is. Since the knowledge description information is obtained by directly matching the knowledge extraction template with the corpus set, the credibility of the knowledge extraction template, that is, the confidence, is crucial to the accuracy of the final extraction result and the effect of subsequent iterations. Because the entire knowledge extraction includes multiple rounds of extraction, each knowledge extraction template can obtain a current confidence in each round of iteration. Among them, determining the current confidence includes obtaining the candidate knowledge description information extracted by the knowledge extraction template, judging the legitimacy of these candidate knowledge description information, and then calculating the current confidence of the knowledge extraction template based on the number of legal candidate knowledge description information and the number of illegal candidate knowledge description information. Then, the template confidence of the knowledge extraction template can be updated according to the current confidence.

[0132] For example, after updating the knowledge extraction template A (as mentioned above) Figure 2b The template confidence of the knowledge extraction template A1 obtained after the x+1th iteration is shown. First, the current confidence of the knowledge extraction template A is determined, and then the confidence of the knowledge extraction template after the previous iteration is obtained (as shown above). Figure 2b The template confidence of the knowledge extraction template A1) obtained after the xth round of iteration is used as the historical confidence of the knowledge extraction template A, and then the template confidence of the knowledge extraction template A after this round of iteration is obtained based on the current confidence and historical confidence of the knowledge extraction template A.

[0133] Specifically, when a certain knowledge extraction template is used to match a corpus set, among the extracted candidate knowledge description information, there may be candidate knowledge description information that describes the same entity as the knowledge description information in the knowledge seed set. The legitimacy of the candidate knowledge description information can be judged based on the knowledge description information in the knowledge seed set. In other words, for a candidate knowledge description information, if the knowledge description information that describes the same entity as it is not found in the knowledge seed set, the knowledge legitimacy of the candidate knowledge description information is determined to be unknown, and the candidate knowledge description information will not participate in the subsequent template confidence calculation process; if the knowledge description information that describes the same entity as it is found in the knowledge seed set, the knowledge description information in the knowledge seed set that describes the same entity as the candidate knowledge description information is used as the knowledge description information to be matched. If the candidate knowledge description information is the same as the knowledge description information to be matched, the knowledge legitimacy of the candidate knowledge description information is determined to be legal; if the candidate knowledge description information is different from the knowledge description information to be matched, the knowledge legitimacy of the candidate knowledge description information is determined to be illegal. For example, the knowledge seed set is {Xiao Ming - Birthday - September 2, Xiao Hua - Birthday - September 9}, and the extracted candidate knowledge description information M1 is Xiao Wang - Birthday - September 5. The entities described by the knowledge description information in the knowledge seed set are Xiao Ming and Xiao Hua, neither of which is the entity Xiao Wang described by the extracted candidate knowledge description information. Therefore, the knowledge legitimacy of the candidate knowledge description information is determined to be unknown; the extracted candidate knowledge description information M2 is Xiao Ming - Birthday - September 5, which describes Xiao Ming's birthday. The knowledge description information that also describes Xiao Ming's birthday can be found in the knowledge seed set. The knowledge description information Xiao Ming - Birthday - September 2 that also describes Xiao Ming's birthday in the knowledge seed set is obtained as the knowledge description information to be matched. By comparing the two, it can be found that the knowledge described by the two is different. The candidate knowledge description information M2 gives Xiao Ming's birthday as September 5, and the knowledge description information to be matched gives Xiao Ming's birthday as September 2, which means that the extracted candidate knowledge description information M2 is wrong, and its knowledge legitimacy is determined to be illegal. If the two are compared and it is found that the knowledge described by the two is the same, the knowledge legitimacy of the candidate knowledge description information is determined to be legal.

[0134] After determining the knowledge validity of all candidate knowledge description information extracted by the knowledge extraction template, the current confidence of the knowledge extraction template in this iteration can be calculated. The template confidence of the knowledge extraction module from the previous iteration is then obtained and used together with the current confidence to determine the template confidence of the knowledge extraction template in this iteration. After calculating the template confidence of all knowledge extraction templates in this round of knowledge extraction, the knowledge confidence of each candidate knowledge description information can be determined. Specifically, the same candidate knowledge description information may be obtained by matching multiple different knowledge extraction templates with the corpus. Therefore, to determine the knowledge confidence of a candidate knowledge description information, it is first necessary to obtain knowledge extraction templates that can extract the candidate knowledge description information from the corpus. The knowledge confidence of the candidate knowledge description information is then determined based on the template confidence of these knowledge extraction templates in this iteration. It is understood that the template confidence of the knowledge extraction template is updated with each iteration, and therefore the knowledge confidence of the candidate knowledge description information is also updated with each iteration.

[0135] S105: Update the knowledge seed set according to the knowledge confidence and the candidate knowledge description information.

[0136] Specifically, the credibility of candidate knowledge description information can be determined through knowledge confidence. Some candidate knowledge description information with high confidence can be selected from the candidate knowledge description information and added to the knowledge seed set. If the updated knowledge seed set does not meet the convergence conditions, the business server will initiate the next round of knowledge extraction, i.e., execute steps S101 to S105 again. The knowledge seed set used in step S101 of the next round of knowledge extraction is the updated knowledge seed set from the current round of knowledge extraction. In other words, if the convergence conditions are not met, the business server will loop through steps S101 to S105, continuously updating the knowledge seed set and adding new knowledge description information. When matched with the corpus, new knowledge extraction templates may be obtained, and then new candidate knowledge description information can be mined from the corpus. This iterative cycle will ultimately expand the knowledge seed set and the template set containing the knowledge extraction template, ultimately achieving the goal of mining all the knowledge description information and knowledge extraction templates that describe relevant intents contained in the corpus.

[0137] Optionally, a credibility confidence threshold can be set. When the knowledge confidence of the candidate knowledge description information exceeds the credibility confidence threshold, the target knowledge description information is treated as credible knowledge description information and added to the knowledge seed set. If the updated knowledge seed set does not converge, the updated knowledge seed set is used as the knowledge seed set for the next iteration.

[0138] S106: When the updated knowledge seed set meets the convergence condition, the knowledge graph is updated according to the updated knowledge seed set.

[0139] Specifically, the updated knowledge seed set meets the convergence condition when no new candidate knowledge description information is mined from the corpus after repeated iterations of S101 to S105. Naturally, no new knowledge description information will be added to the knowledge seed set. At this point, iteration can be stopped, and the candidate knowledge description information obtained by matching the knowledge extraction template in the knowledge seed set, including some reliable candidate knowledge description information, will be selected as the knowledge description information to be stored and added to the knowledge graph.

[0140] Specifically, to determine whether the updated knowledge seed set meets the convergence condition, the knowledge seed set and the updated knowledge seed set may be compared. If the two are the same, it can be considered that the updated knowledge seed set meets the condition.

[0141] Specifically, the present application also provides another convergence detection method: to determine whether the updated knowledge seed set meets the convergence conditions, M historical confidences corresponding to the knowledge extraction template can be obtained, where M is a positive integer; if the absolute value of the difference between the template confidence and each of the M historical confidences is less than the convergence threshold, it is determined that the updated knowledge seed set meets the convergence conditions.

[0142] Optionally, when the updated knowledge seed set meets the convergence criteria, the highly confident knowledge extraction template can be saved and added to the template library. The template library is used to store extraction templates that can directly match text data to obtain reliable knowledge description information. Each time knowledge extraction is performed, in addition to obtaining reliable knowledge description information, a reliable knowledge extraction template can also be obtained. This template is added to the module library and can be directly called upon to match the corpus when knowledge description information is needed in the future.

[0143] In order to more clearly illustrate the iterative process of the knowledge extraction method provided in the embodiment of the present application, please refer to Figure 3b , Figure 3b This is a flow chart of a knowledge extraction method provided by an embodiment of the present application. For ease of understanding, the extracted knowledge description information is described as SPO triples. Figure 3bAs shown in the figure, first, the existing, credible, and small amount of initial SPO knowledge is used as a seed set to mine the extraction template with the corpus, and then the template is simply matched with the corpus to obtain a new rough selection of SPO triples. The accuracy of the mining result can be used to calculate the confidence of the extracted template, and the confidence of the extracted template can be used to deduce the confidence of the extracted SPO triples. The high-confidence SPO knowledge continues to be added to the supplementary seed set, and the obtained SPO set is used as the SPO seed set for the next round of iteration. The specific implementation of the above process can be seen in Figure 3a The description of steps S01-S105 in the corresponding embodiment will not be repeated here.

[0144] like Figure 3b As shown, the next iteration can use the newly introduced SPO triples to mine new templates, thereby repeating the above process to further mine reliable SPO knowledge. This iterative cycle will eventually continuously expand the trusted SPO set and template set, and ultimately achieve the result of mining SPO knowledge and extracting templates for all relevant intents contained in the corpus. When the SPO seed set meets the convergence conditions, the top, highly confident SPO triples are selected from the SPO set and added to the knowledge graph. The top, highly confident extracted templates are selected from the template set and stored in the template library, which can be directly called when performing other knowledge extraction.

[0145] It can be understood that the method provided in the embodiment of the present application can be applied not only to SPO triple knowledge extraction, but also to other knowledge extraction scenarios, such as concept word extraction from knowledge graphs, hyponym extraction from knowledge graphs, event word extraction from knowledge graphs, etc.

[0146] The embodiment of the present application proposes to match the knowledge description information in the knowledge seed set with the text data in the corpus set to obtain a knowledge extraction template, and then use the knowledge extraction template to extract the knowledge description information in the corpus set, calculate the template confidence of the knowledge extraction template based on the knowledge legitimacy of the extracted knowledge description information, and then calculate the confidence of the knowledge description information based on the template confidence, and add the knowledge description information with high confidence to the knowledge seed set, so as to obtain a new seed set to extract the knowledge description information of the corpus. The above process is continuously and automatically iterated, and finally the knowledge seed set and the template set containing the knowledge extraction template are continuously expanded to achieve the result of mining the knowledge description information and extraction template of all relevant intentions contained in the corpus. The embodiment of the present application is initially based on credible, small amounts of knowledge description information and automatically mined knowledge extraction templates, and can iteratively mine more new knowledge description information from the corpus set, that is, it can efficiently extract more and more comprehensive knowledge description information from the massive text data on the Internet, and continuously provide source data with wide coverage and high accuracy for the knowledge graph.

[0147] For further information, see Figure 4 , Figure 4 This is a schematic diagram of the process of generating a knowledge extraction template provided in an embodiment of the present application. Figure 4 Specifically, the above Figure 3a Detailed description of S102 in the corresponding embodiment. The business server will first traverse the text data in the corpus set, and take the paragraphs in the corpus set containing the knowledge description information as the target paragraph corpus; then generate a basic extraction template based on the target paragraph corpus and the knowledge description information; then generate a generalized extraction template based on each basic extraction template; finally, fill the slots of each generalized extraction template to obtain a knowledge extraction template. For ease of understanding, the knowledge description information in the knowledge seed set in the embodiment of the present application is an SPO triple that describes the knowledge of a person's date of birth. Figure 4 As shown, the knowledge seed set contains multiple sets of reliable knowledge description information, such as Xiao Zhang-Date of Birth-September 27, 1961, Xiao Cheng-Date of Birth-April 7, 1954, etc., and the corpus set contains multiple unstructured text paragraphs. First, the business server will traverse the text data in the corpus set, and use the paragraphs in the corpus set that contain the knowledge description information as the target paragraph corpus. For example, the SPO triples of "Xiao Zhang-Date of Birth-September 27, 1961" are matched with the corpus set to find the target paragraph corpus. For this SPO triple, the subject is Xiao Zhang and the object is September 27, 1961. Therefore, when querying, the text paragraph containing Xiao Zhang and September 27, 1961 will be used as the target paragraph corpus. In the example Figure 4 In the corpus shown, "Xiao Zhang, born on September 27, 1961 in XX, native place: Xinhui, Guangdong" contains the entity data for the SPO triple, so this is selected as a target paragraph corpus. Of course, there is no limit to the number of target paragraph corpora; any paragraph containing this entity data can be used as a target paragraph corpus. Once a target paragraph corpus is obtained, matching the SPO triple against the corpus continues until the entire corpus is traversed.

[0148] like Figure 4As shown in the figure, after obtaining the target paragraph corpus, the business server will obtain the text relationship characters corresponding to the knowledge description information; replace the knowledge description information with the text relationship characters in the target paragraph corpus to obtain the basic extraction template. The text relationship characters here refer to the knowledge attributes in the knowledge description information corresponding to the entity data. For SPO triples, the text relationship characters are Subject and Object. Replace Xiao Zhang with Subject and September 27, 1961 with Object to generate a basic extraction template "Subject_Object was born in XX and is from Xinhui, Guangdong." Through knowledge matching and text relationship character replacement, multiple basic extraction templates can be generated.

[0149] Then, a window is slid for each basic extraction template to obtain S generalized extraction templates. To more clearly explain the process of generalizing the basic extraction template, please refer to Figure 5 , Figure 5 This is a schematic diagram of a character window covering a basic extraction template provided by an embodiment of the present application. First, it is necessary to obtain the characters adjacent to the text relationship characters through the character window, such as Figure 5 As shown, taking the SPO triple as an example, there are two text-related characters, S character 401 and P character 402, so two corresponding character windows are required, such as character window 41 and character window 42. It can be understood that the two character windows are slid on the target paragraph corpus respectively until the n characters adjacent to the text-related characters are covered. The character window here needs to cover the n characters adjacent to the text-related characters, which means that it needs to cover the n characters before and the n characters after each text-related character.

[0150] like Figure 5As shown, after character window 41 covers one character adjacent to character S 401 and character window 42 covers one character adjacent to character P 402, the character window actually covers a total of four common characters X, excluding the text-related characters, as the characters to be selected. Character X can be any character. Then, i characters are selected from these characters to be selected, together with the text-related characters, to form a generalized extraction template. Here, i can be 1, 2, 3, or 4, respectively. For example, when i is 1, the generalized extraction templates obtained are: XS, P; SX, P; S, XP; S, PX. "XS, P" is a set of knowledge extraction templates, indicating that there are unknown characters between the matched characters and the template is not a continuous string. For example, using "XS, P" to match XSAAP will successfully match. When i is 2, the generalized extraction templates obtained are: XSX, P; XS, XP; XS, PX; SXXP; SX, PX; S, XPX; S, PXX, and so on. It's understandable that in the generated generalized extraction template, after removing the text-related characters, one end of the resulting string is always adjacent to the text-related characters. For example, in XXSXPXXX, removing S and P yields XX, X, and XXX. It can be seen that in all three strings, there's an X character adjacent to either S or P.

[0151] Assume that S generalized extraction templates include generalized extraction template L i , i is a positive integer less than or equal to S. i Fill the slots, i.e., extract the generalized template L i Perform word segmentation to obtain text relationship characters and associated word segmentation, and determine the knowledge description information and associated word segmentation associated with the text relationship characters as the word segmentation to be replaced; record the text relationship characters in the generalized extraction template L according to the word segmentation to be replaced i The sorting position in is used as the extraction position index; a slot word table is obtained, which includes slot word segmentation, text word segmentation, and a mapping relationship between slot word segmentation and text word segmentation; if a text word segmentation identical to the word to be replaced is found in the slot word table, the slot word segmentation having a mapping relationship with the text word segmentation identical to the word to be replaced is used as the target slot word segmentation; the generalized extraction template is segmented and replaced according to the target slot word segmentation to obtain a slot replacement template; the generalized extraction template L is generated according to the slot replacement template and the extraction position index. i The corresponding knowledge extraction template. Figure 4As shown in the figure, when generalizing the basic generation template "Subject_Object was born in XX, and his hometown is Xinhui, Guangdong", when obtaining the n characters adjacent to the text relationship characters, since there are only characters after Object, when sliding the window, what is actually obtained are the n characters after Object. If the value of n is 5, then 5 generalized extraction templates can be obtained by generalization. After generalizing each basic extraction template, multiple generalized extraction templates are obtained, and then the slots of each generalized extraction template are filled to obtain the knowledge extraction template. Figure 4 The generalized extraction template "Subject_Object was born in XX" is first segmented, resulting in "Subject," "Objec," "born in," and "XX." "Born in" and "XX" are associated segmentations. The business server then obtains the position of the text-related characters within these phrases, with "Subject" in the first position and "Objec" in the second position, recorded as extraction position index_1,2. For these text-related characters, the entity data that replaced them during the basic template generation process must be obtained, namely "Xiao Zhang" and "September 27, 1961." Therefore, the actual phrases requiring part-of-speech analysis and slot replacement include "Xiao Zhang," "September 27, 1961," "born in," and "XX." The business server retrieves the slot word list and then searches the list to see if any segmentation in the phrase has a corresponding slot word. If so, it replaces the segmentation with the slot word; if not, the original segmentation remains unchanged. Slot word lists can be pre-defined slot concepts within the word segmentation tool or generated using custom slots and corresponding text collections. After replacing the slots of the generalized extraction template, and then indexing the extraction position, we can get a knowledge extraction template "[D:person]_[D:date] born in [D:location]_1,2".

[0152] Optionally, after the business server generalizes and expands the basic extraction template, although a large number of generalized extraction templates can be obtained, the quality of these generalized extraction templates varies, and the credibility of some generalized extraction templates is very low. At this time, the posterior rules can be manually set to perform initial filtering on the generalized extraction templates. The filtered generalized extraction templates will no longer participate in the subsequent process. Among them, the posterior rules can be some format verifications and some rules that are easy to verify the low credibility of the template. For example, when the number of unknown characters between the text relationship characters in the obtained generalized extraction template is too large and exceeds the normal situation, such generalized extraction templates can be removed.

[0153] For further information, see Figure 6 , Figure 6 This is a schematic diagram of a process for extracting knowledge description information provided by an embodiment of the present application. Figure 6Specifically, the above Figure 3a For ease of understanding, the embodiment of this application is described in detail in S103 of the corresponding embodiment. Figure 4 The example of the knowledge extraction templates obtained in the embodiment shown is used as an example to extract SPO triples by matching with the corpus set. Figure 4 The corpus collection shown in . Figure 6 As shown, the knowledge extraction templates in the knowledge extraction template set are templates obtained by filling in the slots. In order to more easily match multiple sets of templates with the corpus set, the text data in the corpus set can be filled in the corpus slots at the beginning, using the same method as above. Figure 4 Just like filling slots for generalized extraction templates, the text data is first segmented and analyzed for parts of speech, and then the slot segmentations in the slot vocabulary are obtained to replace the replaceable segmentations. After that, the knowledge extraction templates in the knowledge extraction template set can be used to match the filled sentence set in turn. Among them, the matching can be a direct string matching of the template and the corpus. For text sentences that are successfully matched, the original segmentation data of the corresponding slot will be obtained according to the extraction position index of the knowledge extraction template. Figure 6 As shown in the figure, "[D:person]_[D:date]born[D:location]_1,2" is successfully matched with a text sentence in the corpus collection. According to its extraction position index "_1,2", the business server will obtain the original participle "Xiao Li" corresponding to the first slot participle as the Subject of the extracted knowledge description information, and obtain the original participle "1940" corresponding to the second slot participle as the Objec of the extracted knowledge description information. Therefore, the obtained set of knowledge description information is "Xiao Li-Date of Birth-1940".

[0154] The reliability of the extracted knowledge description information is unknown. Therefore, the extracted knowledge description information is considered as candidate knowledge description information, and its confidence is scored in turn. Finally, the highly reliable candidate knowledge description information is selected based on the confidence score and added to the knowledge graph. Since the candidate knowledge description information is directly matched with the text data in the corpus through the knowledge extraction template, the reliability of the candidate knowledge description information is closely related to the reliability of the knowledge extraction template. Therefore, the confidence of the knowledge extraction template can be calculated first.

[0155] For further information, see Figure 7 , Figure 7 This is a flow chart of calculating the confidence level of a template provided in an embodiment of the present application. Figure 7 Specifically, the above Figure 3aThe calculation of the template confidence of a knowledge extraction template in S104 in the corresponding embodiment is described in detail. Take the knowledge extraction template 70 included in the knowledge extraction template as an example. Figure 7 As shown, the process may include:

[0156] S701: Perform fuzzy matching on the candidate knowledge description information set and the knowledge seed set.

[0157] Specifically, a candidate knowledge description information set 71 consisting of candidate knowledge description information extracted by the knowledge extraction template 70 is obtained, and then fuzzy matching is performed with the knowledge seed set 72 to search for knowledge description information that describes the same entity as the candidate knowledge description information in the knowledge seed set 72. Figure 7 As shown, the candidate knowledge description information set 71 includes candidate knowledge description information 711, candidate knowledge description information 712, candidate knowledge description information 713, and so on; the knowledge seed set 72 includes knowledge description information 721, knowledge description information 722, and knowledge description information 723. Fuzzy matching is performed on the two, that is, for each candidate knowledge description information in the candidate knowledge description information set 71, a fuzzy match is performed in the knowledge seed set 72 to search for the knowledge description information that describes the same entity as it, as the knowledge description information to be matched. For example, candidate knowledge description information 711 and knowledge description information 721 both describe Xiao Ming's date of birth, so knowledge description information 721 is used as the knowledge description information to be matched with candidate knowledge description information 711.

[0158] S702: Verify the knowledge legitimacy of the candidate knowledge description information.

[0159] Specifically, the candidate knowledge description information is compared with the corresponding knowledge description information to be matched. If they are completely consistent, it means that the knowledge legitimacy of the candidate knowledge description information is legal; if they are inconsistent, it means that the knowledge legitimacy of the candidate knowledge description information is illegal; if the candidate knowledge description information has no corresponding knowledge description information to be matched, its knowledge legitimacy is determined to be unknown. Figure 7As shown, by comparing the knowledge description information 721 and the candidate knowledge description information 711, it can be seen that the knowledge described by the two is completely consistent, so the knowledge legitimacy of the candidate knowledge description information 711 is determined to be legal. Similarly, it can be seen that the knowledge description information 723 is the knowledge description information to be matched with the candidate knowledge description information 712. By comparing the two, the birth date of Xiao Zhang in the knowledge description information 723 is September 21, 1961, and the birth date of Xiao Zhang in the candidate knowledge description information 712 is May 1, 1970. There is inconsistency in the described knowledge, so the knowledge legitimacy of the candidate knowledge description information 712 is determined to be illegal. Since no knowledge description information describing the same entity as the candidate knowledge description information 713 is found in the knowledge seed set 72, the knowledge legitimacy of the candidate knowledge description information 713 is determined to be unknown. The determination of the knowledge legitimacy of the remaining candidate knowledge description information is the same as one of the above three, and will not be repeated here.

[0160] S703-S704: Count the results of the legitimacy verification, calculate the confidence of the extraction in this round based on the results, record it as the current confidence, and finally obtain the template confidence of the previous round of the knowledge extraction template as the historical confidence. Based on the current confidence and historical confidence, obtain the template confidence after this round of iteration.

[0161] Specifically, after determining the legitimacy of all candidate knowledge description information, the number of candidate knowledge description information with legal knowledge legitimacy is used as the legal parameter, denoted as p, and the number of candidate knowledge description information with illegal knowledge legitimacy is used as the illegal parameter, denoted as n. Then, the current confidence of the template is calculated based on p and n. The calculation method of the current confidence can be shown as formula (1):

[0162]

[0163] The purpose of calculating the current confidence of the template by comparing the candidate knowledge description information extracted from the template with the knowledge seed set as described above is to automatically determine whether it is correct, thereby avoiding the additional work introduced by manual labeling accuracy affecting the iteration efficiency.

[0164] The knowledge extraction mentioned in the embodiments of the present application is an iterative process. After each round of knowledge description information extraction, a current confidence level can be calculated. In order to integrate the confidence levels of templates from multiple rounds of iterations, a template confidence level is set for each knowledge extraction template. After each round of knowledge description information extraction, the template confidence level is updated based on the current confidence level and the template confidence level of the knowledge extraction template calculated in the previous round. The template confidence level update formula is as follows:

[0165] Conf(P) u =α*Conf(P)o +(1-α)*Conf(P) n Formula (2)

[0166] Among them, Conf(P) o is the relevant historical confidence, that is, the result of updating the template confidence of the knowledge extraction template in the previous round, Conf(P) n is the current confidence, α is the learning rate, Conf(P) u is the template confidence, which is the result of updating the template confidence of the knowledge extraction template in this round.

[0167] The business server records the template confidence of each knowledge extraction template in each iteration. During this iteration, the previously recorded template confidences are collectively referred to as the historical confidence of the knowledge extraction template. The business server obtains the generation timestamp of each historical confidence from the historical confidences and uses the historical confidence corresponding to the largest generation timestamp as the relevant historical confidence.

[0168] like Figure 7 As shown, after calculating the current confidence of the knowledge extraction template 70 in this round of knowledge extraction, the business server will obtain the template confidence obtained by the previous round of knowledge extraction of the knowledge extraction template 70 as the historical confidence, and then obtain the template confidence of the template in this round of knowledge extraction according to formula (2).

[0169] Optionally, although the same set of candidate knowledge description information can be obtained by different knowledge extraction templates, the verification results of its knowledge legitimacy are unique. Therefore, only one knowledge legitimacy verification is required for the set of candidate knowledge description information. In other words, when obtaining candidate knowledge description information extracted by a certain knowledge extraction template and performing knowledge legitimacy verification on it, if the knowledge legitimacy of the candidate knowledge description information has been verified when calculating the template confidence of another knowledge extraction template, the result can be obtained directly without the need for fuzzy matching.

[0170] Optionally, if the number of knowledge extraction templates obtained by the embodiment of the present application is too large, in order to reduce the amount of calculation, a template filtering step can be added to the calculation process to remove some unreasonable or extremely low-credibility knowledge extraction templates. For example, after extracting candidate knowledge description information using the knowledge extraction template, the knowledge extraction templates with extremely low extraction numbers can be filtered out and no longer participate in the template confidence calculation of this round of knowledge extraction. For example, after calculating the current confidence of the knowledge extraction template, the knowledge extraction template with extremely low current confidence is filtered out and no longer participates in the template confidence calculation of this round of knowledge extraction.

[0171] For further information, see Figure 8 , Figure 8 This is a schematic diagram of a knowledge confidence calculation process provided in an embodiment of the present application. Figure 8 Specifically, the above Figure 3a Detailed description of S104 in the corresponding embodiment. Because the candidate knowledge description information is extracted through the above-mentioned knowledge extraction template, the knowledge confidence of the candidate knowledge description information can be calculated based on the template credibility of the knowledge extraction template. The candidate knowledge description information may include knowledge description information K. To calculate the knowledge confidence of the knowledge description information K, the knowledge extraction template that can extract the knowledge description information K may be used as the target knowledge extraction template; the template confidence of the target knowledge extraction template may be used as the target template confidence; and the knowledge confidence of the target knowledge description information K may be determined based on the target confidence. Figure 8 As shown, the knowledge confidence of the knowledge description information K is calculated, and i extraction templates are obtained. Where i is a positive integer greater than or equal to 1. Then, the template confidence corresponding to these i extraction templates is obtained, and then the knowledge confidence of the knowledge description information K is calculated. The calculation formula of knowledge confidence can be shown as formula (3):

[0172] Conf(T)=1-∏(1-Conf(P i ) Formula (3)

[0173] Among them, P i The template confidence of the i-th extracted template that extracts the knowledge description information K. Since the optimized template confidence is updated in each iteration, each iteration triggers the update of the knowledge confidence.

[0174] Further, see Figure 9 , Figure 9 This is a schematic diagram of the structure of a data processing device based on a knowledge graph provided in an embodiment of the present application. The above-mentioned data processing device can be a computer program (including program code) running on a computer device, for example, the data processing device is an application software; the device can be used to execute the corresponding steps of the method provided in the embodiment of the present application. Figure 9 As shown, the data processing device 2 may include: an acquisition module 21, a template generation module 22, a knowledge extraction module 23, a first determination module 24, a second determination module 25, a first update module 26, and a second update module 27.

[0175] An acquisition module 21 is used to acquire a knowledge seed set, where the knowledge seed set includes textual knowledge description information;

[0176] The template generation module 22 is used to match the knowledge description information with the text data in the corpus collection to obtain a knowledge extraction template;

[0177] The knowledge extraction module 23 is used to extract candidate knowledge description information from the corpus collection according to the knowledge extraction template;

[0178] A first determination module 24 is used to determine the template confidence of the knowledge extraction template;

[0179] A second determination module 25 is configured to determine the knowledge confidence of the candidate knowledge description information according to the template confidence;

[0180] A first updating module 26 is configured to update the knowledge seed set according to the knowledge confidence and the candidate knowledge description information;

[0181] The second updating module is used to update the knowledge graph according to the updated knowledge seed set.

[0182] The specific implementation of the acquisition module 21, the template generation module 22, the knowledge extraction module 23, the first determination module 24, the second determination module 25, the first update module 26 and the second update module 27 can be found in the above Figure 3a The description of steps S101-S106 in the corresponding embodiment will not be repeated here.

[0183] See Figure 9 The template generation module 22 may include: a query unit 221 , a basic generation unit 222 , a generalization generation unit 223 and a slot filling unit 224 .

[0184] A query unit 221 is used to traverse the text data in the corpus set and select paragraphs containing knowledge description information in the corpus set as target paragraph corpus;

[0185] A basic generation unit 222 is used to generate a basic extraction template based on the target paragraph corpus and knowledge description information;

[0186] The generalization generation unit 223 is used to generate S generalization extraction templates according to the basic extraction template; S is a positive integer;

[0187] The slot filling unit 224 is used to fill the slots of the S generalized extraction templates respectively to obtain a knowledge extraction template.

[0188] The specific implementation of the query unit 221, the basic generation unit 222, the generalization generation unit 223 and the slot filling unit 224 can be found in the above Figure 3a The description of step S102 in the corresponding embodiment will not be repeated here.

[0189] See Figure 9 The basic generation unit 222 may include: a character acquisition subunit 2221 and a character replacement subunit 2222.

[0190] The character acquisition subunit 2221 is used to acquire textual relationship characters corresponding to the knowledge description information;

[0191] The character replacement subunit 2222 is used to replace the knowledge description information with text relationship characters in the target paragraph corpus to obtain a basic extraction template.

[0192] The specific implementation of the character acquisition subunit 2221 and the character replacement subunit 2222 can be found in the above Figure 3a The description of step S102 in the corresponding embodiment will not be repeated here.

[0193] See Figure 9 The generalization generation unit 223 may include: a window acquisition subunit 2231 and a generalization template generation subunit 2232.

[0194] The window acquisition subunit 2231 is used to acquire text-related characters and characters adjacent to the text-related characters in the basic extraction template by sliding the character window; the adjacent characters are the characters covered by the character window;

[0195] The generalized template generating subunit 2232 is configured to generate S generalized extraction templates according to the text-related characters and the adjacent characters; each generalized extraction template includes at least one of the text-related characters and the adjacent characters.

[0196] The specific implementation of the window acquisition subunit 2231 and the generalized template generation subunit 2232 can be found in the above Figure 3a The description of step S102 in the corresponding embodiment will not be repeated here.

[0197] Among them, S generalized extraction templates include generalized extraction template L i , i is a positive integer less than or equal to S;

[0198] See Figure 9 The slot filling unit 224 may include: a template word segmentation subunit 2241, a position determination subunit 2242, a vocabulary acquisition subunit 2243, a word segmentation search subunit 2244, a word segmentation replacement subunit 2245 and an extraction template generation subunit 2246.

[0199] Template segmentation subunit 2241 is used to extract the generalized template L i Perform word segmentation to obtain text relationship characters and associated word segmentations, and determine the knowledge description information and associated word segmentations associated with the text relationship characters as the word segmentations to be replaced;

[0200] Position determination subunit 2242, based on the relationship character of the word segmentation record to be replaced in the generalized extraction template Li The sorting position in is used as the extraction position index;

[0201] The vocabulary acquisition subunit 2243 is used to acquire a slot vocabulary, which includes slot segmentation words, text segmentation words, and a mapping relationship between slot segmentation words and text segmentation words;

[0202] The word search subunit 2244 is configured to, if a text word identical to the word to be replaced is found in the slot word table, use the slot word that has a mapping relationship with the text word identical to the word to be replaced as the target slot word;

[0203] The word segmentation replacement subunit 2245 is used to replace the generalized extraction template with a word segmentation according to the target slot segmentation to obtain a slot replacement template;

[0204] The extraction template generation subunit 2246 is used to generate a knowledge extraction template corresponding to the generalized extraction template Li according to the slot replacement template and the extraction position index.

[0205] The specific implementation of the template word segmentation subunit 2241, the position determination subunit 2242, the word list acquisition subunit 2243, the word segmentation search subunit 2244, the word segmentation replacement subunit 2245 and the extraction template generation subunit 2246 can be found in the above Figure 4 The detailed description of step S102 in the corresponding embodiment will not be repeated here.

[0206] See Figure 9 The knowledge extraction template 23 may include: a text filling unit 231 , a text matching unit 232 and a knowledge acquisition unit 233 .

[0207] The text filling unit 231 is used to fill the slots of the text data in the corpus collection to obtain replacement text data;

[0208] a text matching unit 232 for matching the knowledge extraction template with the replacement text data;

[0209] The knowledge acquisition unit 233 is configured to acquire an extraction position index if the knowledge extraction template successfully matches the replacement text data, and acquire candidate knowledge description information in the replacement text data according to the extraction position index.

[0210] The specific implementation of the text filling unit 231, the text matching unit 232 and the knowledge acquisition unit 233 can be found in the above Figure 3a The description of step S103 in the corresponding embodiment will not be repeated here.

[0211] See Figure 9The first determination module 24 may include: a legitimacy determination unit 241 , a current confidence calculation unit 242 , a confidence acquisition unit 243 , and a template confidence calculation unit 244 .

[0212] a legitimacy determination unit 241 for verifying the knowledge legitimacy of the candidate knowledge description information based on the candidate knowledge description information and the knowledge seed set;

[0213] A current confidence calculation unit 242 is used to determine the current confidence of the knowledge extraction template according to the knowledge legitimacy;

[0214] The confidence acquisition unit 243 is configured to acquire the generation timestamp of each historical confidence from the historical confidences, and use the historical confidence corresponding to the largest generation timestamp as the relevant historical confidence;

[0215] The template confidence calculation unit 244 is used to determine the template confidence of the knowledge extraction template according to the relevant historical confidence and the current confidence.

[0216] The specific implementation of the legitimacy determination unit 241, the current confidence calculation unit 242, the confidence acquisition unit 243 and the template confidence calculation unit 244 can be found in the above Figure 3a The description of step S104 in the corresponding embodiment will not be repeated here.

[0217] See Figure 9 The legality determination unit 241 may include: a to-be-matched information search subunit 2411 and a legality judgment subunit 2412 .

[0218] The to-be-matched information search subunit 2411 is configured to search the knowledge seed set for knowledge description information that describes the same entity as the candidate knowledge description information, as the to-be-matched knowledge description information;

[0219] The legitimacy judgment subunit 2412 is used to determine the knowledge legitimacy of the candidate knowledge description information based on the candidate knowledge description information and the knowledge description information to be matched.

[0220] The specific implementation of the to-be-matched information search subunit 2411 and the legitimacy judgment subunit 2412 can be found in the above Figure 7 The description of steps S701-S702 in the corresponding embodiment will not be repeated here.

[0221] Among them, the knowledge extraction template includes the knowledge extraction template H g , g is a positive integer less than or equal to the total number of knowledge extraction templates;

[0222] See Figure 9The current confidence calculation unit 242 may include: a statistics subunit 2421 and a current confidence determination subunit 2422.

[0223] Statistics subunit 2421 is used to extract template H through knowledge g Among the extracted candidate knowledge description information, the number of candidate knowledge description information whose knowledge legitimacy is legal is counted as a legal parameter;

[0224] The statistics subunit 2421 is also used to extract the template H through knowledge g Among the extracted candidate knowledge description information, the number of knowledge description information whose knowledge legitimacy is illegal is counted as the illegal parameter;

[0225] The current confidence determination subunit 2422 is used to determine the current confidence of the knowledge extraction template Hg based on the legal parameters and the illegal parameters.

[0226] The specific implementation of the statistical subunit 2421 and the current confidence determination subunit 2422 can be found in the above Figure 7 The description of steps S703-S704 in the corresponding embodiment will not be repeated here.

[0227] The candidate knowledge description information includes the candidate knowledge description information K j , j is a positive integer;

[0228] See Figure 9 The second determination template 25 may include: a target template determination unit 251 , a target confidence determination unit 252 , and a knowledge confidence calculation unit 253 .

[0229] The target template determination unit 251 is used to extract the knowledge description information K j The knowledge extraction template of is used as the target knowledge extraction template;

[0230] a target confidence determination unit 252, configured to use the template confidence of the target knowledge extraction template as the target template confidence;

[0231] The knowledge confidence calculation unit 253 is used to determine the target knowledge description information K according to the target confidence. j knowledge confidence.

[0232] The specific implementation of the target template determination unit 251, the target confidence determination unit 252 and the knowledge confidence calculation unit 253 can be found in the above Figure 7 The description of steps S703-S704 in the corresponding embodiment will not be repeated here.

[0233] See Figure 9The first updating module 26 includes: a trusted knowledge determining unit 261 and a seed set updating unit 262.

[0234] a trusted knowledge determination unit 261, configured to use the target knowledge description information as trusted knowledge description information when the knowledge confidence of the target knowledge description information is greater than a trusted confidence threshold;

[0235] The seed set updating unit 262 is used to add the trusted knowledge description information to the knowledge seed set.

[0236] The specific implementation of the trusted knowledge determination unit 261 and the seed set update unit 262 can be found in the above Figure 3a The description of step S105 in the corresponding embodiment will not be repeated here.

[0237] See Figure 9 The second updating module 29 may include: a knowledge sorting unit 291 and a knowledge adding unit 292 .

[0238] The knowledge sorting unit 291 is used to sort the candidate knowledge description information in the updated knowledge seed set from high to low according to the knowledge confidence, and use the first L knowledge description information as the knowledge description information to be stored; L is a positive integer;

[0239] The knowledge adding unit 292 is used to add the knowledge description information to be stored into the knowledge graph.

[0240] The specific implementation of the knowledge sorting unit 291 and the knowledge adding unit 292 can be found in the above Figure 3a The description of step S106 in the corresponding embodiment will not be repeated here.

[0241] See Figure 9 , the data processing device 2 may further include: a judgment module 27.

[0242] The judgment module 27 is used to determine whether the updated knowledge seed set meets the convergence condition.

[0243] The specific implementation of the judgment module 27 can be found in the above Figure 3a The description of step S106 in the corresponding embodiment will not be repeated here.

[0244] See Figure 9 The judgment module 27 may include: a first judgment unit 271 and a second judgment unit 272.

[0245] A first judgment unit 271 is configured to determine that the updated knowledge seed set meets a convergence condition if the knowledge seed set and the updated knowledge seed set are the same;

[0246] The second judgment unit 272 is configured to determine whether the updated knowledge seed set meets the convergence condition based on the historical confidence of the knowledge extraction template and the template confidence.

[0247] The specific implementation of the first judgment unit 271 and the second judgment unit 272 can be found in the above Figure 3a The description of step S106 in the corresponding embodiment will not be repeated here.

[0248] See Figure 9 The second judgment unit 272 may include a historical confidence acquisition subunit 2721 and a calculation judgment subunit 2722.

[0249] The historical confidence acquisition subunit 2721 is used to obtain M historical confidences corresponding to the knowledge extraction template, where M is a positive integer;

[0250] The calculation and judgment subunit 2722 is used to determine that the updated knowledge seed set meets the convergence condition if the absolute value of the difference between the template confidence and each of the M historical confidences is less than the convergence threshold.

[0251] The specific implementation of the historical confidence acquisition subunit 2721 and the calculation and judgment subunit 2722 can be found in the above Figure 3a The description of step S106 in the corresponding embodiment will not be repeated here.

[0252] The acquisition module is specifically used to obtain the knowledge seed set in the xth iteration;

[0253] See Figure 9 , the data processing device 2 may further include: an iteration module 28.

[0254] Iterative module 28 is used to use the updated knowledge seed set as the knowledge seed set in the x+1th iteration when the updated knowledge seed set does not meet the convergence condition, continue to obtain the knowledge extraction template and candidate knowledge description information in the x+1th iteration based on the knowledge seed set in the x+1th iteration, and iteratively update the knowledge seed set in the x+1th iteration based on the knowledge extraction template and candidate knowledge description information in the x+1th iteration.

[0255] The specific implementation of the iteration module 28 can be found in the above Figure 2b The description of the x+1th round of iterative extraction in the corresponding embodiment will not be repeated here.

[0256] In an embodiment of the present application, a knowledge seed set containing a small amount of credible knowledge description information is constructed, and the knowledge seed set is matched with the corpus set to generate a knowledge extraction template. New knowledge extraction information is then extracted from the corpus set using the knowledge extraction template as candidate knowledge description information. The confidence of the knowledge extraction template and the candidate knowledge description information is determined, and highly credible candidate knowledge description information is selected to be added to the knowledge seed set. The above process is repeated continuously and automatically to expand the knowledge seed set and the knowledge extraction template set. Iteration stops until the knowledge seed set meets the convergence condition. At this point, most of the knowledge description information for the specified intent in the available corpus can be obtained, so the coverage rate is very high.

[0257] Further, see Figure 10 , Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 10 As shown above Figure 9 The device 2 in the corresponding embodiment can be applied to the above-mentioned computer device 8000, which may include: a processor 8001, a network interface 8004 and a memory 8005. In addition, the above-mentioned computer device 8000 also includes: a user interface 8003, and at least one communication bus 8002. Among them, the communication bus 8002 is used to realize the connection and communication between these components. The network interface 8004 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 8005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 8005 may optionally also be at least one storage device located away from the aforementioned processor 8001. As Figure 10 As shown, the memory 8005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application.

[0258] exist Figure 10 In the computer device 8000 shown, the network interface 8004 can provide network communication functions; the user interface 8003 is mainly used to provide an interface for user input; and the processor 8001 can be used to call the device control application stored in the memory 8005 to achieve:

[0259] Acquire a knowledge seed set, where the knowledge seed set includes knowledge description information of the text;

[0260] Match the knowledge description information with the text data in the corpus collection to obtain a knowledge extraction template;

[0261] Extract candidate knowledge description information from the corpus collection according to the knowledge extraction template;

[0262] Determine the template confidence of the knowledge extraction template, and determine the knowledge confidence of the candidate knowledge description information according to the template confidence;

[0263] Update the knowledge seed set according to the knowledge confidence and candidate knowledge description information;

[0264] When the updated knowledge seed set meets the convergence condition, the knowledge graph is updated according to the updated knowledge seed set.

[0265] It should be understood that the computer device 8000 described in the embodiment of the present application can execute the above Figure 3a The description of the data processing method in the corresponding embodiment can also be performed as described above. Figure 9 The description of the data processing device 2 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here either.

[0266] In addition, it should be noted that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the computer device 8000 for data processing mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, the computer program can execute the above-mentioned data processing. Figure 3a The description of the above-mentioned data processing method in the corresponding embodiment will therefore not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0267] The computer-readable storage medium may be the data processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Furthermore, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0268] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method based on knowledge graph, characterized in that: include: Acquire a knowledge seed set, wherein the knowledge seed set includes textual knowledge description information; Traversing the text data in the corpus set, and taking the paragraphs in the corpus set containing the knowledge description information as the target paragraph corpus; Generate a basic extraction template according to the target paragraph corpus and the knowledge description information; Generate S generalized extraction templates according to the basic extraction template; S is a positive integer; Filling the slots of the S generalized extraction templates respectively to obtain a knowledge extraction template; Extracting candidate knowledge description information from the corpus set according to the knowledge extraction template; Determining a template confidence of the knowledge extraction template, and determining a knowledge confidence of the candidate knowledge description information according to the template confidence; Updating the knowledge seed set according to the knowledge confidence and the candidate knowledge description information; When the updated knowledge seed set meets the convergence condition, the knowledge graph is updated according to the updated knowledge seed set.

2. The method according to claim 1, characterized in that The generating of a basic extraction template according to the target paragraph corpus and the knowledge description information includes: Obtaining text relationship characters corresponding to the knowledge description information; The knowledge description information is replaced with the text relationship characters in the target paragraph corpus to obtain a basic extraction template.

3. The method according to claim 2, characterized in that Generating S generalized extraction templates according to the basic extraction template includes: Obtaining the text-related characters and characters adjacent to the text-related characters in the basic extraction template by sliding a character window; the adjacent characters are characters covered by the character window; S generalized extraction templates are generated according to the text-related characters and the adjacent characters; each generalized extraction template includes the text-related characters and at least one of the adjacent characters.

4. The method according to claim 3, characterized in that S generalized extraction templates include generalized extraction template L i , i is a positive integer less than or equal to S; the S generalized extraction templates are respectively slot-filled to obtain a knowledge extraction template, including: The generalized extraction template L i Perform word segmentation to obtain the text-related characters and associated word segmentations, and determine the knowledge description information associated with the text-related characters and the associated word segmentations as the word segmentations to be replaced; According to the word segmentation record to be replaced, the text relation character is extracted in the generalized extraction template L i The sorting position in is used as the extraction position index; Obtaining a slot word table, wherein the slot word table includes slot word segmentation, text word segmentation, and a mapping relationship between the slot word segmentation and the text word segmentation; If a text segmentation identical to the segmentation to be replaced is found in the slot word table, the slot segmentation having a mapping relationship with the text segmentation identical to the segmentation to be replaced is used as the target slot segmentation; Performing word segmentation replacement on the generalized extraction template according to the target slot segmentation to obtain a slot replacement template; Generate the generalized extraction template L according to the slot replacement template and the extraction position index i Corresponding knowledge extraction template.

5. The method according to claim 4, characterized in that Extracting candidate knowledge description information from the corpus set according to the knowledge extraction template includes: Filling slots in the text data in the corpus set to obtain replacement text data; Matching the knowledge extraction template with the replacement text data; If the knowledge extraction template successfully matches the replacement text data, the extraction position index is obtained, and candidate knowledge description information is obtained from the replacement text data according to the extraction position index.

6. The method according to claim 1, characterized in that Determining the template confidence of the knowledge extraction template includes: Verifying the knowledge legitimacy of the candidate knowledge description information according to the candidate knowledge description information and the knowledge seed set; Determining a current confidence level of the knowledge extraction template according to the knowledge legitimacy; Obtain the generation timestamp of each historical confidence in the historical confidence, and use the historical confidence corresponding to the largest generation timestamp as the relevant historical confidence; The template confidence of the knowledge extraction template is determined according to the relevant historical confidence and the current confidence.

7. The method according to claim 6, characterized in that The knowledge extraction template includes a knowledge extraction template H g , g is a positive integer less than or equal to the total number of the knowledge extraction templates; The verifying the knowledge legitimacy of the candidate knowledge description information according to the candidate knowledge description information and the knowledge seed set includes: Searching the knowledge seed set for knowledge description information that describes the same entity as the candidate knowledge description information as the knowledge description information to be matched; If the candidate knowledge description information is the same as the knowledge description information to be matched, determining the knowledge legitimacy of the candidate knowledge description information as legal; If the candidate knowledge description information is different from the knowledge description information to be matched, the knowledge legitimacy of the candidate knowledge description information is determined to be illegal; Then, determining the current confidence of the knowledge extraction template according to the knowledge legitimacy includes: By extracting the knowledge template H g Among the extracted candidate knowledge description information, the number of candidate knowledge description information whose knowledge legitimacy is legal is counted as a legal parameter; By extracting the knowledge template H g Among the extracted candidate knowledge description information, the number of knowledge description information whose knowledge legitimacy is illegal is counted as an illegal parameter; Determine the knowledge extraction template H according to the legal parameters and the illegal parameters g The current confidence level.

8. The method according to claim 1, characterized in that The candidate knowledge description information includes candidate knowledge description information K j , j is a positive integer; The determining of the knowledge confidence of the candidate knowledge description information according to the template confidence includes: The knowledge description information K can be extracted j The knowledge extraction template of is used as the target knowledge extraction template; The template confidence of the target knowledge extraction template is used as the target template confidence; Determine the target knowledge description information K according to the target confidence j knowledge confidence.

9. The method according to claim 1, characterized in that The updating of the knowledge seed set according to the knowledge confidence and the candidate knowledge description information includes: When the knowledge confidence of the candidate knowledge description information is greater than the credibility confidence threshold, the candidate knowledge description information is used as the credible knowledge description information; The trusted knowledge description information is added to the knowledge seed set to obtain an updated knowledge seed set.

10. The method according to claim 1, characterized in that Updating the knowledge graph according to the updated knowledge seed set includes: Sorting the candidate knowledge description information in the updated knowledge seed set from high to low according to the knowledge confidence, and taking the first L knowledge description information as the knowledge description information to be stored; L is a positive integer; Add the knowledge description information to be stored to the knowledge graph.

11. The method according to claim 1, wherein The step of obtaining a knowledge seed set includes: Get the knowledge seed set in the xth iteration; The method further comprises: When the updated knowledge seed set does not meet the convergence condition, the updated knowledge seed set is used as the knowledge seed set in the x+1th round of iteration, and the knowledge extraction template and candidate knowledge description information in the x+1th round of iteration are continued to be obtained based on the knowledge seed set in the x+1th round of iteration. According to the knowledge extraction template and candidate knowledge description information in the x+1th round of iteration, the knowledge seed set in the x+1th round of iteration is iteratively updated.

12. A data processing device based on knowledge graph, characterized in that: include: An acquisition module, configured to acquire a knowledge seed set, wherein the knowledge seed set includes textual knowledge description information; A template generation module is configured to traverse the text data in the corpus set, take the paragraph containing the knowledge description information in the corpus set as the target paragraph corpus, generate a basic extraction template based on the target paragraph corpus and the knowledge description information, and generate S generalized extraction templates based on the basic extraction template; S is a positive integer; The template generation module is further configured to fill slots in the S generalized extraction templates to obtain a knowledge extraction template; A knowledge extraction module, configured to extract candidate knowledge description information from the corpus set according to the knowledge extraction template; a first determining module, configured to determine a template confidence of the knowledge extraction template; A second determining module, configured to determine the knowledge confidence of the candidate knowledge description information according to the template confidence; A first updating module, configured to update the knowledge seed set according to the knowledge confidence and the candidate knowledge description information; The second updating module is used to update the knowledge graph according to the updated knowledge seed set.

13. A computer device, characterized in that: include: processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide a network communication function, the memory is used to store program code, and the processor is used to call the program code to execute the method described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the method according to any one of claims 1 to 11 is executed.

Images