Model training method and device, knowledge extraction method and device, equipment and medium

Abstract

The present disclosure provides a model training method and device, a knowledge extraction method and device, equipment and a medium, relating to the field of artificial intelligence, specifically to the field of knowledge graph technology. The scheme is: analyzing a sample document, determining the level to which each element in the sample document belongs in the sample document, and generating a document tree according to the level to which each element belongs, wherein the nodes of each level in the document tree are used to indicate the elements of the corresponding level in the sample document; for each node in the document tree, querying a target knowledge extraction model matching the type of the element indicated by the node, and using the target knowledge extraction model to extract knowledge from the element indicated by the node to obtain predicted knowledge; and training the target knowledge extraction model according to the difference between the predicted knowledge and the annotated knowledge corresponding to the element indicated by the node on the sample document. Thus, based on deep learning technology, each knowledge extraction model is trained, which can improve the prediction effect of each knowledge extraction model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically the field of knowledge graph technology, and particularly to model training methods and apparatus, knowledge extraction methods and apparatus, devices and media. Background Technology

[0002] Knowledge is extracted from data of different sources and structures, and stored in a knowledge graph, forming the foundation for technologies such as intelligent question answering and intelligent customer service. Thanks to the continuous development of artificial intelligence and deep learning technologies, models can be used to achieve automatic knowledge extraction. Training the model is crucial to improving its predictive performance. Summary of the Invention

[0003] This disclosure provides a method and apparatus for model training, a method and apparatus for knowledge extraction, an equipment, and a medium.

[0004] According to one aspect of this disclosure, a model training method is provided, comprising:

[0005] Obtain at least one sample document from the sample set, and parse the sample document to determine the level to which each element in the sample document belongs in the sample document;

[0006] A document tree is generated based on the level to which each element belongs; wherein the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the sample document;

[0007] For each node in the document tree, based on the type of the element indicated by the node, query the target knowledge extraction model that matches the type;

[0008] The target knowledge extraction model is used to extract knowledge from the elements indicated by the node to obtain predicted knowledge;

[0009] The target knowledge extraction model is trained based on the difference between the predicted knowledge and the labeled knowledge corresponding to the element indicated by the node on the sample document.

[0010] According to another aspect of this disclosure, a knowledge extraction method is provided, comprising:

[0011] Obtain the document to be recognized;

[0012] The document to be identified is parsed to determine the level to which each element in the document belongs.

[0013] A document tree is generated based on the level to which each element belongs; wherein the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the document to be identified;

[0014] For each node in the document tree, based on the type of the element indicated by the node, query the target knowledge extraction model that matches the type;

[0015] The target knowledge is extracted from the elements indicated by the node using the target knowledge extraction model.

[0016] According to another aspect of this disclosure, a model training apparatus is provided, comprising:

[0017] The parsing module is used to obtain at least one sample document from the sample set and parse the sample document to determine the level to which each element in the sample document belongs in the sample document;

[0018] A generation module is used to generate a document tree according to the level to which each element belongs; wherein, the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the sample document;

[0019] The query module is used to query a target knowledge extraction model that matches the type of the element indicated by the node for each node in the document tree.

[0020] An extraction module is used to extract knowledge from the elements indicated by the node using the target knowledge extraction model to obtain predicted knowledge.

[0021] The training module is used to train the target knowledge extraction model based on the difference between the predicted knowledge and the labeled knowledge corresponding to the element indicated by the node on the sample document.

[0022] According to another aspect of this disclosure, a knowledge extraction apparatus is provided, comprising:

[0023] The acquisition module is used to acquire the document to be recognized;

[0024] The parsing module is used to parse the document to be identified in order to determine the level to which each element in the document belongs.

[0025] A generation module is used to generate a document tree according to the level to which each element belongs; wherein, the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the document to be identified;

[0026] The query module is used to query a target knowledge extraction model that matches the type of the element indicated by the node for each node in the document tree.

[0027] The extraction module is used to extract knowledge from the elements indicated by the node using the target knowledge extraction model to obtain target knowledge.

[0028] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0029] At least one processor; and

[0030] A memory communicatively connected to the at least one processor; wherein,

[0031] The memory stores instructions that can be executed by the at least one processor, which, when executed, enable the at least one processor to perform either the model training method proposed in one aspect of this disclosure or the knowledge extraction method proposed in another aspect of this disclosure.

[0032] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided, the computer instructions being used to cause the computer to perform the model training method proposed in one aspect of this disclosure, or to perform the knowledge extraction method proposed in another aspect of this disclosure.

[0033] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the model training method proposed in one aspect of this disclosure, or implements the knowledge extraction method proposed in another aspect of this disclosure.

[0034] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0035] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0036] Figure 1 This is a flowchart illustrating the model training method provided in Embodiment 1 of this disclosure;

[0037] Figure 2 This is a schematic diagram of the document tree structure in an embodiment of this disclosure;

[0038] Figure 3 This is a schematic flowchart of the model training method provided in Embodiment 2 of this disclosure;

[0039] Figure 4 This is a flowchart illustrating the model training method provided in Embodiment 3 of this disclosure;

[0040] Figure 5 This is a schematic flowchart of the model training method provided in Embodiment 4 of this disclosure;

[0041] Figure 6 This is a schematic diagram of the structure of an IDAC system used to implement the method proposed in any embodiment of this disclosure;

[0042] Figure 7 For this disclosure Figure 6 A schematic diagram of the sample factory architecture;

[0043] Figure 8 This is a schematic diagram illustrating the principle of active learning in an embodiment of this disclosure;

[0044] Figure 9 This is a flowchart illustrating the knowledge extraction method provided in Embodiment 5 of this disclosure;

[0045] Figure 10 This is a schematic diagram of the structure of the model training device provided in Embodiment Six of this disclosure;

[0046] Figure 11 This is a schematic diagram of the knowledge extraction device provided in Embodiment 7 of this disclosure;

[0047] Figure 12 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0048] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0049] Knowledge extraction from document data yields knowledge triples in the form of (head entity, entity relation, tail entity) (hereinafter referred to as SPO triples, where S represents the head entity, P represents the entity relation, and O represents the tail entity). These triples form the basis of technologies such as intelligent question answering and intelligent customer service, and have wide applications in various fields including healthcare, finance, public security, and the judiciary. Examples include intelligent customer service, intelligent investment research, intelligent investment advisory, and risk control decision-making in the financial industry, and intelligent search, legal reasoning, intelligent adjudication, and document drafting and review in the legal industry.

[0050] Thanks to the development of artificial intelligence and deep learning technologies, natural language processing technologies such as knowledge extraction have made rapid progress in recent years, achieving good predictive results without the need for manually defined high-level features. However, related system designs are relatively lacking, and the interaction between various steps relies heavily on human intervention, resulting in high usage costs.

[0051] The main implementation schemes of current knowledge extraction systems include the following:

[0052] The first method involves linking manually maintained offline scripts to complete the knowledge extraction process. When the system environment is relatively simple, the entire knowledge extraction process can be completed using offline scripts manually maintained by R&D engineers.

[0053] The second approach, based on the concept of distributed scheduling, optimizes the maintenance cost of offline scripts and enables distributed scheduling and task management.

[0054] However, the above designs are all model-centric, with training data existing as an adjunct to the model. When the number of training data iterations is large, or the training data experiments are complex, the complexity of the entire system will be extremely high.

[0055] To address the aforementioned issues, this disclosure proposes a model training method and apparatus, a knowledge extraction method and apparatus, equipment, and medium.

[0056] The following description, with reference to the accompanying drawings, outlines embodiments of the model training method and apparatus, knowledge extraction method and apparatus, device, and medium of this disclosure.

[0057] Figure 1 This is a schematic flowchart of the model training method provided in Embodiment 1 of this disclosure.

[0058] This disclosure illustrates the example of the model training method being configured in a model training device, which can be applied to any electronic device to enable the electronic device to perform model training functions.

[0059] Among them, electronic devices can be any device with computing capabilities, such as personal computers, mobile terminals, servers, etc. Mobile terminals can be hardware devices with various operating systems, touch screens and / or displays, such as mobile phones, tablets, personal digital assistants, wearable devices, etc.

[0060] like Figure 1 As shown, the model training method may include the following steps:

[0061] Step 101: Obtain at least one sample document from the sample set and parse the sample document to determine the level to which each element in the sample document belongs.

[0062] In this embodiment of the disclosure, the sample set includes multiple sample documents. Each sample document can be obtained from an existing training set, or the sample documents can be collected online, for example, by using web crawler technology to collect sample documents online, or the sample documents can be collected offline, for example, by capturing images of the content of paper documents and then using OCR (Optical Character Recognition) technology to recognize each character in the image to obtain the sample document, etc. This embodiment of the disclosure does not limit this.

[0063] In this embodiment of the disclosure, the sample document is annotated with knowledge information, referred to as annotated knowledge in this disclosure. The annotated knowledge may include at least one of the knowledge triples, which may be an SPO triple, where S refers to the head entity, P refers to the entity relationship between the head entity and the tail entity, and O refers to the tail entity. That is, the annotated knowledge may include at least one of the following: head entity, entity relationship between the head entity and the tail entity, and tail entity.

[0064] In this embodiment of the disclosure, the elements in the sample document may include at least one of the following: title element (optionally, the title element may be further subdivided into document title element, first-level heading element, second-level heading element, and third-level heading element), chapter element, table element (optionally, the table element may be further subdivided into table-content element and table-short text element), content element, and key-value (KV) element. The KV element may be semi-structured information containing attribute-attribute values, such as occupation: singer, actor, birthday: XXXX year XX month XX day.

[0065] In this embodiment of the disclosure, for at least one sample document in the sample set, the sample document can be parsed to determine the level to which each element in the sample document belongs. For example, for a title element, the level to which it belongs in the sample document can be the first level; for a chapter element, the level to which it belongs in the sample document can be the second level; and for body text elements, table elements, or key-value (KV) elements, the level to which they belong in the sample document can be the third level. Optionally, when a table element is divided into long table text elements and short table text elements, the level to which the short table text elements belong in the sample document can be the third level, and the level to which the long table text elements belong in the sample document can be the fourth level.

[0066] Step 102: Generate a document tree based on the level to which each element belongs; wherein the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the sample document.

[0067] In this embodiment of the disclosure, a document tree can be generated according to the level to which each element belongs. The document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the sample document.

[0068] As an example, the structure of a document tree can be as follows: Figure 2 As shown, the node root refers to the root node, the node title refers to the node corresponding to the title element in the document tree, the node chapter refers to the node corresponding to the chapter element in the document tree, the node content refers to the node corresponding to the body text element in the document tree, the node table refers to the node corresponding to the table element (such as the short text element in the table) in the document tree, the node KV refers to the node corresponding to the KV element in the document tree, and the node table-content refers to the node corresponding to the long text element in the table in the document tree.

[0069] Step 103: For each node in the document tree, query the target knowledge extraction model that matches the type of the element indicated by the node.

[0070] In this embodiment of the disclosure, for each node in the document tree, a target knowledge extraction model matching the type of the element indicated by the node can be queried. The target knowledge extraction models can be different or the same when the element types are different; this disclosure does not impose any limitations on this.

[0071] For example, for heading and chapter elements, due to their short text length and hierarchical structure, knowledge extraction can be performed using a Named Entity Recognition (NER) model. That is, the target knowledge extraction model matching the type of a heading element (i.e., heading) can be an NER model, and the target knowledge extraction model matching the type of a chapter element (i.e., chapter) can also be an NER model. For body text elements, since the body text is a relatively complete paragraph, knowledge extraction can be performed using models such as named entity recognition, end-to-end SPO extraction, and slot filling.

[0072] Step 104: Use the target knowledge extraction model to extract knowledge from the element indicated by the node to obtain the predicted knowledge.

[0073] In this embodiment of the disclosure, for each node in the document tree, when a target knowledge extraction model that matches the type of the element indicated by the node is found, the target knowledge extraction model can be used to extract knowledge from the element indicated by the node. In this disclosure, the knowledge extracted by the target knowledge extraction model is recorded as predicted knowledge.

[0074] For example, when the element indicated by the node is a title or chapter element, a Named Entity Recognition (NER) model can be used to extract knowledge from the element, specifically S or SP. When the element indicated by the node is a body text element, models such as Named Entity Recognition (NER), end-to-end SPO extraction, and slot filling can be used to extract knowledge from the element, specifically SPO. When the element indicated by the node is a table element, table recognition models can be used to identify the table structure and content, thereby extracting P, PO, or SPO. When the element indicated by the node is a key-value (KV) element, models such as string matching patterns, reading comprehension, and sequence labeling can be used to extract PO.

[0075] Step 105: Train the target knowledge extraction model based on the difference between the predicted knowledge and the labeled knowledge corresponding to the element indicated by the node in the sample document.

[0076] In this embodiment of the disclosure, the target knowledge extraction model can be trained based on the difference between the predicted knowledge and the labeled knowledge corresponding to the element indicated by the node on the sample document, so as to minimize the above difference.

[0077] The model training method of this disclosure parses sample documents to determine the level to which each element belongs in the sample document, and generates a document tree based on the level of each element. The document tree includes nodes at each level, and each node indicates an element at the corresponding level in the sample document. For each node in the document tree, based on the type of the element indicated by the node, a target knowledge extraction model matching the type is queried, and the target knowledge extraction model is used to extract knowledge from the element indicated by the node to obtain predicted knowledge. The target knowledge extraction model can then be trained based on the difference between the predicted knowledge and the labeled knowledge corresponding to the element indicated by the node in the sample document. Therefore, training each knowledge extraction model based on deep learning technology can improve the prediction performance of each knowledge extraction model. Furthermore, each knowledge extraction model is trained using document elements corresponding to its matching type, enabling targeted training of each knowledge extraction model, thereby improving the prediction performance of each knowledge extraction model and ultimately enhancing the accuracy and reliability of document knowledge extraction.

[0078] It is understandable that when the number of samples in the sample set is small, it may be difficult to guarantee the prediction accuracy of each knowledge extraction model. Therefore, to address the above problem, this disclosure utilizes existing sample documents to expand the sample set, as described below. Figure 3 The above process will be explained in detail.

[0079] Figure 3 This is a schematic flowchart of the model training method provided in Embodiment 2 of this disclosure.

[0080] like Figure 3 As shown, the model training method may include the following steps:

[0081] Step 301: For any sample document in the sample set, perform data augmentation processing on the sample document to obtain an expanded sample.

[0082] In a first possible implementation of this disclosure, for any sample document in the sample set, at least one word in the sample document can be replaced with a synonym to obtain an expanded sample.

[0083] For example, you can replace "draw inferences from one instance" with "learn by analogy" in the sample document, and change "happy" to "joyful" to obtain an expanded sample.

[0084] In a second possible implementation of this disclosure, for any sample document in the sample set, at least one word in the sample document can be replaced with a word of the same type to obtain an expanded sample.

[0085] For example, you can replace "using a sledgehammer to crack a nut" with "making a fuss over nothing" and "grand and spectacular" with "extensive and unrestrained" in the sample document to obtain an expanded sample.

[0086] In a third possible implementation of this disclosure, for any sample document in the sample set, at least one word in the sample document can be shuffled to obtain an expanded sample.

[0087] For example, for a certain sentence in the sample document, such as "I went to a certain place for tourism", the sentence can be scrambled to "I went to a certain place for tourism". The scrambled sentence can then be used to replace the corresponding sentence in the sample document to obtain an expanded sample.

[0088] In a fourth possible implementation of this disclosure, for any sample document in the sample set, a target statement can be generated based on the established knowledge and at least one source statement in the sample document. The target statement is then used to replace the source statement in the sample document, resulting in an expanded sample. In other words, based on existing established knowledge and the source statements in the sample document, a new target statement can be generated using a remote supervision approach. This new target statement is then used to update the source statements in the sample document, resulting in an expanded sample.

[0089] In a fifth possible implementation of the present disclosure, for any sample document in the sample set, the sample document belonging to the first language can be translated into translated text in the second language, and the translated text can be translated to obtain back-translated text in the first language. The back-translated text can then be used to update the sample document to obtain an expanded sample.

[0090] For example, taking Chinese as the first language and English as the second language, a sample document in Chinese can be translated into English, and then the English translation can be translated back into Chinese. This back-translated text can then be used as an expanded sample.

[0091] In a sixth possible implementation of the present disclosure, when the annotation knowledge in the sample document includes each entity word (head entity and / or tail entity) and the entity tag (S and / or O) corresponding to each entity word, for any sample document in the sample set, at least two entity words with the same entity tag in the sample document can be determined, and at least two entity words can be replaced to obtain an expanded sample.

[0092] For example, taking the sample document containing the statements "Liu's wife is Zhu" and "Liang's friend is Zhang" as an example, "Liu" and "Liang" have the same entity label (S). You can replace "Liu" with "Liang" to get "Liang's wife is Zhu", and you can also replace "Liu" with "Liang" to get "Liu's friend is Zhang".

[0093] In a seventh possible implementation of the present disclosure, when the labeled knowledge in the sample document includes each entity word (head entity and / or tail entity) and the entity tag (S and / or O) corresponding to each entity word, for any sample document in the sample set, a target entity tag that is the same as the set entity tag can be determined from the sample document, and the entity word corresponding to the set entity tag is used to replace the entity word corresponding to the target entity tag in the sample document to obtain an expanded sample.

[0094] For example, if we set the entity label as O and the entity word corresponding to this entity label is "Guo Mou", and the entity word corresponding to the target entity label (i.e. O) in the sample document is "Zhu Mou", then we can use "Guo Mou" to replace "Zhu Mou" corresponding to the target entity label in the sample document to obtain an expanded sample.

[0095] In the eighth possible implementation of this disclosure, when the annotation knowledge in the sample document also includes the entity relationship between each entity word (i.e., the entity relationship between the head entity and the tail entity) and the relationship label (i.e. P) corresponding to the entity relationship, for any sample document in the sample set, at least two entity relationships with the same relationship label in the sample document can be determined, and the at least two entity relationships can be replaced to obtain the expanded sample.

[0096] For example, taking the sample document containing the statements "Liu's wife is Zhu" and "Liang's friend is Zhang" as an example, "Liu" and "Liang" have the same entity label (S), "wife" and "friend" have the same relation label (P), and "Zhu" and "Zhang" have the same entity label (O). Therefore, the two entity relations with the same relation label (P) can be "wife" and "friend". After replacing these two entity relations, the corresponding statements in the expanded sample after replacement are: "Liu's friend is Zhu" and "Liang's wife is Zhang".

[0097] In the ninth possible implementation of this disclosure, when the annotation knowledge in the sample document also includes the entity relationship between each entity word (i.e., the entity relationship between the head entity and the tail entity) and the relationship label (i.e., P) corresponding to the entity relationship, for any sample document in the sample set, a target relationship label that is the same as the set relationship label can be determined from the sample document, and the entity relationship corresponding to the target relationship label in the sample document can be replaced by the entity relationship corresponding to the set relationship label to obtain the expanded sample.

[0098] For example, if the entity relation corresponding to the set relation label (i.e., P) is "classmate", then the entity relation "wife" corresponding to the target relation label (P) in the sample document "Liu's wife is Zhu" can be replaced with "classmate", and the corresponding statement in the expanded sample after the replacement will be: "Liu's classmate is Zhu".

[0099] It should be noted that the above examples only demonstrate the individual execution of the nine enhancement methods. In practical applications, multiple combinations of these nine enhancement methods can be executed simultaneously, and this disclosure does not impose any limitations on this. Therefore, data enhancement processing of sample documents can be performed in multiple ways, improving the flexibility and applicability of the method.

[0100] It should be understood that other existing enhancement methods can also be used to perform data augmentation on the sample documents, and this disclosure does not limit this.

[0101] Step 302: Update the sample set by expanding the sample set.

[0102] In this embodiment of the disclosure, after obtaining the expanded sample, the expanded sample can be added to the sample set.

[0103] Step 303: Obtain at least one sample document from the updated sample set, and parse the sample document to determine the level to which each element in the sample document belongs.

[0104] Step 304: Generate a document tree based on the level to which each element belongs; wherein the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the sample document.

[0105] Step 305: For each node in the document tree, query the target knowledge extraction model that matches the type of the element indicated by the node.

[0106] Step 306: Use the target knowledge extraction model to extract knowledge from the element indicated by the node to obtain predicted knowledge.

[0107] Step 307: Train the target knowledge extraction model based on the difference between the predicted knowledge and the labeled knowledge corresponding to the element indicated by the node in the sample document.

[0108] The execution process of steps 303 to 307 can be found in the execution process of any embodiment of this disclosure, and will not be described in detail here.

[0109] The model training method of this disclosure performs data augmentation on sample documents to obtain expanded samples, and then updates the sample set using these expanded samples. This allows for dynamic replenishment of the sample set, increasing the number of samples in the set. By utilizing a large number of sample documents to train each knowledge extraction model, the predictive performance of each knowledge extraction model can be improved, thereby enhancing the accuracy and reliability of knowledge extraction.

[0110] It should be noted that the formats of the sample documents in the sample set may differ. Parsing and processing sample documents of different formats will significantly increase the system's processing burden. To address this issue, in one possible implementation of this disclosure, after obtaining the text document, a unified format conversion can be performed on the sample documents. The following is a collection... Figure 4 The above process will be explained in detail.

[0111] Figure 4 This is a flowchart illustrating the model training method provided in Embodiment 3 of this disclosure.

[0112] like Figure 4 As shown, the model training method may include the following steps:

[0113] Step 401: Obtain at least one sample document from the sample set.

[0114] The execution process of step 401 can be referred to any of the above embodiments, and will not be repeated here.

[0115] Step 402: Obtain the document format of the sample document.

[0116] In this embodiment of the disclosure, the document format may include TXT format, DOC format, PDF format, RTF (Rich Text Field) format, HTML (Hypertext Markup Language) format, etc.

[0117] Step 403: Determine if the document format is the set format. If yes, proceed to step 405; otherwise, proceed to step 404.

[0118] In the embodiments of this disclosure, the format is set to a pre-defined document format, which can be HTML format or other formats, such as DOC format. This disclosure does not limit this.

[0119] Step 404: Convert the format of the sample document to obtain a sample document with the set format.

[0120] In this embodiment of the disclosure, the document format of the sample document can be obtained, and it can be determined whether the document format is a set format. If the document format is not a set format, the sample document can be converted to obtain a sample document with a set format.

[0121] As an example, a sample document can be formatted using a document parsing algorithm to obtain a sample document with a set format. The parsing algorithm includes algorithms such as format conversion and OCR.

[0122] Step 405: Parse the sample document to determine the level to which each element in the sample document belongs.

[0123] Step 406: Generate a document tree based on the level to which each element belongs; wherein the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the sample document.

[0124] Step 407: For each node in the document tree, query the target knowledge extraction model that matches the type of the element indicated by the node.

[0125] Step 408: Use the target knowledge extraction model to extract knowledge from the element indicated by the node to obtain predicted knowledge.

[0126] Step 409: Train the target knowledge extraction model based on the difference between the predicted knowledge and the labeled knowledge corresponding to the element indicated by the node in the sample document.

[0127] The execution process of steps 405 to 409 can be referred to the execution process of any of the above embodiments of this disclosure, and will not be repeated here.

[0128] The model training method of this disclosure obtains the document format of the sample document and determines whether the document format is a predetermined format. If not, the sample document is converted to a predetermined format. This allows for format standardization of all sample documents, facilitating subsequent document parsing, model training, and other processing, reducing processing burden, and improving training efficiency.

[0129] In one possible implementation of this disclosure, to improve the prediction performance of each knowledge extraction model, iterative optimization can be performed on each knowledge extraction model based on unlabeled documents. The following is in conjunction with... Figure 5 The above process will be explained in detail.

[0130] Figure 5 This is a schematic flowchart of the model training method provided in Embodiment 4 of this disclosure.

[0131] like Figure 5 As shown, based on any of the embodiments described above, the model training method may further include the following steps:

[0132] Step 501: Obtain the target document to be annotated.

[0133] In this embodiment of the disclosure, the target document refers to a document that is not labeled with knowledge. The target document can be obtained from an existing test set, or it can be collected online, for example, through web crawling technology, or it can be collected offline, or it can be a document input or selected by the user, etc. This embodiment of the disclosure does not impose any limitations on these aspects.

[0134] Step 502: Extract knowledge from the target document based on each target knowledge extraction model to obtain target knowledge.

[0135] In this embodiment of the disclosure, the target document can be parsed to determine the level to which each element in the target document belongs, and a document tree can be generated based on the level to which each element belongs. The document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the target document. For each node in the document tree, a target knowledge extraction model that matches the type of the element indicated by the node can be queried, and the target knowledge extraction model is used to extract knowledge from the element indicated by the node to obtain the target knowledge.

[0136] Step 503: In response to the update operation for the target knowledge, update the target knowledge and use the updated target knowledge to annotate the target document.

[0137] In this embodiment of the disclosure, the target knowledge can be updated manually. That is, when the annotator manually triggers the update operation of the target knowledge, the target knowledge can be updated in response to the update operation, and the updated target knowledge can be used to annotate the target document.

[0138] Step 504: Retrain each target knowledge extraction model using the labeled target documents.

[0139] In this embodiment of the disclosure, after the target document is annotated, the annotated target document can be used to retrain each target knowledge extraction model. That is, a training process similar to steps 101 to 106 can be performed based on the annotated target document.

[0140] As an example, when this method is applied to an IDAC (Intelligent Document Analysis Center) system, the structure of the IDAC system can be as follows: Figure 6 As shown, the IDAC system mainly consists of two parts. The first part is a sample-centric model, which mainly includes four modules: sample management, sample generation, model retraining, and knowledge annotation. The second part is a knowledge recognition process built based on the prediction results of the model, which mainly completes the knowledge recognition in the document based on the optimal model performance.

[0141] in, Figure 6 In MongoDB, operators include function operators and models, and schema is a data schema.

[0142] In the first part, considering that data centers and model centers are currently the two paradigms for knowledge extraction, and that these two paradigms are suitable for different application scenarios, it is intuitively apparent that model centers are more suitable for situations where sample data does not change significantly and the model is constantly iterating (i.e., the model structure changes and requires continuous iteration). Therefore, by fixing the sample data set, it is easier to focus on the model. Data centers, on the other hand, have greater advantages. They are not only suitable for situations where the model does not change significantly, but also for situations where the model changes significantly, and they also support reasonable changes in sample data, thereby optimizing model performance at low cost.

[0143] Therefore, in this disclosure, Figure 6 The architecture of the sample factory in the example can be as follows: Figure 7 As shown, it mainly consists of the following three functions:

[0144] First, sample management: This mainly involves adding, deleting, modifying, and querying sample data for user convenience. The managed sample information primarily includes sample content, sample generation results, and sample metadata.

[0145] The sample content includes core attributes such as text content and labeled knowledge.

[0146] Sample generation results refer to expanded samples generated based on the sample set through sample augmentation or active learning schemes.

[0147] Sample metadata includes other attribute information of the sample that does not affect the sample semantics, such as the sample's producer, production time, production method (e.g., manual annotation), etc., which record the sample's lifecycle.

[0148] Second, sample augmentation refers to generating new samples based on existing labeled samples in an unsupervised manner. This includes synonym replacement, replacement of similar words, distant supervision (the fourth data augmentation scheme in step 301), back translation (the fifth data augmentation scheme in step 301), character replacement with the same label (the sixth to ninth data augmentation schemes in steps 301), and random shuffling within sentences (the third data augmentation scheme in step 301).

[0149] Third, active learning refers to the continuous iterative optimization of models through collaboration between annotators and existing models. The optimization principle is as follows: Figure 8As shown, the complete active learning process consists of five steps: sample construction → model retraining → using the trained model to predict unlabeled sample data → using the active learning query function Q to select a subset of samples from the model's predictions → manual labeling of the selected samples → sample construction. Considering the large volume of unlabeled sample data and the enormous workload of labelers correcting each sample (making it impossible to provide every sample to labelers), the query function Q is used to select a subset of samples.

[0150] The second part is the knowledge recognition process, which includes steps such as document recognition, knowledge recognition, post-processing, and knowledge review.

[0151] 1. Document Parsing: Using document parsing algorithms, the original documents (including documents in Word, HTML, PDF, etc.) are uniformly converted into a set format, such as HTML documents.

[0152] 2. Document analysis: Parses a formatted document into a document tree for downstream use. The parsed document tree can be used as follows: Figure 2 As shown. The document tree mainly includes the following three parts:

[0153] A. Elements corresponding to nodes (i.e., text information): Except for the table, which is text in the original HTML format, the elements corresponding to other nodes are all in plain text format;

[0154] B. Node hierarchy information: Specifies the parent node and all child nodes of the current node;

[0155] C. Node mapping information in the original document: its location in the original document.

[0156] Among them, KV can be semi-structured information containing attribute-attribute values, such as occupation: singer, actor, birthday: XXXX year XX month XX day.

[0157] 3. Knowledge recognition: Based on the document tree from the previous step, automatically select the element type indicated by each node and call the corresponding operator (i.e., model) to complete the knowledge recognition of the element indicated by the node.

[0158] 4. Post-processing: The knowledge extracted by the operator is processed in a unified manner. For example, the knowledge is cleaned to standardize its format. For instance, the brackets "[]" before and after the entity words corresponding to the book title are removed. Alternatively, the extracted knowledge can be formatted for downstream use.

[0159] Therefore, centering on sample data makes model management and service management more convenient. The main reason is that sample data is plain text content, which has clearer semantics compared to models, and it is also more stable. Furthermore, unchanging sample data represents unchanging requirements, allowing for a greater focus on the model and simplifying the system.

[0160] The model training method of this disclosure involves acquiring target documents to be labeled, extracting knowledge from the target documents based on various target knowledge extraction models to obtain target knowledge, updating the target knowledge in response to an update operation on the target knowledge, and labeling the target documents using the updated target knowledge. The labeled target documents are then used to retrain the target knowledge extraction models. This allows for iterative optimization of each knowledge extraction model based on unlabeled documents, thereby improving the prediction performance of each model.

[0161] The above are various embodiments corresponding to the training method of the knowledge extraction model. This disclosure also proposes an application method of the knowledge extraction model, namely, a method of knowledge extraction using the knowledge extraction model.

[0162] Figure 9 This is a flowchart illustrating the knowledge extraction method provided in Embodiment 5 of this disclosure.

[0163] like Figure 9 As shown, this knowledge extraction method may include the following steps:

[0164] Step 901: Obtain the document to be recognized.

[0165] In the embodiments of this disclosure, the document to be identified can be obtained from an existing test set, or the document to be identified can be collected online, for example, by using web crawler technology to collect the document to be identified online, or the document to be identified can be collected offline, or the document to be identified can be a document input or selected by the user, etc., and the embodiments of this disclosure do not limit this.

[0166] Step 902: Parse the document to be identified to determine the level to which each element in the document belongs.

[0167] In this embodiment of the disclosure, the document to be identified can be parsed to determine the level to which each element in the document belongs. The specific implementation principle is similar to step 101, and will not be repeated here.

[0168] Step 903: Generate a document tree based on the level to which each element belongs; wherein the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the document to be identified.

[0169] In this embodiment of the disclosure, a document tree can be generated according to the level to which each element belongs. The document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the document to be identified.

[0170] Step 904: For each node in the document tree, query the target knowledge extraction model that matches the type of the element indicated by the node.

[0171] Among them, the knowledge extraction models for each target are adopted Figures 1 to 5 The method proposed in any embodiment is used for training.

[0172] In this embodiment of the disclosure, for each node in the document tree, a target knowledge extraction model matching the type of the element indicated by that node can be queried. The specific implementation principle is similar to step 103, and will not be repeated here.

[0173] Step 905: Use the target knowledge extraction model to extract knowledge from the element indicated by the node to obtain the target knowledge.

[0174] In this embodiment of the disclosure, for each node in the document tree, after finding a target knowledge extraction model that matches the type of the element indicated by the node, the target knowledge extraction model can be used to extract knowledge from the element indicated by the node. In this disclosure, the knowledge extracted by the target knowledge extraction model is recorded as target knowledge. The specific implementation principle is similar to step 104, and will not be described in detail here.

[0175] The knowledge extraction method of this disclosure parses the document to be identified to determine the level to which each element in the document belongs, and generates a document tree based on the level of each element. The document tree includes nodes at each level, and each node indicates an element at the corresponding level in the document. For each node in the document tree, a target knowledge extraction model matching the type of the element indicated by the node is queried, and the target knowledge extraction model is used to extract knowledge from the element indicated by the node to obtain the target knowledge. Therefore, based on deep learning technology, knowledge extraction from the document to be identified can improve the accuracy of the knowledge extraction results. Furthermore, by using a target knowledge extraction model matching the type of each element in the document to extract knowledge from that element, the accuracy of the knowledge extraction results can be further improved.

[0176] With the above Figures 1 to 5 Corresponding to the model training method provided in the embodiments, this disclosure also provides a model training apparatus. Since the model training apparatus provided in the embodiments of this disclosure is similar to the one described above... Figures 1 to 5The model training method provided in the embodiments corresponds to the model training device provided in the embodiments of this disclosure, and will not be described in detail in the embodiments of this disclosure.

[0177] Figure 10 This is a schematic diagram of the model training device provided in Embodiment Six of this disclosure.

[0178] like Figure 10 As shown, the model training device 1000 may include: a parsing module 1010, a generation module 1020, a query module 1030, an extraction module 1040, and a training module 1050.

[0179] The parsing module 1010 is used to obtain at least one sample document from the sample set and parse the sample document to determine the level to which each element in the sample document belongs.

[0180] The generation module 1020 is used to generate a document tree based on the level to which each element belongs; wherein the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the sample document.

[0181] The query module 1030 is used to query the target knowledge extraction model that matches the type of the element indicated by the node for each node in the document tree.

[0182] The extraction module 1040 is used to extract knowledge from the elements indicated by the nodes using a target knowledge extraction model in order to obtain predicted knowledge.

[0183] The training module 1050 is used to train the target knowledge extraction model based on the difference between the predicted knowledge and the labeled knowledge corresponding to the elements indicated by the nodes on the sample documents.

[0184] In one possible implementation of this disclosure, the model training device 1000 may further include:

[0185] The enhancement module is used to perform data augmentation on sample documents to obtain expanded samples.

[0186] The update module is used to update the sample set by expanding the sample set.

[0187] Specifically, the enhancement module is used to perform at least one of the following:

[0188] Perform synonym replacement on at least one word in the sample document;

[0189] Replace at least one word in the sample document with a word of the same type;

[0190] Randomize at least one word in the sample document;

[0191] Based on the defined knowledge and at least one source statement in the sample document, generate a target statement and use the target statement to replace the source statement in the sample document.

[0192] In one possible implementation of this disclosure, the enhancement module is further configured to: translate the sample document belonging to the first language into translated text in the second language; translate the translated text to obtain back-translated text in the first language; and update the sample document using the back-translated text.

[0193] In one possible implementation of this disclosure, the annotation knowledge includes each entity word and the entity tag corresponding to each entity word; the enhancement module is further configured to perform at least one of the following:

[0194] Identify at least two entity words with the same entity tag in the sample document, and replace at least two entity words.

[0195] Identify the target entity tag that is the same as the set entity tag from the sample document, and replace the entity words corresponding to the target entity tag in the sample document with the entity words corresponding to the set entity tag.

[0196] In one possible implementation of this disclosure, the annotation knowledge further includes entity relationships between entity words and relationship tags corresponding to those entity relationships; the enhancement module is also configured to perform at least one of the following:

[0197] Identify at least two entity relationships with the same relation tag in the sample document, and replace at least two entity relationships;

[0198] Identify the target relation label from the sample document that is identical to the set relation label, and replace the entity relation corresponding to the target relation label in the sample document with the entity relation corresponding to the set relation label.

[0199] In one possible implementation of this disclosure, the model training device 1000 may further include:

[0200] The first acquisition module is used to acquire the document format of the sample document.

[0201] The judgment module is used to determine whether the document format is the set format.

[0202] The conversion module is used to convert the format of a sample document to the specified format when the document format is not the set format.

[0203] In one possible implementation of this disclosure, the model training device 1000 may further include:

[0204] The second acquisition module is used to acquire the target document to be labeled.

[0205] The extraction module 1040 is also used to extract knowledge from target documents based on various target knowledge extraction models to obtain target knowledge.

[0206] The update module is used to update the target knowledge in response to update operations on the target knowledge.

[0207] The annotation module is used to annotate target documents using updated target knowledge.

[0208] The training module 1050 is also used to retrain each target knowledge extraction model using labeled target documents.

[0209] The model training apparatus of this disclosure analyzes sample documents to determine the level to which each element belongs within the document. Based on the level of each element, a document tree is generated, comprising nodes at each level, each node indicating an element at the corresponding level in the sample document. For each node in the document tree, a target knowledge extraction model matching the type of the element indicated by the node is queried. The target knowledge extraction model is then used to extract knowledge from the element indicated by the node to obtain predicted knowledge. The target knowledge extraction model can be trained based on the difference between the predicted knowledge and the labeled knowledge corresponding to the element indicated by the node in the sample document. Therefore, training each knowledge extraction model based on deep learning technology can improve the prediction performance of each model. Furthermore, each knowledge extraction model is trained using document elements corresponding to its matching type, enabling targeted training of each model and thus improving its prediction performance, thereby enhancing the accuracy and reliability of document knowledge extraction.

[0210] With the above Figure 9 Corresponding to the knowledge extraction method provided in the embodiments, this disclosure also provides a knowledge extraction device. Because the knowledge extraction device provided in the embodiments of this disclosure is similar to the one described above... Figure 9 The knowledge extraction method provided in the embodiments corresponds to the knowledge extraction device provided in the embodiments of this disclosure, and will not be described in detail in the embodiments of this disclosure.

[0211] Figure 11 This is a schematic diagram of the knowledge extraction device provided in Embodiment 7 of this disclosure.

[0212] like Figure 11As shown, the knowledge extraction device 1100 may include: an acquisition module 1110, a parsing module 1120, a generation module 1130, a query module 1140, and an extraction module 1150.

[0213] The acquisition module 1110 is used to acquire the document to be recognized.

[0214] The parsing module 1120 is used to parse the document to be identified in order to determine the level to which each element in the document belongs.

[0215] The generation module 1130 is used to generate a document tree based on the level to which each element belongs; wherein the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the document to be identified.

[0216] The query module 1140 is used to query the target knowledge extraction model that matches the type of the element indicated by the node for each node in the document tree.

[0217] The extraction module 1150 is used to extract knowledge from the elements indicated by the nodes using a target knowledge extraction model in order to obtain target knowledge.

[0218] The knowledge extraction apparatus of this disclosure analyzes a document to be identified to determine the level to which each element in the document belongs. Based on the level of each element, a document tree is generated, comprising nodes at each level, each node indicating an element at the corresponding level in the document. For each node in the document tree, a target knowledge extraction model matching the type of the element indicated by the node is queried. The target knowledge extraction model is then used to extract knowledge from the element indicated by the node to obtain target knowledge. Therefore, based on deep learning technology, knowledge extraction from the document to be identified can improve the accuracy of the knowledge extraction results. Furthermore, by using a target knowledge extraction model matching the type of each element in the document to extract knowledge from that element, the accuracy of the knowledge extraction results can be further improved.

[0219] To implement the above embodiments, this disclosure also provides an electronic device, which may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the model training method proposed in any of the above embodiments of this disclosure, or to execute the knowledge extraction method proposed in the above embodiments of this disclosure.

[0220] To implement the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the model training method proposed in any of the above embodiments of this disclosure, or to execute the knowledge extraction method proposed in the above embodiments of this disclosure.

[0221] To implement the above embodiments, this disclosure also provides a computer program product, which includes a computer program that, when executed by a processor, implements the model training method proposed in any of the above embodiments of this disclosure, or implements the knowledge extraction method proposed in the above embodiments of this disclosure.

[0222] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0223] Figure 12 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0224] like Figure 12 As shown, device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in ROM (Read-Only Memory) 1202 or loaded from storage unit 1207 into RAM (Random Access Memory) 1203. RAM 1203 may also store various programs and data required for the operation of device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. I / O (Input / Output) interface 1205 is also connected to bus 1204.

[0225] Multiple components in device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, etc.; output unit 1207, such as various types of monitors, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0226] The computing unit 1201 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as the model training or knowledge extraction methods described above. For example, in some embodiments, the model training or knowledge extraction methods described above can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by computing unit 1201, one or more steps of the model training or knowledge extraction method described above can be performed. Alternatively, in other embodiments, computing unit 1201 can be configured to perform the above-described model training or knowledge extraction method by any other suitable means (e.g., by means of firmware).

[0227] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0228] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0229] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0230] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0231] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0232] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers integrated with blockchain technology.

[0233] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0234] According to the technical solution of this disclosure, by parsing a sample document, the level to which each element in the sample document belongs is determined, and a document tree is generated based on the level to which each element belongs. The document tree includes nodes at each level, and each node at each level indicates an element at the corresponding level in the sample document. For each node in the document tree, based on the type of the element indicated by the node, a target knowledge extraction model matching the type is queried, and the target knowledge extraction model is used to extract knowledge from the element indicated by the node to obtain predicted knowledge. Thus, the target knowledge extraction model can be trained based on the difference between the predicted knowledge and the labeled knowledge corresponding to the element indicated by the node in the sample document. Therefore, training each knowledge extraction model based on deep learning technology can improve the prediction performance of each knowledge extraction model. Furthermore, each knowledge extraction model is trained using document elements corresponding to its matching type, enabling targeted training of each knowledge extraction model, thereby improving the prediction performance of each knowledge extraction model and ultimately enhancing the accuracy and reliability of document knowledge extraction.

[0235] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0236] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A model training method, the method comprising: At least one sample document is obtained from the sample set, and the sample document is parsed to determine the level to which each element in the sample document belongs. The elements in the sample document include title elements, chapter elements, table elements, body elements, and key-value elements. The title element belongs to the first level in the sample document, the chapter element belongs to the second level in the sample document, and the body element, the table element, or the key-value element belongs to the third level in the sample document. A document tree is generated based on the level to which each element belongs; wherein the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the sample document; For each node in the document tree, based on the type of the element indicated by the node, a target knowledge extraction model matching the type is queried. Specifically, when the element indicated by the node is a title element or a chapter element, the target knowledge extraction model is a model for extracting S or SP from the element; when the element indicated by the node is a body text element, the target knowledge extraction model is a model for extracting SPO from the element; when the element indicated by the node is a table element, the target knowledge extraction model is a model for extracting P, PO, or SPO from the element; and when the element indicated by the node is a key-value element, the target knowledge extraction model is a model for extracting PO from the element. The target knowledge extraction model is used to extract knowledge from the elements indicated by the node to obtain predicted knowledge; The target knowledge extraction model is trained based on the difference between the predicted knowledge and the labeled knowledge corresponding to the element indicated by the node on the sample document.

2. The method according to claim 1, wherein, After obtaining at least one sample document from the sample set, the method further includes: The sample document is subjected to data augmentation processing to obtain an expanded sample; The sample set is updated using the expanded samples; The data augmentation process includes at least one of the following processes: At least one word in the sample document is replaced with a synonym; At least one word in the sample document is replaced with a word of the same type; At least one word in the sample document is shuffled. Based on the set knowledge and at least one source statement in the sample document, a target statement is generated, and the target statement is used to replace the source statement in the sample document.

3. The method according to claim 2, wherein, The data augmentation process also includes: Translate the sample document, which is in the first language, into translated text in the second language; The translated text is then translated to obtain a back-translated text in the first language; The sample document is updated using the translated text.

4. The method according to claim 2, wherein, The labeled knowledge includes each entity word and the entity label corresponding to each entity word; The data augmentation process also includes at least one of the following processes: Identify at least two entity words with the same entity tag in the sample document, and replace the at least two entity words. The target entity tag that is the same as the set entity tag is determined from the sample document, and the entity word corresponding to the set entity tag is used to replace the entity word corresponding to the target entity tag in the sample document.

5. The method according to claim 4, wherein, The annotation knowledge also includes the entity relationships between the entity words and the relationship tags corresponding to the entity relationships; The data augmentation process also includes at least one of the following processes: Identify at least two entity relationships with the same relation tag in the sample document, and replace the at least two entity relationships; From the sample document, determine the target relationship label that is the same as the set relationship label, and use the entity relationship corresponding to the set relationship label to replace the entity relationship corresponding to the target relationship label in the sample document.

6. The method according to any one of claims 1-5, wherein, After obtaining at least one sample document from the sample set, the method further includes: Obtain the document format of the sample document; Determine whether the document format is the set format; If the document format is not the set format, the sample document is converted to obtain a sample document in the set format.

7. The method according to any one of claims 1-5, wherein, The method further includes: Obtain the target document to be annotated; Based on the aforementioned target knowledge extraction models, knowledge is extracted from the target document to obtain target knowledge; In response to the update operation for the target knowledge, the target knowledge is updated, and the target document is annotated using the updated target knowledge; The target knowledge extraction models are retrained using the labeled target documents.

8. A knowledge extraction method, the method comprising: Obtain the document to be recognized; The document to be identified is parsed to determine the level to which each element in the document belongs. A document tree is generated based on the level to which each element belongs; wherein the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the document to be identified; For each node in the document tree, based on the type of the element indicated by the node, a target knowledge extraction model matching the type is queried, wherein the target knowledge extraction model is obtained using the model training method as described in any one of claims 1-7; The target knowledge is extracted from the elements indicated by the node using the target knowledge extraction model.

9. A model training apparatus, the apparatus comprising: The parsing module is used to obtain at least one sample document from the sample set and parse the sample document to determine the level to which each element in the sample document belongs. The elements in the sample document include title elements, chapter elements, table elements, body elements, and key-value elements. The title element belongs to the first level in the sample document, the chapter element belongs to the second level in the sample document, and the body element, the table element, or the key-value element belongs to the third level in the sample document. A generation module is used to generate a document tree according to the level to which each element belongs; wherein, the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the sample document; The query module is used to query a target knowledge extraction model that matches the type of the element indicated by the node for each node in the document tree. Specifically, when the element indicated by the node is a title element or a chapter element, the target knowledge extraction model is a model for extracting S or SP from the element; when the element indicated by the node is a body text element, the target knowledge extraction model is a model for extracting SPO from the element; when the element indicated by the node is a table element, the target knowledge extraction model is a model for extracting P, PO, or SPO from the element; and when the element indicated by the node is a key-value element, the target knowledge extraction model is a model for extracting PO from the element. An extraction module is used to extract knowledge from the elements indicated by the node using the target knowledge extraction model to obtain predicted knowledge. The training module is used to train the target knowledge extraction model based on the difference between the predicted knowledge and the labeled knowledge corresponding to the element indicated by the node on the sample document.

10. The apparatus according to claim 9, wherein, The device further includes: The enhancement module is used to perform data augmentation processing on the sample document to obtain an expanded sample; The update module is used to update the sample set using the expanded samples; Specifically, the enhancement module is used to perform at least one of the following: At least one word in the sample document is replaced with a synonym; At least one word in the sample document is replaced with a word of the same type; At least one word in the sample document is shuffled. Based on the set knowledge and at least one source statement in the sample document, a target statement is generated, and the target statement is used to replace the source statement in the sample document.

11. The apparatus according to claim 10, wherein, The enhancement module is also used for: Translate the sample document, which is in the first language, into translated text in the second language; The translated text is then translated to obtain a back-translated text in the first language; The sample document is updated using the translated text.

12. The apparatus according to claim 10, wherein, The labeled knowledge includes each entity word and the entity label corresponding to each entity word; The enhancement module is also configured to perform at least one of the following: Identify at least two entity words with the same entity tag in the sample document, and replace the at least two entity words. The target entity tag that is the same as the set entity tag is determined from the sample document, and the entity word corresponding to the set entity tag is used to replace the entity word corresponding to the target entity tag in the sample document.

13. The apparatus according to claim 12, wherein, The annotation knowledge also includes the entity relationships between the entity words and the relationship tags corresponding to the entity relationships; The enhancement module is also configured to perform at least one of the following: Identify at least two entity relationships with the same relation tag in the sample document, and replace the at least two entity relationships; From the sample document, determine the target relationship label that is the same as the set relationship label, and use the entity relationship corresponding to the set relationship label to replace the entity relationship corresponding to the target relationship label in the sample document.

14. The apparatus according to any one of claims 9-13, wherein, The device further includes: The first acquisition module is used to acquire the document format of the sample document; The judgment module is used to determine whether the document format is a set format; The conversion module is used to convert the format of the sample document to obtain the sample document in the set format when the document format is not the set format.

15. The apparatus according to any one of claims 9-13, wherein, The device further includes: The second acquisition module is used to acquire the target document to be annotated. The extraction module is further configured to extract knowledge from the target document based on each of the target knowledge extraction models to obtain target knowledge; An update module is used to update the target knowledge in response to an update operation on the target knowledge; The annotation module is used to annotate the target document using the updated target knowledge; The training module is also used to retrain each of the target knowledge extraction models using the labeled target documents.

16. A knowledge extraction device, the device comprising: The acquisition module is used to acquire the document to be recognized; The parsing module is used to parse the document to be identified in order to determine the level to which each element in the document belongs. A generation module is used to generate a document tree according to the level to which each element belongs; wherein, the document tree includes nodes at each level, and each node at each level is used to indicate the element at the corresponding level in the document to be identified; The query module is used to query a target knowledge extraction model that matches the type of the element indicated by the node for each node in the document tree, wherein the target knowledge extraction model is obtained by the model training method as described in any one of claims 1-7; The extraction module is used to extract knowledge from the elements indicated by the node using the target knowledge extraction model to obtain target knowledge.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the model training method of any one of claims 1-7, or to perform the knowledge extraction method of claim 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the model training method of any one of claims 1-7, or to perform the knowledge extraction method of claim 8.

19. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the model training method of any one of claims 1-7, or implements the steps of the knowledge extraction method of claim 8.

Images