Knowledge extraction processing method, device and equipment

Abstract

The invention provides a knowledge extraction processing method, device and equipment, and relates to the technical field of information and the petroleum industry technology, and the method comprises the steps: obtaining a knowledge extraction request; obtaining a to-be-extracted text according to the knowledge extraction request; wherein the to-be-extracted text comprises terms and paraphrases of the terms, and the paraphrases are composed of a plurality of entities; the knowledge extraction request is used for determining terms of the to-be-extracted text, paraphrases of the terms and entities in the paraphrases. According to a preset concept extraction model, performing concept extraction processing on the to-be-extracted text to obtain terms and paraphrases of the terms; wherein the concept extraction model is obtained by training according to the marked concept extraction data set. According to a preset entity extraction model, performing named entity extraction on the paraphrases to obtain a plurality of entities; wherein the entity extraction model is obtained by training according to the marked entity extraction data set. According to the method provided by the invention, the technical problem that the knowledge extraction difficulty of the complex text is relatively high is solved.

Description

Technical Field

[0001] The present application relates to the field of information technology and petroleum industry technology, and in particular to a method, device and equipment for extracting and processing knowledge. Background Art

[0002] At present, with the rapid development of artificial intelligence, knowledge extraction has become more and more convenient. Knowledge extraction refers to extracting structured knowledge or information from text. This knowledge can be concepts (such as terms and corresponding interpretations), entities (such as names of people, places, and organizations), relationships (such as relationships between people, and relationships between entities and attributes), etc. Knowledge extraction plays a key role in many NLP tasks, including question-answering systems, information retrieval, text summarization, semantic analysis, etc.

[0003] In the prior art, knowledge extraction is usually performed through a single concept extraction or named entity recognition method. For example, the concept extraction or named entity recognition method is based on a preset extraction type, and directly performs target extraction on the text to obtain target information.

[0004] However, in the prior art, the relevant texts in the field of drilling and completion contain a wealth of drilling and completion terminology and corresponding term definitions. For example, in the text "Low-toxic oil mud means an oil-based drilling fluid (mud) prepared with low-toxic or non-toxic emulsifiers and wetting agents and low-toxic mineral oil.", there is the term "low-toxic oil mud", and its term definition is "an oil-based drilling fluid (mud) prepared with low-toxic or non-toxic emulsifiers and wetting agents and low-toxic mineral oil." In this term definition, drilling and completion related entities such as "emulsifier", "wetting agent", and "oil-based drilling fluid" can be extracted. Therefore, for this kind of progressive knowledge entity, it is impossible to extract knowledge through a single concept extraction or named entity recognition method, which makes it difficult to extract knowledge from texts in complex fields such as the drilling and completion field. Summary of the invention

[0005] The present application provides a knowledge extraction processing method, device and equipment to solve the technical problem of the high difficulty of knowledge extraction from complex texts.

[0006] In a first aspect, the present application provides a method for extracting knowledge, comprising:

[0007] Obtain a knowledge extraction request; and according to the knowledge extraction request, obtain a text to be extracted; wherein the text to be extracted includes a term and an interpretation of the term, and the interpretation is composed of a plurality of entities; the knowledge extraction request is used to determine the term of the text to be extracted, the interpretation of the term, and the entities in the interpretation;

[0008] According to a preset concept extraction model, the text to be extracted is subjected to concept extraction processing to obtain terms and definitions of the terms; wherein the concept extraction model is obtained by training based on a labeled concept extraction data set;

[0009] According to a preset entity extraction model, named entity extraction is performed on the interpretation to obtain multiple entities; wherein the entity extraction model is trained based on a labeled entity extraction data set.

[0010] Furthermore, the process of performing concept extraction on the text to be extracted according to a preset concept extraction model to obtain terms and definitions of the terms includes:

[0011] Obtaining word vectors in the text to be extracted through a first pre-trained model in a preset concept extraction model;

[0012] Input the word vector into a neural network in a preset concept extraction model to learn context information and obtain target data; and decode the target data to output a concept label sequence; wherein the concept label sequence includes a concept label of a term and a concept label corresponding to the interpretation of the term;

[0013] Determine the term corresponding to each concept tag in the concept tag sequence and the interpretation of the term.

[0014] Furthermore, the explanation is subjected to named entity extraction according to a preset entity extraction model to obtain a plurality of entities, including:

[0015] Obtaining the word vector of the interpretation through a second pre-trained model in a preset entity extraction model;

[0016] Inputting the word vector of each character in the interpretation into a neural network in a preset entity extraction model for context information learning to obtain positive output data and negative output data; and performing weighted summation processing on the forward output data and the negative output data to obtain target output data;

[0017] The target output data is transmitted to the conditional random field model layer in the entity extraction model to obtain an entity tag sequence; wherein the entity tag sequence includes multiple entity tags, and the entity tags represent entities.

[0018] Furthermore, the method further comprises:

[0019] Acquire a data set, and perform error correction on the data set to obtain an error-corrected data set;

[0020] In response to the first annotation operation, a concept extraction data set is obtained; wherein the first annotation operation is to annotate the error-corrected data set according to a preset concept label; the concept extraction data set includes a plurality of terms, definitions, concept labels corresponding to each of the terms, and concept labels corresponding to each of the definitions;

[0021] Training a concept candidate model according to the concept extraction data set to obtain a concept extraction model;

[0022] In response to the second labeling operation, an entity extraction data set is obtained; wherein the second labeling operation is to label the error-corrected data set according to a preset entity label; the entity extraction data set includes multiple entities and an entity label corresponding to each of the entities;

[0023] According to the entity extraction data set, the entity candidate model is trained to obtain an entity extraction model.

[0024] Furthermore, the error correction of the data set to obtain the error-corrected data set includes:

[0025] According to a preset wrong character comparison table and a standard dictionary, the data set is corrected to obtain a corrected data set; wherein the wrong character comparison table is generated by comparing the characters in the preset reference dictionary before conversion and the characters in the reference dictionary after conversion; the standard dictionary is a dictionary obtained by expanding the word vectors of the entries in the reference dictionary;

[0026] The step of correcting the data set according to a preset wrong character comparison table and a standard dictionary to obtain a corrected data set includes:

[0027] Generate a list of wrong characters according to the preset wrong character comparison table;

[0028] For each character in the data set, if it is determined that the character in the data set is in the wrong character list, the character is determined to be a wrong character, and a correct character corresponding to the wrong character is searched in the comparison table;

[0029] Traversing the standard dictionary to obtain a vocabulary list containing the correct word; wherein each word in the vocabulary list contains the correct word;

[0030] For each word in the vocabulary list, if it is determined that the prefix and suffix of the correct word in the vocabulary are the same as the prefix and suffix of the wrong word, the wrong word is changed to a correct word and a corrected data set is generated.

[0031] Furthermore, the step of training the concept candidate model according to the concept extraction data set to obtain the concept extraction model includes:

[0032] Based on the mapping relationship between the terms and the concept tags of the terms, and the mapping relationship between the definitions and the concept tags of the definitions, the concept candidate model is trained according to the concept extraction data set until the concept candidate model converges to obtain a concept extraction model.

[0033] Furthermore, the entity candidate model is trained according to the entity extraction data set to obtain the entity extraction model, including:

[0034] Performing data enhancement on the entity extraction dataset to obtain an enhanced entity extraction dataset;

[0035] Based on the mapping relationship between the preset entity and the entity label, the entity candidate model is trained according to both the entity extraction dataset and the enhanced entity extraction dataset until the entity candidate model converges to obtain an entity extraction model.

[0036] Furthermore, the data enhancement of the entity extraction dataset to obtain an enhanced entity extraction dataset includes:

[0037] In the entity extraction data set, according to the entity label, a sentence set including the entity and a preset expansion scale value are determined;

[0038] For each sentence in the sentence set, determining the number of entities in the sentence and the entity position of each entity;

[0039] According to the expansion scale value, the sentence is copied multiple times to generate an extended sentence set;

[0040] For each sentence in the extended sentence set, determine the similarity between each entity in the sentence and a word in a preset vector knowledge base, and replace the entity at the corresponding entity position with a word whose similarity is greater than a preset similarity threshold to obtain a replaced sentence;

[0041] In the expanded sentence set, a target number of sentences are randomly selected as an enhanced entity extraction data set; wherein the target number is the same as the expansion scale value.

[0042] In a second aspect, the present application provides a knowledge extraction and processing device, comprising:

[0043] The first acquisition module is used to acquire a knowledge extraction request; and acquire the text to be extracted according to the knowledge extraction request; wherein the text to be extracted includes a term and an interpretation of the term, and the interpretation is composed of a plurality of entities; the knowledge extraction request is used to determine the term of the text to be extracted, the interpretation of the term, and the entities in the interpretation;

[0044] A first extraction module is used to perform concept extraction processing on the text to be extracted according to a preset concept extraction model to obtain terms and definitions of the terms; wherein the concept extraction model is trained based on a labeled concept extraction data set;

[0045] The second extraction module is used to perform named entity extraction on the interpretation according to a preset entity extraction model to obtain multiple entities; wherein the entity extraction model is trained based on a labeled entity extraction data set.

[0046] Furthermore, the first extraction module includes:

[0047] A first conversion unit, used for obtaining a word vector in the text to be extracted through a first pre-trained model in a preset concept extraction model;

[0048] A first output unit is used to input the word vector into a neural network in a preset concept extraction model to learn context information and obtain target data; and decode the target data to output a concept label sequence; wherein the concept label sequence includes a concept label of a term and a concept label corresponding to the interpretation of the term;

[0049] Determine the term corresponding to each concept tag in the concept tag sequence and the interpretation of the term.

[0050] Furthermore, the second extraction module includes:

[0051] A second conversion unit is used to obtain the word vector of the interpretation through a second pre-trained model in a preset entity extraction model;

[0052] A second output unit is used to input the word vector of each character in the interpretation into a neural network in a preset entity extraction model for context information learning to obtain positive output data and negative output data; and perform weighted sum processing on the forward output data and the negative output data to obtain target output data;

[0053] The third output unit is used to transmit the target output data to the conditional random field model layer in the entity extraction model to obtain an entity label sequence; wherein the entity label sequence includes multiple entity labels, and the entity labels represent entities.

[0054] Furthermore, the device also includes:

[0055] A second acquisition module is used to acquire a data set;

[0056] An error correction module, used to correct errors in the data set to obtain a corrected data set;

[0057] A first annotation module is used to obtain a concept extraction data set in response to a first annotation operation; wherein the first annotation operation is to annotate the error-corrected data set according to a preset concept label; the concept extraction data set includes a plurality of terms, definitions, concept labels corresponding to each term, and concept labels corresponding to each definition;

[0058] A first training module is used to train a concept candidate model according to the concept extraction data set to obtain a concept extraction model;

[0059] A second labeling module, configured to obtain an entity extraction data set in response to a second labeling operation; wherein the second labeling operation is to label the error-corrected data set according to a preset entity label; the entity extraction data set includes a plurality of entities and an entity label corresponding to each of the entities;

[0060] The second training module is used to train the entity candidate model according to the entity extraction data set to obtain the entity extraction model.

[0061] Furthermore, the error correction module includes:

[0062] The error correction unit is used to correct the data set according to a preset wrong character comparison table and a standard dictionary to obtain a corrected data set; wherein the wrong character comparison table is generated by comparing the characters in the preset reference dictionary before conversion and the characters in the reference dictionary after conversion; the standard dictionary is a dictionary obtained by expanding the word vectors of the entries in the reference dictionary;

[0063] The error correction unit is specifically used for:

[0064] Generate a list of wrong characters according to the preset wrong character comparison table;

[0065] For each character in the data set, if it is determined that the character in the data set is in the wrong character list, the character is determined to be a wrong character, and a correct character corresponding to the wrong character is searched in the comparison table;

[0066] Traversing the standard dictionary to obtain a vocabulary list containing the correct word; wherein each word in the vocabulary list contains the correct word;

[0067] For each word in the vocabulary list, if it is determined that the prefix and suffix of the correct word in the vocabulary are the same as the prefix and suffix of the wrong word, the wrong word is changed to a correct word and a corrected data set is generated.

[0068] Furthermore, the first training module is specifically used for:

[0069] Based on the mapping relationship between the terms and the concept tags of the terms, and the mapping relationship between the definitions and the concept tags of the definitions, the concept candidate model is trained according to the concept extraction data set until the concept candidate model converges to obtain a concept extraction model.

[0070] Furthermore, the second training module includes:

[0071] An enhancement unit, used to perform data enhancement on the entity extraction data set to obtain an enhanced entity extraction data set;

[0072] A training unit is used to train an entity candidate model based on a mapping relationship between a preset entity and the entity label according to both the entity extraction data set and the enhanced entity extraction data set until the entity candidate model converges to obtain an entity extraction model.

[0073] Furthermore, the enhancement unit is specifically used for:

[0074] In the entity extraction data set, according to the entity label, a sentence set including the entity and a preset expansion scale value are determined;

[0075] For each sentence in the sentence set, determining the number of entities in the sentence and the entity position of each entity;

[0076] According to the expansion scale value, the sentence is copied multiple times to generate an extended sentence set;

[0077] For each sentence in the extended sentence set, determine the similarity between each entity in the sentence and a word in a preset vector knowledge base, and replace the entity at the corresponding entity position with a word whose similarity is greater than a preset similarity threshold to obtain a replaced sentence;

[0078] In the expanded sentence set, a target number of sentences are randomly selected as an enhanced entity extraction data set; wherein the target number is the same as the expansion scale value.

[0079] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0080] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0081] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0082] The present application provides a method, device and equipment for extracting knowledge, obtaining a knowledge extraction request; and obtaining a text to be extracted according to the knowledge extraction request; wherein the text to be extracted includes terms and their definitions, and the definitions are composed of multiple entities; the knowledge extraction request is used to determine the terms, the definitions of the terms, and the entities in the definitions of the text to be extracted. According to a preset concept extraction model, the text to be extracted is subjected to concept extraction processing to obtain terms and their definitions; wherein the concept extraction model is obtained by training based on annotated concept extraction data sets. According to a preset entity extraction model, named entity extraction is performed on the definitions to obtain multiple entities; wherein the entity extraction model is obtained by training based on annotated entity extraction data sets. In this scheme, concepts (i.e., terms and their definitions) are first extracted from long texts, and then named entity recognition is performed on the definitions of the terms to extract relevant named entities in the field of drilling and completion, enrich the drilling and completion knowledge base, establish relationships between knowledge points, improve the drilling and completion knowledge graph, and provide data support for drilling and completion knowledge reasoning. Therefore, it is possible to automatically obtain concepts, definitions, and named entities for a given text document, realize the organization and aggregation of knowledge, improve the comprehensibility of the text, sort out the relationships between nouns in the text, save researchers' reading energy, and solve the more difficult technical problem of knowledge extraction from complex texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0084] Figure 1 A flowchart of a knowledge extraction method provided in an embodiment of the present application;

[0085] Figure 2 A flowchart of another method for extracting knowledge provided in an embodiment of the present application;

[0086] Figure 3 A flowchart of another method for extracting knowledge provided in an embodiment of the present application;

[0087] Figure 4 A flowchart of another method for extracting knowledge provided in an embodiment of the present application;

[0088] Figure 5 A flowchart of another method for extracting knowledge provided in an embodiment of the present application;

[0089] Figure 6 A flowchart of another method for extracting and processing knowledge provided in an embodiment of the present application;

[0090] Figure 7 A schematic diagram of the overall process of a knowledge extraction and processing method provided in an embodiment of the present application;

[0091] Figure 8 A schematic diagram of the structure of a knowledge extraction and processing device provided in an embodiment of the present application;

[0092] Fig. 9 A schematic diagram of the structure of another knowledge extraction and processing device provided in an embodiment of the present application;

[0093] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0094] Fig.11 A block diagram of an electronic device provided in an embodiment of the present application.

[0095] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0096] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0097] At present, with the rapid development of artificial intelligence, knowledge extraction has become more and more convenient. Knowledge extraction refers to extracting structured knowledge or information from text. This knowledge can be concepts (such as terms and corresponding interpretations), entities (such as names of people, places, and organizations), relationships (such as relationships between people, and relationships between entities and attributes), etc. Knowledge extraction plays a key role in many NLP tasks, including question-answering systems, information retrieval, text summarization, semantic analysis, etc.

[0098] Concept extraction in knowledge extraction is the extraction of terms and definitions, which extracts the term types in the text and their corresponding definitions. However, most of the research on term extraction is aimed at Western languages ​​such as English. Compared with Western languages, Chinese term extraction has its own characteristics and difficulties, mainly as follows: ① Chinese vocabulary is an open set. No matter how large the dictionary is, it is impossible to exhaust all words, and a large number of new words will continue to appear over time. For example, unregistered words include various proper nouns and some terms, abbreviations, new words, etc. ② For Chinese, there is no space in the text to mark the word boundary, and there is no obvious feature such as capitalization of the first letter to represent a term. The term recognition process often has to be combined with the Chinese word segmentation process, which makes Chinese term extraction more difficult. ③ The expression of Chinese is flexible and changeable, and it is also mixed with a large number of ambiguous phenomena, which often involves semantic level analysis and is difficult to mark correctly.

[0099] Entity extraction in knowledge extraction is also known as named entity recognition. Named entity recognition (NER) refers to the identification of entities with certain special meanings in text. In other words, it is necessary to identify the boundaries of entities in natural text and classify them into a specific category. Named entity recognition is the basis of research in the field of natural language processing, involving the construction of knowledge graphs, syntactic analysis, question-answering systems, machine translation, etc. In some general fields, the research on knowledge extraction is more than that in some professional fields, while for professional fields, due to the uniqueness of Chinese and the specificity of the field, there are fewer studies.

[0100] In one example, knowledge extraction is usually performed through a single concept extraction or named entity recognition method. For example, both concept extraction and named entity recognition methods are based on a preset extraction type, and directly perform target extraction on the text to obtain target information. However, in the prior art, in the relevant texts in the field of drilling and completion, there are rich drilling and completion field terms and corresponding term interpretations. For example, in the text "Low-toxic oil mud means an oil-based drilling fluid (mud) prepared with low-toxic or non-toxic emulsifiers and wetting agents and low-toxic mineral oil.", there is a term "low-toxic oil mud", and its term interpretation is "oil-based drilling fluid (mud) prepared with low-toxic or non-toxic emulsifiers and wetting agents and low-toxic mineral oil." In this term interpretation, drilling and completion related entities such as "emulsifier", "wetting agent", and "oil-based drilling fluid" can be extracted. Therefore, for this type of progressive knowledge entity, it is impossible to extract knowledge through a single concept extraction or named entity recognition method, which makes it difficult to extract knowledge from texts in complex fields such as drilling and completion.

[0101] The present application provides a knowledge extraction and processing method, device and equipment, which are intended to solve the above technical problems in the prior art.

[0102] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0103] Figure 1 A flow chart of a knowledge extraction method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method includes:

[0104] Step 101, obtain a knowledge extraction request; and according to the knowledge extraction request, obtain the text to be extracted; wherein the text to be extracted includes terms and definitions of the terms, and the definitions are composed of multiple entities; the knowledge extraction request is used to determine the terms of the text to be extracted, the definitions of the terms, and the entities in the definitions.

[0105] For example, the execution subject of this embodiment may be an electronic device, or a terminal device, or a knowledge extraction processing device or device, or other devices or equipment that can execute this embodiment, and there is no limitation on this. In this embodiment, the execution subject is introduced as an electronic device.

[0106] First, it is necessary to obtain a knowledge extraction request, which includes the identifier of the text to be extracted. According to the knowledge extraction request, the text to be extracted corresponding to the identifier of the text to be extracted is obtained. Among them, the text to be extracted is a progressive content, including terms and their definitions, and the definitions are composed of multiple entities; the knowledge extraction request is used to extract and determine the terms, definitions of terms, and entities in the definitions of the text to be extracted.

[0107] Step 102: Perform concept extraction processing on the text to be extracted according to a preset concept extraction model to obtain terms and their definitions; wherein the concept extraction model is trained based on a labeled concept extraction data set.

[0108] Exemplarily, the concept extraction model is trained based on an annotated concept extraction data set, which includes multiple terms, concept labels of terms, definitions of terms, and concept labels of definitions. Therefore, the concept extraction model trained based on the annotated concept extraction data set is used to extract terms and definitions from text. In this step, the text to be extracted is output to a preset concept extraction model, and concept extraction processing is performed on the text to be extracted to obtain terms and definitions of terms.

[0109] Step 103: perform named entity extraction on the interpretation according to a preset entity extraction model to obtain multiple entities; wherein the entity extraction model is trained according to a labeled entity extraction data set.

[0110] Exemplarily, the entity extraction model is trained based on a labeled entity extraction data set, and the entity extraction data set includes multiple entities and entity labels of the entities. Therefore, the entity extraction model trained based on the labeled entity extraction data set is used to extract entities in the text. In this step, the interpretation is output to the preset entity extraction model, and the interpretation is further subjected to entity extraction processing to obtain the entity in the interpretation.

[0111] In the embodiment of the present application, a knowledge extraction request is obtained; and according to the knowledge extraction request, a text to be extracted is obtained; wherein the text to be extracted includes terms and their definitions, and the definitions are composed of multiple entities; the knowledge extraction request is used to determine the terms, the definitions of the terms, and the entities in the definitions of the text to be extracted. According to a preset concept extraction model, the text to be extracted is subjected to concept extraction processing to obtain terms and their definitions; wherein the concept extraction model is obtained by training based on annotated concept extraction data sets. According to a preset entity extraction model, named entity extraction is performed on the definitions to obtain multiple entities; wherein the entity extraction model is obtained by training based on annotated entity extraction data sets. In this scheme, concepts (i.e., terms and their definitions) are first extracted from long texts, and then named entity recognition is performed on the definitions of the terms to extract relevant named entities in the field of drilling and completion, enrich the drilling and completion knowledge base, establish relationships between knowledge points, improve the drilling and completion knowledge graph, and provide data support for drilling and completion knowledge reasoning. Therefore, it is possible to automatically obtain concepts, definitions, and named entities for a given text document, realize the organization and aggregation of knowledge, improve the comprehensibility of the text, sort out the relationships between nouns in the text, save researchers' reading energy, and solve the more difficult technical problem of knowledge extraction from complex texts.

[0112] Figure 2 A flowchart of another method for extracting knowledge provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method includes:

[0113] Step 201: Obtain a data set.

[0114] Exemplarily, corpus collection is performed, and the collected corpus is converted to obtain a data set. Specifically, according to the text field to be extracted, the electronic dictionary of the corresponding field is determined. Taking drilling and completion and related fields as an example, multiple electronic dictionaries in drilling and completion and related fields are collected, and the PDF version of the electronic dictionary is converted into an editable text format using an OCR tool. Furthermore, for different text formats of different dictionaries, a matching method of terms and definitions based on regular expressions is proposed. By writing regular expressions, terms and definitions can be matched from the text, and the results can be saved in a TXT file to obtain multiple term-definition pairs. Multiple term-definition pairs are the acquired data sets. Some examples of regular expressions are shown in Table 1:

[0115] Table 1 Regular expression examples

[0116]

[0117] Step 202: Correct the data set to obtain a corrected data set.

[0118] In one example, step 202 includes: correcting a data set according to a preset typo comparison table and a standard dictionary to obtain a corrected data set; wherein the typo comparison table is generated by comparing characters in a preset reference dictionary before conversion and characters in a reference dictionary after conversion; and the standard dictionary is a dictionary obtained by expanding word vectors of entries in the reference dictionary.

[0119] In one example, "correcting a data set according to a preset typo comparison table and a standard dictionary to obtain a corrected data set" includes: generating a typo list according to a preset typo comparison table; for each character in the data set, if it is determined that the character in the data set is in the typo list, determining that the character is a typo, and searching the comparison table for a correct character corresponding to the typo; traversing the standard dictionary to obtain a vocabulary list containing correct characters; wherein each word in the vocabulary list contains a correct character; for each word in the vocabulary list, if it is determined that the prefix and suffix of the correct character in the vocabulary are the same as the prefix and suffix of the typo, changing the typo to a correct character, and generating a corrected data set.

[0120] Exemplarily, due to the existence of some professional terms or rare characters in the drilling and completion field, when converting multiple electronic dictionaries through PDF recognition technology, there will be problems with typos, where one character is misrecognized as another. For example, "矽" is misrecognized as "釸", "烃" is misrecognized as "烧", "液" is misrecognized as "夜", "噻" is misrecognized as "嚏", and so on. When these typos are part of relevant concepts or entities in the drilling and completion field, it will have a greater impact on this type of recognition research. For example, for the entity "drilling fluid" in the drilling and completion field, when "液" is recognized as "夜", we will label it as an entity in the drilling and completion field during annotation, but in fact, it no longer belongs to this type of entity. Therefore, it is necessary to correct the typos in the dataset based on the neighbor correction method of the professional dictionary to ensure the correctness of the corpus text and reduce the impact of the corpus on entity recognition research.

[0121] First, according to the preset reference dictionary, generate a typo comparison table and a standard dictionary. Taking the standard dictionary in the drilling and completion field as an example, based on the neighbor correction method of the professional dictionary, through the standard dictionary in the drilling and completion field, judge the neighbor characters of each character in the converted text to complete the correction. The specific implementation method includes: First, use PDF recognition technology to select one of the multiple reference dictionaries as the reference dictionary, and use PDF recognition technology to identify and obtain all the entries from this reference dictionary as part of the standard dictionary in the drilling and completion field, denoted as Dict1. Second, since using PDF recognition technology will also generate typos, manually compare each of the above recognition results item by item, find the typos among them, and modify them, recording the correct character before conversion and the incorrect character after conversion to form a typo comparison table (the conversion errors of these characters are representative). The typo comparison table is shown in Table 2:

[0122] Table 2 Sample of Typo Comparison Table

[0123]

[0124] Then, expand Dict1 based on the preset word vector technology. Specifically, use the preset word vector conversion model to perform vector conversion on all the entries in Dict1 to obtain the entity word embedding vectors of all the entries in Dict1, denoted as V1 N , where N represents the number of entries in Dict1. Obtain the preset public word library, and denote the Chinese word vectors in the word library as V2 M , where M is the number of word vectors in this word library. Calculate each vector in V1 N with V2 MThe similarity between each vector in Dict1 is calculated, and the top K vectors with a similarity threshold greater than the preset similarity threshold are selected as relevant vectors, and the Chinese words corresponding to these relevant vectors are added to the standard dictionary in the field of drilling and completion. Therefore, the final standard dictionary in the field of drilling and completion contains the entity words in Dict1 and the Chinese words of the relevant vectors similar to each entity word.

[0125] Finally, based on the above comparison table and the standard dictionary in the field of drilling and completion, the data set is corrected, including: generating a list of wrong characters errorTokenList based on the wrong character comparison table. For each character in the data set, determine whether the character is in the list of wrong characters errorTokenList. If it is not in the list of wrong characters errorTokenList, it means that the character is a correct character and no correction is required. If it is in the list of wrong characters errorTokenList, it is determined that the character is a wrong character errorToken, and the correct character correctToken corresponding to the wrong character errorToken is obtained from the wrong character. Traverse all the words in the dictionary of the field of drilling and completion to obtain a word list wordList including the correct character correctToken, and each word in the word list contains the correct character. For each word in the word list wordList, determine whether the prefix and suffix of the correct character correctToken in the word are the same as the prefix and suffix of the wrong character errorToken. If the prefix and suffix are the same, replace the wrong character errorToken with the correct character correctToken to generate a corrected data set.

[0126] Therefore, first of all, a term and explanation matching method based on regular expressions is proposed. By writing regular expressions, terms and explanations are matched from the text. Then a neighbor error correction method based on professional dictionaries is given. By constructing a dictionary in the field of drilling and completion, the neighbor words of each word in the text are judged to complete error correction, which greatly improves the correctness of the corpus.

[0127] Step 203, in response to the first labeling operation, a concept extraction data set is obtained; wherein the first labeling operation is to label the corrected data set according to a preset concept label; the concept extraction data set includes multiple terms, definitions, concept labels corresponding to each term, and concept labels corresponding to each definition.

[0128] For example, the concept extraction in the field of drilling and completion includes two types of tags: terminology and interpretation. As shown in Table 3 below:

[0129] Table 3. Label types of concepts extracted from drilling and completion domain

[0130]

[0131] The physical extraction in the field of drilling and completion includes drilling fluid, drilling reagents, drilling methods, drilling equipment, etc., as shown in Table 4 below:

[0132] Table 4 Examples of entity extraction label types in the drilling and completion domain

[0133]

[0134] In this step, the training corpus for concept extraction is annotated based on concept tags using the BIO annotation method. BIO annotation is to annotate each element as "BX", "IX" or "O". Among them, "BX" means that the current element in a word belongs to category X and the position is the beginning of the word, "IX" means that the current element in a word belongs to category X and the position is the middle or end of the word, and "O" means that it does not belong to any type.

[0135] Therefore, the corrected data set obtained in the corpus collection stage is used as a concept extraction data set. In order to verify the effectiveness of the entity recognition method proposed in this application on a small sample data set, only a preset number of data in the concept extraction data set can be selected for annotation. For example, 100 items are selected for annotation to form an annotated data set, and the annotated data set is divided into a training set and a test set in a ratio of 7:3.

[0136] Step 204: Train the concept candidate model based on the concept extraction data set to obtain a concept extraction model.

[0137] In one example, step 204 includes: based on the mapping relationship between terms and their concept tags, and the mapping relationship between definitions and their concept tags, training the concept candidate model according to the concept extraction data set until the concept candidate model converges to obtain a concept extraction model.

[0138] Exemplarily, the concept extraction model is a model based on XLM-RoBERTa-BiLSTM, that is, the concept extraction model is a combination of the XLM-RoBERTa model and BiLSTM, and there is no limitation on the combined model. The XLM-RoBERTa model is an improved version based on the RoBERTa model. When processing Chinese text, it adopts the same masking strategy as the RoBERTa model, that is, modeling continuous text segments as a complete unit. Due to the use of this masking strategy, the differences in pre-training data and tasks, and the optimization of multilingual processing, XLM-RoBERT a The model can better capture the semantic and contextual information between words. To further improve the performance in drilling domain tasks, you can try fine-tuning XLM-RoBERT aModel parameters and how to add a BiLSTM decoder.

[0139] Specifically, during the model training process, first, through XLM-RoBERT a The model obtains word vectors. XLM-RoBERT a The model is used to capture the contextual information of the input text sequence. The field of drilling and completion involves some specific terms, such as drilling fluid, drill bit, drill pipe, drilling platform, borehole wellbore, drilling fluid circulation system, etc. XLM-RoBERT a The model can learn relevant semantic and contextual information from a large amount of drilling and completion field text by modeling these terms, and provide accurate term extraction results. The input sentence is split into sub-word units, such as "drilling fluid" may be split into three parts: "drill", "well", and "fluid". Then, these sub-word units are mapped to embedding vectors and processed through the Transformer layer. Assuming the input text is "Drilling fluid is a key mud component", its embedding representation is:

[0140] X = [Emb(drill), Emb(well), Emb(fluid), ...Emb(form), Emb(point)] (1)

[0141] The output of the Transformer layer is a context-aware representation H:

[0142] H = TransformerLayers(X) (2)

[0143] Here, H represents the contextual representation of each input word. These representations capture the importance of “drilling fluid” as a term in the sentence and its association with “key mud components”.

[0144] The second step is to learn the bidirectional dependencies between different words through the Bidirectional Long Short-Term Memory (BiLSTM) layer. The BiLSTM layer is used to further process XLM-RoBERT. a The context representation of the model output learns the bidirectional dependencies between words in the sequence. By modeling the context, the dependencies and semantic information between terms can be fully captured. This helps to more accurately identify and extract specific terms in the field of drilling and completion. The outputs of the forward LSTM and the reverse LSTM are and Combined to form a bidirectional context representation h t :

[0145]

[0146] The third step is to calculate the label probability through the Softmax function. Specifically, the output h of BiLSTM t After a fully connected layer and classification using the Softmax function, each word is assigned to the most likely label (e.g. B-TERM, I-TERM, I-TERM). For example, the model predicts "drilling fluid" as a term and uses Softmax to calculate the probability of each label:

[0147]

[0148] in, It is the label probability distribution calculated by the Softmax function, which indicates the probability that the word at time step t belongs to each label. For each word, It will be predicted whether it is part of a term (B-TERM or I-TERM) or an ordinary word (O); exp refers to the exponential function with the real number e (e≈2.71828) as the base; ht is the context representation output from BiLSTM, and each time step t has a corresponding h t For example, when the model processes the term "drilling fluid", h t contains the information of the term and its context; C is the total number of labels, indicating the number of categories in the classification problem; W j is the weight matrix for class j, b j is the corresponding bias vector. For each category j, there are independent weights and bias terms; W is the weight matrix, and b is the bias vector. These two parameters belong to the trainable parameters of the classification layer and are used to convert the output h of BiLSTM t Mapped to different category spaces, that is, different term labels (such as B-TERM, I-TERM, O, etc.).

[0149] The fourth step is to measure the prediction error through the loss function. Specifically, the cross entropy loss function is used to measure the difference between the model's generated sequence (referring to the predicted label sequence predicted during the training process) and the target sequence (referring to the labeled label sequence in the text), and the back propagation algorithm is used to optimize the model parameters. Through an iterative approach, the model is allowed to identify patterns and relationships in the data by learning the data in the training set, continuously optimizing the model's performance and the ability to extract terms and interpretations in the field of oil drilling and completion, and by continuously adjusting internal parameters to minimize the prediction error, thereby improving the fit to the training data.

[0150]

[0151] Where L refers to the cross entropy loss function; n is the length of the input sequence, indicating how many words there are in the sentence; C is the total number of labels, representing the number of categories in the term extraction task, such as B-TERM, I-TERM, and O; yij It refers to the indicator variable of the true label, indicating whether the i-th word is true in category j. It is a binary variable. When word i belongs to category j, y ij =1, otherwise 0; It refers to the probability that the i-th word predicted by the model belongs to category j, which is calculated by the Softmax function.

[0152] Finally, perform model evaluation. Since the test set usually does not participate in the model training process, it provides a data environment independent of the training set to test the generalization ability of the model. Through the performance on the test set, it can be evaluated whether the model can handle new data well. The trained model can be evaluated using the test set, that is, the performance of the test model on unknown data to obtain the model's accuracy, recall rate, harmonic mean, etc. And optimize the model based on the evaluation results, such as adjusting model parameters, increasing the amount of training data, etc., until the concept candidate model converges and obtains the concept extraction model. And the data that passes the test is updated and integrated into the drilling and completion field dictionary.

[0153] Step 205, in response to the second labeling operation, an entity extraction data set is obtained; wherein the second labeling operation is to label the corrected data set according to a preset entity label; the entity extraction data set includes multiple entities and an entity label corresponding to each entity.

[0154] Exemplarily, the training corpus for entity extraction is based on preset entity labels, and the corrected data set is annotated using the BIO annotation method to obtain an entity extraction data set, which includes multiple entities and entity labels corresponding to each entity. Among them, the BIO annotation refers to the annotation method of the concept label, which will not be repeated here.

[0155] Step 206: Train the entity candidate model based on the entity extraction data set to obtain an entity extraction model.

[0156] In one example, step 206 includes: performing data enhancement on the entity extraction dataset to obtain an enhanced entity extraction dataset; based on a mapping relationship between a preset entity and an entity label, training the entity candidate model according to both the entity extraction dataset and the enhanced entity extraction dataset until the entity candidate model converges to obtain an entity extraction model.

[0157] In one example, "performing data enhancement on an entity extraction dataset to obtain an enhanced entity extraction dataset" includes: in the entity extraction dataset, determining a sentence set including entities and a preset expansion scale value according to entity labels; for each sentence in the sentence set, determining the number of entities in the sentence and the entity position of each entity; according to the expansion scale value, copying the sentence multiple times to generate an extended sentence set; for each sentence in the extended sentence set, determining the similarity between each entity in the sentence and a vocabulary in a preset vector knowledge base, and replacing the entity at the corresponding entity position with a vocabulary having a similarity greater than a preset similarity threshold to obtain a replaced sentence; in the extended sentence set, randomly selecting a target number of sentences as an enhanced entity extraction dataset; wherein the target number is the same as the expansion scale value.

[0158] For example, taking the field of drilling and completion as an example, Figure 3 A flowchart of another method for extracting knowledge provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, it includes: based on the definition of terms, obtaining the original data set in the field of drilling and completion, that is, the entity extraction data set; performing data enhancement on the data set to obtain enhanced data in the field of drilling and completion; the entity extraction data set and the enhanced data in the field of drilling and completion together constitute a new data set in the field of drilling and completion; training an entity extraction model based on the new data set in the field of drilling and completion. Since there are few annotated data in the field of drilling and completion, it takes a lot of time to build a data set through a small amount of manual annotation, which only meets the basic requirements of training, resulting in poor training results and many domain entities cannot be correctly identified. The entity extraction data set can be enhanced according to a preset data enhancement algorithm to obtain an enhanced entity extraction data set, wherein the enhancement algorithm is not limited. Then, based on the mapping relationship between the preset entity and the entity label, the entity candidate model is trained according to the entity extraction data set and the enhanced entity extraction data set until the entity candidate model converges to obtain the entity extraction model.

[0159] Furthermore, the data augmentation algorithm introduces an external entity knowledge base, semantically replaces the entities in the external entity knowledge base with the entities in the annotated data, and merges the annotated entity extraction data set with the enhanced data as an enhanced data set for entity recognition in the field of drilling and completion. Specifically, data augmentation is a technology that generates new training samples by performing a series of transformations and perturbations on the original data. These transformations and perturbations can be various operations on the text, such as replacing words, inserting or deleting words, changing the word order, etc. The goal of data augmentation is to expand the scale of the training data set and increase the diversity of samples, thereby improving model performance. Existing data augmentation methods, such as synonym replacement and random deletion, focus more on random transformation of data, resulting in problems such as changes in the meaning of entities in the generated text or no increase in entity diversity. In response to these problems, this application proposes a data enhancement method for the field of drilling and completion. The algorithm flow is as follows: a. Obtain a sentence set SentenceList including entities and a preset expansion scale m. For each sentence Sentence in SentenceList, execute steps bd; b. Determine the number n of entities in Sentence and their positions loc; c. Copy Sentence m*n times to generate an extended sentence set ext_List; d. For each sentence in ext_List, replace the entities in it in sequence according to the semantic similarity according to the entity position loc. Calculation method: Use the preset vector conversion model to convert the entities in the sentence, obtain the entity word embedding vector, and then obtain the vocabulary with high similarity to the entity word embedding vector from the preset knowledge base as the replacement vocabulary. e. Randomly select m sentences from ext_List as enhanced data to obtain an enhanced entity extraction data set.

[0160] For example, suppose a sentence contains n entities in the drilling field, and the data enhancement cardinality is m. The sentence is copied m*n times, and the entities in each original sentence are semantically understood to complete entity replacement. Finally, by setting a random number, m sentences are selected from the total m*n sentences as enhanced data. For example, the text "Use water-based drilling fluid to drill deep wells" contains two entities in the drilling and completion field: "water-based drilling fluid" and "deep well". Table 5 gives a list of replacement words based on the knowledge base. The text is data enhanced, and the data enhancement cardinality is 2. Then the text is copied 2*2=4 times. According to Table 5, the 4 copied texts are replaced by entities. The replacement results are shown in Table 6. Randomly select 2 from Table 6 as enhanced texts. Tables 5 and 6 are as follows:

[0161] Table 5. Some named entity replacement results

[0162]

[0163] Table 6 Enhancement algorithm examples

[0164]

[0165] Furthermore, if Figure 3 As shown in the figure, the entity extraction model is a model based on BERT-WWM-neural network-CRF, that is, the concept extraction model is a combination of BERT-WWM model, neural network and CRF. The neural network can be a BiLSTM layer. There is no limitation on the combined model. The model training process includes: the first step is to convert the input text sequence into a word vector using a preset pre-trained language model. BERT and BERT-WWM are commonly used pre-trained language models. BERT is a character-level random masking strategy. In the input text sequence, a certain proportion of words will be randomly replaced with "[MASK]". For example, in the sentence "There is conglomerate at the bottom of the Guantao Formation in this well, biogenic limestone in the first section of the sand, and basalt in the third section of the sand", BERT will mask some words in "the first section of the sand" and "the third section of the sand", such as "the sand [MASK] section", resulting in a word being masked into multiple characters, affecting the overall representation of the stratigraphic name. BERT-WWM uses a full-word masking strategy, which masks the entire word "Sha Section 1" or "Sha Section 3". This method can better capture the meaning of words in Chinese and other languages, and maintain the integrity of the words. This is especially important for processing Chinese texts, because the word boundaries of professional Chinese words are not as clear as those in English. Masking the entire word at one time can avoid the loss of word meaning due to partial masking. Therefore, BERT-WWM's full-word masking strategy helps the model learn more accurate word-level representations, ultimately enhancing the model's ability to understand professional terms and improving entity recognition results. Given an input sequence X = {x1, x2, ..., x N}, where N is the length of the sequence, we will i Convert them into word fragments, and then input them into the pre-trained BERT-WWM module, finally obtaining the sequence representation R = {r1,r2,…,r N}.

[0166] The second step is to input the word vector into the neural network to learn the context information. N} Input the context information into the neural network to obtain the forward LSTM output and the reverse LSTM output in the neural network, and perform the weighted summation of the forward LSTM output and the reverse LSTM output to obtain the output H. The formula is shown in the following equations (6), (7), and (8):

[0167]

[0168] Therefore, the final output H = {h1,h2,…,h N}∈R N×d, where d represents the dimension of word representation.

[0169] The third step is to predict the output sequence through the CRF layer. Send H to the CRF layer and predict the output sequence Y = {y1,y2,…,y N The score of} can be expressed by formula (9), which is the sum of transition probability and state probability:

[0170]

[0171] Where A represents the transfer matrix, It means that from a state y i Transfer to another state y i+1 The probability of; P represents the output score matrix of BiLSTM, Indicates that when the i-th word is given, each state y i The generation probability of ; S(X,Y) refers to the score of the sequence. Then the normalized exponential function softmax is used to obtain the probability value of the label sequence Y, as shown in formula (10):

[0172]

[0173] Among them, P(Y|X) refers to the probability value, and X refers to the input sequence.

[0174] The fourth step is to calculate the optimal label sequence through a dynamic programming algorithm. The CRF layer takes into account the constraints between labels and can obtain more coherent and reasonable labeling results. For example, a sentence starts with "B-" or "O" instead of "I-"; or "B-label1" is followed by "I-label1" instead of "I-label2" inside an entity of another type of label, etc., which reduces the number of incorrectly predicted labels and improves the recognition accuracy.

[0175] Finally, based on multiple test data sets, the model effect is evaluated from the following two aspects to obtain the final entity extraction model.

[0176] (1) The impact of pre-trained models on model performance. Based on multiple training data sets, the BERT-neural network-CRF and BERT-WWM-neural network-CRF structures were used for training according to the above method, and then the trained models were applied to multiple test data sets to evaluate the model performance, as shown in Table 7 below:

[0177] Table 7 Experimental results of pre-training model

[0178]

[0179] According to Table 7, BERT-WWM-neural network-CRF has the best effect, with an F1 value of 73.09%, which is 0.49% higher than the F1 value of BERT.

[0180] (2) The impact of pre-data enhancement on model performance. For multiple training data sets, different enhancement bases are used to expand the training data sets, and then the expanded data sets are used to train the model; and the trained model is applied to multiple test data sets to evaluate the model performance, as shown in Table 8 below:

[0181] Table 8 Effect of different enhancement bases of EEKR on model performance

[0182]

[0183] According to Table 8, when no data enhancement is performed, the F1 value is 73.09%; when the data enhancement cardinality is 2, the F1 value is 79.85%; when the data enhancement cardinality is 4, the F1 value is 80.86%; when the data enhancement cardinality is 8, the F1 value is 80.72%. When the data enhancement cardinality is 4, the F1 value is increased by 7.77% compared with the case without data enhancement, and the effect is optimal. The effectiveness of this method lies in that it does not destroy the true meaning of the entity, and at the same time it can enrich the entity, meet the data standards in the field of drilling and completion, and effectively solve the problem of data scarcity in the field of drilling and completion. At the same time, according to the experimental results, it can be concluded that when the data enhancement cardinality reaches a certain level, the enhancement effect will decrease.

[0184] Step 207, obtain a knowledge extraction request; and obtain the text to be extracted based on the knowledge extraction request; wherein the text to be extracted includes terms and definitions of the terms, and the definitions are composed of multiple entities; the knowledge extraction request is used to determine the terms of the text to be extracted, the definitions of the terms, and the entities in the definitions.

[0185] For example, this step can be referred to Figure 1 Step 101 in the above is not described in detail.

[0186] Step 208: Obtain word vectors in the text to be extracted through the first pre-trained model in the preset concept extraction model.

[0187] For example, Figure 4 A flowchart of another method for extracting knowledge provided in an embodiment of the present application is shown in FIG. Figure 4As shown, for the dictionary and text to be extracted in the field of drilling and completion, a drilling and completion terminology and interpretation data set, that is, a concept extraction data set, is obtained through regular expressions and annotations, and the data set is integrated into the drilling and completion terminology vocabulary. For details, please refer to step 201, which will not be repeated here. The first pre-trained model is an XLM-RoBERTa model, etc., which is not limited to this. According to the first pre-trained model in the preset concept extraction model, the text to be extracted is input into the XLM-RoBERTa model, and the text to be extracted is encoded according to the encoder in the XLM-RoBERTa model to obtain the converted word vector.

[0188] Step 209: input the word vector into the neural network in the preset concept extraction model to learn the context information and obtain the target data; decode the target data and output a concept label sequence; wherein the concept label sequence includes the concept label of the term and the concept label corresponding to the definition of the term; determine the term and the definition of the term corresponding to each concept label in the concept label sequence.

[0189] For example, Figure 4 As shown, the neural network in the concept extraction model takes the BiLSTM layer as an example, inputs the word vector into the BiLSTM layer in the preset concept extraction model for context information learning, obtains the forward output data of the forward LSTM in the BiLSTM layer and the reverse output data of the reverse LSTM, and performs weighted summation processing on the forward output data and the reverse output data to obtain the target data output by the BiLSTM layer. And according to the multi-layer Transformer decoder in the concept extraction model, the target data is decoded, and the concept label sequence is gradually generated and output, which is extracted into a dictionary word library format, and the term and the interpretation of each concept label in the concept label sequence are determined, and the drilling and completion term library is updated; wherein, the concept label sequence includes the concept label of the term and the concept label corresponding to the interpretation of the term.

[0190] For example, Figure 5 A flowchart of another method for extracting knowledge provided in an embodiment of the present application is shown in FIG. Figure 5 As shown in the figure, the concept extraction model uses the XLM-RoBERTa model as the basic model, and converts the feature vector into a text vector representation through a multi-layer Transformer encoder; the text vector is input into the decoder of the BiLSTM layer to gradually generate a concept label sequence, and then the term and the term's interpretation are determined according to the concept label in the concept label sequence.

[0191] Step 210: Obtain the word vector of the interpretation through the second pre-trained model in the preset entity extraction model.

[0192] For example, after obtaining the term and its definition, named entity recognition is performed on the definition of the term. Figure 3 As shown, the second pre-trained model can be a BERT-WWM model, etc., and there is no limitation on this. Based on the second pre-trained model in the preset entity extraction model, the input interpretation, that is, the text sequence, is converted into a word vector. For example, the BERT-WWM model adopts a full-word masking strategy, which will mask the entire word "Sha Section 1" or "Sha Section 3", enhance the model's ability to understand professional terms, and improve entity recognition results.

[0193] Step 211: input the word vector of each character in the interpretation into the neural network in the preset entity extraction model for context information learning to obtain positive output data and negative output data; and perform weighted summation processing on the forward output data and the reverse output data to obtain the target output data.

[0194] For example, Figure 3 As shown, the neural network takes the BiLSTM layer as an example, inputs the word vector of each character in the interpretation into the BiLSTM layer in the preset entity extraction model to learn the context information, obtains the forward output data and the reverse output data, and performs weighted summation processing on the forward output data and the reverse output data to obtain the target output data output by the BiLSTM layer.

[0195] Step 212: transmit the target output data to the conditional random field model layer in the entity extraction model to obtain an entity label sequence; wherein the entity label sequence includes multiple entity labels, and the entity label represents the entity.

[0196] For example, Figure 3 As shown in FIG. 1 , an entity extraction model is used for entity recognition tasks, and the target output data is sent to the conditional random field model layer (ie, CRF layer) in the entity extraction model, and the optimal entity label sequence is calculated through the dynamic programming algorithm in the CRF.

[0197] For example, in vertical fields, a named entity recognition method based on a pre-trained language model is generally used. The entity recognition process of the entity extraction model BERT-WWM-BiLSTM-CRF in this application is as follows: Figure 6 As shown, Figure 6 A flowchart of another method for extracting knowledge provided in an embodiment of the present application, according to Figure 6 It can be seen that through BERT-WWM and BiLSTM, an entity label sequence can be obtained, and then the entity can be determined based on the entity label in the entity label sequence.

[0198] In an embodiment of the present application, a data set is obtained, and the data set is corrected to obtain a corrected data set. In response to a first annotation operation, a concept extraction data set is obtained; wherein the first annotation operation is to annotate the corrected data set according to a preset concept label; the concept extraction data set includes multiple terms, definitions, concept labels corresponding to each term, and concept labels corresponding to each definition. According to the concept extraction data set, a concept candidate model is trained to obtain a concept extraction model. In response to a second annotation operation, an entity extraction data set is obtained; wherein the second annotation operation is to annotate the corrected data set according to a preset entity label; the entity extraction data set includes multiple entities and entity labels corresponding to each entity. According to the entity extraction data set, an entity candidate model is trained to obtain an entity extraction model. A knowledge extraction request is obtained; and according to the knowledge extraction request, a text to be extracted is obtained; wherein the text to be extracted includes terms and definitions of terms, and the definitions are composed of multiple entities; the knowledge extraction request is used to determine the terms, definitions of terms, and entities in the definitions of the text to be extracted. The word vectors in the text to be extracted are obtained through the first pre-trained model. The word vector is input into the neural network in the preset concept extraction model for context information learning to obtain the target data; and the target data is decoded to output a concept label sequence; wherein the concept label sequence includes the concept label of the term and the concept label corresponding to the interpretation of the term. The word vector of the interpretation is obtained through the second pre-training model. The word vector of each character in the interpretation is input into the neural network in the preset entity extraction model for context information learning to obtain the forward output data and the reverse output data; and the forward output data and the reverse output data are weighted summed to obtain the target output data. The target output data is transmitted to the conditional random field model layer in the entity extraction model to obtain the entity label sequence; wherein the entity label sequence includes multiple entity labels, and the entity label represents the entity. Therefore, concepts, interpretations and named entities can be automatically obtained for a given text document, which realizes the organization and aggregation of knowledge, improves the comprehensibility of the text, sorts out the relationship between the nouns in the text, saves the reading energy of researchers, and solves the difficult technical problem of knowledge extraction from complex texts.

[0199] In one example, Figure 7 The overall flow chart of a knowledge extraction method provided in the embodiment of the present application is as follows: Figure 7As shown, taking the drilling and completion field text as an example, for the progressive knowledge existing in the drilling and completion field text, the method includes: performing concept extraction on the drilling and completion field text to obtain drilling and completion field terms and interpretations of drilling and completion field terms; performing named entity recognition on the interpretations to obtain drilling and completion field entities, thereby enriching the knowledge in the drilling and completion field. Specifically, this application includes: data set construction, drilling and completion field knowledge extraction based on concepts and entities, wherein the data set construction includes 4 steps: corpus collection, corpus error correction, label definition, data annotation, etc.; knowledge extraction is divided into two stages: stage one uses XLM-RoBERTa-BiLSTM for concept extraction to extract terms and term interpretations from the text; stage two uses BERT-WWM-BiLSTM-CRF to further extract entities from the term interpretations.

[0200] For example, firstly, concepts (i.e., terms and definitions) are extracted from long texts, and then entities are extracted from the definitions of terms, relationships between knowledge points are established, and the drilling and completion knowledge graph is improved to provide data support for drilling and completion knowledge reasoning. The obtained terms, definitions of terms, and entities in definitions are shown in Table 9 below:

[0201] Table 9

[0202]

[0203] Therefore, according to Table 9, in order to accurately extract the hierarchical knowledge entities existing in the text of drilling and completion, identify and extract terms, definitions of terms, key entities in the definitions, such as geological structure, drilling fluid, formula, etc., and improve the domain knowledge base according to the extraction results to provide knowledge services for intelligent drilling design.

[0204] Figure 8 A schematic diagram of the structure of a knowledge extraction and processing device provided in an embodiment of the present application, such as Figure 8 As shown, the device comprises:

[0205] The first acquisition module 31 is used to obtain a knowledge extraction request; and according to the knowledge extraction request, obtain the text to be extracted; wherein the text to be extracted includes terms and definitions of the terms, and the definitions are composed of multiple entities; the knowledge extraction request is used to determine the terms of the text to be extracted, the definitions of the terms, and the entities in the definitions.

[0206] The first extraction module 32 is used to perform concept extraction processing on the text to be extracted according to a preset concept extraction model to obtain terms and their definitions; wherein the concept extraction model is trained based on a labeled concept extraction data set.

[0207] The second extraction module 33 is used to perform named entity extraction on the interpretation according to a preset entity extraction model to obtain multiple entities; wherein the entity extraction model is trained according to a labeled entity extraction data set.

[0208] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0209] Fig. 9 A schematic diagram of the structure of another knowledge extraction and processing device provided in an embodiment of the present application, in Figure 8 Based on the embodiment shown, Fig. 9 As shown, the first extraction module 32 includes:

[0210] The first conversion unit 321 is used to obtain the word vector in the text to be extracted through the first pre-trained model in the preset concept extraction model.

[0211] The first output unit 322 is used to input the word vector into the neural network in the preset concept extraction model to learn the context information and obtain the target data; and decode the target data and output the concept label sequence; wherein the concept label sequence includes the concept label of the term and the concept label corresponding to the interpretation of the term;

[0212] Determine the term and the definition of each concept tag in the concept tag sequence.

[0213] In one example, the second extraction module 33 includes:

[0214] The second conversion unit 331 is used to obtain the word vector of the interpretation through a second pre-trained model in a preset entity extraction model.

[0215] The second output unit 332 is used to input the word vector of each character in the interpretation into the neural network in the preset entity extraction model for context information learning to obtain positive output data and negative output data; and perform weighted sum processing on the forward output data and the reverse output data to obtain target output data;

[0216] The third output unit 333 is used to transmit the target output data to the conditional random field model layer in the entity extraction model to obtain an entity label sequence; wherein the entity label sequence includes multiple entity labels, and the entity label represents the entity.

[0217] In one example, the device further includes:

[0218] A second acquisition module 41, used to acquire a data set;

[0219] The error correction module 42 is used to correct errors in the data set to obtain a corrected data set.

[0220] The first annotation module 43 is used to obtain a concept extraction data set in response to a first annotation operation; wherein the first annotation operation is to annotate the corrected data set according to a preset concept label; the concept extraction data set includes multiple terms, definitions, concept labels corresponding to each term, and concept labels corresponding to each definition.

[0221] The first training module 44 is used to train the concept candidate model according to the concept extraction data set to obtain the concept extraction model.

[0222] The second labeling module 45 is used to obtain an entity extraction data set in response to a second labeling operation; wherein the second labeling operation is to label the corrected data set according to a preset entity label; the entity extraction data set includes multiple entities and an entity label corresponding to each entity.

[0223] The second training module 46 is used to train the entity candidate model according to the entity extraction data set to obtain the entity extraction model.

[0224] In one example, the error correction module 42 includes:

[0225] The error correction unit 421 is used to correct the data set according to a preset wrong character comparison table and a standard dictionary to obtain a corrected data set; wherein the wrong character comparison table is generated by comparing the characters in the preset reference dictionary before conversion and the characters in the reference dictionary after conversion; the standard dictionary is a dictionary obtained by expanding the word vectors of the entries in the reference dictionary.

[0226] The error correction unit 421 is specifically used for:

[0227] Generate a list of typos based on the preset typo comparison table.

[0228] For each character in the data set, if it is determined that the character in the data set is in the wrong character list, the character is determined to be a wrong character, and the correct character corresponding to the wrong character is searched in the comparison table.

[0229] The standard dictionary is traversed to obtain a vocabulary list containing the correct word; wherein each word in the vocabulary list contains the correct word.

[0230] For each word in the vocabulary list, if it is determined that the prefix and suffix of the correct word in the vocabulary are the same as the prefix and suffix of the wrong word, the wrong word is changed to the correct word and a corrected data set is generated.

[0231] In one example, the first training module 44 is specifically configured to:

[0232] Based on the mapping relationship between terms and their concept labels, and the mapping relationship between definitions and their concept labels, the concept candidate model is trained according to the concept extraction data set until the concept candidate model converges to obtain a concept extraction model.

[0233] In one example, the second training module 46 includes:

[0234] The enhancement unit 461 is used to perform data enhancement on the entity extraction data set to obtain an enhanced entity extraction data set.

[0235] The training unit 462 is used to train the entity candidate model based on the mapping relationship between the preset entity and the entity label, according to both the entity extraction data set and the enhanced entity extraction data set, until the entity candidate model converges to obtain the entity extraction model.

[0236] In one example, the enhancement unit 461 is specifically configured to:

[0237] In the entity extraction data set, according to the entity label, the sentence set including the entity and the preset expansion scale value are determined;

[0238] For each sentence in the sentence set, determine the number of entities in the sentence and the entity position of each entity;

[0239] According to the expansion scale value, the sentence is copied multiple times to generate an expanded sentence set;

[0240] For each sentence in the extended sentence set, determine the similarity between each entity in the sentence and the vocabulary in the preset vector knowledge base, and replace the entity at the corresponding entity position with the vocabulary whose similarity is greater than the preset similarity threshold to obtain the replaced sentence;

[0241] In the expanded sentence set, a target number of sentences are randomly selected as the enhanced entity extraction dataset; where the target number is the same as the expansion scale value.

[0242] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0243] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Fig.10 As shown, the electronic device includes: a memory 51 and a processor 52.

[0244] The memory 51 stores a computer program that can be executed on the processor 52 .

[0245] The processor 52 is configured to execute the method provided in the above embodiment.

[0246] The electronic device further includes a receiver 53 and a transmitter 54. The receiver 53 is used to receive instructions and data sent by an external device, and the transmitter 54 is used to send instructions and data to the external device.

[0247] Fig.11 This is a block diagram of an electronic device provided in an embodiment of the present application. The electronic device may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0248] The device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0249] The processing component 602 generally controls the overall operation of the device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0250] The memory 604 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0251] The power supply component 606 provides power to the various components of the device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 600.

[0252] The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0253] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the device 600 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0254] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0255] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the device 600. For example, the sensor assembly 614 can detect the open / closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, the sensor assembly 614 can also detect the position change of the device 600 or a component of the device 600, the presence or absence of user contact with the device 600, the orientation or acceleration / deceleration of the device 600 and the temperature change of the device 600. The sensor assembly 614 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0256] The communication component 616 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0257] In an exemplary embodiment, the apparatus 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method. In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the above instructions may be executed by a processor 620 of the apparatus 600 to perform the above method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0258] An embodiment of the present application also provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the method provided by the above embodiment.

[0259] An embodiment of the present application also provides a computer program product, which includes: a computer program, which is stored in a readable storage medium, and at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0260] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0261] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for extracting knowledge, characterized in that: include: Get knowledge extraction request; and obtaining a text to be extracted according to the knowledge extraction request; wherein the text to be extracted includes a term and an interpretation of the term, and the interpretation is composed of a plurality of entities; the knowledge extraction request is used to determine the term of the text to be extracted, the interpretation of the term, and the entities in the interpretation; According to a preset concept extraction model, the text to be extracted is subjected to concept extraction processing to obtain terms and definitions of the terms; wherein the concept extraction model is obtained by training based on a labeled concept extraction data set; According to a preset entity extraction model, named entity extraction is performed on the interpretation to obtain multiple entities; wherein the entity extraction model is trained based on a labeled entity extraction data set.

2. The method according to claim 1, characterized in that The method of performing concept extraction processing on the text to be extracted according to a preset concept extraction model to obtain terms and definitions of the terms includes: Obtaining word vectors in the text to be extracted through a first pre-trained model in a preset concept extraction model; Input the word vector into a neural network in a preset concept extraction model to learn context information and obtain target data; and decode the target data to output a concept label sequence; wherein the concept label sequence includes a concept label of a term and a concept label corresponding to the interpretation of the term; Determine the term corresponding to each concept tag in the concept tag sequence and the interpretation of the term.

3. The method according to claim 2, characterized in that The method further comprises performing named entity extraction on the interpretation according to a preset entity extraction model to obtain a plurality of entities, including: Obtaining the word vector of the interpretation through a second pre-trained model in a preset entity extraction model; Inputting the word vector of each character in the interpretation into a neural network in a preset entity extraction model for context information learning to obtain positive output data and negative output data; and performing weighted summation processing on the forward output data and the negative output data to obtain target output data; The target output data is transmitted to the conditional random field model layer in the entity extraction model to obtain an entity tag sequence; wherein the entity tag sequence includes multiple entity tags, and the entity tags represent entities.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Acquire a data set, and perform error correction on the data set to obtain an error-corrected data set; In response to the first annotation operation, a concept extraction data set is obtained; wherein the first annotation operation is to annotate the error-corrected data set according to a preset concept label; the concept extraction data set includes a plurality of terms, definitions, concept labels corresponding to each of the terms, and concept labels corresponding to each of the definitions; Training a concept candidate model according to the concept extraction data set to obtain a concept extraction model; In response to the second labeling operation, an entity extraction data set is obtained; wherein the second labeling operation is to label the error-corrected data set according to a preset entity label; the entity extraction data set includes multiple entities and an entity label corresponding to each of the entities; According to the entity extraction data set, the entity candidate model is trained to obtain an entity extraction model.

5. The method according to claim 4, characterized in that The step of performing error correction on the data set to obtain an error-corrected data set includes: According to a preset wrong character comparison table and a preset standard dictionary, the data set is corrected to obtain a corrected data set; wherein the wrong character comparison table is generated by comparing the characters in the preset reference dictionary before conversion and the characters in the reference dictionary after conversion; the standard dictionary is a dictionary obtained by expanding the word vectors of the entries in the reference dictionary; The step of correcting the data set according to a preset wrong character comparison table and a standard dictionary to obtain a corrected data set includes: Generate a list of wrong characters according to the preset wrong character comparison table; For each character in the data set, if it is determined that the character in the data set is in the wrong character list, the character is determined to be a wrong character, and a correct character corresponding to the wrong character is searched in the comparison table; Traversing the standard dictionary to obtain a vocabulary list containing the correct word; wherein each word in the vocabulary list contains the correct word; For each word in the vocabulary list, if it is determined that the prefix and suffix of the correct word in the vocabulary are the same as the prefix and suffix of the wrong word, the wrong word is changed to a correct word and a corrected data set is generated.

6. The method according to claim 4, characterized in that The step of training the concept candidate model according to the concept extraction data set to obtain the concept extraction model includes: Based on the mapping relationship between the terms and the concept tags of the terms, and the mapping relationship between the definitions and the concept tags of the definitions, the concept candidate model is trained according to the concept extraction data set until the concept candidate model converges to obtain a concept extraction model.

7. The method according to claim 4, characterized in that The step of training the entity candidate model according to the entity extraction data set to obtain the entity extraction model includes: Performing data enhancement on the entity extraction dataset to obtain an enhanced entity extraction dataset; Based on the mapping relationship between the preset entity and the entity label, the entity candidate model is trained according to both the entity extraction dataset and the enhanced entity extraction dataset until the entity candidate model converges to obtain an entity extraction model.

8. The method according to claim 7, characterized in that The step of performing data enhancement on the entity extraction dataset to obtain an enhanced entity extraction dataset includes: In the entity extraction data set, according to the entity label, a sentence set including the entity and a preset expansion scale value are determined; For each sentence in the sentence set, determining the number of entities in the sentence and the entity position of each entity; According to the expansion scale value, the sentence is copied multiple times to generate an extended sentence set; For each sentence in the extended sentence set, determine the similarity between each entity in the sentence and a word in a preset vector knowledge base, and replace the entity at the corresponding entity position with a word whose similarity is greater than a preset similarity threshold to obtain a replaced sentence; In the expanded sentence set, a target number of sentences are randomly selected as an enhanced entity extraction data set; wherein the target number is the same as the expansion scale value.

9. A knowledge extraction processing device, characterized in that: include: A first acquisition module, used to acquire a knowledge extraction request; and obtaining a text to be extracted according to the knowledge extraction request; wherein the text to be extracted includes a term and an interpretation of the term, and the interpretation is composed of a plurality of entities; the knowledge extraction request is used to determine the term of the text to be extracted, the interpretation of the term, and the entities in the interpretation; A first extraction module is used to perform concept extraction processing on the text to be extracted according to a preset concept extraction model to obtain terms and definitions of the terms; wherein the concept extraction model is trained based on a labeled concept extraction data set; The second extraction module is used to perform named entity extraction on the interpretation according to a preset entity extraction model to obtain multiple entities; wherein the entity extraction model is trained based on a labeled entity extraction data set.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images