Knowledge extraction method, device and computer storage medium

Abstract

The application provides a knowledge extraction method, device and computer storage medium. In an embodiment, a plurality of documents are acquired; a first file is acquired, the first file including content extraction rules, entity extraction rules of a plurality of entity types and entity relationship extraction rules of the plurality of entity types; first information using second extraction rules is determined from the plurality of documents based on the content extraction rules; entity extraction is performed on the first information based on the entity extraction rules to determine an entity extraction result; wherein the entity extraction result includes a plurality of entities and attribute information corresponding to each of the plurality of entities, the attribute information including any entity type in the plurality of entity types; relationships between part or all of the plurality of entities are established based on the entity relationship extraction rules and the attribute information corresponding to each of the plurality of entities to determine an entity relationship extraction result. The rule definition can be patterned according to the entity type, the extraction model framework is unified, and the extraction efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, and particularly relates to a knowledge extraction method, device and computer storage medium. BACKGROUND

[0002] With the increasing demand for information intelligentization in today's society, providing intelligent information is a common problem that needs to be considered at present. The current mainstream of document intelligentization is to reconstruct knowledge of text by using a knowledge graph. The first step of constructing a knowledge graph for a document is to extract document data, which requires extracting knowledge of a triple from a large amount of text. Here, the triple is entity-relation-entity.

[0003] At present, a data extraction feature is defined for a type of text. A tool (for example, python) is used to define a batch processing script design extraction scheme for a type of data, so as to realize data extraction of this type of text. If a new type of text is used, the data extraction feature needs to be redefined. The tool needs to be reprogrammed or modified to adapt to the new data extraction feature.

[0004] The above scheme identifies features for a type of text, and a tool script is newly defined to extract this type of data. Each time a new type of feature data is extracted, the features need to be re-identified, and personnel need to be reprogrammed to write a tool script, which is low in extraction efficiency. SUMMARY

[0005] Embodiments of the present application provide a knowledge extraction method, device and computer storage medium, which can define entity extraction rules and entity relationship extraction rules according to entity types, unify an extraction model framework, and improve extraction efficiency.

[0006] In a first aspect, embodiments of the present application provide a knowledge extraction method, including: obtaining a plurality of documents; obtaining a first file; wherein the first file includes content extraction rules, entity extraction rules of a plurality of entity types, and entity relationship extraction rules of the plurality of entity types; determining first information using the entity extraction rules from the plurality of documents based on the content extraction rules; performing entity extraction on the first information based on the entity extraction rules to determine an entity extraction result; wherein the entity extraction result includes a plurality of entities and attribute information corresponding to each of the plurality of entities, and the attribute information includes any entity type in the plurality of entity types; establishing relationships between part or all of the plurality of entities based on the entity relationship extraction rules and the attribute information corresponding to each of the plurality of entities to determine an entity relationship extraction result.

[0007] In the present scheme, the rules can be defined according to the entity types, the extraction model framework is unified, and the extraction efficiency is improved.

[0008] In a possible implementation, the relation extraction rule includes a plurality of relation rules, the relation rules indicate that entities of two entity types in the plurality of entity types have a first relation when a first condition is met; and the establishing of the relations between part or all of the plurality of entities based on the entity relation extraction rule and the attribute information corresponding to each of the plurality of entities includes: matching the plurality of relation rules based on the attribute information corresponding to each of the plurality of entities, to establish the relations between part or all of the plurality of entities.

[0009] In a possible implementation, the plurality of entities include at least one first entity and at least one second entity, and the at least one second entity is extracted from the attribute information of the at least one first entity.

[0010] In a possible implementation, the attribute information corresponding to the first entity includes entity meaning.

[0011] In a possible implementation, the plurality of documents include first page navigation information, and the first page navigation information includes part or all of the plurality of entities; and the establishing of the relations between part or all of the plurality of entities based on the entity relation extraction rule and the attribute information corresponding to each of the plurality of entities includes: establishing the relations between part or all of the plurality of entities based on the entity relation extraction rule, the attribute information corresponding to each of the plurality of entities, and the first page navigation information.

[0012] In this scheme, the attribute information of the entities and the page navigation information are comprehensively considered, so that the relations between the plurality of entities can be determined more accurately and comprehensively.

[0013] In a possible implementation, the plurality of documents include documents that meet the HyperText Markup Language.

[0014] In a possible implementation, the content extraction rule includes at least one of the following parameters: a first document information type, a first page information type, a navigation level, and at least one entity attribute.

[0015] In a possible implementation, the first file is a configuration file.

[0016] In a second aspect, an embodiment of the present application provides a knowledge extraction apparatus, including:

[0017] a document acquisition module, configured to acquire a plurality of documents;

[0018] a rule acquisition module, configured to acquire a first file; the first file includes a content extraction rule, an entity extraction rule of a plurality of entity types, and an entity relation extraction rule of the plurality of entity types.

[0019] The screening module is configured to determine, based on the content extraction rule, first information from the plurality of documents by using the entity extraction rule;

[0020] The entity extraction module is configured to perform entity extraction on the first information based on the entity extraction rule to determine an entity extraction result, wherein the entity extraction result comprises a plurality of entities and attribute information corresponding to each of the plurality of entities, and the attribute information comprises any entity type in the plurality of entity types;

[0021] The relationship extraction module is configured to establish relationships between part or all of the plurality of entities based on the entity relationship extraction rule and the attribute information corresponding to each of the plurality of entities to determine an entity relationship extraction result.

[0022] The beneficial effects of the present scheme are described above and will not be repeated here.

[0023] In a possible implementation, the relationship extraction rule comprises a plurality of relationship rules, the relationship rules indicate that entities of two entity types in the plurality of entity types have a first relationship when a first condition is met; and the relationship extraction module is configured to match the plurality of relationship rules based on the attribute information corresponding to each of the plurality of entities to establish relationships between part or all of the plurality of entities.

[0024] In a possible implementation, the plurality of entities comprises at least one first entity and at least one second entity, and the at least one second entity is extracted from attribute information of the at least one first entity.

[0025] In a possible implementation, the attribute information corresponding to the first entity comprises entity meaning.

[0026] In a possible implementation, the plurality of documents comprises first page navigation information, and the first page navigation information comprises part or all of the plurality of entities; and the relationship extraction module is configured to establish relationships between part or all of the plurality of entities based on the entity relationship extraction rule, the attribute information corresponding to each of the plurality of entities, and the first page navigation information.

[0027] In a possible implementation, the plurality of documents comprises documents satisfying a HyperText Markup Language.

[0028] In a possible implementation, the content extraction rule comprises at least one of the following parameters: a first document information type, a first page information type, a navigation level, and at least one entity attribute.

[0029] In a possible implementation, the first file is a configuration file.

[0030] In a third aspect, an embodiment of the present application provides a knowledge extraction device, characterized in that the device runs computer program instructions to execute the method provided in the first aspect. Exemplarily, the device can be a chip or a processor.

[0031] In one example, the apparatus may include a processor, which may be coupled to a memory, read instructions from the memory, and perform the method provided in the first aspect according to the instructions. The memory may be integrated into a chip or processor, or may be independent of the chip or processor.

[0032] In a fourth aspect, an embodiment of the present application provides a knowledge extraction device, comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method provided in the first aspect.

[0033] In a fifth aspect, an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method provided in the first aspect.

[0034] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of a knowledge extraction method provided in an embodiment of the present application;

[0036] Figure 2a This is a schematic diagram of the structure of a document provided in an embodiment of the present application. Figure 1 ;

[0037] Figure 2b This is a second structural diagram of a document provided in an embodiment of the present application;

[0038] Figure 3a is a schematic diagram of content filtering provided by an embodiment of the present application;

[0039] Figure 3b This is a schematic diagram of the structure of entity extraction provided by an embodiment of the present application;

[0040] Figure 3c Schematic diagram of entity relationship extraction provided by an embodiment of the present application;

[0041] Figure 4 This is a schematic diagram of the content of a first information provided in an embodiment of the present application;

[0042] Figure 5is a structural schematic diagram of a knowledge extraction device provided by an embodiment of the present application.

[0043] Figure 6 is a structural schematic diagram of a knowledge extraction device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below with reference to the drawings.

[0045] In the description of the embodiments of the present application, the words "exemplary", "for example", or "for instance" are used to mean serving as an example, instance or illustration. Any embodiment or design solution described as "exemplary", "for example" or "for instance" in the embodiments of the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design solutions. In fact, the words "exemplary", "for example" or "for instance" are used in the sense of presenting a particular concept in a concrete manner.

[0046] In the description of the embodiments of the present application, the term "and / or" merely describes an association relationship of associated objects, and means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, B alone, and A and B simultaneously. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems mean two or more systems, and multiple terminals mean two or more terminals.

[0047] In addition, the terms "first", "second" are used for description purposes only, and should not be interpreted or implied to indicate or imply relative importance or implicitly indicate the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more features. The terms "include", "contain", "have" and their variants mean "include but are not limited to", unless otherwise specifically emphasized.

[0048] With the increasing demand for information intelligence in today's society, providing intelligent information is a common problem that needs to be considered at present. The current mainstream of document intelligence is to use the knowledge graph to reconstruct the knowledge of the text. The first step of constructing the knowledge graph for the document is to extract the document data, which needs to extract the information in a large amount of text into the "entity-relation-entity" triple knowledge. Among them, knowledge extraction can be understood as a process of extracting each knowledge point in the document into a certain form into the knowledge base by recognizing, understanding, screening and formatting. The purpose is to enhance the usability and reusability of information, and this process can also be regarded as a semantic annotation process for existing unstructured information.

[0049] Currently, knowledge extraction mainly targets HTML (Hyper Text Markup Language) document types, and there are two main extraction schemes.

[0050] Solution 1: Use tools (such as Python) to define a batch script design extraction plan for a type of data to extract information with a certain type of text features, and realize data extraction of this type of text.

[0051] For solution 1, for feature recognition of a certain type of feature, it is necessary to redefine the script through the tool to extract this type of data, that is, it is necessary to re-employ personnel to write tool scripts, which has the disadvantages of low efficiency and high investment.

[0052] Solution 2: By using machine learning and model training, text information is extracted according to the defined algorithm, and the extraction results are continuously trained and optimized.

[0053] Solution 2 is mainly suitable for unstructured data extraction scenarios, and requires the design of extraction algorithms and a large amount of model training. Usually, the investment cost is high and the extraction accuracy will decrease.

[0054] In order to solve the above problems, this application uses a method of extracting text by defining extraction rules in a pattern, unifying the extraction model framework, and defining unified grammatical rules for the features of such texts to support category-by-category and rule-based extraction.

[0055] In practical applications, this extraction method is carried out through a configuration file, and the extraction tool is used to implement the document-based knowledge extraction of this logic, thereby realizing the extraction of triple data from the graph.

[0056] Figure 1 This is a flow chart of the knowledge extraction method provided by the embodiment of the present application. This embodiment can be applied to electronic devices, specifically to servers or general computers. Figure 1 As shown, the knowledge extraction method provided in the embodiment of the present application includes at least the following steps:

[0057] Step 101: Determine multiple documents.

[0058] In the embodiment of this application, Figure 2a and Figure 2bAs shown, the document includes a metadata file and at least one page. In addition, the document also includes a plurality of components and page navigation information. For example, the components can be used to navigate to the pages, and the pages can be navigated to each other, and the page navigation information indicates the hierarchical link relationship between the components and the pages. For example, the document can be a document of a website, and the document can include a plurality of components, and each component can link to a page, and the page can link to other pages. For example, as shown in FIG. 1, the document includes a metadata file and a plurality of components, and each component links to a page, and the pages can be navigated to each other. Figure 2b As shown, the page navigation information is a navigation tree, and the navigation tree includes a root node, a child node and a leaf node, wherein the root node is the topmost node of the tree, and is usually a component, the child node is a node other than the root node, and has a node connected below it, and can be a component or a page, and the leaf node is a node that no longer has a node connected to it (i.e. the end), and is usually a page.

[0059] The metadata file can include document metadata, page metadata and component metadata.

[0060] Here, the document metadata includes document description information, and is used for the source of the document, the type of information recorded by the document (for the sake of description and distinction, it can be referred to as document information type), and the technical field covered by the information recorded by the document. For example, assuming that the document is used to describe a website, the source of the document can include an overview of the website of a company, such as which company developed it, which department developed and designed it, the knowledge field it belongs to such as network technology, what platform it belongs to, and what use it is for such as introduction, selling products of the company, and information for establishing cooperation with the company. In addition, the type of information recorded by the document can be alarm type, operation type and measurement type. The document of the alarm type is used to describe the alarm conditions of various products, the document of the operation type can be used to describe the operation process of the product and other operation-related information such as product manual. The document of the measurement type is used to describe the measurement indicators of various products, and is generally used to implement monitoring of the product. For example, if the product is software, the document of the measurement type is used to record how to monitor the software and what indicators need to be monitored. In addition, the technical field covered by the information recorded by the document can be the communication field, the terminal field, the sensor field, etc.

[0061] Here, the page metadata includes page description information, which is used to describe the type of information recorded in the page (for the sake of description and distinction, it can be referred to as page information type) and the technical field to which the information recorded in the page belongs. In an example, the type can be a description type, such as a type used to describe a product of a company; in an example, the type can be a reference type, such as a type used to describe a successful case of a company for users to refer to, or a type used to describe a cooperative company of the company for users to refer to. It should be noted that the type of information recorded in the page belongs to the type of information recorded in the document. It should be noted that the technical field in the document description information is a general description of the technical field to which the information of the entire document belongs, while the technical field in the page description information is limited to the information recorded in the page, and generally, one page only describes the knowledge of one technical field.

[0062] Further, the page metadata can further include an identifier such as an ID of the page and a uniform resource locator (URL) of the page.

[0063] Here, the component metadata includes an identifier such as an ID of the component.

[0064] In the embodiments of the present application, the format of each page of one document is generally the same, such as pages in an Html format. The document is often a complete file, such as an article. The embodiments of the present application are described by taking an HTML document as an example. In order to facilitate the understanding of the following steps, the syntax structure of HTML is described as follows. Figure 2b As shown in the following table, the syntax structure of HTML is as follows:

[0065]

[0066] The <!DOCTYPE> tag is located at the very beginning of the document, and is used to indicate which HTML or XHTML standard specification is used for the current document. The <!DOCTYPE> tag must be used at the beginning to specify the XHTML version and type for all XHTML documents, so that the web page can be regarded as a valid XHTML document and parsed according to the specified standard specification.

[0067] The tag is used to define the header information of the HTML document, also known as the header tag, and is mainly used to encapsulate other tags located in the header of the document, such as the "document title" in the following table. <title>, <meta>, <link> and <style> are used to describe the title, author and relationship with other documents, etc. An HTML document can only contain a pair of <head> tags, and most of the data in the document header will not be actually displayed on the page.

[0068] Among them, the <body> tag is used to define the content to be displayed in the HTML document, also known as the body tag. All text, images, audio and video information displayed in the browser must be located within the <body> tag, and the information in the <body> tag is the final display to the user.

[0069] It should be noted that an HTML page is composed of multiple <tag> content < / tag> tags, including many tags, and each tag has its own function in actual application. For example, commonly used tags include: partition display, <link> link an external style sheet, <h1>-<h6> title, hyperlink, picture, <buton> button, <input> input box, line break, paragraph, <center> center, unordered list, ordered list, <hr> horizontal dividing line, table. Usually HTML is composed of multiple <tag> content < / tag> tags. For example, <h1> title < / h1> or hyperlink , where <h1>< / h1> and are tags in HTML.

[0070] In addition, an HTML page can build a DOM tree, which is a finite hierarchical tree structure, and the tree order is determined by the HTML document. There are two kinds of DOM trees, one is an element tree (all tags in the document are considered a tree), and the other is a node tree (all contents in the document are considered a tree). The DOM tree consists of root nodes, child nodes and leaf nodes. The root node is the topmost node of the tree, the child node is a node that has a node below it other than the root node, and the leaf node is a node that no longer has a node (i.e. the end). For example, child nodes mainly have three types: the first is an element node, such as <html>, , <body>, etc., which are element nodes, i.e. tags. The second is a text node, which is the content displayed to the user, such as <title>...< / title> The third is an attribute node, which is an attribute of the element identified by the element node, such as the following table. The href attribute in the tag. In actual applications, the HTML page includes a DOM tree of the head and a DOM tree of the body.

[0071] Step 102, obtaining a first file; wherein the first file includes content extraction rules, entity extraction rules of a plurality of entity types, and entity relationship extraction rules of the plurality of entity types.

[0072] In actual applications, the content extraction rules, the entity extraction rules of the plurality of entity types, and the entity relationship extraction rules of the plurality of entity types can be obtained by reading the first file through an extraction tool.

[0073] The content extraction rules are used to implement content filtering to filter information that can be processed by the entity extraction rules.

[0074] The entity extraction rules indicate what type of entity needs to be extracted, and what attribute value of the attribute of the entity needs to be extracted.

[0075] The entity relationship extraction rules are used to describe rules based on which relationships between entity types are defined.

[0076] In one example, the first file can be a configuration file.

[0077] Step 103, determining first information that can adopt the entity extraction rules from a plurality of documents based on the content extraction rules.

[0078] According to one possible implementation, the content extraction rules are parsed by the extraction tool to perform information extraction on the plurality of documents, and first information that can adopt the second extraction rules is determined from the plurality of documents. The first information can include metadata files of the documents, pages, fragments in the pages, and the like.

[0079] In one example, the content extraction rules can include a plurality of filtering rules, and the plurality of filtering rules can be set from a plurality of dimensions. For example, the plurality of dimensions include dimension 1: document information type, dimension 2: page information type, dimension 3: navigation level, such as child node, leaf node, and the like, dimension 4: content template, and the like. The content template can define a plurality of attributes that the entity needs to have, such as entity and meaning of the entity. In addition, the content template can define a label in which the entity exists, such as <h1> Label.

[0080] For example, as Figure 3a As shown, the filtering rule can be defined from the document metadata, the page navigation information, the DOM tree of the head of the page, and the DOM tree of the body of the page, in combination with the above four dimensions. For the DOM tree of the head, a plurality of tags can be obtained through tag positioning, and then the filtering conditions of the tag screening are set to obtain the required tags. For the DOM tree of the body, a plurality of tags can be obtained through tag positioning, and then the filtering conditions of the content in the tags are set to obtain the required content, so as to lock the extraction range and determine what information is extracted from the tags. For example, the filtering rule can be to extract information satisfying the following conditions:

[0081] Condition 1: the document information type is type A;

[0082] Condition 2: the page information type is type B;

[0083] Condition 3: the page is a leaf node in the page navigation information;

[0084] Condition 4: the page includes the meaning of the entity.

[0085] Step 104, performing entity extraction on the first information based on the entity extraction rule to determine an entity extraction result; wherein the entity extraction result includes a plurality of entities and attribute information corresponding to each of the plurality of entities, and the attribute information includes any entity type in a plurality of entity types.

[0086] The attribute information of the entity includes a plurality of attributes of the entity. At least the entity type is included in the attributes.

[0087] Specifically, the plurality of entities can be divided into a plurality of first entities and a plurality of second entities, and the plurality of second entities are extracted from the attribute information of the plurality of first entities.

[0088] Here, the attribute information of the first entity can also include the entity meaning. Further, the plurality of second entities can be extracted from the entity meaning. In actual applications, the entity meaning is used to explain and describe the entity. For example, assuming that the entity is the N7 session authorization request success rate, the entity meaning is to count the N7 session authorization request success rate of the N7 interface of the UPCF, which is used to evaluate the PCC access capability of the mobile data network and report the N7 session authorization request failure rate.

[0089] In addition, in actual applications, the entity type of the entity can not exist in the first information. In actual applications, an entity library can be established in advance, and the entity library includes entity types and entities under the entity types. For the entity whose entity type cannot be extracted from the first information, the entity type of the entity can be determined through the pre-set entity library.

[0090] It should be noted that the attribute information of the entity can directly include the document information type and the page information type.

[0091] In one example, the entity extraction rule can include a plurality of rules. The plurality of rules can be set from a plurality of dimensions. For example, dimension 1: the text in the h1 tag is an entity, dimension 2: in the div tag, the div content meeting the following conditions is the meaning value: class = section, class = sectiontile value is "meaning". Wherein, div in web design means block, area, such as when you make a product selling webpage, product 1 and product 2 are divided into an area, which can define two divs, two blocks, class can indicate that you can give the block some attributes such as length and width, background color, you can give these attributes a name, class = chanpin indicates that you give the attribute set a name of chanpin. Wherein, class is an attribute in the html tag, which is used to define the class name of the element, corresponding to the class selector in css, the syntax "<element class = "classname">"; "classname" specifies the name of the element class. If multiple classes are specified for an element, separate the class names with spaces.

[0092] Specifically, the filtering rule can be defined from the DOM tree of the page, the metadata file, and the text information of the page, in combination with the above dimensions. For the DOM tree of the page, including the tree under head and body, a plurality of tags can be located to obtain the content filtering conditions of the tags, and then the required content is obtained to lock the extraction range and determine what information is extracted from the tags, or the information under the tags is directly extracted. For the metadata file, the entity can be extracted from the document metadata, component id, page id, page url, and component metadata, and the attribute information of the entity (for the sake of description and distinction, referred to as entity attribute information) can also be extracted. For the extracted entity attribute information, the entity can also be extracted from the entity attribute information.

[0093] It is worth noting that the entity extraction rule defines the description information of the entity and the description information of the attribute of the entity, based on which the entity can be extracted based on the description information of the entity, and the attribute of the entity and the attribute value of the attribute can be extracted based on the description information of the attribute of the entity. Here, the description information of the entity can be the tag where the entity is located, and the description information of the attribute of the entity can be the tag where the attribute is located.

[0094] Step 105, based on the entity relationship extraction rule and the attribute information corresponding to each of the plurality of entities, establishing the relationship between part or all of the plurality of entities, and determining the entity relationship extraction result.

[0095] According to an implementation, the entity relationship extraction rules can include a plurality of relationship rules. The plurality of relationship rules indicate what conditions are met between two entity types and what relationship is met. Correspondingly, whether the conditions defined by the relationship rules are met between two entity types can be determined based on attribute information of the two entities, and if the conditions are met, the two entities have the relationship defined by the matching relationship rules.

[0096] In one example, the plurality of relationship rules can be set from the following multiple dimensions.

[0097] Dimension 1: matching extraction based on entity attribute information. For example, when the attribute value of the "meaning" attribute of "entity type A" contains an entity of "entity type B", the entities under entity type A and entity type B have a statistical relationship. For example, when the attribute value of the "meaning" attribute of "entity type A" contains an entity of "entity type B", the entities under entity type A and entity type B have a causing relationship. Table 1 below shows the relationship between entities and entities.

[0098]

[0099]

[0100] Table 1

[0101] Among them, signaling is in a wireless communication system, in addition to transmitting user information, in order to make the whole network orderly work, the control signal needed to ensure normal communication, signaling allows program-controlled exchange, network database, other "intelligent" nodes in the network exchange the following information: call establishment, monitoring, demolition, distributed application process required information, network management information.

[0102] Dimension 2: extraction based on hierarchical relationship between the identities of the pages in the page navigation information. For example, when the next level of the measurement unit is all subordinate measurement indicators, a containing relationship between the measurement unit and the measurement indicators can be established.

[0103] It should be noted that the entity relationship extraction rules can extract a variety of relationships, such as containing relationship, statistical relationship, configuration relationship, and reporting relationship.

[0104] Under this dimension, the plurality of documents include first page navigation information, and the first page navigation information includes part or all of the plurality of entities.

[0105] Further, based on the steps 101-105, it further includes establishing a relationship between a plurality of entities in the first page navigation information based on the first page navigation information.

[0106] In actual application, the document has page navigation information, the page navigation information includes hierarchical conditions between multiple pages, and usually includes identifiers of multiple pages. The identifier of the page can be an entity. Therefore, a relationship between entities represented by identifiers of the pages can be established based on a hierarchical relationship between the identifiers of the pages, for example, there is no link relationship between the identifiers of the pages, and the identifiers of the pages are in a parallel relationship, for another example, identifier A of the page is linked to identifier B of the page, and there is a containing relationship between identifier A of the page and identifier B of the page.

[0107] In the scheme, the rule definition can be patterned according to the entity type, the model framework is unified, and the extraction efficiency is improved.

[0108] In addition, in actual use, the embodiment of the application defines the configuration file (content extraction rule, entity extraction rule and entity relationship extraction rule) according to a type of document feature, and realizes extraction of entities and relationships through execution of the extraction tool. For documents with different features, the configuration file can be flexibly defined to adapt to various special scenarios, and the knowledge can be accurately and effectively extracted, and the quality of the extracted data is good.

[0109] In addition, according to such extraction logic, a plurality of configuration files can be supported for one-time extraction of documents with different features, and the extraction efficiency is also very high for massive diversified documents.

[0110] Therefore, the embodiment of the application refines and abstracts the extraction rules, summarizes the extraction logic and extraction grammar, forms a set of extraction mode, can simultaneously extract different types of documents, realizes batch extraction of massive data, and can also design flexible grammar rules to consider accurate extraction of content in special scenarios.

[0111] It should be noted that the embodiment of the application can also be applied to an excel table, and the content extraction rule, the entity extraction rule and the relationship extraction rule need to be set based on the file name of the excel table, the field name represented by the cell and the label of the worksheet.

[0112] The following takes extraction of a document of a measurement index as an embodiment of the scheme of the application. The measurement index can be understood as an index representing measurement. It is assumed that the measurement index document has the following characteristics: the document information type is "measurement index", the page information type is "UPCF"; the pages in the page navigation information are leaf nodes; the title of the page is< / h1> <h1>to< / h1> <h6>The content in the section is an entity, the content fragment in the section is a value of the meaning attribute, and when the entity of the signaling error code entity type is described in the meaning, the measurement index and the signaling error code are in a statistical relationship.

[0113] The content extraction rule can be defined as: the document information type is "measurement index"; the page information type is "UPCF"; the page in the page navigation information is a leaf node; and the page content has a meaning section. At this time, a page as shown in Figure 4 may be obtained.

[0114] The entity extraction rule in the definition of the entity extraction rule is that the text in the h1 tag is an entity, and the content of the div tag that meets the following conditions is a value of the meaning: class = section, and the value of class = sectiontile is "meaning".

[0115] The entity relationship rule in the definition of the relationship extraction rule is that when the "meaning" attribute of the "measurement index" contains the "signaling error code", the two have a statistical relationship.

[0116] Based on the same concept as the method embodiments of the present application, the knowledge extraction device provided in the embodiments of the present application is also provided. The knowledge extraction device includes a plurality of modules, each module is used to execute each step in the knowledge extraction method provided in the embodiments of the present application, and the division of the modules is not limited here. Those skilled in the art can clearly understand that in actual application, each step in the knowledge extraction method provided in the embodiments of the present application can be distributed and completed by different modules according to needs, that is, the internal structure of the device is divided into different modules to complete all or part of the functions described above. Each module in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more modules can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit. In addition, the specific names of each module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the modules in the device can refer to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0117] For example, the knowledge extraction device is used to execute the knowledge extraction method provided in the embodiments of the present application, Figure 5 is a structural schematic diagram of the knowledge extraction device provided in the embodiments of the present application. As Figure 5 shown, the knowledge extraction device provided in the embodiments of the present application includes:

[0118] The document acquisition module 501 is used to acquire a plurality of documents.

[0119] A rule acquisition module 502 is configured to acquire a first file, wherein the first file includes content extraction rules, entity extraction rules for multiple entity types, and entity relationship extraction rules for the multiple entity types;

[0120] A screening module 503, configured to determine, based on the content extraction rule, first information adopting the entity extraction rule from the plurality of documents;

[0121] An entity extraction module 504 is configured to perform entity extraction on the first information based on the entity extraction rule and determine an entity extraction result; wherein the entity extraction result includes a plurality of entities and attribute information corresponding to each of the plurality of entities, and the attribute information includes any entity type among the plurality of entity types;

[0122] The relationship extraction module 505 is configured to establish relationships between some or all of the entities based on the entity relationship extraction rules and attribute information corresponding to each of the entities, and determine entity relationship extraction results.

[0123] The details of each module can be found in the detailed description of the knowledge extraction method above, which will not be repeated here.

[0124] Exemplary electronic devices

[0125] Based on the same concept as the embodiment of the method of the present application, the embodiment of the present application also provides an electronic device. This electronic device is also a knowledge extraction device. This electronic device can be a server or a terminal device. The terminal device can be a mobile phone, a tablet computer, etc.

[0126] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0127] like Figure 6 As shown, the electronic device 600 includes a processor 601 , a memory 602 and a network interface 603 .

[0128] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0129] The memory 602 can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0130] For example, the memory 602 can store a computer program, and the processor 601 executes the computer program to implement the steps of the knowledge extraction method embodiments described above, such as steps 101 to 105 shown in the figure. Alternatively, the processor 601 executes the computer program to implement the functions of the modules in the device embodiments described above. For example, the computer program can be divided into one or more modules / units, which can be a series of computer program instruction segments capable of completing a specific function, and the one or more modules / units are stored in the memory 602 and executed by the processor 601 to complete the present application. For example, the computer program can be divided into a document acquisition module 501, a rule acquisition module 502, a filtering module 503, an entity extraction module 504, and an entity relationship extraction module 505. The specific functions of each module are described above. Figure 1

[0131] The network interface 603 is used for transceiving data, for example, sending data processed by the processor 601 to other electronic devices, or receiving data sent by other electronic devices, etc.

[0132] Of course, in order to simplify, Figure 6 ​Only some of the components of the electronic device 600 related to the present application are shown, and components such as a bus, an input / output interface, and the like are omitted. In addition, the electronic device 600 can include any other appropriate components according to specific application cases. In addition, the electronic device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. Those skilled in the art can understand that Figure 6 The electronic device 600 is only an example and does not constitute a limitation on the electronic device, and can include more or fewer components than shown, or combine certain components, or different components, for example, the electronic device can also include an input device, an output device, a network access device, a bus, and the like. For example, the input device can be a microphone array, and can also include, for example, a keyboard, a mouse, and the like. For example, the output device can output various information to the outside, and can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0133] In addition to the above method, device and electronic device, the embodiments of the present application can also provide a computer program product including computer program instructions, which, when executed by a processor, cause the processor to perform the steps of the knowledge extraction method described in the above "method" part of the present application. Wherein the computer program product can be written in one or more program design languages in any combination for computer program code for executing the operations of the embodiments of the present application, the program design languages include object-oriented program design languages such as Java, C++, and the like, and also include conventional procedural program design languages such as "C" language or similar program design languages. Wherein the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer program code can be executed completely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or completely on a remote computing device or server.

[0134] Furthermore, the embodiments of the present application can also provide a computer readable storage medium, having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the steps of the knowledge extraction method according to various embodiments of the present disclosure described in the above "Method" section of the specification. The computer readable storage medium can employ any combination of one or more non-transitory media. The non-transitory media can be a non-transitory signal medium or a non-transitory storage medium. The non-transitory storage medium can include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the non-transitory storage medium include the following: an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. It should be noted that the computer readable medium contained herein can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0135] The method steps in the embodiments of the present application can be implemented in the form of hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC.

[0136] In the embodiments described above, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted by the computer readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, solid state disk (SSD)) and the like.

[0137] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0138] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0139] The basic principles of the present application are described above in combination with specific embodiments. However, it should be pointed out that the advantages, advantages, effects and the like mentioned in the present application are only examples and not limitations, and these advantages, advantages, effects and the like cannot be considered as the necessary possession of each embodiment of the present application. In addition, the above specific details are only for the purpose of example and understanding, and are not limited to the above specific details. The above details do not limit the present disclosure to the above specific details.

[0140] The block diagrams of the devices, apparatuses, equipment, systems involved in the present disclosure are only as illustrative examples and are not intended to require or imply that the connection, arrangement, configuration must be as shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have", and the like are open-ended words, mean "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0141] It is also necessary to point out that in the devices, apparatuses and methods of the present disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of the present disclosure.

[0142] The above description has been given for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0143] It can be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of differentiation in the description, and are not used to limit the scope of the embodiments of the present application.< / h6>

Claims

1. A method of knowledge extraction, characterized by, The method comprises: obtaining a plurality of documents; obtaining a first file; wherein the first file comprises content extraction rules, entity extraction rules of a plurality of entity types, and entity relationship extraction rules of the plurality of entity types; determining first information using the entity extraction rules from the plurality of documents based on the content extraction rules; performing entity extraction on the first information based on the entity extraction rules to determine an entity extraction result; wherein the entity extraction result comprises a plurality of entities and attribute information corresponding to each of the plurality of entities, and the attribute information comprises any entity type in the plurality of entity types; establishing relationships between part or all of the plurality of entities based on the entity relationship extraction rules and the attribute information corresponding to each of the plurality of entities to determine an entity relationship extraction result; the entity relationship extraction rules comprise a plurality of relationship rules, and the relationship rules indicate that entities of two entity types in the plurality of entity types have a first relationship when a first condition is met; establishing relationships between part or all of the plurality of entities based on the entity relationship extraction rules and the attribute information corresponding to each of the plurality of entities comprises: matching the plurality of relationship rules based on the attribute information corresponding to each of the plurality of entities to establish relationships between part or all of the plurality of entities.

2. The method of claim 1, wherein, The plurality of entities comprises at least one first entity and at least one second entity, and the at least one second entity is extracted from attribute information of the at least one first entity.

3. The method of claim 2, wherein, The attribute information corresponding to the first entity comprises entity meaning.

4. The knowledge extraction method of claim 1, wherein, The plurality of documents comprises first page navigation information, and the first page navigation information comprises part or all of the plurality of entities. The method further comprises: establishing relationships between a plurality of entities in the first page navigation information based on the first page navigation information.

5. The knowledge extraction method according to any one of claims 1 to 4, characterized by, The plurality of documents comprises documents satisfying the HyperText Markup Language.

6. The knowledge extraction method according to any one of claims 1 to 4, characterized by, The content extraction rules comprise at least one parameter: a first document information type, a first page information type, a navigation level, and at least one entity attribute.

7. The knowledge extraction method according to any one of claims 1 to 4, characterized by, The first file is a configuration file.

8. An entity extraction device, characterized by, The method comprises: at least one memory for storing a program; at least one processor for executing the program stored in the memory, and when the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1-7.

9. A computer storage medium, characterized in that The computer storage medium stores instructions, and when the instructions run on the computer, the computer executes the method according to any one of claims 1-7.

Images