Key information extraction method and system for demand document

Abstract

The invention provides a method and a system for extracting key information of a demand document, belongs to the technical field of document analysis, and aims to solve the problems of low efficiency and insufficient accuracy in the existing demand analysis process. Comprising the following steps: 1, preprocessing a demand document; 2, identifying a relationship between key entities and entities in the demand document, and forming a triple; 3, learning the triads identified in the step 2, and generating a plurality of triads newly embedded in the entity 1 based on a message function; 4, calculating a relative attention value of each newly embedded triple by applying an attention mechanism; 5, carrying out weighted average on the relative attention value of each newly embedded triple to obtain the final new embedding of the entity 1; according to the method, the manufacturing industry demand document is converted into the knowledge graph and further converted into the triple for embedding, so that the text and the structured data are effectively fused, and the comprehensiveness and the accuracy of information extraction are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of document analysis, and in particular relates to a method and system for extracting key information from a demand document. Background Art

[0002] In the process of informatization of manufacturing industry, demand analysis, as a key link, directly affects the quality and efficiency of subsequent system development. However, traditional demand analysis methods mainly rely on manual reading and interpretation of documents, which is not only time-consuming and labor-intensive, but also easily affected by personal subjective judgment, which may lead to omission or misunderstanding of key information. With the rapid development of natural language processing technology, especially the introduction of attention mechanism, new possibilities are provided to solve this problem. These advanced technologies have greatly improved the model's ability to capture long-distance dependencies and key information.

[0003] However, manufacturing requirements documents usually contain a lot of professional terms and industry-specific knowledge, which requires that information extraction methods must not only have general language understanding capabilities, but also incorporate manufacturing domain knowledge. Therefore, developing a method that can automatically identify and extract key information from manufacturing requirements documents can not only improve efficiency but also ensure accuracy, which is of great significance for enhancing the competitiveness of the entire manufacturing industry. The development of this method will combine the latest natural language processing technology and manufacturing-specific domain knowledge to provide strong assistance to requirements analysts, thereby promoting the development of the manufacturing industry in a more efficient and accurate direction. Summary of the invention

[0004] In view of this, the present invention provides a method and system for extracting key information from a requirement document to solve the problems of low efficiency, insufficient accuracy and lack of intelligent assistance in the current requirement analysis process.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A method for extracting key information from a requirement document, comprising:

[0007] Step 1: Preprocess the requirement documents and convert them into parsable text formats.

[0008] First, clean up useless information in the document, such as extra spaces, headers, footers, footnotes, etc., and standardize the text to convert different formats of the requirements document (PDF, Word, etc.) into a parsable text format.

[0009] Step 2: Identify the key entities and the relationships between entities in the preprocessed requirement document, and form the identified key entities, entities, and relationships into a triple (entity 1, relationship, entity 2) to indicate that there is a relationship connection from entity 1 to entity 2, where entity 1 represents the key entity and entity 2 represents the entity;

[0010] Named Entity Recognition (NER) is used to identify key entities in the text and the relationships between entities, such as equipment management, safety management, production operations, employees, and the relationship between employees and equipment management, and the relationship between equipment management and safety management. Figure 1 shown.

[0011] Step 3: Learn the triples identified in step 2 and generate multiple triples of new embeddings of entity 1 based on the message function;

[0012] The step 3 specifically includes:

[0013] Through word embedding technology, entities and relationships are mapped into key entity vectors, entity vectors and relationship vectors respectively and connected in series. The corresponding message function m(i, r, j) is constructed through linear transformation, as shown in the following formula:

[0014] m(i, r, j) = W[e i ][e r ][e j ]=We i +W(e i .e r ) (1)

[0015] In the formula, the vector e i 、e j and e r They represent the embedding of entity i, j and relation r respectively, and W represents the linear transformation matrix.

[0016] like Figure 2 As shown, this method uses message functions to represent how to learn information from adjacent nodes and edges. In order to obtain entity e i The new embedding learned is similar to e i The representation of each triplet is learned across the graph to generate messages. Inspired by the CrossE model, this method learns these embeddings by linearly transforming the concatenation of entity and relation feature vectors, resulting in a feature vector corresponding to a specific message function m(i, r, j).

[0017] Step 4: Apply the attention mechanism to calculate the absolute and relative attention values ​​of each triplet of the new embedding;

[0018] The step 4 specifically comprises the following steps:

[0019] Step 4.1: Get the absolute attention value b of the triple based on the activation function irj , as shown below:

[0020] b irj =LeakyyReLU(WATT m(i, r, j)) (2)

[0021] Where b irj represents the absolute attention value, W ATT represents a linear transformation parameterized by a weight matrix, and LeakyReLU represents the activation function:

[0022] Step 4.2: Obtain relative attention value α based on Softmax function irj , as shown below:

[0023] α irj =Softmax(b irj ) (3).

[0024] Step 5: Take a weighted average of the relative attention values ​​of each triplet of the new embedding to get the final new embedding of entity 1.

[0025] The step 5 is implemented by the following formula:

[0026]

[0027] in, Represents entity e i Neighborhood of Represents entity e i and e j , and H represents the number of attention layers.

[0028] In step 5, e is represented by the cosine similarity metric. i Through the relationship r After connecting with e j The degree of neighbor is as follows:

[0029]

[0030] Where S represents the set of valid triples obtained after updating.

[0031] A key information extraction system for a requirement document, comprising:

[0032] Preprocessing module: preprocesses the requirement documents and converts the requirement documents in different formats into parsable text formats;

[0033] Identification module: Identify key entities and relationships between entities in the preprocessed requirement document, and combine the identified key entities, entities, and relationships into a triple (entity 1, relationship, entity 2) to indicate that there is a relationship connection from entity 1 to entity 2, where entity 1 represents the key entity and entity 2 represents the entity;

[0034] New embedding module: learns the triples identified in the recognition module and generates multiple triplets of new embeddings of entities based on the message function;

[0035] Computation module: Apply the attention mechanism to calculate the relative attention value of each triplet of the new embedding;

[0036] Weighted average module: The relative attention values ​​of each triplet of the new embedding are weighted averaged to obtain the final new embedding of entity 1.

[0037] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0038] 1. In the present invention, by converting the manufacturing demand document into a knowledge graph and further converting it into triple embedding, the text and structured data are effectively integrated, and the comprehensiveness and accuracy of information extraction are improved;

[0039] 2. In the present invention, by calculating the interactive relationship between entities and relationships, it is more conducive to mining reliable entity relationship link paths;

[0040] 3. In the present invention, adaptive extraction of key information is achieved through a multi-head attention layer, so that the model can jointly focus on information from different relational parameter subspaces. At the same time, a simplified cosine similarity loss function is used to reduce the calculation dimension. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0042] Figure 1 It is a schematic diagram of the entity relationship triple representation method of the present invention;

[0043] Figure 2 Schematic diagram of the attention model structure of the present invention;

[0044] Figure 3 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0046] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0048] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0049] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0050] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0051] Example

[0052] like Figure 1-3 As shown, an embodiment of the present invention discloses a method for extracting key information of a requirement document, comprising:

[0053] Step 1: Preprocess the requirement documents and convert them into parsable text formats.

[0054] First, clean up useless information in the document, such as extra spaces, headers, footers, footnotes, etc., and standardize the text to convert different formats of the requirements document (PDF, Word, etc.) into a parsable text format.

[0055] Step 2: Identify the key entities and the relationships between entities in the preprocessed requirement document, and form the identified key entities, entities, and relationships into a triple (entity 1, relationship, entity 2) to indicate that there is a relationship connection from entity 1 to entity 2, where entity 1 represents the key entity and entity 2 represents the entity;

[0056] Named Entity Recognition (NER) is used to identify key entities in the text and the relationships between entities, such as equipment management, safety management, production operations, employees, and the relationship between employees and equipment management, and the relationship between equipment management and safety management. Figure 1 shown.

[0057] Step 3: Learn the triples identified in step 2 and generate multiple triples of new embeddings of entity 1 based on the message function;

[0058] The step 3 specifically includes:

[0059] Through word embedding technology, entities and relationships are mapped into key entity vectors, entity vectors and relationship vectors respectively and connected in series. The corresponding message function m(i, r, j) is constructed through linear transformation, as shown in the following formula:

[0060] m(i, r, j) = W[e i ][e r ][e j ]=We i +W(e i .e r ) (1)

[0061] In the formula, the vector e i 、e j and e r They represent the embedding of entity i, j and relation r respectively, and W represents the linear transformation matrix.

[0062] like Figure 2 As shown, this method uses message functions to represent how to learn information from adjacent nodes and edges. In order to obtain entity e i The new embedding learned is similar to e i The representation of each triplet is learned across the graph to generate messages. Inspired by the CrossE model, this method learns these embeddings by linearly transforming the concatenation of entity and relation feature vectors, resulting in a feature vector corresponding to a specific message function m(i, r, j).

[0063] Step 4: Apply the attention mechanism to calculate the absolute and relative attention values ​​of each triplet of the new embedding;

[0064] The step 4 specifically comprises the following steps:

[0065] Step 4.1: Get the absolute attention value b of the triple based on the activation function irj , as shown below:

[0066] b irj =LeakyReLU(W ATTm(i,r,j)) (2)

[0067] Where b irj represents the absolute attention value, W ATT represents a linear transformation parameterized by a weight matrix, and LeakyReLU represents the activation function;

[0068] Step 4.2: Obtain relative attention value α based on Softmax function irj , as shown below:

[0069] α irj =Softmax(b irj ) (3).

[0070] Step 5: Take a weighted average of the relative attention values ​​of each triplet of the new embedding to get the final new embedding of entity 1.

[0071] The step 5 is implemented by the following formula:

[0072]

[0073] in, Represents entity e i Neighborhood of Represents entity e i and e j , and H represents the number of attention layers.

[0074] In step 5, e is represented by the cosine similarity metric. i Through the relationship r After connecting with e j The degree of neighbor is as follows:

[0075]

[0076] Where S represents the set of valid triples obtained after updating.

[0077] A key information extraction system for a requirement document, comprising:

[0078] Preprocessing module: preprocesses the requirement documents and converts the requirement documents in different formats into parsable text formats;

[0079] Identification module: Identify key entities and relationships between entities in the preprocessed requirement document, and combine the identified key entities, entities, and relationships into a triple (entity 1, relationship, entity 2) to indicate that there is a relationship connection from entity 1 to entity 2, where entity 1 represents the key entity and entity 2 represents the entity;

[0080] New embedding module: learns the triples identified in the recognition module and generates multiple triplets of new embeddings of entities based on the message function;

[0081] Computation module: Apply the attention mechanism to calculate the relative attention value of each triplet of the new embedding;

[0082] Weighted average module: The relative attention values ​​of each triplet of the new embedding are weighted averaged to obtain the final new embedding of entity 1.

[0083] The circuits, electronic components and modules involved are all prior art and can be fully implemented by those skilled in the art. Needless to say, the content protected by the present invention does not involve improvements to software and methods.

[0084] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0085] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for extracting key information from a requirement document, characterized in that: include: Step 1: Preprocess the requirement documents and convert them into parsable text formats. Step 2: Identify the key entities and the relationships between entities in the preprocessed requirement document, and form the identified key entities, entities, and relationships into a triple (entity 1, relationship, entity 2) to indicate that there is a relationship connection from entity 1 to entity 2, where entity 1 represents the key entity and entity 2 represents the entity; Step 3: Learn the triples identified in step 2 and generate multiple triples of new embeddings of entity 1 based on the message function; Step 4: Apply the attention mechanism to calculate the relative attention value of each triplet of the new embedding; Step 5: Take a weighted average of the relative attention values ​​of each triplet of the new embedding to get the final new embedding of entity 1.

2. The method for extracting key information from a requirement document according to claim 1, characterized in that: The step 3 specifically includes: Through word embedding technology, entities and relationships are mapped into key entity vectors, entity vectors and relationship vectors respectively and connected in series. The corresponding message function m(i, r, j) is constructed through linear transformation, as shown in the following formula: m(i,r,j)=W[e i ][have been r ][have been j ]=We i +W(ie i .have been r ) (1) In the formula, the vector e i 、e j and e r They represent the embedding of entity i, j and relation r respectively, and W represents the linear transformation matrix.

3. The method for extracting key information from a requirement document according to claim 1, characterized in that: The step 4 specifically comprises the following steps: Step 4.1: Get the absolute attention value b of the triple based on the activation function irj , as shown below: b irj =LeakyReLU(W ATT m(i,r,j)) (2) Where b irj represents the absolute attention value, W ATT represents a linear transformation parameterized by a weight matrix, and LeakyReLU represents the activation function; Step 4.2: Obtain relative attention value α based on Softmax function irj , as shown below: α irj =Softmax(b irj ) (3)。 4. The method for extracting key information from a requirement document according to claim 1, characterized in that: The step 5 is implemented by the following formula: in, Represents entity e i Neighborhood of Represents entity e i and e j , H represents the number of attention layers, and h is an indicator variable of the sum symbol ∑, with a value range of 1 to H.

5. The method for extracting key information from a requirement document according to claim 4, characterized in that: In step 5, e is represented by the cosine similarity metric. i Through the relationship r After connecting with e j The degree of neighbor is as follows: Where S represents the set of valid triples obtained after updating.

6. A key information extraction system for a requirement document, characterized in that: A method for extracting key information from a requirement document according to claims 1 to 5, comprising: Preprocessing module: preprocesses the requirement documents and converts the requirement documents in different formats into parsable text formats; Identification module: Identify key entities and relationships between entities in the preprocessed requirement document, and form the identified key entities, entities and relationships into triples (entity 1, relationship, entity 2) to indicate that there is a relationship connection between entity 1 and entity 2, where entity 1 represents the key entity and entity 2 represents the entity, to indicate that there is a relationship connection between entity 1 and entity 2, where entity 1 represents the key entity and entity 2 represents the entity; New embedding module: learns the triples identified in step 2 and generates multiple triples of new embeddings of entity 1 based on the message function; Computation module: Apply the attention mechanism to calculate the relative attention value of each triplet of the new embedding; Weighted average module: The relative attention values ​​of each triplet of the new embedding are weighted averaged to obtain the final new embedding of entity 1.

Images