Information extraction method and apparatus thereof

Abstract

The application discloses an information extraction method and device, relates to the field of artificial intelligence, and in particular relates to natural language processing (NLP) and deep learning technologies. The specific implementation scheme is as follows: a feature vector of a to-be-processed text and a dependency relation graph of the to-be-processed text are acquired; a corresponding distance matrix is generated according to the dependency relation graph; an attention mechanism is used to calculate an attention value between any two words in the to-be-processed text according to the feature vector of the to-be-processed text and the distance matrix; semantic feature representation of the to-be-processed text is acquired according to the feature vector of the to-be-processed text and the attention value between any two words in the to-be-processed text; and information extraction is performed on the to-be-processed text according to the semantic feature representation of the to-be-processed text. The dependency relation information in the to-be-processed text is encoded in a display coding manner, so that more accurate semantic feature representation can be acquired. Information extraction is performed by using the semantic feature representation, so that the information extraction effect can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to natural language processing (NLP) and deep learning technologies, and specifically relates to an information extraction method and device. BACKGROUND

[0002] Information extraction refers to extracting specified types of entity, relationship, event and other factual information from natural language text, and forming structured data output. Information extraction is a technology for extracting specific information from text data. Text data is composed of specific units, such as sentences, paragraphs, chapters, and text information is composed of specific units, such as words, phrases, sentences, paragraphs, or combinations of these specific units. Extracting noun phrases, names, and places from text data is text information extraction, and of course, the information extracted by the text information extraction technology can be various types of information.

[0003] In related technologies, the text data is usually semantically expressed to obtain semantic feature representation of the text data, and the information extraction is performed on the text data by using the semantic feature representation. Therefore, how to obtain more accurate semantic feature representation to improve the information extraction effect is a problem to be solved. SUMMARY

[0004] The present application provides an information extraction method, device, electronic equipment and storage medium.

[0005] According to a first aspect of the present application, an information extraction method is provided, comprising:

[0006] obtaining a feature vector of a to-be-processed text and a dependency relation graph of the to-be-processed text;

[0007] generating a corresponding distance matrix according to the dependency relation graph;

[0008] calculating attention values between any two words in the to-be-processed text by using an attention mechanism according to the feature vector of the to-be-processed text and the distance matrix;

[0009] obtaining a semantic feature representation of the to-be-processed text according to the feature vector of the to-be-processed text and the attention values between any two words in the to-be-processed text;

[0010] performing information extraction on the to-be-processed text according to the semantic feature representation of the to-be-processed text.

[0011] According to a second aspect of the present application, an information extraction device is provided, comprising:

[0012] The first obtaining module is configured to obtain a feature vector of a to-be-processed text and a dependency relation graph of the to-be-processed text.

[0013] The generating module is configured to generate a corresponding distance matrix according to the dependency relation graph.

[0014] The calculating module is configured to calculate an attention value between any two words in the to-be-processed text by using an attention mechanism according to the feature vector of the to-be-processed text and the distance matrix.

[0015] The second obtaining module is configured to obtain a semantic feature representation of the to-be-processed text according to the feature vector of the to-be-processed text and the attention value between any two words in the to-be-processed text.

[0016] The information extraction module is configured to perform information extraction on the to-be-processed text according to the semantic feature representation of the to-be-processed text.

[0017] According to a third aspect of the present application, an electronic device is provided, comprising:

[0018] at least one processor; and

[0019] a memory connected with the at least one processor in communication; wherein

[0020] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the first aspect.

[0021] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of the first aspect.

[0022] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of the method of the first aspect.

[0023] According to the technical solution of the present application, the dependency relation information in the to-be-processed text is encoded in a display coding manner, and the distance matrix obtained after coding is introduced into the attention mechanism, so that each node can interact with more other nodes in the feature extraction process, and can focus on nodes with closer distances, thereby obtaining more accurate semantic feature representation. Using the semantic feature representation for information extraction can improve the information extraction effect.

[0024] It should be appreciated that the description set forth in this section is not intended to identify key or essential features of the embodiments of the application, nor is it intended to limit the scope of the application. Other features of the application will be readily apparent to those skilled in the art upon reviewing the following description and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0025] The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this application. In the drawings:

[0026] Figure 1 A flowchart of an information extraction method provided for an embodiment of the application;

[0027] Figure 2a A flowchart of another information extraction method provided for an embodiment of the application;

[0028] Figure 2b An example graph of a dependency relation graph of a text to be processed provided for an embodiment of the application;

[0029] Figure 3 A flowchart of a method for calculating an attention value between any two words in a text provided for an embodiment of the application;

[0030] Figure 4 An example graph of a network structure of an attention mechanism provided for an embodiment of the application;

[0031] Figure 5 A flowchart of another method for calculating an attention value between any two words in a text provided for an embodiment of the application;

[0032] Figure 6 An example graph of a network structure of another attention mechanism provided for an embodiment of the application;

[0033] Figure 7 An example graph of an output result of each layer in an attention mechanism provided for an embodiment of the application;

[0034] Figure 8 A block diagram of an information extraction device provided for an embodiment of the application;

[0035] Figure 9 A block diagram of an electronic device for implementing the information extraction method of an embodiment of the application. DETAILED DESCRIPTION

[0036] Exemplary embodiments of the present application are described herein with reference to the accompanying drawings, which are cited by way of example only. The various details of the embodiments of the application can be understood with reference to the figures, which are intended to illustrate the application for clarity and conciseness. Therefore, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted herein.

[0037] Figure 1 A flowchart of an information extraction method according to an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the method can include, but is not limited to, the following steps. Figure 1

[0038] In step 101, a feature vector of a text to be processed and a dependency graph of the text to be processed are obtained.

[0039] Optionally, in the embodiment of the present application, the feature vector of the text to be processed can be understood as a text representation (or word vector) of the text to be processed.

[0040] In a possible implementation, a preset word vector model Word2vec can be used to process the text to be processed to obtain a text representation of the text to be processed, and the text representation of the text to be processed is the feature vector of the text to be processed. Alternatively, other implementation manners can also be used to obtain the feature vector of the text to be processed, for example, One-Hot vector encoding technology or word embedding technology, etc. The present application does not make specific limitations on this, and will not be described again.

[0041] In the embodiment of the present application, the dependency graph of the text to be processed refers to a graph that can indicate the dependency relationship between the words in the text to be processed. The nodes in the dependency graph can be the words in the text to be processed, and the edges in the dependency graph can be the dependency relationship between the corresponding nodes.

[0042] In a possible implementation, dependency syntax analysis technology can be used to perform syntax analysis on the text to be processed to obtain a syntax analysis result of the text to be processed, and the dependency graph of the text to be processed is constructed according to the syntax analysis result. The words in the text to be processed are taken as nodes, and the nodes are connected by edges according to the dependency relationship between the words to construct the dependency graph of the text to be processed.

[0043] In step 102, a corresponding distance matrix is generated according to the dependency graph.

[0044] ​In the related art, an adjacency matrix is usually generated according to the connection relationship of nodes in the dependency graph, and the adjacency matrix is introduced into the graph attention network for enhanced representation to obtain semantic feature representation of the text to be processed. However, the inventors of the present application found through experiments that the features learned by the adjacency matrix generated in this way are not good, which is because the dependency graph is relatively sparse (for example, for a text of L words, the size of the adjacency matrix is L*L, and only L elements are not 0), which leads to limited information that each node can aggregate when updating the features, and further leads to poor quality of the learned features.

[0045] To alleviate this situation, in one possible implementation, a corresponding adjacency matrix (directed and with reverse edges) can be generated according to the dependency graph, and the adjacency matrix can be densified according to the position information of the nodes in the dependency graph in the text to be processed to obtain a distance matrix.

[0046] Optionally, an adjacency matrix can be generated according to the connection relationship of nodes in the dependency graph, and the value of the corresponding element position in the adjacency matrix can be adjusted according to the position information of each node in the text to be processed to obtain a distance matrix. In this way, the dependency relationship information in the text is encoded in a display coding manner, and the distance matrix obtained after encoding is introduced into the semantic stage, so that more accurate semantic feature representation can be obtained.

[0047] In step 103, an attention value between any two words in the text to be processed is calculated using an attention mechanism according to the feature vector of the text to be processed and the distance matrix.

[0048] In the embodiments of the present application, the dependency relationship information in the text is encoded in a display coding manner, and the distance matrix obtained after encoding is introduced into the attention mechanism, so that each node can interact with more other nodes in the feature extraction process, and can also pay attention to nodes with closer distances.

[0049] In step 104, the semantic feature representation of the text to be processed is obtained according to the feature vector of the text to be processed and the attention value between any two words in the text to be processed.

[0050] In one possible implementation, a value vector can be determined according to the feature vector of the text to be processed, and the value vector and the attention value between any two words in the text to be processed can be multiplied to obtain the semantic feature representation of the text to be processed.

[0051] As an example, the feature vector of the text to be processed can be converted into a value vector V, and the attention value between any two words in the text to be processed can be weighted with the corresponding elements in the value vector V to obtain the semantic feature representation of the text to be processed.

[0052] In step 105, information extraction is performed on the to-be-processed text according to the semantic feature representation of the to-be-processed text.

[0053] In a possible implementation, entity recognition can be performed on the semantic feature representation of the to-be-processed text based on the semantic feature representation, to obtain recognized entity information in the to-be-processed text. Optionally, the semantic feature representation of the to-be-processed text is decoded based on a named entity recognition task, a probability distribution of each word is predicted at each prediction time step, and the word with the highest probability is taken as a predicted word, and the recognized entity information in the to-be-processed text is obtained by analysis.

[0054] By implementing the embodiments of the present application, the dependency relationship information in the to-be-processed text is encoded in a display coding manner, and the distance matrix obtained after encoding is introduced into the attention mechanism, so that each node can interact with more other nodes in the feature extraction process, and can focus on nodes with closer distances, thereby obtaining more accurate semantic feature representation. Using the semantic feature representation for information extraction can improve the information extraction effect.

[0055] Figure 2a Another flowchart of an information extraction method provided by the embodiments of the present application is shown in FIG. 2. As shown in FIG. 2, the method can include but is not limited to the following steps. Figure 2a

[0056] In step 201, a feature vector of the to-be-processed text and a dependency relationship graph of the to-be-processed text are obtained.

[0057] In the embodiments of the present application, step 201 can be implemented in any of the embodiments of the present application, and the embodiments of the present application do not limit this and will not be repeated.

[0058] In step 202, a corresponding adjacency matrix is generated according to the dependency relationship graph.

[0059] Optionally, the adjacency matrix can be generated according to the connection relationship of the nodes in the dependency relationship graph.

[0060] In step 203, the adjacency matrix is densified according to the position information of the nodes in the to-be-processed text in the dependency relationship graph, to obtain a distance matrix.

[0061] Optionally, the value of the corresponding element position in the adjacency matrix can be adjusted according to the position information of each node in the to-be-processed text, to obtain the distance matrix. In a possible implementation, the implementation of step 203 can include the following steps:

[0062] Step 203a, determining the position information of the word corresponding to the node in the to-be-processed text in the dependency relationship graph.​

[0063] In one possible implementation, the position information of the words corresponding to each node in the dependency graph in the text to be processed can be determined.

[0064] Step 203b: In the case that there is no dependency relationship between two nodes in the dependency graph, determine the distance value of the words corresponding to the two nodes in the text to be processed based on the position information, and set the distance value between the two nodes in the adjacency matrix as the distance value of the words corresponding to the two nodes in the text to be processed.

[0065] Step 203c: When there is a dependency relationship between two nodes in the dependency graph, set the distance value between the two nodes in the adjacency matrix to the first value to obtain the distance matrix.

[0066] For example, suppose the text to be processed is "word 1 word 5, word 6 word 2, word 3 word 4.", such as Figure 2b The diagram shows the dependency graph of the text to be processed. An adjacency matrix can be generated based on the connection relationships of the nodes in the dependency graph. As shown in formula (1) below, the elements corresponding to nodes with dependency relationships in the adjacency matrix are 1, and the elements corresponding to other nodes without dependency relationships are 0. The position information of the words corresponding to the nodes in the dependency graph in the text to be processed is determined. If there is a dependency relationship between two nodes, the distance between the two nodes is set to 1. Otherwise, the distance between the two nodes is the distance between the two nodes in the text, as shown in formula (2) below. Using the position information of the words corresponding to the nodes in the dependency graph in the text to be processed, the elements in formula (1) are adjusted to obtain the distance matrix. The rows and columns in formula (1) and formula (2) are: “word1”, “word2”, “word3”, “word4”, “word5”, “word6”, “,”, and “.”, respectively.

[0067]

[0068]

[0069] In step 204, based on the feature vector and distance matrix of the text to be processed, an attention mechanism is used to calculate the attention value between any two words in the text to be processed.

[0070] In the embodiments of this application, step 204 can be implemented in any of the ways described in the various embodiments of this application. The embodiments of this application do not limit this, nor will they be described in detail.

[0071] In step 205, a semantic feature representation of the text to be processed is obtained according to the feature vector of the text to be processed and the attention value between any two words in the text to be processed.

[0072] In the embodiments of the present application, step 205 can be implemented in any of the embodiments of the present application, and the embodiments of the present application do not limit this and will not be repeated here.

[0073] In step 206, information extraction is performed on the text to be processed according to the semantic feature representation of the text to be processed.

[0074] In the embodiments of the present application, step 206 can be implemented in any of the embodiments of the present application, and the embodiments of the present application do not limit this and will not be repeated here.

[0075] By implementing the embodiments of the present application, the dependency relationship information in the text is encoded by a display coding manner, and the distance matrix obtained after coding is introduced into the attention mechanism, so that each node can interact with more other nodes in the feature extraction process, and can focus on nodes with closer distances.

[0076] Figure 3 A flowchart of a method for calculating the attention value between any two words in a text according to an embodiment of the present application is provided. As shown in Figure 3 The method can include but is not limited to the following steps.

[0077] In step 301, a query vector and a key vector are determined according to the feature vector of the text to be processed.

[0078] Optionally, the feature vector of the text to be processed can be converted into a query vector Q and a key vector K in the attention mechanism.

[0079] In step 302, a double-affine calculation is performed according to the query vector and the key vector to obtain a first intermediate matrix, and a matrix multiplication operation is performed on the query vector and the key vector to obtain a second intermediate matrix.

[0080] In step 303, the data in the first intermediate matrix is modified according to the distance matrix to obtain a third intermediate matrix.

[0081] In a possible implementation, the data in the first intermediate matrix is divided by the data at the corresponding position in the distance matrix to obtain the third intermediate matrix.

[0082] In step 304, a matrix addition operation is performed on the second intermediate matrix and the third intermediate matrix to obtain a fourth intermediate matrix.

[0083] In step 305, the fourth intermediate matrix is scaled, masked and Softmax operation is performed on the regression function to obtain the attention value between any two words in the text to be processed.

[0084] For example, as shown in the network structure of the attention mechanism used in the present application, it includes Biaffine Matmul module, Distance Scaling module, Add module, Scaling module, Mask (opt.) module, Softmax module and Matmul module. The calculation formula of the Biaffine Matmul module is as follows. Figure 4

[0085]

[0086] wherein and denote the edge r between the l-th layer node i and node j. ij The corresponding Biaffine parameter. Table 1: Element in the first intermediate matrix. Through Biaffine calculation, edge information is integrated and interacts with node information.

[0087] The output of the Biaffine Matmul module is input into the Distance Scaling module. Through the Distance Scaling module, the output of the Biaffine Matmul module is modified to obtain the third intermediate matrix. The calculation formula of the Distance Scaling module is as follows.

[0088]

[0089] wherein d ij is the distance between node i and node j, i.e. the element in the distance matrix.

[0090] Then, the output of the Distance Scaling module is added to the output of the Matmul module in the Add module, and the calculation formula of the Add module is as follows.

[0091]

[0092] Then, the output of the Add module is scaled, masked and Softmax operation is performed on the regression function to obtain the attention value between any two words in the text to be processed. It can be understood that, as Figure 4 ​​​​The calculation method behind the Add module shown and the attention calculation in related technologies are the same, and will not be described here.

[0093] By implementing the embodiments of the present application, the dependency relationship information in the text is encoded in a display coding manner, and the distance matrix obtained after encoding is introduced into the attention mechanism, so that each node can interact with more other nodes in the feature extraction process, and can focus on nodes with closer distances.

[0094] Figure 5 The flowchart of another method for calculating the attention value between any two words in the text provided by the embodiments of the present application. As shown in the figure, the method can include but is not limited to the following steps. Figure 5

[0095] In step 501, according to the feature vector of the text to be processed, the query vector and the key vector are determined.

[0096] Optionally, the feature vector of the text to be processed can be converted into the query vector Q and the key vector K in the attention mechanism.

[0097] In step 502, the double affine calculation is performed according to the query vector and the key vector to obtain a first intermediate matrix, and the matrix multiplication operation is performed on the query vector and the key vector to obtain a second intermediate matrix.

[0098] In step 503, the first intermediate matrix and the second intermediate matrix are added to obtain a third intermediate matrix.

[0099] In step 504, the third intermediate matrix is scaled, masked and subjected to the regression function Softmax operation to obtain the corresponding attention matrix.

[0100] In step 505, the data in the attention matrix is corrected and normalized according to the distance matrix to obtain the attention value between any two words in the text to be processed.

[0101] In one possible implementation, the data in the attention matrix is divided by the data at the corresponding position in the distance matrix to obtain a fourth intermediate matrix, and the data in the fourth intermediate matrix is normalized to obtain the attention value between any two words in the text to be processed.

[0102] For example, as shown in the figure, Figure 6 ​The diagram shows the network structure of the attention mechanism used in this application, which includes a Biaffine Matmul module, a Distance Scaling module, an Add module, a Scaling module, a Mask (opt.) module, a Softmax (regression function) module, and a Matmul (matrix multiplication) module. The calculation formula for the Biaffine Matmul module is as follows.

[0103]

[0104] in and The edge r between node i and node j in the l-th layer is represented by... ij The corresponding double affine parameters. The elements in the first intermediate matrix are represented. Edge information is incorporated and interacts with node information through biaffine computation.

[0105] The output of the Biaffine Matmul module Enter the Add module. The Add module adds the results of the BiaffineMatmul module and the Matmul module. The calculation formula for the Add module can be expressed as follows:

[0106]

[0107] Then, the output of the Add module is scaled proportionally, masked, and subjected to the Softmax regression function to obtain the corresponding attention matrix.

[0108] It is worth noting that, Figure 6 The attention network structure shown is similar to Figure 4 The attention network structure shown differs from the one in that the Distance Scaling module follows the Softmax module. However, because this step requires the output to be normalized, it is different from the previous one. Figure 4 The Distance Scaling module calculations shown are slightly different. The calculation formula for the Distance Scaling module is as follows.

[0109]

[0110]

[0111] In other words, after performing the softmax operation, the result of the softmax operation can be... The input is fed into the Distance Scaling module for calculation. The result of the softmax operation is then processed by the Distance Scaling module. Divide the result by the elements in the distance matrix and normalize it using the formula (8) above to obtain the normalized result. This is the attention value between node i and node j.

[0112] By implementing the embodiments of this application, dependency information in the text is encoded by explicit encoding, and the distance matrix obtained after encoding is introduced into the attention mechanism, so that each node can interact with more other nodes during the feature extraction process, and can pay attention to nodes that are closer.

[0113] It should be noted that, in the embodiments of this application, regarding the attention mechanism, such as Figure 7 Dense connections can be used to connect each layer in the attention mechanism, that is:

[0114] H i = [H1; H2; ...; H i-1 (9)

[0115] Among them, H i Let represent the output of the i-th layer, and represent the concatenation operation. Thus, a densely connected attention network is used, which not only increases information interaction between multiple layers but also adds a gradient backpropagation mechanism, making the network easier to train. Increasing the number of layers also alleviates the gradient vanishing problem.

[0116] Figure 8 This is a block diagram of an information extraction device provided in an embodiment of this application. Figure 8 As shown, the information extraction device may include: a first acquisition module 801, a generation module 802, a calculation module 803, a second acquisition module 804, and an information extraction module 805.

[0117] The first acquisition module 801 is used to acquire the feature vector of the text to be processed and the dependency graph of the text to be processed.

[0118] The generation module 802 is used to generate a corresponding distance matrix based on the dependency graph. In one possible implementation, the generation module 802 is specifically used to: generate a corresponding adjacency matrix based on the dependency graph; and densify the adjacency matrix based on the position information of the nodes in the dependency graph in the text to be processed to obtain the distance matrix.

[0119] Optionally, in an implementation, the generating module 802 is specifically configured to: determine position information of the words corresponding to the nodes in the dependency graph in the to-be-processed text; in the case that there is no dependency relationship between two nodes in the dependency graph, determine a distance value of the words corresponding to the two nodes respectively in the to-be-processed text according to the position information, and set the distance value between the two nodes in the adjacency matrix as the distance value of the words corresponding to the two nodes respectively in the to-be-processed text; in the case that there is a dependency relationship between two nodes in the dependency graph, set the distance value between the two nodes in the adjacency matrix as a first numerical value, to obtain the distance matrix.

[0120] The computing module 803 is configured to calculate the attention value between any two words in the to-be-processed text by using an attention mechanism according to the feature vector of the to-be-processed text and the distance matrix.

[0121] In a possible implementation, the computing module 803 is specifically configured to: determine a query vector and a key vector according to the feature vector of the to-be-processed text; perform bi-affine calculation on the query vector and the key vector to obtain a first intermediate matrix, and perform matrix multiplication operation on the query vector and the key vector to obtain a second intermediate matrix; modify the data in the first intermediate matrix according to the distance matrix to obtain a third intermediate matrix; perform matrix addition operation on the second intermediate matrix and the third intermediate matrix to obtain a fourth intermediate matrix; and perform per-ratio scaling, masking and regression function Softmax operation on the fourth intermediate matrix to obtain the attention value between any two words in the to-be-processed text.

[0122] Optionally, the computing module 803 divides the data in the first intermediate matrix by the data at the corresponding position in the distance matrix to obtain the third intermediate matrix.

[0123] In another possible implementation, the computing module 803 is specifically configured to: determine a query vector and a key vector according to the feature vector of the to-be-processed text; perform bi-affine calculation on the query vector and the key vector to obtain a first intermediate matrix, and perform matrix multiplication operation on the query vector and the key vector to obtain a second intermediate matrix; perform matrix addition operation on the first intermediate matrix and the second intermediate matrix to obtain a third intermediate matrix; perform per-ratio scaling, masking and regression function Softmax operation on the third intermediate matrix to obtain a corresponding attention matrix; and modify and normalize the data in the attention matrix according to the distance matrix to obtain the attention value between any two words in the to-be-processed text.

[0124] Optionally, the computing module 803 divides data in the attention matrix by data in the distance matrix at corresponding positions to obtain a fourth intermediate matrix, and normalizes data in the fourth intermediate matrix to obtain the attention value between any two words in the text to be processed.

[0125] The second obtaining module 804 is configured to obtain a semantic feature representation of the text to be processed according to the feature vector of the text to be processed and the attention value between any two words in the text to be processed. In a possible implementation, the second obtaining module 804 determines a value vector according to the feature vector of the text to be processed, and performs a product operation on the value vector and the attention value between any two words in the text to be processed to obtain the semantic feature representation of the text to be processed.

[0126] The information extraction module 805 is configured to perform information extraction on the text to be processed according to the semantic feature representation of the text to be processed. In a possible implementation, the information extraction module 805 performs entity recognition on the text to be processed based on the semantic feature representation to obtain recognized entity information in the text to be processed.

[0127] As to the apparatus in the above-described embodiments, specific manners in which various modules perform operations have been described in details in the embodiments of the method, and will not be described in details here.

[0128] According to the embodiments of the present application, the present application further provides an electronic device and a readable storage medium.

[0129] As shown in Figure 9 is a block diagram of an electronic device according to the method of information extraction of the embodiments of the present application. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown in the figure, their connections and relationships, and their functions, are merely examples, and are not intended to limit the implementations described and / or claimed in this document.

[0130] As shown in Figure 9As shown, the electronic device includes one or more processors 901, memory 902, and interfaces 903 for connecting various parts of the electronic device, including a high-speed interface and a low-speed interface. Various parts are connected using various buses, and can be mounted on a common motherboard or in other manners, as desired. The processor can process instructions for execution within the electronic device, including instructions stored in the memory or on the memory to display graphical information for a GUI on an external input / output device, such as a display device coupled to the high-speed interface. In other implementations, multiple processors and / or multiple buses can be employed as desired to implement these functions, and multiple memories and types of memory can be used. Also, various parts of the electronic device can be implemented as part of one or more pieces of hardware, such as the display device coupled to the high-speed interface, which can be implemented as a touch screen, or a keyboard, or a television, or a monitor, or a combination thereof. Figure 9 The processor 901 is taken as an example in the embodiment.

[0131] The memory 902 is a non-transitory computer readable storage medium provided by the present application. The memory stores instructions executable by at least one processor, so that the at least one processor executes the information extraction method provided by the present application. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to execute the information extraction method provided by the present application.

[0132] The memory 902 is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the information extraction method in the embodiment of the present application. The processor 901 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions and modules stored in the memory 902, that is, implements the information extraction method in the above method embodiment.

[0133] The memory 902 can include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function; the data storage area can store data created according to the use of the electronic device, etc. In addition, the memory 902 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 902 can optionally include a memory remotely arranged with respect to the processor 901, which can be connected to the electronic device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0134] The electronic device can further include an input device 903 and an output device 904. The processor 901, the memory 902, the input device 903, and the output device 904 can be connected through a bus or other means, Figure 9 The connection through the bus is taken as an example.

[0135] The input device 903 can receive input digital or character information and generate key signal input related to user settings and function controls of the electronic device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 904 can include a display device, an auxiliary lighting device (e.g., an LED), a haptic feedback device (e.g., a vibration motor), etc. The display device can include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device can be a touch screen.

[0136] Various embodiments of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0137] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0138] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0139] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0140] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server can also be a server of a distributed system, or a server combined with a blockchain.

[0141] It should be understood that various forms of flow shown above can be used, with steps reordered, added, or removed. For example, the steps described in the present application can be executed in parallel, in sequence, or in a different order, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and the present application does not limit this.

[0142] The above detailed description does not limit the scope of the application. Various modifications, combinations, sub-combinations and alternatives can be made to the detailed embodiment disclosed herein without departing from the spirit and the principles of the application. Any modification, equivalent replacement or improvement made within the spirit and principles of the application shall fall within the scope of the application.

Claims

1. An information extraction method, comprising: obtaining a feature vector of a to-be-processed text and a dependency graph of the to-be-processed text; generating a corresponding distance matrix according to the dependency graph, comprising: generating a corresponding adjacency matrix according to the dependency graph; and performing dense processing on the adjacency matrix according to position information of nodes in the to-be-processed text in the dependency graph to obtain the distance matrix; calculating attention values between any two words in the to-be-processed text by using an attention mechanism according to the feature vector of the to-be-processed text and the distance matrix; obtaining semantic feature representation of the to-be-processed text according to the feature vector of the to-be-processed text and the attention values between any two words in the to-be-processed text; performing information extraction on the to-be-processed text according to the semantic feature representation of the to-be-processed text; wherein the performing dense processing on the adjacency matrix according to position information of nodes in the to-be-processed text in the dependency graph to obtain the distance matrix comprises: determining position information of words corresponding to nodes in the to-be-processed text in the dependency graph; in the case that there is no dependency relationship between two nodes in the dependency graph, determining distance values of words corresponding to the two nodes in the to-be-processed text according to the position information, and setting distance values between the two nodes in the adjacency matrix to the distance values of the words corresponding to the two nodes in the to-be-processed text; in the case that there is a dependency relationship between two nodes in the dependency graph, setting distance values between the two nodes in the adjacency matrix to a first numerical value to obtain the distance matrix.

2. The method of claim 1, wherein, the calculating attention values between any two words in the to-be-processed text by using an attention mechanism according to the feature vector of the to-be-processed text and the distance matrix comprises: determining a query vector and a key vector according to the feature vector of the to-be-processed text; performing double-affine calculation according to the query vector and the key vector to obtain a first intermediate matrix, and performing matrix multiplication operation on the query vector and the key vector to obtain a second intermediate matrix; modifying data in the first intermediate matrix according to the distance matrix to obtain a third intermediate matrix; performing matrix addition operation on the second intermediate matrix and the third intermediate matrix to obtain a fourth intermediate matrix; performing scaling, masking and regression function Softmax operation on the fourth intermediate matrix to obtain the attention values between any two words in the to-be-processed text.

3. The method of claim 2, wherein, the modifying data in the first intermediate matrix according to the distance matrix to obtain a third intermediate matrix comprises: performing division operation on data in the first intermediate matrix and data at corresponding positions in the distance matrix to obtain the third intermediate matrix.

4. The method of claim 1, wherein, the calculating attention values between any two words in the to-be-processed text by using an attention mechanism according to the feature vector of the to-be-processed text and the distance matrix comprises: determine a query vector and a key vector according to the feature vector of the to-be-processed text; perform bi-affine calculation on the query vector and the key vector to obtain a first intermediate matrix, and perform matrix multiplication operation on the query vector and the key vector to obtain a second intermediate matrix; perform matrix addition operation on the first intermediate matrix and the second intermediate matrix to obtain a third intermediate matrix; perform scaling, masking and regression function Softmax operation on the third intermediate matrix to obtain a corresponding attention matrix; perform correction and normalization processing on data in the attention matrix according to the distance matrix to obtain an attention value between any two words in the to-be-processed text.

5. The method of claim 4, wherein, The correction and normalization processing on the data in the attention matrix according to the distance matrix to obtain the attention value between any two words in the to-be-processed text comprises: perform division operation on the data in the attention matrix and the data at the corresponding position in the distance matrix to obtain a fourth intermediate matrix; perform normalization processing on the data in the fourth intermediate matrix to obtain the attention value between any two words in the to-be-processed text.

6. The method of any one of claims 1 to 5, wherein, The obtaining of the semantic feature representation of the to-be-processed text according to the feature vector of the to-be-processed text and the attention value between any two words in the to-be-processed text comprises: determine a value vector according to the feature vector of the to-be-processed text; perform product operation on the value vector and the attention value between any two words in the to-be-processed text to obtain the semantic feature representation of the to-be-processed text.

7. The method of any one of claims 1 to 5, wherein, The information extraction on the to-be-processed text according to the semantic feature representation of the to-be-processed text comprises: perform entity recognition on the to-be-processed text based on the semantic feature representation to obtain recognized entity information in the to-be-processed text. 8.An information extraction apparatus, comprising: a first obtaining module configured to obtain a feature vector of a to-be-processed text and a dependency relation graph of the to-be-processed text; a generating module configured to generate a corresponding distance matrix according to the dependency relation graph, comprising: generating a corresponding adjacency matrix according to the dependency relation graph; and performing densification processing on the adjacency matrix according to position information of nodes in the to-be-processed text in the dependency relation graph to obtain the distance matrix; a calculating module configured to calculate an attention value between any two words in the to-be-processed text by using an attention mechanism according to the feature vector of the to-be-processed text and the distance matrix; a second obtaining module configured to obtain a semantic feature representation of the to-be-processed text according to the feature vector of the to-be-processed text and the attention value between any two words in the to-be-processed text; an information extraction module configured to perform information extraction on the to-be-processed text according to the semantic feature representation of the to-be-processed text. The generation module is specifically configured to: determine position information of words corresponding to nodes in the dependency relation graph in the to-be-processed text; in the case that there is no dependency relation between two nodes in the dependency relation graph, determine distance values of the words corresponding to the two nodes respectively in the to-be-processed text according to the position information, and set distance values between the two nodes in the adjacency matrix as the distance values of the words corresponding to the two nodes respectively in the to-be-processed text; in the case that there is a dependency relation between two nodes in the dependency relation graph, set distance values between the two nodes in the adjacency matrix as a first numerical value, to obtain the distance matrix.

9. The apparatus of claim 8, wherein, The calculation module is specifically configured to: determine a query vector and a key vector according to the feature vector of the to-be-processed text; perform bi-affine calculation on the query vector and the key vector to obtain a first intermediate matrix, and perform matrix multiplication operation on the query vector and the key vector to obtain a second intermediate matrix; perform correction on data in the first intermediate matrix according to the distance matrix to obtain a third intermediate matrix; perform matrix addition operation on the second intermediate matrix and the third intermediate matrix to obtain a fourth intermediate matrix; perform per-ratio scaling, masking and regression function Softmax operation on the fourth intermediate matrix to obtain attention values between any two words in the to-be-processed text.

10. The apparatus of claim 9, wherein, The calculation module is specifically configured to: perform division operation on data in the first intermediate matrix and data at corresponding positions in the distance matrix to obtain a third intermediate matrix.

11. The apparatus of claim 8, wherein, The calculation module is specifically configured to: determine a query vector and a key vector according to the feature vector of the to-be-processed text; perform bi-affine calculation on the query vector and the key vector to obtain a first intermediate matrix, and perform matrix multiplication operation on the query vector and the key vector to obtain a second intermediate matrix; perform matrix addition operation on the first intermediate matrix and the second intermediate matrix to obtain a third intermediate matrix; perform per-ratio scaling, masking and regression function Softmax operation on the third intermediate matrix to obtain a corresponding attention matrix; perform correction and normalization processing on data in the attention matrix according to the distance matrix to obtain attention values between any two words in the to-be-processed text.

12. The apparatus of claim 11, wherein, The calculation module is specifically configured to: perform division operation on data in the attention matrix and data at corresponding positions in the distance matrix to obtain a fourth intermediate matrix; perform normalization processing on data in the fourth intermediate matrix to obtain attention values between any two words in the to-be-processed text.

13. The apparatus of any one of claims 8 to 12, wherein, The second acquisition module is specifically configured to: determine a value vector according to the feature vector of the to-be-processed text; perform product operation on the value vector and the attention values between any two words in the to-be-processed text to obtain semantic feature representation of the to-be-processed text.

14. The apparatus of any one of claims 8 to 12, wherein, The information extraction module is specifically configured to: perform entity recognition on the to-be-processed text based on the semantic feature representation to obtain recognized entity information in the to-be-processed text. 15.An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

16. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1 to 7.

17. A computer program product comprising a computer program, wherein, The computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 7.

Images