Open information extraction method and system based on set sequence generation network

Abstract

The application provides an open information extraction method and system based on a set sequence generation network, a storage medium and an electronic device, and relates to the technical field of open information extraction. In the application, the set sequence generation network comprises a text information coding layer, an information tuple coding layer and an information tuple sequence generation layer, the open information extraction task is expressed as a set prediction problem based on the network, the set prediction network and the text generation network are combined, an open information extraction method based on the set sequence generation network is proposed, the relationship between information tuples is effectively processed, and the open information extraction method can be used for information extraction of enterprises, products and technologies in specific fields.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of open information extraction, and particularly relates to an open information extraction method and system based on a set sequence generation network, a storage medium and an electronic device. BACKGROUND

[0002] Open information extraction (Open IE) is a technology that extracts structured representations of information from unstructured text. The extracted information is usually represented as a semantic relationship between a set of arguments and a relational phrase in the form of a triple or n-tuple information.

[0003] In processing sentences containing multiple information tuples, existing Open IE systems are all independently predicting each information tuple, ignoring the relationship between tuples. For example, NeuralOIE models open information extraction as a sequence-to-sequence task and develops a neural open information extraction model using an encoder-decoder framework. Similarly to NeuralOIE, for example, an open information extraction method using a Transformer architecture; a Chinese-oriented sequence generation model Logicial model that uses a coverage mechanism and gated dependency attention to extract relational tuples from sentences; extending NeuralOIE by autoregressively generating multiple tuples, each of which is conditioned on all previously extracted tuples, etc. SUMMARY

[0004] (I) Technical problems solved

[0005] In view of the deficiencies of the prior art, the present application provides an open information extraction method and system based on a set sequence generation network, a storage medium and an electronic device, which solves the technical problem of ignoring the relationship between tuples.

[0006] (II) Technical solutions

[0007] To achieve the above object, the present application is implemented by the following technical solutions:

[0008] An open information extraction method based on a set sequence generation network, characterized in that the set sequence generation network comprises a text information encoding layer, an information tuple encoding layer and an information tuple sequence generation layer; the open information extraction method comprises:

[0009] S1, obtaining a plurality of sentences to be processed;

[0010] S2, taking each sentence as the input of the text information encoding layer to obtain the corresponding context character embedding;

[0011] S3, generating a set of information tuple queries with the same size for each sentence, and taking the context character embeddings as the input of the information tuple encoding layer, obtaining the hidden state of each information tuple based on the encoder of Transformers;

[0012] S4, taking the context character embeddings and hidden states as the input of the information tuple sequence generation layer, and decoding to obtain the corresponding information tuple sequence based on the single-layer LSTM network.

[0013] Preferably, BERT is used as the text information encoder in S2, as follows:

[0014]

[0015] wherein, represents the d1-dimensional context character embedding of character w i .

[0016] Preferably, S3 includes:

[0017] A set of information tuple queries with the same size is generated for each sentence, which is initialized by m learnable embedding vectors Q=(q1, q2,..., q m m represents a hyperparameter;

[0018] The encoder based on Transformers is composed of N identical Transformers modules; in any Transformers module, the relationship between information tuples is modeled by a non-masked multi-head self-attention network, and the relationship between information tuples and their corresponding input sentences is modeled by a non-masked multi-head cross-attention network; as follows:

[0019]

[0020] wherein, d k is the dimension of the mapping vector, and u, k and v represent the query mapping vector, the key mapping vector and the value mapping vector, respectively; for the self-attention network, u, k and v are linearly transformed from the information tuple query; for the cross-attention network, the difference is that v is linearly transformed from the context character embedding;

[0021] The m information tuple queries are converted into the hidden state of the m information tuples: Q′=(q1′, q2′,..., q m ′).

[0022] Preferably, S4 includes:

[0023] The hidden state of the current character at time step t is generated as follows: ​

[0024]

[0025] wherein, q′ i , respectively represent the hidden state of the previous predicted character, the embedding vector of the previous predicted character, the hidden state of the i-th information tuple, and the sentence encoding vector obtained by the attention mechanism and the copy mechanism;

[0026] adopt a linear layer to project to the target character table V, and calculate the probability distribution of the current predicted character by SoftMax operation

[0027]

[0028] wherein, respectively represent the weight matrix and the bias of the projection layer, and |V| represents the size of the output character table;

[0029] The character with the highest probability is selected as the predicted character at time step t, and m variable-length information tuples Y = {Y1, Y1, …, Y m} are obtained, wherein the information tuple sequence The start symbol, the end symbol and the predicted character at time step t of the j-th information tuple sequence are represented by and respectively.

[0030] Preferably, the obtaining process of the sentence encoding vector obtained by the attention mechanism comprises:

[0031]

[0032]

[0033]

[0034] wherein, w g and b g respectively represent the corresponding weight matrix and bias, represents the vector of character w i , tanh represents the tangent function, exp represents the exponential with natural logarithm e as the base, represents the state of the j-th information tuple at time t-1;

[0035] and / or the obtaining process of the sentence encoding vector obtained by the copy mechanism comprises:

[0036]

[0037] wherein, wc and b c respectively denote the corresponding weight matrix and bias, and tanh denotes the hyperbolic tangent function.

[0038] Preferably, after obtaining the hidden state of each information tuple in S3, the validity of the hidden state of each information tuple is determined, specifically including:

[0039] An independent output gate is used to calculate the hidden state q' of the jth information tuple j The effective probability o j , as shown below:

[0040] o j = softmax(W q q' + b j ) q

[0041] wherein, b q ∈ R 2 respectively denote the weight matrix and bias of the output gate;

[0042] If o j is greater than or equal to a preset threshold, it is determined that the hidden state q' of the jth information tuple is valid; otherwise, it is invalid. j

[0043] Preferably, S4 further includes:

[0044] According to the validity of the hidden state of each information tuple, the corresponding information tuple sequence is filtered to obtain the final valid information tuple sequence.

[0045] Preferably, in the training stage of the set sequence generation network, a bipartite matching loss function is selected, specifically as follows:

[0046] (1) The loss function of all matched predicted information tuples and real information tuples

[0047]

[0048]

[0049]

[0050] wherein, π* ∈ Ω n denotes the permutation with the lowest cost containing n objects, Ω n denotes the space of all permutations of n objects, is the pair-wise matching cost between the real information tuple Y j and the predicted information tuple with index π(j) in the permutation;

[0051] ​​(2) Information tuple validity prediction loss selects cross-entropy loss function

[0052] Defining the predicted information tuple belonging to the best bipartite matching is equivalent to the real information tuple, indirectly obtaining the real validity of the predicted information tuple; the predicted validity of the predicted information tuple is obtained through the independent output gate; then

[0053]

[0054] Wherein, o and respectively represent the real validity and the predicted validity of the predicted information tuple; o j and respectively represent the real validity and the predicted validity of the jth predicted information tuple.

[0055] (3) Combined loss function

[0056]

[0057] Wherein, σ1 and σ2 are respectively and learnable weights.

[0058] An open information extraction system based on a set sequence generation network, the set sequence generation network comprising a text information encoding layer, an information tuple encoding layer and an information tuple sequence generation layer; the open information extraction system comprising:

[0059] An acquisition module for acquiring a plurality of sentences to be processed;

[0060] An embedding module for taking each sentence as an input of the text information encoding layer to obtain a corresponding context character embedding;

[0061] An encoding module for generating a group of information tuples with the same size for each sentence, taking the context character embedding as an input of the information tuple encoding layer, and obtaining a hidden state of each information tuple based on a Transformers-based encoder;

[0062] A decoding module for taking the context character embedding and the hidden state as an input of the information tuple sequence generation layer, and decoding to obtain a corresponding information tuple sequence based on a single-layer LSTM network.

[0063] A storage medium storing a computer program for open information extraction based on a set sequence generation network, wherein the computer program causes a computer to execute the open information extraction method as described above.

[0064] An electronic device comprising:

[0065] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs comprising instructions for performing the open information extraction method as described above.

[0066] (III) Beneficial Effects

[0067] The present application provides an open information extraction method, system, storage medium and electronic equipment based on a set sequence generation network. Compared with the prior art, the following beneficial effects are achieved:

[0068] In the present application, the set sequence generation network includes a text information encoding layer, an information tuple encoding layer and an information tuple sequence generation layer. The open information extraction task is expressed as a set prediction problem based on the network, and the set prediction network and the text generation network are combined to propose an open information extraction method based on a set sequence generation network, which effectively handles the relationship between information tuples and can be used for information extraction of enterprise-level, product-level and technology-level specific fields. BRIEF DESCRIPTION OF DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0070] Figure 1 A whole framework diagram of a set sequence generation network provided by the embodiment of the present application;

[0071] Figure 2 A block diagram of an open information extraction method based on a set sequence generation network provided by the embodiment of the present application;

[0072] Figure 3 A comparison diagram of improved and unimproved bipartite matching loss provided by the embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme in the embodiments of the present application is described clearly and completely. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0074] The embodiment of the application provides an open information extraction method and system based on a set sequence generation network, a storage medium and an electronic device, and solves the technical problem of ignoring the relationship between tuples.

[0075] The technical solution in the embodiment of the application is as follows to solve the above technical problem:

[0076] In view of the problems in the background art, the embodiment of the application expresses the open information extraction task as a set prediction problem, combines a set prediction network and a text generation network, and proposes an open information extraction method and system based on a set sequence generation network.

[0077] As shown in the formula (I), the Open IE system (namely, the set sequence generation network) proposed in the embodiment of the application comprises three network structures: a text information encoding layer, an information tuple encoding layer and an information tuple sequence generation layer. Specifically: Figure 1

[0078] Firstly, in the text information encoding layer, each character in the input sentence is expressed as a vector based on the bidirectional context thereof.

[0079] Secondly, in the information tuple encoding layer, each information tuple is initialized through a learnable embedding vector (query vector), and the hidden state of the information tuple is generated by modeling the relationship between information tuples and the relationship between the information tuple and the corresponding input sentence. The validity of each information tuple is also predicted through an independent output gate.

[0080] Finally, in the information tuple sequence generation layer, the hidden state of the information tuple is decoded, and each information tuple sequence is generated using a copy attention mechanism.

[0081] In order to better understand the above technical solution, the above technical solution will be described in detail in combination with the drawings of the specification and the specific embodiments.

[0082] Embodiment 1:

[0083] As shown in the formula (I), the Open IE system (namely, the set sequence generation network) proposed in the embodiment of the application comprises three network structures: a text information encoding layer, an information tuple encoding layer and an information tuple sequence generation layer. Specifically: Figure 2

[0084] S1, obtaining a plurality of sentences to be processed;

[0085] S2, taking each sentence as the input of the text information encoding layer to obtain the corresponding context character embedding;

[0086] ​​S3, generate a set of information tuple queries of the same size for each sentence, and take the context character embedding as the input of an information tuple encoding layer, obtain the hidden state of each information tuple based on a Transformer-based encoder;

[0087] S4, take the context character embedding and the hidden state as the input of an information tuple sequence generation layer, obtain the corresponding information tuple sequence based on a single-layer LSTM network.

[0088] The embodiment of the application formulates the open information extraction task as a set prediction problem, and combines a set prediction network and a text generation network, and proposes an open information extraction method and system based on a set sequence generation network, which effectively processes the relationship between information tuples and can be used for information extraction of enterprises, products, and technologies in specific fields.

[0089] Next, each step of the above technical solution will be described in detail:

[0090] In step S1, a plurality of sentences to be processed are obtained.

[0091] In step S2, each sentence is taken as the input of a text information encoding layer to obtain the corresponding context character embedding.

[0092] The purpose of this step is to represent each character in the input sentence as a vector, and BERT is specifically used as a text information encoder. In the text information encoding layer, the input sentence is first segmented into continuous characters (w1, w2, …, w l ) by the BERT tokenizer. The tokenizer marks the beginning and end of the sentence with special characters [CLS] and [SEP] respectively. Next, the characters are passed to the BERT model to generate their corresponding vector representations:

[0093]

[0094] wherein, represents the d1-dimensional context character embedding of the character w i .

[0095] In step S3, a set of information tuple queries of the same size is generated for each sentence, and the context character embedding is taken as the input of an information tuple encoding layer, and the hidden state of each information tuple is obtained based on a Transformer-based encoder.

[0096] The step encodes the relationship between information tuples and the relationship between information tuples and their corresponding input sentences based on a Transformers-based encoder. The Transformers-based encoder has two advantages: it can encode information tuples based on bidirectional information, and it avoids learning the extraction order between information tuples.

[0097] Correspondingly, the S3 includes:

[0098] Before encoding, a set of information tuple queries of the same size is generated for each sentence, which is composed of m learnable embedding vectors Q = (q1, q2, …, qm), where m represents a hyperparameter, and the hyperparameter m is set to a value that is significantly larger than the typical number of information tuples in most sentences. m ) initialization.

[0099] Subsequently, encoding is performed using a Transformers-based encoder composed of N identical Transformers modules for self-attention and cross-attention calculations. In any Transformers module, the relationship between information tuples is modeled by a non-masked multi-head self-attention network, and the relationship between information tuples and their corresponding input sentences is modeled by a non-masked multi-head cross-attention network.

[0100] Specifically, each multi-head attention network simultaneously uses multiple attention modules for individual calculations to capture semantic information from multiple levels, and the output of the multi-head attention network is a concatenation of the results of individual attention network calculations. A single attention module uses a query mapping vector and a set of key-value mapping vectors as input and outputs a weighted sum of value mapping vectors. The complete operation can be summarized by the following expression:

[0101]

[0102] where d k is the dimension of the mapping vector, u, k, and v represent the query mapping vector, the key mapping vector, and the value mapping vector, respectively; for self-attention networks, u, k, and v are linearly transformed from information tuple queries; for cross-attention networks, the difference is that v is linearly transformed from context character embeddings;

[0103] Convert m information tuple queries into m information tuple hidden states: Q' = (q1', q2', …, qm'). m

[0104] ​The hidden state of each information tuple is decoded into an information tuple sequence in the subsequent information tuple sequence generation layer. However, the number of valid information tuples for most sentence generation is much smaller than m, and therefore a large part of the generated information tuple sequence will be redundant extraction. In order to prevent these redundant extractions from affecting the performance of the method, the embodiment of the present application further pre-judges the validity of each information tuple hidden state in this step before sending it to the subsequent network layer.

[0105] Specifically, the probability o j is calculated by an independent output gate as follows:

[0106] o j = softmax(W q q j ′+b q )

[0107] wherein, b q ∈R 2 respectively represent the weight matrix and bias of the output gate;

[0108] If o j is greater than or equal to a preset threshold (such as 0.5), it is determined that the jth information tuple hidden state q′ j is valid; otherwise, it is invalid.

[0109] In step S4, the context character embedding and the hidden state are taken as inputs of the information tuple sequence generation layer, and a corresponding information tuple sequence is obtained by decoding based on a single-layer LSTM network.

[0110] The purpose of this step is to decode the hidden state of the information tuple by combining the input sentence to generate an information tuple sequence, and a single-layer LSTM network is used as a decoder.

[0111] Correspondingly, the S4 includes:

[0112] First, the hidden state q is generated for the current character at time step t as follows:

[0113]

[0114] wherein, q′ i , respectively represent the hidden state of the last predicted character, the embedding vector of the previous predicted character, the hidden state of the ith information tuple, and the sentence encoding vector obtained by the attention mechanism and the copy mechanism;

[0115] Subsequently, a linear layer is used to obtain is projected to the target character vocabulary V and the probability distribution over the current predicted character is computed by a SoftMax operation

[0116]

[0117] where, and b denote the weight matrix and bias of the projection layer respectively, and |V| denotes the size of the output character vocabulary;

[0118] Finally, the character with the highest probability is selected as the predicted character at time step t, and m variable-length information tuples Y = {Y1, Y1,..., Y m}, where the sequence of information tuples The start symbol, end symbol and predicted character at time step t of the jth information tuple sequence are denoted by and respectively.

[0119] The attention mechanism is commonly used in generative networks, responsible for updating the weights of the encoder hidden states in real time according to the decoder hidden states at each time step. Specifically, the sentence encoding vector is computed as the weighted sum of the context character embeddings. This process is represented by the following expression:

[0120]

[0121]

[0122]

[0123] where w g and b g denote the corresponding weight matrix and bias respectively, denotes the vector of character w i , tanh denotes the hyperbolic tangent function, exp denotes the exponential with base natural logarithm e, denotes the state of the jth information tuple at time step t-1

[0124] The copy mechanism is designed for specific generative networks where the characters in the output sequence are mostly from the input sequence. The copy mechanism is very suitable for the Open IE task because most of the output characters are directly from the input sentence. The method uses a simplified version of the copy mechanism to generate the sequence of information tuples, where the sentence encoding vector can be computed by the following formula:

[0125]

[0126] where w c and bc respectively, and tanh denotes the hyperbolic tangent function.

[0127] The output of the information tuple sequence generation is based on the above two mechanisms, i.e., m information tuple sequences. Then, the generated results are filtered using the information tuple effectiveness calculated in the previous step, i.e., the final effective information tuple sequence is obtained.

[0128] In particular, the embodiments of the present application recognize that since the prediction of each information tuple is unordered, the above task belongs to a set prediction problem, and a bipartite matching loss function needs to be used during model training. The bipartite matching loss regards the matching of multiple predicted values and true values as a assignment problem in operations research, and uses the Hungarian algorithm to process it. The Hungarian algorithm can find the best bipartite matching between predicted values and true values. The bipartite matching loss usually contains three calculation steps: 1) calculate the cost matrix generated by pairing predicted values and true values two by two; 2) find the optimal (i.e. lowest cost) bipartite matching using the Hungarian algorithm; 3) calculate the loss function using the optimal bipartite matching.

[0129] To solve this problem, the embodiments of the present application design a learning strategy using partial matching to speed up the training process. The proposed learning strategy only uses the first n predicted information tuples to perform bipartite matching with the true information tuples (C∈R n*n ). The proposed learning strategy is also called "differentiated learning strategy" in the embodiments of the present application, because it can make each decoder learn the information tuple sequence generation method under different information conditions.

[0130] More specifically, when the number of true information tuples is different, the information received by each decoder is different. For example, when there is only one true information tuple, only the first decoder will participate in training, and when there are two true information tuples, only the first two decoders will participate in training.

[0131] Figure 3 The matching schematic diagram of the bipartite matching loss before and after improvement is shown.

[0132] The process of calculating the difference between the predicted information tuples and the true information tuples using the improved bipartite matching loss will be described in detail below:

[0133] After creating the cost matrix, the Hungarian algorithm is used to find the best bipartite matching between the predicted information tuples and the true information tuples. Specifically, the permutation π*∈Ω n that contains n objects is searched, which has the lowest cost.

[0134]

[0135] where Ωn a space representing all permutations of n objects, is the real information tuple Y j and the pairwise matching cost between the predicted information tuple indexed by π(j) in the permutation and the real information tuple. The matching cost is obtained by summing the negative log probability of each element in the real information tuple, i.e.

[0136]

[0137] The loss function of all matching predicted information tuples and real information tuples is given by the following formula:

[0138]

[0139] Although the information tuple encoding layer describes how to calculate the effectiveness of the predicted information tuple, the real effectiveness of the predicted information tuple is unknown. Since the predicted information tuple belonging to the optimal bipartite matching is equivalent to the real information tuple, the real effectiveness of the predicted information tuple can be generated indirectly. Specifically, if the predicted information tuple belongs to the optimal bipartite matching, its real effectiveness is set to True; otherwise, it is set to False. The information tuple effectiveness prediction loss is given by the cross-entropy loss function:

[0140]

[0141] where o and respectively represent the real effectiveness and the predicted effectiveness of the predicted information tuple; o j and respectively represent the real effectiveness and the predicted effectiveness of the jth predicted information tuple;

[0142] The above two losses are independent of each other and have different classification targets. Therefore, this belongs to a multi-task learning problem. The relative weight of each loss function in the multi-task learning system will significantly affect the performance of the model. Therefore, the embodiment of the present application uses homoscedastic uncertainty to weigh the two losses to further improve the training performance, so as to obtain a combined loss function:

[0143]

[0144] where σ1 and σ2 are the learnable weights of and respectively.

[0145] In order to better prove the superiority of the open information extraction method provided by the embodiment of the present application, the following specific examples are provided:

[0146] Model training: A large-scale Chinese open semantic relation extraction dataset, the SAOKE dataset, was used to develop the model. The dataset contains 46930 sentences randomly collected from Baidu Baike, each sentence is annotated with information tuples by trained workers, a total of 120258 information tuples. Subsequently, the SAOKE dataset is randomly divided into training set, validation set and test set according to the ratio of 8:1:1 to develop the model proposed in the patent.

[0147] Specifically, taking cars as an example, a large number of car-related articles for open information extraction were collected from multiple encyclopedia websites (Baidu Encyclopedia, Quick Encyclopedia), car portal websites (Car Home, Pacific Auto Network) and knowledge websites (Zhihu). The data collection rules are as follows: 1) Encyclopedia website: select popular car type encyclopedia description articles; 2) Portal website: search for hot car type launch / professional review articles; 3) Knowledge website: search for car professional knowledge articles with “car design” and “car research and development” as keywords. Then, through clause cutting processing of the collected articles, a total of 1533 effective sentences were obtained.

[0148] Through the use of the trained model to perform open information extraction on the above data, a total of 3984 effective information tuples were obtained. Each information tuple contains a relation phrase and at least one argument, wherein the relation phrase is the combination of the predicate and the qualifier, and the argument 2 is the combination of the object and the time and place (distinguished by the symbol “#”).

[0149] Several representative fragmented product information extraction cases are shown in Tables 1-3:

[0150] Table 1 Case 1

[0151]

[0152] Table 2 Case 2

[0153]

[0154] Table 3 Case 3

[0155]

[0156] The embodiment of the application also provides an open information extraction system based on a set sequence generation network, the set sequence generation network comprising a text information encoding layer, an information tuple encoding layer and an information tuple sequence generation layer; the open information extraction system comprising:

[0157] An acquisition module is configured to acquire a plurality of sentences to be processed.

[0158] An embedding module is configured to take each sentence as an input of the text information encoding layer to obtain a corresponding context character embedding.

[0159] An encoding module is configured to generate a set of information tuple queries of the same size for each sentence, and input the context character embedding as an information tuple encoding layer into a Transformer-based encoder to obtain a hidden state of each information tuple;

[0160] A decoding module is configured to input the context character embedding and the hidden state as an information tuple sequence generation layer into a single-layer LSTM network to decode to obtain a corresponding information tuple sequence.

[0161] The embodiment of the present application also provides a storage medium storing a computer program for open information extraction based on a set sequence generation network, wherein the computer program causes a computer to execute the open information extraction method.

[0162] The embodiment of the present application also provides an electronic device, comprising:

[0163] one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs comprise a program for executing the open information extraction method.

[0164] It can be understood that the open information extraction system based on a set sequence generation network, the storage medium and the electronic device provided by the embodiment of the present application correspond to the open information extraction method based on a set sequence generation network provided by the embodiment of the present application, and the explanation, examples and beneficial effects of the related content can refer to the corresponding part in the open information extraction method, which will not be repeated here.

[0165] To sum up, compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0166] 1、The embodiment of the present application formulates the open information extraction task as a set prediction problem, and combines a set prediction network and a text generation network to propose an open information extraction method based on a set sequence generation network, which effectively processes the relationship between information tuples and can be used for information extraction of enterprises, products and technologies in specific fields.

[0167] 2、The Open IE system proposed by the embodiment of the present application can learn the relationship between information tuples in a non-autoregressive manner to avoid learning the extraction order of information tuples; and an independent output gate is used to evaluate the effectiveness of the generated information tuples, thereby effectively preventing redundant extraction.

[0168] 3、The embodiment of the present application proposes a differentiated learning strategy to improve the stability of the binary matching loss, thereby improving the training speed to realize fast convergence.

[0169] It is to be noted that the relationship terms, such as first and second, etc., are used only to differentiate one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Also, the terms "comprising", "containing" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0170] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent ones; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An open information extraction method based on a set sequence generation network, characterized in that: The set sequence generation network includes a text information encoding layer, an information tuple encoding layer, and an information tuple sequence generation layer; the open information extraction method includes: S1. Obtain several sentences to be processed; S2. Take each sentence as the input of the text information encoding layer and obtain the corresponding context character embedding; S3. Generate a set of information tuple queries of the same size for each sentence, and use the context character embedding as the input of the information tuple encoding layer. The Transformers-based encoder obtains the hidden state of each information tuple. S4, using the context character embedding and hidden state as the input of the information tuple sequence generation layer, and decoding based on a single-layer LSTM network to obtain the corresponding information tuple sequence; The S3 includes: Generate a set of information tuple queries of the same size for each sentence, which consists of m learnable embedding vectors Q = (q1, q2, ..., q m ) initialization, where m represents the hyperparameter; The Transformers-based encoder consists of N identical Transformers modules. In any Transformers module, the relationship between information tuples is modeled by an unmasked multi-head self-attention network, and the relationship between information tuples and their corresponding input sentences is modeled by an unmasked multi-head cross-attention network. Among them, d k is the dimension of the mapping vector, u, k, and v represent the query mapping vector, key mapping vector, and value mapping vector, respectively; for the self-attention network, u, k, and v are linearly transformed from the information tuple query; for the cross-attention network, the difference is that v is linearly transformed from the context character embedding; Convert the m information tuple queries into the hidden states of m information tuples: Q′=(q1′, q2′, …, q m ′).

2. The open information extraction method based on the set sequence generation network according to claim 1, characterized in that: In S2, BERT is used as the text information encoder, as shown below: in, Represents the character w i The contextual character embedding of d1 dimension.

3. The open information extraction method based on the set sequence generation network according to claim 1, characterized in that: The S4 includes: Generate hidden state for the current character at time step t As shown below: in, Represent the hidden state of the last predicted character, the embedding vector of the previous predicted character, the hidden state of the i-th information tuple, and the sentence encoding vector obtained by the attention mechanism and the copy mechanism; Using a linear layer Project it to the target character table V and calculate the probability distribution of the current predicted character through the SoftMax operation in, Represent the weight matrix and bias of the projection layer respectively, |V| represents the size of the output character table; Select the character with the highest probability as the predicted character at time step t, and obtain m variable-length information tuples Y = {Y1, Y1, ..., Y m }, where the information tuple sequence The starting symbol, ending symbol and predicted character of the j-th information tuple sequence at time step t are represented by and express.

4. The open information extraction method based on a set sequence generation network according to claim 3, characterized in that: The process of obtaining the sentence encoding vector obtained by the attention mechanism includes: Among them, w g and b g Represent the corresponding weight matrix and bias respectively, Represents the character w i A vector of , tanh represents the tangent function, exp represents the exponential with the natural logarithm e as the base, Represents the state of the j-th information tuple at time t-1; And / or the acquisition process of the sentence encoding vector acquired by the replication mechanism includes: Among them, w c and b c Represent the corresponding weight matrix and bias respectively, and tanh represents the tangent function.

5. The open information extraction method based on a set sequence generation network according to claim 1, characterized in that: After obtaining the hidden state of each information tuple in S3, the validity of the hidden state of each information tuple is determined, which specifically includes: Use an independent output gate to calculate the hidden state q′ of the jth information tuple j The probability of effectiveness j , as shown below: o j =softmax(W q q′ j +b q ) in, b q ∈R 2 Represent the weight matrix and bias of the output gate respectively; If o j If it is greater than or equal to the preset threshold, the hidden state q′ of the jth information tuple is determined j Valid; otherwise invalid; Said S4 further comprises: According to the validity of the hidden state of each information tuple, the corresponding information tuple sequence is filtered to obtain a final valid information tuple sequence.

6. The open information extraction method based on a set sequence generation network according to claim 5, characterized in that: During the training phase of the set sequence generation network, a bipartite matching loss function is selected, as follows: (1) Loss function of all matched predicted information tuples and true information tuples Among them, π*∈Ω n represents the lowest cost permutation containing n objects, Ω n represents the space of all permutations of n objects, is the real information tuple Y j The pairwise matching cost between the predicted information tuple with index π(j) in the permutation; (2) The cross entropy loss function is used to predict the loss of information tuple validity. The prediction information tuple belonging to the best bipartite matching is defined as equivalent to the real information tuple, and the real validity of the prediction information tuple is indirectly obtained; the prediction validity of the prediction information tuple is obtained through the independent output gate; then Among them, o and Represent the true validity and predicted validity of the prediction information tuple respectively; o j and They represent the true validity and predicted validity of the j-th prediction information tuple respectively; (3) Combined loss function Among them, σ1 and σ2 are and The learnable weights.

7. An open information extraction system based on a set sequence generation network, characterized in that: The set sequence generation network includes a text information encoding layer, an information tuple encoding layer and an information tuple sequence generation layer; The open information extraction system is used to execute the open information extraction method based on the set sequence generation network according to claim 1, comprising: An acquisition module is used to acquire a number of sentences to be processed; The embedding module is used to take each sentence as the input of the text information encoding layer and obtain the corresponding context character embedding; The encoding module is used to generate a set of information tuple queries of the same size for each sentence, and use the context character embedding as the input of the information tuple encoding layer, and obtain the hidden state of each information tuple based on the Transformers encoder; The decoding module is used to embed the context characters and the hidden state as the input of the information tuple sequence generation layer, and decode and obtain the corresponding information tuple sequence based on a single-layer LSTM network.

8. A storage medium, characterized in that: The computer program for extracting open information based on a set sequence generation network is stored therein, wherein the computer program enables a computer to execute the open information extraction method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including instructions for executing the open information extraction method according to any one of claims 1 to 6.

Images