Information extraction method and device

Abstract

The present invention discloses an information extraction method and device, which converts a text to be extracted into a first target input vector; passes the first target input vector to a subject extraction model to obtain a target subject, and the subject extraction model is constructed based on a first Bert model, a first fully connected layer, and a first sigmoid function; the target subject is spliced ​​with the text to be extracted and converted into a second target input vector; the second target input vector is passed to a prediction object and relationship model to obtain a target object and a target relationship, and the prediction object and relationship model is constructed based on a second Bert model, a second fully connected layer, a second sigmoid function, a relative distance, a punctuation state vector, and a subject vector. In the above process, the target subject, the target object, and the target relationship are extracted based on the subject extraction model and the prediction object and relationship model, and the extraction based on the model ensures the accuracy of the extraction, and manpower is saved because no human participation is required.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to an information extraction method and device. Background Art

[0002] An important step in building a knowledge graph is the extraction of triples, where triples refer to the extraction of subjects, objects, and the relationship between subjects and objects from text data. In the knowledge graph, data is stored in the form of nodes and the relationship between nodes. For example, in the financial field, the financial situation between companies can be constructed through the knowledge graph, which is of great significance for analyzing the systemic risk of the entire market and the spread of risks in various companies. In order to analyze these shareholding information and debt information, it is necessary to extract these structured information from the company's announcement. For example, a company's announcement is described as follows: "Company A holds 30% of the outstanding shares of Company B, and Company A holds Company B's bonds with an annual interest rate of 6% with a remaining maturity of ten years." For this sentence, we hope to extract two pieces of structured information: the first one: subject: Company A, object: Company B, relationship: equity investment, shareholding type: outstanding shares, shareholding ratio: 30%. The second one: subject: Company A, object: Company B, relationship: bond financing, maturity time: ten years, annual interest rate: 6%. Once we obtain the relevant triple data, we can store this data in a graph database or relational database to build a knowledge graph.

[0003] The inventors have found that the use of manpower to extract triples in the prior art is not only time-consuming and labor-intensive, but also has a high error rate. Summary of the invention

[0004] In view of this, the present invention provides an information extraction method and device, which is used to solve the problem that the existing technology uses manpower to extract triples, which is not only time-consuming and labor-intensive, but also has a high error rate. The specific scheme is as follows:

[0005] An information extraction method, comprising:

[0006] Receiving a text to be extracted, and converting the text to be extracted into a first target input vector;

[0007] Passing the first target input vector to a subject extraction model for extraction to obtain a target subject, wherein the subject extraction model is constructed based on a first Bert model, a first fully connected layer, and a first sigmoid function;

[0008] Splicing the target subject with the text to be extracted, and converting the spliced ​​text into a second target input vector;

[0009] The second target input vector is passed to the prediction object and relationship model for extraction to obtain the target object and the target relationship, wherein the prediction object and relationship model is constructed based on a second Bert model, a second fully connected layer, a second sigmoid function, a relative distance, a punctuation state vector and a subject vector.

[0010] The above method may optionally convert the text to be extracted into a first target input vector, including:

[0011] Segmenting the text to be extracted according to the granularity of characters to obtain individual characters;

[0012] Convert each word into a content representation vector and a position representation vector;

[0013] The content representation vector and the position representation vector are superimposed to obtain a first target input vector.

[0014] The above method, optionally, passes the first target input vector to a subject extraction model for subject extraction to obtain a target subject, including:

[0015] Passing the first target input vector to the first Bert model to obtain a first semantic vector;

[0016] The first semantic vector is passed to the first sigmoid function of the first fully connected layer to predict the target subject in the text to be extracted.

[0017] The above method, optionally, passes the second target input vector to a prediction object and relationship model to obtain a target object and a target relationship, including:

[0018] Passing the second target input vector to the second Bert model to obtain a second semantic vector;

[0019] Adding the relative distance, the punctuation state vector and the subject vector to the second semantic vector to obtain a third semantic vector;

[0020] Determine the correlation between the subject words and the non-subject words in the third semantic vector, and add the product of the correlation and the subject vector to the corresponding third semantic vector to obtain a target semantic vector;

[0021] The target semantic vector is passed to the second sigmoid function of the second fully connected layer to predict the target object and target relationship in the text to be extracted.

[0022] The above method may optionally further include: the subject extraction model and the prediction object and relationship model are at least one;

[0023] Acquire each triple, wherein the triple includes: a target subject, a target object, and a target relationship;

[0024] Selecting a triplet to be predicted from the triples;

[0025] Passing the triple to be predicted to a triple prediction model to obtain a rational probability of the triple combination to be predicted, wherein the triple prediction model is constructed based on a third Bert model, a third fully connected layer and a third sigmoid function;

[0026] Select the predicted triples whose probability values ​​are greater than the preset probability threshold from each rationality probability as the target triples

[0027] An information extraction device, comprising:

[0028] A first conversion module, configured to receive a text to be extracted and convert the text to be extracted into a first target input vector;

[0029] A first extraction module, used for passing the first target input vector to a subject extraction model for extraction to obtain a target subject, wherein the subject extraction model is constructed based on a first Bert model, a first fully connected layer and a first sigmoid function;

[0030] A second conversion module, used for concatenating the target subject with the text to be extracted, and converting the concatenated text into a second target input vector;

[0031] The second extraction module is used to pass the second target input vector to the prediction object and relationship model for extraction to obtain the target object and target relationship, wherein the prediction object and relationship model is constructed based on a second Bert model, a second fully connected layer, a second sigmoid function, a relative distance, a punctuation state vector and a subject vector.

[0032] In the above device, optionally, the first conversion module includes:

[0033] A word segmentation unit is used to segment the text to be extracted according to the granularity of words to obtain individual words;

[0034] A conversion unit, used for converting each word into a content representation vector and a position representation vector;

[0035] The superposition unit is used to superimpose the content representation vector and the position representation vector to obtain a first target input vector.

[0036] In the above device, optionally, the first extraction module includes:

[0037] A first computing unit, configured to pass the first target input vector to the first Bert model to obtain a first semantic vector;

[0038] The first prediction unit is used to pass the first semantic vector to the first sigmoid function of the first fully connected layer to predict the target subject in the text to be extracted.

[0039] In the above device, optionally, the second extraction module includes:

[0040] A second computing unit, configured to pass the second target input vector to the second Bert model to obtain a second semantic vector;

[0041] A first adding unit, configured to add the relative distance, the punctuation state vector and the subject vector to the second semantic vector to obtain a third semantic vector;

[0042] A second adding unit is used to determine the correlation between the subject words and the non-subject words in the third semantic vector, and add the product of the correlation and the subject vector to the corresponding third semantic vector to obtain a target semantic vector;

[0043] The second prediction unit is used to pass the target semantic vector to the second sigmoid function of the second fully connected layer to predict the target object and target relationship in the text to be extracted.

[0044] The above device may optionally further include: at least one of the subject extraction model and the predicted object and relationship model;

[0045] An acquisition module, used for acquiring each triple, wherein the triple includes: a target subject, a target object and a target relationship;

[0046] A first selection module, used for selecting a triple to be predicted from the triples;

[0047] A prediction module, used for passing the triple to be predicted to a triple prediction model to obtain a rational probability of the triple combination to be predicted, wherein the triple prediction model is constructed based on a third Bert model, a third fully connected layer and a third sigmoid function;

[0048] The second selection module selects the to-be-predicted triples whose probability values ​​are greater than a preset probability threshold value from among the various rationality probabilities as the target triples.

[0049] Compared with the prior art, the present invention has the following advantages:

[0050] The present invention discloses an information extraction method and device, including: receiving a text to be extracted, converting each word in the text to be extracted into a first target input vector; passing the first target input vector to a subject extraction model for extraction to obtain a target subject, wherein the subject extraction model is constructed based on a first Bert model, a first fully connected layer, and a first sigmoid function; splicing the target subject with the text to be extracted, and converting the spliced ​​text into a second target input vector; passing the second target input vector to a predicted object and relationship model for extraction to obtain a target object and a target relationship, wherein the predicted object and relationship model is constructed based on a second Bert model, a second fully connected layer, a second sigmoid function, a relative distance, a punctuation state vector, and a subject vector. In the above process, based on the subject extraction model and the predicted object and relationship model, the target subject, the target object, and the target relationship are extracted based on the model to ensure the accuracy of the extraction, and because no manual participation is required, manpower is saved.

[0051] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 A flow chart of an information extraction method disclosed in an embodiment of the present application;

[0054] Figure 2 This is a structural block diagram of an information extraction device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

[0057] The present invention discloses an information extraction method and apparatus, which is applied to obtaining triples in text data during the construction of a knowledge graph. The triples of the text data include: a subject, an object, and the relationship between the subject and the object. The execution process of the method is as Figure 1 shown, and includes the steps:

[0058] S101. Receive the text to be extracted, and convert the text to be extracted into a first target input vector;

[0059] In an embodiment of the present invention, the text to be extracted is received. The text to be extracted may be a paragraph or a sentence. The text to be extracted is split according to the character granularity to obtain each character. For example, for "Zhang San likes to eat apples", after word segmentation, it becomes ["Zhang", "San", "Xi", "Huan", "Chi", "Ping", "Guo"]. Each character is converted into a content representation vector and a position representation vector. The content vector represents the meaning of the word, and the position vector represents the position of the word in the whole sentence. The reason for having an additional position representation vector is to convey the order information of a word in the sentence to the model. For example, the words in "Zhang San likes to eat apples" and "Likes to eat apples Zhang San" are exactly the same, but the order of the words is different. Therefore, they are different sentences, so a position representation vector needs to be prepared so that the model knows that it wants to express "Zhang San wants to eat apples" rather than "Wants to eat apples Zhang San". For the content representation vector: each word has a corresponding 512-dimensional vector in the Bert model to represent it, which is the content representation vector. For the position representation vector: the Bert model uses absolute position encoding, that is, a vector is prepared separately for each position. For example, the x1 vector represents position 1, the x2 vector represents position 2, and so on. For example, "Zhang" is the first word in the sentence, so its position is represented by the x1 vector, and "Guo" is the 6th word in the sentence, so this position is represented by the x6 vector. Further, after determining the content representation vector and the position representation vector, the content representation vector and the position representation vector are superimposed to obtain a first target input vector.

[0060] S102, passing the first target input vector to a subject extraction model for extraction to obtain a target subject, wherein the subject extraction model is constructed based on a first Bert model, a first fully connected layer and a first sigmoid function;

[0061] In an embodiment of the present invention, the subject extraction model is constructed based on a first Bert model, a first fully connected layer and a first sigmoid function, wherein the first Bert model is pre-trained, and the first Bert model is used to convert the first target input vector into the first semantic vector, and the first semantic vector includes the context information of the first target input vector. The first Bert model is essentially used to pay attention to the context information of each word in the text to be extracted to obtain the encoded first semantic vector. For example, the text to be extracted is "Zhang San likes to eat apples." Each word in it has an index in the Bert dictionary, such as the corresponding index [2,4,65,7,45,23,6,233], index 2 represents the word "Zhang", index 4 represents the word "San", and each word corresponds to the initial encoding. When [2,4,65,7,45,23,6,233] is passed to Bert, Bert will automatically convert these indexes into corresponding first semantic vectors, specifically using a 512-dimensional vector to represent a word.

[0062] Further, the first semantic vector is passed to the first sigmoid function of the first fully connected layer. Preferably, the first fully connected layer is two parallel fully connected layers. The reason for using two parallel fully connected layers is that I use a semi-pointer method to predict the position of the subject, one of which is used to predict the starting position of the subject, and the other is used to predict the ending position of the subject. The target subject is determined based on the starting position and the ending position. Preferably, the target subject is at least one.

[0063] S103, concatenating the target subject with the text to be extracted, and converting the concatenated text into a second target input vector;

[0064] In an embodiment of the present invention, the target body is spliced ​​with the text to be extracted to obtain a spliced ​​text. Preferably, the target body is in front of the text to be extracted, and the spliced ​​text is segmented using the same processing method as the first target input vector. The corresponding characters obtained by the segmentation are converted into content representation vectors and position representation vectors, and the corresponding content representation vectors and position representation vectors are superimposed to obtain a second target input vector.

[0065] S104. Pass the second target input vector to the prediction object and relationship model for extraction to obtain the target object and target relationship, wherein the prediction object and relationship model is constructed based on a second Bert model, a second fully connected layer, a second sigmoid function, a relative distance, a punctuation state vector and a subject vector.

[0066] In an embodiment of the present invention, the predicted object and relationship model is constructed based on a second Bert model, a second fully connected layer, a second sigmoid function, a relative distance, a punctuation state vector and a subject vector, wherein the second Bert model is pre-trained, and the second Bert model is used to convert the second target input vector into the second semantic vector, and the second semantic vector includes the context information of the second target input vector. The second Bert model is essentially used to pay attention to the respective context information of each word in the spliced ​​text to obtain the encoded second semantic vector.

[0067] Furthermore, the subject vector is the vector corresponding to the target subject, the relative distance is used to represent the distance between each vector, and the punctuation state vector is used to represent the relationship between punctuation marks between vectors, such as the sentence "Zhang San likes to eat apples. Li Si likes to eat pears". The first vector v1 represents a relative distance of 1, the second vector v12 represents a relative distance of 2, and so on. Select the corresponding relative distance and add it to the encoded vector of the corresponding word. The punctuation state vector regards the period and semicolon as triggers. Initialize some vectors. For the case where there are 0 triggers between words, use vector x0 to represent this situation. For the case where there is 1 trigger between words, use vector x1 to represent this situation. In this sentence, there is no trigger between "apple" and "three", then add x0 to the encoded vector of "apple", and add x1 to the encoded vector of "pear". The purpose of adding the punctuation state vector is to let the model know more semantic information. For example, in this sentence, the subject of the previous sentence is "Zhang San", and the subject of the latter sentence is "Li Si". "Pear" is the object of "Li Si", so when predicting the object of "Zhang San", we hope that the model knows that "Zhang San" and "Pear" are not related. Therefore, by passing punctuation information to the model, the model can better perceive this point and not predict "Pear" as the object of "Zhang Zhang San".

[0068] After the second Bert model encoding, a trainable state vector representing relative distance and punctuation is added, and then the first vector and the last vector of the subject are averaged to obtain the third semantic vector, and relative position encoding is used to determine the correlation between the subject and non-subject words in the third semantic vector, and the correlation is used to weight the subject vector and added to the word vector. The calculation formula of the correlation weight is as follows:

[0069]

[0070] The function of formula (1) is to calculate the correlation between the subject and the word. Because the object corresponding to the subject has a great semantic relationship with the subject, we hope to calculate the correlation between the subject and the word and pass this correlation to the model so that the model can identify the object more accurately.

[0071] in, represents the correlation weight between the i-th word and the j-th word in the sentence, E represents the word encoding, refers to the 512-dimensional vector of the i-th word (here it is the averaged subject vector), W q and W k are the multi-head attention transformation matrices for subject and word vectors respectively. The subscript q indicates the transformation of query (where query refers to the averaged subject vector), W k represents the relative distance, that is, the distance between the subject and the word, and u and v are global vectors used to measure the importance of the word in the entire sentence. T and v T Each is a 512-dimensional vector, which means to represent the entire sentence.

[0072] Furthermore, the first two terms of formula (1) are used to calculate the size of the association and the distance between the subject and the object (because it involves the operation of the subject vector and the word vector, and the operation between the subject vector and the vector representing the relative distance). The last two terms of formula (1) are used to calculate the importance of each word in the entire sentence. (Because it involves the calculation of u and the word vector and the calculation of v for the distance vector)

[0073] According to the above formula, the closer the word is to the subject and the more semantically related it is to the subject, the greater the relevance weight will be. By adding the state vector representing the relative distance and punctuation and the subject vector that has been weighted by attention calculation relevance to the initial non-subject word vector to form the final word vector representation, the model can more accurately perceive the object attributes of the word in the sentence relative to a specific subject.

[0074] Further, add the product of the calculated correlation and the topic vector to the vector corresponding to the third semantic vector to obtain the target semantic vector. Concatenate the probability value output by the second fully connected layer using the second sigmoid activation function to the target semantic vector, and determine the start position and the end position of the target object, as well as the start position and the end position of the target relationship based on the probability value. Determine the target object and the target relationship respectively based on the corresponding start position and end position. The corresponding start position or end position adopts one-hot encoding. For the word at the correct position, such as "苹" in "苹果" being the starting word of the object, then set its value to 1 in the label for predicting the start position of the object. Preferably, there is at least one group of the target object and the target relationship.

[0075] Taking the prediction object and relationship model as an example, for instance, the text to be extracted is "《月光奏鸣曲》是贝多芬创作的". The word "贝多芬" which was previously predicted as the target subject is concatenated at the front of this sentence. As described above, the encoding process of the second Bert model essentially makes each word in the sentence pay attention to the information of the words in its context to obtain its information in the sentence. In the prediction object and relationship model, to predict the object "《月光奏鸣曲》", during the calculation process of the model, it will pay attention to the information of the target subject "贝多芬", so that in the finally encoded vector, it contains the information that it is the target subject "贝多芬", thus enabling prediction. In other words, it is hoped that the prediction object and relationship model can predict what the relationship between each word in "《月光奏鸣曲》是贝多芬创作的" and the target subject "贝多芬" is. Through the second Bert model, the encoded vector of the words in the sentence can contain this information for prediction. Regarding the prediction of the target relationship: multiple affine functions are used to achieve the prediction of different relationships. The relationship between the subject and the object may be diverse, such as: "composing" relationship, "directing" relationship, etc. For each relationship, the object is predicted in a semi-pointer manner, that is, the output encoded vector is transformed into a 1-dimensional vector using an affine function and then converted into a probability value between 0 and 1 through the softmax function to determine whether it is the starting word or the ending word of the object. This is the way to predict the object for one relationship. Specifically, for the "composing" relationship, two affine functions are prepared for object prediction. For the "directing" relationship, we also prepare two corresponding affine functions for object prediction, and so on.

[0076] The loss function uses multi-class cross-entropy, and the training method adopts the Adam algorithm. The formula for multi-class cross-entropy:

[0077]

[0078] Among them: H(p) represents the loss function, C represents the number of classification types, p(x i ) represents the true probability that x is the i-th class, q(x i ) represents the probability that x calculated by the model is of the i-th category.

[0079] Through this method, as the model is trained and the loss function decreases, the probability predicted by the model will gradually approach the true probability.

[0080] In the embodiment of the present invention, the Bert model is a pre-trained model trained based on a large amount of text data, and has achieved excellent results in multiple language tasks. This model uses Bert to extract subjects to achieve a high accuracy rate. In addition, the fusion of prior knowledge and pre-trained model knowledge is creatively proposed. Specifically, some trainable state vectors are prepared in advance. These vectors are used to represent the distance and punctuation information between words. When predicting the object corresponding to the subject and its relationship, after the pre-trained model completes the encoding of the words in the sentence, these state vectors and word vectors are added. At the same time, in order to allow the model to perceive more non-subject word vectors and the relationship between the subject vector, the present invention uses relative position encoding to calculate the correlation between the word vector and the subject vector, and multiplies the correlation and the subject vector and adds it to the word vector, so that the information of the word vector and the subject vector is further combined, which promotes the model to perceive the object attributes of the word vector more accurately, and then allows the model to determine whether these words are objects. The logic behind this is that in a sentence, the object corresponding to a subject often has multiple different positions, and how to determine the true position of the object is a difficult problem. In semantics, the closer the object is to the subject, the more likely it is to be the true object. In addition, the punctuation between the subject and the object also expresses the information of the sentence. For example, if there is a period between an object A and a subject, and a comma between another object B and the subject, then B is more likely to be the object corresponding to the subject. By adding this prior knowledge, the model can obtain more information about the relationship between a specific subject and other words, and therefore can better identify the target object and target relationship.

[0081] The present invention discloses an information extraction method, including: receiving a text to be extracted, converting each word in the text to be extracted into a first target input vector; passing the first target input vector to a subject extraction model for extraction to obtain a target subject, wherein the subject extraction model is constructed based on a first Bert model, a first fully connected layer and a first sigmoid function; splicing the target subject with the text to be extracted, and converting the spliced ​​text into a second target input vector; passing the second target input vector to a predicted object and relationship model for extraction to obtain a target object and a target relationship, wherein the predicted object and relationship model is constructed based on a second Bert model, a second fully connected layer, a second sigmoid function, a relative distance, a punctuation state vector and a subject vector. In the above process, based on the subject extraction model and the predicted object and relationship model, the target subject, the target object and the target relationship are extracted based on the model to ensure the accuracy of the extraction, and because no manual participation is required, manpower is saved.

[0082] In an embodiment of the present invention, in order to ensure the accuracy of the extraction results, multiple groups of subject extraction models and predicted object and relationship models can be selected to extract the text to be extracted, and a combination of multiple triples can be obtained. Further, each triple is screened to screen out the triples to be predicted that meet the requirements. The triples to be predicted are at least one. In an embodiment of the present invention, the screening rules can be that the same triples reach a preset number or a preset percentage threshold, etc., and the specific screening rules are not limited.

[0083] Further, the triple to be predicted is passed to the triple prediction model to obtain the rational probability of the triple combination to be predicted, wherein the triple prediction model is constructed based on the third Bert model, the third fully connected layer and the third sigmoid function, and the specific processing process of the triple prediction model is as follows:

[0084] The subject and object of the triple to be predicted and their relationship are spliced ​​to the front of the triple to be extracted, passed to the third Bert for encoding, and then passed to the third full link layer and the third sigmoid activation function, using the first character <cls>The output value represents its rationality probability. Among the rationality probabilities, the predicted triplet with a probability value greater than a preset probability threshold is selected as the target triplet. The preset probability threshold can be set based on experience or specific circumstances. Preferably, the preset probability threshold is 0.5. Based on the above processing process, the accuracy of triplet extraction is further guaranteed.

[0085] In an embodiment of the present invention, the first Bert model, the second Bert model and the third Bert model may be the same or different, and are not specifically limited in the embodiment of the present invention. Furthermore, the first fully connected layer, the second fully connected layer and the third fully connected layer and the first sigmoid function, the second sigmoid function and the third sigmoid function are the same as the limitations of the Bert model and are not repeated here.

[0086] The extraction method described in the implementation of the present invention uses the pre-trained language model BERT as a word embedding layer, which is used to encode the original text to generate a dynamic word vector containing context information, and the present invention extracts prior information in the sentence, such as the relative position between the word and the subject, the status of the punctuation between the word and the subject, and the semantic relevance between the word and the subject, and fuses this information into the word vector, which can increase the model's perception of the semantic structure of the sentence and help the model extract triples more accurately. The above extraction method can handle most of the complex triples, including triples with the same object but different subjects, triples with the same subject but different objects, and triples with the same subject and object but different relationships, which can greatly improve the model's ability to extract triples and reduce information loss.

[0087] Based on the above-mentioned information extraction method, an information extraction device is also provided in an embodiment of the present invention. The structural block diagram of the extraction device is as follows: Figure 2 As shown, including:

[0088] A first conversion module 201 , a first extraction module 202 , a second conversion module 203 and a second extraction module 204 .

[0089] in,

[0090] The first conversion module 201 is used to receive the text to be extracted and convert the text to be extracted into a first target input vector;

[0091] The first extraction module 202 is used to pass the first target input vector to a subject extraction model for extraction to obtain a target subject, wherein the subject extraction model is constructed based on a first Bert model, a first fully connected layer and a first sigmoid function;

[0092] The second conversion module 203 is used to concatenate the target subject with the text to be extracted, and convert the concatenated text into a second target input vector;

[0093] The second extraction module 204 is used to pass the second target input vector to the prediction object and relationship model for extraction to obtain the target object and target relationship, wherein the prediction object and relationship model is constructed based on a second Bert model, a second fully connected layer, a second sigmoid function, a relative distance, a punctuation state vector and a subject vector.

[0094] The present invention discloses an information extraction device, comprising: receiving a text to be extracted, converting each word in the text to be extracted into a first target input vector; passing the first target input vector to a subject extraction model for extraction to obtain a target subject, wherein the subject extraction model is constructed based on a first Bert model, a first fully connected layer and a first sigmoid function; splicing the target subject with the text to be extracted, and converting the spliced ​​text into a second target input vector; passing the second target input vector to a predicted object and relationship model for extraction to obtain a target object and a target relationship, wherein the predicted object and relationship model is constructed based on a second Bert model, a second fully connected layer, a second sigmoid function, a relative distance, a punctuation state vector and a subject vector. In the above process, based on the subject extraction model and the predicted object and relationship model, the target subject, the target object and the target relationship are extracted based on the model to ensure the accuracy of the extraction, and because no manual participation is required, manpower is saved.

[0095] In the embodiment of the present invention, the first extraction module 201 includes:

[0096] word segmentation unit 205 , conversion unit 206 and superposition unit 207 .

[0097] in,

[0098] The word segmentation unit 205 is used to segment the text to be extracted according to the granularity of words to obtain individual words;

[0099] The conversion unit 206 is used to convert each word into a content representation vector and a position representation vector;

[0100] The superposition unit 207 is used to superimpose the content representation vector and the position representation vector to obtain a first target input vector.

[0101] In the embodiment of the present invention, the first extraction module 202 includes:

[0102] A first calculation unit 208 and a first prediction unit 209 .

[0103] in,

[0104] The first calculation unit 208 is used to pass the first target input vector to the first Bert model to obtain a first semantic vector;

[0105] The first prediction unit 209 is used to pass the first semantic vector to the first sigmoid function of the first fully connected layer to predict the target subject in the text to be extracted.

[0106] In the embodiment of the present invention, the second extraction module 203 includes:

[0107] a second calculation unit 210 , a first adding unit 211 , a second adding unit 212 , and a second prediction unit 213 .

[0108] in,

[0109] The second calculation unit 210 is used to pass the second target input vector to the second Bert model to obtain a second semantic vector;

[0110] The first adding unit 211 is used to add the relative distance, the punctuation state vector and the subject vector to the second semantic vector to obtain a third semantic vector;

[0111] The second adding unit 212 is used to determine the correlation between the subject words and the non-subject words in the third semantic vector, and add the product of the correlation and the subject vector to the corresponding third semantic vector to obtain a target semantic vector;

[0112] The second prediction unit 213 is used to pass the target semantic vector to the second sigmoid function of the second fully connected layer to predict the target object and target relationship in the text to be extracted.

[0113] In an embodiment of the present invention, the Zhouqu device further comprises: at least one of the subject extraction model and the predicted object and relationship model;

[0114] An acquisition module 214 , a first selection module 215 , a prediction module 216 and a second selection module 217 .

[0115] in,

[0116] The acquisition module 214 is used to acquire each triple, wherein the triple includes: a target subject, a target object and a target relationship;

[0117] The first selection module 215 is used to select a triplet to be predicted from the triplet;

[0118] The prediction module 216 is used to pass the triple to be predicted to a triple prediction model to obtain the rational probability of the triple to be predicted, wherein the triple prediction model is constructed based on the third Bert model, the third fully connected layer and the third sigmoid function;

[0119] The second selection module 217 selects, from among the rationality probabilities, a to-be-predicted triple whose probability value is greater than a preset probability threshold as the target triple.

[0120] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0121] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0124] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0125] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0126] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0127] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, commodity or device including the element.

[0128] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0129] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.< / cls>

Claims

1. An information extraction method, characterized in that: include: Receiving a text to be extracted, and segmenting the text to be extracted according to the granularity of characters to obtain individual characters; Convert each word into a content representation vector and a position representation vector; Superimposing the content representation vector and the position representation vector to obtain a first target input vector; Passing the first target input vector to a subject extraction model for extraction to obtain a target subject, wherein the subject extraction model is constructed based on a first Bert model, a first fully connected layer and a first sigmoid function; the first fully connected layer is two parallel fully connected layers; Splicing the target subject with the text to be extracted, and converting the spliced ​​text into a second target input vector; Passing the second target input vector to the prediction object and relationship model for extraction to obtain the target object and the target relationship, wherein the prediction object and relationship model is constructed based on the second Bert model, the second fully connected layer, the second sigmoid function, the relative distance, the punctuation state vector and the subject vector; Passing the second target input vector to the prediction object and relationship model to obtain the target object and target relationship includes: Passing the second target input vector to the second BERT model to obtain a second semantic vector; the first BERT model is different from the second BERT model; Adding the relative distance, the punctuation state vector and the subject vector to the second semantic vector to obtain a third semantic vector; Determine the correlation between the subject words and the non-subject words in the third semantic vector, and add the product of the correlation and the subject vector to the corresponding third semantic vector to obtain a target semantic vector; The target semantic vector is passed to the second sigmoid function of the second fully connected layer to predict the target object and target relationship in the text to be extracted.

2. The method according to claim 1, characterized in that Passing the first target input vector to the subject extraction model to extract the subject, and obtaining the target subject, including: Passing the first target input vector to the first Bert model to obtain a first semantic vector; The first semantic vector is passed to the first sigmoid function of the first fully connected layer to predict the target subject in the text to be extracted.

3. The method according to claim 1, characterized in that Also includes: The subject extraction model and the prediction object and relationship model are at least one; Acquire each triple, wherein the triple includes: a target subject, a target object, and a target relationship; Selecting a triplet to be predicted from the triples; Passing the triple to be predicted to a triple prediction model to obtain a rational probability of the triple combination to be predicted, wherein the triple prediction model is constructed based on a third Bert model, a third fully connected layer and a third sigmoid function; Among the rationality probabilities, the triples to be predicted whose probability values ​​are greater than the preset probability threshold are selected as the target triples.

4. An information extraction device, characterized in that: include: A first conversion module is used to receive a text to be extracted, and segment the text to be extracted according to the granularity of characters to obtain individual characters; Convert each word into a content representation vector and a position representation vector; Superimposing the content representation vector and the position representation vector to obtain a first target input vector; A first extraction module, used for passing the first target input vector to a subject extraction model for extraction to obtain a target subject, wherein the subject extraction model is constructed based on a first Bert model, a first fully connected layer and a first sigmoid function; the first fully connected layer is two parallel fully connected layers; A second conversion module, used for concatenating the target subject with the text to be extracted, and converting the concatenated text into a second target input vector; a second extraction module, configured to pass the second target input vector to a prediction object and relationship model for extraction, to obtain a target object and a target relationship, wherein the prediction object and relationship model is constructed based on a second Bert model, a second fully connected layer, a second sigmoid function, a relative distance, a punctuation state vector, and a subject vector; Passing the second target input vector to the prediction object and relationship model to obtain the target object and target relationship includes: Passing the second target input vector to the second BERT model to obtain a second semantic vector; the first BERT model is different from the second BERT model; Adding the relative distance, the punctuation state vector and the subject vector to the second semantic vector to obtain a third semantic vector; Determine the correlation between the subject words and the non-subject words in the third semantic vector, and add the product of the correlation and the subject vector to the corresponding third semantic vector to obtain a target semantic vector; The target semantic vector is passed to the second sigmoid function of the second fully connected layer to predict the target object and target relationship in the text to be extracted.

5. The device according to claim 4, characterized in that The first extraction module comprises: A first computing unit, configured to pass the first target input vector to the first Bert model to obtain a first semantic vector; The first prediction unit is used to pass the first semantic vector to the first sigmoid function of the first fully connected layer to predict the target subject in the text to be extracted.

6. The device according to claim 4, characterized in that Also includes: The subject extraction model and the prediction object and relationship model are at least one; An acquisition module, used for acquiring each triple, wherein the triple includes: a target subject, a target object and a target relationship; A first selection module, used for selecting a triple to be predicted from the triples; A prediction module, used for passing the triple to be predicted to a triple prediction model to obtain a rational probability of the triple combination to be predicted, wherein the triple prediction model is constructed based on a third Bert model, a third fully connected layer and a third sigmoid function; The second selection module selects the to-be-predicted triples whose probability values ​​are greater than a preset probability threshold value from among the various rationality probabilities as the target triples.

Images